Commit Graph

1687 Commits

Author SHA1 Message Date
Alexey Milovidov
47eaffbe63 Additional checks 2020-07-10 11:21:40 +03:00
Alexey Milovidov
4b86f36d37 Check arguments of bloom filter index 2020-07-10 11:13:21 +03:00
alesapin
5cae87e664
Merge pull request #12335 from ClickHouse/fix_alter_exit_codes
Fix alter rename error messages
2020-07-10 11:05:20 +03:00
Peng Jian
958bb1f3aa COS (Tencent Cloud Object Storage) integration
To allow import and export to COS.
2020-07-10 15:26:55 +08:00
Alexey Milovidov
276b3a0215 Avoid exception when negative or floating point constant is used in WHERE condition for indexed tables #11905 2020-07-10 09:30:49 +03:00
Alexey Milovidov
a4b35a8a6f Allow to CLEAR column even if there are depending DEFAULT expressions #12333 2020-07-10 08:54:35 +03:00
alexey-milovidov
c16d8e094b
Merge pull request #12308 from ClickHouse/fix-codec-bad-exception-code
Fix wrong exception code in codecs Delta, DoubleDelta #12110
2020-07-10 08:40:46 +03:00
Alexey Milovidov
afc00fa0b8 Merge branch 'master' into fix-codec-bad-exception-code 2020-07-10 04:12:24 +03:00
Alexey Milovidov
7fc90aa070 Fix error 2020-07-10 02:45:29 +03:00
Alexey Milovidov
ad6fcd57b2 Merge branch 'master' into fix-race-condition-replicated-merge-tree-queue 2020-07-10 02:21:24 +03:00
Anton Popov
80a62977f1 fix hashing in DataPartWriterCompact 2020-07-09 21:26:54 +03:00
Anton Popov
9f72baad1f Merge remote-tracking branch 'upstream/master' into HEAD 2020-07-09 20:51:36 +03:00
alesapin
0156f43ed3 Human readable errors in alter rename queries 2020-07-09 17:30:38 +03:00
alesapin
9dea4ab323 Initial version 2020-07-09 17:14:44 +03:00
Vladimir Chebotarev
faf6be6576
Implemented single part uploads for DiskS3 (#12026)
* Implemented single part uploads for DiskS3.
* Added `min_multi_part_upload_size` to disk configuration.
2020-07-09 17:09:17 +03:00
Alexander Tokmakov
445acd0294 Merge branch 'master' into database_atomic_improvements 2020-07-09 16:02:54 +03:00
Denis Glazachev
5a4762bb8a
Merge branch 'master' into ldap-per-user-authentication 2020-07-09 14:50:02 +04:00
alesapin
47f05dcadd
Merge pull request #12304 from CurtizJ/fix-ttl-rename
Fix TTL after renaming column.
2020-07-09 13:06:27 +03:00
Alexey Milovidov
39e79cfe3e Merge branch 'master' into fix-codec-bad-exception-code 2020-07-09 07:50:18 +03:00
Alexey Milovidov
6c458a6287 Fix race condition in ReplicatedMergeTreeQueue 2020-07-09 07:43:28 +03:00
alexey-milovidov
827990d681
Merge pull request #12305 from ClickHouse/fix-virtual-columns-filter
Fix virtual columns filter
2020-07-09 07:06:07 +03:00
Alexey Milovidov
ea970fd57c Remove bad ugliness 2020-07-09 04:00:16 +03:00
Alexey Milovidov
b78e1145e8 Fix filtering by virtual columns #12166 2020-07-09 02:52:57 +03:00
Anton Popov
0e4871eec2 fix TTL after renaming column 2020-07-09 02:25:31 +03:00
Alexey Milovidov
eaa13d8176 Fix strange code CC @Enmk. Prove: g++ -xc++ -include vector - <<< 'int main() { return std::vector<const char>{1, 2, 3}.size(); }' 2020-07-09 02:04:42 +03:00
Alexander Tokmakov
9c4efa1f36 fix, add test 2020-07-09 01:50:15 +03:00
tavplubix
33c3545125
Merge pull request #12197 from nikitamikhaylov/csv-file-bugfix
Headers for CSVWithNames in StorageFile
2020-07-08 20:42:49 +03:00
alesapin
84f8bf1e14
Merge pull request #10697 from CurtizJ/polymorphic-parts
Polymorphic parts (in-memory format)
2020-07-08 13:25:45 +03:00
alexey-milovidov
ffdd6e1af6
Merge pull request #12173 from nikitamikhaylov/summing-bug-fix
Exclude partition key columns from SummingMergeTree
2020-07-08 01:46:12 +03:00
Denis Glazachev
9effacfbc1 Merge branch 'master' into ldap-per-user-authentication
* master: (1102 commits)
  Update README.md
  Update README.md
  Update README.md
  Update index.md
  [docs] add intrdocution for statements page (#12189)
  Revert "Run perf tests with memory sampling (for allocations >1M)"
  Sanitize LINK_LIBRARIES property for the directories (#12160)
  [docs] refactor Domains overview (#12186)
  DOCS-647: toStartOfSecond (#12190)
  [docs] add intrdocution for commercial page (#12187)
  DOCSUP-1348 Russian translation for new functions (#133) (#12194)
  changelog fixes
  Update index.md (#12191)
  Update zh kafka.md title (#12192)
  Added test for #3767
  style fix for #12152
  Tests for fixed issues #10846 and #7347
  changelog fixes
  [docs] introduction for special table engines (#12170)
  [docs] introduction for third-party interfaces (#12175)
  ...

# Conflicts:
#	src/Access/ya.make
#	src/Common/ErrorCodes.cpp
2020-07-08 00:42:09 +04:00
alexey-milovidov
546fcc9ed6
Merge pull request #12151 from ClickHouse/fix-transform-query-aliases
Fix transform query for external databases in presense of aliases
2020-07-07 21:46:35 +03:00
Nikita Mikhaylov
270551e599 done 2020-07-07 14:45:20 +03:00
Anton Popov
416d9648c2 remove questionable functionality 2020-07-07 12:58:14 +03:00
tavplubix
488a8cbc9f
Merge pull request #12120 from filimonov/query_context_for_system_logs
Add query context for system logs and to Buffer
2020-07-07 09:55:20 +03:00
Anton Popov
d6434f61dc support codecs in compact parts 2020-07-07 03:15:02 +03:00
Mikhail Filimonov
93517b4e82 Same change for Kafka - just in case, and to make it conform. 2020-07-06 20:48:33 +02:00
alesapin
3f90f65678 Merge remote-tracking branch 'origin' into CurtizJ-polymorphic-parts 2020-07-06 21:34:17 +03:00
Nikita Mikhaylov
2b539e6ace fix style 2020-07-06 20:26:15 +03:00
Mikhail Filimonov
8038383f06 Fix #10437, CR fixes 2020-07-06 19:24:33 +02:00
Nikita Mikhaylov
d31ed58f01 done 2020-07-06 17:33:31 +03:00
Alexander Tokmakov
d1be5ec641 fix symlinks, add short syntax of attach database 2020-07-06 16:41:43 +03:00
Alexander Kuzmenkov
d648628a4b
Merge pull request #12075 from azat/StorageKafka-SIGSEGV-fix
Fix SIGSEGV in StorageKafka on DROP TABLE
2020-07-06 13:56:06 +03:00
Alexey Milovidov
95a7a09c37 Fix transform query for external databases in presense of aliases #12032 2020-07-06 04:50:45 +03:00
alexey-milovidov
ea0fb005fb
Merge pull request #12116 from vitlibar/fix-table-dictionary-dependency
Fix handling dictionary-table dependency
2020-07-06 04:19:43 +03:00
alexey-milovidov
eecf7ae3ee
Merge pull request #12140 from ClickHouse/fix-division
Fix potential overflow in integer division
2020-07-05 19:00:41 +03:00
myrrc
8c3417fbf7
ILIKE operator (#12125)
* Integrated CachingAllocator into MarkCache

* fixed build errors

* reset func hotfix

* upd: Fixing build

* updated submodules links

* fix 2

* updating grabber allocator proto

* updating lost work

* updating CMake to use concepts

* some other changes to get it building (integration into MarkCache)

* further integration into caches

* updated Async metrics, fixed some build errors

* and some other errors revealing

* added perfect forwarding to some functions

* fix: forward template

* fix: constexpr modifier

* fix: FakePODAllocator missing member func

* updated PODArray constructor taking alloc params

* fix: PODArray overload with n restored

* fix: FakePODAlloc duplicating alloc() func

* added constexpr variable for alloc_tag_t

* split cache values by allocators, provided updates

* fix: memcpy

* fix: constexpr modifier

* fix: noexcept modifier

* fix: alloc_tag_t for PODArray constructor

* fix: PODArray copy ctor with different alloc

* fix: resize() signature

* updating to lastest working master

* syncing with 273267

* first draft version

* fix: update Searcher to case-insensitive

* added ILIKE test

* fixed style errors, updated test, split like and ilike,  added notILike

* replaced inconsistent comments

* fixed show tables ilike

* updated missing test cases

* regenerated ya.make

* Update 01355_ilike.sql

Co-authored-by: myrrc <me-clickhouse@myrrec.space>
Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
2020-07-05 18:57:59 +03:00
alexey-milovidov
cb126a23ef
Update StorageDictionary.h 2020-07-05 18:45:05 +03:00
alexey-milovidov
1a760e18a5
Update StorageDictionary.h 2020-07-05 18:44:28 +03:00
Vitaly Baranov
99e9b15e00 Make code clearer: use enum instead of bool internal. 2020-07-05 17:14:29 +03:00
Alexey Milovidov
73a5c38398 Fix potential overflow in integer division #12119 2020-07-05 03:29:03 +03:00
alexey-milovidov
c696354dd4
Merge pull request #12115 from ianton-ru/disk-type-in-system-disks
Add type column in system.disks
2020-07-05 01:04:17 +03:00
alexey-milovidov
e05283d5d3
Merge pull request #12060 from ClickHouse/fix-12053
Check wrong type for filter.
2020-07-04 23:46:34 +03:00
Anton Popov
73676f5022
Improve performace of reading in order of sorting key. (#11696)
* simplify reading in order of sorting key

* add perf test for reading many parts

* Revert "simplify reading in order of sorting key"

This reverts commit 7267d7c46e.

* add threshold for preliminary merge for reading in order

* better threshold

* limit threads in test
2020-07-04 15:48:51 +03:00
alesapin
a2b6d58053
Merge pull request #11069 from kssenii/add-storage-rabbitmq-read-only
Add storage RabbitMQ
2020-07-04 10:38:50 +03:00
alesapin
bfa8cdc7a4
Move skip lists for tests to skip_list.json file (#12107)
* Add unbundled mode flag

* Move skip lists to clickhouse-test

* More verbose message about skip

* Make skip-list optional

* Update clickhouse-test
2020-07-04 10:35:45 +03:00
alesapin
8dc204350f
Fix version column in replicated version collapsing merge tree (#12121) 2020-07-04 10:35:17 +03:00
alexey-milovidov
6677e0a503
Merge pull request #12073 from zhang2014/feature/atfer_column_modify_column
ISSUES-4006 support first for ALTER ADD|MODIFY COLUMN
2020-07-03 22:54:05 +03:00
Vitaly Baranov
ab343132de Fix handling dependency of table with ENGINE=Dictionary on dictionary. 2020-07-03 22:05:53 +03:00
Anton Ivashkin
23b44ca6fe Add type column in system.disks 2020-07-03 15:45:54 +03:00
alesapin
0a6f3ca9fd Add unbundled mode flag 2020-07-03 13:11:16 +03:00
alesapin
c57edd2018 Tiny fixes 2020-07-03 11:02:35 +03:00
Azat Khuzhin
bd5e5e9462 kafka: remove outdated comment
As stated by @filimonov it is not relevant (after #11599)
2020-07-03 09:23:39 +03:00
Azat Khuzhin
de8bc99d48 kafka: avoid superior polling after DROP/DETACH TABLE
Before this patch isStalled() was checked before polledDataUnusable(),
and after DROP TABLE isStalled() == true (although this looks tricky).
2020-07-03 09:23:39 +03:00
Azat Khuzhin
8508200020 kafka: improve logging during engine shutdown
This will help with tracking possible issues, when you need to know was
buffer released or not.
2020-07-03 09:23:39 +03:00
Azat Khuzhin
e7179c402c kafka: check that the data is still usable after parsing 2020-07-03 09:23:38 +03:00
Azat Khuzhin
6104872cae kafka: fix SIGSEGV on DROP TABLE
After #11599 it is possible that messages of the
ReadBufferFromKafkaConsumer will be cleaned-up right in
read_kafka_message callback (from KafkaBlockInputStream) if the stop
flag isset (i.e. DROP TABLE is waiting the consumer), and if
read_kafka_message already processed some rows it will not return 0 and
the loop after will try to get current topic from the buffer, which uses
messages in the underlying and this will got SIGSEGV:

    12:14:56.173262 [ 55421 ] {f7930856-d478-4e41-af56-24ce7b693e95} <Debug> executeQuery: (from 0.0.0.0:0, user: ) DROP TABLE IF EXISTS data.queue
    12:14:56.173285 [ 55421 ] {f7930856-d478-4e41-af56-24ce7b693e95} <Trace> StorageKafka (newly_queue): Waiting for cleanup
    12:14:56.180016 [ 55390 ] {} <Trace> BaseDaemon: Received signal 11
    12:14:56.180267 [ 4914 ] {} <Fatal> BaseDaemon: ########################################
    12:14:56.181879 [ 4914 ] {} <Fatal> BaseDaemon: (version 20.6.1.1, build id: 4CE0298F08583658) (from thread 55468) (no query) Received signal Segmentation fault (11)
    12:14:56.181900 [ 4914 ] {} <Fatal> BaseDaemon: Address: 0x8 Access: read. Address not mapped to object.
    12:14:56.181909 [ 4914 ] {} <Fatal> BaseDaemon: Stack trace:
    12:14:56.184676 [ 4914 ] {} <Fatal> BaseDaemon: 3. /ch/contrib/cppkafka/include/cppkafka/message.h:111: DB::KafkaBlockInputStream::readImpl() @ 0xe343f1c in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.185553 [ 4914 ] {} <Fatal> BaseDaemon: 4. /ch/contrib/libcxx/include/vector:1003: DB::IBlockInputStream::read() @ 0xd9d95bd in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.188238 [ 4914 ] {} <Fatal> BaseDaemon: 5. /ch/src/DataStreams/copyData.cpp:26: DB::copyData() @ 0xd9f712a in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.188780 [ 4914 ] {} <Fatal> BaseDaemon: 6. /ch/contrib/libcxx/include/vector:1532: DB::StorageKafka::streamToViews() @ 0xe335e73 in /usr/lib/debug/usr/bin/clickhouse
    12:14:56.189331 [ 4914 ] {} <Fatal> BaseDaemon: 7. /ch/src/Storages/Kafka/StorageKafka.cpp:491: DB::StorageKafka::threadFunc() @ 0xe336738 in /usr/lib/debug/usr/bin/clickhouse

55421 thread (shows that it still waiting for deactivation):

    5  std::__1::lock_guard<>::lock_guard () at ../contrib/libcxx/include/__mutex_base:90
    6  DB::BackgroundSchedulePoolTaskInfo::deactivate (this=0x7fc7e4465f20) at ../src/Core/BackgroundSchedulePool.cpp:59
    7  DB::StorageKafka::shutdown (this=0x7fc7e45e4600) at ../contrib/libcxx/include/memory:3821

And just in case thread where read_kafka_message is called:

    0  DB::ReadBufferFromKafkaConsumer::nextImpl (this=0x7fd4901d4118) at ../contrib/libcxx/include/atomic:1491
    1  DB::ReadBuffer::next (this=0x7fd4901d4118) at ../src/IO/ReadBuffer.h:59
    2  DB::ReadBuffer::eof (this=0x7fd4901d4118) at ../src/IO/ReadBuffer.h:81
    3  DB::skipWhitespaceIfAny (buf=...) at ../src/IO/ReadHelpers.h:945
    4  DB::JSONEachRowRowInputFormat::readRow (ext=..., columns=..., this=0x7fd499a7a020) at ../src/Processors/Formats/Impl/JSONEachRowRowInputFormat.cpp:222
    5  DB::JSONEachRowRowInputFormat::readRow (this=0x7fd499a7a020, columns=..., ext=...) at ../src/Processors/Formats/Impl/JSONEachRowRowInputFormat.cpp:218
    6  DB::IRowInputFormat::generate (this=0x7fd499a7a020) at ../src/Processors/Formats/IRowInputFormat.cpp:64
    7  DB::ISource::work (this=0x7fd499a7a020) at ../src/Processors/ISource.cpp:48
    8  DB::KafkaBlockInputStream::<lambda()>::operator()(void) const () at ../contrib/libcxx/include/memory:3826
    9  DB::KafkaBlockInputStream::readImpl (this=0x7fd46e718820) at ../contrib/libcxx/include/new:340

Cc: @filimonov
2020-07-03 09:22:44 +03:00
Anton Popov
ca346ea13c rename fsync-related settings 2020-07-03 02:41:37 +03:00
Alexander Tokmakov
383c583bdd add uuid to atomic databases 2020-07-02 23:39:31 +03:00
Anton Popov
4422df2e37 Merge remote-tracking branch 'upstream/master' into HEAD 2020-07-02 20:18:21 +03:00
alesapin
c6c7ee30d9 Less race conditions 2020-07-02 19:44:04 +03:00
alesapin
96df2e6b71 Better shutdown and conversion 2020-07-02 17:35:10 +03:00
alexey-milovidov
82061da77c
Merge pull request #12040 from azat/dist-fixes
Tiny fixes for distributed queries
2020-07-02 17:02:56 +03:00
alesapin
64583ceb22 Merge branch 'master' into add-storage-rabbitmq-read-only 2020-07-02 16:08:29 +03:00
Anton Popov
71059e4dff
Merge pull request #12062 from nvartolomei/nv/set-index-tuple-types
Try fix pk in tuple performance
2020-07-02 15:51:03 +03:00
Nikolai Kochetov
79cd33a567 Fix tests. 2020-07-02 10:44:47 +03:00
Alexander Kazakov
cc2cadb7f5
Merge pull request #11893 from excitoon-favorites/autostartmoves
In *MergeTree: Parts moving task shall be started if new storage policy needs them
2020-07-02 10:44:38 +03:00
zhang2014
9ed5c6d925 ISSUES-4006 support first for ALTER ADD|MODIFY COLUMN 2020-07-01 22:58:52 +08:00
Nicolae Vartolomei
3854ce6d84 Rewrite Set lookup to make it more readable 2020-07-01 15:05:54 +01:00
Anton Popov
68c9896fa2 fix arcadia build 2020-07-01 00:15:39 +03:00
Anton Popov
d45697a1b5 in-memory-parts: lazy creation of WAL 2020-06-30 21:47:12 +03:00
Nikolai Kochetov
3585700506 Fix tests. 2020-06-30 19:41:43 +03:00
Vitaly Baranov
c39eb8f71b Fix partial revokes (complex cases). 2020-06-30 18:47:02 +03:00
Nicolae Vartolomei
8f1845185e Try fix pk in tuple performance
Possible approach for fixing #10574

The problem is that prepared sets are built correctly, it is a hash map of key -> set
where key is a hash of AST and list of data types (when we a list of
tuples of literals).

However, when the key is built from the index to try and find if there
exists a prepared set that would match it looks for data types of the
primary key (see how data_types is populated) because the primary key
has only one field (v in my example) it can not find the prepared set.

The patch looks for any prepared indexes where data types match for the
subset of fields found in primary key, we are not interested in other
fields anyway for the purpose of primary key pruning.
2020-06-30 16:33:38 +01:00
Nikolai Kochetov
66da0733ab Check type of filter for prewhere. 2020-06-30 17:20:27 +03:00
Vladimir Chebotarev
bb8da71eff Moves task shall be started if new storage policy needs them. 2020-06-30 15:36:44 +03:00
kssenii
f797efb04c Fix build 2020-06-30 01:48:11 +00:00
Anton Popov
53e955c6dd several fixes 2020-06-29 23:36:18 +03:00
Azat Khuzhin
9c94993f14 Fix "Sharding key is not deterministic" message 2020-06-29 23:00:14 +03:00
kssenii
fd9b416871 Fix and simplify code 2020-06-29 15:47:25 +00:00
Anton Popov
a43cb93be5 remove questionable functionality 2020-06-29 18:46:50 +03:00
alesapin
718262e7ce
Merge pull request #11973 from ClickHouse/alter-enum-partition-key
Allow to perform "metadata-only" ALTER of partition key.
2020-06-29 16:01:07 +03:00
kssenii
88ece429c9 Move writing to RabbitMQ into background task 2020-06-29 12:33:01 +00:00
kssenii
3d2cc9d4b2 Remove libevent library 2020-06-29 12:11:17 +00:00
kssenii
5fc0b93400 Use ConcurentBoundedQueue instead of vector 2020-06-29 09:50:33 +00:00
alesapin
e72bc39654
Merge pull request #11983 from ClickHouse/storage_mongodb
Merging #10931 (Storage MongoDB)
2020-06-29 11:49:18 +03:00
Alexey Milovidov
cc597bb30b Implement for ReplicatedMergeTree 2020-06-28 22:39:31 +03:00
Alexey Milovidov
6aa89d9073 Typos 2020-06-28 22:39:31 +03:00
Alexey Milovidov
e3410ca617 Write partition key changes to ZooKeeper 2020-06-28 22:39:31 +03:00
Alexey Milovidov
e5f4a4b4b0 Whitespace 2020-06-28 22:39:31 +03:00
Alexey Milovidov
1f9c27de34 Fix clang-tidy 2020-06-28 22:39:31 +03:00
Alexey Milovidov
b6e31f0a5e Only allow Enum extension within the same width 2020-06-28 22:39:31 +03:00
Alexey Milovidov
e4d8cc2b44 Slightly more safe 2020-06-28 22:39:31 +03:00
Alexey Milovidov
97ad23b905 Allow to ALTER partition key in some cases 2020-06-28 22:39:31 +03:00
Alexey Milovidov
678d127503 Whitespace 2020-06-28 22:39:31 +03:00
Alexey Milovidov
c4d5ca485e Remove the usage of obsolete setting 2020-06-28 22:39:31 +03:00
kssenii
649eb8e348 Move reading from RabbitMQ into background task 2020-06-28 18:20:18 +00:00
Alexey Milovidov
d1f4a0e473 Fix array size overflow in generateRandom 2020-06-28 17:33:02 +03:00
alexey-milovidov
935d2be826
Merge pull request #12003 from ClickHouse/fix-typos
Fix typos in code
2020-06-28 15:49:17 +03:00
alexey-milovidov
5a99cdb0d9
Merge pull request #11996 from BohuTANG/mysql_engine_enum
ISSUES-3985 Support MySQL engine reading Enums type
2020-06-27 22:18:25 +03:00
Alexey Milovidov
1462a66d1e Fix typos 2020-06-27 22:05:00 +03:00
Anton Popov
b2aa565a37 allow to turn on fsync on inserts, merges and fetches 2020-06-27 17:52:16 +03:00
Alexey Milovidov
46caa211d5 Better diagnostics of "Replica {} appears to be already active" message 2020-06-27 16:55:00 +03:00
BohuTANG
f609fd34f7 Support Enums type for MySQL engine #3985 2020-06-27 09:10:07 +08:00
alesapin
6b3092290d Trying to fix ya.make 2020-06-26 19:33:51 +03:00
alesapin
b5f79bc28c Remove strange header 2020-06-26 19:32:15 +03:00
alesapin
4969da85d9 Fixes 2020-06-26 17:28:00 +03:00
alesapin
11f88340a5 Merge branch 'mongo' of https://github.com/ageraab/ClickHouse into storage_mongodb 2020-06-26 16:03:06 +03:00
alesapin
9710a67704 Fix style check 2020-06-26 15:27:12 +03:00
alesapin
6caf2e7a3f Fix protocol check 2020-06-26 14:38:37 +03:00
alesapin
6f1824f0ea Correct merge with master 2020-06-26 14:30:23 +03:00
alesapin
e9c47dc89c Merge branch 'master' into CurtizJ-polymorphic-parts 2020-06-26 14:27:19 +03:00
alesapin
b3c31370c2
Merge pull request #11970 from ClickHouse/fix-estimation-of-the-number-of-marks
Fix estimation of the number of marks while reading from MergeTree
2020-06-26 11:28:36 +03:00
alexey-milovidov
2163716e45
Merge pull request #11955 from ClickHouse/revive-mmap-2
Allow to use direct_io and mmap_io for secondary indices.
2020-06-26 03:16:56 +03:00
alexey-milovidov
178a5217bc
Merge pull request #11953 from ClickHouse/revive-mmap
Revive mmap IO + add a test
2020-06-26 03:15:07 +03:00
alexey-milovidov
8f429f6027
Merge pull request #11935 from zlobober/allocated_bytes
Consider allocatedBytes() instead of bytes() in Storage{Buffer,Memory}.
2020-06-26 00:57:47 +03:00
Alexey Milovidov
8eed47857b Fix estimation of the number of marks for various thresholds 2020-06-25 23:20:22 +03:00
Alexey Milovidov
8872417d00 Respect direct_io/mmap settings while reading secondary indices 2020-06-25 22:31:54 +03:00
Alexey Milovidov
5608f15749 Revive mmap IO 2020-06-25 22:15:41 +03:00
Anton Popov
f8f67a788e allow to turn on fsync on inserts and merges 2020-06-25 19:55:45 +03:00
alexey-milovidov
a34032cace
Merge pull request #11300 from blinkov/sentry
Opt-in support for sending crash reports
2020-06-25 17:20:50 +03:00
alesapin
36eb2c3028 Simplify code around locks 2020-06-25 13:25:22 +03:00
alesapin
d5847d29d6 Avoid memory leaks 2020-06-25 12:44:39 +03:00
Maxim Akhmedov
29d2928f93 Consider allocatedBytes() instead of bytes() in Storage{Buffer,Memory}. 2020-06-25 00:27:53 +03:00
alesapin
addee61bcb Trying to do everything on top of libuv, add heartbeats 2020-06-25 00:14:49 +03:00
alesapin
cb30dbfe28 Correct merge with master 2020-06-24 20:32:57 +03:00
alesapin
3fc65b3269 Merge branch 'master' into kssenii-rabbit-mq 2020-06-24 20:14:28 +03:00
alesapin
63f8f89ca0
bump CI 2020-06-24 19:06:01 +03:00
alesapin
08f5ed6da3 Fix unitialized memory 2020-06-24 17:12:44 +03:00
Nikolai Kochetov
f7f16fdbf7
Merge pull request #11895 from ClickHouse/fix-nullable-prewhere-type-3
Fix nullable prewhere type 3
2020-06-24 14:32:08 +03:00
alexey-milovidov
18eb141ea1
Merge pull request #11715 from azat/dist-optimize_skip_unused_shards-fixes
Control nesting level for shards skipping and disallow non-deterministic functions
2020-06-24 12:54:58 +03:00
tavplubix
2401bc9143
Merge pull request #11896 from ClickHouse/merging_drop_replica
Merging #10679
2020-06-24 11:42:39 +03:00
Anton Popov
8c3f496826 add test with alters and in-memory parts 2020-06-24 01:16:29 +03:00
Ivan Blinkov
45d1ca2567 merge master 2020-06-24 00:18:16 +03:00
alexey-milovidov
33dfbdcbe9
Update StorageSystemTables.cpp 2020-06-23 22:28:26 +03:00
Alexander Tokmakov
7b5de16da9 minor improvements 2020-06-23 22:15:01 +03:00
Nikolai Kochetov
658a2d0dc0 Fix anotehr one nullable prewhere column. 2020-06-23 21:52:32 +03:00
amudong
2a51286527 Optimize code 2020-06-23 16:22:53 +08:00
alesapin
4185fe9975 Merge branch 'master' into atomic_metadata5 2020-06-23 11:13:05 +03:00
alesapin
292016e045 Add missed lock 2020-06-23 11:04:43 +03:00
amudong
cca8df6b78 format comment 2020-06-23 12:24:05 +08:00
amudong
0101947403 rewrite static StorageReplicatedMergeTree::dropReplica 2020-06-23 12:12:30 +08:00
amudong
7723dc4935 add drop relica database and the whole replica
fix removeReplicaByZKPath

fix bug: add zkpath empty judge

fix: rewrite code

delete useless code.

fix:ast fromat

fix bug

add test_drop_replica

add drop_replica doc

add drop databse checkAccess

refactor dropReplica

update tests

add static method StorageReplicatedMergeTree::dropReplicaByZkPath

update doc and delete useless code

fix conflict

fix doc

fix doc

fix StorageReplicatedMergeTree::dropReplica

fix bug

delete useless code
2020-06-23 12:12:30 +08:00
amudong
2e6a3eff8e fix: can not drop local table and replace ALTER with SYSTEM
fix test_drop_replica

fix drop replica '/path/to/zk/' ending in '/' and update doc
2020-06-23 12:12:30 +08:00
sundy-li
15ad830290 only drop inactive replica
update doc

Increase timeout to release the zookeeper Ephemeral nodes

Fix code comment

use PartitionManager

make integrations test passed
2020-06-23 12:12:30 +08:00
sundy-li
34df59baf8 Add tests and docs
Fix integration-tests
2020-06-23 12:12:30 +08:00
sundy-li
d4aa3cea4b fix merge state
Fix style && build
2020-06-23 12:12:30 +08:00
sundy-li
906a43e4a8 Add drop replica alter support 2020-06-23 12:12:30 +08:00
alexey-milovidov
a7b224a578
Merge pull request #11869 from ClickHouse/fix-nullable-prewhere-type-2
Fix nullable prewhere type 2
2020-06-23 02:48:59 +03:00
Anton Popov
78d28be8cf check in-memory parts, comments and style-fixes 2020-06-22 21:57:50 +03:00
robot-clickhouse
4bd47b8adc Auto version update to [20.6.1.1] [54436] 2020-06-22 20:40:26 +03:00
alesapin
ef89a6f728
Merge pull request #11786 from ClickHouse/fix_alter_with_materialized
Fix alter key when materialized column passed
2020-06-22 19:06:25 +03:00
Nikolai Kochetov
ae21aca3a3 Fix header for nullable prewhere column. 2020-06-22 19:01:59 +03:00
alesapin
c76b4b3481 Clang-tidy fixes 2020-06-22 18:51:11 +03:00
alesapin
f262c4f652 Better comments 2020-06-22 12:49:21 +03:00
alesapin
b1e8976df4 Merge with master 2020-06-22 12:04:27 +03:00
alesapin
b9e74f4e82 Merge branch 'master' into atomic_metadata5 2020-06-22 12:03:53 +03:00
Ivan Blinkov
2c0ff29c48 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-22 10:06:36 +03:00
Alexey Milovidov
0e6fb617aa Merge branch 'master' into manual-write-duplicate-parts-to-replicas 2020-06-22 01:54:03 +03:00
Denis Glazachev
6c1b2a4831 Merge branch 'master' into ldap-per-user-authentication
* master: (114 commits)
  Update alter.md (#11823)
  Update PODArray.h
  Added a test for #6549
  Update create.md
  Update create.md
  replacingmergetree translate into "替换合并树“ better (#11814)
  Whitespace
  Fix style
  Added a test
  Fix another 10% of bad code #11529
  Update join.md (#11798)
  Update memory.md (#11800)
  Whitespace
  Check for tests with "fail" in their names
  Fix 10% of bad code #11756
  Update PODArray.h
  avoid duplicate short system name
  adapt recent result as well
  [website] tune benchmark pages styling + add extra checks
  Added MSan suppression for Hyperscan
  ...

# Conflicts:
#	src/Common/ErrorCodes.cpp
2020-06-20 22:01:12 +04:00
alexey-milovidov
9bfefc99ab
Merge pull request #11599 from filimonov/better_kafka_states
Kafka work with formats based on PeekableReadBuffer and other improvements
2020-06-20 20:36:54 +03:00
alexey-milovidov
076fe8e8ba
Update KafkaBlockInputStream.cpp 2020-06-20 20:34:42 +03:00
Alexey Milovidov
ad672f7ce2 Whitespace 2020-06-20 10:26:33 +03:00
Alexey Milovidov
b8769e1ada Fix style 2020-06-20 10:24:51 +03:00
alesapin
4c0879ae30 Better logging in storages 2020-06-19 20:17:13 +03:00
alesapin
c9fa5d2ec3 Better naming 2020-06-19 18:39:41 +03:00
alesapin
a2e7e9f232 Remove unused constructor and rename method 2020-06-19 18:21:48 +03:00
alesapin
3c47faa9da Move partition key initialization into registerMergeTree 2020-06-19 18:14:08 +03:00
Alexey Milovidov
468eaa8d93 Remove unused method 2020-06-19 17:21:26 +03:00
Alexey Milovidov
5211a42c04 Remove leader election, step 3 2020-06-19 17:18:58 +03:00
alesapin
016ee6316f Add missed check 2020-06-19 15:05:29 +03:00
alesapin
ce0c5faf9e Merge branch 'fix_alter_with_materialized' into atomic_metadata5 2020-06-19 14:00:32 +03:00
alesapin
4b6db63fff Fix alter key when materialized column passed 2020-06-19 13:53:20 +03:00
Mikhail Filimonov
9609bd9dee Kafka better states, formats based on PeekableReadBuffer, and other minor fixes.
Add formats tests, fixes for JSONCompactEachRowWithNamesAndTypes, TSVWithNamesAndTypes. Some CR fixes
Add sanitizing for kafka_max_block_size and kafka_poll_max_batch_size
2020-06-19 10:49:05 +02:00
Denis Glazachev
5db60202b6 Merge branch 'master' into ldap-per-user-authentication 2020-06-19 00:11:08 +04:00
alesapin
009977c20c Fix locks 2020-06-18 22:45:37 +03:00
Azat Khuzhin
d34e6217bc Add logging of adjusting conditional settings for distributed queries 2020-06-18 21:49:29 +03:00
Azat Khuzhin
041533eae2 Disable optimize_skip_unused_shards if sharding_key has non-deterministic func
Example of such functions is rand()

And this patch disables only optimize_skip_unused_shards, i.e. INSERT
code path does not changed, so it will work as before.
2020-06-18 21:49:29 +03:00
alesapin
8b131e2079 Remove int contention 2020-06-18 20:19:11 +03:00
alesapin
c666763cc0 Remove unused method better 2020-06-18 20:09:06 +03:00
alesapin
e888dafdc2 Remove unused method 2020-06-18 20:03:42 +03:00
alesapin
b3ee8967dc Fix style 2020-06-18 19:28:20 +03:00
alesapin
54e5fe7dbc Less locks 2020-06-18 19:19:40 +03:00
alesapin
d79982f497 Better locks in Storages 2020-06-18 19:10:47 +03:00
alesapin
c8a58299ac Fix storage buffer metadata 2020-06-18 14:42:48 +03:00
alesapin
1a69c3234a Fix style 2020-06-18 14:09:55 +03:00
alesapin
d4c49816ab Really atomic metadata 2020-06-18 14:02:31 +03:00
alesapin
4de5331b0d Fix SystemTables 2020-06-18 13:39:33 +03:00
alesapin
aab4ce6394 Truncate with metadata 2020-06-18 13:29:13 +03:00
alesapin
35ce47951d Fix storage merge 2020-06-18 12:22:54 +03:00
alesapin
760e9a8488 Fix crash 2020-06-18 12:08:24 +03:00
alesapin
f4037b8f54 Fix build 2020-06-18 12:00:43 +03:00
alesapin
ebb36bec8a Merge branch 'master' into atomic_metadata5 2020-06-18 11:57:16 +03:00
alesapin
d0beeaf7c4
Merge pull request #11744 from ClickHouse/stop_fetches_before_metadata_alter
Cancel fetches before metadata alter
2020-06-18 11:32:16 +03:00
alesapin
870d891257 Cancel fetches before metadata alter 2020-06-17 23:25:43 +03:00
Azat Khuzhin
caa88777be Parse metadata stored in zookeeper before checking for equality
After #11325 trailing whitespaces has been removed for data skipping
indicies, and it may be different, if you have multiple skip indices,
and in this case new server will not load such tables, because metadata
will be different.

Fix this by re-parse metadata in zookeeper.
2020-06-17 21:03:22 +03:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
alesapin
ef8781cce7 Better getVirtuals method 2020-06-17 17:37:21 +03:00
alesapin
33c27de54d Check methods in metadata 2020-06-17 17:32:25 +03:00
alesapin
31abbe5dbd Select query in metadata 2020-06-17 17:06:22 +03:00
alesapin
eaaef83742 Settings changes in StorageInMemoryMetadata 2020-06-17 16:46:01 +03:00
alesapin
ed8f3b2fc4 TTL in storage in memory metadata 2020-06-17 16:39:26 +03:00
Vitaly Baranov
7011401cd4
Merge pull request #11670 from vitlibar/rbac-improve-parser-multiple-entities
RBAC: improve syntax
2020-06-17 16:26:02 +03:00
alesapin
7064a366e2 Missed change for primary key 2020-06-17 15:40:27 +03:00
alesapin
1afdebeebd Primary key in storage metadata 2020-06-17 15:39:20 +03:00
alesapin
1da393b218 Sampling key in StorageInMemoryMetadata 2020-06-17 15:07:09 +03:00
alesapin
eca6e9087e Fix race condition 2020-06-17 14:52:19 +03:00
alesapin
ba04d02f1e Compilable sorting key in metadata 2020-06-17 14:05:11 +03:00
Nikita Mikhaylov
d2be6c036b
Merge pull request #11602 from ClickHouse/fix-buffer-sample
Fix buffer sample
2020-06-17 14:44:48 +04:00
Nikita Mikhaylov
ff0262626a
Merge pull request #11645 from azat/load-balancing-round-robin
Add round_robin load_balancing
2020-06-17 14:34:59 +04:00
alesapin
ab61abccc1 Partition key in StorageInMemoryMetadata 2020-06-17 13:34:23 +03:00
alesapin
62f2c17a66 Secondary indices in StorageInMemoryMetadata 2020-06-17 12:38:47 +03:00
alesapin
2fac3290f0 Merge branch 'master' into atomic_metadata5 2020-06-17 11:14:03 +03:00
alexey-milovidov
e07d02015c
Merge pull request #11661 from ClickHouse/return-not-nullable-from-count-distinct-2
Return non-Nullable results from COUNT(DISTINCT), more complete.
2020-06-17 09:25:45 +03:00
alexey-milovidov
598fd75caf
Merge pull request #11639 from ClickHouse/remove-leader-election-2
Remove leader election, step 2: allow multiple leaders
2020-06-17 09:25:23 +03:00
alexey-milovidov
6e61f708d7
Merge pull request #11690 from ClickHouse/odbc-bridge-timeouts
Pass proper timeouts when communicating with XDBC bridge.
2020-06-17 03:18:29 +03:00
Alexey Milovidov
a655765040 Merge branch 'master' into return-not-nullable-from-count-distinct-2 2020-06-17 01:15:57 +03:00
alexey-milovidov
f387752cf1
Merge pull request #11702 from ClickHouse/fix-mv-header
Fix mv header
2020-06-16 22:43:38 +03:00
Alexey Milovidov
0a6054eece Merge branch 'master' into return-not-nullable-from-count-distinct-2 2020-06-16 22:31:33 +03:00
Alexey Milovidov
b7ec5f94f2 Merge branch 'master' into odbc-bridge-timeouts 2020-06-16 22:14:43 +03:00
alesapin
fa60903620 Fix race condition 2020-06-16 21:41:11 +03:00
alesapin
d214a504d7
Merge pull request #11681 from ClickHouse/fix_mutations_premature_finish
Fix mutations premature "finish"
2020-06-16 21:26:14 +03:00
alesapin
ccc2bda666 getConstraints() in StorageInMemoryMetadata (suspicious commit, but pretend to work) 2020-06-16 19:55:04 +03:00
alesapin
1ddeb3d149 Buildable getSampleBlock in StorageInMemoryMetadata 2020-06-16 18:51:29 +03:00
alesapin
71f99a274d Compileable getSampleBlockWithColumns in StorageInMemoryMetadata 2020-06-16 17:25:08 +03:00
Ivan Blinkov
7e2bb1fd83 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-16 16:02:55 +03:00
alesapin
08b9aa6b2e getSampleBlockWithVirtuals in StorageInMemoryMetadata 2020-06-16 15:58:05 +03:00
alesapin
53cb5210de Move getSampleBlockNonMaterialized to StorageInMemoryMetadata 2020-06-16 15:48:10 +03:00
alesapin
0bcd22008a Get column dependencies in StorageInMemoryMetadata 2020-06-16 15:19:21 +03:00
alesapin
0f286ac133 Copy some methods to metdata 2020-06-16 15:03:27 +03:00
alesapin
eae901f58d Merge branch 'master' into atomic_metadata5 2020-06-16 14:44:45 +03:00
alesapin
0a33ccb77b Fix remove method 2020-06-16 14:12:52 +03:00
alesapin
d1052f2dfe Better logic 2020-06-16 13:50:47 +03:00
alesapin
d5b76b2fe1 Fix parts count in mutations 2020-06-16 13:34:59 +03:00
alesapin
0cb14c836c
Merge pull request #11691 from ClickHouse/fix-race-condition-system-sync-replicas
Fix race condition in SYSTEM SYNC REPLICA
2020-06-16 13:29:18 +03:00
alesapin
362aaf1385
Merge pull request #11516 from ClickHouse/consistent_metadata4
Storage metadata as a single struct.
2020-06-16 13:28:22 +03:00
alesapin
605c904fc6 Add method for compatibility 2020-06-16 11:39:12 +03:00
Nikolai Kochetov
66c0ef2610 Do not materialize header for merge tables. 2020-06-16 11:08:32 +03:00
Anton Popov
b19d48a11c Merge remote-tracking branch 'upstream/master' into HEAD 2020-06-16 06:37:55 +03:00
Alexey Milovidov
125b007360 Merge branch 'master' into remove-leader-election-2 2020-06-16 05:56:29 +03:00
Alexey Milovidov
22a92faab6 Avoid connection to replica when fetches are cancelled 2020-06-16 05:14:53 +03:00
Alexey Milovidov
4360e326ed Fix clang-static-analyzer 2020-06-16 04:17:02 +03:00
Alexey Milovidov
6da1a0f153 Fix style 2020-06-16 04:13:45 +03:00
Alexey Milovidov
53d985909e Fix race condition in SYSTEM SYNC REPLICA 2020-06-16 01:54:19 +03:00
Alexey Milovidov
186d336e31 Use proper timeouts when communicating with xdbc-bridge 2020-06-16 01:35:15 +03:00
Anton Popov
a3ac224ae4 in-memory parts: sync insert to replicated 2020-06-15 22:57:12 +03:00
Alexey Milovidov
d10109dc38 Remove excessive statement #11131 2020-06-15 22:18:33 +03:00
alesapin
36ba0192df Metadata in read and write methods of IStorage 2020-06-15 22:08:58 +03:00
Vitaly Baranov
15132d47c9 Slightly improve syntax of CREATE POLICY.
Also rename column source=>storage in table system.row_policies.
2020-06-15 22:07:42 +03:00
Alexey Milovidov
e44e1ad0d4 Don't ignore duplicate parts written to replicas 2020-06-15 21:57:38 +03:00
alesapin
b47a7327fd All set methods in metadata 2020-06-15 21:08:05 +03:00
alesapin
33a74a3ea0 TTL methods in StorageInMemoryMetadata 2020-06-15 20:50:53 +03:00
alesapin
e667eb57b2 Working set*Keys methods 2020-06-15 20:24:49 +03:00
alesapin
5fc41c7ecc Move set*Key methods to StorageInMemoryMetadata 2020-06-15 20:17:06 +03:00
Vitaly Baranov
4bd00b02e2 Improve syntax of CREATE QUOTA. Now resource types and key types could be written with underscores.
Also rename columns key_type=>keys and source=>storage in table system.quotas.
2020-06-15 20:10:34 +03:00
Vitaly Baranov
92b9f4a88d Rename ExtendedRoleSet => RolesOrUsersSet. 2020-06-15 20:10:34 +03:00
alesapin
aa30649ce5 Working setColumns, setConstraints, setIndices 2020-06-15 20:10:14 +03:00
alesapin
af2fe2ba55 Compilable setColumns, setConstraints, setIndices 2020-06-15 19:55:33 +03:00
alesapin
8f66c1b22c Merge with master 2020-06-15 17:31:37 +03:00
alesapin
8945d0073a Fix misunderstanding bug in mutations finalization 2020-06-15 16:37:40 +03:00
Artem Zuikov
24059efad5
Change push down logic in VIEW (#11513) 2020-06-15 15:36:10 +03:00
alesapin
2c9ce0f3fa
Bump CI 2020-06-15 13:14:36 +03:00
alesapin
bc9a3c0458 Merge branch 'master' into consistent_metadata4 2020-06-15 11:20:30 +03:00
alesapin
c20ce687cf
bump ci 2020-06-15 11:12:01 +03:00
alexey-milovidov
fd6115cccf
Merge pull request #11648 from ClickHouse/simplify-code-functions-comparison-3
Allow comparison with String in index analysis; simplify code.
2020-06-15 05:47:03 +03:00
Alexey Milovidov
d2c66f9688 Added comments 2020-06-15 05:14:59 +03:00
Alexey Milovidov
bbe5f4c909 Revert "Remove "current_password" because it is harmful"
This reverts commit 66ccb2f6b1.
2020-06-15 05:13:41 +03:00
Alexey Milovidov
5866401f60 Less noise in cleanup thread 2020-06-15 05:12:06 +03:00
Alexey Milovidov
66ccb2f6b1 Remove "current_password" because it is harmful 2020-06-15 04:12:01 +03:00
Alexey Milovidov
689b6901f8 Fix typo 2020-06-15 04:04:42 +03:00
Alexey Milovidov
1c438a133e Leader election both backward and forward compatible 2020-06-15 03:59:12 +03:00
Alexey Milovidov
b596831342 Merge branch 'master' into remove-leader-election-2 2020-06-15 03:16:41 +03:00
Alexey Milovidov
8dac30ae95 Split file for better build times 2020-06-14 21:42:10 +03:00
kssenii
9e1b8b2872 Better exchanges, fix build, better comments, better tests 2020-06-14 18:36:56 +00:00
Alexey Milovidov
f6c52fe1c2 Allow comparison with String in index analysis; simplify code #11630 2020-06-14 21:31:42 +03:00
alexey-milovidov
561d4b5b01
Merge pull request #11655 from ClickHouse/position-in-system-columns
Added column "position" to system.columns and "column_position" to system.parts_columns
2020-06-14 21:01:41 +03:00
Alexey Milovidov
394fb64a9c Better way of implementation 2020-06-14 20:42:11 +03:00
alexey-milovidov
87223b2cb7
Merge pull request #11652 from ClickHouse/allow-drop-and-rename-without-zookeeper
Allow to drop table if there is no metadata in ZooKeeper; allow to rename.
2020-06-14 09:00:38 +03:00
Alexey Milovidov
d6cf62e587 Added column "position" to system.columns and "column_position" to system.parts_columns 2020-06-14 04:56:22 +03:00
Alexey Milovidov
400e9fb64f Allow to drop table if there is no metadata in ZooKeeper; allow to rename 2020-06-14 04:23:53 +03:00
Azat Khuzhin
c139a05370 Forward declaration in StorageDistributed 2020-06-14 01:09:21 +03:00
kssenii
dcd7b7351c Support headers-exchange type 2020-06-13 21:47:22 +00:00
Alexey Milovidov
2cd82a25f5 Remove trivial count query optimization if row-level security is set #11352 2020-06-13 23:59:20 +03:00
kssenii
b8a4c7708a Make local exchanges unique for each table 2020-06-13 18:15:59 +00:00
alexey-milovidov
f53da4d36b
Merge pull request #11608 from ClickHouse/fix-nullable-prewhere-type
Fix nullable prewhere type
2020-06-13 20:29:12 +03:00
alesapin
3427da1c43 Fix incorrect usage of rows TTL 2020-06-13 13:39:25 +03:00
alesapin
18b58e8483 Revert too strict lock 2020-06-13 12:12:45 +03:00
alesapin
d636cdf4b0 Remove outdated comment 2020-06-13 11:55:03 +03:00
alesapin
31b852c46d Remove redundant locks 2020-06-13 11:53:40 +03:00
alesapin
fcfb6d3bc2 Merge with master 2020-06-13 11:51:07 +03:00
alesapin
5d3de747cd Merge remote-tracking branch 'origin/master' into consistent_metadata4 2020-06-13 11:50:27 +03:00
Alexey Milovidov
6f0db5ef10 Fix error 2020-06-13 04:35:49 +03:00
Alexey Milovidov
21897f2abd Instrument 2020-06-13 04:35:49 +03:00
Alexey Milovidov
6ff671b092 Improvement 2020-06-13 04:35:49 +03:00
Alexey Milovidov
ab00e34305 Miscellaneous 2020-06-13 04:35:49 +03:00
Alexey Milovidov
85c0706901 Step 2: allow multiple leaders 2020-06-13 04:35:49 +03:00
Alexey Milovidov
b8b55a5b99 More LeaderElection to Storage/MergeTree 2020-06-13 04:35:49 +03:00
Alexey Milovidov
52ac009754 Update versions on merge and mutation 2020-06-13 04:34:05 +03:00
Alexey Milovidov
833c8178ae Whitespaces 2020-06-13 04:34:05 +03:00
Alexey Milovidov
6869c122a8 Step 1 towards removing leader election: check and modify version of the "log" node 2020-06-13 04:34:05 +03:00
alexey-milovidov
54b77ff967
Merge pull request #11633 from ClickHouse/avoid-errors-using-zk-api
Avoid errors due to implicit int<->bool conversions when using ZK API
2020-06-13 04:32:32 +03:00
kssenii
462e8bcdc9 Support transactions for publishing 2020-06-12 18:10:13 +00:00
kssenii
3b0a3e00c0 Some fixes 2020-06-12 18:09:49 +00:00
alexey-milovidov
fc8da2abd3
Merge pull request #11613 from azat/system.zookeeper-path-prepend-slash
Prepend slash for path while querying system.zookeeper
2020-06-12 18:19:10 +03:00
Alexey Milovidov
72257061d5 Avoid errors due to implicit int<->bool conversions when using ZK API 2020-06-12 18:09:12 +03:00
Nikolai Kochetov
83155e139c Try fix tests. 2020-06-12 17:59:14 +03:00
alesapin
0edf5ff7a2 Fix race condition 2020-06-12 17:32:47 +03:00
Ivan Blinkov
e98a91e9f7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-12 16:53:00 +03:00
alesapin
465c4b65b7 Slightly better interfaces and comments 2020-06-12 12:37:52 +03:00
alesapin
4eb8a1cfef Fix gcc build 2020-06-12 00:03:25 +03:00
Alexey Milovidov
540f8532cb Fix error 2020-06-11 22:59:15 +03:00
Azat Khuzhin
176ebb26e0 Prepend slash to path for system.zookeeper queries
path should starts with '/', otherwise ZBADARGUMENTS will be thrown in
ZooKeeper::sendThread and the session will fail.
2020-06-11 20:58:53 +03:00
Nikolai Kochetov
7b8c1657a5 Fix nullable prewhere column type. 2020-06-11 19:34:36 +03:00
alesapin
a39d010f94 Merge branch 'master' into consistent_metadata4 2020-06-11 18:47:41 +03:00
alexey-milovidov
5f073c3ef4
Update StorageReplicatedMergeTree.cpp 2020-06-11 17:11:13 +03:00
Nikolai Kochetov
ed0826efa7 Remove extra columns from sample in StorageBuffer. 2020-06-11 15:18:45 +03:00
kssenii
626eb53baa Fix multiple bindings for single queue & rm hardcoded strings 2020-06-11 11:13:13 +00:00
kssenii
9350472ee4 Support multiple bindings 2020-06-11 10:29:08 +00:00
Alexey Milovidov
790bc4e8e0 Better code 2020-06-11 06:35:59 +03:00
Alexey Milovidov
b72a095e99 Fix race conditions in CREATE/DROP of different replicas of ReplicatedMergeTree 2020-06-11 06:24:52 +03:00
kssenii
18820814f5 Merge 2020-06-10 23:25:33 +00:00
kssenii
d7b2694806 Support all exchange types 2020-06-10 23:06:23 +00:00
Denis Glazachev
9e3a28a6b8 Merge branch 'master' into ldap-per-user-authentication
* master: (414 commits)
  Update file.md
  Update merge.md
  Update dictionary.md
  Update external-data.md
  Update distributed.md
  Update null.md
  Update set.md
  Update join.md
  Update url.md
  Update view.md
  Update materializedview.md
  Update memory.md
  Update buffer.md
  Update generate.md
  removed a sentence about global lock during rename (#11577)
  greatCircleAngle en translation (#11584)
  Update configuration-files.md
  try fix flacky test
  Update why.html
  Update rich.html
  ...

# Conflicts:
#	src/Common/ErrorCodes.cpp
#	utils/ci/jobs/quick-build/run.sh
2020-06-11 03:06:17 +04:00
kssenii
472b04b69c Better producer 2020-06-10 19:59:37 +00:00
alesapin
dd3cf0fe0a Don't miss columns TTLs update 2020-06-10 21:02:45 +03:00
kssenii
897a592ee6 Move subscription 2020-06-10 15:41:49 +00:00
alesapin
24db834046 Style 2020-06-10 14:27:16 +03:00
alesapin
8be957ecb5 Better checks around metadata 2020-06-10 14:16:31 +03:00
Ivan Blinkov
90ee4d52f4 merge master 2020-06-10 13:18:41 +03:00
alesapin
ec933d9d03 Better naming 2020-06-10 12:09:51 +03:00
alesapin
75977bd939 Gcc fixed 2020-06-10 00:22:01 +03:00
alesapin
4f398b5dea Merge remote-tracking branch 'origin/master' into consistent_metadata4 2020-06-09 23:53:38 +03:00
alesapin
4c77ba664a Better assignment 2020-06-09 20:42:04 +03:00
alesapin
d2fcf5aea5 Fixes for gcc 2020-06-09 20:28:29 +03:00
alesapin
936cc9d573 Update key 2020-06-09 20:22:03 +03:00
alesapin
4c2e0561f5 Fix primary key update 2020-06-09 20:21:38 +03:00
alesapin
6f673b0981 Fix comments bug 2020-06-09 19:33:28 +03:00
Nikolai Kochetov
5cc2f318c9
Merge pull request #11495 from vzakaznikov/live_view_support_for_prewhere
Adding support for PREWHERE in live view tables.
2020-06-09 19:09:17 +03:00
Ivan Blinkov
6f7064928f Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-09 19:07:57 +03:00
alesapin
f7a397242f Merge branch 'master' into consistent_metadata4 2020-06-09 18:59:59 +03:00
Alexander Kuzmenkov
dd41f6a574
Merge pull request #11527 from azat/dist-insert-local-fix
Fix async INSERT into Distributed for prefer_localhost_replica=0 and w/o internal_replication
2020-06-09 15:21:05 +03:00
Ivan
47ad338cb2
Refactor CMake build files (#11390)
* Get rid of lib_name.cmake
* Refactor Boost and HyperScan libraries
* Refactor lz4
* Fix build with xxHash
2020-06-09 13:54:49 +03:00
Ivan Blinkov
5b3fef526e Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-09 11:51:29 +03:00
alesapin
223f45685f Review fixes 2020-06-09 05:03:22 +03:00
alesapin
de59629b38 Fix benign race condition during shutdown 2020-06-09 04:48:30 +03:00
Azat Khuzhin
d2383f0f5d Fix async INSERT into Distributed for prefer_localhost_replica=0 and w/o internal_replication 2020-06-08 21:58:56 +03:00
alesapin
0d375e4522 Fix constraints 2020-06-08 21:49:54 +03:00
alesapin
a5a9048109 At least fixed gcc-9 build 2020-06-08 21:23:26 +03:00
alesapin
7b63a88120 Remove redundant error code 2020-06-08 18:19:41 +03:00
alesapin
d625162fce Better comment 2020-06-08 17:23:32 +03:00
alesapin
37a1d128c9 Better 2020-06-08 17:22:11 +03:00
alesapin
a30b72a128 Less debug 2020-06-08 17:21:12 +03:00
alesapin
2354939994 Remove log debug 2020-06-08 17:19:20 +03:00
alesapin
638cd50ce4 Fix several bugs 2020-06-08 17:18:38 +03:00
alesapin
2226f79f16 Fix some bugs 2020-06-08 15:57:25 +03:00
alesapin
ac67d0485c Merge branch 'master' into consistent_metadata4 2020-06-08 15:06:51 +03:00
Vitaliy Zakaznikov
5fe67c4292 * Removing supportsPrewhere() from StorageLiveView.h as it is not valid.
* Updating test to check using PREWHERE in query against live view table.
* Updating test to check using PREWHERE in the stored query against the table that does
  not support PREWHWERE.
2020-06-08 13:12:40 +02:00
kssenii
17e7cc03c0 Add consumer failure handler 2020-06-08 01:11:48 +00:00
Vitaliy Zakaznikov
75c97ca82a Adding support for PREWHERE in live view tables. 2020-06-08 00:19:23 +02:00
Alexey Milovidov
f2cfb7d132 Whitespaces 2020-06-08 00:05:36 +03:00
kssenii
ce448d9291 Better event handler 2020-06-07 11:24:10 +00:00
alexey-milovidov
e3c528e661
Merge pull request #11174 from excitoon-favorites/s3tests
Added tests for improved S3 URL parsing
2020-06-07 02:44:40 +03:00
Ivan
d2178cbacd
Merge pull request #11388 from filimonov/kafka_better_settings
Better settings for Kafka
2020-06-07 01:26:21 +03:00
alexey-milovidov
5ff63d9ad5
Merge pull request #11486 from ClickHouse/speed-up-build
Speed up build by removing old example programs
2020-06-07 00:22:03 +03:00
Ivan
657ce426d3
Fix build
Checked locally - no PR
2020-06-06 22:27:48 +03:00
BayoNet
e9b7919e12
DOCS-271: Updated the MergeTree() ORDER BY description (#11433)
* CLICKHOUSEDOCS-271: Updated the MergeTree() ORDER BY description.

* CLICKHOUSEDOCS-271: Fixes grammar.

* CLICKHOUSEDOCS-271: Updated by comments.

Co-authored-by: Sergei Shtykov <bayonet@yandex-team.ru>
2020-06-06 20:44:48 +03:00
Ivan
058821fa02
Merge branch 'master' into kafka_better_settings 2020-06-06 19:50:38 +03:00
Alexey Milovidov
d3d02a67f4 Speed up build by removing old example programs 2020-06-06 19:19:16 +03:00
alexey-milovidov
3e84b17cf8
Merge pull request #11394 from azat/system.distribution_queue
Add system.distribution_queue table
2020-06-06 18:59:59 +03:00
alexey-milovidov
82e849e6a1
Update StorageDistributed.cpp 2020-06-06 18:57:52 +03:00
Anton Popov
5c42408add
Merge pull request #9113 from dimarub2000/group_by_in_order_optimization
[WIP] Optimization of GROUP BY with respect to table sorting key.
2020-06-06 14:25:59 +03:00
Artem Zuikov
c58f9bd43d refactoring: remove TableWithColumnNames 2020-06-06 00:17:00 +03:00
Anton Popov
66e31d4311 in-memory parts: several fixes 2020-06-05 23:49:24 +03:00
Ivan Blinkov
653a9afda0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-05 22:51:28 +03:00
alesapin
abaf47f0cd Make metadata single structure 2020-06-05 20:29:40 +03:00
kssenii
cb618a32b8 Fix style 2020-06-05 14:27:56 +00:00
kssenii
68b94c5c20 Fixes 2020-06-05 13:46:20 +00:00
alesapin
2b23d1aa33 Fix reference 2020-06-05 15:13:24 +03:00
alesapin
5bc7f67e61 Better metadata for select query and renames 2020-06-05 14:54:54 +03:00
Alexander Kuzmenkov
eaaf005a03
Merge pull request #11410 from ClickHouse/aku/logical-error-synonyms
Remove assorted synonyms of LOGICAL_ERROR
2020-06-05 13:10:40 +03:00
Azat Khuzhin
ff85125326 Fix readability-qualified-auto 2020-06-04 20:23:46 +03:00
Alexander Kuzmenkov
029005be7c clang-tidy 2020-06-04 17:26:16 +03:00
alesapin
22235c1951 Tiny fixes for storage s3 2020-06-04 16:48:20 +03:00
alesapin
bba558d68e Merge branch 'custom-auth' of https://github.com/GrigoryPervakov/ClickHouse into merging_s3_auth 2020-06-04 16:19:04 +03:00
Vladimir Chebotarev
d62e85f875 Improved logging of globs in S3. 2020-06-04 16:12:08 +03:00
Nikita Mikhaylov
4e4218e10d
Merge pull request #11385 from ClickHouse/akz/bg-processing-pool-update1
Extended BackgroundProcessingPool::startTask() to control task execution concurrency
2020-06-04 16:19:29 +04:00
Mikhail Filimonov
26d93fdfba Fix the test, adjust default for queued.min.messages 2020-06-04 14:14:24 +02:00
Alexander Kuzmenkov
30d7d41ebc
Merge pull request #11381 from ClickHouse/aku/fuzz-related
Fuzzing-related changes.
2020-06-04 15:06:11 +03:00
Alexander Kuzmenkov
0943ef3e4b Do not set global context twice in unit tests. 2020-06-04 15:04:05 +03:00
Anton Popov
4a3d3c6e54
Merge pull request #11419 from CurtizJ/polymorphic-parts-2
Return lost comments and default values
2020-06-04 13:32:38 +03:00
kssenii
972611e31b Fix consumer 2020-06-04 06:22:53 +00:00
kssenii
5624066195 Fix producer 2020-06-04 06:21:28 +00:00
Azat Khuzhin
86c5465bf8 Rewrite StorageSystemDistributionQueue interfaces 2020-06-04 03:04:32 +03:00
Azat Khuzhin
aa13bd9530 Fix possible SIGSEGV in StorageSystemDistributionQueue
Refs: #11348
2020-06-04 03:04:28 +03:00
Azat Khuzhin
f0050adc51 Make system.distribution_queue metrics non racy 2020-06-04 02:36:16 +03:00
Azat Khuzhin
09c3ca9c6c Add last_exception into system.distribution_queue 2020-06-04 02:36:16 +03:00
Azat Khuzhin
389f78ceee Add system.distribution_queue
system.distribution_queue contains the following columns:
- database
- table
- data_path
- is_blocked
- error_count
- data_files
- data_compressed_bytes
2020-06-04 02:36:16 +03:00
Azat Khuzhin
60d10f1bac Fix typo in StorageDistributed 2020-06-04 02:36:16 +03:00
Anton Popov
b312ac9786 in-memory parts: fix columns sizes 2020-06-04 01:53:04 +03:00
Alexey Milovidov
484d269c22 Fix typo 2020-06-04 01:15:13 +03:00
Alexey Milovidov
a0fd9b021a Fix MSan failure in MergeTree 2020-06-04 01:11:06 +03:00
Anton Popov
df3dfd5b81 fix clang-tidy build 2020-06-04 01:00:02 +03:00
Anton Popov
1980d58ba6 add lost comments and default values 2020-06-04 00:30:45 +03:00
Anton Popov
11c4e9dde3 in-memory parts: fix 'check' query 2020-06-03 22:19:49 +03:00
Mikhail Filimonov
eef914cbdf Remove debug print 2020-06-03 20:34:22 +02:00
Mikhail Filimonov
94261b9786 Fix issues detected by clang tidy, fix issue with flush timeout 2020-06-03 19:02:24 +02:00
Mikhail Filimonov
95f4c46aa4 Better control of context copies, some cosmetic changes, client.software.name and client.software.version now initialized 2020-06-03 19:01:45 +02:00
Mikhail Filimonov
4956ade67d Support for all format settings in Kafka, expose some setting on table level, better defaults. 2020-06-03 19:01:41 +02:00
Alexander Kuzmenkov
1ab3201454 Merge remote-tracking branch 'origin/master' into HEAD 2020-06-03 16:36:22 +03:00
Anton Popov
1ce09e1faa Merge remote-tracking branch 'upstream/master' into polymorphic-parts 2020-06-03 16:27:54 +03:00
Anton Popov
caf1e4e8cc in-memory-parts: fixes 2020-06-03 12:51:23 +03:00
Alexander Kuzmenkov
c7d9094a7a
Merge pull request #11259 from ClickHouse/consistent_metadata3
More consistent metadata for secondary indices
2020-06-03 12:23:21 +03:00
Alexander Kuzmenkov
f7bb6d57a4 build 2020-06-03 11:50:27 +03:00
Alexander Kazakov
f999021ee1 Submit to clang10's demands of syntactic purity 2020-06-03 11:12:56 +03:00
alexey-milovidov
66eb56631c
Merge pull request #11355 from ClickHouse/fix_some_integration_tests
Fix several flaky integration tests
2020-06-03 02:35:33 +03:00
Alexander Kazakov
0cd4e6e5ba Trying to fix clang10 build 2020-06-03 00:52:29 +03:00
Denis Glazachev
ca1381d475 Write empty json in more efficient way 2020-06-03 01:17:59 +04:00
Denis Glazachev
9331387340 Merge branch 'master' into ldap-per-user-authentication
* master: (29 commits)
  DOCS-620: Background pools settings (#11358)
  Update formats.md
  ya.make
  DOCS-611: background_buffer_flush_schedule_pool_size setting (#11176)
  Remove structure lock on parts send
  Update adopters.md
  This word "collapse" in Chinese means "折叠” (#11353)
  interval type link (#11351)
  Try fix tets.
  DOCS-573: system.data_type_families (#11344)
  fix flacky test with ttl
  Added test result
  Fix error with Lazy database
  Fix nullptr dereference in SYSTEM query and Lazy database
  Better code
  Added a test
  Fix unit test with UBSan
  Fix tests.
  Try fix build.
  Try fix build.
  ...
2020-06-03 00:48:21 +04:00
Alexander Kazakov
9d025337f5 Fixed minor inconsistency with master branch 2020-06-02 21:56:36 +03:00
alesapin
6253e9b97b Revert disabled tests 2020-06-02 21:41:29 +03:00
Pervakov Grigorii
bc9381406f Add default credentials and custom headers for s3 table functions. 2020-06-02 20:48:49 +03:00
Alexander Kuzmenkov
a859afa174 macos build fix 2020-06-02 19:57:19 +03:00
kssenii
786874e867 Better publish & some fixes 2020-06-02 15:57:30 +00:00
Alexander Kazakov
8986983392 insignificant typo 2020-06-02 18:22:05 +03:00
Alexander Kazakov
9fbd72ab34 Renamed method 2020-06-02 18:18:36 +03:00
Alexander Kazakov
013dc2d7c2 Updated startTask() method + reworked some of the worker loop 2020-06-02 17:37:10 +03:00
alexey-milovidov
ff7262968a
Merge pull request #11354 from ClickHouse/get_rid_of_lock_during_send
Remove structure lock on parts send
2020-06-02 17:05:24 +03:00
Alexander Kuzmenkov
0a5cc96b67 Fuzzing-related changes.
* More LOGICAL_ERROR
* Proper cloning of some Asts
* Field::safeGet for user-supplied values
2020-06-02 16:56:10 +03:00
alexey-milovidov
d47d0d7108
Merge pull request #11346 from ClickHouse/optimize-final-should-force-merge-2
OPTIMIZE FINAL should force merge even if concurrent merges are performed (experimental, try 2)
2020-06-02 16:40:20 +03:00
alexey-milovidov
9b07989954
Merge pull request #11348 from ClickHouse/lazy-database-segfault-2
Fix potential segfault when using Lazy database.
2020-06-02 16:37:23 +03:00
Nikolai Kochetov
9a283de670
Merge pull request #11339 from ClickHouse/remove-experimantal-use-processors-flag-5
Remove some streams.
2020-06-02 14:46:02 +03:00
Denis Glazachev
e75bb335e3 Fix: set auth_params column value for each row, by default to "{}" empty JSON object 2020-06-02 13:40:32 +04:00
Denis Glazachev
920e792f33 Add USE_LDAP to table system.build_options 2020-06-02 13:38:20 +04:00
Denis Glazachev
c61cbe8e98 Merge branch 'master' into ldap-per-user-authentication
* master: (114 commits)
  Update PushingToViewsBlockOutputStream.cpp
  Update PushingToViewsBlockOutputStream.cpp
  make clang-10 happy
  Fix sync_async test (remove timeout)
  CLICKHOUSEDOCS-631: temporary_files_codec, join_on_disk_max_files_to_merge settings. (#11242)
  Suppress output of cancelled queries in clickhouse-client #9473
  Better log messages in ConfigReloader
  fix select from StorageJoin
  Fix unit tests under MSan
  Added test.
  Fix build.
  Fix arguments for AggregateFunctionQuantile/
  Update style.md
  Add a guide on error messages.
  Report dictionary name on dictionary load errors.
  more types in ASOF JOIN (#11301)
  Fix part_log test
  Update test.
  Add perftest.
  Parallel processing for PushingToViewsBlockOutputStream::writeSuffix
  ...
2020-06-02 12:42:29 +04:00
alesapin
42b6370b67 Remove structure lock on parts send 2020-06-02 11:10:21 +03:00
alesapin
e5eb6071a4 Merge with master 2020-06-02 10:46:58 +03:00
alesapin
1ab3047201 Merge branch 'master' into consistent_metadata3 2020-06-02 10:37:12 +03:00
Ilya Yatsishin
a101b17df7
Merge pull request #11335 from ClickHouse/stop_merges_before_alter
Stop merges before taking exclusive lock :(
2020-06-02 09:34:05 +03:00
Ivan Blinkov
4bd678a3fd Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-02 08:21:30 +03:00
Alexey Milovidov
a6bf4b5d70 Fix error with Lazy database 2020-06-02 05:06:16 +03:00
Alexey Milovidov
78d966f4e5 Better code 2020-06-02 03:41:52 +03:00
kssenii
806fd27395 Fix build & fix style & fix 2020-06-01 20:48:24 +00:00
alesapin
ad6447398f Fix indices calculation during merge 2020-06-01 22:58:30 +03:00
Artem Zuikov
b7958d4f06 make clang-10 happy 2020-06-01 22:38:51 +03:00
kssenii
d3b069e37b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into add-storage-rabbitmq-read-only 2020-06-01 18:25:36 +00:00
Anton Popov
e1970f6d28 in-memory parts: fix reading of nested 2020-06-01 20:53:18 +03:00
Artem Zuikov
bfa59adbfe fix select from StorageJoin 2020-06-01 20:27:22 +03:00
Nikolai Kochetov
53d12f5ab8 Try fix build. 2020-06-01 20:06:21 +03:00
Nikolai Kochetov
d25326e75c Try fix build. 2020-06-01 20:03:57 +03:00
kssenii
386dc4d95e Fixes 2020-06-01 17:02:56 +00:00
kssenii
5757dd1d57 Add insert part 2020-06-01 17:01:41 +00:00
Nikolai Kochetov
325393d340 Try fix build. 2020-06-01 19:58:36 +03:00
Nikolai Kochetov
3a0acb00c1 Remove some streams. 2020-06-01 19:31:06 +03:00
alesapin
ce6db6927b
boop CI 2020-06-01 19:22:33 +03:00
alesapin
f287a28fc5 Stop merges before taking exclusive lock :( 2020-06-01 18:16:10 +03:00
alesapin
e0e77194f4 Remove redundant parts from MergeTree 2020-06-01 15:39:20 +03:00
alesapin
b49be4c7f2 Better names 2020-06-01 15:11:23 +03:00
alesapin
254d55f2ab Better method name 2020-06-01 15:02:36 +03:00
alesapin
663e92b1c5 Rename to methods 2020-06-01 14:29:11 +03:00
alesapin
d9a817f578 Less virtual methods in IStorage 2020-06-01 14:17:18 +03:00
Ivan Blinkov
89212fb174 Merge branch 'sentry' of github.com:blinkov/ClickHouse; branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-01 14:14:44 +03:00
alesapin
69a9c856a6 Better variable name 2020-06-01 13:54:31 +03:00
alesapin
72c17a8e3a Slightly more comments 2020-06-01 13:48:52 +03:00
alesapin
3847ea892d Merge branch 'master' into consistent_metadata3 2020-06-01 13:17:59 +03:00
alesapin
c22e8f6b9b
Merge pull request #11283 from filimonov/kafka_virtual_column_headers
Virtual columns for Kafka headers
2020-06-01 12:06:00 +03:00
alesapin
364ebe9246
Merge pull request #11260 from filimonov/kafka_virtual_column_timestamp_ms
Add _timestamp_ms virtual columns to Kafka engine
2020-06-01 12:04:17 +03:00
Ivan Blinkov
fcc7ff0ef9 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-01 08:42:03 +03:00
alexey-milovidov
256f84ddc2
Merge pull request #11230 from Jokser/s3-poco-http-client
S3 Poco Http Client
2020-06-01 01:20:21 +03:00
alexey-milovidov
9984e989c1
Merge pull request #11317 from ClickHouse/logger-fmt-2
Preparation for structured logging
2020-06-01 00:11:29 +03:00
Alexey Milovidov
07c14e12dd Fix build after merge of #11162 with master 2020-05-31 19:53:30 +03:00
alexey-milovidov
8accde79b5
Merge pull request #11162 from azat/data-skip-index-merging-params-fix
[RFC] Fix data skipping indexes for columns with additional actions during merge
2020-05-31 19:10:53 +03:00
alexey-milovidov
9416e82b75
Update MergeTreeDataMergerMutator.cpp 2020-05-31 19:06:59 +03:00
alexey-milovidov
9a362eb02e
Merge pull request #11252 from filimonov/kafka-clientid
Kafka clientid (finishing #11073)
2020-05-31 18:37:30 +03:00
Denis Glazachev
e8144976fe Merge branch 'master' into ldap-per-user-authentication 2020-05-31 15:09:16 +04:00
kssenii
e80b405359 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into add-storage-rabbitmq-read-only 2020-05-31 10:30:53 +00:00
kssenii
037ed3a02c Code fix & style fix & merge fix 2020-05-31 09:34:57 +00:00
kssenii
8266715c49 Fix build & fix style 2020-05-31 08:39:22 +00:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Alexey Milovidov
5aff138956 Preparation for structured logging 2020-05-31 00:35:52 +03:00
Dmitry
4b0d32f026 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-31 00:21:02 +03:00
alexey-milovidov
3eea042d16
Merge pull request #11243 from ClickHouse/remove-experimental-use-processors-flag-4
Remove some code.
2020-05-30 21:05:14 +03:00
alexey-milovidov
1dbaf80a5d
Merge pull request #11303 from ClickHouse/fix-index-analysis-empty-array
Fix issue #11286; add a test
2020-05-30 17:28:00 +03:00
Alexey Milovidov
b78c48310b Whitespaces 2020-05-30 01:37:44 +03:00
Alexey Milovidov
8c8821475c Fix issue #11286; add a test 2020-05-30 00:32:35 +03:00
Ilya Yatsishin
393db8ff6c
Merge pull request #11261 from filimonov/kafka_log_with_prefix_and_facility
Add prefix & facility to librdkafka logs
2020-05-29 23:02:42 +03:00
Ivan Blinkov
95ee742ee7 merge master 2020-05-29 22:56:10 +03:00
tavplubix
a4e40fb5f2
Merge pull request #11269 from ClickHouse/add_context_to_idatabase_methods
Add context to IDatabase methods
2020-05-29 21:15:35 +03:00
Anton Popov
e6a9a33010
Merge pull request #11183 from ClickHouse/consistent_metadata2
Refactor TTLs metadata (move them to IStorage).
2020-05-29 20:00:11 +03:00
Anton Popov
fa05641b06 in-memory parts: partition commands 2020-05-29 19:58:08 +03:00
kssenii
0362bb2d2f Make connection between concurrent consumers shared - not private 2020-05-29 16:28:52 +00:00
Anton Popov
c919840722 in-memory parts: partition commands 2020-05-29 18:02:12 +03:00
alesapin
a429760641 Fix debug build 2020-05-29 17:10:09 +03:00
alesapin
748e8fbfc3 Merge with master 2020-05-29 14:03:11 +03:00
alesapin
2ff7bbb511 Better names 2020-05-29 13:56:04 +03:00
alesapin
0170250c5e Merge branch 'master' into consistent_metadata2 2020-05-29 13:48:09 +03:00
Denis Glazachev
cd5d3ed618 Merge branch 'master' into ldap-per-user-authentication
* master:
  Remove error code
  Better
  Remove redundant header
  Fix unit tests
  Better error codes
2020-05-29 14:07:42 +04:00
Denis Glazachev
bb68768ce1 Change auth_params type to string
Write LDAP_SERVER params (server name) to auth_params in JSON
2020-05-29 13:30:07 +04:00
Mikhail Filimonov
b885337454 Virtual columns for accessing kafka message headers 2020-05-29 10:44:10 +02:00
Anton Popov
4769ce7271 in-memory parts: several fixes 2020-05-29 02:59:13 +03:00
Alexander Tokmakov
c071e0a473 add context to DatabaseCatalog methods 2020-05-29 02:01:18 +03:00
Alexander Tokmakov
3cff070f89 add context to IDatabase methods 2020-05-28 23:10:45 +03:00
alesapin
026244b07f Better 2020-05-28 21:57:09 +03:00
alesapin
93801a1a9d Fix unit tests 2020-05-28 21:52:08 +03:00
Mikhail Filimonov
7211af9404 Fixing style 2020-05-28 20:19:10 +02:00
Mikhail Filimonov
b97ff8136b Make function static 2020-05-28 20:10:54 +02:00
Mikhail Filimonov
16de65a6f0 Add prefix & facility to librdkafka logs 2020-05-28 19:52:59 +02:00
Mikhail Filimonov
e39c4cdf71 Add _timestamp_ms virtual columns to Kafka engine 2020-05-28 19:41:57 +02:00
alesapin
eb5a8dccfa Fix ya.make 2020-05-28 20:32:22 +03:00
Vitaly Baranov
d7cc703233
Merge pull request #11080 from vitlibar/add-authentication-type-to-system-users
Show authentication type in system.users table
2020-05-28 20:20:41 +03:00
alesapin
66ccb30299 Fix style check 2020-05-28 20:17:05 +03:00
alesapin
860dc2c814 Merge with consistent_metadata2 2020-05-28 19:45:51 +03:00
alesapin
c5fe87e123 Less includes 2020-05-28 19:32:35 +03:00
alesapin
acc3e6b15a Better structure 2020-05-28 19:31:27 +03:00
alesapin
c6f82590ad Add missed files 2020-05-28 18:34:33 +03:00
alesapin
61aff798d3 Compileable code 2020-05-28 18:33:44 +03:00
Nikolai Kochetov
ea3f638b30 Try fix tests. 2020-05-28 17:11:36 +03:00
alesapin
099bfc4657 Fix bug with empty column 2020-05-28 17:03:21 +03:00
Mikhail Filimonov
4de1d6d421 Fixing style 2020-05-28 15:53:27 +02:00
alesapin
5f8b69547b More readable code 2020-05-28 16:45:08 +03:00
Mikhail Filimonov
6b694d54b4 Better kafka client.id 2020-05-28 15:39:55 +02:00
alesapin
f5e5290c0a Better naming 2020-05-28 16:09:03 +03:00
alesapin
5b5f749592 Split files 2020-05-28 16:03:18 +03:00
alesapin
26a9829e04 Indices description as vector 2020-05-28 15:47:17 +03:00
alesapin
52ca6b2051 I'm able to build it 2020-05-28 15:37:05 +03:00
Nikolai Kochetov
235fdc831b Merged with master. 2020-05-28 15:23:23 +03:00
tavplubix
801c1c1240
Merge pull request #11205 from ClickHouse/fix-view-totals
Fix totals and extremes header for view.
2020-05-28 15:05:34 +03:00
Nikolai Kochetov
da0052858d Fix build. 2020-05-28 13:57:04 +03:00
Nikolai Kochetov
1fa795988f Remove some code. 2020-05-28 11:24:59 +03:00
madianjun
e3c7ab2ca7 Add Kafka client.id 2020-05-28 08:32:51 +02:00
Vitaly Baranov
98172deffc Show authentication type in table system.users and while executing SHOW CREATE USER query. 2020-05-28 08:34:10 +03:00
alexey-milovidov
69d5fd0edb
Merge pull request #11220 from ClickHouse/keep-metric-distributed-queue-on-exception
Keep the value of DistributedFilesToInsert metric on exceptions
2020-05-28 01:23:11 +03:00
Anton Popov
d8342e5b12 in-memory parts: better restore and clear stale wal files 2020-05-27 23:05:55 +03:00
Pavel Kovalenko
677cb56a06 S3 Poco HTTP Client WIP. 2020-05-27 22:41:49 +03:00
Ivan Blinkov
709b4f42c8 Prototype sending crash reports on segfaults 2020-05-27 22:11:04 +03:00
alesapin
380c60a33f Intermediate stage (doesn't compile) 2020-05-27 21:38:34 +03:00
alexey-milovidov
dffeec8637
Merge pull request #10537 from expl0si0nn/ttl_expr_data_rollup
Support of WHERE and GROUP BY in TTL expressions
2020-05-27 21:35:55 +03:00
alesapin
18656841a3 Implement methods for index struct 2020-05-27 16:34:23 +03:00
alesapin
c4b2f72fe6 Add common struct for skip indices 2020-05-27 14:25:02 +03:00
Nikolai Sorokin
57555dbabf Fix after rebase 2020-05-27 14:00:17 +03:00
alexey-milovidov
992f0dfec5
Merge pull request #11149 from filimonov/kafka_reschedule
Fixed reschedule issue in Kafka
2020-05-27 13:26:55 +03:00
Alexey Milovidov
146370934a Keep the value of DistributedFilesToInsert metric on exceptions 2020-05-27 13:07:38 +03:00
Anton Popov
f62adfd588 Fix Arcadia build 2020-05-27 12:53:15 +03:00
Nikolai Sorokin
141ed88751 Allow functions in group by keys; Add default aggregate function; Add more tests 2020-05-27 12:53:15 +03:00
Nikolai Sorokin
66496dc7e3 Fix trailing spaces 2020-05-27 12:53:15 +03:00
Nikolai Sorokin
eeaf608aa9 Refactor code; support prefix of PK as GROUP BY key 2020-05-27 12:53:15 +03:00
Nikolai Sorokin
61974e0047 TTL Expressions WHERE and GROUP BY draft 2020-05-27 12:53:15 +03:00
alesapin
c7cda399c7 Merge branch 'master' into consistent_metadata2 2020-05-27 12:51:56 +03:00
alesapin
3399e573d1
Merge pull request #11199 from ClickHouse/fix_primary_key_method_call
Fix primary key check
2020-05-27 11:55:12 +03:00
Mikhail Filimonov
efc0b97707 Fix Kafka performance issue related to reschedules based on limits, which were always applied. 2020-05-27 07:39:06 +02:00
bobrovskij artemij
8d69223aaf show_privileges test fix, one more build fix 2020-05-27 01:20:25 +03:00
Dmitry
41d1cd1c9b fix bad merge 2020-05-27 01:17:32 +03:00
kssenii
5e472af425 Fix merge & small fix 2020-05-26 20:43:20 +00:00
Dmitry
38c585f867 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-26 21:27:50 +03:00
kssenii
ad1c0de8db Merge 2020-05-26 17:45:39 +00:00
kssenii
14c67c6ae6 Fixes 2020-05-26 17:34:57 +00:00
alesapin
9d0c28d96c Add missing methods 2020-05-26 17:14:08 +03:00
alesapin
8614487a5a Fix primary key check 2020-05-26 16:46:19 +03:00
Nikolai Kochetov
12384551da Fix totals and extremes header for views. 2020-05-26 16:19:24 +03:00
alesapin
6626ef08e8 Fix PVS studio errors 2020-05-26 15:14:50 +03:00
Vitaly Baranov
e27ded8798
Merge pull request #11172 from vitlibar/grpc-improve-build-scripts
Improve build scripts for protobuf & gRPC
2020-05-26 13:27:54 +03:00
alesapin
5780240e88 Fix primry key check 2020-05-26 12:46:04 +03:00
alesapin
7bbb064b34 Fix metadata update 2020-05-26 11:58:32 +03:00
Alexey Milovidov
c858f4d89c Clarifications 2020-05-26 02:47:21 +03:00
alexey-milovidov
d4acd1af3a
Merge pull request #11179 from excitoon-favorites/s3globfix
Fixed S3 globbing which could fail in case of more than 1000 keys and some backends
2020-05-26 02:45:42 +03:00
Alexey Milovidov
4bb3381bc9 Fix build after merge 2020-05-26 00:53:29 +03:00
filimonov
73a0ef6c0e
Fix for the hang during deletion of engine=Kafka (one more time) (#11145)
* Added flag for safer rdkafka destruction, but more testing detected another hang case, which has no straigt solutions and can be workarounded currenly only by draining the consumer queue, so destructor is back

* After review fixes

* After review fixes2

* After review fixes3
2020-05-26 00:13:10 +03:00
bobrovskij artemij
27c6eefc85 Merge branch 'master' into mongo 2020-05-25 23:33:38 +03:00
alesapin
355fbadb31 Fix style 2020-05-25 22:18:01 +03:00
alesapin
211d7e229f Better using 2020-05-25 21:03:04 +03:00
alesapin
adc7439a4c Comments and refactoring 2020-05-25 20:57:08 +03:00
alesapin
c3a6571036 Compilable code 2020-05-25 20:22:20 +03:00
alesapin
9b3cc9e525 Almost working ttl in IStorage 2020-05-25 20:07:14 +03:00
Vitaly Baranov
58a2172154 Improve build scripts related to protobuf and gRPC. 2020-05-25 18:46:57 +03:00
Vladimir Chebotarev
7f9a257c31 Fixed S3 globbing which could fail in case of more than 1000 keys and some backends. 2020-05-25 15:57:32 +03:00
alesapin
6281dd6893
Merge pull request #11115 from ClickHouse/consistent_metadata
Refactoring in storage metadata (more consistent keys)
2020-05-25 13:09:56 +03:00
bobrovskij artemij
a4d74601a6 build warnings fix 2020-05-25 00:42:05 +03:00
Alexey Milovidov
4ebf63fb23 Fixed error 2020-05-24 22:17:15 +03:00
Azat Khuzhin
c36708caf9 Split data skipping indexes from the order by to evaluate order by before merge 2020-05-24 16:12:08 +03:00
Azat Khuzhin
71731b2e99 Fix data skipping indexes for columns with additional actions during merge
For example for AggregatingMergeTree if the column in the data skipping
index is under some aggregate function, then data skipping index will be
incorrect.

Fix this by calculating data skipping indexes after aggregation has been
done.
2020-05-24 16:12:08 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
af2a59e49b Progress on task 2020-05-24 00:50:34 +03:00
Alexey Milovidov
c87fb91532 Progress on task 2020-05-24 00:41:35 +03:00
Alexey Milovidov
ce0619dabf Progress on task 2020-05-24 00:26:45 +03:00
Alexey Milovidov
2d7d5a1547 Apply all transformations again 2020-05-24 00:16:27 +03:00
Alexey Milovidov
bab24879e9 Progress on task 2020-05-24 00:16:05 +03:00
Alexey Milovidov
e1695feb7f Apply all transformations again 2020-05-23 23:40:32 +03:00
Alexey Milovidov
85f84550ba Progress on task 2020-05-23 23:37:37 +03:00
Alexey Milovidov
f75587e562 Progress on task 2020-05-23 23:13:34 +03:00
Alexey Milovidov
18febd7b97 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | while read file; do perl -pne 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/${1}_FORMATTED(${2}, "${3}{}${5}{}${7}{}${9}{}${11}", ${4}, ${6}, ${8}, ${10});/' $file > ${file}.tmp; mv ${file}.tmp $file; done 2020-05-23 22:56:05 +03:00
Alexey Milovidov
cd6d3c7509 Progress on task 2020-05-23 22:55:06 +03:00
Alexey Milovidov
d5b7883451 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7{}\9", \4, \6, \8);/' 2020-05-23 22:41:08 +03:00
Alexey Milovidov
7e2fb9ad65 Apply all transformations again 2020-05-23 22:38:30 +03:00
Alexey Milovidov
eacff92d0e Progress on task 2020-05-23 22:35:08 +03:00
Alexey Milovidov
29762240de Remove duplicate whitespaces (preparation) 2020-05-23 22:31:54 +03:00
Alexey Milovidov
7fed65cbe2 Remove duplicate whitespaces (preparation) 2020-05-23 22:14:58 +03:00
Alexey Milovidov
86b4d5a86f Remove duplicate whitespaces (preparation) 2020-05-23 22:09:06 +03:00
Alexey Milovidov
ab0562a574 Make all LOG in single line (preparation) 2020-05-23 22:05:41 +03:00
Alexey Milovidov
9d2a0d2dd7 Apply all transformations again 2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f Remove duplicate whitespaces (preparation) 2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
3384f3b77e Progress on task 2020-05-23 21:07:38 +03:00
Alexey Milovidov
fc412f2d9d Progress on task 2020-05-23 20:59:03 +03:00
Alexey Milovidov
9d24908e53 Progress on task 2020-05-23 20:52:11 +03:00
Alexey Milovidov
3a09f9ca1c find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\W_]+\([^,]+, [^<+)"]+\);' | xargs sed -i -r -e 's/(LOG_[^\W_]+)\(([^,]+), ([^<+)"]+)\);/\1_FORMATTED(\2, \3);/' 2020-05-23 20:29:56 +03:00
Alexey Milovidov
b3f4fe12dd Progress on task 2020-05-23 20:25:19 +03:00
Alexey Milovidov
d6bc4d2bd0 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "{}\4{}", \3, \5);/' 2020-05-23 20:14:57 +03:00
Alexey Milovidov
241f8c5431 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+ \+ "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+) \+ "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 20:10:21 +03:00
Alexey Milovidov
f69cbdcbfc find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 20:09:37 +03:00
Alexey Milovidov
f68d1ceb4f find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}\7{}", \4, \6, \8);/' 2020-05-23 20:02:09 +03:00
Alexey Milovidov
533f86278a find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7", \4, \6);/' 2020-05-23 20:00:41 +03:00
Alexey Milovidov
8042e5febe find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/' 2020-05-23 19:58:15 +03:00
Alexey Milovidov
e391b77d81 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 19:56:05 +03:00
Alexey Milovidov
ee4ffbc332 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 19:47:56 +03:00
Alexey Milovidov
8d2e80a5e2 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/' 2020-05-23 19:42:39 +03:00
Alexey Milovidov
35d79e1252 fmt-style logging, part 1 2020-05-23 19:33:07 +03:00
alexey-milovidov
ac6deaf8ec
Merge pull request #11036 from excitoon-favorites/s3fixes
Fixed parsing of S3 URL
2020-05-23 17:06:59 +03:00
Azat Khuzhin
bc4b75dead Add table name into logs for StorageDistributed 2020-05-23 11:57:14 +03:00
alesapin
cd83ebc599
Merge pull request #11106 from ClickHouse/fix_deadlock_system_logs_startup
Fix deadlock in system tables during server startup
2020-05-22 20:53:07 +03:00
alesapin
e812fa15d1 Merge branch 'consistent_metadata' into consistent_metadata2 2020-05-22 17:41:53 +03:00
alesapin
e971ff627e Fix gcc-9 build 2020-05-22 17:39:49 +03:00
alesapin
f026fa7de1 Rename PartDestinationType to DataDestionationType, because there is no part-specific information 2020-05-22 16:29:33 +03:00
alesapin
a935db4169
boop CI 2020-05-22 15:34:26 +03:00
alexey-milovidov
b82c633716
Merge pull request #11048 from filimonov/kafka_missed_data_during_drop
Fixes the potential missed data during termination of Kafka engine table
2020-05-22 14:11:10 +03:00
alesapin
f4b0d475af Fix system.tables output 2020-05-22 14:04:15 +03:00
alesapin
98ffefd90c Fix bug with return type 2020-05-22 13:09:24 +03:00
Vladimir Chebotarev
98f31374d5 Added improved S3 URL parsing to DiskS3. 2020-05-22 11:18:34 +03:00
alesapin
cbcdb6219e Slighty more verbose comment 2020-05-22 11:17:57 +03:00
Vladimir Chebotarev
02cb08ea2f Fixed parsing of S3 URL. 2020-05-22 11:17:10 +03:00
alesapin
b9dfa61774
Merge pull request #11042 from CurtizJ/modify-ttl
Apply TTL for old data, after ALTER MODIFY TTL query
2020-05-22 11:12:14 +03:00
Dmitry
47778c0259 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-21 23:45:12 +03:00
alesapin
59b3bc0c05
Merge branch 'master' into fix_deadlock_system_logs_startup 2020-05-21 22:57:52 +03:00
alesapin
09034157cd Remove comments 2020-05-21 22:53:46 +03:00
alesapin
327d17ac6a Better 2020-05-21 22:46:03 +03:00
alesapin
41914588fd More clear 2020-05-21 22:07:18 +03:00
alesapin
1906762119
Merge pull request #11109 from ClickHouse/less_verbose_logging
Less verbose logging in mutation finalization task
2020-05-21 21:53:08 +03:00
Alexander Kuzmenkov
f98ffdbc4c
Merge pull request #11087 from azat/context-fwd-decl
[RFC] Forward declaration for Context as much as possible.
2020-05-21 19:43:29 +03:00
alesapin
87eab514fb Merge branch 'master' into consistent_metadata 2020-05-21 18:54:52 +03:00
alesapin
5676340f6b Less verbose logging in mutation finalization task 2020-05-21 18:52:11 +03:00
alesapin
c036af0261 Fix deadlock after clickhouse-server update (with changes in one of system log tables structure) during startup between concurrent merge and table rename. 2020-05-21 17:11:56 +03:00
alexey-milovidov
765510d5c3
Merge pull request #10991 from ClickHouse/enable-clang-tidy-for-programs-and-utils
Enable clang-tidy for programs and utils
2020-05-21 15:12:07 +03:00
Alexey Milovidov
2fa264d83e Fix style #10591 2020-05-21 12:46:00 +03:00
alexey-milovidov
b6b62dbdbc
Merge pull request #10591 from excitoon-favorites/ttl_introspection
Added `move_ttl_info` to `system.parts`
2020-05-21 12:43:49 +03:00
BohuTANG
c9d315654c Fix Storages/MergeTree typo 2020-05-21 17:00:44 +08:00
Azat Khuzhin
d93b9a57f6 Forward declaration for Context as much as possible.
Now after changing Context.h 488 modules will be recompiled instead of 582.
2020-05-21 01:53:18 +03:00
Anton Popov
6d8749b100 in-memory parts: faster computation of checksums 2020-05-21 01:16:08 +03:00
kssenii
4dc659b763 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into add-storage-rabbitmq-read-only 2020-05-20 18:17:41 +00:00
alesapin
616902a995 Sorting and primary key (broken) 2020-05-20 21:11:38 +03:00
alesapin
9fb28f5ac0 Add sampling key 2020-05-20 18:16:39 +03:00
Anton Popov
d62fe980ef Merge remote-tracking branch 'upstream/master' into modify-ttl 2020-05-20 16:52:30 +03:00
alesapin
8bc527eecd Compile partition key 2020-05-20 15:16:55 +03:00
Anton Popov
4fb6492b08 in-memory parts: delay merges 2020-05-20 15:02:02 +03:00
alexey-milovidov
da33858d39 Update StorageSystemParts.cpp 2020-05-20 14:07:31 +03:00
Vladimir Chebotarev
02b52df86d Added delete_ttl_info_min and delete_ttl_info_max to system.parts. 2020-05-20 14:07:31 +03:00
Vladimir Chebotarev
6f68979662 Switched system.parts.move_ttl_info to separate arrays. 2020-05-20 14:07:31 +03:00
Vladimir Chebotarev
3ed5d9434a Added move_ttl_info to system.parts. 2020-05-20 14:07:31 +03:00
kssenii
aeffab3fdb Enable SELECT and CREATE MV queries with engine RabbitMQ 2020-05-20 09:40:49 +00:00
Mikhail Filimonov
e8a13842af Better fix, previous fix was wrong (was leaving the cycle by eof condition) 2020-05-20 11:02:02 +02:00
alesapin
dc330ca7cb
Merge pull request #11037 from ClickHouse/alter_rename_and_modify
Fix alter rename with different types of alter
2020-05-20 11:28:13 +03:00
kssenii
3b75f214c5 Register RabbitMQ storage 2020-05-20 09:09:20 +03:00
Mikhail Filimonov
34198336ed Style 2020-05-19 19:02:37 +02:00
Mikhail Filimonov
b50480cbde Fixes the potential missed data during termination of Kafka engine table 2020-05-19 18:55:37 +02:00
alesapin
7d382cc7b5 Return missed command 2020-05-19 19:03:10 +03:00
alesapin
4906dbef41 Remove redundant logs 2020-05-19 13:46:56 +03:00
alesapin
2b37a418f2 More clear logic 2020-05-19 13:44:53 +03:00
alesapin
111c576579 Slightly less garbage 2020-05-19 13:03:05 +03:00
alesapin
70e5553204 At least something works 2020-05-19 12:54:56 +03:00
Anton Popov
58ed04dc24 materialize TTL after its modifying 2020-05-19 04:53:01 +03:00
alesapin
07cb21ccb7
Merge branch 'master' into refactor-reservations 2020-05-18 11:43:48 +03:00
Alexey Milovidov
f54435e7fd Fix clang-tidy 2020-05-18 11:08:55 +03:00
alexey-milovidov
7efd2a825f
Merge pull request #10463 from ClickHouse/parallel-final
Parallel final
2020-05-18 05:20:18 +03:00
alexey-milovidov
073dc2e1e3
Merge pull request #10986 from ClickHouse/try-fix-use-after-free-mergetree
Try to fix use-after-free error in MergeTree
2020-05-18 03:41:53 +03:00
alexey-milovidov
3820075813
Merge pull request #10983 from azat/configure-librdkafka
Configure librdkafka (logging and thread names)
2020-05-17 23:25:18 +03:00
Alexey Milovidov
70c31cbdc4 Fix error 2020-05-17 22:44:31 +03:00
Alexey Milovidov
81a7930c3e Merge branch 'master' of github.com:yandex/ClickHouse into try-fix-use-after-free-mergetree 2020-05-17 22:42:34 +03:00
Alexey Milovidov
20dbd6c3ef Fix error 2020-05-17 22:41:49 +03:00
alexey-milovidov
ca1a9b890d
Revert "Try to fix use-after-free error in MergeTree" 2020-05-17 22:36:27 +03:00
alexey-milovidov
8d3c37992a
Merge pull request #10533 from ClickHouse/alter-key-column-better-exception-message
Better exception message and error code for ALTER of key column #9078
2020-05-17 22:21:14 +03:00
Azat Khuzhin
e3d69eea88 Configure thread names for librdkafka
First of all it is nice to have part of the table name in it.

And secondly, librdkafka uses pthread_setname_np(), but due to
glibc-compatibility it is a noop in clickhouse sources:

  # librdkafka uses rdk: prefix for thread names by default
  $ grep rdk /proc/$(pgrep clickhouse-serv)/task/*/comm
  # just in case:
  $ grep rdk /proc/*/task/*/comm
  $ grep rdk /proc/*/comm

  (gdb) disas pthread_setname_np
  Dump of assembler code for function pthread_setname_np:
     0x000000000c603250 <+0>:     xor    %eax,%eax
     0x000000000c603252 <+2>:     retq
  End of assembler dump.
2020-05-17 20:40:18 +03:00
Azat Khuzhin
dbe781451a Redirect librdkafka logs from stderr to clickhouse logger 2020-05-17 20:27:36 +03:00
alexey-milovidov
7cf3538840
Merge pull request #10270 from ClickHouse/quota-key-in-client
Support quota_key for Native client
2020-05-17 14:09:40 +03:00
alexey-milovidov
7ee35f102d
Merge pull request #10867 from azat/dist-INSERT-load-balancing
Respect prefer_localhost_replica/load_balancing on INSERT into Distributed
2020-05-17 11:11:35 +03:00
Alexey Milovidov
7971f62776 Fix test 2020-05-17 10:47:53 +03:00
alexey-milovidov
421eb6df8f
Merge pull request #10970 from ClickHouse/try-fix-use-after-free-mergetree
Try to fix use-after-free error in MergeTree
2020-05-17 09:09:45 +03:00
alexey-milovidov
a3346e9331
Merge pull request #10912 from ClickHouse/static-mutate
Static mutate
2020-05-17 09:05:35 +03:00
Alexey Milovidov
397859ccb8 Fix error 2020-05-17 08:45:20 +03:00
Alexey Milovidov
9fcfe5fb14 Better exception message and error code for ALTER of key column #9078 2020-05-17 07:46:25 +03:00
alexey-milovidov
0fd0711b51
Merge pull request #10910 from filimonov/kafka_drop_hang_fix
Fix for the hang during deletion of engine=Kafka
2020-05-17 07:13:54 +03:00
Alexey Milovidov
88ee76186b Try to fix use-after-free error in MergeTree 2020-05-17 07:12:33 +03:00
alexey-milovidov
0d76091fd9
Merge pull request #10940 from azat/dist-send-partially-written-read-fix
Avoid sending partially written files by the DistributedBlockOutputStream
2020-05-17 06:53:51 +03:00
Gleb Novikov
1a25ac6e1f Merge branch 'master' into refactor-reservations 2020-05-16 23:34:45 +03:00
Gleb Novikov
45b84d491e small fixes of review remarks 2020-05-16 23:31:17 +03:00