Commit Graph

755 Commits

Author SHA1 Message Date
Alexey Milovidov
6ff671b092 Improvement 2020-06-13 04:35:49 +03:00
Alexey Milovidov
ab00e34305 Miscellaneous 2020-06-13 04:35:49 +03:00
Alexey Milovidov
85c0706901 Step 2: allow multiple leaders 2020-06-13 04:35:49 +03:00
Alexey Milovidov
b8b55a5b99 More LeaderElection to Storage/MergeTree 2020-06-13 04:35:49 +03:00
Alexey Milovidov
52ac009754 Update versions on merge and mutation 2020-06-13 04:34:05 +03:00
Alexey Milovidov
833c8178ae Whitespaces 2020-06-13 04:34:05 +03:00
Alexey Milovidov
6869c122a8 Step 1 towards removing leader election: check and modify version of the "log" node 2020-06-13 04:34:05 +03:00
alexey-milovidov
54b77ff967
Merge pull request #11633 from ClickHouse/avoid-errors-using-zk-api
Avoid errors due to implicit int<->bool conversions when using ZK API
2020-06-13 04:32:32 +03:00
alexey-milovidov
fc8da2abd3
Merge pull request #11613 from azat/system.zookeeper-path-prepend-slash
Prepend slash for path while querying system.zookeeper
2020-06-12 18:19:10 +03:00
Alexey Milovidov
72257061d5 Avoid errors due to implicit int<->bool conversions when using ZK API 2020-06-12 18:09:12 +03:00
Nikolai Kochetov
83155e139c Try fix tests. 2020-06-12 17:59:14 +03:00
alesapin
0edf5ff7a2 Fix race condition 2020-06-12 17:32:47 +03:00
alesapin
465c4b65b7 Slightly better interfaces and comments 2020-06-12 12:37:52 +03:00
alesapin
4eb8a1cfef Fix gcc build 2020-06-12 00:03:25 +03:00
Alexey Milovidov
540f8532cb Fix error 2020-06-11 22:59:15 +03:00
Azat Khuzhin
176ebb26e0 Prepend slash to path for system.zookeeper queries
path should starts with '/', otherwise ZBADARGUMENTS will be thrown in
ZooKeeper::sendThread and the session will fail.
2020-06-11 20:58:53 +03:00
Nikolai Kochetov
7b8c1657a5 Fix nullable prewhere column type. 2020-06-11 19:34:36 +03:00
alesapin
a39d010f94 Merge branch 'master' into consistent_metadata4 2020-06-11 18:47:41 +03:00
alexey-milovidov
5f073c3ef4
Update StorageReplicatedMergeTree.cpp 2020-06-11 17:11:13 +03:00
Nikolai Kochetov
ed0826efa7 Remove extra columns from sample in StorageBuffer. 2020-06-11 15:18:45 +03:00
Alexey Milovidov
790bc4e8e0 Better code 2020-06-11 06:35:59 +03:00
Alexey Milovidov
b72a095e99 Fix race conditions in CREATE/DROP of different replicas of ReplicatedMergeTree 2020-06-11 06:24:52 +03:00
alesapin
dd3cf0fe0a Don't miss columns TTLs update 2020-06-10 21:02:45 +03:00
alesapin
24db834046 Style 2020-06-10 14:27:16 +03:00
alesapin
8be957ecb5 Better checks around metadata 2020-06-10 14:16:31 +03:00
alesapin
ec933d9d03 Better naming 2020-06-10 12:09:51 +03:00
alesapin
75977bd939 Gcc fixed 2020-06-10 00:22:01 +03:00
alesapin
4f398b5dea Merge remote-tracking branch 'origin/master' into consistent_metadata4 2020-06-09 23:53:38 +03:00
alesapin
4c77ba664a Better assignment 2020-06-09 20:42:04 +03:00
alesapin
d2fcf5aea5 Fixes for gcc 2020-06-09 20:28:29 +03:00
alesapin
936cc9d573 Update key 2020-06-09 20:22:03 +03:00
alesapin
4c2e0561f5 Fix primary key update 2020-06-09 20:21:38 +03:00
alesapin
6f673b0981 Fix comments bug 2020-06-09 19:33:28 +03:00
Nikolai Kochetov
5cc2f318c9
Merge pull request #11495 from vzakaznikov/live_view_support_for_prewhere
Adding support for PREWHERE in live view tables.
2020-06-09 19:09:17 +03:00
alesapin
f7a397242f Merge branch 'master' into consistent_metadata4 2020-06-09 18:59:59 +03:00
Alexander Kuzmenkov
dd41f6a574
Merge pull request #11527 from azat/dist-insert-local-fix
Fix async INSERT into Distributed for prefer_localhost_replica=0 and w/o internal_replication
2020-06-09 15:21:05 +03:00
Ivan
47ad338cb2
Refactor CMake build files (#11390)
* Get rid of lib_name.cmake
* Refactor Boost and HyperScan libraries
* Refactor lz4
* Fix build with xxHash
2020-06-09 13:54:49 +03:00
alesapin
223f45685f Review fixes 2020-06-09 05:03:22 +03:00
alesapin
de59629b38 Fix benign race condition during shutdown 2020-06-09 04:48:30 +03:00
Azat Khuzhin
d2383f0f5d Fix async INSERT into Distributed for prefer_localhost_replica=0 and w/o internal_replication 2020-06-08 21:58:56 +03:00
alesapin
0d375e4522 Fix constraints 2020-06-08 21:49:54 +03:00
alesapin
a5a9048109 At least fixed gcc-9 build 2020-06-08 21:23:26 +03:00
alesapin
7b63a88120 Remove redundant error code 2020-06-08 18:19:41 +03:00
alesapin
d625162fce Better comment 2020-06-08 17:23:32 +03:00
alesapin
37a1d128c9 Better 2020-06-08 17:22:11 +03:00
alesapin
a30b72a128 Less debug 2020-06-08 17:21:12 +03:00
alesapin
2354939994 Remove log debug 2020-06-08 17:19:20 +03:00
alesapin
638cd50ce4 Fix several bugs 2020-06-08 17:18:38 +03:00
alesapin
2226f79f16 Fix some bugs 2020-06-08 15:57:25 +03:00
alesapin
ac67d0485c Merge branch 'master' into consistent_metadata4 2020-06-08 15:06:51 +03:00
Vitaliy Zakaznikov
5fe67c4292 * Removing supportsPrewhere() from StorageLiveView.h as it is not valid.
* Updating test to check using PREWHERE in query against live view table.
* Updating test to check using PREWHERE in the stored query against the table that does
  not support PREWHWERE.
2020-06-08 13:12:40 +02:00
Vitaliy Zakaznikov
75c97ca82a Adding support for PREWHERE in live view tables. 2020-06-08 00:19:23 +02:00
Alexey Milovidov
f2cfb7d132 Whitespaces 2020-06-08 00:05:36 +03:00
alexey-milovidov
e3c528e661
Merge pull request #11174 from excitoon-favorites/s3tests
Added tests for improved S3 URL parsing
2020-06-07 02:44:40 +03:00
Ivan
d2178cbacd
Merge pull request #11388 from filimonov/kafka_better_settings
Better settings for Kafka
2020-06-07 01:26:21 +03:00
alexey-milovidov
5ff63d9ad5
Merge pull request #11486 from ClickHouse/speed-up-build
Speed up build by removing old example programs
2020-06-07 00:22:03 +03:00
Ivan
657ce426d3
Fix build
Checked locally - no PR
2020-06-06 22:27:48 +03:00
BayoNet
e9b7919e12
DOCS-271: Updated the MergeTree() ORDER BY description (#11433)
* CLICKHOUSEDOCS-271: Updated the MergeTree() ORDER BY description.

* CLICKHOUSEDOCS-271: Fixes grammar.

* CLICKHOUSEDOCS-271: Updated by comments.

Co-authored-by: Sergei Shtykov <bayonet@yandex-team.ru>
2020-06-06 20:44:48 +03:00
Ivan
058821fa02
Merge branch 'master' into kafka_better_settings 2020-06-06 19:50:38 +03:00
Alexey Milovidov
d3d02a67f4 Speed up build by removing old example programs 2020-06-06 19:19:16 +03:00
alexey-milovidov
3e84b17cf8
Merge pull request #11394 from azat/system.distribution_queue
Add system.distribution_queue table
2020-06-06 18:59:59 +03:00
alexey-milovidov
82e849e6a1
Update StorageDistributed.cpp 2020-06-06 18:57:52 +03:00
Anton Popov
5c42408add
Merge pull request #9113 from dimarub2000/group_by_in_order_optimization
[WIP] Optimization of GROUP BY with respect to table sorting key.
2020-06-06 14:25:59 +03:00
Artem Zuikov
c58f9bd43d refactoring: remove TableWithColumnNames 2020-06-06 00:17:00 +03:00
alesapin
abaf47f0cd Make metadata single structure 2020-06-05 20:29:40 +03:00
alesapin
2b23d1aa33 Fix reference 2020-06-05 15:13:24 +03:00
alesapin
5bc7f67e61 Better metadata for select query and renames 2020-06-05 14:54:54 +03:00
Alexander Kuzmenkov
eaaf005a03
Merge pull request #11410 from ClickHouse/aku/logical-error-synonyms
Remove assorted synonyms of LOGICAL_ERROR
2020-06-05 13:10:40 +03:00
Azat Khuzhin
ff85125326 Fix readability-qualified-auto 2020-06-04 20:23:46 +03:00
Alexander Kuzmenkov
029005be7c clang-tidy 2020-06-04 17:26:16 +03:00
alesapin
22235c1951 Tiny fixes for storage s3 2020-06-04 16:48:20 +03:00
alesapin
bba558d68e Merge branch 'custom-auth' of https://github.com/GrigoryPervakov/ClickHouse into merging_s3_auth 2020-06-04 16:19:04 +03:00
Vladimir Chebotarev
d62e85f875 Improved logging of globs in S3. 2020-06-04 16:12:08 +03:00
Nikita Mikhaylov
4e4218e10d
Merge pull request #11385 from ClickHouse/akz/bg-processing-pool-update1
Extended BackgroundProcessingPool::startTask() to control task execution concurrency
2020-06-04 16:19:29 +04:00
Mikhail Filimonov
26d93fdfba Fix the test, adjust default for queued.min.messages 2020-06-04 14:14:24 +02:00
Alexander Kuzmenkov
30d7d41ebc
Merge pull request #11381 from ClickHouse/aku/fuzz-related
Fuzzing-related changes.
2020-06-04 15:06:11 +03:00
Alexander Kuzmenkov
0943ef3e4b Do not set global context twice in unit tests. 2020-06-04 15:04:05 +03:00
Anton Popov
4a3d3c6e54
Merge pull request #11419 from CurtizJ/polymorphic-parts-2
Return lost comments and default values
2020-06-04 13:32:38 +03:00
Azat Khuzhin
86c5465bf8 Rewrite StorageSystemDistributionQueue interfaces 2020-06-04 03:04:32 +03:00
Azat Khuzhin
aa13bd9530 Fix possible SIGSEGV in StorageSystemDistributionQueue
Refs: #11348
2020-06-04 03:04:28 +03:00
Azat Khuzhin
f0050adc51 Make system.distribution_queue metrics non racy 2020-06-04 02:36:16 +03:00
Azat Khuzhin
09c3ca9c6c Add last_exception into system.distribution_queue 2020-06-04 02:36:16 +03:00
Azat Khuzhin
389f78ceee Add system.distribution_queue
system.distribution_queue contains the following columns:
- database
- table
- data_path
- is_blocked
- error_count
- data_files
- data_compressed_bytes
2020-06-04 02:36:16 +03:00
Azat Khuzhin
60d10f1bac Fix typo in StorageDistributed 2020-06-04 02:36:16 +03:00
Alexey Milovidov
484d269c22 Fix typo 2020-06-04 01:15:13 +03:00
Alexey Milovidov
a0fd9b021a Fix MSan failure in MergeTree 2020-06-04 01:11:06 +03:00
Anton Popov
1980d58ba6 add lost comments and default values 2020-06-04 00:30:45 +03:00
Mikhail Filimonov
eef914cbdf Remove debug print 2020-06-03 20:34:22 +02:00
Mikhail Filimonov
94261b9786 Fix issues detected by clang tidy, fix issue with flush timeout 2020-06-03 19:02:24 +02:00
Mikhail Filimonov
95f4c46aa4 Better control of context copies, some cosmetic changes, client.software.name and client.software.version now initialized 2020-06-03 19:01:45 +02:00
Mikhail Filimonov
4956ade67d Support for all format settings in Kafka, expose some setting on table level, better defaults. 2020-06-03 19:01:41 +02:00
Alexander Kuzmenkov
1ab3201454 Merge remote-tracking branch 'origin/master' into HEAD 2020-06-03 16:36:22 +03:00
Alexander Kuzmenkov
c7d9094a7a
Merge pull request #11259 from ClickHouse/consistent_metadata3
More consistent metadata for secondary indices
2020-06-03 12:23:21 +03:00
Alexander Kuzmenkov
f7bb6d57a4 build 2020-06-03 11:50:27 +03:00
Alexander Kazakov
f999021ee1 Submit to clang10's demands of syntactic purity 2020-06-03 11:12:56 +03:00
alexey-milovidov
66eb56631c
Merge pull request #11355 from ClickHouse/fix_some_integration_tests
Fix several flaky integration tests
2020-06-03 02:35:33 +03:00
Alexander Kazakov
0cd4e6e5ba Trying to fix clang10 build 2020-06-03 00:52:29 +03:00
Alexander Kazakov
9d025337f5 Fixed minor inconsistency with master branch 2020-06-02 21:56:36 +03:00
alesapin
6253e9b97b Revert disabled tests 2020-06-02 21:41:29 +03:00
Pervakov Grigorii
bc9381406f Add default credentials and custom headers for s3 table functions. 2020-06-02 20:48:49 +03:00
Alexander Kuzmenkov
a859afa174 macos build fix 2020-06-02 19:57:19 +03:00
Alexander Kazakov
8986983392 insignificant typo 2020-06-02 18:22:05 +03:00
Alexander Kazakov
9fbd72ab34 Renamed method 2020-06-02 18:18:36 +03:00
Alexander Kazakov
013dc2d7c2 Updated startTask() method + reworked some of the worker loop 2020-06-02 17:37:10 +03:00
alexey-milovidov
ff7262968a
Merge pull request #11354 from ClickHouse/get_rid_of_lock_during_send
Remove structure lock on parts send
2020-06-02 17:05:24 +03:00
Alexander Kuzmenkov
0a5cc96b67 Fuzzing-related changes.
* More LOGICAL_ERROR
* Proper cloning of some Asts
* Field::safeGet for user-supplied values
2020-06-02 16:56:10 +03:00
alexey-milovidov
d47d0d7108
Merge pull request #11346 from ClickHouse/optimize-final-should-force-merge-2
OPTIMIZE FINAL should force merge even if concurrent merges are performed (experimental, try 2)
2020-06-02 16:40:20 +03:00
alexey-milovidov
9b07989954
Merge pull request #11348 from ClickHouse/lazy-database-segfault-2
Fix potential segfault when using Lazy database.
2020-06-02 16:37:23 +03:00
Nikolai Kochetov
9a283de670
Merge pull request #11339 from ClickHouse/remove-experimantal-use-processors-flag-5
Remove some streams.
2020-06-02 14:46:02 +03:00
alesapin
42b6370b67 Remove structure lock on parts send 2020-06-02 11:10:21 +03:00
alesapin
e5eb6071a4 Merge with master 2020-06-02 10:46:58 +03:00
alesapin
1ab3047201 Merge branch 'master' into consistent_metadata3 2020-06-02 10:37:12 +03:00
Ilya Yatsishin
a101b17df7
Merge pull request #11335 from ClickHouse/stop_merges_before_alter
Stop merges before taking exclusive lock :(
2020-06-02 09:34:05 +03:00
Alexey Milovidov
a6bf4b5d70 Fix error with Lazy database 2020-06-02 05:06:16 +03:00
Alexey Milovidov
78d966f4e5 Better code 2020-06-02 03:41:52 +03:00
alesapin
ad6447398f Fix indices calculation during merge 2020-06-01 22:58:30 +03:00
Artem Zuikov
b7958d4f06 make clang-10 happy 2020-06-01 22:38:51 +03:00
Artem Zuikov
bfa59adbfe fix select from StorageJoin 2020-06-01 20:27:22 +03:00
Nikolai Kochetov
53d12f5ab8 Try fix build. 2020-06-01 20:06:21 +03:00
Nikolai Kochetov
d25326e75c Try fix build. 2020-06-01 20:03:57 +03:00
Nikolai Kochetov
325393d340 Try fix build. 2020-06-01 19:58:36 +03:00
Nikolai Kochetov
3a0acb00c1 Remove some streams. 2020-06-01 19:31:06 +03:00
alesapin
ce6db6927b
boop CI 2020-06-01 19:22:33 +03:00
alesapin
f287a28fc5 Stop merges before taking exclusive lock :( 2020-06-01 18:16:10 +03:00
alesapin
e0e77194f4 Remove redundant parts from MergeTree 2020-06-01 15:39:20 +03:00
alesapin
b49be4c7f2 Better names 2020-06-01 15:11:23 +03:00
alesapin
254d55f2ab Better method name 2020-06-01 15:02:36 +03:00
alesapin
663e92b1c5 Rename to methods 2020-06-01 14:29:11 +03:00
alesapin
d9a817f578 Less virtual methods in IStorage 2020-06-01 14:17:18 +03:00
alesapin
69a9c856a6 Better variable name 2020-06-01 13:54:31 +03:00
alesapin
72c17a8e3a Slightly more comments 2020-06-01 13:48:52 +03:00
alesapin
3847ea892d Merge branch 'master' into consistent_metadata3 2020-06-01 13:17:59 +03:00
alesapin
c22e8f6b9b
Merge pull request #11283 from filimonov/kafka_virtual_column_headers
Virtual columns for Kafka headers
2020-06-01 12:06:00 +03:00
alesapin
364ebe9246
Merge pull request #11260 from filimonov/kafka_virtual_column_timestamp_ms
Add _timestamp_ms virtual columns to Kafka engine
2020-06-01 12:04:17 +03:00
alexey-milovidov
256f84ddc2
Merge pull request #11230 from Jokser/s3-poco-http-client
S3 Poco Http Client
2020-06-01 01:20:21 +03:00
alexey-milovidov
9984e989c1
Merge pull request #11317 from ClickHouse/logger-fmt-2
Preparation for structured logging
2020-06-01 00:11:29 +03:00
Alexey Milovidov
07c14e12dd Fix build after merge of #11162 with master 2020-05-31 19:53:30 +03:00
alexey-milovidov
8accde79b5
Merge pull request #11162 from azat/data-skip-index-merging-params-fix
[RFC] Fix data skipping indexes for columns with additional actions during merge
2020-05-31 19:10:53 +03:00
alexey-milovidov
9416e82b75
Update MergeTreeDataMergerMutator.cpp 2020-05-31 19:06:59 +03:00
alexey-milovidov
9a362eb02e
Merge pull request #11252 from filimonov/kafka-clientid
Kafka clientid (finishing #11073)
2020-05-31 18:37:30 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Alexey Milovidov
5aff138956 Preparation for structured logging 2020-05-31 00:35:52 +03:00
Dmitry
4b0d32f026 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-31 00:21:02 +03:00
alexey-milovidov
3eea042d16
Merge pull request #11243 from ClickHouse/remove-experimental-use-processors-flag-4
Remove some code.
2020-05-30 21:05:14 +03:00
alexey-milovidov
1dbaf80a5d
Merge pull request #11303 from ClickHouse/fix-index-analysis-empty-array
Fix issue #11286; add a test
2020-05-30 17:28:00 +03:00
Alexey Milovidov
b78c48310b Whitespaces 2020-05-30 01:37:44 +03:00
Alexey Milovidov
8c8821475c Fix issue #11286; add a test 2020-05-30 00:32:35 +03:00
Ilya Yatsishin
393db8ff6c
Merge pull request #11261 from filimonov/kafka_log_with_prefix_and_facility
Add prefix & facility to librdkafka logs
2020-05-29 23:02:42 +03:00
tavplubix
a4e40fb5f2
Merge pull request #11269 from ClickHouse/add_context_to_idatabase_methods
Add context to IDatabase methods
2020-05-29 21:15:35 +03:00
Anton Popov
e6a9a33010
Merge pull request #11183 from ClickHouse/consistent_metadata2
Refactor TTLs metadata (move them to IStorage).
2020-05-29 20:00:11 +03:00