Peng Jian
091f7065cd
fix build
2020-11-19 15:44:47 +08:00
Peng Jian
2df0c3e0b0
Add helper function for extracting zookeeper name and path.
2020-11-19 15:44:47 +08:00
Peng Jian
a0683ce460
Support mulitple ZooKeeper clusters
2020-11-19 15:44:47 +08:00
Pavel Kruglov
547ec19fb3
Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert
2020-11-18 12:01:59 +03:00
Mikhail Filimonov
234c671e52
After CR fixes
2020-11-18 08:45:44 +01:00
Mikhail Filimonov
97fef77ed1
execute_merges_on_single_replica
2020-11-18 08:45:33 +01:00
Pavel Kruglov
6a57c0a8cf
Move merge in MergeTreeDataWriter
2020-11-13 10:55:56 +03:00
alesapin
8d3858fc22
Merge pull request #16895 from CurtizJ/remove-empty-parts
...
Remove empty parts after they were pruned by TTL [2]
2020-11-13 10:00:18 +03:00
alesapin
bbd85c3b2d
Merge pull request #16390 from ClickHouse/add_separate_pool_for_fetches
...
Add separate pool for fetches
2020-11-12 22:42:28 +03:00
Anton Popov
4cecd89e84
fix dropping of empty parts
2020-11-12 20:37:54 +03:00
alesapin
2cac5e5d97
Merge pull request #16875 from CurtizJ/refactor-alter-partition
...
Avoid code duplication in alterPartition
2020-11-12 13:19:59 +03:00
alesapin
a8d466a645
Merge branch 'master' into add_separate_pool_for_fetches
2020-11-12 12:51:02 +03:00
tavplubix
058aa8f85e
Merge pull request #16824 from ClickHouse/replace_stringstreams_with_buffers
...
Replace std::*stringstreams with DB::*Buffers
2020-11-12 01:11:44 +03:00
Anton Popov
6f0dc08ba7
avoid code duplication in alterPartition
2020-11-11 16:34:07 +03:00
Alexander Tokmakov
7da5e92754
fix
2020-11-11 13:41:12 +03:00
Pavel Kruglov
287d71b363
Fix build
2020-11-11 13:34:32 +03:00
Pavel Kruglov
dd06ff4f94
Minor change
2020-11-11 00:02:11 +03:00
Pavel Kruglov
c648f62629
Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize_final_optimization
2020-11-10 22:58:21 +03:00
Alexander Tokmakov
b94cc5c4e5
remove more stringstreams
2020-11-10 21:22:26 +03:00
Pavel Kruglov
9120189d8a
Add SelectPartsDecision enum class
2020-11-10 17:42:56 +03:00
Nikolai Kochetov
195c941c4e
Merge branch 'master' into storage-read-query-plan
2020-11-10 15:02:22 +03:00
alesapin
2e357516a6
Merge branch 'master' into add_separate_pool_for_fetches
2020-11-10 13:53:08 +03:00
Nikolai Kochetov
046830967c
Try fix tests.
2020-11-10 13:26:26 +03:00
Vladimir Chebotarev
059357d51e
ALTER UPDATE/DELETE ... IN PARTITION with partition pruning in ReplicatedMergeTree (#13403 )
...
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2020-11-10 13:23:46 +03:00
alesapin
e347f4b72f
Add some comments
2020-11-09 12:14:20 +03:00
alesapin
72046313d4
Merge branch 'master' into nvartolomei-drop-part
2020-11-09 10:02:02 +03:00
alesapin
4e846d2197
Merge branch 'master' into add_separate_pool_for_fetches
2020-11-09 10:00:21 +03:00
alexey-milovidov
0e6ae4aff7
Merge pull request #16253 from amosbird/pf
...
Prune partition in verbatim way.
2020-11-08 18:58:02 +03:00
alexey-milovidov
f4ba5f1f9a
Merge pull request #16772 from ClickHouse/fix-stringstream
...
Fix "server failed to start" error
2020-11-08 14:27:08 +03:00
Alexey Milovidov
5314185e25
Merge branch 'master' into azat-optimize_skip_unused_shards-optimization
2020-11-08 00:17:59 +03:00
Alexey Milovidov
fd84d16387
Fix "server failed to start" error
2020-11-07 03:14:53 +03:00
Nikolai Kochetov
c10f733587
Merge branch 'master' into storage-read-query-plan
2020-11-06 15:43:46 +03:00
Amos Bird
2b0085c106
Pruning is different from counting
2020-11-06 19:58:03 +08:00
Amos Bird
30bf5e6d26
Prune partition in verbatim way.
2020-11-06 09:56:13 +08:00
alexey-milovidov
ee3e289ec2
Merge pull request #15983 from ClickHouse/no_background_pool_no_more
...
Replace BackgroundProcessingPool with SchedulePool task and ThreadPool.
2020-11-05 21:38:22 +03:00
Anton Ivashkin
1742fb3256
Merge master
2020-11-03 12:27:16 +03:00
alesapin
da8ac948bf
Disallow to drop parts with parallel quorum
2020-11-03 12:24:10 +03:00
alesapin
294489f6ad
Merge branch 'master' into nvartolomei-drop-part
2020-11-03 10:19:03 +03:00
alesapin
dac6705995
DROP PART and clear blocks in a single zookeeper op
2020-11-02 20:30:59 +03:00
alesapin
67d22b3e57
Merge branch 'master' into nvartolomei-drop-part
2020-11-02 19:09:09 +03:00
alesapin
7f8560e0c0
Merge pull request #16538 from amosbird/fp2
...
Allow fetching duplicate parts to detach
2020-11-02 16:10:42 +03:00
Amos Bird
18c5abc44e
Allow fetching duplicate parts to detach
2020-10-30 10:55:50 +08:00
alesapin
40fc512e79
Merge branch 'master' into no_background_pool_no_more
2020-10-29 12:53:34 +03:00
alesapin
0931658c2f
Merge branch 'master' into add_separate_pool_for_fetches
2020-10-28 14:20:56 +03:00
tavplubix
4a63f1271b
Merge pull request #16244 from ClickHouse/fix_race_data_parts_exchange_endpoint
...
Fix race on StorageReplicatedMergeTree::data_parts_exchange_endpoint
2020-10-28 13:25:55 +03:00
alesapin
47ebc9f74e
Add separate pool for fetches
2020-10-26 14:02:47 +03:00
hchen9
824c04f873
Fix conflict code
2020-10-25 18:12:17 -07:00
hchen9
feaab260d8
Fix conflict
2020-10-23 12:54:33 -07:00
alesapin
39e47c5338
Less strange lambdas
2020-10-23 11:54:00 +03:00
alesapin
7a63b373f9
Merge branch 'blind_fix_of_read_in_order' into no_background_pool_no_more
2020-10-22 15:47:11 +03:00
Nikolai Kochetov
7fa045cff8
Merge branch 'master' into storage-read-query-plan
2020-10-22 13:31:10 +03:00
Anton Ivashkin
f833501c77
Merge branch 'master' into s3_zero_copy_replication
2020-10-22 11:17:21 +03:00
alexey-milovidov
adeba6bdd8
Merge pull request #15074 from amosbird/btc
...
Extend trivial count optimization.
2020-10-22 02:50:57 +03:00
Alexander Tokmakov
ac42199ae5
fix race on StorageReplicatedMergeTree::data_parts_exchange_endpoint
2020-10-21 22:25:22 +03:00
alesapin
ce8ee01d75
Fix background moves start
2020-10-20 14:27:50 +03:00
alesapin
d126aa8a09
Merge branch 'master' into no_background_pool_no_more
2020-10-20 12:54:42 +03:00
Alexander Kuzmenkov
defcd29b31
Merge pull request #16036 from ClickHouse/stress_test_improvemets
...
Minor improvements of stress test
2020-10-19 16:12:30 +03:00
alesapin
e09713ece1
Merge branch 'master' into no_background_pool_no_more
2020-10-19 10:58:31 +03:00
Alexander Tokmakov
026896835c
fixes
2020-10-16 15:20:30 +03:00
alesapin
7edbe4d276
Add some comments
2020-10-16 13:12:31 +03:00
Anton Ivashkin
14a78f87b0
Zero copy replication over S3: fetch instead of merge
2020-10-15 19:14:47 +03:00
alesapin
4c1f5cab13
Sleep less in background task
2020-10-15 19:10:22 +03:00
alesapin
2002289003
Fix bug in alter primary key for replicated versioned collapsing merge tree
2020-10-15 16:02:39 +03:00
Pavel Kruglov
25c39689c3
Check expired ttl
2020-10-15 15:15:02 +03:00
alesapin
583d7042e9
Get rid of some deadlocks
2020-10-15 11:29:18 +03:00
alesapin
c5419083e3
Better method name
2020-10-15 10:43:50 +03:00
alesapin
483893cdd4
Slightly refactored code
2020-10-14 17:56:42 +03:00
alesapin
0b14a31ba9
Get rid of background processing pool
2020-10-14 15:44:10 +03:00
alesapin
4014e0f08d
Something very similar to working code
2020-10-14 15:32:35 +03:00
alesapin
efd3126b5d
Moving pool
2020-10-14 10:22:48 +03:00
Pavel Kruglov
1ccf10505a
add tests and comments
2020-10-13 21:26:40 +03:00
Pavel Kruglov
6d9b310f61
Add optimize_skip_merged_partitions
2020-10-13 21:26:35 +03:00
alesapin
fd35368c59
First simple implementation
2020-10-13 17:25:42 +03:00
alesapin
cb7db6d232
Split select and process merges
...
(cherry picked from commit 1889eb0ff1
2020-10-13 15:04:16 +03:00
Nikolai Kochetov
7e58f99f64
Merge branch 'master' into storage-read-query-plan
2020-10-12 13:12:39 +03:00
Alexey Milovidov
5b482f4191
Cleanups
2020-10-10 19:31:10 +03:00
alexey-milovidov
0fd007ad9e
Merge pull request #15601 from alexelex/alexelex-master
...
Improvement of Quorum Inserts in ClickHouse
2020-10-09 22:43:53 +03:00
Anton Ivashkin
9272ed06b4
Move Zookeeper lock for S3 shared part in IMergeTreeDataPart
2020-10-09 17:24:10 +03:00
Anton Ivashkin
766dbfd2be
Merge master
2020-10-09 15:00:14 +03:00
Alexandra Latysheva
0594a77b57
fix thread restart for parallel quorum inserts
2020-10-09 11:20:20 +00:00
tavplubix
7b5f2b2bb0
Merge pull request #15701 from ClickHouse/better_clone_replica
...
Improvements in lost replica recovery
2020-10-08 23:19:23 +03:00
Anton Ivashkin
acf86568a7
S3 zero copy replication proof of concept
2020-10-08 18:45:10 +03:00
Alexandra Latysheva
6877f4aa9c
for parallel and non-parallel quorum insertion at the same time
2020-10-08 15:35:41 +00:00
tavplubix
6065a5b65c
Update StorageReplicatedMergeTree.cpp
2020-10-08 16:37:32 +03:00
Pervakov Grigorii
3a2068f19d
Use tmp disk for vertical merge files
2020-10-08 13:49:41 +03:00
Amos Bird
867216103f
Extend trivial count optimization.
2020-10-08 18:08:17 +08:00
hchen9
ed2ce4a6fc
Resolve conflict in StorageMemory
2020-10-07 17:19:59 -07:00
Alexander Tokmakov
ffb19fa4bc
fix
2020-10-07 22:47:31 +03:00
Latysheva Alexandra
f43ad338ed
Merge pull request #12 from alexelex/quorum_inserts_2
...
style check
2020-10-07 18:56:14 +06:00
Alexandra Latysheva
8f8872dc42
style check
2020-10-07 12:24:46 +00:00
Latysheva Alexandra
6c26f77380
Merge pull request #11 from alexelex/quorum_inserts_2
...
changing the condition for updateQuorum, specialized it and reduced t…
2020-10-07 17:49:20 +06:00
Alexandra Latysheva
a43cac2c1a
changing the condition for updateQuorum, specialized it and reduced the number of calls to the zk
2020-10-07 11:28:48 +00:00
Nikolai Kochetov
7e02152b50
Merge branch 'master' into storage-read-query-plan
2020-10-07 14:27:21 +03:00
Nikolai Kochetov
c5cb05f5f3
Try fix tests.
2020-10-07 14:26:29 +03:00
tavplubix
041015545c
Merge pull request #15537 from ClickHouse/mutation_hangs_after_replace_partition
...
Fix mutation may hang after REPLACE/DROP PARTITION
2020-10-07 14:25:29 +03:00
Alexander Tokmakov
47e9fe9ff1
fix
2020-10-07 03:05:48 +03:00
Alexander Tokmakov
ea16370024
improvements in lost replica recovery
2020-10-07 02:29:33 +03:00
Latysheva Alexandra
991e51a80e
Merge pull request #10 from alexelex/quorum_inserts_2
...
style check fix
2020-10-07 04:38:30 +06:00
Alexandra Latysheva
9077c92862
style check fix
2020-10-06 22:36:42 +00:00
Latysheva Alexandra
85bcc07c0b
Merge pull request #9 from alexelex/quorum_inserts_2
...
Quorum inserts [version 3]
2020-10-07 04:04:41 +06:00
Alexandra Latysheva
644091459f
style fixes and remove small changes
2020-10-06 22:01:30 +00:00
Alexandra Latysheva
f549ecf9d1
insert_quorum_parallel: status nodes in /quorum/parallel and allow merges
2020-10-06 21:49:48 +00:00
Latysheva Alexandra
d495a2fb2d
Merge pull request #7 from alexelex/quorum_inserts_2
...
build checker fix (style) and change comments
2020-10-06 02:33:29 +06:00
Alexandra Latysheva
41e0148d72
build checker fix (style) and change comments
2020-10-05 20:30:50 +00:00
Alexander Tokmakov
418766e92b
fix another bug in mutations
2020-10-05 22:16:28 +03:00
alesapin
8ec58c17f3
More consistent metadata usage
2020-10-05 19:41:46 +03:00
Alexander Tokmakov
30a18f7567
make it better
2020-10-05 16:52:03 +03:00
Latysheva Alexandra
7e7cedd73f
Merge pull request #4 from alexelex/quorum_inserts_2
...
style check fix
2020-10-05 16:43:14 +06:00
Alexandra Latysheva
0662432c5c
style check
2020-10-05 10:41:49 +00:00
Latysheva Alexandra
a63084adaf
Merge pull request #3 from alexelex/quorum_inserts_2
...
Quorum inserts 2
2020-10-05 03:28:58 +06:00
Alexandra Latysheva
d54b6c9542
add question in comments
2020-10-04 20:32:08 +00:00
Alexandra Latysheva
ff8cbe6745
remove debug things
2020-10-04 20:19:05 +00:00
Alexandra Latysheva
8263e62298
working copy (with some debug info)
2020-10-04 19:55:39 +00:00
hchen9
cbf3af401d
Fix comments
2020-10-04 02:42:03 -07:00
Azat Khuzhin
b838214a35
Pass non-const SelectQueryInfo (and drop mutable qualifiers)
2020-10-02 22:42:35 +03:00
hchen9
b3949db00f
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into trivial_count_seq_consistency
2020-10-01 21:15:26 -07:00
Nikolai Kochetov
ec64def384
Use QueryPlan while reading from MergeTree.
2020-10-01 20:34:22 +03:00
Nicolae Vartolomei
7197a0c803
Possibly fix the issue with quorum
2020-10-01 11:38:50 +01:00
hchen9
b4494578d7
Support select_sequential_consistency in StorageReplicatedMergeTree.totalRows and totalBytes
2020-09-30 16:50:58 -07:00
Alexandra Latysheva
e89a56969f
first part
2020-09-30 23:16:27 +00:00
Alexander Tokmakov
34addcf61f
add review suggestions
2020-09-26 22:18:28 +03:00
alesapin
5ffee8808a
Alter remove column properties and TTLs ( #14742 )
2020-09-20 16:27:33 +03:00
alesapin
e38b537017
Merge pull request #11684 from ClickHouse/manual-write-duplicate-parts-to-replicas
...
Don't ignore duplicate parts written to replicas
2020-09-18 16:50:00 +03:00
Alexey Milovidov
a5560cd62d
Better drop replicated tables
2020-09-17 14:27:17 +03:00
alesapin
40b2f203b6
Merge branch 'master' into manual-write-duplicate-parts-to-replicas
2020-09-17 13:21:00 +03:00
Alexander Tokmakov
da6beb1a91
Merge branch 'master' into enable_some_tests_with_database_atomic
2020-09-14 23:00:14 +03:00
bharatnc
3b3b7bd5e7
minor changes to comment
2020-09-10 09:09:48 -07:00
Alexander Tokmakov
20b3823770
do not remove directory on attach
2020-09-10 13:21:13 +03:00
Alexander Tokmakov
dee1fefeb4
add more tests
2020-09-09 20:39:49 +03:00
bharatnc
fe6d6f16ea
Merge remote-tracking branch 'upstream/master' into ncb/zk-conn-fail-do-cleanup
2020-09-09 09:25:34 -07:00
Alexander Tokmakov
b68782d285
enable more tests with Atomic database
2020-09-09 16:32:50 +03:00
bharatnc
f37814b367
Merge remote-tracking branch 'upstream/master' into ncb/zk-conn-fail-do-cleanup
2020-09-08 23:45:20 -07:00
Nicolae Vartolomei
97d0b5ab23
Remove unused query argument
2020-09-08 10:56:10 +01:00
bharatnc
0c00b992d5
StorageReplicatedMergeTree - cleanup data dir after Zk exceptions
...
It's possible for `getZooKeeper()` to timeout if zookeeper host(s) can't
be reached. In such cases `Poco::Exception` is thrown after a connection
timeout - refer to `src/Common/ZooKeeper/ZooKeeperImpl.cpp:866` for more info.
Side effect of this is that the CreateQuery gets interrupted and it exits.
But the data Directories for the tables being created aren't cleaned up.
This unclean state will hinder table creation on any retries and will
complain that the Directory for table already exists.
To achieve a clean state on failed table creations, catch this error if
the exception is of type Poco::Exception and call `dropIfEmpty()` method,
then proceed throwing the exception. Without this, the Directory for the
tables need to be manually deleted before retrying the CreateQuery.
2020-09-07 18:09:09 -07:00
Nicolae Vartolomei
6060a1ab57
ALTER TABLE ... DROP|DETACH PART for ReplicatedMergeTree
2020-09-07 17:01:45 +01:00
alesapin
ea7168580b
Fixes
2020-09-04 16:55:07 +03:00
alesapin
e42d0f60da
Fix several bugs
2020-09-04 14:27:27 +03:00
alesapin
4079636f81
Merge
2020-09-04 13:52:51 +03:00
alesapin
82c56349a5
Some comments
2020-09-04 13:08:09 +03:00
alesapin
61ecaebcb1
Simplify settings for TTL merges
2020-09-04 09:55:19 +03:00
alesapin
f4c7ff0376
Add fixed size of Merge TTLS
2020-09-03 16:00:13 +03:00
alesapin
fbb37c37df
Simplier interface
2020-09-02 11:28:46 +03:00
alesapin
b08056fa8c
Better selection of Merges with TTL
2020-09-02 11:18:50 +03:00
alesapin
5b0822902b
Merge pull request #14155 from amosbird/fzk
...
Allow FETCH PARTITION from other zookeepers
2020-08-31 09:54:51 +03:00
alesapin
2d33a4029b
Merge pull request #14220 from ClickHouse/remove_redundant_flag
...
Disable force TTL on optimize final
2020-08-29 10:40:36 +03:00
alesapin
232c264d7d
Disable force TTL on optimise
2020-08-28 16:45:42 +03:00
alesapin
1ea00d5e7a
Fix test and better messages
2020-08-28 14:12:51 +03:00
Alexey Milovidov
5763737d97
Fixups
2020-08-28 03:53:22 +03:00
Amos Bird
078b14610d
ALTER MODIFY SAMPLE BY
2020-08-27 22:31:30 +08:00
Amos Bird
0c1cf22c00
Allow FETCH PARTITION from other zookeepers
2020-08-27 22:19:18 +08:00
