Pavel Kruglov
9120189d8a
Add SelectPartsDecision enum class
2020-11-10 17:42:56 +03:00
Nikolai Kochetov
195c941c4e
Merge branch 'master' into storage-read-query-plan
2020-11-10 15:02:22 +03:00
alesapin
2e357516a6
Merge branch 'master' into add_separate_pool_for_fetches
2020-11-10 13:53:08 +03:00
Nikolai Kochetov
046830967c
Try fix tests.
2020-11-10 13:26:26 +03:00
Vladimir Chebotarev
059357d51e
ALTER UPDATE/DELETE ... IN PARTITION with partition pruning in ReplicatedMergeTree (#13403 )
...
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2020-11-10 13:23:46 +03:00
alesapin
e347f4b72f
Add some comments
2020-11-09 12:14:20 +03:00
alesapin
72046313d4
Merge branch 'master' into nvartolomei-drop-part
2020-11-09 10:02:02 +03:00
alesapin
4e846d2197
Merge branch 'master' into add_separate_pool_for_fetches
2020-11-09 10:00:21 +03:00
alexey-milovidov
0e6ae4aff7
Merge pull request #16253 from amosbird/pf
...
Prune partition in verbatim way.
2020-11-08 18:58:02 +03:00
alexey-milovidov
f4ba5f1f9a
Merge pull request #16772 from ClickHouse/fix-stringstream
...
Fix "server failed to start" error
2020-11-08 14:27:08 +03:00
Alexey Milovidov
5314185e25
Merge branch 'master' into azat-optimize_skip_unused_shards-optimization
2020-11-08 00:17:59 +03:00
Alexey Milovidov
fd84d16387
Fix "server failed to start" error
2020-11-07 03:14:53 +03:00
Nikolai Kochetov
c10f733587
Merge branch 'master' into storage-read-query-plan
2020-11-06 15:43:46 +03:00
Amos Bird
2b0085c106
Pruning is different from counting
2020-11-06 19:58:03 +08:00
Amos Bird
30bf5e6d26
Prune partition in verbatim way.
2020-11-06 09:56:13 +08:00
alexey-milovidov
ee3e289ec2
Merge pull request #15983 from ClickHouse/no_background_pool_no_more
...
Replace BackgroundProcessingPool with SchedulePool task and ThreadPool.
2020-11-05 21:38:22 +03:00
Anton Ivashkin
1742fb3256
Merge master
2020-11-03 12:27:16 +03:00
alesapin
da8ac948bf
Disallow to drop parts with parallel quorum
2020-11-03 12:24:10 +03:00
alesapin
294489f6ad
Merge branch 'master' into nvartolomei-drop-part
2020-11-03 10:19:03 +03:00
alesapin
dac6705995
DROP PART and clear blocks in a single zookeeper op
2020-11-02 20:30:59 +03:00
alesapin
67d22b3e57
Merge branch 'master' into nvartolomei-drop-part
2020-11-02 19:09:09 +03:00
alesapin
7f8560e0c0
Merge pull request #16538 from amosbird/fp2
...
Allow fetching duplicate parts to detach
2020-11-02 16:10:42 +03:00
Amos Bird
18c5abc44e
Allow fetching duplicate parts to detach
2020-10-30 10:55:50 +08:00
alesapin
40fc512e79
Merge branch 'master' into no_background_pool_no_more
2020-10-29 12:53:34 +03:00
alesapin
0931658c2f
Merge branch 'master' into add_separate_pool_for_fetches
2020-10-28 14:20:56 +03:00
tavplubix
4a63f1271b
Merge pull request #16244 from ClickHouse/fix_race_data_parts_exchange_endpoint
...
Fix race on StorageReplicatedMergeTree::data_parts_exchange_endpoint
2020-10-28 13:25:55 +03:00
alesapin
47ebc9f74e
Add separate pool for fetches
2020-10-26 14:02:47 +03:00
hchen9
824c04f873
Fix conflict code
2020-10-25 18:12:17 -07:00
hchen9
feaab260d8
Fix conflict
2020-10-23 12:54:33 -07:00
alesapin
39e47c5338
Less strange lambdas
2020-10-23 11:54:00 +03:00
alesapin
7a63b373f9
Merge branch 'blind_fix_of_read_in_order' into no_background_pool_no_more
2020-10-22 15:47:11 +03:00
Nikolai Kochetov
7fa045cff8
Merge branch 'master' into storage-read-query-plan
2020-10-22 13:31:10 +03:00
Anton Ivashkin
f833501c77
Merge branch 'master' into s3_zero_copy_replication
2020-10-22 11:17:21 +03:00
alexey-milovidov
adeba6bdd8
Merge pull request #15074 from amosbird/btc
...
Extend trivial count optimization.
2020-10-22 02:50:57 +03:00
Alexander Tokmakov
ac42199ae5
fix race on StorageReplicatedMergeTree::data_parts_exchange_endpoint
2020-10-21 22:25:22 +03:00
alesapin
ce8ee01d75
Fix background moves start
2020-10-20 14:27:50 +03:00
alesapin
d126aa8a09
Merge branch 'master' into no_background_pool_no_more
2020-10-20 12:54:42 +03:00
Alexander Kuzmenkov
defcd29b31
Merge pull request #16036 from ClickHouse/stress_test_improvemets
...
Minor improvements of stress test
2020-10-19 16:12:30 +03:00
alesapin
e09713ece1
Merge branch 'master' into no_background_pool_no_more
2020-10-19 10:58:31 +03:00
Alexander Tokmakov
026896835c
fixes
2020-10-16 15:20:30 +03:00
alesapin
7edbe4d276
Add some comments
2020-10-16 13:12:31 +03:00
Anton Ivashkin
14a78f87b0
Zero copy replication over S3: fetch instead of merge
2020-10-15 19:14:47 +03:00
alesapin
4c1f5cab13
Sleep less in background task
2020-10-15 19:10:22 +03:00
alesapin
2002289003
Fix bug in alter primary key for replicated versioned collapsing merge tree
2020-10-15 16:02:39 +03:00
Pavel Kruglov
25c39689c3
Check expired ttl
2020-10-15 15:15:02 +03:00
alesapin
583d7042e9
Get rid of some deadlocks
2020-10-15 11:29:18 +03:00
alesapin
c5419083e3
Better method name
2020-10-15 10:43:50 +03:00
alesapin
483893cdd4
Slightly refactored code
2020-10-14 17:56:42 +03:00
alesapin
0b14a31ba9
Get rid of background processing pool
2020-10-14 15:44:10 +03:00
alesapin
4014e0f08d
Something very similar to working code
2020-10-14 15:32:35 +03:00
alesapin
efd3126b5d
Moving pool
2020-10-14 10:22:48 +03:00
Pavel Kruglov
1ccf10505a
add tests and comments
2020-10-13 21:26:40 +03:00
Pavel Kruglov
6d9b310f61
Add optimize_skip_merged_partitions
2020-10-13 21:26:35 +03:00
alesapin
fd35368c59
First simple implementation
2020-10-13 17:25:42 +03:00
alesapin
cb7db6d232
Split select and process merges
...
(cherry picked from commit 1889eb0ff1
2020-10-13 15:04:16 +03:00
Nikolai Kochetov
7e58f99f64
Merge branch 'master' into storage-read-query-plan
2020-10-12 13:12:39 +03:00
Alexey Milovidov
5b482f4191
Cleanups
2020-10-10 19:31:10 +03:00
alexey-milovidov
0fd007ad9e
Merge pull request #15601 from alexelex/alexelex-master
...
Improvement of Quorum Inserts in ClickHouse
2020-10-09 22:43:53 +03:00
Anton Ivashkin
9272ed06b4
Move Zookeeper lock for S3 shared part in IMergeTreeDataPart
2020-10-09 17:24:10 +03:00
Anton Ivashkin
766dbfd2be
Merge master
2020-10-09 15:00:14 +03:00
Alexandra Latysheva
0594a77b57
fix thread restart for parallel quorum inserts
2020-10-09 11:20:20 +00:00
tavplubix
7b5f2b2bb0
Merge pull request #15701 from ClickHouse/better_clone_replica
...
Improvements in lost replica recovery
2020-10-08 23:19:23 +03:00
Anton Ivashkin
acf86568a7
S3 zero copy replication proof of concept
2020-10-08 18:45:10 +03:00
Alexandra Latysheva
6877f4aa9c
for parallel and non-parallel quorum insertion at the same time
2020-10-08 15:35:41 +00:00
tavplubix
6065a5b65c
Update StorageReplicatedMergeTree.cpp
2020-10-08 16:37:32 +03:00
Pervakov Grigorii
3a2068f19d
Use tmp disk for vertical merge files
2020-10-08 13:49:41 +03:00
Amos Bird
867216103f
Extend trivial count optimization.
2020-10-08 18:08:17 +08:00
hchen9
ed2ce4a6fc
Resolve conflict in StorageMemory
2020-10-07 17:19:59 -07:00
Alexander Tokmakov
ffb19fa4bc
fix
2020-10-07 22:47:31 +03:00
Latysheva Alexandra
f43ad338ed
Merge pull request #12 from alexelex/quorum_inserts_2
...
style check
2020-10-07 18:56:14 +06:00
Alexandra Latysheva
8f8872dc42
style check
2020-10-07 12:24:46 +00:00
Latysheva Alexandra
6c26f77380
Merge pull request #11 from alexelex/quorum_inserts_2
...
changing the condition for updateQuorum, specialized it and reduced t…
2020-10-07 17:49:20 +06:00
Alexandra Latysheva
a43cac2c1a
changing the condition for updateQuorum, specialized it and reduced the number of calls to the zk
2020-10-07 11:28:48 +00:00
Nikolai Kochetov
7e02152b50
Merge branch 'master' into storage-read-query-plan
2020-10-07 14:27:21 +03:00
Nikolai Kochetov
c5cb05f5f3
Try fix tests.
2020-10-07 14:26:29 +03:00
tavplubix
041015545c
Merge pull request #15537 from ClickHouse/mutation_hangs_after_replace_partition
...
Fix mutation may hang after REPLACE/DROP PARTITION
2020-10-07 14:25:29 +03:00
Alexander Tokmakov
47e9fe9ff1
fix
2020-10-07 03:05:48 +03:00
Alexander Tokmakov
ea16370024
improvements in lost replica recovery
2020-10-07 02:29:33 +03:00
Latysheva Alexandra
991e51a80e
Merge pull request #10 from alexelex/quorum_inserts_2
...
style check fix
2020-10-07 04:38:30 +06:00
Alexandra Latysheva
9077c92862
style check fix
2020-10-06 22:36:42 +00:00
Latysheva Alexandra
85bcc07c0b
Merge pull request #9 from alexelex/quorum_inserts_2
...
Quorum inserts [version 3]
2020-10-07 04:04:41 +06:00
Alexandra Latysheva
644091459f
style fixes and remove small changes
2020-10-06 22:01:30 +00:00
Alexandra Latysheva
f549ecf9d1
insert_quorum_parallel: status nodes in /quorum/parallel and allow merges
2020-10-06 21:49:48 +00:00
Latysheva Alexandra
d495a2fb2d
Merge pull request #7 from alexelex/quorum_inserts_2
...
build checker fix (style) and change comments
2020-10-06 02:33:29 +06:00
Alexandra Latysheva
41e0148d72
build checker fix (style) and change comments
2020-10-05 20:30:50 +00:00
Alexander Tokmakov
418766e92b
fix another bug in mutations
2020-10-05 22:16:28 +03:00
alesapin
8ec58c17f3
More consistent metadata usage
2020-10-05 19:41:46 +03:00
Alexander Tokmakov
30a18f7567
make it better
2020-10-05 16:52:03 +03:00
Latysheva Alexandra
7e7cedd73f
Merge pull request #4 from alexelex/quorum_inserts_2
...
style check fix
2020-10-05 16:43:14 +06:00
Alexandra Latysheva
0662432c5c
style check
2020-10-05 10:41:49 +00:00
Latysheva Alexandra
a63084adaf
Merge pull request #3 from alexelex/quorum_inserts_2
...
Quorum inserts 2
2020-10-05 03:28:58 +06:00
Alexandra Latysheva
d54b6c9542
add question in comments
2020-10-04 20:32:08 +00:00
Alexandra Latysheva
ff8cbe6745
remove debug things
2020-10-04 20:19:05 +00:00
Alexandra Latysheva
8263e62298
working copy (with some debug info)
2020-10-04 19:55:39 +00:00
hchen9
cbf3af401d
Fix comments
2020-10-04 02:42:03 -07:00
Azat Khuzhin
b838214a35
Pass non-const SelectQueryInfo (and drop mutable qualifiers)
2020-10-02 22:42:35 +03:00
hchen9
b3949db00f
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into trivial_count_seq_consistency
2020-10-01 21:15:26 -07:00
Nikolai Kochetov
ec64def384
Use QueryPlan while reading from MergeTree.
2020-10-01 20:34:22 +03:00
Nicolae Vartolomei
7197a0c803
Possibly fix the issue with quorum
2020-10-01 11:38:50 +01:00
hchen9
b4494578d7
Support select_sequential_consistency in StorageReplicatedMergeTree.totalRows and totalBytes
2020-09-30 16:50:58 -07:00
Alexandra Latysheva
e89a56969f
first part
2020-09-30 23:16:27 +00:00
Alexander Tokmakov
34addcf61f
add review suggestions
2020-09-26 22:18:28 +03:00
alesapin
5ffee8808a
Alter remove column properties and TTLs ( #14742 )
2020-09-20 16:27:33 +03:00
alesapin
e38b537017
Merge pull request #11684 from ClickHouse/manual-write-duplicate-parts-to-replicas
...
Don't ignore duplicate parts written to replicas
2020-09-18 16:50:00 +03:00
Alexey Milovidov
a5560cd62d
Better drop replicated tables
2020-09-17 14:27:17 +03:00
alesapin
40b2f203b6
Merge branch 'master' into manual-write-duplicate-parts-to-replicas
2020-09-17 13:21:00 +03:00
Alexander Tokmakov
da6beb1a91
Merge branch 'master' into enable_some_tests_with_database_atomic
2020-09-14 23:00:14 +03:00
bharatnc
3b3b7bd5e7
minor changes to comment
2020-09-10 09:09:48 -07:00
Alexander Tokmakov
20b3823770
do not remove directory on attach
2020-09-10 13:21:13 +03:00
Alexander Tokmakov
dee1fefeb4
add more tests
2020-09-09 20:39:49 +03:00
bharatnc
fe6d6f16ea
Merge remote-tracking branch 'upstream/master' into ncb/zk-conn-fail-do-cleanup
2020-09-09 09:25:34 -07:00
Alexander Tokmakov
b68782d285
enable more tests with Atomic database
2020-09-09 16:32:50 +03:00
bharatnc
f37814b367
Merge remote-tracking branch 'upstream/master' into ncb/zk-conn-fail-do-cleanup
2020-09-08 23:45:20 -07:00
Nicolae Vartolomei
97d0b5ab23
Remove unused query argument
2020-09-08 10:56:10 +01:00
bharatnc
0c00b992d5
StorageReplicatedMergeTree - cleanup data dir after Zk exceptions
...
It's possible for `getZooKeeper()` to timeout if zookeeper host(s) can't
be reached. In such cases `Poco::Exception` is thrown after a connection
timeout - refer to `src/Common/ZooKeeper/ZooKeeperImpl.cpp:866` for more info.
Side effect of this is that the CreateQuery gets interrupted and it exits.
But the data Directories for the tables being created aren't cleaned up.
This unclean state will hinder table creation on any retries and will
complain that the Directory for table already exists.
To achieve a clean state on failed table creations, catch this error if
the exception is of type Poco::Exception and call `dropIfEmpty()` method,
then proceed throwing the exception. Without this, the Directory for the
tables need to be manually deleted before retrying the CreateQuery.
2020-09-07 18:09:09 -07:00
Nicolae Vartolomei
6060a1ab57
ALTER TABLE ... DROP|DETACH PART for ReplicatedMergeTree
2020-09-07 17:01:45 +01:00
alesapin
ea7168580b
Fixes
2020-09-04 16:55:07 +03:00
alesapin
e42d0f60da
Fix several bugs
2020-09-04 14:27:27 +03:00
alesapin
4079636f81
Merge
2020-09-04 13:52:51 +03:00
alesapin
82c56349a5
Some comments
2020-09-04 13:08:09 +03:00
alesapin
61ecaebcb1
Simplify settings for TTL merges
2020-09-04 09:55:19 +03:00
alesapin
f4c7ff0376
Add fixed size of Merge TTLS
2020-09-03 16:00:13 +03:00
alesapin
fbb37c37df
Simplier interface
2020-09-02 11:28:46 +03:00
alesapin
b08056fa8c
Better selection of Merges with TTL
2020-09-02 11:18:50 +03:00
alesapin
5b0822902b
Merge pull request #14155 from amosbird/fzk
...
Allow FETCH PARTITION from other zookeepers
2020-08-31 09:54:51 +03:00
alesapin
2d33a4029b
Merge pull request #14220 from ClickHouse/remove_redundant_flag
...
Disable force TTL on optimize final
2020-08-29 10:40:36 +03:00
alesapin
232c264d7d
Disable force TTL on optimise
2020-08-28 16:45:42 +03:00
alesapin
1ea00d5e7a
Fix test and better messages
2020-08-28 14:12:51 +03:00
Alexey Milovidov
5763737d97
Fixups
2020-08-28 03:53:22 +03:00
Amos Bird
078b14610d
ALTER MODIFY SAMPLE BY
2020-08-27 22:31:30 +08:00
Amos Bird
0c1cf22c00
Allow FETCH PARTITION from other zookeepers
2020-08-27 22:19:18 +08:00
alexey-milovidov
c1c5474730
Merge pull request #14051 from javisantana/fix/info_in_read_only_table
...
adds some info to the exception about zookeeper path
2020-08-26 02:27:01 +03:00
alexey-milovidov
ea4aa34697
Little bit more pretty
2020-08-25 20:09:35 +03:00
Javi Santana
4942d85090
adds some info to the exception about zookeeper path
2020-08-25 18:09:22 +02:00
Alexey Milovidov
44fb1ec107
Fix typo
2020-08-16 13:15:35 +03:00
Alexey Milovidov
8c5e5ec1cf
Fix flaky test 01305_replica_create_drop_zookeeper
2020-08-15 15:30:17 +03:00
Nikolai Kochetov
9b67cd9faf
Merge branch 'master' into refactor-pipes-3
2020-08-10 10:50:17 +03:00
Alexey Milovidov
edd89a8610
Fix half of typos
2020-08-08 03:47:03 +03:00
Nikolai Kochetov
09fbce1b1e
Merge branch 'master' into refactor-pipes-3
2020-08-04 11:32:34 +03:00
Nikolai Kochetov
2cca4d5fcf
Refactor Pipe [part 2].
2020-08-03 16:54:14 +03:00
Alexander Tokmakov
5b92f43f12
Merge branch 'master' into database_atomic_improvements
2020-08-03 14:35:40 +03:00
Alexey Milovidov
084e261687
Trigger CI
2020-08-02 01:06:09 +03:00
alesapin
45894fcf99
Better comments
2020-07-31 15:22:32 +03:00
alexey-milovidov
9c9ef0db9b
Merge branch 'master' into database_atomic_improvements
2020-07-31 14:56:49 +03:00
alesapin
14928874a1
Fix mutations status check
2020-07-31 14:37:16 +03:00
alexey-milovidov
fc35ce69fa
Merge pull request #12666 from ClickHouse/forward_alter_error_to_client
...
Forward exceptions to client if exception happened during ALTER
2020-07-30 21:36:06 +03:00
alesapin
7fc1e45c3b
Review fixe
2020-07-29 11:32:52 +03:00
alesapin
c75830e878
Alter table freeze with verbose output
2020-07-28 18:10:36 +03:00
alesapin
8c8bdd5070
Fix tests and work with multiple mutations
2020-07-22 18:19:54 +03:00
alesapin
de846e5e3c
Forward exception to client during alter
2020-07-22 15:36:19 +03:00
Alexander Tokmakov
a612080df1
Merge branch 'master' into database_atomic_improvements
2020-07-17 00:43:59 +03:00
Alexey Milovidov
e1e2204279
Whitespace
2020-07-15 19:37:52 +03:00
Alexander Tokmakov
2fe2833a0d
fix build
2020-07-15 13:18:30 +03:00
Alexander Tokmakov
09abe71b60
Merge branch 'master' into database_atomic_improvements
2020-07-15 13:07:08 +03:00
alesapin
4a53264a86
Remove redundant and duplicated code
2020-07-13 19:19:08 +03:00
Alexander Tokmakov
fdb02edc7c
Merge branch 'master' into database_atomic_improvements
2020-07-13 17:28:48 +03:00
Alexander Tokmakov
04c3e7cab6
add implicit uuid macro
2020-07-13 16:18:38 +03:00
Alexey Milovidov
7fc90aa070
Fix error
2020-07-10 02:45:29 +03:00
Alexey Milovidov
6c458a6287
Fix race condition in ReplicatedMergeTreeQueue
2020-07-09 07:43:28 +03:00
Anton Popov
4422df2e37
Merge remote-tracking branch 'upstream/master' into HEAD
2020-07-02 20:18:21 +03:00
Vladimir Chebotarev
bb8da71eff
Moves task shall be started if new storage policy needs them.
2020-06-30 15:36:44 +03:00
Anton Popov
a43cb93be5
remove questionable functionality
2020-06-29 18:46:50 +03:00
Alexey Milovidov
cc597bb30b
Implement for ReplicatedMergeTree
2020-06-28 22:39:31 +03:00
Alexey Milovidov
6aa89d9073
Typos
2020-06-28 22:39:31 +03:00
Alexey Milovidov
e3410ca617
Write partition key changes to ZooKeeper
2020-06-28 22:39:31 +03:00
Alexey Milovidov
e5f4a4b4b0
Whitespace
2020-06-28 22:39:31 +03:00
Alexey Milovidov
1462a66d1e
Fix typos
2020-06-27 22:05:00 +03:00
alesapin
6f1824f0ea
Correct merge with master
2020-06-26 14:30:23 +03:00
alesapin
e9c47dc89c
Merge branch 'master' into CurtizJ-polymorphic-parts
2020-06-26 14:27:19 +03:00
tavplubix
2401bc9143
Merge pull request #11896 from ClickHouse/merging_drop_replica
...
Merging #10679
2020-06-24 11:42:39 +03:00
Alexander Tokmakov
7b5de16da9
minor improvements
2020-06-23 22:15:01 +03:00
amudong
2a51286527
Optimize code
2020-06-23 16:22:53 +08:00
amudong
cca8df6b78
format comment
2020-06-23 12:24:05 +08:00
amudong
0101947403
rewrite static StorageReplicatedMergeTree::dropReplica
2020-06-23 12:12:30 +08:00
amudong
7723dc4935
add drop relica database and the whole replica
...
fix removeReplicaByZKPath
fix bug: add zkpath empty judge
fix: rewrite code
delete useless code.
fix:ast fromat
fix bug
add test_drop_replica
add drop_replica doc
add drop databse checkAccess
refactor dropReplica
update tests
add static method StorageReplicatedMergeTree::dropReplicaByZkPath
update doc and delete useless code
fix conflict
fix doc
fix doc
fix StorageReplicatedMergeTree::dropReplica
fix bug
delete useless code
2020-06-23 12:12:30 +08:00
sundy-li
15ad830290
only drop inactive replica
...
update doc
Increase timeout to release the zookeeper Ephemeral nodes
Fix code comment
use PartitionManager
make integrations test passed
2020-06-23 12:12:30 +08:00
sundy-li
d4aa3cea4b
fix merge state
...
Fix style && build
2020-06-23 12:12:30 +08:00
sundy-li
906a43e4a8
Add drop replica alter support
2020-06-23 12:12:30 +08:00
alesapin
b1e8976df4
Merge with master
2020-06-22 12:04:27 +03:00
alesapin
b9e74f4e82
Merge branch 'master' into atomic_metadata5
2020-06-22 12:03:53 +03:00
Alexey Milovidov
468eaa8d93
Remove unused method
2020-06-19 17:21:26 +03:00
Alexey Milovidov
5211a42c04
Remove leader election, step 3
2020-06-19 17:18:58 +03:00
alesapin
009977c20c
Fix locks
2020-06-18 22:45:37 +03:00
alesapin
8b131e2079
Remove int contention
2020-06-18 20:19:11 +03:00
alesapin
c666763cc0
Remove unused method better
2020-06-18 20:09:06 +03:00
alesapin
e888dafdc2
Remove unused method
2020-06-18 20:03:42 +03:00
alesapin
54e5fe7dbc
Less locks
2020-06-18 19:19:40 +03:00
alesapin
d79982f497
Better locks in Storages
2020-06-18 19:10:47 +03:00
alesapin
aab4ce6394
Truncate with metadata
2020-06-18 13:29:13 +03:00
alesapin
f4037b8f54
Fix build
2020-06-18 12:00:43 +03:00
alesapin
ebb36bec8a
Merge branch 'master' into atomic_metadata5
2020-06-18 11:57:16 +03:00
alesapin
d0beeaf7c4
Merge pull request #11744 from ClickHouse/stop_fetches_before_metadata_alter
...
Cancel fetches before metadata alter
2020-06-18 11:32:16 +03:00
alesapin
870d891257
Cancel fetches before metadata alter
2020-06-17 23:25:43 +03:00
Azat Khuzhin
caa88777be
Parse metadata stored in zookeeper before checking for equality
...
After #11325 trailing whitespaces has been removed for data skipping
indicies, and it may be different, if you have multiple skip indices,
and in this case new server will not load such tables, because metadata
will be different.
Fix this by re-parse metadata in zookeeper.
2020-06-17 21:03:22 +03:00
alesapin
dffdece350
getColumns in StorageInMemoryMetadta (only compilable)
2020-06-17 19:39:58 +03:00
alesapin
ed8f3b2fc4
TTL in storage in memory metadata
2020-06-17 16:39:26 +03:00
alesapin
ba04d02f1e
Compilable sorting key in metadata
2020-06-17 14:05:11 +03:00
alesapin
ab61abccc1
Partition key in StorageInMemoryMetadata
2020-06-17 13:34:23 +03:00
alesapin
2fac3290f0
Merge branch 'master' into atomic_metadata5
2020-06-17 11:14:03 +03:00
alexey-milovidov
598fd75caf
Merge pull request #11639 from ClickHouse/remove-leader-election-2
...
Remove leader election, step 2: allow multiple leaders
2020-06-17 09:25:23 +03:00
alesapin
ccc2bda666
getConstraints() in StorageInMemoryMetadata (suspicious commit, but pretend to work)
2020-06-16 19:55:04 +03:00
alesapin
1ddeb3d149
Buildable getSampleBlock in StorageInMemoryMetadata
2020-06-16 18:51:29 +03:00
alesapin
71f99a274d
Compileable getSampleBlockWithColumns in StorageInMemoryMetadata
2020-06-16 17:25:08 +03:00
alesapin
0bcd22008a
Get column dependencies in StorageInMemoryMetadata
2020-06-16 15:19:21 +03:00
alesapin
eae901f58d
Merge branch 'master' into atomic_metadata5
2020-06-16 14:44:45 +03:00
alesapin
0cb14c836c
Merge pull request #11691 from ClickHouse/fix-race-condition-system-sync-replicas
...
Fix race condition in SYSTEM SYNC REPLICA
2020-06-16 13:29:18 +03:00
Anton Popov
b19d48a11c
Merge remote-tracking branch 'upstream/master' into HEAD
2020-06-16 06:37:55 +03:00
Alexey Milovidov
53d985909e
Fix race condition in SYSTEM SYNC REPLICA
2020-06-16 01:54:19 +03:00
Anton Popov
a3ac224ae4
in-memory parts: sync insert to replicated
2020-06-15 22:57:12 +03:00
alesapin
36ba0192df
Metadata in read and write methods of IStorage
2020-06-15 22:08:58 +03:00
alesapin
b47a7327fd
All set methods in metadata
2020-06-15 21:08:05 +03:00
alesapin
bc9a3c0458
Merge branch 'master' into consistent_metadata4
2020-06-15 11:20:30 +03:00
Alexey Milovidov
bbe5f4c909
Revert "Remove "current_password" because it is harmful"
...
This reverts commit 66ccb2f6b1
2020-06-15 05:13:41 +03:00
Alexey Milovidov
66ccb2f6b1
Remove "current_password" because it is harmful
2020-06-15 04:12:01 +03:00
Alexey Milovidov
b596831342
Merge branch 'master' into remove-leader-election-2
2020-06-15 03:16:41 +03:00
Alexey Milovidov
400e9fb64f
Allow to drop table if there is no metadata in ZooKeeper; allow to rename
2020-06-14 04:23:53 +03:00
alesapin
5d3de747cd
Merge remote-tracking branch 'origin/master' into consistent_metadata4
2020-06-13 11:50:27 +03:00
Alexey Milovidov
21897f2abd
Instrument
2020-06-13 04:35:49 +03:00
Alexey Milovidov
52ac009754
Update versions on merge and mutation
2020-06-13 04:34:05 +03:00
Alexey Milovidov
833c8178ae
Whitespaces
2020-06-13 04:34:05 +03:00
Alexey Milovidov
6869c122a8
Step 1 towards removing leader election: check and modify version of the "log" node
2020-06-13 04:34:05 +03:00
Alexey Milovidov
72257061d5
Avoid errors due to implicit int<->bool conversions when using ZK API
2020-06-12 18:09:12 +03:00
alesapin
465c4b65b7
Slightly better interfaces and comments
2020-06-12 12:37:52 +03:00
alesapin
4eb8a1cfef
Fix gcc build
2020-06-12 00:03:25 +03:00
Alexey Milovidov
540f8532cb
Fix error
2020-06-11 22:59:15 +03:00
alesapin
a39d010f94
Merge branch 'master' into consistent_metadata4
2020-06-11 18:47:41 +03:00
alexey-milovidov
5f073c3ef4
Update StorageReplicatedMergeTree.cpp
2020-06-11 17:11:13 +03:00
Alexey Milovidov
790bc4e8e0
Better code
2020-06-11 06:35:59 +03:00
Alexey Milovidov
b72a095e99
Fix race conditions in CREATE/DROP of different replicas of ReplicatedMergeTree
2020-06-11 06:24:52 +03:00
alesapin
dd3cf0fe0a
Don't miss columns TTLs update
2020-06-10 21:02:45 +03:00
alesapin
8be957ecb5
Better checks around metadata
2020-06-10 14:16:31 +03:00
alesapin
ec933d9d03
Better naming
2020-06-10 12:09:51 +03:00
alesapin
75977bd939
Gcc fixed
2020-06-10 00:22:01 +03:00
alesapin
f7a397242f
Merge branch 'master' into consistent_metadata4
2020-06-09 18:59:59 +03:00
alesapin
223f45685f
Review fixes
2020-06-09 05:03:22 +03:00
alesapin
de59629b38
Fix benign race condition during shutdown
2020-06-09 04:48:30 +03:00
alesapin
a5a9048109
At least fixed gcc-9 build
2020-06-08 21:23:26 +03:00
Anton Popov
66e31d4311
in-memory parts: several fixes
2020-06-05 23:49:24 +03:00
alesapin
abaf47f0cd
Make metadata single structure
2020-06-05 20:29:40 +03:00
Nikita Mikhaylov
4e4218e10d
Merge pull request #11385 from ClickHouse/akz/bg-processing-pool-update1
...
Extended BackgroundProcessingPool::startTask() to control task execution concurrency
2020-06-04 16:19:29 +04:00
Alexey Milovidov
484d269c22
Fix typo
2020-06-04 01:15:13 +03:00
Alexey Milovidov
a0fd9b021a
Fix MSan failure in MergeTree
2020-06-04 01:11:06 +03:00
Anton Popov
df3dfd5b81
fix clang-tidy build
2020-06-04 01:00:02 +03:00
Anton Popov
1ce09e1faa
Merge remote-tracking branch 'upstream/master' into polymorphic-parts
2020-06-03 16:27:54 +03:00
Alexander Kazakov
9fbd72ab34
Renamed method
2020-06-02 18:18:36 +03:00
alesapin
e5eb6071a4
Merge with master
2020-06-02 10:46:58 +03:00
alesapin
f287a28fc5
Stop merges before taking exclusive lock :(
2020-06-01 18:16:10 +03:00
alesapin
b49be4c7f2
Better names
2020-06-01 15:11:23 +03:00
alesapin
3847ea892d
Merge branch 'master' into consistent_metadata3
2020-06-01 13:17:59 +03:00
Alexander Tokmakov
c071e0a473
add context to DatabaseCatalog methods
2020-05-29 02:01:18 +03:00
alesapin
52ca6b2051
I'm able to build it
2020-05-28 15:37:05 +03:00
Anton Popov
d8342e5b12
in-memory parts: better restore and clear stale wal files
2020-05-27 23:05:55 +03:00
alesapin
c3a6571036
Compilable code
2020-05-25 20:22:20 +03:00
alesapin
6281dd6893
Merge pull request #11115 from ClickHouse/consistent_metadata
...
Refactoring in storage metadata (more consistent keys)
2020-05-25 13:09:56 +03:00
Alexey Milovidov
4ebf63fb23
Fixed error
2020-05-24 22:17:15 +03:00
Alexey Milovidov
7e1813825b
Return old names of macros
2020-05-24 01:24:01 +03:00
Alexey Milovidov
af2a59e49b
Progress on task
2020-05-24 00:50:34 +03:00
Alexey Milovidov
bab24879e9
Progress on task
2020-05-24 00:16:05 +03:00
Alexey Milovidov
e1695feb7f
Apply all transformations again
2020-05-23 23:40:32 +03:00
Alexey Milovidov
85f84550ba
Progress on task
2020-05-23 23:37:37 +03:00
Alexey Milovidov
d5b7883451
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7{}\9", \4, \6, \8);/'
2020-05-23 22:41:08 +03:00
Alexey Milovidov
7e2fb9ad65
Apply all transformations again
2020-05-23 22:38:30 +03:00
Alexey Milovidov
eacff92d0e
Progress on task
2020-05-23 22:35:08 +03:00
Alexey Milovidov
29762240de
Remove duplicate whitespaces (preparation)
2020-05-23 22:31:54 +03:00
Alexey Milovidov
9d2a0d2dd7
Apply all transformations again
2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f
Remove duplicate whitespaces (preparation)
2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65
Make all LOG in single line (preparation)
2020-05-23 21:31:37 +03:00
Alexey Milovidov
b3f4fe12dd
Progress on task
2020-05-23 20:25:19 +03:00
Alexey Milovidov
f68d1ceb4f
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}\7{}", \4, \6, \8);/'
2020-05-23 20:02:09 +03:00
Alexey Milovidov
533f86278a
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7", \4, \6);/'
2020-05-23 20:00:41 +03:00
Alexey Milovidov
8042e5febe
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/'
2020-05-23 19:58:15 +03:00
Alexey Milovidov
e391b77d81
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/'
2020-05-23 19:56:05 +03:00
Alexey Milovidov
ee4ffbc332
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/'
2020-05-23 19:47:56 +03:00
Alexey Milovidov
8d2e80a5e2
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/'
2020-05-23 19:42:39 +03:00
alesapin
b9dfa61774
Merge pull request #11042 from CurtizJ/modify-ttl
...
Apply TTL for old data, after ALTER MODIFY TTL query
2020-05-22 11:12:14 +03:00
alesapin
41914588fd
More clear
2020-05-21 22:07:18 +03:00
alesapin
1906762119
Merge pull request #11109 from ClickHouse/less_verbose_logging
...
Less verbose logging in mutation finalization task
2020-05-21 21:53:08 +03:00
alesapin
5676340f6b
Less verbose logging in mutation finalization task
2020-05-21 18:52:11 +03:00
Azat Khuzhin
d93b9a57f6
Forward declaration for Context as much as possible.
...
Now after changing Context.h 488 modules will be recompiled instead of 582.
2020-05-21 01:53:18 +03:00
alesapin
616902a995
Sorting and primary key (broken)
2020-05-20 21:11:38 +03:00
Anton Popov
d62fe980ef
Merge remote-tracking branch 'upstream/master' into modify-ttl
2020-05-20 16:52:30 +03:00
Anton Popov
58ed04dc24
materialize TTL after its modifying
2020-05-19 04:53:01 +03:00
Gleb Novikov
1a25ac6e1f
Merge branch 'master' into refactor-reservations
2020-05-16 23:34:45 +03:00
Anton Popov
14e1a46a8c
Merge remote-tracking branch 'upstream/master' into polymorphic-parts
2020-05-15 03:48:27 +03:00
Anton Popov
4c03f48703
in-memory parts: better restoring
2020-05-15 03:39:15 +03:00
alexey-milovidov
623b2e5a43
Merge pull request #10849 from ClickHouse/fix_optimize_and_alter_hangs
...
Fix mutations and OPTIMIZE hangs when replica becomes inactive
2020-05-13 23:41:13 +03:00
Alexander Tokmakov
032197b015
fix alter and optimize hangs when replica becomes inactive
2020-05-13 10:06:57 +03:00
alesapin
13c8ffc0c9
Fix exception message after server shutdown ( #10841 )
...
* After shutdown cancel background logs pulling after task were deleted from pool
* Fix comment
* Cancel with lock
* Remove whitspace
2020-05-13 06:35:43 +03:00
Gleb Novikov
d255bb998a
VolumePtr instead of DiskPtr in IMergedBlockOutputStream and inherited classes
2020-05-11 20:20:26 +03:00
Gleb Novikov
57b6fb6d80
Merge branch 'master' into refactor-reservations
2020-05-10 13:01:45 +03:00
Alexey Milovidov
36d9d646a9
Fix race condition in ReplicatedMergeTreeQueue
2020-05-10 05:18:10 +03:00
Gleb Novikov
390b39b272
VolumePtr instead of DiskPtr in MergeTreeData*
2020-05-10 00:24:15 +03:00
Anton Popov
4a396090f8
fix build
2020-05-07 01:24:00 +03:00
Anton Popov
387041a97e
Merge remote-tracking branch 'upstream/master' into polymorphic-parts
2020-05-06 19:22:07 +03:00
Gleb Novikov
c637d99e07
Volumes and storages refactoring:
...
1. Moved Volume to separate file
2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
4. Removed old Volume completely
5. Moved StoragePolicy to separated files
6. Moved DiskSelector to separated files
7. Removed DiskSpaceMonitor file
2020-05-04 23:15:38 +03:00
Anton Popov
aa723d245f
Merge remote-tracking branch 'upstream/master' into polymorphic-parts
2020-04-30 18:27:39 +03:00
Anton Popov
42997bce86
im-memory parts: replication
2020-04-29 20:14:49 +03:00
alesapin
228dbf475e
Finalize mutations in background ( #10526 )
2020-04-27 19:19:04 +03:00
Alexander Tokmakov
04d6b59ac0
Merge branch 'master' into database_atomic
2020-04-23 17:31:37 +03:00
Nikita Mikhaylov
3520755ab5
fix c++20 usage
2020-04-23 12:00:43 +03:00
alexey-milovidov
4f6bb00424
Merge pull request #10420 from ClickHouse/clang-tidy-8
...
Fixes for clang-tidy on clang-10
2020-04-23 00:33:41 +03:00
Alexander Tokmakov
fefbbd37df
Merge branch 'master' into database_atomic
2020-04-22 17:02:30 +03:00
Alexey Milovidov
1e325a9fd9
Checkpoint
2020-04-22 09:22:14 +03:00
Nikita Mikhaylov
2173bcf28e
Merge branch 'master' of github.com:ClickHouse/ClickHouse into quorum_bug
2020-04-21 14:35:06 +03:00
Alexander Tokmakov
b29bddac12
Merge branch 'master' into database_atomic
2020-04-20 14:09:09 +03:00
Nikita Mikhaylov
05206df88c
better
2020-04-20 13:56:59 +03:00
alesapin
1bc84ec8b4
Fix comments
2020-04-20 13:56:23 +03:00
alesapin
8767ea7efa
Add setting which restrict replica to merge parts by itself and always prefer dowloading from other replicas.
2020-04-20 13:53:29 +03:00
Alexey Milovidov
47e4d14ece
Whitespace
2020-04-20 09:13:04 +03:00
Alexey Milovidov
a3195d56ae
Fix translation
2020-04-20 07:19:50 +03:00
Nikita Mikhaylov
9a80eb6e47
Merge branch 'master' of github.com:ClickHouse/ClickHouse into quorum_bug
2020-04-16 21:47:34 +03:00
Nikita Mikhaylov
85fbf413c1
server crashes, just to save
2020-04-16 21:47:20 +03:00
Alexander Tokmakov
605f3b2119
fix DROP vs OPTIMIZE race in ReplicatedMergeTree
2020-04-16 18:30:18 +03:00
Alexey Milovidov
cdeda4ab91
Fix usage of max_parser_depth setting; remove harmful default function arguments
2020-04-16 04:06:10 +03:00
Nikita Mikhaylov
89f80a6353
Merge branch 'master' into quorum_bug
2020-04-13 23:25:55 +04:00
Nikita Mikhaylov
441bb13d4a
better
2020-04-13 18:21:05 +03:00
Nikita Mikhaylov
d5c055ab03
remove logs
2020-04-13 15:13:13 +03:00
Alexander Tokmakov
5e6d4b9449
Merge branch 'master' into database_atomic
2020-04-12 16:35:44 +03:00
Nikita Mikhaylov
a8d72d01c3
mvp fix
2020-04-11 00:29:54 +03:00
Alexander Tokmakov
033d7f9b32
Merge branch 'master' into database_atomic
2020-04-10 17:12:09 +03:00
Alexander Kazakov
497df3086f
Merge branch 'master' into timed_rwlock
...
Change-Id: I620bfde2121ff013773b001d514b40b1e796a58b
2020-04-10 11:38:20 +03:00
Alexander Kazakov
26dd6140b2
Added new config settings to control timeouts
...
* "lock_acquire_timeout" controls for how long a query will continue to
acquire each lock on its argument tables
* "lock_acquire_timeout_for_background_operations" is a per-table
setting for storages of *MergeTree family
2020-04-09 21:10:27 +03:00
Alexander Tokmakov
140cd88c20
fixes
2020-04-09 02:59:39 +03:00
alexey-milovidov
c389fee7e7
Update StorageReplicatedMergeTree.cpp
2020-04-08 22:48:01 +03:00
Alexander Tokmakov
dd1590830b
Merge branch 'master' into database_atomic
2020-04-08 22:00:46 +03:00
alesapin
ded306f016
Fix adaptive granularity compatibility
2020-04-08 12:51:04 +03:00
alexey-milovidov
ead277b717
Update StorageReplicatedMergeTree.cpp
2020-04-07 20:23:26 +03:00
alesapin
27777c1d2b
Better comment
2020-04-07 18:55:23 +03:00
Alexander Tokmakov
4c48b7dd80
better rename
2020-04-07 18:31:33 +03:00
alesapin
82a87bc0d2
Fix replicated tables startup when updating from old version
2020-04-07 18:28:29 +03:00
Alexander Tokmakov
08bae4668d
Merge branch 'master' into database_atomic
2020-04-06 16:18:07 +03:00
alesapin
1cb072d58a
Merge branch 'master' into alter_rename_column
2020-04-06 11:40:27 +03:00
Ivan Lezhankin
06446b4f08
dbms/ → src/
2020-04-03 18:14:31 +03:00
