Alexey Milovidov
e6b44756dc
Fixed build
2023-02-12 07:45:28 +01:00
alesapin
5c83ddabf9
Merge branch 'master' into mutations_rename_hang
2023-02-11 12:18:58 +01:00
alesapin
b94f9f8006
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica
2023-02-06 12:38:39 +01:00
alesapin
b265300a19
Merge branch 'master' into mutations_rename_hang
2023-02-06 12:25:29 +01:00
mateng915
3e5d3914ba
Merge branch 'master' into replicas_status_api_optimize
2023-02-06 09:58:31 +08:00
alesapin
f91c10d09d
Initial support for version
2023-02-03 20:49:07 +01:00
alesapin
7a0d0f0c53
Some code which doesn't work
2023-02-03 19:14:49 +01:00
alesapin
bdc530dead
Fix style
2023-02-02 17:30:38 +01:00
Sema Checherinda
41a1dddc7a
Merge branch 'master' into better_postpone_reason
2023-02-01 15:16:34 +01:00
Nikolai Kochetov
e2c32ccbca
Aggregate Projections analysis using query plan [In progress]
2023-01-31 20:33:01 +00:00
Alexander Gololobov
863548114d
force_wait flag is no longer needed
2023-01-30 18:38:28 +01:00
SmitaRKulkarni
b22fd40d92
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica
2023-01-27 15:00:47 +01:00
Smita Kulkarni
8c885ffeae
Updated checking of SYSTEM SYNC REPLICA
...
Implementation:
* Updated to wait for current last entry to be processed (after pulling shared log) instead of queue size becoming 0.
* Updated Subscriber to notify both queue size and removed log_entry_id.
2023-01-26 11:40:09 +01:00
Anton Popov
8e3698c91f
refactoring of code near merge tree parts
2023-01-25 17:34:09 +00:00
Frank Chen
1fe9bde594
Set replica name to zero copy exclusive lock
2023-01-20 15:48:33 +08:00
mateng915
5dc81fac2b
Merge branch 'master' into replicas_status_api_optimize
2023-01-20 10:19:40 +08:00
Han Fei
3481b4d50a
fix style
2023-01-16 10:41:35 +01:00
Han Fei
a258a39eb1
Merge branch 'master' into hanfei/async-cache
2023-01-13 20:17:58 +01:00
mateng0915
eae0f9957c
Feature: Optimize the replica delay api logic
...
Description:
============
Currently if in the Whole CK instance has one table is read only status
then the API /replicas_status will throw error, xxx table is read only
For make this monitor can work in prod env, we can catch the read only
status instead of directly throw error
Solution:
=========
Return other normal table's delay value even if the CK instance has readonly Replicatedxxx table
Please enter the commit message for your changes. Lines starting
2023-01-13 14:44:27 +08:00
Raúl Marín
9f121d5d9c
Fix background_fetches_pool_size config reload ( #45189 )
2023-01-12 23:41:25 +01:00
Han Fei
bef6168c05
support cache for async inserts block ids
2023-01-10 13:19:12 +01:00
alesapin
616e37ba43
Merge pull request #44847 from ClickHouse/fix_rare_race
...
Fix rare race which can lead to queue hang
2023-01-10 12:09:10 +01:00
Alexander Gololobov
742887ce53
Merge pull request #44718 from ClickHouse/lwd_mutation_always_sync
...
Make lightweight deletes always synchronous
2023-01-09 15:58:02 +01:00
Nikita Mikhaylov
857799fbca
Parallel distributed insert select with s3Cluster [3] ( #44955 )
...
* Revert "Revert "Resurrect parallel distributed insert select with s3Cluster (#41535 )""
This reverts commit b8d9066004
2023-01-09 13:30:32 +01:00
alesapin
da83e97b4c
Fix startup issue
2023-01-06 15:36:53 +01:00
alesapin
2a5b4067e4
Review fixes
2023-01-04 15:15:14 +01:00
Alexander Gololobov
2c64f0042a
Added force_wait parameter to IStorage::mutate()
2022-12-30 16:23:03 +01:00
alesapin
5dfefb43a3
Fix deadlock in attach thread
2022-12-29 18:52:20 +01:00
alesapin
fa467b3398
Minor code polishing
2022-12-22 14:31:42 +01:00
Han Fei
3c11aca040
address comments
2022-12-07 23:40:52 +01:00
Han Fei
28564df6f1
Merge branch 'master' into hanfei/aysnc-insert
2022-11-24 09:12:43 +01:00
Han Fei
32c7fbec55
use another dedup path for async inserts
2022-11-22 15:12:00 +01:00
Han Fei
b80a2d6c89
address comments
2022-11-18 17:22:05 +01:00
Han Fei
14701670b5
Merge branch 'master' into hanfei/aysnc-insert
2022-11-16 17:24:36 +01:00
Alexander Tokmakov
c64109a2de
Merge branch 'master' into fetch-remove-covered
2022-11-14 18:23:16 +03:00
Han Fei
11f93dec3f
tmp commit
2022-11-14 15:53:45 +01:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)""
2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)"
2022-11-10 14:12:24 +03:00
Igor Nikonov
379113f8f1
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-11-09 13:03:32 +00:00
Azat Khuzhin
2bd4a71150
Merge branch 'master' into fetch-remove-covered
2022-11-08 00:55:00 +01:00
Alexander Tokmakov
e67d05604c
Merge branch 'master' into fix_intersecting_parts2
2022-11-02 19:30:27 +01:00
Alexander Tokmakov
00c9e50ee3
fix race between INSERT and ALTER PARTITION
2022-11-02 16:25:26 +01:00
Igor Nikonov
9cc64a0fcc
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-31 18:21:21 +00:00
Alexander Tokmakov
c68ab231f9
fix accessing part in Deleting state
2022-10-30 17:30:51 +01:00
Igor Nikonov
3682913172
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-27 22:01:53 +00:00
Azat Khuzhin
8db31beee6
Remove covered parts for fetched part
...
Here is an example that I found on production, simplified.
Consider the following queue (nothing of this had been processed on this
replica):
- GET_PART all_0_0_0 (queue-0000000001)
- GET_PART all_1_1_0 (queue-0000000002)
...
- GET_PART all_0_1_1 (queue-0000000003)
- GET_PART all_2_2_0 (queue-0000000004)
...
- MERGE_PARTS from [all_0_1_1, all_2_2_0] to all_0_2_2 (queue-0000000005)
And now queue-0000000005 started to executing (either because
of reording, or because at that time GET_PART fails), and it
does not have any required parts, so it will fetch them, but
not all_0_0_0 and all_1_1_0, so this replica delay will set to
the time of min(queue-0000000001, queue-0000000002), while it
is not true, since it already have parts that covers those
parts.
and since MERGE_PARTS takes 30min, it increased the replica delay
eventually to 30min, for the time range of 30min, which is pretty huge.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-26 11:10:50 +03:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part
2022-10-25 14:31:05 +02:00
Igor Nikonov
54013009d3
Code migration from #41734 to align diff and git blame.
...
Initial implementation was different and it changed the entire
ReplicatedMergeTreeSink::commitPart() which change history provided by git blame.
Then RetriesControl.retryLoop() was introduced later which significantly reduces
the diff since it's like while() used before.
So, check outing the current version will keep more original history in
git blame, which is useful here
2022-10-24 08:20:11 +00:00
Anton Popov
b40d9200d2
better semantic of constsness of DataPartStorage
2022-10-23 15:24:20 +00:00
Igor Nikonov
c0d1be9cab
Merge pull request #42560 from ClickHouse/igor/replicated_merge_tree_some_polishing
...
Some polishing: replicated merge tree
2022-10-23 15:15:24 +02:00
Anton Popov
56e5daba0c
remove DataPartStorageBuilder
2022-10-23 00:23:15 +00:00
Igor Nikonov
851f556d65
Remove unused parameter
2022-10-21 14:35:37 +02:00
Igor Nikonov
13d31eefbc
Replicated merge tree polishing
...
+ make read-only members const
+ reduce memory allocations
2022-10-21 14:02:52 +02:00
Azat Khuzhin
4e76629aaf
Fixes for -Wshorten-64-to-32
...
- lots of static_cast
- add safe_cast
- types adjustments
- config
- IStorage::read/watch
- ...
- some TODO's (to convert types in future)
P.S. That was quite a journey...
v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Alexander Tokmakov
b8d9066004
Revert "Resurrect parallel distributed insert select with s3Cluster ( #41535 )"
...
This reverts commit 860e34e760
2022-10-07 15:53:30 +02:00
alesapin
3373480ede
Merge pull request #42036 from ClickHouse/fix_one_more_no_such_key
...
Fix No such key during table drop
2022-10-06 14:50:02 +02:00
Nikita Mikhaylov
860e34e760
Resurrect parallel distributed insert select with s3Cluster ( #41535 )
2022-10-06 13:47:32 +02:00
alesapin
210882b9c4
Better fix
2022-10-03 23:30:50 +02:00
alesapin
3114ddcad2
Fix failure during table drop
2022-10-03 19:06:12 +02:00
Nikita Mikhaylov
dbcba1490c
Try to make MergeTreeRestartingThread more reactive in case of expired sessions ( #41092 )
2022-09-23 14:40:04 +02:00
alesapin
acde29e314
Nasty code
2022-09-20 17:55:06 +02:00
alesapin
cfe398cbbc
Fix lost table shared id during server restart
2022-09-20 14:51:16 +02:00
Alexander Tokmakov
4d146b05a9
Merge pull request #38262 from PolyProgrammist/fix-ordinary-system-unfreeze
...
Fix SYSTEM UNFREEZE for ordinary database
2022-09-13 14:55:11 +03:00
Alexander Tokmakov
da4fbb552d
Merge branch 'master' into fix_parts_removal
2022-09-12 17:49:29 +03:00
alesapin
21ab379026
Fix comment
2022-09-11 13:47:04 +02:00
alesapin
ac53830000
Fix hung check
2022-09-11 13:37:39 +02:00
alesapin
e3fc78aa5f
Fix bad warnings in part fetches
2022-09-09 12:14:42 +02:00
ianton-ru
39e1fc7a0f
Merge branch 'master' into fix-ordinary-system-unfreeze
2022-09-05 17:10:59 +03:00
Luck-Chang
1ac8e739c9
fix some typos and clang-tidy warnings
2022-09-05 09:50:24 +08:00
Alexander Tokmakov
eae2667a1c
Merge pull request #40148 from ClickHouse/replicate-tables-readonly
...
`ReplicatedMergeTree` tables as readonly until attach is finished
2022-08-25 19:39:19 +03:00
Antonio Andelic
797e21761a
Address PR comments
2022-08-24 18:32:03 +00:00
Antonio Andelic
da0d6056f4
Merge branch 'master' into replicate-tables-readonly
2022-08-24 17:29:33 +00:00
alesapin
eeb0a3584e
Merge branch 'master' into data_source_description
2022-08-24 15:26:18 +02:00
Alexander Tokmakov
f9f85a0e8b
Revert "Parallel distributed insert select from *Cluster table functions ( #39107 )"
...
This reverts commit d3cc234986
2022-08-24 15:17:15 +03:00
alesapin
d8664c3227
Add shortcut for backups
2022-08-19 16:58:30 +02:00
Antonio Andelic
ee3df77fda
Simplify logic
2022-08-19 11:12:20 +00:00
Antonio Andelic
5e2c8869a9
Merge branch 'master' into replicate-tables-readonly
2022-08-19 09:41:11 +00:00
Antonio Andelic
a8f3b62bf6
Use global zookeeper
2022-08-19 08:49:51 +00:00
Alexander Tokmakov
832d7c6e7c
Merge branch 'master' into replicated_database_improvements
2022-08-16 15:49:13 +02:00
Antonio Andelic
2ed42bca7e
Merge branch 'master' into replicate-tables-readonly
2022-08-16 08:17:53 +00:00
Nikita Mikhaylov
d3cc234986
Parallel distributed insert select from *Cluster table functions ( #39107 )
2022-08-15 12:41:17 +02:00
Antonio Andelic
c6b3a897d2
Test replicated table without ZK, small fixes
2022-08-12 11:56:46 +00:00
Antonio Andelic
1c0f85ccdb
Attach table in background
2022-08-12 09:32:13 +00:00
Alexander Tokmakov
6f88065cc1
check metadata consistency
2022-07-29 18:33:16 +02:00
jianmei zhang
c1834d183b
ReplicatedMergeTree support lightweight delete
2022-07-26 11:50:09 +08:00
Vadim Volodin
85bf022258
review zookeeper and partition matching
2022-07-21 16:57:28 +03:00
Vitaly Baranov
7689e0c36f
Improve gathering metadata for backup - part 6.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
461a31f237
Improve gathering metadata for backup - part 2.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
64b51a3772
Improve gathering metadata for backup.
2022-06-30 08:37:17 +02:00
Vadim Volodin
3296ba2532
Fix SYSTEM UNFREEZE for ordinary database
2022-06-24 20:46:16 +03:00
Nikolai Kochetov
7452d04e3a
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-20 15:31:02 +02:00
Vitaly Baranov
c2c35fad82
Refactoring of the code getting create table queries for backup.
2022-06-15 20:32:35 +02:00
Vitaly Baranov
592f568f83
Move backup/restore code to storages and databases - part 2.
2022-06-15 20:32:31 +02:00
Vitaly Baranov
724bc4dc57
Move backup/restore code to storages and databases - part 1.
2022-06-15 20:28:43 +02:00
Vitaly Baranov
73b1894a21
Rework collecting replicated parts.
2022-06-15 20:28:42 +02:00
Vitaly Baranov
5cabdbd982
Restore parts of MergeTree in correct order.
2022-06-15 20:28:40 +02:00
Nikolai Kochetov
2a9a63ac7b
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-15 15:35:26 +02:00
Vadim Volodin
637d293fbd
Add SYSTEM UNFREEZE query
2022-06-08 15:21:14 +03:00
Nikolai Kochetov
678d978acf
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-07 15:23:00 +00:00
Nikolai Kochetov
89c5855d20
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-02 12:19:07 +02:00
