alesapin
5dfefb43a3
Fix deadlock in attach thread
2022-12-29 18:52:20 +01:00
alesapin
fa467b3398
Minor code polishing
2022-12-22 14:31:42 +01:00
Han Fei
3c11aca040
address comments
2022-12-07 23:40:52 +01:00
Han Fei
28564df6f1
Merge branch 'master' into hanfei/aysnc-insert
2022-11-24 09:12:43 +01:00
Han Fei
32c7fbec55
use another dedup path for async inserts
2022-11-22 15:12:00 +01:00
Han Fei
b80a2d6c89
address comments
2022-11-18 17:22:05 +01:00
Han Fei
14701670b5
Merge branch 'master' into hanfei/aysnc-insert
2022-11-16 17:24:36 +01:00
Alexander Tokmakov
c64109a2de
Merge branch 'master' into fetch-remove-covered
2022-11-14 18:23:16 +03:00
Han Fei
11f93dec3f
tmp commit
2022-11-14 15:53:45 +01:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)""
2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)"
2022-11-10 14:12:24 +03:00
Igor Nikonov
379113f8f1
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-11-09 13:03:32 +00:00
Azat Khuzhin
2bd4a71150
Merge branch 'master' into fetch-remove-covered
2022-11-08 00:55:00 +01:00
Alexander Tokmakov
e67d05604c
Merge branch 'master' into fix_intersecting_parts2
2022-11-02 19:30:27 +01:00
Alexander Tokmakov
00c9e50ee3
fix race between INSERT and ALTER PARTITION
2022-11-02 16:25:26 +01:00
Igor Nikonov
9cc64a0fcc
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-31 18:21:21 +00:00
Alexander Tokmakov
c68ab231f9
fix accessing part in Deleting state
2022-10-30 17:30:51 +01:00
Igor Nikonov
3682913172
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-27 22:01:53 +00:00
Azat Khuzhin
8db31beee6
Remove covered parts for fetched part
...
Here is an example that I found on production, simplified.
Consider the following queue (nothing of this had been processed on this
replica):
- GET_PART all_0_0_0 (queue-0000000001)
- GET_PART all_1_1_0 (queue-0000000002)
...
- GET_PART all_0_1_1 (queue-0000000003)
- GET_PART all_2_2_0 (queue-0000000004)
...
- MERGE_PARTS from [all_0_1_1, all_2_2_0] to all_0_2_2 (queue-0000000005)
And now queue-0000000005 started to executing (either because
of reording, or because at that time GET_PART fails), and it
does not have any required parts, so it will fetch them, but
not all_0_0_0 and all_1_1_0, so this replica delay will set to
the time of min(queue-0000000001, queue-0000000002), while it
is not true, since it already have parts that covers those
parts.
and since MERGE_PARTS takes 30min, it increased the replica delay
eventually to 30min, for the time range of 30min, which is pretty huge.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-26 11:10:50 +03:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part
2022-10-25 14:31:05 +02:00
Igor Nikonov
54013009d3
Code migration from #41734 to align diff and git blame.
...
Initial implementation was different and it changed the entire
ReplicatedMergeTreeSink::commitPart() which change history provided by git blame.
Then RetriesControl.retryLoop() was introduced later which significantly reduces
the diff since it's like while() used before.
So, check outing the current version will keep more original history in
git blame, which is useful here
2022-10-24 08:20:11 +00:00
Anton Popov
b40d9200d2
better semantic of constsness of DataPartStorage
2022-10-23 15:24:20 +00:00
Igor Nikonov
c0d1be9cab
Merge pull request #42560 from ClickHouse/igor/replicated_merge_tree_some_polishing
...
Some polishing: replicated merge tree
2022-10-23 15:15:24 +02:00
Anton Popov
56e5daba0c
remove DataPartStorageBuilder
2022-10-23 00:23:15 +00:00
Igor Nikonov
851f556d65
Remove unused parameter
2022-10-21 14:35:37 +02:00
Igor Nikonov
13d31eefbc
Replicated merge tree polishing
...
+ make read-only members const
+ reduce memory allocations
2022-10-21 14:02:52 +02:00
Azat Khuzhin
4e76629aaf
Fixes for -Wshorten-64-to-32
...
- lots of static_cast
- add safe_cast
- types adjustments
- config
- IStorage::read/watch
- ...
- some TODO's (to convert types in future)
P.S. That was quite a journey...
v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Alexander Tokmakov
b8d9066004
Revert "Resurrect parallel distributed insert select with s3Cluster ( #41535 )"
...
This reverts commit 860e34e760
.
2022-10-07 15:53:30 +02:00
alesapin
3373480ede
Merge pull request #42036 from ClickHouse/fix_one_more_no_such_key
...
Fix No such key during table drop
2022-10-06 14:50:02 +02:00
Nikita Mikhaylov
860e34e760
Resurrect parallel distributed insert select with s3Cluster ( #41535 )
2022-10-06 13:47:32 +02:00
alesapin
210882b9c4
Better fix
2022-10-03 23:30:50 +02:00
alesapin
3114ddcad2
Fix failure during table drop
2022-10-03 19:06:12 +02:00
Nikita Mikhaylov
dbcba1490c
Try to make MergeTreeRestartingThread more reactive in case of expired sessions ( #41092 )
2022-09-23 14:40:04 +02:00
alesapin
acde29e314
Nasty code
2022-09-20 17:55:06 +02:00
alesapin
cfe398cbbc
Fix lost table shared id during server restart
2022-09-20 14:51:16 +02:00
Alexander Tokmakov
4d146b05a9
Merge pull request #38262 from PolyProgrammist/fix-ordinary-system-unfreeze
...
Fix SYSTEM UNFREEZE for ordinary database
2022-09-13 14:55:11 +03:00
Alexander Tokmakov
da4fbb552d
Merge branch 'master' into fix_parts_removal
2022-09-12 17:49:29 +03:00
alesapin
21ab379026
Fix comment
2022-09-11 13:47:04 +02:00
alesapin
ac53830000
Fix hung check
2022-09-11 13:37:39 +02:00
alesapin
e3fc78aa5f
Fix bad warnings in part fetches
2022-09-09 12:14:42 +02:00
ianton-ru
39e1fc7a0f
Merge branch 'master' into fix-ordinary-system-unfreeze
2022-09-05 17:10:59 +03:00
Luck-Chang
1ac8e739c9
fix some typos and clang-tidy warnings
2022-09-05 09:50:24 +08:00
Alexander Tokmakov
eae2667a1c
Merge pull request #40148 from ClickHouse/replicate-tables-readonly
...
`ReplicatedMergeTree` tables as readonly until attach is finished
2022-08-25 19:39:19 +03:00
Antonio Andelic
797e21761a
Address PR comments
2022-08-24 18:32:03 +00:00
Antonio Andelic
da0d6056f4
Merge branch 'master' into replicate-tables-readonly
2022-08-24 17:29:33 +00:00
alesapin
eeb0a3584e
Merge branch 'master' into data_source_description
2022-08-24 15:26:18 +02:00
Alexander Tokmakov
f9f85a0e8b
Revert "Parallel distributed insert select from *Cluster table functions ( #39107 )"
...
This reverts commit d3cc234986
.
2022-08-24 15:17:15 +03:00
alesapin
d8664c3227
Add shortcut for backups
2022-08-19 16:58:30 +02:00
Antonio Andelic
ee3df77fda
Simplify logic
2022-08-19 11:12:20 +00:00
Antonio Andelic
5e2c8869a9
Merge branch 'master' into replicate-tables-readonly
2022-08-19 09:41:11 +00:00
Antonio Andelic
a8f3b62bf6
Use global zookeeper
2022-08-19 08:49:51 +00:00
Alexander Tokmakov
832d7c6e7c
Merge branch 'master' into replicated_database_improvements
2022-08-16 15:49:13 +02:00
Antonio Andelic
2ed42bca7e
Merge branch 'master' into replicate-tables-readonly
2022-08-16 08:17:53 +00:00
Nikita Mikhaylov
d3cc234986
Parallel distributed insert select from *Cluster table functions ( #39107 )
2022-08-15 12:41:17 +02:00
Antonio Andelic
c6b3a897d2
Test replicated table without ZK, small fixes
2022-08-12 11:56:46 +00:00
Antonio Andelic
1c0f85ccdb
Attach table in background
2022-08-12 09:32:13 +00:00
Alexander Tokmakov
6f88065cc1
check metadata consistency
2022-07-29 18:33:16 +02:00
jianmei zhang
c1834d183b
ReplicatedMergeTree support lightweight delete
2022-07-26 11:50:09 +08:00
Vadim Volodin
85bf022258
review zookeeper and partition matching
2022-07-21 16:57:28 +03:00
Vitaly Baranov
7689e0c36f
Improve gathering metadata for backup - part 6.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
461a31f237
Improve gathering metadata for backup - part 2.
2022-06-30 08:37:17 +02:00
Vitaly Baranov
64b51a3772
Improve gathering metadata for backup.
2022-06-30 08:37:17 +02:00
Vadim Volodin
3296ba2532
Fix SYSTEM UNFREEZE for ordinary database
2022-06-24 20:46:16 +03:00
Nikolai Kochetov
7452d04e3a
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-20 15:31:02 +02:00
Vitaly Baranov
c2c35fad82
Refactoring of the code getting create table queries for backup.
2022-06-15 20:32:35 +02:00
Vitaly Baranov
592f568f83
Move backup/restore code to storages and databases - part 2.
2022-06-15 20:32:31 +02:00
Vitaly Baranov
724bc4dc57
Move backup/restore code to storages and databases - part 1.
2022-06-15 20:28:43 +02:00
Vitaly Baranov
73b1894a21
Rework collecting replicated parts.
2022-06-15 20:28:42 +02:00
Vitaly Baranov
5cabdbd982
Restore parts of MergeTree in correct order.
2022-06-15 20:28:40 +02:00
Nikolai Kochetov
2a9a63ac7b
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-15 15:35:26 +02:00
Vadim Volodin
637d293fbd
Add SYSTEM UNFREEZE query
2022-06-08 15:21:14 +03:00
Nikolai Kochetov
678d978acf
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-07 15:23:00 +00:00
Nikolai Kochetov
89c5855d20
Merge branch 'master' into refactor-something-in-part-volumes
2022-06-02 12:19:07 +02:00
Nikolai Kochetov
c71256ea38
Remove some commented code.
2022-05-30 13:18:20 +00:00
Nikolai Kochetov
1b85f2c1d6
Merge branch 'master' into refactor-read-metrics-and-callbacks
2022-05-25 16:27:40 +02:00
Nikolai Kochetov
fd97a9d885
Move some resources
2022-05-23 19:47:32 +00:00
Alexander Tokmakov
db66efb1a7
fix logical error on truncate table
2022-05-06 16:12:31 +02:00
Nikolai Kochetov
35095191eb
Merge branch 'master' into refactor-something-in-part-volumes
2022-05-03 17:51:47 +02:00
Robert Schulze
777b5bc15b
Don't let storages inherit from boost::noncopyable
...
... IStorage has deleted copy ctor / assignment already
2022-05-03 09:07:08 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
...
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
previously allowed.
Hence, this change
- removes shared_ptr_helper and as a result all inherited create() methods,
- instead, Storage objects are now created using make_shared<>() by the
caller (for that to work, many constructors had to be made public), and
- all Storage classes were marked as noncopyable using boost::noncopyable.
In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Nikolai Kochetov
e44af67fee
Merge branch 'master' into refactor-something-in-part-volumes
2022-04-26 21:08:00 +02:00
Vitaly Baranov
68a020ecea
Implement BACKUP/RESTORE ON CLUSTER.
2022-04-25 16:34:33 +02:00
Vitaly Baranov
acd28d8a1d
Implement RESTORE for replicated tables.
2022-04-25 16:34:33 +02:00
alesapin
ebdd53ef8f
Fix shared fetch
2022-04-22 19:18:18 +02:00
Nikolai Kochetov
8c00692844
Part 8
2022-04-22 16:58:09 +00:00
Nikolai Kochetov
9133e398b8
Part 7
2022-04-21 19:19:13 +00:00
alesapin
cbfe00bb47
Fix tidy
2022-04-21 00:30:13 +02:00
alesapin
c14e2e0b96
Fix more
2022-04-20 21:08:26 +02:00
alesapin
40c15222f8
Merge branch 'master' into fix_trash
2022-04-20 12:45:49 +02:00
tavplubix
2c54381520
Merge pull request #36200 from ClickHouse/fix_create_as_replicated_default_args
...
Fix CREATE AS for ReplicatedMergeTree with default arguments
2022-04-20 12:01:13 +03:00
alesapin
7cb7c120cc
Less ugly
2022-04-19 15:53:10 +02:00
alesapin
cc06bc3d99
Add some clarifications
2022-04-19 14:01:30 +02:00
alesapin
bd7b3847c1
Some code
2022-04-19 01:09:09 +02:00
alesapin
1706ae9e15
Some trash implementation
2022-04-15 18:36:23 +02:00
alesapin
5a8419a48e
Remove more trash
2022-04-15 17:05:17 +02:00
Alexander Tokmakov
40fea2b833
do not unfold {uuid} macro
2022-04-13 16:51:59 +02:00
Alexander Tokmakov
457a9e9691
fixes for ReplicatedMergeTree
2022-04-12 14:14:26 +02:00
Alexander Tokmakov
37a06eec1a
fixes
2022-04-05 17:36:53 +02:00
Alexander Tokmakov
7cba97aeab
Merge branch 'master' into resubmit_21474
2022-03-21 12:09:00 +01:00
alesapin
f2c5e2d3a0
Don't spam logs in zero copy replication
2022-03-19 17:31:33 +01:00