thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
99,731 Commits 9,315 Branches 1,783 Tags 2.1 GiB
56e5daba0c
Commit Graph

1187 Commits

Author SHA1 Message Date
Alexander Tokmakov
06f80770b8 fix stuck REPALCE_RANGE 2022-06-01 20:11:53 +02:00
HeenaBansal2009
b7eb6bbd38 Fixed clang-tidy-CheckTriviallyCopyableMove-errors 2022-05-30 11:09:03 -07:00
Nikolai Kochetov
c71256ea38 Remove some commented code. 2022-05-30 13:18:20 +00:00
Nikolai Kochetov
5b4658aa5e Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-30 09:47:35 +00:00
Alexander Tokmakov
562eec591e
Revert "Implemented changing comment to a ReplicatedMergeTree table" 2022-05-29 09:28:47 +03:00
alesapin
f63fa9bcc6
Merge pull request #37416 from Enmk/fix_ReplicatedMergeTree_comments 
Implemented changing comment to a ReplicatedMergeTree table
 2022-05-27 18:29:34 +02:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
Azat Khuzhin
dc9ca3d70c
Fix LOGICAL_ERROR in getMaxSourcePartsSizeForMerge during merges (#37413) 2022-05-26 14:14:58 +02:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
Vasily Nemkov
59b4d4a643 ALTER COMMENT is now local-only operation and immediately observable 2022-05-24 21:08:30 +03:00
Nikolai Kochetov
fd97a9d885 Move some resources 2022-05-23 19:47:32 +00:00
mergify[bot]
747aa5575c
Merge branch 'master' into remove-useless-code-2 2022-05-22 17:41:57 +00:00
Vasily Nemkov
c12f826d22 Implemented changing comment to a ReplicatedMergeTree table 2022-05-21 20:08:30 +03:00
alesapin
e99c90118e Merge branch 'master' into i_object_storage 2022-05-18 12:36:22 +02:00
Vitaly Baranov
72bff5242a
Merge pull request #37168 from vitlibar/backup-improvements-5 
Backups Improvements 5
 2022-05-16 21:25:54 +02:00
Vitaly Baranov
dfa1053b9f Use query scopes for async backup/restore. 2022-05-13 10:35:02 +02:00
alesapin
5dcd25be23 Initial implementation 2022-05-12 00:04:54 +02:00
Anton Popov
aec30c4076
Merge pull request #37053 from CurtizJ/remove-streams-comments 
Remove last mentions of data streams
 2022-05-10 13:38:13 +02:00
alesapin
41737d66b0
Merge pull request #37014 from ClickHouse/better_delete_outdated_parts 
Fix forgotten parts in cleanup thread
 2022-05-10 12:34:34 +02:00
Anton Popov
e911900054 remove last mentions of data streams 2022-05-09 19:15:24 +00:00
alesapin
f8f170462c Fix text 2022-05-09 15:24:50 +02:00
alesapin
684bfe35ee Merge branch 'better_delete_outdated_parts' of github.com:ClickHouse/ClickHouse into better_delete_outdated_parts 2022-05-09 15:23:05 +02:00
alesapin
018ed10684 Add test 2022-05-09 15:21:21 +02:00
alesapin
aad246f833
Update StorageReplicatedMergeTree.cpp 2022-05-08 16:40:41 +02:00
alesapin
d4e0127b0a Followup 2022-05-08 13:28:59 +02:00
alesapin
46712f1d98 Fix forgotten parts in cleanup thread 2022-05-08 00:53:55 +02:00
Alexander Tokmakov
db66efb1a7 fix logical error on truncate table 2022-05-06 16:12:31 +02:00
Alexey Milovidov
4c6f964421
Merge pull request #36909 from ClickHouse/fixes_for_replicated_merge_tree 
Some fixes for replicated merge tree
 2022-05-06 00:44:21 +03:00
Vitaly Baranov
dd51265218
Merge pull request #36864 from vitlibar/backup-improvements-4 
Backup improvements
 2022-05-05 15:37:51 +02:00
Alexander Tokmakov
dd2fe7f3b4 fix REPLACE_RANGE stuck in rare cases 2022-05-04 17:40:53 +02:00
Alexander Tokmakov
af9b4a5b9c fix intersecting parts 2022-05-04 16:22:06 +02:00
mergify[bot]
64084b5e32
Merge branch 'master' into shared_ptr_helper3 2022-05-03 20:46:16 +00:00
Nikolai Kochetov
fc3645a483 Fix some tests. 2022-05-03 19:32:24 +00:00
Nikolai Kochetov
49e613a666 Fixing build. 2022-05-03 17:58:47 +00:00
Nikolai Kochetov
35095191eb Merge branch 'master' into refactor-something-in-part-volumes 2022-05-03 17:51:47 +02:00
Vitaly Baranov
cb9d867f5f Fix restore coordination for creating tables in replicated databases. 2022-05-03 11:03:16 +02:00
Vitaly Baranov
409edfd3fa Rework RestoreCoordination: make restore deterministic. 2022-05-03 11:01:44 +02:00
Azat Khuzhin
da974585f1 Remove unimplemented RAID1 support 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-02 13:59:13 +03:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying 
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
 2022-05-02 08:46:52 +02:00
mergify[bot]
cc08ccb420
Merge branch 'master' into remove-useless-code-2 2022-04-30 12:48:15 +00:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Nikolai Kochetov
e44af67fee Merge branch 'master' into refactor-something-in-part-volumes 2022-04-26 21:08:00 +02:00
Vitaly Baranov
68a020ecea Implement BACKUP/RESTORE ON CLUSTER. 2022-04-25 16:34:33 +02:00
Vitaly Baranov
acd28d8a1d Implement RESTORE for replicated tables. 2022-04-25 16:34:33 +02:00
alesapin
2b04c19435 Satisfy clang-tidy 2022-04-23 11:58:56 +02:00
Nikolai Kochetov
e7e6125b46 More fixes 2022-04-22 20:38:18 +00:00
alesapin
ebdd53ef8f Fix shared fetch 2022-04-22 19:18:18 +02:00
Nikolai Kochetov
8c00692844 Part 8 2022-04-22 16:58:09 +00:00
alesapin
2559d62d74 Better logging and bugfixes 2022-04-22 15:35:25 +02:00
Nikolai Kochetov
9133e398b8 Part 7 2022-04-21 19:19:13 +00:00
alesapin
5465415751 Fix replace/move partition with zero copy replication 2022-04-21 14:39:12 +02:00
alesapin
c14e2e0b96 Fix more 2022-04-20 21:08:26 +02:00
alesapin
829854113b Add a little thread safety 2022-04-20 15:10:36 +02:00
alesapin
e8fba1644c Fix bug 2022-04-20 14:13:29 +02:00
alesapin
40c15222f8 Merge branch 'master' into fix_trash 2022-04-20 12:45:49 +02:00
Alexander Tokmakov
1d30a97fd2 Merge branch 'master' into remove-useless-code-2 2022-04-20 11:45:56 +02:00
tavplubix
2c54381520
Merge pull request #36200 from ClickHouse/fix_create_as_replicated_default_args 
Fix CREATE AS for ReplicatedMergeTree with default arguments
 2022-04-20 12:01:13 +03:00
alesapin
7919485db2 Fix 2022-04-19 15:57:25 +02:00
alesapin
7cb7c120cc Less ugly 2022-04-19 15:53:10 +02:00
alesapin
cc06bc3d99 Add some clarifications 2022-04-19 14:01:30 +02:00
alesapin
c81cb9e563 Better 2022-04-19 01:12:07 +02:00
alesapin
bd7b3847c1 Some code 2022-04-19 01:09:09 +02:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains" 
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
 2022-04-18 23:53:11 +02:00
alesapin
1706ae9e15 Some trash implementation 2022-04-15 18:36:23 +02:00
alesapin
5a8419a48e Remove more trash 2022-04-15 17:05:17 +02:00
alesapin
eb7593f786 Remove more trash 2022-04-15 16:24:38 +02:00
tavplubix
fd23c42ea7
Update StorageReplicatedMergeTree.cpp 2022-04-13 22:33:55 +03:00
Alexander Tokmakov
66fdf35dfd remove outdated parts immediately on drop partition 2022-04-13 18:01:22 +02:00
Alexander Tokmakov
40fea2b833 do not unfold {uuid} macro 2022-04-13 16:51:59 +02:00
Alexander Tokmakov
457a9e9691 fixes for ReplicatedMergeTree 2022-04-12 14:14:26 +02:00
Alexey Milovidov
a54c01cf72 Remove useless code in ReplicatedMergeTreeRestartingThread 2022-04-11 00:44:30 +02:00
Alexander Tokmakov
7f54e7b422 Merge branch 'master' into mvcc_prototype 2022-04-07 15:14:06 +02:00
Alexander Tokmakov
1309e781b6 apply suggestion 2022-04-06 13:56:26 +02:00
Alexander Tokmakov
37a06eec1a fixes 2022-04-05 17:36:53 +02:00
Alexander Tokmakov
d9e3e9b69f Merge branch 'master' into faster_replicated_ddl 2022-04-05 14:38:38 +02:00
Alexander Tokmakov
1fe50ad201 Merge branch 'master' into mvcc_prototype 2022-04-05 14:38:02 +02:00
alesapin
eaa72ef002
Merge pull request #35904 from ClickHouse/more_logs_on_unsuccessful_part_removal 
More logs on unsuccessful part removal
 2022-04-05 11:29:08 +02:00
Alexander Tokmakov
4e9ec5dc2f make some replicated DDL faster 2022-04-05 00:51:48 +02:00
alesapin
885447342c More logs on unsuccessful part removal 2022-04-04 13:17:33 +02:00
jewisliu
9a4cc78dfb support ALTER TABLE t DETACH PARTITION ALL syntax 2022-04-01 13:13:07 +08:00
Alexander Tokmakov
208b242188 Merge branch 'master' into mvcc_prototype 2022-03-28 19:58:06 +02:00
Alexander Tokmakov
9ec0951de5 update comment for mismatching checksums 2022-03-25 15:54:05 +01:00
Alexander Tokmakov
3c762f566d Merge branch 'master' into mvcc_prototype 2022-03-21 20:16:29 +01:00
Alexander Tokmakov
7cba97aeab Merge branch 'master' into resubmit_21474 2022-03-21 12:09:00 +01:00
Alexander Tokmakov
9e05b12d2c Merge branch 'master' into mvcc_prototype 2022-03-20 22:42:26 +01:00
alesapin
f2c5e2d3a0 Don't spam logs in zero copy replication 2022-03-19 17:31:33 +01:00
Alexander Tokmakov
c28008a2df more comments, minor fixes 2022-03-18 12:01:26 +01:00
Alexander Tokmakov
d04dc03fa4 Merge branch 'master' into mvcc_prototype 2022-03-17 15:24:32 +01:00
alesapin
92ab33f560
Merge pull request #35348 from ClickHouse/fix_yet_another_zero_copy_bug 
Fix race in zero copy replication
 2022-03-17 11:34:38 +01:00
alesapin
03ff6d52f6 Fix race in zero copy replication 2022-03-16 21:51:33 +01:00
Alexander Tokmakov
c2ac8d4a5c review fixes 2022-03-16 21:05:34 +01:00
Alexander Tokmakov
9702b5177d Merge branch 'master' into mvcc_prototype 2022-03-14 21:45:38 +01:00
Anton Popov
36ec379aeb Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-14 16:28:35 +00:00
zhanghuajie
53a8987b3b fix build fail with gcc --fix warnings without disabling some parameters 2022-03-11 21:59:19 +08:00
Alexander Tokmakov
061fa6a6f2 Merge branch 'master' into mvcc_prototype 2022-03-10 13:13:04 +01:00
Azat Khuzhin
caffc144b5 Fix possible "Part directory doesn't exist" during INSERT 
In #33291 final part commit had been defered, and now it can take
significantly more time, that may lead to "Part directory doesn't exist"
error during INSERT:

    2022.02.21 18:18:06.979881 [ 11329 ] {insert} <Debug> executeQuery: (from 127.1:24572, user: default) INSERT INTO db.table (...) VALUES
    2022.02.21 20:58:03.933593 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18044_18044_0 to 20220214_270654_270654_0.
    2022.02.21 21:16:50.961917 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18197_18197_0 to 20220214_270689_270689_0.
    ...
    2022.02.22 21:16:57.632221 [ 64878 ] {} <Warning> db.table: Removing temporary directory /clickhouse/data/db/table/tmp_insert_20220214_18232_18232_0/
    ...
    2022.02.23 12:23:56.277480 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18232_18232_0 to 20220214_273459_273459_0.
    2022.02.23 12:23:56.299218 [ 11329 ] {insert} <Error> executeQuery: Code: 107. DB::Exception: Part directory /clickhouse/data/db/table/tmp_insert_20220214_18232_18232_0/ doesn't exist. Most likely it is a logical error. (FILE_DOESNT_EXIST) (version 22.2.1.1) (from 127.1:24572) (in query: INSERT INTO db.table (...) VALUES), Stack trace (when copying this message, always include the lines below):

Follow-up for: #28760
Refs: #33291

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-08 07:44:11 +03:00
Anton Popov
fcdebea925 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-25 13:41:30 +03:00
Alexander Tokmakov
aa6b9a2abc Merge branch 'master' into mvcc_prototype 2022-02-23 23:22:03 +03:00
Alexander Tokmakov
dae044f86b Merge branch 'master' into mvcc_prototype 2022-02-17 13:49:37 +03:00
Jianmei Zhang
ef0c3b99ff Merge remote-tracking branch 'upstream/master' 2022-02-17 14:02:27 +08:00
Jianmei Zhang
25c761b3b6 Early break for multiple leaders case when log updated by other leader 2022-02-16 16:06:41 +08:00
Alexander Tokmakov
e37ef4560c fix 2022-02-15 18:00:45 +03:00
alesapin
447cd56cb9 Fix comments 2022-02-15 12:11:50 +03:00
Alexander Tokmakov
ae5aa8c12d write part version before other files 2022-02-15 02:24:51 +03:00
alesapin
b75d551281 Fix clang tidy and add check for master 2022-02-14 14:37:41 +03:00
alesapin
b2886a429b Fix lock during fetch 2022-02-14 12:20:27 +03:00
alesapin
beb4400978 Fix 'same local part' check 2022-02-13 23:08:29 +03:00
Alexander Tokmakov
07e66e690d Merge branch 'master' into mvcc_prototype 2022-02-11 15:53:32 +03:00
alesapin
3af06b23f8 POC 2022-02-10 22:45:52 +03:00
Anton Popov
18940b8637 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-09 23:38:38 +03:00
alesapin
02a93cb852 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-08 19:42:27 +03:00
alesapin
8e9ccbd077
Merge pull request #33933 from sunny19930321/fix-substr-zk-metadata 
Better local metadata comparison with ZooKeeper metadata
 2022-02-08 14:58:46 +03:00
alesapin
3af6012cb4 Revert "Revert "Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer"""" 
This reverts commit 2bc2ea485e.
 2022-02-08 11:01:26 +03:00
alesapin
2bc2ea485e Revert "Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer""" 
This reverts commit fb77d7a7d5.
 2022-02-08 10:56:29 +03:00
alesapin
fb77d7a7d5 Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer"" 
This reverts commit 875e5413ad.
 2022-02-07 12:36:54 +03:00
alesapin
bb361e3b01 Fix bug with existing part 2022-02-07 12:33:37 +03:00
Alexander Tokmakov
45be75b4db Merge branch 'master' into mvcc_prototype 2022-02-06 23:36:08 +03:00
alesapin
c028269e6f Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-04 18:02:25 +03:00
tavplubix
22de534fdc
Merge pull request #34297 from ClickHouse/fix_restarting_thread 
Try to fix race between pullLogsToQueue and RestartingThread
 2022-02-04 17:30:17 +03:00
alesapin
875e5413ad Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer" 
This reverts commit b92efed350, reversing
changes made to ecce006cb2.
 2022-02-04 14:30:33 +03:00
alesapin
2ed45b2a98 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-04 11:23:46 +03:00
Alexander Tokmakov
897e94c16c make restarting thread less bad 2022-02-03 23:29:24 +03:00
Alexander Tokmakov
ca5f951558 Merge branch 'master' into mvcc_prototype 2022-02-03 18:56:44 +03:00
alesapin
25375bc76b Remove unused param 2022-02-03 11:21:19 +03:00
alesapin
80800e051e Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-02 19:42:04 +03:00
Nikolai Kochetov
ea044fc6b2 Merge branch 'master' into revert-34212-revert-33291-add-pool-to-s3-write-buffer 2022-02-02 19:40:41 +03:00
alesapin
b9c118524f Fix race condition on hardlink/erase/read metadata 2022-02-02 19:40:21 +03:00
mergify[bot]
d8bea598b2
Merge branch 'master' into fix-substr-zk-metadata 2022-02-01 14:10:12 +00:00
Anton Popov
836a348a9c Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-01 15:23:07 +03:00
Alexander Tokmakov
d93c9f0be4 fix build after merge 2022-02-01 15:05:05 +03:00
alesapin
e87ed0ff02 Fix in-memory parts 2022-02-01 14:37:12 +03:00
Azat Khuzhin
5be76bc969 Use proper fmt-like logging 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:04 +03:00
Azat Khuzhin
bedf208cbd Use fmt::runtime() for LOG_* for non constexpr 
Here is oneliner:

    $ gg 'LOG_\(DEBUG\|TRACE\|INFO\|TEST\|WARNING\|ERROR\|FATAL\)([^,]*, [a-zA-Z]' -- :*.cpp :*.h | cut -d: -f1 | sort -u | xargs -r sed -E -i 's#(LOG_[A-Z]*)\(([^,]*), ([A-Za-z][^,)]*)#\1(\2, fmt::runtime(\3)#'

Note, that I tried to do this with coccinelle (tool for semantic
patchin), but it cannot parse C++:

    $ cat fmt.cocci
    @@
    expression log;
    expression var;
    @@

    -LOG_DEBUG(log, var)
    +LOG_DEBUG(log, fmt::runtime(var))

I've also tried to use some macros/templates magic to do this implicitly
in logger_useful.h, but I failed to do so, and apparently it is not
possible for now.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

v2: manual fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:03 +03:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3"" 2022-02-01 13:36:51 +03:00
Alexander Tokmakov
2e4ae37d98 Merge branch 'master' into mvcc_prototype 2022-02-01 13:20:03 +03:00
alexey-milovidov
99392b5ca7
Merge pull request #13544 from amosbird/mdha 
Multi-Disk auto-recovery.
 2022-02-01 06:13:26 +03:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3" 2022-02-01 05:49:40 +03:00
Amos Bird
ec7d367814
DiskLocal checker 
Add DiskLocal checker so that ReplicatedMergeTree can recover data when some of its disks are broken.
 2022-02-01 05:55:27 +08:00
Nikolai Kochetov
321fa4a9e8
Merge pull request #33291 from ClickHouse/add-pool-to-s3-write-buffer 
Add pool to WriteBufferFromS3
 2022-01-31 19:37:40 +03:00
Maksim Kita
8513f20cfd
Merge pull request #34145 from kitaisreal/bitset-sort-performance-check 
pdqsort performance check
 2022-01-31 12:35:13 +01:00
tavplubix
d19e24f530
Merge pull request #34096 from ClickHouse/fix_race_merge_selecting_task 
Fix race between mergeSelectingTask and queue reinitialization
 2022-01-31 12:16:29 +03:00
Maksim Kita
5ef83deaa6 Update sort to pdqsort 2022-01-30 19:49:48 +00:00
Anton Popov
78b9f15abb Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-30 03:24:37 +03:00
Alexander Tokmakov
fb9b2d5326 Merge branch 'master' into mvcc_prototype 2022-01-28 21:18:36 +03:00
Alexander Tokmakov
e0304c2a58 review fixes, write tid into mutation entry 2022-01-28 20:47:37 +03:00
Nikolai Kochetov
1c9f026178 Merge branch 'master' into add-pool-to-s3-write-buffer 2022-01-28 16:01:42 +03:00
Alexander Tokmakov
b3ddc601a5 fix race between mergeSelectingTask and queue reinitialization 2022-01-28 15:50:58 +03:00
liyang830
c2bdeb349c Merge branch 'fix-substr-zk-metadata' of github.com:sunny19930321/ClickHouse into fix-substr-zk-metadata 2022-01-27 16:35:02 +08:00
liyang830
eca0453564 fix local metadata differ zk metadata 2022-01-27 16:33:40 +08:00
Nikolai Kochetov
9b2998c639 Review fixes. 2022-01-26 18:08:01 +00:00
Nikolai Kochetov
506ee8c024 Refactor some code. 2022-01-24 15:55:29 +00:00
tavplubix
da9a38655b
Merge pull request #33847 from ClickHouse/fix_33806 
Fix logical error on RESTORE REPLICA
 2022-01-24 13:28:37 +03:00
tavplubix
34baca1298
Update src/Storages/StorageReplicatedMergeTree.cpp 
Co-authored-by: Bharat Nallan <bharatnc@gmail.com>
 2022-01-24 12:12:08 +03:00
tavplubix
2f566d2af9
Update src/Storages/StorageReplicatedMergeTree.cpp 
Co-authored-by: Bharat Nallan <bharatnc@gmail.com>
 2022-01-24 12:12:03 +03:00
Nikolai Kochetov
b3cbb63487 Merge branch 'master' into add-pool-to-s3-write-buffer 2022-01-21 21:41:54 +03:00
Anton Popov
e8ce091e68 Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-21 20:11:18 +03:00
Alexander Tokmakov
e3b1397001 Merge branch 'master' into mvcc_prototype 2022-01-21 18:59:01 +03:00
Alexander Tokmakov
8921725d71 fix logical error on RESTORE REPLICA 2022-01-21 14:32:19 +03:00
Anton Popov
6c4c9bc04a
Merge pull request #33234 from CurtizJ/fix-flush-in-memory-parts 
Fix flushing of in-memory parts
 2022-01-21 02:18:03 +03:00
Alexander Tokmakov
e9a5a64a71 Merge branch 'master' into mvcc_prototype 2022-01-19 21:41:23 +03:00
alesapin
e5ec1aa446
Merge pull request #33642 from ClickHouse/add_integration_test_for_zero_duplication 
Fix bug with background moves in zero copy replication
 2022-01-18 18:32:16 +03:00
alesapin
eff85d3352 Review fixes 2022-01-18 11:27:01 +03:00
alesapin
4caef03e6a Add ephemeral lock in zookeeper 2022-01-17 14:52:51 +03:00
Nikolai Kochetov
6d49a62666 Some more async writes. 2022-01-14 19:53:55 +00:00
alesapin
bf803472b1 Add a tests for a TTL bug in zero copy replication 2022-01-14 18:44:10 +03:00
Nikolai Kochetov
28f2012b06 Add some more async writing. 2022-01-11 19:02:48 +00:00
Azat Khuzhin
aee034a597 Use explicit template instantiation for SystemLog 
- Move some code into module part to avoid dependency from IStorage in SystemLog
- Remove extra headers from SystemLog.h
- Rewrite some code that was relying on headers that was included by SystemLog.h

v2: rebase
v3: squash move into module part with explicit template instantiation
    (to make each commit self compilable after rebase)
 2022-01-10 22:01:41 +03:00
Alexander Tokmakov
bf2b6c2c37 Merge branch 'master' into mvcc_prototype 2022-01-10 19:53:50 +03:00
mergify[bot]
6fcd5a7d41
Merge branch 'master' into MDB-15474 2022-01-07 10:12:25 +00:00
Alexander Tokmakov
187c43eba8 rename Committed state to Active 2021-12-30 23:45:38 +03:00
alesapin
91e1ac437e Tiny improvements 2021-12-30 12:57:38 +03:00
alesapin
cbdba89d65 Merge branch 'master' into ianton-ru-MDB-15474 2021-12-30 11:21:19 +03:00
avogar
8112a71233 Implement schema inference for most input formats 2021-12-29 12:18:56 +03:00
Anton Popov
4ebf61b809 proper checking of atomic flags 2021-12-29 01:03:55 +03:00
Anton Ivashkin
2d87f0a0e3 Fix debug build 2021-12-28 11:45:38 +03:00
Anton Ivashkin
0465aef15d Fixes by code review responces 2021-12-27 19:27:06 +03:00
Anton Popov
a83832c03e fix flushing of in-memory parts 2021-12-27 18:54:28 +03:00
ianton-ru
e6fd4bfb50
Merge branch 'master' into MDB-15474 2021-12-21 17:38:36 +03:00
Anton Ivashkin
33cbfc89ef Move logic for replicated part to StorageReplicatedMergeTree class 2021-12-21 17:29:50 +03:00
Anton Ivashkin
0c0bf66334 Merge master 2021-12-21 17:27:54 +03:00
Alexander Tokmakov
9cd49bc0ec Merge branch 'master' into mvcc_prototype 2021-12-20 22:06:22 +03:00
Anton Ivashkin
c724b074ae Remove zero-copy version converter 2021-12-20 20:23:25 +03:00
Anton Ivashkin
f0b9a4327a Use table UUID in zero-copy shared label in ZooKeeper 2021-12-20 16:19:00 +03:00
Anton Popov
99ebabd822 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-17 19:02:29 +03:00
alesapin
6bd7e425c6
Merge pull request #22535 from CurtizJ/sparse-serialization 
Sparse serialization and ColumnSparse
 2021-12-17 15:26:17 +03:00
Alexander Tokmakov
e185ad260b fix skipping of some mutations 2021-12-15 21:28:24 +03:00
Alexander Tokmakov
1b02136941 fix skipping of some mutations 2021-12-15 21:19:29 +03:00
Alexander Tokmakov
d7ad72838c Merge branch 'master' into mvcc_prototype 2021-12-14 23:07:52 +03:00
Anton Popov
16312e7e4a Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-14 18:58:17 +03:00
tavplubix
25427719d4
Try fix 'Directory tmp_merge_<part_name>' already exists (#32201) 
* try fix 'directory tmp_merge_<part_name>' already exists

* fix

* fix

* fix

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-12-10 16:29:51 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
Anton Popov
d8367334a3 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-08 18:26:19 +03:00
tavplubix
785d79531d
Update StorageReplicatedMergeTree.cpp 2021-12-08 15:27:32 +03:00
tavplubix
4f46ac6b30
Remove LeaderElection (#32140) 
* remove LeaderElection

* try fix tests

* Update test.py

* Update test.py
 2021-12-07 19:55:55 +03:00
Alexander Tokmakov
7fcb79ae72 Merge branch 'master' into mvcc_prototype 2021-12-07 14:39:29 +03:00
tavplubix
e9489ccd6f
Update StorageReplicatedMergeTree.cpp 2021-12-02 13:18:02 +03:00
Anton Ivashkin
e0a16a483e Merge master 2021-12-01 19:03:31 +03:00
Anton Ivashkin
80ab73c691 Fix Zero-Copy replication lost locks, fix remove used remote data in DROP DETACHED PART 2021-12-01 16:11:26 +03:00
Anton Popov
6f4d9a53b2 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-12-01 15:54:33 +03:00
Anton Popov
54f51444c0 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-01 15:49:02 +03:00
tavplubix
b623a387af
Merge pull request #31887 from ClickHouse/fix_cannot_create_empty_part 
Parse partition key value from `partition_id` when need to create part in empty partition
 2021-12-01 15:38:46 +03:00
tavplubix
78224ef273
Merge pull request #31738 from ClickHouse/fix_segfault_on_attach_partition 
Fix segfault on attach parititon
 2021-11-30 14:32:50 +03:00
Anton Ivashkin
0f9038ebed Zero-copy: move shared mark outside table node in ZooKeeper 2021-11-29 19:05:31 +03:00
Alexander Tokmakov
2fb00172a9 try parse partition key value from partition_id 2021-11-27 15:07:08 +03:00
Raúl Marín
b2cfa70541 Reduce dependencies on ASTFunction.h 
481 -> 230
 2021-11-26 18:21:54 +01:00
Alexander Tokmakov
1ab8b5d4df fix segfault on attach parititon 2021-11-25 13:04:39 +03:00
Anton Popov
ccd78e3838 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-22 17:19:35 +03:00
Vitaly Baranov
6634fcbac7 Rename Quota::ResourceType -> QuotaType and move it to Access/Common. 2021-11-19 00:14:23 +03:00
Alexander Tokmakov
0a4647f927 support alter partition 2021-11-17 21:14:14 +03:00
Alexander Tokmakov
ce2f692bb4 Merge branch 'master' into mvcc_prototype 2021-11-15 14:56:50 +03:00
Kseniia Sumarokova
2313981fd7
Merge pull request #31260 from azat/external-cleanup 
Cleanup extern ProfileEvents/CurrentMetrics and add a style check
 2021-11-12 00:02:57 +03:00
Alexander Tokmakov
672157b817 Merge branch 'master' into mvcc_prototype 2021-11-11 22:01:34 +03:00
tavplubix
de6a6f3227
Merge pull request #31180 from ClickHouse/merge_selecting_backoff 
Slightly better backoff for merge selecting task
 2021-11-11 20:57:40 +03:00
Azat Khuzhin
baf14444e6 Cleanup ProfileEvents and CurrentMetrics 2021-11-10 21:15:27 +03:00
Anton Popov
a20922b2d3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-11-09 15:36:25 +03:00
Alexander Tokmakov
51bd98294e better merge selecting backoff 2021-11-09 15:26:51 +03:00
Amos Bird
33dfed584d
Fix exception message 2021-11-09 18:19:00 +08:00
Amos Bird
94fe925773
Better 2021-11-09 18:02:17 +08:00
Anton Popov
66973a2a28 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-08 21:27:45 +03:00
Alexander Tokmakov
92eec74ad7 Merge branch 'master' into mvcc_prototype 2021-11-06 21:08:36 +03:00
Amos Bird
710f7d64e4
Add retry to check alter_partition_version 2021-11-03 15:48:16 +08:00
Anton Popov
d50137013c Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-01 16:55:53 +03:00
Azat Khuzhin
60a411581f Fix possible "The local set of parts of X doesn't look like the set of parts in ZooKeeper" error 
If during removing replica_path from zookeeper, some error occurred
(zookeeper goes away), then it may not remove everything from zookeeper.

And on DETACH/ATTACH (or server restart, like stress tests does in the
analysis from this comment [1]), it will trigger an error:

    The local set of parts of table test_1.alter_table_4 doesn't look like the set of parts in ZooKeeper:

  [1]: https://github.com/ClickHouse/ClickHouse/pull/28296#issuecomment-915829943

Fix this, by removing "metadata" at first, and only after this
everything else, this will avoid this error, since on ATTACH such table
will be marked as read-only.

v2: forget to remove remote_replica_path itself
v3: fix test_drop_replica by adding a check for remote_replica_path existence
 2021-11-01 08:59:28 +03:00
Azat Khuzhin
ca22e7acfb Use existing local remote_replica_path var in StorageReplicatedMergeTree::dropReplica() 2021-11-01 08:53:31 +03:00
Anton Popov
0099dfd523 refactoring of SerializationInfo 2021-10-29 20:21:02 +03:00
tavplubix
f50b2b651b
Merge pull request #30822 from ClickHouse/fix_29052 
Fix ambiguity when extracting auxiliary ZooKeeper name
 2021-10-29 12:15:29 +03:00
tavplubix
bb4a8e91f7
Create StorageReplicatedMergeTree.cpp 2021-10-28 20:25:31 +03:00
Alexander Tokmakov
c1310841d9 fix ambiguity when extracting auxiliary zk name 2021-10-28 19:19:41 +03:00
tavplubix
33ffe11d03
Merge pull request #30346 from ClickHouse/fix_some_fetches_may_stuck 
Try fix 'some fetches may stuck'
 2021-10-28 13:10:15 +03:00
Alexander Tokmakov
5e622d4c98 fix another issue 2021-10-27 15:59:26 +03:00
alesapin
2af950d4d5
Merge pull request #30658 from ClickHouse/std_mutex_alter_lock 
Change `alter_lock` from `RWLock` to `std::timed_mutex`
 2021-10-27 11:14:11 +03:00
Alexander Tokmakov
6be1e98342 fix 2021-10-26 17:06:48 +03:00
mergify[bot]
ce88a84e88
Merge branch 'master' into nv/parts-uuid-move-shard-kill 2021-10-26 11:09:19 +00:00
Alexander Tokmakov
2e7e195e77 change alter_lock to std::timed_mutex 2021-10-26 13:37:00 +03:00
Alexander Tokmakov
94088f8197 minor fixes 2021-10-25 17:01:36 +03:00
Alexander Tokmakov
075c3a37f4 Merge branch 'master' into fix_some_fetches_may_stuck 2021-10-25 15:28:54 +03:00
Anton Popov
b1a74af44d
Merge pull request #30180 from CurtizJ/remove-sample-by 
Allow to remove `SAMPLE BY` expression
 2021-10-21 17:09:53 +03:00
tavplubix
5e24f337a0
Update StorageReplicatedMergeTree.cpp 2021-10-20 09:29:41 +03:00
Alexander Tokmakov
caf972db48 fix 2021-10-19 14:56:04 +03:00
Alexander Tokmakov
89fe606d15 try fix 'some fetches may stuck' 2021-10-18 23:16:02 +03:00
Anton Popov
d71ffc355a Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-18 15:18:22 +03:00
alexey-milovidov
4f11cfa59d
Merge pull request #28760 from azat/mutator-forbid-cleaner 
Forbid cleaning of tmp directories that can be used by an active mutation/merge.
 2021-10-17 03:18:01 +03:00
Azat Khuzhin
07e8b2b3c7 Do not try to remove temporary paths that is currently in written by merge/mutation 
v2: rebase against MergeTask
v3: rebase due to conflicts in src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp
v4:
- rebase due to conflicts in src/Storages/MergeTree/MergeTask.cpp
- drop common/scope_guard_safe.h (not used)
 2021-10-16 00:43:52 +03:00
Anton Popov
7aa6068fb2 Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-14 19:44:08 +03:00
Anton Popov
476d7a411f allow to remove SAMPLE BY expression 2021-10-14 17:05:20 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Nikolai Kochetov
340b53ef85 Remove some more streams. 2021-10-08 17:03:54 +03:00
Nikolai Kochetov
9eddee5517
Merge pull request #29768 from ClickHouse/remove-merging-streams 
Remove some merging streams
 2021-10-08 13:15:10 +03:00
Alexander Tokmakov
4119848a41 better diagnostic for OPTIMIZE 2021-10-07 13:08:10 +03:00
mergify[bot]
ab55225c8f
Merge branch 'master' into remove-merging-streams 2021-10-05 14:16:34 +00:00