thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
109,109 Commits 9,326 Branches 1,783 Tags 2.1 GiB
2d8060d843
Commit Graph

5370 Commits

Author SHA1 Message Date
Alexander Gololobov
249e670c9f Properly preserve columns that are required by next steps 2023-02-12 21:54:23 +01:00
Alexander Gololobov
45ef2f6d60 Split prewhere actions into separate conjuctive steps 2023-02-12 21:54:23 +01:00
alesapin
1683cd5d6f
Merge branch 'master' into mutations_rename_hang 2023-02-12 13:06:48 +01:00
Alexey Milovidov
b8cbc947ce Remove mutex from replicas_status 2023-02-12 07:39:16 +01:00
Alexey Milovidov
5f00bc49f8
Merge branch 'master' into addParts 2023-02-12 05:55:18 +03:00
Alexey Milovidov
a97f9b2caf
Update MergeTreeData.cpp 2023-02-12 05:55:07 +03:00
Alexey Milovidov
395d6b6bd5
Merge pull request #46282 from CurtizJ/return-back-vertical-merges 
Allow vertical merges from compact to wide parts [2]
 2023-02-12 05:51:12 +03:00
serxa
df6b4c4372 fix typo 2023-02-11 20:18:19 +00:00
serxa
9a7f70f595 fix comment 2023-02-11 16:54:57 +00:00
serxa
eb8cf6f816 Merge branch 'round-robin-merge-scheduler' of github.com:ClickHouse/ClickHouse into round-robin-merge-scheduler 2023-02-11 16:45:55 +00:00
serxa
d2f8377c47 fix check 2023-02-11 16:45:45 +00:00
Sergei Trifonov
297587531b
Merge branch 'master' into round-robin-merge-scheduler 2023-02-11 17:19:27 +01:00
serxa
c58b165b0f add config option to select scheduling policy 2023-02-11 16:18:42 +00:00
alesapin
5c83ddabf9 Merge branch 'master' into mutations_rename_hang 2023-02-11 12:18:58 +01:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Sergei Trifonov
fbedf4d520
Merge branch 'master' into round-robin-merge-scheduler 2023-02-11 10:30:26 +01:00
kssenii
3545568440 Fix build 2023-02-10 22:15:44 +01:00
Kseniia Sumarokova
3be36c7979
Merge pull request #41976 from kssenii/allow-single-disk-instead-of-storage-policy 
Allow configuring storage as `SETTINGS disk='<disk_name>'` (instead of `storage_policy`) and explicit disk creation `SETTINGS disk=disk(<disk_configuration>)`
 2023-02-10 21:38:50 +01:00
Anton Popov
1205932a19 add setting to disable vertical merges from compact parts 2023-02-10 16:33:46 +00:00
kssenii
468d5216da Remove logging because of test 00002_log_and_exception_messages_formatting 2023-02-10 17:25:26 +01:00
Anton Popov
a252d3a139 Revert "Merge pull request #46236 from ClickHouse/revert-45681-compact-parts-vertical-merge" 
This reverts commit 27bc317ef9, reversing
changes made to ccf7b6f4c1.
 2023-02-10 14:13:29 +00:00
Anton Popov
27bc317ef9
Merge pull request #46236 from ClickHouse/revert-45681-compact-parts-vertical-merge 
Revert "Allow vertical merges from compact to wide parts"
 2023-02-10 14:31:33 +01:00
alesapin
808a939ad2 Small rehacktoing 2023-02-10 14:25:19 +01:00
KitKatKKK
d31457de44
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-02-10 21:14:42 +08:00
KitKatKKK
7664114349
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-02-10 21:14:33 +08:00
kssenii
571aabdb69 Fix clang-tidy 2023-02-10 12:39:05 +01:00
Nikolai Kochetov
8c22c740a4
Merge pull request #46205 from ClickHouse/fix-write-buffer-destruction-order-for-vertical-merge 
Fix write buffer destruction order for vertical merge.
 2023-02-10 11:47:23 +01:00
KitKatKKK
ea300ce13b
Merge branch 'master' into addParts 2023-02-10 16:48:04 +08:00
KitKatKKK
fa73b00a66
Update src/Storages/MergeTree/MergeTreeData.h 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-02-10 16:46:57 +08:00
attack204
7f41173a1f rename API 2023-02-10 16:24:10 +08:00
attack204
93e92dab3b rename API 2023-02-10 16:05:50 +08:00
attack204
52960d4e7d add API to MergeTreeData 2023-02-10 11:00:11 +08:00
Nikolai Kochetov
4aa6e4d683 Revert "Add return flag for preFinalize." 
This reverts commit 3261378ef8.
 2023-02-09 19:51:09 +00:00
Nikolai Kochetov
3261378ef8 Add return flag for preFinalize. 2023-02-09 19:50:54 +00:00
Nikolai Kochetov
98765cc20b Disable delayed streams for Vertical merges for non-remote disks. 2023-02-09 19:39:34 +00:00
serxa
bf299b6696 Merge branch 'round-robin-merge-scheduler' of github.com:ClickHouse/ClickHouse into round-robin-merge-scheduler 2023-02-09 19:33:17 +00:00
serxa
cc3c76944b remove redundant code 2023-02-09 19:31:54 +00:00
Sergei Trifonov
e3727d9b0e
Merge branch 'master' into round-robin-merge-scheduler 2023-02-09 20:04:59 +01:00
serxa
f61f95fcec Replace background executor scheduler merges and mutations with round-robin 2023-02-09 18:54:15 +00:00
Antonio Andelic
f96d480563
Merge branch 'master' into add-support-for-settings-alias 2023-02-09 16:07:45 +01:00
Anton Popov
612fe0bd00
Revert "Allow vertical merges from compact to wide parts" 2023-02-09 15:42:46 +01:00
alesapin
9b23068d5e
Merge pull request #45821 from ClickHouse/explicit_drop_part_entry_type 
Make separate DROP_PART log entry type
 2023-02-09 15:11:13 +01:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy 2023-02-09 13:16:55 +01:00
kssenii
2a48a96f83 Fix tests 2023-02-09 12:26:15 +01:00
Nikolai Kochetov
65ec9bd7d0 Fix write buffer destruction order for vertical merge. 2023-02-09 10:58:49 +00:00
alesapin
859f528fe1
Merge pull request #45648 from ClickHouse/45508_Update_strategy_for_system_sync_replica 
Updated checking of SYSTEM SYNC REPLICA
 2023-02-09 11:51:05 +01:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
avogar
c3e8dd8984 Fix low cardinality case 2023-02-08 19:14:28 +00:00
alesapin
a28d10a810 Fxi 2023-02-08 14:50:30 +01:00
Smita Kulkarni
87ec32f235 Updated to use znode_name instead of log_entry_id - Updated checking of SYSTEM SYNC REPLICA 2023-02-08 14:07:06 +01:00
Azat Khuzhin
151ba92a2e Fix reading of non existing nested columns with multiple level in compact parts 
Consider the following example:

    CREATE TABLE data (root.array_str Array(UInt8)) ENGINE = MergeTree() ORDER BY tuple();
    INSERT INTO data VALUES ([]);
    ALTER TABLE data ADD COLUMN root.nested_array Array(Array(UInt8));

In this case the first part will not have data for root.nested_array,
and thanks to #37152 it will simply read offsets column from
root.array_str, however since root.nested_array is a nested array, it
will try to read elements from the same offsets stream and if you are
lucky enough you will get one of the following errors:

- Cannot read all data. Bytes read: 1. Bytes expected: 8.: (while reading column root.nested_array): While executing MergeTreeInOrder. (CANNOT_READ_ALL_DATA)
- DB::Exception: Array size is too large: 8233460228287709730: (while reading column serp.serp_features): While executing MergeTreeInOrder.

So to address this, findColumnForOffsets() had been changed to return
the level of the column too, to allow to read only up to this level.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-08 10:21:40 +01:00
alesapin
bbf044f477 Fxi 2023-02-07 22:30:38 +01:00
Smita Kulkarni
8c9d994fd7 Extended scope of state_mutex while adding subscriber - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 22:04:17 +01:00
alesapin
bc58fd156b Merge remote-tracking branch 'origin' into mutations_rename_hang 2023-02-07 20:53:04 +01:00
alesapin
36f41da6e0 One more bugfix 2023-02-07 18:53:47 +01:00
Smita Kulkarni
ecea28a832 Fixed style check - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 17:27:34 +01:00
Nikolai Kochetov
3912f5a333
Merge pull request #45681 from CurtizJ/compact-parts-vertical-merge 
Allow vertical merges from compact to wide parts
 2023-02-07 17:09:17 +01:00
Smita Kulkarni
34341ddabe Updated locks and removed getLogEntryIds function - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 16:57:58 +01:00
Sema Checherinda
52d4e78b7f
Merge pull request #45907 from CheSema/forbid-wal-on-s3 
do not run wal on remote disks
 2023-02-07 14:48:00 +01:00
Yakov Olkhovskiy
4e7dffafe4
Merge pull request #46087 from ClickHouse/fix-ip-bloom 
Allow bloom filter for IPv4 and IPv6
 2023-02-07 08:29:28 -05:00
alesapin
1fd9baa3f3 Fix fetch: 2023-02-07 12:49:51 +01:00
Smita Kulkarni
fb765696b6 Removed state_mutex lock from addSubscriber - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 12:19:58 +01:00
Vitaly Baranov
f021c0eab6
Merge pull request #46058 from vitlibar/remove-unnecessary-execute-for-const-expr 
Remove unnecessary execute() while evaluating a constant expression.
 2023-02-07 12:09:37 +01:00
Smita Kulkarni
ef29b61786 Updated callback to return log entry ids - Updated checking of SYSTEM SYNC REPLICA 2023-02-07 10:02:38 +01:00
Anton Popov
7dfea59267 remove an assertion 2023-02-07 00:50:41 +00:00
alesapin
dd3a98e88d Fixes 2023-02-06 22:30:31 +01:00
alesapin
aac5da77e8 fix stupid bug 2023-02-06 21:32:40 +01:00
kssenii
6cea67d58b Cleanup 2023-02-06 18:11:33 +01:00
alesapin
2a0484ca41 Add commnents, fix bugs 2023-02-06 18:06:14 +01:00
Yakov Olkhovskiy
e433c26cf9 allow bloom filter for IPv4 and IPv6 2023-02-06 16:30:18 +00:00
Alexander Tokmakov
98a4e1ab2c better comment 2023-02-06 15:08:30 +01:00
alesapin
30a7198411
Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp 2023-02-06 15:06:28 +01:00
alesapin
bd4a3ce2d5
Update src/Storages/MergeTree/DataPartsExchange.cpp 2023-02-06 15:05:13 +01:00
Alexander Tokmakov
86feaae253 Merge branch 'master' into explicit_drop_part_entry_type 2023-02-06 14:59:59 +01:00
alesapin
e51e385017 fix style 2023-02-06 14:16:17 +01:00
Alexander Tokmakov
e9f770d89d fix 2023-02-06 14:03:14 +01:00
alesapin
b94f9f8006
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-02-06 12:38:39 +01:00
alesapin
b265300a19 Merge branch 'master' into mutations_rename_hang 2023-02-06 12:25:29 +01:00
Sema Checherinda
734d3a318b
const auto ref in the loop 2023-02-06 12:09:33 +01:00
Vitaly Baranov
a3423ce196 Remove unnecessary execute() while evaluating a constant expression. 2023-02-05 20:26:34 +01:00
Han Fei
532b341de9
Merge pull request #45975 from ucasfl/_part 
use LowCardnality for _part and _partition_id virtual column
 2023-02-05 18:00:46 +01:00
Antonio Andelic
12569da984 Merge branch 'master' into add-support-for-settings-alias 2023-02-05 16:08:57 +00:00
kssenii
a7eb1b1419 Fix integration test 2023-02-04 22:57:30 +01:00
kssenii
5abbc7cad6 Cleanup 2023-02-04 19:41:41 +01:00
kssenii
1cbff88a7c Fix integration test 2023-02-04 19:31:19 +01:00
Alexey Milovidov
22bd0b6f69
Merge pull request #38983 from CurtizJ/randomize-mt-settings 
Allow to randomize merge tree settings in tests
 2023-02-04 02:59:52 +01:00
alesapin
fd4ae0990b Tiny fix 2023-02-03 21:07:34 +01:00
kssenii
5f124263a6 Revert changes with complex SettingsChanges 2023-02-03 21:04:45 +01:00
alesapin
9416401406 Fix 2023-02-03 20:52:10 +01:00
alesapin
f91c10d09d Initial support for version 2023-02-03 20:49:07 +01:00
alesapin
7a0d0f0c53 Some code which doesn't work 2023-02-03 19:14:49 +01:00
Antonio Andelic
8929838fcc Merge branch 'master' into add-support-for-settings-alias 2023-02-03 17:47:00 +00:00
Alexander Tokmakov
3f11948bb0
Merge branch 'master' into stack_trace_in_part_log 2023-02-03 20:05:00 +03:00
Alexander Tokmakov
7e6f7c79f2
Merge pull request #45457 from FrankChen021/exception_time 
Add last_exception_time to replication_queue
 2023-02-03 20:00:15 +03:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] (#43772) 2023-02-03 14:34:18 +01:00
Alexey Milovidov
3e3df376c0
Merge pull request #45995 from CurtizJ/check-dynamic-columns 
Check dynamic columns of part before its commit
 2023-02-03 15:39:54 +03:00
alesapin
4a3277b1fd Merge branch 'master' into mutations_rename_hang 2023-02-03 12:35:37 +01:00
Smita Kulkarni
123f3cfe1f Used unordered_set for log entries and fixed the callback function - Updated checking of SYSTEM SYNC REPLICA 2023-02-03 08:48:16 +01:00
Frank Chen
d38adfab30 Merge two overridden functions as one 2023-02-03 15:27:45 +08:00
flynn
d119367d7d fix 2023-02-03 07:06:58 +00:00
Frank Chen
d3a05a11da Merge remote-tracking branch 'remotes/github/master' into stack_trace_in_part_log 2023-02-03 11:39:43 +08:00
Frank Chen
7af5ae7f2e
Merge branch 'master' into better_postpone_reason 2023-02-03 11:20:37 +08:00
Anton Popov
8ca602a148 check dynamic columns of part before its commit 2023-02-03 02:58:28 +00:00
alesapin
b8b0f1c3d3 Make RENAME COLUMN barrier 2023-02-02 21:11:19 +01:00
kssenii
a391de18a1 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-02-02 20:49:41 +01:00
alesapin
fc6c62e2ff Add comment 2023-02-02 17:41:36 +01:00
alesapin
bdc530dead Fix style 2023-02-02 17:30:38 +01:00
flynn
bc38ebaf52 use LowCardnality for _part and _partition_id virtual column 
fix
 2023-02-02 16:20:29 +00:00
alesapin
dea46e58fb Fixes for ordinary merge tree 2023-02-02 17:03:24 +01:00
alesapin
174bdba912 Fix bugs in alter rename column 2023-02-02 16:57:46 +01:00
Alexander Gololobov
ec3bb0c04e
Merge pull request #45794 from ClickHouse/parallelize_is_storage_touched_by_mutations 
Do not restrict count() query to 1 thread in isStorageTouchedByMutations()
 2023-02-02 16:44:11 +01:00
Antonio Andelic
f613bfd8d2 Merge branch 'master' into add-support-for-settings-alias 2023-02-02 11:15:39 +00:00
Sema Checherinda
7c416477ac do not run wal on remote disks 2023-02-01 20:03:45 +01:00
Sema Checherinda
3e1b17e4cf
fix build 2023-02-01 17:09:43 +01:00
Anton Popov
9f26e16963 loose the assertion 2023-02-01 15:31:06 +00:00
Sema Checherinda
41a1dddc7a
Merge branch 'master' into better_postpone_reason 2023-02-01 15:16:34 +01:00
Antonio Andelic
85cfee4bb9 Better alias definition 2023-02-01 13:54:03 +00:00
Alexander Tokmakov
aba4a00a17 fix 2023-02-01 14:47:57 +01:00
Antonio Andelic
ec04cf6876 Merge branch 'master' into add-support-for-settings-alias 2023-02-01 09:01:01 +00:00
Anton Popov
1c4174750d fix mistake after refactoring 2023-01-31 15:48:14 +00:00
Alexander Tokmakov
7862990721 Merge branch 'master' into explicit_drop_part_entry_type 2023-01-31 13:49:04 +01:00
Alexander Tokmakov
2e8100b6e4 make separate DROP_PART log entry type 2023-01-31 13:37:56 +01:00
Alexander Tokmakov
51b0f66e6b
Merge pull request #45793 from ClickHouse/fix_part_check_cancellation 
Fix race condition on a part check cancellation
 2023-01-31 14:15:17 +03:00
Alexander Gololobov
4eea3de654 Do not restrict count() query to 1 thread in isStorageTouchedByMutations() 2023-01-30 22:29:28 +01:00
Alexander Tokmakov
e42cd3163f fix race condition on a part check cancellation 2023-01-30 18:48:20 +01:00
Arthur Passos
48e5211454
Merge branch 'ClickHouse:master' into add_select_final_mt_setting 2023-01-30 13:01:36 -03:00
alesapin
be58d5d1af Fix bug in tables drop which can lead to potential query hung 2023-01-30 17:00:28 +01:00
Smita Kulkarni
6cf2498609 Fixed clang-tidy build for comparing strings and updated to notify after adding subscriber if queue is already empty- Updated checking of SYSTEM SYNC REPLICA 2023-01-30 08:18:20 +01:00
SmitaRKulkarni
809354dc30
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-01-29 11:28:21 +01:00
Alexey Milovidov
49a59ff9bd Fix bad comment 2023-01-29 03:57:59 +01:00
Smita Kulkarni
9b457a57a9 Fixed clang-tidy build - Updated checking of SYSTEM SYNC REPLICA 2023-01-28 21:43:41 +01:00
Smita Kulkarni
2bbe08ce8b Updated destructor to notify even when queue empty - Updated checking of SYSTEM SYNC REPLICA 2023-01-27 22:37:42 +01:00
Anton Popov
d8fe2bcbaa
Merge branch 'master' into randomize-mt-settings 2023-01-27 16:03:00 +01:00
Anton Popov
3f1db52d51 Merge remote-tracking branch 'upstream/master' into compact-parts-vertical-merge 2023-01-27 14:38:37 +00:00
Nikolai Kochetov
313947e0b0
Merge branch 'master' into issue_45590 2023-01-27 15:08:28 +01:00
SmitaRKulkarni
b22fd40d92
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica 2023-01-27 15:00:47 +01:00
Anton Popov
f10e82355e
Merge pull request #45630 from CurtizJ/fix-performance-columns-description 
Fix performance of short queries with `Array` columns
 2023-01-27 14:59:55 +01:00
Anton Popov
9c1717b42d fix computation of granularity in vertical merges 2023-01-27 13:35:04 +00:00
Nikolai Kochetov
f6ad6296ba Restore. 2023-01-27 12:45:41 +00:00
Alexander Tokmakov
1494262a38
Update DataPartStorageOnDiskFull.cpp 2023-01-27 15:42:41 +03:00
Nikolai Kochetov
b716e2d754 A bit cleaner fix. 2023-01-27 12:37:23 +00:00
alesapin
716d25c33f
Merge pull request #45619 from CurtizJ/refactoring-merge-tree-parts 
Refactoring of code near merge tree parts
 2023-01-27 13:36:33 +01:00
Smita Kulkarni
a95609e0bc Updated to validate & remove each element instead of just last element - Updated checking of SYSTEM SYNC REPLICA 2023-01-27 10:35:12 +01:00
Amos Bird
716516f84a
Fix test 2023-01-27 16:58:41 +08:00
Amos Bird
fd60954081
Fix key description on duplicate primary keys 
This can happen in projections. See
https://github.com/ClickHouse/ClickHouse/issues/45590 for details.
 2023-01-27 13:01:36 +08:00
Anton Popov
fb3f7409fa allow vertical merges from compact to wide parts 2023-01-26 22:18:04 +00:00
Alexander Tokmakov
061026c307 fix 2023-01-26 19:01:29 +01:00
Alexander Tokmakov
067b1f5f13 Merge branch 'master' into exception_message_patterns4 2023-01-26 15:20:58 +01:00
Antonio Andelic
714fad1529 Add support for settings alias 2023-01-26 14:06:46 +00:00
Alexander Tokmakov
f676e9eaff
Merge pull request #45613 from azat/replicated-improve-log 
Improve "at least part X is missing" error message
 2023-01-26 16:48:18 +03:00