thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 12:32:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
144,024 Commits 9,194 Branches 1,782 Tags 2.2 GiB
b1b9aaf2ec
Commit Graph

58502 Commits

Author SHA1 Message Date
josh-hildred
e495bf0742 Update src/Interpreters/getCustomKeyFilterForParallelReplicas.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2024-06-06 05:28:16 -07:00
Joshua Hildred
0c0c6bee32 Address review comments 2024-06-06 05:28:15 -07:00
Joshua Hildred
bde8d19f45 Address review comments 2024-06-06 05:28:15 -07:00
Joshua Hildred
0302f218e4 Add additional check to ensure range is larger than the number of replicas 2024-06-06 05:28:15 -07:00
Joshua Hildred
01c7123c10 Fix issue with rational cast 2024-06-06 05:28:15 -07:00
Joshua Hildred
9ed52af56f Add new settings to SettingsChangesHistory.h 2024-06-06 05:28:13 -07:00
Joshua Hildred
d455116b7c fix style 2024-06-06 05:26:57 -07:00
Joshua Hildred
a7230e3c6c Add settings to allow parallel replicas custom key 
with range filter to use a custom range
 2024-06-06 05:26:57 -07:00
Mikhail Gorshkov
f0c21da1d8 Style fixed 2024-06-06 11:06:16 +00:00
Mikhail Gorshkov
2f50e070ef Scale argument restricted to types (U)Int8 - (U)Int64 2024-06-06 10:34:59 +00:00
János Benjamin Antal
eb724ad8c7
Merge pull request #64255 from wudidapaopao/fix_integer_accurate_cast 
Fix unexpected accurateCast from string to integer
 2024-06-06 09:53:58 +00:00
Ilya Golshtein
aefe7ffc31 time_virtual_col: slightly works 2024-06-06 09:48:32 +00:00
Ilya Golshtein
d0a2abe17b time_virtual_col: initial 2024-06-06 09:48:32 +00:00
alesapin
71f37910ba
Merge pull request #64899 from ClickHouse/revert-61973-mt/rename-without-lock 
Revert "Reduce lock contention for MergeTree tables (by renaming parts without holding lock)"
 2024-06-06 09:43:42 +00:00
alesapin
7900fe5836
Revert "Reduce lock contention for MergeTree tables (by renaming parts without holding lock)" 2024-06-06 11:36:57 +02:00
János Benjamin Antal
d74b93413d
Merge pull request #64230 from ClickHouse/const-settings-changes-history 
Make `settings_changes_history` const
 2024-06-06 09:31:25 +00:00
Antonio Andelic
fcee260b25
Update src/Interpreters/TraceCollector.h 
Co-authored-by: alesapin <alesapin@clickhouse.com>
 2024-06-06 11:22:48 +02:00
Antonio Andelic
1d3cf17053 Fix global trace collector 2024-06-06 11:18:31 +02:00
Smita Kulkarni
9ce824d41c Addressed review comments 2024-06-06 11:04:38 +02:00
Julia Kartseva
94a8c7edb1
Merge pull request #64720 from korowa/fix-split-tokenized-indices 
fix: function filters with token-based text indexes
 2024-06-06 09:02:23 +00:00
Sergei Trifonov
feb60b566c
Merge branch 'master' into precise-memtracking 2024-06-06 10:52:28 +02:00
serxa
f9ac18d74a better description for history of a setting changes 2024-06-06 08:16:11 +00:00
Michael Kolupaev
fd93097130 Fix writing ORC statistics for unsigned types 2024-06-06 06:39:07 +00:00
Alexey Milovidov
ce244e126d
Merge pull request #61973 from azat/mt/rename-without-lock 
Reduce lock contention for MergeTree tables (by renaming parts without holding lock)
 2024-06-06 05:36:28 +02:00
Alexey Milovidov
c0a02475c1
Merge pull request #64860 from ClickHouse/settings-changes 
Add new settings to changes history.
 2024-06-06 05:30:31 +02:00
Alexey Milovidov
cb2c085535
Merge pull request #64610 from thomoco/master 
Updated Advanced Dashboard for both open-source and ClickHouse Cloud versions to include a chart for 'Maximum concurrent network connections'
 2024-06-06 05:21:33 +02:00
Alexey Milovidov
f9c243064f A tiny fix for fancy quotes 2024-06-06 04:20:20 +02:00
Alexey Milovidov
6c24bb48c5
Merge pull request #64770 from ClickHouse/backtothefuture 
Fix assert in IObjectStorageIteratorAsync
 2024-06-06 03:37:34 +02:00
Alexey Milovidov
53f26f614e
Merge pull request #64769 from ClickHouse/pufit/rbac-fix-on-cluster-grant-with-default-database 
Fix default database with grant on cluster
 2024-06-06 03:35:32 +02:00
Alexey Milovidov
9ab06931d2
Merge pull request #64845 from ClickHouse/follow-up-s3-queue 
Follow-up to #64349
 2024-06-06 02:56:58 +02:00
Alexey Milovidov
81c647e680
Merge pull request #64844 from CurtizJ/fix-memory-tracker 
Fix untracked memory in `MemoryTrackerSwitcher`
 2024-06-06 02:18:43 +02:00
Alexey Milovidov
9d8bc4d54d
Merge pull request #59427 from seandhaynes/master 
Use scheduleOrThrow in MergeTree data selection and initialization to avoid deadlocks
 2024-06-06 02:06:22 +02:00
Alexey Milovidov
8863736459 Fix style 2024-06-05 22:53:08 +02:00
Nikita Mikhaylov
efbf40ad28
Merge pull request #64846 from rschu1ze/simplify-inverted-indexes 
Simplify handling of old 'inverted' indexes
 2024-06-05 20:04:16 +00:00
pufit
e7b7c3aebe Update query before replication 2024-06-05 16:00:08 -04:00
Nikita Mikhaylov
f42452d51e Add settings to changes history 2024-06-05 21:57:23 +02:00
Nikita Mikhaylov
106c1529ed Introduce an alias 2024-06-05 21:14:26 +02:00
wudidapaopao
c96800e96d
Merge branch 'ClickHouse:master' into fix_integer_accurate_cast 2024-06-06 03:06:09 +08:00
Nikita Mikhaylov
eb8520758a Done 2024-06-05 20:51:07 +02:00
Azat Khuzhin
6cfd5b2165 Fix possible assertion when size of precommitted_parts <= precommitted_parts_need_rename 
CI founds [1]:

    Logical error: 'precommitted_parts.size() >= precommitted_parts_need_rename.size()'

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/61973/5c1e6a3e956917bdbb7eaa467934e5b75f17a923/stateless_tests__tsan__s3_storage__[5_5].html

The problem is that after precommitted_parts cleaned from detached parts
it may be less then precommitted_parts_need_rename, so to avoid this,
let's just copy it to a new container.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-06-05 19:38:24 +02:00
Azat Khuzhin
6f522c1d61 Do not remove detached parts in Transaction::rollback 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-06-05 19:38:24 +02:00
Azat Khuzhin
3675c27fe9 Require explicit rename of parts in transaction 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-06-05 19:38:24 +02:00
Azat Khuzhin
ca2c720d0e Avoid race between cleanup thread and renameMergedTemporaryPart() 
The problem was that with this patch set renameMergedTemporaryPart() is
called without temporary_directory_lock holded (in MergeTask), since it
is reseted just before calling renameMergedTemporaryPart(), and this can
be seen in logs:

    2024.03.29 19:56:42.126919 [ 1341 ] {ea7a3fd2-cf47-4ec7-91a5-51c69fba1b95::-8_0_138_2_2} <Trace> test_btnct5cr.alter_table_0 (ea7a3fd2-cf47-4ec7-91a5-51c69fba1b95) (MergerMutator): Merged 50 parts: [-8_0_0_0_2, -8_138_138_0] -> -8_0_138_2_2
    2024.03.29 19:56:42.127034 [ 1341 ] {ea7a3fd2-cf47-4ec7-91a5-51c69fba1b95::-8_0_138_2_2} <Debug> test_btnct5cr.alter_table_0 (ea7a3fd2-cf47-4ec7-91a5-51c69fba1b95): Committing part -8_0_138_2_2 to zookeeper
    2024.03.29 19:56:42.128462 [ 884 ] {} <Warning> test_btnct5cr.alter_table_0 (ea7a3fd2-cf47-4ec7-91a5-51c69fba1b95): Removing temporary directory /var/lib/clickhouse/store/ea7/ea7a3fd2-cf47-4ec7-91a5-51c69fba1b95/tmp_merge_-8_0_138_2_2/
    2024.03.29 19:56:42.128647 [ 1341 ] {ea7a3fd2-cf47-4ec7-91a5-51c69fba1b95::-8_0_138_2_2} <Debug> test_btnct5cr.alter_table_0 (ea7a3fd2-cf47-4ec7-91a5-51c69fba1b95): Part -8_0_138_2_2 committed to zookeeper
    ...
    2024.03.29 19:56:54.586084 [ 57841 ] {bf240267-0620-4294-afc1-479c58e6be89} <Error> executeQuery: std::exception. Code: 1001, type: std::__1::__fs::filesystem::filesystem_error, e.what() = filesystem error: in file_size: No such file or directory ["/var/lib/clickhouse/store/ea7/ea7a3fd2-cf47-4ec7-91a5-51c69fba1b95/-8_0_138_2_2/data.cmrk3"]

This should fix failures of 00993_system_parts_race_condition_drop_zookeeper in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/61973/f6f826c85dd5b7bb8db16286fd10dcf441a440f7/stateless_tests__coverage__[4_6].html

Though now it looks hackish...

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-06-05 19:38:24 +02:00
Azat Khuzhin
b41d08a2b6 Use renameParts() explicitly to avoid leaving parts in detached 
Since there is an assertion that does not allows to remove detached
parts during cleanup, which sounds good in general, but breaks this new
code.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-06-05 19:38:24 +02:00
Azat Khuzhin
ee546fa00a Fix replacing parts with empty 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-06-05 19:38:24 +02:00
Azat Khuzhin
6c3db34aae Remove unused locked_parts from MergeTreeData::Transaction 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-06-05 19:38:24 +02:00
Azat Khuzhin
78088ce59a Reduce lock contention for MergeTree tables (by renaming parts without holding lock) 
Under heavy load, or not so heavy but with fsync_part_directory=1,
time that renameTo() holds DataPartsLock will be increased, and this
will affect almost every operation with this table.

On one of production clusters I saw ~60 seconds with
fsync_part_directory=1.

Move the renameTo() out from the critical section.

v2: instead of using DataPartsLock.lock.lock()/unlock() move the renameTo() into MergeTreeData::Transaction::commit()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-06-05 19:38:23 +02:00
Azat Khuzhin
66a2962cce Add reason into "Part {} is broken and need manual correction" message 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-06-05 19:38:23 +02:00
alesapin
966b08f192 Fix segfault for a very tricky case 2024-06-05 19:26:20 +02:00
Nikolai Kochetov
8bbf46ce2f Trying to fix a test. 2024-06-05 17:15:48 +00:00
Sergei Trifonov
9c7a8727d0
Merge pull request #64244 from kirillgarbar/check_zk_path 
Prevent conversion to Replicated if zookeeper path already exists
 2024-06-05 16:36:10 +00:00
Nikolai Kochetov
ee5100171e Avoid duplicating names for new inputs of ActionsDAG::split result. 2024-06-05 15:50:27 +00:00
vdimir
5251febf03
Merge pull request #64393 from ClickHouse/vdimir/blob_storage_log_fix_recursion 
Prevent recursive logging in blob_storage_log
 2024-06-05 14:28:01 +00:00
kssenii
98b7805697 Fix unused field 2024-06-05 16:19:17 +02:00
Mikhail Gorshkov
7ce67265c7 PR post-review fixes 2024-06-05 14:06:31 +00:00
Robert Schulze
8bdd291049
Simplify handling of old 'inverted' indexes 2024-06-05 13:27:07 +00:00
kssenii
2a30c77346 Fix compatibility 2024-06-05 15:21:51 +02:00
Mikhail Gorshkov
83901b82c9 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-1892 2024-06-05 13:13:06 +00:00
Anton Popov
5e9a41bd9f fix untacked memory in MemoryTrackerSwitcher 2024-06-05 13:10:37 +00:00
Han Fei
ac430bb01d
Merge pull request #59357 from hanfei1991/hanfei/stats_uniq 
Introduce statistics of type "number of distinct values"
 2024-06-05 12:56:52 +00:00
vdimir
d2d8618d27
Merge pull request #64775 from bigo-sg/debug_inequal_join 
Mixed join condition with function `in`
 2024-06-05 12:40:30 +00:00
Robert Schulze
1628c2985d Some fixups 2024-06-05 11:55:55 +00:00
Han Fei
9b7944e3a5
Merge pull request #64827 from ClickHouse/fix-clang-tidy-s3-queue 
Fix clang-tidy
 2024-06-05 11:54:14 +00:00
Kseniia Sumarokova
b6b19ca6d3
Merge pull request #64762 from azat/mt-write-once-fixes 
Do not try to write columns.txt if it does not exists for write-once storages
 2024-06-05 11:22:54 +00:00
Igor Nikonov
22d50dcd3e
Merge pull request #64579 from ClickHouse/store-analysis-result 
Cache first analysis result in ReadFromMergeTree
 2024-06-05 11:21:50 +00:00
Han Fei
c04e7e64af Merge branch 'master' into hanfei/stats_uniq 2024-06-05 13:09:15 +02:00
Kseniia Sumarokova
f50fbfa411
Merge pull request #64803 from ClickHouse/fix-memory-leak-in-slru 
Fix memory leak in slru cache policy
 2024-06-05 09:58:12 +00:00
Blargian
5f3bc4271f rename forgoten isEscapeSequence to is_escape_sequence 2024-06-05 10:58:30 +02:00
kssenii
c25f8fa28c Fix 2024-06-05 10:58:08 +02:00
Robert Schulze
ec3b82ba63
Merge pull request #64606 from rschu1ze/map-stuff 
Double-checking #59318 and docs for `Map`
 2024-06-05 07:56:29 +00:00
Robert Schulze
4639b108e7
Merge pull request #64823 from rschu1ze/fix-tidy 
Fix clang-tidy build
 2024-06-05 07:42:05 +00:00
Robert Schulze
3bb1157a0e
Fix clang-tidy 2024-06-05 07:32:57 +00:00
Antonio Andelic
0f23f9b384
Merge pull request #64784 from ClickHouse/keeper-correct-snapshot-size 
Fix Keeper snapshot size in `mntr`
 2024-06-05 07:17:57 +00:00
Alexey Milovidov
2c5ebf9ca6
Merge branch 'master' into master 2024-06-05 04:23:37 +02:00
Alexey Milovidov
02a3049876
Merge pull request #58285 from canhld94/fix_a_test 
Hot reload storage policy for distributed tables when adding a new disk
 2024-06-05 04:11:49 +02:00
lgbo-ustc
8c94832c20 fixed #64513. mixed join condition with function 'in' 2024-06-05 09:33:04 +08:00
Alexey Milovidov
09c2151f3b
Fix style 2024-06-05 03:08:58 +02:00
Alexey Milovidov
bb9091ad5a
Merge pull request #64582 from arenadata/ADQM-1868 
Binary math functions Decimal support
 2024-06-05 02:15:40 +02:00
Alexey Milovidov
78506304b7
Merge pull request #55061 from amosbird/support-empty-tuple-2 
Support empty tuple [second implementation]
 2024-06-05 02:00:03 +02:00
Han Fei
f7ca33868b Merge branch 'master' into hanfei/stats_uniq 2024-06-05 01:21:49 +02:00
Julia Kartseva
1638dd0837
Merge pull request #64483 from korowa/read-in-order-nullable 
fix: optimize_read_in_order for nullable keys
 2024-06-04 22:49:27 +00:00
Alexey Milovidov
5a5bdd87cb
Merge pull request #64592 from tomershafir/xray 
xray: add global xray instrumentation support
 2024-06-05 01:25:25 +02:00
Alexey Milovidov
a6fe50c103
Merge pull request #42544 from ClickHouse/try-about-on-current-thread-join 
Try abort on current thread join.
 2024-06-05 01:13:25 +02:00
Julia Kartseva
12bf33c519
Merge pull request #64778 from jkartseva/fix-plain-rewritable-cache 
Fix crash in a local cache over `plain_rewritable` disk
 2024-06-04 20:34:58 +00:00
Kseniia Sumarokova
d044d228ba
Merge pull request #64349 from ClickHouse/s3queue-fix-ordered-mode 
S3Queue rework ordered mode parallel processing
 2024-06-04 20:27:46 +00:00
Robert Schulze
46434f9040
Merge pull request #63578 from ElderlyPassionFruit/add-compression-sorts-optimization 
Best-effort sorting to improve compressability
 2024-06-04 19:02:55 +00:00
Robert Schulze
9cf8bdcdb2
Merge pull request #64656 from rschu1ze/deprecated-inverted-index 
Be more graceful with existing tables with `inverted` indexes
 2024-06-04 18:45:03 +00:00
Robert Schulze
c061a42f35
Merge pull request #64742 from Beetelbrox/fjurado-61548/feat-add-parseReadableSize-function 
Add `parseReadableSize` function
 2024-06-04 18:39:33 +00:00
Alexey Milovidov
c0aad2a2b5
Merge pull request #64777 from ClickHouse/remove-iostream-debug-helpers 
Remove iostream debug helpers
 2024-06-04 20:44:30 +02:00
Eduard Karacharov
f1a56080af fix: optimize_read_in_order for nullable keys 2024-06-04 20:56:45 +03:00
kssenii
1790c85465 Fix build without s3 in fast test 2024-06-04 19:42:31 +02:00
Raúl Marín
499514f57d
Merge pull request #64785 from Algunenano/comment 
Remove unnecessary comment
 2024-06-04 17:33:31 +00:00
kssenii
c05d4e52d3 Fix memory leak in slry 2024-06-04 19:26:12 +02:00
Nikolai Kochetov
1d77cda70b Fix distributed array join by nested 2024-06-04 17:13:19 +00:00
Sema Checherinda
dd6ae2f77c
Merge pull request #63427 from arthurpassos/fix_wrong_request_protocol_proxy 
Several minor fixes to proxy support in ClickHouse
 2024-06-04 16:35:03 +00:00
Alexey Milovidov
5b16681311 Merge branch 'master' into try-about-on-current-thread-join 2024-06-04 18:11:34 +02:00
Mikhail Gorshkov
830acd3a89 Support non-const scale arguments in rounding functions 2024-06-04 15:32:38 +00:00
Igor Markelov
613e82c39b Merge branch 'master' into add-compression-sorts-optimization 2024-06-04 15:05:44 +00:00
Antonio Andelic
0ae901c479 Better 2024-06-04 16:51:04 +02:00
Igor Nikonov
9372768239 Merge remote-tracking branch 'origin/master' into store-analysis-result 2024-06-04 14:41:52 +00:00