thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
106,908 Commits 9,326 Branches 1,783 Tags 2.1 GiB
33877b5e00
Commit Graph

5088 Commits

Author SHA1 Message Date
Han Fei
701dc88d6f
Merge branch 'master' into hanfei/support-advance-dedup 2023-01-15 19:46:28 +01:00
Han Fei
c859f8dbe5
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2023-01-15 19:46:16 +01:00
Han Fei
bb2c0914e9
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2023-01-15 19:46:09 +01:00
Alexander Tokmakov
d857d62a03 remove another set of macros 2023-01-13 20:34:31 +01:00
Alexander Tokmakov
2d7773fccc Merge branch 'master' into text_log_add_pattern 2023-01-13 20:33:46 +01:00
Han Fei
ed49ebf01a update setting explain 2023-01-13 20:26:08 +01:00
Han Fei
2fb2f503e3 Update src/Storages/MergeTree/MergeTreeSettings.h 
Co-authored-by: Sergei Trifonov <svtrifonov@gmail.com>
 2023-01-13 20:20:08 +01:00
Han Fei
9e99c7e116 Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp 
Co-authored-by: Sergei Trifonov <svtrifonov@gmail.com>
 2023-01-13 20:19:13 +01:00
Han Fei
a258a39eb1 Merge branch 'master' into hanfei/async-cache 2023-01-13 20:17:58 +01:00
Anton Popov
487de70d01 fix locking at loading outdated data parts 2023-01-13 17:05:32 +00:00
Anton Popov
71188c22ee fix race on 'relative_data_path' 2023-01-13 12:19:41 +00:00
Alexander Tokmakov
51d94314d6
Merge pull request #45235 from ClickHouse/more_verbose_logs_about_replication_log_entries 
More verbose logs about replication log entries
 2023-01-13 15:05:21 +03:00
Alexander Tokmakov
d9b8cf9c04 more verbose logs about replication log entries 2023-01-12 20:32:39 +01:00
serxa
693489a8ad review fixes 2023-01-12 15:51:04 +00:00
Sergei Trifonov
12d8543578
Merge branch 'master' into cancellable-mutex-integration 2023-01-12 16:03:49 +01:00
Anton Popov
8bdf63f8e5
Merge pull request #45191 from ClickHouse/fix-ip-part-name 
Fix part ID generation for IP types for backward compatibility
 2023-01-12 12:15:06 +01:00
Igor Nikonov
edbcb24da0
Merge pull request #44954 from ClickHouse/fix-insert-delay 
Fix: insert delay calculation
 2023-01-12 09:43:00 +01:00
HarryLeeIBM
6d576cf157 Added more documentation 2023-01-11 13:40:20 -08:00
Yakov Olkhovskiy
9c4ea5a16b fix part ID generation for IP types for backward compatibility 2023-01-11 18:51:37 +00:00
Alexander Tokmakov
0fba3d6d81
Merge pull request #44468 from ClickHouse/fix_get_part_name 
Fix incorrect usages of `getPartName()`
 2023-01-11 16:24:02 +03:00
Igor Nikonov
1d002e45a9 Fix test & review comments 2023-01-11 12:49:01 +00:00
Nikolai Kochetov
5e7a6ac619
Merge pull request #45122 from ClickHouse/revert-45121-revert-44653-custom-reading-for-mutation 
Revert "Revert "Custom reading for mutation""
 2023-01-11 12:37:32 +01:00
Nikolai Kochetov
d0c1079242
Merge pull request #45133 from ClickHouse/fix-additinal-tables-filter-with-count-projection 
Fix additional_table_filters with minmax/count projection.
 2023-01-11 12:25:40 +01:00
Anton Popov
85734bb321 fix build 2023-01-11 08:32:33 +00:00
Anton Popov
6c4f53bdca Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-11 01:00:57 +00:00
HarryLeeIBM
150f9b48a4 Fix functional test for full text search 2023-01-10 14:35:06 -08:00
Igor Nikonov
543221b8b9 Merge remote-tracking branch 'origin/master' into fix-insert-delay 2023-01-10 22:15:41 +00:00
Igor Nikonov
e75df5e76b Test 
+ fix some review comments
 2023-01-10 22:14:54 +00:00
Alexander Tokmakov
881b17492f Merge branch 'master' into fix_get_part_name 2023-01-10 21:39:35 +01:00
Alexander Tokmakov
255881a2ff
Merge pull request #44466 from ClickHouse/parallel_parts_cleanup_with_zero_copy 
Parallel parts cleanup with zero copy replication
 2023-01-10 22:44:39 +03:00
Ilya Yatsishin
0efceae6ce
Merge pull request #44917 from azat/mt-fix-log-message 
Fix logging message in MergeTreeDataMergerMutator (about merged parts)
 2023-01-10 19:49:31 +01:00
Nikolai Kochetov
9a81f27fb2 Fix additional_table_filters with minmax/count projection. 2023-01-10 17:32:26 +00:00
HarryLeeIBM
bc48b17817 Addressed more review comments. 2023-01-10 08:26:27 -08:00
Nikolai Kochetov
4673b3fe1d
Revert "Revert "Custom reading for mutation"" 2023-01-10 16:31:01 +01:00
Alexander Tokmakov
c8ec130be4
Revert "Custom reading for mutation" 2023-01-10 17:51:30 +03:00
Han Fei
bef6168c05 support cache for async inserts block ids 2023-01-10 13:19:12 +01:00
Anton Popov
b9bf92fa5d check for network errors while loading parts 2023-01-10 11:55:08 +00:00
Nikolai Kochetov
11418963c0
Merge pull request #44653 from ClickHouse/custom-reading-for-mutation 
Custom reading for mutation
 2023-01-10 12:16:24 +01:00
alesapin
616e37ba43
Merge pull request #44847 from ClickHouse/fix_rare_race 
Fix rare race which can lead to queue hang
 2023-01-10 12:09:10 +01:00
Nikolai Kochetov
3e430b1b6e Add comments. Remove unused flag from Context. 2023-01-09 19:52:11 +00:00
Anton Popov
1e5d6e44a5 fix deadlock at shutdown 2023-01-09 13:07:46 +00:00
Sergei Trifonov
81d2ea30ba
Merge branch 'master' into cancellable-mutex-integration 2023-01-07 19:37:46 +01:00
Anton Popov
1f32ffedf8
Merge pull request #43221 from ClickHouse/refactoring-ip-types 
Replace domain IP types (IPv4, IPv6) with native
 2023-01-07 12:01:21 +01:00
Alexey Milovidov
9fcd111724
Merge pull request #45003 from amosbird/issue_44979 
Fix minmax_count_projection with _partition_value
 2023-01-07 03:36:56 +03:00
serxa
15bb127b01 replace every std::shared_mutex with DB::FastSharedMutex 2023-01-06 23:35:38 +00:00
Amos Bird
200d40c53d
Fix minmax_count_projection with _partition_value 
Check if virtual column `_partition_value` can be used when building
minmax_count_projection block.
 2023-01-07 05:38:28 +08:00
Dmitry Novik
9d44c7a737
Merge pull request #44541 from ClickHouse/cleanup-resolved-function 
Followup #43761
 2023-01-06 18:09:32 +01:00
alesapin
da83e97b4c Fix startup issue 2023-01-06 15:36:53 +01:00
Anton Popov
b25f875674
Merge pull request #44875 from ClickHouse/fix-another-one-cannot-read-all-data-for-lc-dict-error 
Fix right offset for reading LowCardinality dictionary from remote fs
 2023-01-06 15:24:36 +01:00
alesapin
b20f07b2cc Merge branch 'master' into fix_rare_race 2023-01-06 14:20:08 +01:00
Anton Popov
fcdbc8123d better loading of outdated parts 2023-01-06 11:59:54 +00:00
Alexey Milovidov
b88be9df8c
Merge pull request #44956 from ClickHouse/fix_read_result_consistency_check_on_exception 
Do not check read result consistency when unwinding
 2023-01-06 06:22:42 +03:00
Alexey Milovidov
a205b41c56
Merge pull request #44947 from Algunenano/background_pool_size_logical 
Change error code on invalid background_pool_size config
 2023-01-06 03:19:40 +03:00
Alexander Gololobov
1b31b8bb91 Do not check read result consistency when unwinding 2023-01-06 00:26:08 +01:00
Igor Nikonov
a67afdff6a Fix: insert delay calculation 2023-01-05 22:43:41 +00:00
Dmitry Novik
ebead611b6
Merge branch 'master' into cleanup-resolved-function 2023-01-05 19:27:08 +01:00
Raúl Marín
b0715ec5eb Change error code on invalid background_pool_size config 2023-01-05 18:47:02 +01:00
Anton Popov
ed00399eb6 Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-05 14:12:02 +00:00
Larry Luo
f2bc5bb335 Added initializer 2023-01-05 05:41:29 -08:00
Igor Nikonov
2a105d1862 Fix stupid mistake - max() -> min() 2023-01-05 12:38:27 +00:00
Igor Nikonov
db46267a95 Fast fix: force upper bound for time to delay INSERT 2023-01-05 12:38:27 +00:00
Larry Luo
fd0c6cccc8 Fixed style error 2023-01-04 20:08:28 -08:00
Larry Luo
7a4d499cb1 Added density to control Inverted index size 2023-01-04 19:42:45 -08:00
Azat Khuzhin
d06ac2d468 Fix logging message in MergeTreeDataMergerMutator (about merged parts) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-04 19:07:01 +01:00
Yakov Olkhovskiy
7a5a36cbed
Merge branch 'master' into refactoring-ip-types 2023-01-04 11:11:06 -05:00
alesapin
2cd7247619 Merge branch 'master' into fix_rare_race 2023-01-04 15:06:20 +01:00
Kruglov Pavel
9056925c7d
Merge pull request #44874 from Algunenano/time_fix 
Do not throw DB::Exception when folders do not exist
 2023-01-04 01:17:57 +01:00
Nikolai Kochetov
c51f5e6fac Fix typo. 2023-01-03 18:41:37 +00:00
Nikolai Kochetov
da26f62a9b Fix right offset for reading LowCardinality dictionary from remote fs in case if right mark was in the middle of compressed block. 2023-01-03 18:19:51 +00:00
Dmitry Novik
a84318af5f
Merge branch 'master' into cleanup-resolved-function 2023-01-03 17:52:29 +01:00
Raúl Marín
0785bf6b29 Do not throw DB::Exception when folders do not exist 2023-01-03 17:05:54 +01:00
Nikolai Kochetov
6b1a697b12
Merge pull request #44850 from ClickHouse/respect-context-in-projection-optimization 
Save context created from projection analysis.
 2023-01-03 13:30:51 +01:00
Han Fei
7d7de3833b
Merge pull request #44651 from hanfei1991/hanfei/fix-async-insert-cleanup 
fix bug that async blocks cleanup not work
 2023-01-02 21:58:27 +01:00
Dmitry Novik
929300a721
Update MergeTreeData.cpp 2023-01-02 21:18:16 +01:00
Dmitry Novik
04fb410a40
Merge branch 'master' into parallel_parts_cleanup_with_zero_copy 2023-01-02 20:48:39 +01:00
Dmitry Novik
c646048af9
Merge pull request #44461 from ClickHouse/parts-removal-limit 
Add part removal limit for one CleanupThread iteration
 2023-01-02 20:04:19 +01:00
Alexey Milovidov
e855d3519a
Merge branch 'master' into refactoring-ip-types 2023-01-02 21:58:53 +03:00
Nikolai Kochetov
09325ff17e Store context created from projection analysis. 2023-01-02 17:00:48 +00:00
alesapin
6fc63878b4 Fix rare race which can lead to queue hang 2023-01-02 16:57:25 +01:00
Han Fei
eef0136415 add debug log 2023-01-02 11:59:44 +01:00
Larry Luo
7cc84c6743 refactor filter functions support 2022-12-31 18:05:07 -08:00
Larry Luo
71da1b1379 Updated inverted index to use ActionsDAG 2022-12-31 18:05:06 -08:00
Larry Luo
f0a2c06261 Fixed errors caused by merging. 2022-12-31 18:05:06 -08:00
HarryLeeIBM
d7a7de9601 Minor fixes for addressing reviews 2022-12-31 18:05:06 -08:00
HarryLeeIBM
6a5fef632d Added more tests and experimental flag, etc. 2022-12-31 18:05:01 -08:00
Harry-Lee
b43ddde50a Refactor full text search according to reviews 2022-12-31 18:03:14 -08:00
Larry Luo
00e4411740 Created FST namespace to hide FST details 2022-12-31 18:03:14 -08:00
Larry Luo
64f807f31a fixed an edge case that index column has no data 2022-12-31 18:03:14 -08:00
Larry Luo
da93210cc6 Updated MergeTreeConditionGinFilter due to SelectQueryInfo changes. 2022-12-31 18:03:14 -08:00
Larry Luo
a33232cb0a Clean up code 2022-12-31 18:03:14 -08:00
Larry Luo
7e5f311da9 Fix MemorySanitizer error 2022-12-31 18:03:14 -08:00
Larry Luo
0a47378e5f Added initializer for current segment 2022-12-31 18:03:14 -08:00
Larry Luo
514e17d037 Code clean up 2022-12-31 18:03:14 -08:00
Larry Luo
1d9bf9962c Code clean up 2022-12-31 18:03:14 -08:00
Larry Luo
657ce7c614 Cleanup code and tests 2022-12-31 18:03:13 -08:00
Larry Luo
3652740535 Code clean up 2022-12-31 18:03:13 -08:00
Larry Luo
0b63d6b5be Resolve conflicts and style errors. 2022-12-31 18:03:13 -08:00
HarryLeeIBM
bc8be368bb Fix coding styles 2022-12-31 18:03:13 -08:00
Larry Luo
91b79b26e3 Use DataPartStorage in GinIndexStore 2022-12-31 18:03:02 -08:00
Harry-Lee
4b29c02f35 Remove useless FUNCTION_LIKE 2022-12-31 18:00:17 -08:00
HarryLeeIBM
e58f0ef3d2 Remove roaring bmp cleanup code which can cause memory leak 2022-12-31 18:00:17 -08:00
HarryLeeIBM
0839ff60ee Initial Checkin of full text search 2022-12-31 18:00:17 -08:00
Alexey Milovidov
50bb7f90d8
Merge pull request #44809 from ClickHouse/fix-race-in-mergetreedatapart 
Fix race in `system.parts` and `system.parts_columns`
 2023-01-01 01:09:46 +03:00
Han Fei
a036c6bc01
Merge branch 'master' into hanfei/fix-async-insert-cleanup 2022-12-31 20:52:16 +01:00
Alexey Milovidov
3a5cea8281 Fix race in system.parts and system.parts_columns 2022-12-31 15:07:26 +01:00
Alexey Milovidov
26eb2f57e1 Fix race in system.parts and system.parts_columns 2022-12-31 15:06:57 +01:00
Azat Khuzhin
e7bf2cc959 Add a comment for lack of fsync in MergedBlockOutputStream::finalizePartOnDisk() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-30 22:39:13 +01:00
Azat Khuzhin
9fb6004c92 Add missing fsync for small files (ttl.txt, columns.txt, ...) in mutations 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-30 22:22:39 +01:00
Nikolai Kochetov
758f30cc17 Cleanup. 2022-12-30 17:53:46 +00:00
Han Fei
454f40ab17 fix test 2022-12-30 16:30:33 +01:00
Alexey Milovidov
13b76ad760
Merge pull request #44719 from ClickHouse/fix_attach_thread_deadlock 
Fix deadlock in attach thread
 2022-12-30 13:56:18 +03:00
Nikolay Degterinsky
dfe93b5d82
Merge pull request #42284 from Algunenano/perf_experiment 
Performance experiment
 2022-12-30 03:14:22 +01:00
Alexander Gololobov
fbae502499
Merge pull request #42126 from ClickHouse/fix_invalid_number_of_rows_in_chunk_with_prewhere 
Fixing "Invalid number of rows in Chunk" with lightweight deletes
 2022-12-29 21:46:12 +01:00
alesapin
5dfefb43a3 Fix deadlock in attach thread 2022-12-29 18:52:20 +01:00
Dmitry Novik
81c0db3b67 Reuse FunctionBase for IndexHint 2022-12-29 15:47:43 +00:00
Dmitry Novik
bd480f18c1
Merge branch 'master' into parts-removal-limit 2022-12-29 16:27:54 +01:00
Alexander Gololobov
a02a631d51 Cleanups based on code review 2022-12-29 15:00:42 +01:00
Alexander Gololobov
7df137e460 Replaced asserts with logical errors 2022-12-29 14:33:11 +01:00
alesapin
02c3e708af
Merge pull request #44694 from Algunenano/old_part_improvement 
Improve handling of old parts
 2022-12-29 13:14:52 +01:00
alesapin
b1ed48a977
Merge pull request #44628 from ClickHouse/CurtizJ-fix-restart-after-quorum-insert 
Fix restart after quorum insert
 2022-12-29 12:51:54 +01:00
Alexander Gololobov
059ec6f747 Cleanups 2022-12-29 01:22:47 +01:00
Alexey Milovidov
4311e6c147
Merge pull request #44683 from CurtizJ/fix-flaky-test-4 
Fix test `01130_in_memory_parts`
 2022-12-29 01:53:24 +03:00
Raúl Marín
32d1662503 Improve handling of old parts 2022-12-28 21:22:40 +01:00
Anton Popov
8d2788445b fix style check 2022-12-28 19:29:56 +00:00
Raúl Marín
e915ce1e95 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-12-28 20:15:43 +01:00
alesapin
697617548c Tiny improvement 2022-12-28 18:59:24 +01:00
Anton Popov
bc122d368a Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-28 17:45:04 +00:00
Nikolai Kochetov
763fcdc939
Update MergeTreeSequentialSource.cpp 2022-12-28 18:17:40 +01:00
Alexander Gololobov
10a058d138 More cleanups in the logic of applying current step filter and final filter 2022-12-28 18:07:36 +01:00
Alexander Gololobov
fd5d328fae Test accumulating filters ignoring prewhere_info->need_filter flag 2022-12-28 18:07:36 +01:00
Alexander Gololobov
a7adc0a91b Cleanups 2022-12-28 18:07:36 +01:00
Alexander Gololobov
b22711baa3 Reset need_filter flag when filter is applied 2022-12-28 18:07:36 +01:00
Alexander Gololobov
ada6422985 Restored old logic for filling _part_offset 2022-12-28 18:07:36 +01:00
Alexander Gololobov
4cebc6f3a4 Cleanups 2022-12-28 18:07:36 +01:00
Alexander Gololobov
13e457c754 Cleanups 2022-12-28 18:07:36 +01:00
Alexander Gololobov
a1cdad4815 Fix for reading _part_offset along with newly added column that has only default value 2022-12-28 18:07:35 +01:00
Alexander Gololobov
ac1549f6b3 Skip filtering if there are no rows after optimize() 2022-12-28 18:07:35 +01:00
Alexander Gololobov
f273f8712d Avoid filtering same column in block_before_prewhere if it is present in the result 2022-12-28 18:07:35 +01:00
Alexander Gololobov
f3646248c5 Avoid unneeded work if all rows were filtered 2022-12-28 18:07:35 +01:00
Alexander Gololobov
75152ddabb Apply filter only if needed 2022-12-28 18:07:35 +01:00
Alexander Gololobov
a18850458c Test applying current filter at each step 2022-12-28 18:07:35 +01:00
Alexander Gololobov
29b5c4af07 Test dirty intermediate changes 2022-12-28 18:07:35 +01:00
Alexander Gololobov
c561acb774 Properly handle low cardinality column as prewhere filter 2022-12-28 18:07:35 +01:00
Alexander Gololobov
aa276b230b Don't need to save filter and rows_per_granule from previous step 2022-12-28 18:07:35 +01:00
Alexander Gololobov
c4a01cbd5b Fix for propely cleaning rows_per_granule_original between prewhere steps 2022-12-28 18:07:35 +01:00
Alexander Gololobov
abbb58107c Fix for "out of bound" in ColumnVector::insertRangeFrom called from shrink() 2022-12-28 18:07:35 +01:00
Alexander Gololobov
79874e8733 Fix for "Invalid number of rows in Chunk" 2022-12-28 18:07:35 +01:00
Alexander Gololobov
bdf51545f7 Added FilterWithCachedCount class instead of caching counts in filter_bytes_map 2022-12-28 18:07:35 +01:00
Azat Khuzhin
31a88d4eae Fix PREWHERE with row-level filters (when row filter is always true/false) 
In case of row-level filters optimized out, i.e. converted to
always true/false, it is possible for MergeTreeRangeReader to reuse
incorrect statistics for the filter (countBytesInResultFilter()), and
because of this it simply does not apply other filters, since it assume
that this filter does not need to filter anything.

Fixes: #40956
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-28 18:07:35 +01:00
Alexander Gololobov
d44392b366 Checking the fix for "Invalid number of rows in Chunk" 2022-12-28 18:07:35 +01:00