Commit Graph

5131 Commits

Author SHA1 Message Date
Robert Schulze
1ef2704539
Cosmetics 2023-01-20 11:39:23 +00:00
Robert Schulze
463cc843de
"segment file" --> "segment metadata file" 2023-01-20 11:26:22 +00:00
Robert Schulze
58df3953bb
Move some code around (no other changes) 2023-01-20 11:24:23 +00:00
Robert Schulze
3267ac2787
Prefix more typedefs in DB namespace with "Gin" 2023-01-20 11:19:07 +00:00
Robert Schulze
919b67f117
Cosmetics 2023-01-20 11:15:28 +00:00
Robert Schulze
98e117dca6
SegmentDictionary --> GinSegmentDictionary, also move typedef 2023-01-20 11:09:49 +00:00
Robert Schulze
908fa83f72
Move some typedefs around 2023-01-20 11:08:19 +00:00
Robert Schulze
44618927f9
Inline two short methods + uppercase 2023-01-20 11:04:35 +00:00
Robert Schulze
5c3cc5283f
"term dictionary" --> "dictionary" 2023-01-20 10:53:41 +00:00
Robert Schulze
be936b257c
Make version enum private 2023-01-20 10:48:43 +00:00
Robert Schulze
0653f86de9
Various cosmetic cleanups 2023-01-20 10:45:35 +00:00
Frank Chen
f867c4c206 Add an overriden fromCurrentException to simply the code 2023-01-20 16:43:10 +08:00
Frank Chen
38799eb852 Merge two methods as one 2023-01-20 16:35:28 +08:00
Frank Chen
be8ceeb4bb Save stack trace in part_log 2023-01-20 16:22:40 +08:00
Frank Chen
1fe9bde594 Set replica name to zero copy exclusive lock 2023-01-20 15:48:33 +08:00
Frank Chen
fb8452aa49 Add last_exception_time to replication_queue 2023-01-20 15:20:57 +08:00
kssenii
4ce8950712 Minor changes 2023-01-19 17:53:10 +01:00
larryluogit
52ae33dba7
Merge branch 'master' into ftsearch 2023-01-19 11:34:11 -05:00
Sema Checherinda
ae1dfb9ce5
Update src/Storages/MergeTree/MergeTreeData.cpp
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-01-18 15:21:11 +01:00
Han Fei
e51123c9b0 fix data race 2023-01-18 13:11:07 +01:00
Sergei Trifonov
c443c1ece0
Merge branch 'master' into hanfei/async-cache 2023-01-18 00:19:49 +01:00
Alexander Tokmakov
5cd90c1a3e Merge branch 'master' into exception_message_patterns 2023-01-17 20:04:04 +01:00
Alexander Tokmakov
72e8615bec formatting of some exception messages 2023-01-17 20:03:56 +01:00
Alexander Tokmakov
8b13b85ea0
Merge pull request #44543 from ClickHouse/text_log_add_pattern
Add a column with a message pattern to text_log
2023-01-17 20:19:32 +03:00
Han Fei
8a74238fe0 improve 2023-01-17 15:47:52 +01:00
HarryLeeIBM
e7add8218f Addressed more review comments and ClangTidy errors 2023-01-17 06:29:13 -08:00
Alexander Tokmakov
522686f78b less empty patterns 2023-01-17 01:19:44 +01:00
Alexander Tokmakov
870cfcc36a less fmt::runtime usages 2023-01-17 00:11:59 +01:00
Sema Checherinda
90fa1ecd49 make that old_parts_lifetime=0 deletes files instantly at drop/truncate 2023-01-16 16:47:12 +01:00
Sema Checherinda
c51f4d7be1 do not merge over a gap with uotdate parts, delete empty parts with respect to old_parts_lifetime 2023-01-16 16:47:11 +01:00
Sema Checherinda
25e16388d7 better message in MergeTreeDataMergerMutator when parts intersect 2023-01-16 16:47:11 +01:00
Anton Popov
6863cd152f
Merge pull request #42181 from CurtizJ/optimize-loading-parts
Do not load inactive parts at startup
2023-01-16 14:38:50 +01:00
Han Fei
30a798182a Merge branch 'master' into hanfei/async-cache 2023-01-16 14:07:36 +01:00
Han Fei
5617f7f616 address comments 2023-01-15 22:51:10 +01:00
Han Fei
701dc88d6f
Merge branch 'master' into hanfei/support-advance-dedup 2023-01-15 19:46:28 +01:00
Han Fei
c859f8dbe5
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp
Co-authored-by: alesapin <alesapin@gmail.com>
2023-01-15 19:46:16 +01:00
Han Fei
bb2c0914e9
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp
Co-authored-by: alesapin <alesapin@gmail.com>
2023-01-15 19:46:09 +01:00
Alexander Tokmakov
d857d62a03 remove another set of macros 2023-01-13 20:34:31 +01:00
Alexander Tokmakov
2d7773fccc Merge branch 'master' into text_log_add_pattern 2023-01-13 20:33:46 +01:00
Han Fei
ed49ebf01a update setting explain 2023-01-13 20:26:08 +01:00
Han Fei
2fb2f503e3 Update src/Storages/MergeTree/MergeTreeSettings.h
Co-authored-by: Sergei Trifonov <svtrifonov@gmail.com>
2023-01-13 20:20:08 +01:00
Han Fei
9e99c7e116 Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp
Co-authored-by: Sergei Trifonov <svtrifonov@gmail.com>
2023-01-13 20:19:13 +01:00
Han Fei
a258a39eb1 Merge branch 'master' into hanfei/async-cache 2023-01-13 20:17:58 +01:00
Anton Popov
487de70d01 fix locking at loading outdated data parts 2023-01-13 17:05:32 +00:00
Anton Popov
71188c22ee fix race on 'relative_data_path' 2023-01-13 12:19:41 +00:00
Alexander Tokmakov
51d94314d6
Merge pull request #45235 from ClickHouse/more_verbose_logs_about_replication_log_entries
More verbose logs about replication log entries
2023-01-13 15:05:21 +03:00
Alexander Tokmakov
d9b8cf9c04 more verbose logs about replication log entries 2023-01-12 20:32:39 +01:00
serxa
693489a8ad review fixes 2023-01-12 15:51:04 +00:00
Sergei Trifonov
12d8543578
Merge branch 'master' into cancellable-mutex-integration 2023-01-12 16:03:49 +01:00
Anton Popov
ca80240d84 Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-12 12:13:45 +00:00
Anton Popov
8bdf63f8e5
Merge pull request #45191 from ClickHouse/fix-ip-part-name
Fix part ID generation for IP types for backward compatibility
2023-01-12 12:15:06 +01:00
Igor Nikonov
edbcb24da0
Merge pull request #44954 from ClickHouse/fix-insert-delay
Fix: insert delay calculation
2023-01-12 09:43:00 +01:00
HarryLeeIBM
6d576cf157 Added more documentation 2023-01-11 13:40:20 -08:00
Yakov Olkhovskiy
9c4ea5a16b fix part ID generation for IP types for backward compatibility 2023-01-11 18:51:37 +00:00
Alexander Tokmakov
0fba3d6d81
Merge pull request #44468 from ClickHouse/fix_get_part_name
Fix incorrect usages of `getPartName()`
2023-01-11 16:24:02 +03:00
Igor Nikonov
1d002e45a9 Fix test & review comments 2023-01-11 12:49:01 +00:00
Nikolai Kochetov
5e7a6ac619
Merge pull request #45122 from ClickHouse/revert-45121-revert-44653-custom-reading-for-mutation
Revert "Revert "Custom reading for mutation""
2023-01-11 12:37:32 +01:00
Nikolai Kochetov
d0c1079242
Merge pull request #45133 from ClickHouse/fix-additinal-tables-filter-with-count-projection
Fix additional_table_filters with minmax/count projection.
2023-01-11 12:25:40 +01:00
Anton Popov
85734bb321 fix build 2023-01-11 08:32:33 +00:00
Anton Popov
6c4f53bdca Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-11 01:00:57 +00:00
HarryLeeIBM
150f9b48a4 Fix functional test for full text search 2023-01-10 14:35:06 -08:00
Igor Nikonov
543221b8b9 Merge remote-tracking branch 'origin/master' into fix-insert-delay 2023-01-10 22:15:41 +00:00
Igor Nikonov
e75df5e76b Test
+ fix some review comments
2023-01-10 22:14:54 +00:00
Alexander Tokmakov
881b17492f Merge branch 'master' into fix_get_part_name 2023-01-10 21:39:35 +01:00
Alexander Tokmakov
255881a2ff
Merge pull request #44466 from ClickHouse/parallel_parts_cleanup_with_zero_copy
Parallel parts cleanup with zero copy replication
2023-01-10 22:44:39 +03:00
Ilya Yatsishin
0efceae6ce
Merge pull request #44917 from azat/mt-fix-log-message
Fix logging message in MergeTreeDataMergerMutator (about merged parts)
2023-01-10 19:49:31 +01:00
Nikolai Kochetov
9a81f27fb2 Fix additional_table_filters with minmax/count projection. 2023-01-10 17:32:26 +00:00
HarryLeeIBM
bc48b17817 Addressed more review comments. 2023-01-10 08:26:27 -08:00
Nikolai Kochetov
4673b3fe1d
Revert "Revert "Custom reading for mutation"" 2023-01-10 16:31:01 +01:00
Alexander Tokmakov
c8ec130be4
Revert "Custom reading for mutation" 2023-01-10 17:51:30 +03:00
Han Fei
bef6168c05 support cache for async inserts block ids 2023-01-10 13:19:12 +01:00
Anton Popov
b9bf92fa5d check for network errors while loading parts 2023-01-10 11:55:08 +00:00
Nikolai Kochetov
11418963c0
Merge pull request #44653 from ClickHouse/custom-reading-for-mutation
Custom reading for mutation
2023-01-10 12:16:24 +01:00
alesapin
616e37ba43
Merge pull request #44847 from ClickHouse/fix_rare_race
Fix rare race which can lead to queue hang
2023-01-10 12:09:10 +01:00
Nikolai Kochetov
3e430b1b6e Add comments. Remove unused flag from Context. 2023-01-09 19:52:11 +00:00
Anton Popov
1e5d6e44a5 fix deadlock at shutdown 2023-01-09 13:07:46 +00:00
Sergei Trifonov
81d2ea30ba
Merge branch 'master' into cancellable-mutex-integration 2023-01-07 19:37:46 +01:00
Anton Popov
1f32ffedf8
Merge pull request #43221 from ClickHouse/refactoring-ip-types
Replace domain IP types (IPv4, IPv6) with native
2023-01-07 12:01:21 +01:00
Alexey Milovidov
9fcd111724
Merge pull request #45003 from amosbird/issue_44979
Fix minmax_count_projection with _partition_value
2023-01-07 03:36:56 +03:00
serxa
15bb127b01 replace every std::shared_mutex with DB::FastSharedMutex 2023-01-06 23:35:38 +00:00
Amos Bird
200d40c53d
Fix minmax_count_projection with _partition_value
Check if virtual column `_partition_value` can be used when building
minmax_count_projection block.
2023-01-07 05:38:28 +08:00
Dmitry Novik
9d44c7a737
Merge pull request #44541 from ClickHouse/cleanup-resolved-function
Followup #43761
2023-01-06 18:09:32 +01:00
alesapin
da83e97b4c Fix startup issue 2023-01-06 15:36:53 +01:00
Anton Popov
b25f875674
Merge pull request #44875 from ClickHouse/fix-another-one-cannot-read-all-data-for-lc-dict-error
Fix right offset for reading LowCardinality dictionary from remote fs
2023-01-06 15:24:36 +01:00
alesapin
b20f07b2cc Merge branch 'master' into fix_rare_race 2023-01-06 14:20:08 +01:00
Anton Popov
fcdbc8123d better loading of outdated parts 2023-01-06 11:59:54 +00:00
Alexey Milovidov
b88be9df8c
Merge pull request #44956 from ClickHouse/fix_read_result_consistency_check_on_exception
Do not check read result consistency when unwinding
2023-01-06 06:22:42 +03:00
Alexey Milovidov
a205b41c56
Merge pull request #44947 from Algunenano/background_pool_size_logical
Change error code on invalid background_pool_size config
2023-01-06 03:19:40 +03:00
Alexander Gololobov
1b31b8bb91 Do not check read result consistency when unwinding 2023-01-06 00:26:08 +01:00
Igor Nikonov
a67afdff6a Fix: insert delay calculation 2023-01-05 22:43:41 +00:00
Dmitry Novik
ebead611b6
Merge branch 'master' into cleanup-resolved-function 2023-01-05 19:27:08 +01:00
Raúl Marín
b0715ec5eb Change error code on invalid background_pool_size config 2023-01-05 18:47:02 +01:00
Anton Popov
ed00399eb6 Merge remote-tracking branch 'upstream/master' into HEAD 2023-01-05 14:12:02 +00:00
Larry Luo
f2bc5bb335 Added initializer 2023-01-05 05:41:29 -08:00
Igor Nikonov
2a105d1862 Fix stupid mistake - max() -> min() 2023-01-05 12:38:27 +00:00
Igor Nikonov
db46267a95 Fast fix: force upper bound for time to delay INSERT 2023-01-05 12:38:27 +00:00
Larry Luo
fd0c6cccc8 Fixed style error 2023-01-04 20:08:28 -08:00
Larry Luo
7a4d499cb1 Added density to control Inverted index size 2023-01-04 19:42:45 -08:00
Azat Khuzhin
d06ac2d468 Fix logging message in MergeTreeDataMergerMutator (about merged parts)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-04 19:07:01 +01:00
Yakov Olkhovskiy
7a5a36cbed
Merge branch 'master' into refactoring-ip-types 2023-01-04 11:11:06 -05:00
alesapin
2cd7247619 Merge branch 'master' into fix_rare_race 2023-01-04 15:06:20 +01:00
Kruglov Pavel
9056925c7d
Merge pull request #44874 from Algunenano/time_fix
Do not throw DB::Exception when folders do not exist
2023-01-04 01:17:57 +01:00
Nikolai Kochetov
c51f5e6fac Fix typo. 2023-01-03 18:41:37 +00:00
Nikolai Kochetov
da26f62a9b Fix right offset for reading LowCardinality dictionary from remote fs in case if right mark was in the middle of compressed block. 2023-01-03 18:19:51 +00:00
Dmitry Novik
a84318af5f
Merge branch 'master' into cleanup-resolved-function 2023-01-03 17:52:29 +01:00
Raúl Marín
0785bf6b29 Do not throw DB::Exception when folders do not exist 2023-01-03 17:05:54 +01:00
Nikolai Kochetov
6b1a697b12
Merge pull request #44850 from ClickHouse/respect-context-in-projection-optimization
Save context created from projection analysis.
2023-01-03 13:30:51 +01:00
Han Fei
7d7de3833b
Merge pull request #44651 from hanfei1991/hanfei/fix-async-insert-cleanup
fix bug that async blocks cleanup not work
2023-01-02 21:58:27 +01:00
Dmitry Novik
929300a721
Update MergeTreeData.cpp 2023-01-02 21:18:16 +01:00
Dmitry Novik
04fb410a40
Merge branch 'master' into parallel_parts_cleanup_with_zero_copy 2023-01-02 20:48:39 +01:00
Dmitry Novik
c646048af9
Merge pull request #44461 from ClickHouse/parts-removal-limit
Add part removal limit for one CleanupThread iteration
2023-01-02 20:04:19 +01:00
Alexey Milovidov
e855d3519a
Merge branch 'master' into refactoring-ip-types 2023-01-02 21:58:53 +03:00
Nikolai Kochetov
09325ff17e Store context created from projection analysis. 2023-01-02 17:00:48 +00:00
alesapin
6fc63878b4 Fix rare race which can lead to queue hang 2023-01-02 16:57:25 +01:00
Han Fei
eef0136415 add debug log 2023-01-02 11:59:44 +01:00
Larry Luo
7cc84c6743 refactor filter functions support 2022-12-31 18:05:07 -08:00
Larry Luo
71da1b1379 Updated inverted index to use ActionsDAG 2022-12-31 18:05:06 -08:00
Larry Luo
f0a2c06261 Fixed errors caused by merging. 2022-12-31 18:05:06 -08:00
HarryLeeIBM
d7a7de9601 Minor fixes for addressing reviews 2022-12-31 18:05:06 -08:00
HarryLeeIBM
6a5fef632d Added more tests and experimental flag, etc. 2022-12-31 18:05:01 -08:00
Harry-Lee
b43ddde50a Refactor full text search according to reviews 2022-12-31 18:03:14 -08:00
Larry Luo
00e4411740 Created FST namespace to hide FST details 2022-12-31 18:03:14 -08:00
Larry Luo
64f807f31a fixed an edge case that index column has no data 2022-12-31 18:03:14 -08:00
Larry Luo
da93210cc6 Updated MergeTreeConditionGinFilter due to SelectQueryInfo changes. 2022-12-31 18:03:14 -08:00
Larry Luo
a33232cb0a Clean up code 2022-12-31 18:03:14 -08:00
Larry Luo
7e5f311da9 Fix MemorySanitizer error 2022-12-31 18:03:14 -08:00
Larry Luo
0a47378e5f Added initializer for current segment 2022-12-31 18:03:14 -08:00
Larry Luo
514e17d037 Code clean up 2022-12-31 18:03:14 -08:00
Larry Luo
1d9bf9962c Code clean up 2022-12-31 18:03:14 -08:00
Larry Luo
657ce7c614 Cleanup code and tests 2022-12-31 18:03:13 -08:00
Larry Luo
3652740535 Code clean up 2022-12-31 18:03:13 -08:00
Larry Luo
0b63d6b5be Resolve conflicts and style errors. 2022-12-31 18:03:13 -08:00
HarryLeeIBM
bc8be368bb Fix coding styles 2022-12-31 18:03:13 -08:00
Larry Luo
91b79b26e3 Use DataPartStorage in GinIndexStore 2022-12-31 18:03:02 -08:00
Harry-Lee
4b29c02f35 Remove useless FUNCTION_LIKE 2022-12-31 18:00:17 -08:00
HarryLeeIBM
e58f0ef3d2 Remove roaring bmp cleanup code which can cause memory leak 2022-12-31 18:00:17 -08:00
HarryLeeIBM
0839ff60ee Initial Checkin of full text search 2022-12-31 18:00:17 -08:00
Alexey Milovidov
50bb7f90d8
Merge pull request #44809 from ClickHouse/fix-race-in-mergetreedatapart
Fix race in `system.parts` and `system.parts_columns`
2023-01-01 01:09:46 +03:00
Han Fei
a036c6bc01
Merge branch 'master' into hanfei/fix-async-insert-cleanup 2022-12-31 20:52:16 +01:00
Alexey Milovidov
3a5cea8281 Fix race in system.parts and system.parts_columns 2022-12-31 15:07:26 +01:00
Alexey Milovidov
26eb2f57e1 Fix race in system.parts and system.parts_columns 2022-12-31 15:06:57 +01:00
Azat Khuzhin
e7bf2cc959 Add a comment for lack of fsync in MergedBlockOutputStream::finalizePartOnDisk()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-30 22:39:13 +01:00
Azat Khuzhin
9fb6004c92 Add missing fsync for small files (ttl.txt, columns.txt, ...) in mutations
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-30 22:22:39 +01:00
Nikolai Kochetov
758f30cc17 Cleanup. 2022-12-30 17:53:46 +00:00
Han Fei
454f40ab17 fix test 2022-12-30 16:30:33 +01:00
Alexey Milovidov
13b76ad760
Merge pull request #44719 from ClickHouse/fix_attach_thread_deadlock
Fix deadlock in attach thread
2022-12-30 13:56:18 +03:00
Nikolay Degterinsky
dfe93b5d82
Merge pull request #42284 from Algunenano/perf_experiment
Performance experiment
2022-12-30 03:14:22 +01:00
Alexander Gololobov
fbae502499
Merge pull request #42126 from ClickHouse/fix_invalid_number_of_rows_in_chunk_with_prewhere
Fixing "Invalid number of rows in Chunk" with lightweight deletes
2022-12-29 21:46:12 +01:00
alesapin
5dfefb43a3 Fix deadlock in attach thread 2022-12-29 18:52:20 +01:00
Dmitry Novik
81c0db3b67 Reuse FunctionBase for IndexHint 2022-12-29 15:47:43 +00:00