thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-12 01:12:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
109,599 Commits 9,297 Branches 1,782 Tags 2.1 GiB
6cd4507f2f
Commit Graph

11810 Commits

Author SHA1 Message Date
Han Fei
b7eef62458
Merge pull request #45491 from azat/dist/async-send-refactoring 
[RFC] Rewrite distributed sends to avoid using filesystem as a queue, use in-memory queue instead
 2023-03-06 12:32:33 +01:00
Vladimir C
1455e7602d
Merge pull request #47154 from azat/prewhere-columns-fix 2023-03-06 10:03:31 +01:00
Anton Popov
6f3e4d4137
Merge pull request #46118 from CurtizJ/fix-issues-with-sparse 
Randomize setting `ratio_of_defaults_for_sparse_serialization`
 2023-03-05 22:28:18 +01:00
Robert Schulze
10753481f9
Merge pull request #47242 from ClickHouse/rs/docs-index-support-has-hasall-hasany 
Docs: Update index support of has(), hasAny(), hasAll()
 2023-03-04 18:49:24 +01:00
Robert Schulze
11930ad685
Docs: Update index support of has(), hasAny(), hasAll() 2023-03-04 17:27:47 +00:00
Sema Checherinda
85178c9609
Merge pull request #47102 from CheSema/merge-mutate-do-thread-group 
do flushUntrackedMemory when context switches
 2023-03-03 20:23:25 +01:00
Anton Popov
9389cc29da better mutations of sparse columns 2023-03-03 16:52:01 +00:00
Alexander Tokmakov
cadaf06829
Merge pull request #46681 from ClickHouse/fix_insert_cancellation_in_native_protocol 
Fix queries cancellation when a client dies
 2023-03-03 16:27:20 +03:00
Konstantin Bogdanov
1bbf5acd47
Pass headers from StorageURL to WriteBufferFromHTTP (#46996) 
* Pass headers from StorageURL to WriteBufferFromHTTP

* Add a test

* Lint

* `time.sleep(1)`

* Start echo server earlier

* Add proper handling for mock server start

* Automatic style fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
 2023-03-03 13:55:52 +01:00
Maksim Kita
3cd88003dd
Merge pull request #47101 from kitaisreal/analyzer-planner-fixes-before-enable-by-default 
Analyzer Planner fixes before enable by default
 2023-03-02 22:16:39 +03:00
Sema Checherinda
7a9f4aab8e do not call adjustWithUntrackedMemory when untracked_memory is 0 2023-03-02 19:21:03 +01:00
Azat Khuzhin
167e4903a8 Fix concrete columns PREWHERE support 
This is the fix for the IStorage::supportedPrewhereColumns() API.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-02 18:51:23 +01:00
Alexander Tokmakov
c518f3690e Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-02 18:24:08 +01:00
Anton Popov
fdf8a418f5 better interface 2023-03-02 14:46:26 +00:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
Kruglov Pavel
ce8e49a9a0
Merge pull request #45693 from Avogar/fix-kafka-protobuf 
Fix formats parser resetting, test processing bad messages in kafka
 2023-03-02 11:50:45 +01:00
Alexey Milovidov
97c0f8a0e0
Merge pull request #47027 from amosbird/clang16 
Prepare to build with clang 16
 2023-03-02 02:10:03 +03:00
Anton Popov
c4a999ce29 Merge remote-tracking branch 'upstream/master' into HEAD 2023-03-01 19:33:16 +00:00
Maksim Kita
d39be3ac9c Fixed tests 2023-03-01 18:05:07 +01:00
Maksim Kita
51ee007e01 Fixed tests 2023-03-01 18:05:07 +01:00
alesapin
ebba37b18f
Merge pull request #47010 from ClickHouse/zero_copy_fetch 
Fix bug in zero-copy replication disk choice during fetch
 2023-03-01 15:55:00 +01:00
alesapin
b7370865a7
Merge pull request #47032 from ClickHouse/fix_message_for_zero_copy 
Better exception message when move was unseccessful
 2023-03-01 15:26:53 +01:00
Alexander Tokmakov
a97e15e36f Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-01 14:26:41 +01:00
alesapin
d72c310faa
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:54:22 +01:00
alesapin
8ff77c89eb
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:50:35 +01:00
alesapin
be5ef666ba
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:50:29 +01:00
alesapin
5de8a8e776
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:50:23 +01:00
Alexey Milovidov
fe446afb7d Increase max_replicated_merges_in_queue 2023-03-01 06:02:59 +01:00
Amos Bird
c0fb4ddc3e
Make thread analyzer happy 2023-03-01 11:07:36 +08:00
Azat Khuzhin
d06a4b50d6 Latest review fixes (variable naming: s/monitor/queue) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:48:07 +01:00
Azat Khuzhin
591fca57f3 Fix function names for opentelemetry spans in StorageDistributed 
Fixes: 02417_opentelemetry_insert_on_distributed_table
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
7063c20b3c Change noisy "Skipping send data over distributed table." message to test 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
16bfef3c8a Fix processing current_batch.txt on init 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
752d27d663 Fix lossing files during distributed batch send 
v2: do not suppress exceptions in case of errors
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
263c042c6a Fix opentelemetry for distributed batch sends 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
00115c6615 Rename readDistributedAsyncInsertHeader() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
a76d7b22c1 Use existing public methods of StorageDistributed in DistributedSink 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
e83699a8d3 Improve comment for DistributedAsyncInsertDirectoryQueue::initializeFilesFromDisk() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
e10fb142fd Fix race for distributed sends from disk 
Before it was initialized from disk only on startup, but if some INSERT
can create the object before, then, it will lead to the situation when
it will not be initialized.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
b5434eac3b Rename StorageDistributedDirectoryMonitor to DistributedAsyncInsertDirectoryQueue 
Since #44922 it is not a directory monitor anymore.

v2: Remove unused error codes
v3: Contains some header fixes due to conflicts with master
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
1c4659b8e7 Separate out Batch as DistributedAsyncInsertBatch (and also some helpers) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
33b13549ad Separate out DirectoryMonitorSource as DistributedAsyncInsertSource 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
325a7b2305 Separate out DistributedHeader as DistributedAsyncInsertHeader 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
22a39e29f7 Add a comment for StorageDistributedDirectoryMonitor::Batch::recovered 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
0c19a75a1c Add log message for batch restore 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
13a3e03f19 Introduce StorageDistributedDirectoryMonitor::Batch::{de,}serialize() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
f0a2efa630 Always manipulate with absolute file paths in DirectoryMonitor 
Otherwise on batch restore we can get the difference in file paths.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
ef1e642e05 Add log message to StorageDistributedDirectoryMonitor::addAndSchedule() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
16646c0923 Rename DirectoryMonitor::current_batch_file to current_file 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
3f892e52ab Revert "Revert "Merge pull request #44922 from azat/dist/async-INSERT-metrics"" 
This is the revert of revert since there will be follow up patches to
address the issues.

This reverts commit a55798626a.
 2023-02-28 22:33:36 +01:00