tavplubix
22de534fdc
Merge pull request #34297 from ClickHouse/fix_restarting_thread
...
Try to fix race between pullLogsToQueue and RestartingThread
2022-02-04 17:30:17 +03:00
Maksim Kita
ab696e6b59
Merge pull request #34310 from ClickHouse/fix-parallel-loading-parts
...
Fix parallel loading of data parts
2022-02-04 13:10:24 +01:00
Nikolai Kochetov
b92efed350
Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer
...
Revert "Revert "Add pool to WriteBufferFromS3""
2022-02-04 11:00:29 +03:00
Alexey Milovidov
78eebd5c7c
Fix parallel loading of data parts
2022-02-04 02:29:46 +03:00
Alexander Tokmakov
897e94c16c
make restarting thread less bad
2022-02-03 23:29:24 +03:00
Anton Popov
9b844c6b42
Merge pull request #32748 from CurtizJ/read-in-order-fixed-prefix
...
Support `optimize_read_in_order` if prefix of sorting key is already sorted
2022-02-03 18:17:08 +03:00
Nikolai Kochetov
a2a9c10617
Merge pull request #34221 from ClickHouse/revert-34201-revert-34189-less-logging-for-remote_fs_execute_merges_on_single_replica_time_threshold
...
Revert "Revert "Additionally check remote_fs_execute_merges_on_single_replica_time_threshold inside ReplicatedMergeTreeQueue""
2022-02-03 13:09:44 +03:00
Nikolai Kochetov
ea044fc6b2
Merge branch 'master' into revert-34212-revert-33291-add-pool-to-s3-write-buffer
2022-02-02 19:40:41 +03:00
Nikolai Kochetov
1117e6095d
Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
...
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2022-02-02 18:34:05 +03:00
Azat Khuzhin
15993cb13b
Add missing fmt::runtime() in MergeTreeBackgroundExecutor
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-02 11:37:53 +03:00
alesapin
47d538b52f
Merge pull request #34232 from ClickHouse/cancelled-merging-parts-error-message
...
Change severity of the "Cancelled merging parts" message in logs
2022-02-02 10:29:07 +03:00
Sergei Trifonov
68bc456830
Merge pull request #34223 from azat/bump-fmt
...
Bump fmtlib from 7.0.0 to 8.1.1
2022-02-02 00:03:25 +03:00
Nikolai Kochetov
dd81eff301
Fix tests.
2022-02-01 15:28:22 +00:00
Azat Khuzhin
5dfafd68a7
ReplicatedMergeTreeQueue: Fix fmt:: and reduce copy-paste of logging and out reason
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:04 +03:00
Azat Khuzhin
5be76bc969
Use proper fmt-like logging
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:04 +03:00
Azat Khuzhin
3b3635c6d5
Fix formatting error in logging messages
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:04 +03:00
Azat Khuzhin
743096a883
Use proper fmt:: like Exception ctor in DataPartsExchange
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:03 +03:00
Azat Khuzhin
bedf208cbd
Use fmt::runtime() for LOG_* for non constexpr
...
Here is oneliner:
$ gg 'LOG_\(DEBUG\|TRACE\|INFO\|TEST\|WARNING\|ERROR\|FATAL\)([^,]*, [a-zA-Z]' -- :*.cpp :*.h | cut -d: -f1 | sort -u | xargs -r sed -E -i 's#(LOG_[A-Z]*)\(([^,]*), ([A-Za-z][^,)]*)#\1(\2, fmt::runtime(\3)#'
Note, that I tried to do this with coccinelle (tool for semantic
patchin), but it cannot parse C++:
$ cat fmt.cocci
@@
expression log;
expression var;
@@
-LOG_DEBUG(log, var)
+LOG_DEBUG(log, fmt::runtime(var))
I've also tried to use some macros/templates magic to do this implicitly
in logger_useful.h, but I failed to do so, and apparently it is not
possible for now.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: manual fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:03 +03:00
Nikolai Kochetov
a6cc61bd14
Revert "Revert "Additionally check remote_fs_execute_merges_on_single_replica_time_threshold inside ReplicatedMergeTreeQueue""
2022-02-01 14:17:46 +03:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3""
2022-02-01 13:36:51 +03:00
alexey-milovidov
06477c2a7e
Update ReplicatedMergeTreeSink.cpp
2022-02-01 09:22:40 +01:00
Igor Nikonov
f4c0b64420
Clean up: insert_deduplication_token setting for INSERT statement
...
+ reduce number of allocations on replication merge tree path
+ bash test: move insert block settings into variable
Issue: ClickHouse#7461
2022-02-01 09:22:33 +01:00
alexey-milovidov
99392b5ca7
Merge pull request #13544 from amosbird/mdha
...
Multi-Disk auto-recovery.
2022-02-01 06:13:26 +03:00
Alexey Milovidov
7dbf0dede5
Change severity of the "Cancelled merging parts" message in logs
2022-02-01 05:55:07 +03:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3"
2022-02-01 05:49:40 +03:00
mergify[bot]
e229487817
Merge branch 'master' into mdha
2022-02-01 01:22:16 +00:00
Dmitry Novik
3a0ed0f8f5
Merge pull request #33324 from azat/buffer-different-structure
...
Simplify different block structure (i.e. after ALTER) support for Buffer
2022-02-01 02:19:49 +03:00
alexey-milovidov
15e4fe5c78
Revert "Additionally check remote_fs_execute_merges_on_single_replica_time_threshold inside ReplicatedMergeTreeQueue"
2022-02-01 01:51:39 +03:00
Amos Bird
ec7d367814
DiskLocal checker
...
Add DiskLocal checker so that ReplicatedMergeTree can recover data when some of its disks are broken.
2022-02-01 05:55:27 +08:00
alesapin
dd61d1c2de
Merge pull request #34172 from ClickHouse/fix_race_in_some_engines
...
Fix benign race condition for storage HDFS, S3, URL
2022-01-31 22:41:54 +03:00
alesapin
75d73d2785
Merge pull request #34139 from ClickHouse/fix_buf_s3_low_cardinality
...
Fix bug with bounded S3 reads and LowCardinality
2022-01-31 22:41:14 +03:00
Nikolai Kochetov
348d72266a
Merge pull request #34189 from ClickHouse/less-logging-for-remote_fs_execute_merges_on_single_replica_time_threshold
...
Additionally check remote_fs_execute_merges_on_single_replica_time_threshold inside ReplicatedMergeTreeQueue
2022-01-31 21:39:04 +03:00
Nikolai Kochetov
a207cdf28f
Additionally check remote_fs_execute_merges_on_single_replica_time_threshold inside ReplicatedMergeTreeQueue.
2022-01-31 17:53:28 +00:00
Nikolai Kochetov
321fa4a9e8
Merge pull request #33291 from ClickHouse/add-pool-to-s3-write-buffer
...
Add pool to WriteBufferFromS3
2022-01-31 19:37:40 +03:00
Maksim Kita
8513f20cfd
Merge pull request #34145 from kitaisreal/bitset-sort-performance-check
...
pdqsort performance check
2022-01-31 12:35:13 +01:00
alesapin
5230c59f05
Smaller scope
2022-01-31 14:03:06 +03:00
alesapin
1adc7538fe
Fix benign race condition for storage HDFS, S3, URL
2022-01-31 13:50:28 +03:00
tavplubix
d19e24f530
Merge pull request #34096 from ClickHouse/fix_race_merge_selecting_task
...
Fix race between mergeSelectingTask and queue reinitialization
2022-01-31 12:16:29 +03:00
Kruglov Pavel
a9d0beb7ae
Fix data race in StorageFile ( #34113 )
...
* Fix data race in StorageFile
* Update StorageFile.h
* Fix
2022-01-31 11:58:40 +03:00
alesapin
55c7936257
Fix incorrect range for index
2022-01-31 11:11:32 +03:00
Maksim Kita
5ef83deaa6
Update sort to pdqsort
2022-01-30 19:49:48 +00:00
alesapin
4f1b902342
Fix compact parts as well
2022-01-30 22:36:19 +03:00
alesapin
4bedcc19b5
Better invariants
2022-01-30 20:40:09 +03:00
alesapin
c237c03c50
Fix
2022-01-30 18:39:26 +03:00
alesapin
bf918892ac
More clear code with less getMark calls
2022-01-30 18:21:05 +03:00
alesapin
cb45a348f1
Merge branch 'master' into fix_buf_s3_low_cardinality
2022-01-30 17:30:55 +03:00
alesapin
3f3e90c8ba
Remove redundant code
2022-01-29 23:55:45 +03:00
alexey-milovidov
2b43bad923
Update MergeTreeReaderStream.cpp
2022-01-29 19:28:39 +03:00
alesapin
7ada8227cf
Fix bug with bounded S3 reads and LowCardinality
2022-01-29 18:28:40 +03:00
alexey-milovidov
6535b75322
Merge pull request #34001 from azat/memory-tracker-fix
...
Fix memory accounting for queries that uses < max_untracker_memory
2022-01-29 00:59:53 +03:00