Commit Graph

106427 Commits

Author SHA1 Message Date
Azat Khuzhin
54ef885a61 Fsync all small files at once after mutation
Everything else is handled in MergedBlockOutputStream::finalizePart()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-24 14:08:37 +01:00
Azat Khuzhin
4b775ccdd5 Do fsync all files at once for fetched parts to decrease latency
For filessystems like ext4, fsync of one file will handle all operations
before, so this can be pretty time consuming.

And in case of you write multiple files in a loop, and at the end of
each iteration sync each file, then during writing of this file there
can be other operations in journal, and hence more work for fsync.

Let's call fsync for all files at once instead, like
MergedBlockOutputStream does.

Hope that keeping all file buffers till the end, will not cause troubles
(buffering and so forth).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-24 14:08:37 +01:00
Azat Khuzhin
b370e3faa5 Move fsync inside transaction callback in DataPartStorageOnDisk::rename()
Otherwise, it is useless.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-24 14:08:37 +01:00
SmitaRKulkarni
d387835774
Merge branch 'master' into 45486_Fix_flaky_test_for_disallowing_concurrent_backups_restores 2023-01-24 13:49:32 +01:00
DanRoscigno
72b1a65984 WIP 2023-01-24 07:45:59 -05:00
Anton Popov
5c0307bc6a fix race in StorageURL and StorageHDFS 2023-01-24 12:34:43 +00:00
vdimir
d8d3da951e
Ignore utf errors in clickhouse-test reportLogStats 2023-01-24 12:30:19 +00:00
Vladimir C
f818e3070d
Merge pull request #45391 from ClickHouse/vdimir/join_prewhere_bug_44062 2023-01-24 13:23:31 +01:00
Antonio Andelic
39cf1f4418
Merge branch 'master' into keeper-fix-race-asio-listener 2023-01-24 13:21:48 +01:00
Alexander Tokmakov
c25e5ff8de
Merge pull request #45545 from ClickHouse/revert_45439
Revert "Merge pull request #45493 from azat/fix-detach"
2023-01-24 15:21:33 +03:00
Mikhail f. Shiryaev
b8474fd17f
Merge pull request #45466 from ClickHouse/cherrypick-search
CherryPick: Fix a wrong staring search date
2023-01-24 13:20:16 +01:00
Mikhail f. Shiryaev
d32a564ff0
Merge pull request #45421 from ClickHouse/chrm-releases
Get rid of artifactory in favor of r2 + ch-repos-manager
2023-01-24 13:17:48 +01:00
Nikolai Kochetov
e4c0608a01
Merge pull request #45150 from ClickHouse/fix-ip-function-mod
Binary arithmetic for IPv4
2023-01-24 13:11:02 +01:00
Vladimir C
0403f801a8
Merge pull request #44238 from ClickHouse/vdimir/grace_join_mem 2023-01-24 13:02:11 +01:00
Maksim Kita
2f1092e6d2
Merge pull request #45425 from ClickHouse/fix-ipv6-parser
Fix ipv6 parser
2023-01-24 14:59:47 +03:00
Igor Nikonov
7108189b45 Merge remote-tracking branch 'origin/master' into revert-45414-revert-43905-igor/remove_redundant_order_by 2023-01-24 11:46:24 +00:00
Igor Nikonov
ba5fff2a8a Disable the optimization to avoid sort.xml perf test fail in other PRs 2023-01-24 11:42:56 +00:00
Alexander Tokmakov
00b3376a9d
Merge pull request #45516 from ClickHouse/tavplubix-patch-2
Update test_system_merges/test.py
2023-01-24 14:37:52 +03:00
Alexander Tokmakov
bfd2ee1918
Update stress 2023-01-24 14:36:29 +03:00
Alexander Tokmakov
9dc4f02d15 Revert "Merge pull request #45493 from azat/fix-detach"
This reverts commit a182a6b851, reversing
changes made to c47a29a089.
2023-01-24 12:20:37 +01:00
Antonio Andelic
ed8fc2d9b8 Merge branch 'master' into keeper-fix-race-asio-listener 2023-01-24 11:16:58 +00:00
Antonio Andelic
71f754b6f6 Fix race in NuRaft 2023-01-24 11:16:55 +00:00
Robert Schulze
574cab5d7e
Remove transitory parameter 2023-01-24 11:05:29 +00:00
Kseniia Sumarokova
ac1e7264fd
Merge pull request #45528 from ClickHouse/add-distinct-to-intersect
Add DISTINCT to INTERSECT and EXCEPT docs
2023-01-24 11:14:32 +01:00
Robert Schulze
aaf7653108
Merge remote-tracking branch 'origin/master' into block-non-float-gorilla-v2 2023-01-24 10:14:10 +00:00
Robert Schulze
9ff2bfcbf5
Merge remote-tracking branch 'origin/master' into inv-index-cleanup
src/Interpreters/GinFilter.cpp
	src/Interpreters/InterpreterCreateQuery.cpp
	src/Storages/MergeTree/MergeTreeData.cpp
	src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp
	src/Storages/MergeTree/MergeTreeIndexInverted.cpp
2023-01-24 10:09:42 +00:00
Alexey Milovidov
43eac58e62
Merge pull request #45430 from ClickHouse/tavplubix-patch-1
Disable parallel quorum inserts with deprecated *MergeTree syntax
2023-01-24 13:04:48 +03:00
Alexey Milovidov
dda6a69f9a
Merge branch 'master' into tavplubix-patch-1 2023-01-24 13:04:25 +03:00
Alexey Milovidov
54bc501d39
Update StorageReplicatedMergeTree.cpp 2023-01-24 13:03:28 +03:00
Alexey Milovidov
851f6bf910
Merge pull request #45448 from azat/fix-query-hang
Fix possible (likely distributed) query hung
2023-01-24 13:00:37 +03:00
Alexey Milovidov
116a7edf25
Merge pull request #45504 from ClickHouse/teepopen-timeout
Improve logging for TeePopen.timeout exceeded
2023-01-24 12:53:44 +03:00
Alexey Milovidov
cb1e8afda6
Merge pull request #45524 from kssenii/fix-cache-policies-getter
Add proper default to cache policy getter
2023-01-24 12:46:33 +03:00
Alexey Milovidov
e5ea600902
Merge pull request #45526 from ClickHouse/rfraposa-patch-2
Update link to Protobuf website in formats.md
2023-01-24 12:44:56 +03:00
Maksim Kita
4d947eaed6 Analyzer add test for CREATE TABLE AS SELECT 2023-01-24 10:15:21 +01:00
Smita Kulkarni
642f9ca549 Merge branch 'master' into 45486_Fix_flaky_test_for_disallowing_concurrent_backups_restores 2023-01-24 09:37:28 +01:00
Smita Kulkarni
9ae5ac2388 Moved concurrency checks inside functions - Updated backup/restore status when concurrent backups & restores are not allowed 2023-01-24 09:31:51 +01:00
avogar
3c5a81f9f9 Add unit test for recursive checkpoints 2023-01-24 01:20:13 +00:00
Anton Popov
8a1ea62aec fix race on cancelation of query in StorageS3 2023-01-24 01:12:01 +00:00
Alexander Tokmakov
9d13ca4a16 fix 2023-01-24 01:49:16 +01:00
ltrk2
ee5818f27b Fix the build with ENABLE_VECTORSCAN disabled 2023-01-23 16:43:03 -08:00
Alexander Tokmakov
6ddd76c5a0 fix 2023-01-24 01:30:26 +01:00
Alexander Tokmakov
c6910f39b9 fix 2023-01-24 01:11:58 +01:00
rfraposa
33770303a0 Add DISTINCT to INTERSECT and EXCEPT 2023-01-23 17:10:30 -07:00
Alexander Tokmakov
414693feb2 fixes 2023-01-24 00:46:03 +01:00
Kruglov Pavel
cd1cd904a7
Merge branch 'master' into tsv-csv-detect-header 2023-01-23 23:49:56 +01:00
Rich Raposa
429e93965c
Update formats.md
Google has a new website for Protocol Buffers. The old link expires on Jan 31, 2023
2023-01-23 15:42:35 -07:00
ltrk2
9710b8643c Refine the solution 2023-01-23 14:27:48 -08:00
Alexander Tokmakov
bb4c8e169f check number of parameters in format string 2023-01-23 23:16:16 +01:00
Anton Popov
b987ed5037
Merge pull request #45499 from CurtizJ/add-fields-system-formats
Add fields to table `system.formats`
2023-01-23 23:08:35 +01:00
Nikita Taranov
95d17d59fb
impl (#45289) 2023-01-23 22:55:52 +01:00