thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-30 03:22:14 +00:00
Code Issues Packages Projects Releases Wiki Activity
101,453 Commits 9,189 Branches 1,782 Tags 2.2 GiB
177cbbac4b
Commit Graph

101453 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Azat Khuzhin
177cbbac4b Avoid race condition for updating system.distribution_queue values 
Previously it was possible to have a race while updating
files_count/bytes_count, since INSERT updates it those counters from one
thread and the same metrics are updated from filesystem in a separate
thread, and even though the access is synchronized with the mutex it
avoids the race only for accessing the variables not the logical race,
since it is possible that getFiles() from a separate thread will
increment counters and later addAndSchedule() will increment them again.

Here you can find an example of this race [1].

  [1]: https://pastila.nl/?00950e00/41a3c7bbb0a7e75bd3f2922c58b02334

Note, that I analyzed logs from production system with lots of async
Distributed INSERT and everything is OK there, even though the logs
contains the following:

    2022.11.20 02:21:15.459483 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Files set to 35 (was 34)
    2022.11.20 02:21:15.459515 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Bytes set to 4035418 (was 3929008)
    2022.11.20 02:21:15.819488 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Files set to 1 (was 2)
    2022.11.20 02:21:15.819502 [ 11528 ] {} <Trace> v21.dist_out.DirectoryMonitor: Bytes set to 190072 (was 296482)

As you may see it first increases the counters and next update
decreases (and 4035418-3929008 == 296482-190072)

Refs: #23885
Reported-by: @tavplubix
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-20 14:13:01 +01:00
Nikita Taranov
0b4e643c27
Add check to CompressionCodecDelta (#43255) 
* Add check to CompressionCodecDelta

* Apply suggestions from code review

* Update src/Compression/CompressionCodecDelta.cpp
 2022-11-19 14:16:14 +01:00
alesapin
93a4950dae
Merge pull request #43222 from zhongyuankai/optimize_ttl 
Priority is given to deleting completely expired Parts
 2022-11-19 13:51:51 +01:00
alesapin
3d497cb9d4
Merge pull request #43373 from ClickHouse/trying_to_split_checks 
Splitting checks in CI more
 2022-11-19 13:42:15 +01:00
zhongyuankai
a3a3261745
Merge branch 'master' into optimize_ttl 2022-11-19 16:32:26 +08:00
zhongyuankai
ce9b4866ea
Update MergeTreeDataMergerMutator.cpp 2022-11-19 16:31:47 +08:00
Alexey Milovidov
b4664cd525
Merge pull request #43365 from ClickHouse/update-security-on-tag 
Update SECURITY.md on new stable tags
 2022-11-18 22:39:17 +01:00
Alexey Milovidov
ec1c30d63e
Merge pull request #43375 from vitaliyf/docs-zookeeper-tips 
docs: Clarified ZooKeeper tips comments.
 2022-11-18 22:36:36 +01:00
alesapin
c7d648174b
Merge pull request #43328 from ClickHouse/better-format-version-check 
Check content of `format_version` file in `MergeTreeData`
 2022-11-18 22:34:37 +01:00
Kruglov Pavel
82d6f5357c
Merge pull request #43156 from azat/dist/insert-apply-timeouts 
Apply connection timeouts settings for Distributed async INSERT from the query
 2022-11-18 21:27:57 +01:00
alesapin
f12309ac8e Fix finish check 2022-11-18 19:21:37 +01:00
Vitaliy
9e38b14fc1
docs: Clarified ZooKeeper tips comments. 2022-11-18 12:08:07 -05:00
alesapin
2dbf07f58a Splitting checks 2022-11-18 17:38:17 +01:00
Sergei Trifonov
d05223e70b
Merge pull request #43335 from ClickHouse/revert-43306-revert-43014-disk-s3-throttler 
Revert "Revert "S3 request per second rate throttling""
 2022-11-18 16:22:30 +01:00
Antonio Andelic
25697fd87d
Merge pull request #43289 from ClickHouse/fix-another-race-rmt 
Fix race on write in `ReplicatedMergeTree`
 2022-11-18 16:09:50 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info 
Improve masking sensitive info
 2022-11-18 15:37:50 +01:00
Mikhail f. Shiryaev
11a5103dfb
Add a comment how the SECURITY.md is generated 2022-11-18 15:10:32 +01:00
Mikhail f. Shiryaev
609986f8b3
Merge pull request #43349 from ClickHouse/auto/v22.11.1.1360-stable 
Update version_date.tsv and changelogs after v22.11.1.1360-stable
 2022-11-18 13:58:09 +01:00
Mikhail f. Shiryaev
e3e2ff5d5d
Update SECURITY.md on new tags 2022-11-18 13:46:00 +01:00
Mikhail f. Shiryaev
ed2957adbf
Regenerate SECURITY.md with the new generator 2022-11-18 13:45:59 +01:00
Mikhail f. Shiryaev
318a31ab6e
Rewrite SECURITY.md generator to python 2022-11-18 13:45:58 +01:00
Mikhail f. Shiryaev
de562a654b
Merge pull request #43346 from ClickHouse/improve-insall-template 
Suggest users installation troubleshooting
 2022-11-18 12:28:44 +01:00
Nikolay Degterinsky
e7e21adf17
Merge pull request #43207 from freedomDR/improve_unhex 
unhex support FixedString arguments
 2022-11-18 03:09:58 +01:00
Nikolay Degterinsky
a97afbb78b
Merge pull request #43282 from evillique/fix-identifier-cloning 
Fix cloning of ASTIdentifier
 2022-11-18 02:38:20 +01:00
Anton Popov
9569113169
Merge pull request #43269 from CurtizJ/fix-fuzzer-bad-cast 
Fix race in `IColumn::dumpStructure`
 2022-11-18 01:31:28 +01:00
Mikhail f. Shiryaev
aa208fdc02
Merge pull request #43348 from ClickHouse/22.12-prepare 
Update version after release
 2022-11-17 23:21:59 +01:00
Mikhail f. Shiryaev
7a39a513bf
Generate SECURITY.md 2022-11-17 23:21:36 +01:00
robot-clickhouse
77500b7065 Update version_date.tsv and changelogs after v22.11.1.1360-stable 2022-11-17 21:20:45 +00:00
Mikhail f. Shiryaev
f942ab1bd2
Update version to 22.12.1.1 2022-11-17 22:15:12 +01:00
Alexey Milovidov
a5821f89cc
Merge pull request #43345 from ClickHouse/nickitat-patch-8 
Add #43072
 2022-11-17 22:13:26 +01:00
Mikhail f. Shiryaev
1d181b8939
Suggest users installation troubleshooting 2022-11-17 21:28:18 +01:00
Nikita Taranov
b826772a6c
Add #43072 2022-11-17 21:03:18 +01:00
Mikhail f. Shiryaev
3727f21ac2
Merge pull request #43019 from ClickHouse/release-only-on-marked-commits 
Allow release only from ready commits
 2022-11-17 20:51:57 +01:00
Alexey Milovidov
0d211ed198
Merge pull request #43316 from ClickHouse/disable-delta-hudi-readonly 
Disable `deltaLake` and `hudi` table functions in readonly mode
 2022-11-17 19:27:43 +01:00
Vitaly Baranov
86e486d051 Remove unused argument hilite in formatWithSecretHidden(). 2022-11-17 18:16:33 +01:00
Vitaly Baranov
050df6ac7f Move InDepthNodeVisitor.h back to src/Interpreters. 2022-11-17 18:16:32 +01:00
Vitaly Baranov
6e0762f1fa Move function findFunctionSecretArguments() to ASTFunction.cpp 2022-11-17 18:16:28 +01:00
serxa
e6270065b9 test: .sql -> .sh to avoid AST fuzzing 2022-11-17 17:01:00 +00:00
Sergei Trifonov
f2f0676bcc
Revert "Revert "S3 request per second rate throttling"" 2022-11-17 17:35:04 +01:00
Vitaly Baranov
2b9ed89721 Rename formatWithHiddenSecrets() -> formatWithSecretsHidden(). 2022-11-17 17:30:37 +01:00
Alexey Milovidov
ef1278f5c4
Merge pull request #43313 from ClickHouse/pmod 
`pmod`: compatibility with Spark, better documentation
 2022-11-17 17:13:45 +01:00
Antonio Andelic
d324e8cc3e Write to first non-readonly disk 2022-11-17 14:52:35 +00:00
robot-clickhouse
926b35b7c4 Automatic style fix 2022-11-17 14:46:25 +00:00
Antonio Andelic
d41af8b741 Check and compare content of format_version file 2022-11-17 13:41:36 +00:00
Igor Nikonov
00920b1aa9
Merge pull request #43117 from ClickHouse/igor/keeper_retries_enable_fault_injection_in_tests 
Enable keeper fault injection for inserts in functional tests
 2022-11-17 14:36:09 +01:00
Alexander Tokmakov
5d959ab6a0
Merge pull request #43287 from filimonov/patch-7 
Allow autoremoval of old parts if detach_not_byte_identical_parts enabled
 2022-11-17 16:11:02 +03:00
Nikita Taranov
7beb58b0cf
Optimize merge of uniqExact without_key (#43072) 
* impl for uniqExact

* rm unused (read|write)Text methods

* fix style

* small fixes

* impl for variadic uniqExact

* refactor

* fix style

* more agressive inlining

* disable if max_threads=1

* small improvements

* review fixes

* Revert "rm unused (read|write)Text methods"

This reverts commit a7e7480584.

* encapsulate is_able_to_parallelize_merge in Data

* encapsulate is_exact & argument_is_tuple in Data
 2022-11-17 13:19:02 +01:00
Vladimir C
a29c5b9e3a
Merge pull request #43232 from ClickHouse/vdimir/max_map_null_totals_bug 2022-11-17 13:10:05 +01:00
Sema Checherinda
8e21d2c777
Merge pull request #43285 from CheSema/ubsan-max-ast-depth 
check limits for an AST in select parser fuzzer
 2022-11-17 12:59:02 +01:00
Kruglov Pavel
a406557263
Merge branch 'master' into dist/insert-apply-timeouts 2022-11-17 12:51:47 +01:00