thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-21 09:10:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
112,037 Commits 8,936 Branches 1,766 Tags 2 GiB
7638320fee
Commit Graph

112037 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Rich Raposa
7638320fee
Merge pull request #48437 from ClickHouse/style-fix 
Update replication.md
 2023-04-05 14:14:42 -06:00
robot-ch-test-poll2
bf003c7595
Merge pull request #48390 from Avogar/protobuf-tuple 
Allow write/read unnamed tuple as nested Message in Protobuf format
 2023-04-05 22:14:28 +02:00
Alexander Gololobov
0ebfdbdf69
Merge pull request #48413 from azat/tests/02352_lightweight_delete 
Fix possible flakiness of lightweight delete tests (due to index granularity randomization)
 2023-04-05 20:58:20 +02:00
Alexander Gololobov
7e1dfbb72e
Merge branch 'master' into tests/02352_lightweight_delete 2023-04-05 20:57:38 +02:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling 
Per-query/per-server throttling for remote IO/local IO/BACKUPs
 2023-04-05 19:32:37 +02:00
rfraposa
a016e8fcd1 Update replication.md 2023-04-05 11:30:59 -06:00
Robert Schulze
90a1a75424
Merge pull request #48412 from ClickHouse/rs/fix-ast-fuzzer 
AST fuzzer: Fix assertion in TopK serialization
 2023-04-05 19:16:50 +02:00
Kruglov Pavel
faf6d49f1b
Merge pull request #48356 from Avogar/fix-protobuf-abort 
Avoid abort in protobuf library in debug build
 2023-04-05 18:26:59 +02:00
Kruglov Pavel
9331c6c260
Merge pull request #48122 from Avogar/bson-more-types 
Support Enum output/input in BSONEachRow, allow all map key types and avoid extra calculations
 2023-04-05 18:26:19 +02:00
Nikolai Kochetov
48caf26b63
Merge pull request #48231 from ClickHouse/fix-possible-pipeline-stuck-on-partial-query-result 
Do not  partially cancel processors added from expand pipeline.
 2023-04-05 18:25:42 +02:00
Nikolai Kochetov
ab87cf7ff7
Merge branch 'master' into fix-possible-pipeline-stuck-on-partial-query-result 2023-04-05 18:25:09 +02:00
Vitaly Baranov
862a4d6fcd
Merge pull request #48142 from DimasKovas/json_functions_to_h_file 
Move FunctionsJSON implementation to header file
 2023-04-05 16:52:48 +02:00
Alexander Tokmakov
d7879c16e4
Merge pull request #48417 from ClickHouse/fix-test-keeper-snapshots 
Fix flaky `test_keeper_snapshots`
 2023-04-05 16:19:20 +03:00
Robert Schulze
62608f74d3
Merge pull request #48396 from ClickHouse/rs/clang-tidy-cleanup 
Cleanup mess in .clang-tidy
 2023-04-05 14:39:28 +02:00
Alexander Tokmakov
3b5ed9242c
Merge pull request #48391 from azat/fix-sync-replica-stuck 
Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION
 2023-04-05 15:20:46 +03:00
Kruglov Pavel
bd318950b3
Fix special build 2023-04-05 13:35:12 +02:00
Nikita Mikhaylov
7b526d3df9
Always communicate with local replica through loopback interface for parallel replicas (#48246) 2023-04-05 13:32:46 +02:00
Robert Schulze
1be81db885
Fix build, pt. II 2023-04-05 11:23:09 +00:00
Kruglov Pavel
8bd997d680
Update docs/en/interfaces/formats.md 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-04-05 11:57:41 +02:00
Kruglov Pavel
96a3307bda
Merge branch 'master' into fix-protobuf-abort 2023-04-05 11:57:18 +02:00
Kruglov Pavel
8d6b709f1a
Merge pull request #48257 from Avogar/capnp-more-types 
Support more types in CapnProto format, allow integer conversions
 2023-04-05 11:56:46 +02:00
avogar
f46f098c78 Better 2023-04-05 09:55:49 +00:00
Vladimir C
1f22fca3c3
Merge pull request #46428 from bigo-sg/quantile_gk 2023-04-05 11:25:48 +02:00
Antonio Andelic
f7586e38ed Fix flaky test_keeper_snapshots 2023-04-05 08:56:01 +00:00
Azat Khuzhin
4008155a6e Add max_ prefix for backup_bandwidth settings 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 10:54:25 +02:00
Robert Schulze
0a71656ee6
Fix build 2023-04-05 08:45:59 +00:00
Azat Khuzhin
8f06972dc5 Set index granularity settings for other two lightweight delete tests 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 10:35:11 +02:00
Azat Khuzhin
7f1569dd6e Fix 02352_lightweight_delete flakiness (due to index granularity randomization) 
CI: https://s3.amazonaws.com/clickhouse-test-reports/48242/c8da660d96bea153fa0c1dc8fbdb4a560e139a56/stateless_tests__debug__[4/5].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 10:35:11 +02:00
Robert Schulze
598b050628
AST Fuzzer: Fix assertion in TopK serialization 
Problem:
  https://s3.amazonaws.com/clickhouse-test-reports/0/fa5b2bd4a5b02336bca8837c473a7124f8ecedf2/fuzzer_astfuzzerasan/report.html

The new assertion in the Varint code was introduced with (*). It rejects
values whose serialization cannot be deserialized (and this behavior
cannot be changed due to historical reasons). Such values should be
exceptionally rare in practice but AST fuzzer managers to trigger them.

The fix is similar to (**): Bypass the check by limiting the value to
the maximum allowed value.

(if AST fuzzer triggers finds more violations of the assertion, we might
consider throwing an exception instead)

(*) https://github.com/ClickHouse/ClickHouse/pull/48154
(**) https://github.com/ClickHouse/ClickHouse/pull/48154/files#diff-653c0a18dfdaa86262c78dc6b25550add0487f165b4ad053e86f530388f6203a
 2023-04-05 08:07:37 +00:00
Antonio Andelic
e428ed5543
Merge pull request #48384 from ClickHouse/less-flaky-keeper-test 
Fix flaky `test_keeper_mntr_data_size`
 2023-04-05 09:59:28 +02:00
Azat Khuzhin
f0f58de79c Remove throttler for IDisk::copyFile() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:41:27 +02:00
Azat Khuzhin
dd9f0f409b Remove knowledge about throttling from IBackupWriter::supportNativeCopy() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:19 +02:00
Azat Khuzhin
c8597fbb9a Do not throttle S3-S3 backups if native copy is possible 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
61405b827d Use MAKE_DEPRECATED_BY_SERVER_CONFIG() for deprecated throttle settings 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
684ecc24e2 Ignore deprecated values max_remote_{read,write}_network_bandwidth_for_server 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
ad96d3c249 Do not require proper Poco Application for Context 
$ yes | head | clickhouse-obfuscator --structure 'id String' --seed foo --input-format TSV --output-format TSV
    Poco::Exception. Code: 1000, e.code() = 0, Null pointer: _pInstance in file "/src/ch/clickhouse/base/poco/Util/include/Poco/Util/Application.h", line 446, Stack trace (when copying this message, always include the lines below):

    0. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x15605622 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    1. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x156055ed in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    2. /src/ch/clickhouse/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x2ebd2d80 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    3. /src/ch/clickhouse/base/poco/Foundation/src/Exception.cpp:132: Poco::LogicException::LogicException(String const&, int) @ 0x2ebd3667 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    4. /src/ch/clickhouse/base/poco/Foundation/src/Exception.cpp:134: Poco::NullPointerException::NullPointerException(String const&, int) @ 0x2ebd3da7 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    5. /src/ch/clickhouse/base/poco/Foundation/src/Bugcheck.cpp:42: Poco::Bugcheck::nullPointer(char const*, char const*, int) @ 0x2ebc5851 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    6. /src/ch/clickhouse/base/poco/Util/include/Poco/Util/Application.h:446: Poco::Util::Application::instance() @ 0x1ee3e6cd in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    7. /src/ch/clickhouse/src/Interpreters/Context.cpp:358: DB::ContextSharedPart::ContextSharedPart() @ 0x263b0cca in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    8. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if<DB::ContextSharedPart>::__unique_single std::make_unique[abi:v15000]<DB::ContextSharedPart>() @ 0x26387b6e in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    9. /src/ch/clickhouse/src/Interpreters/Context.cpp:652: DB::Context::createShared() @ 0x26361a3c in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    10. /src/ch/clickhouse/programs/obfuscator/Obfuscator.cpp:1293: mainEntryClickHouseObfuscator(int, char**) @ 0x1ef6eb26 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    11. /src/ch/clickhouse/programs/main.cpp:481: main @ 0x155e569a in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    12. ? @ 0x7ffff7dd2790 in ?
    13. __libc_start_main @ 0x7ffff7dd284a in ?
    14. _start @ 0x155e51ae in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
     (version 23.3.1.2537)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
a8c1407bc4 Use existing ServerSettings in Context instead of reading it again 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
c332d290d8 Keep only one throttler for BACKUPs IO (instead of separate read/write) 
There is no need in separate read/write throttling, because you cannot
write faster then read anyway, and plus this makes the code less cleaner

(and also it will allow avoid implementing throttling backups to S3,
since it does not use common S3 writer).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:17 +02:00
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis 
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:48 +02:00
Azat Khuzhin
a25dd1d348 Add ability to throttle local IO on per-query/server basis 
Server settings:
- max_local_read_bandwidth_for_server
- max_local_write_bandwidth_for_server

Query settings:
- max_local_read_bandwidth
- max_local_write_bandwidth

This is the preparation for adding ability to throttle BACKUPs

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
b3406beeb7 Add per-query network throttling 
Controlled with:
- max_remote_read_network_bandwidth
- max_remote_write_network_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
abdb682048 Move some methods for Context into module part 
This will be required later, for per-query bandwidth limiting.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
40cb87c2ac Make remote bandwidth settings server settings 
Right now those two settings are applied only at start:
- max_remote_read_network_bandwidth_for_server
- max_remote_write_network_bandwidth_for_server

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
robot-clickhouse-ci-1
fb3af065f4
Merge pull request #48294 from ClickHouse/update-arrow-2 
Try to update arrow library to release 11.0.0
 2023-04-05 04:43:51 +02:00
Rich Raposa
25be18c83d
Merge pull request #48408 from ClickHouse/serxa-patch-3 
doc entry for SYSTEM WAIT LOADING PARTS
 2023-04-04 19:02:36 -06:00
Alexander Tokmakov
baed245710
Merge pull request #48409 from ClickHouse/better_ci_bot 
Make CI slack bot less noisy
 2023-04-05 03:28:18 +03:00
Alexander Tokmakov
e2679ccb53 make CI slack bot less noisy 2023-04-05 02:16:03 +02:00
Sergei Trifonov
1bea16ea63
doc entry for SYSTEM WAIT LOADING PARTS 2023-04-05 01:34:25 +02:00
FFFFFFFHHHHHHH
6e8f77ee9c
Add statistical aggregate function kolmogorovSmirnovTest (#48325) 2023-04-05 01:18:54 +02:00
Robert Schulze
db864891f8
Merge pull request #47610 from murfel/format-hiliting-fix 
Fix minor hiliting issues in clickhouse-format
 2023-04-04 22:59:46 +02:00