thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-29 02:52:13 +00:00
Code Issues Packages Projects Releases Wiki Activity
149,704 Commits 9,188 Branches 1,782 Tags 2.2 GiB
c7b94ca43a
Commit Graph

149704 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
avogar
a499cd25c7 Fix for integers 2024-08-05 09:31:41 +00:00
Anton Popov
834fa7fb56 Merge remote-tracking branch 'upstream/master' into HEAD 2024-08-05 09:26:08 +00:00
Azat Khuzhin
9ce55b69b4 Fix possible CANNOT_READ_ALL_DATA during server startup in performance tests 
CI [1]:

    2024.08.04 22:09:11.646800 [ 1052 ] {} <Error> Application: Code: 33. DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4.: While checking access for disk backups. (CANNOT_READ_ALL_DATA), Stack trace (when copying this message, always include the lines below):

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/64955/6702acf6f2e4a0ee9697066e38006631fc7f69df/performance_comparison__aarch64__[2_4].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-05 11:23:53 +02:00
vdimir
547765760e
Merge branch 'master' into vdimir/fix_02434_cancel_insert_when_client_dies 2024-08-05 11:16:54 +02:00
Azat Khuzhin
cdbc4f3573 ci: fail the test if the entrypoint script failed 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-05 11:16:08 +02:00
Azat Khuzhin
b47f8a733f ci: fix basic errors collecting after stateless tests 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-05 11:16:08 +02:00
János Benjamin Antal
517bb64636 Merge remote-tracking branch 'origin/master' into fix-flaky-test_system_kafka_consumers_rebalance 2024-08-05 09:04:37 +00:00
Robert Schulze
d0bc728d52
Bump rocksdb to v8.9.1 2024-08-05 08:11:40 +00:00
Robert Schulze
cebdc5ecf6
Bump rocksdb to v8.0.0 2024-08-05 08:11:23 +00:00
Azat Khuzhin
2a7ad3a1f9 Update test references for new REPLACE modifier syntax 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-05 08:59:29 +02:00
Azat Khuzhin
8bca80f4dd Fix REPLACE modifier formatting (forbid omitting brackets) 
It is too tricky to verify does brackets required or not, i.e. "SELECT *
REPLACE(1/3/3 AS dummy)" will be formatted to "SELECT * REPLACE (1/3)/3 AS dummy"
which is already invalid query.

So let's simply always print them.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: move the fix into correct place ASTColumnsReplaceTransformer::formatImpl() instead of ASTColumnsReplaceTransformer::Replacement::formatImpl()
 2024-08-05 08:59:28 +02:00
Azat Khuzhin
6b1e184e12 Print original query for AST formatting check on CI 
It may be tricky to understand the root cause of the AST formatting
issue in case of syntax error, so add one knob to control this -
debug_ast_formatting_print_original_query.

And CI contains core dumps anyway, so let's enable for CI.

P.S. There was concern from @al13n321 that printing original query even
in debug build is not a good idea [1], hence a knob for this.

  [1]: https://github.com/ClickHouse/ClickHouse/pull/63357/files#r1674809348

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-05 08:37:45 +02:00
Alexey Milovidov
681eafef79
Merge pull request #67752 from azat/ci/stateless 
ci: better stateless runner (correctly collect artifacts and also some basic errors capturing)
 2024-08-05 02:53:11 +00:00
robot-clickhouse
7adeaf9c28 Automatic style fix 2024-08-04 22:35:16 +00:00
Alexey Milovidov
5a860fcc3a
Update 02099_tsv_raw_format.sh 2024-08-05 00:25:56 +02:00
Alexey Milovidov
270dddc00f Fix test 02231_bloom_filter_sizing 2024-08-05 00:22:48 +02:00
Alexey Milovidov
4504326b2a Merge branch 'no-flaky-check' into fix-log-and-exception-messages-formatting 2024-08-05 00:03:25 +02:00
Alexey Milovidov
57108fcc2c Merge branch 'master' into no-flaky-check 2024-08-05 00:02:41 +02:00
Alexey Milovidov
538761b43d Fix flaky check 2024-08-05 00:02:32 +02:00
Alexey Milovidov
a573b2926e Fixes for the script 2024-08-04 23:59:44 +02:00
Alexey Milovidov
0bdaa57023 Fix diagnostics in the test script 2024-08-04 23:53:54 +02:00
Alexey Milovidov
83be27cdf5 Fix test 02833_concurrent_sessions, Fix test 02835_drop_user_during_session 2024-08-04 23:34:01 +02:00
Azat Khuzhin
43cf85ef28 ci: collect basic issues in stateless tests (dmesg, fatal and similar) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Azat Khuzhin
c200f43774 ci: fix stateless runner to correctly gather artifacts when server failed 
Right now it simply does not work due to "set -e", with it you cannot
use "foo=$(false)" since bash will break execution after, rewrite it to
a plain shell, with "if".

Also use ZSTD everywhere (ugh)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Alexey Milovidov
1e19459610
Merge pull request #67334 from petern48/aliasFeature 
New Feature: Add camelCase aliases denseRank() and percentRank()
 2024-08-04 20:52:31 +00:00
Alexey Milovidov
16e8321d48
Merge pull request #67751 from azat/dict-fix-async-loading 
Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading (also fixes possible stress tests hung)
 2024-08-04 20:46:57 +00:00
Azat Khuzhin
a49924b383 tests: avoid leaving leftovers after test_storage_mongodb (fixes flaky chec) 
CI: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 22:01:08 +02:00
Alexey Milovidov
bd50224329 Whitespaces 2024-08-04 20:57:22 +02:00
Alexey Milovidov
576e0952b5 Merge branch 'master' into add-assert-into-timer-descriptor 2024-08-04 19:53:39 +02:00
Alexey Milovidov
ce58549592
Merge pull request #67557 from rschu1ze/icu-s390x 
Re-enable ICU on s390/x
 2024-08-04 17:51:41 +00:00
Alexey Milovidov
6f9d78d873
Merge pull request #67743 from azat/tests/test_storage_rabbitmq 
Try to fix RabbitMQ test failures
 2024-08-04 19:26:23 +02:00
Alexey Milovidov
a433115434 Fix typo 2024-08-04 19:18:00 +02:00
Alexey Milovidov
334eaf201f Merge branch 'master' into no-flaky-check 2024-08-04 19:17:32 +02:00
Nikita Fomichev
c4a0e7665c
Merge pull request #67747 from azat/tests/test_ttl_move 
Fix test_ttl_move::test_alter_with_merge_work flakiness
 2024-08-04 16:47:24 +00:00
Alexey Milovidov
37da0b3c30 Add a test 2024-08-04 18:27:37 +02:00
Alexey Milovidov
87e0cf6b76 Fix stack overflow in JSONMergePatch 2024-08-04 18:25:36 +02:00
Alexey Milovidov
0e013ba62e
Merge pull request #67306 from ucasfl/uniq-crash 
Fix crash of uniq and uniqTheta with empty tuple argument
 2024-08-04 16:12:00 +00:00
Alexey Milovidov
701db12687 Merge branch 'no-flaky-check' into fix-log-and-exception-messages-formatting 2024-08-04 18:08:31 +02:00
Alexey Milovidov
6fd4552fad Merge branch 'master' into fix-log-and-exception-messages-formatting 2024-08-04 18:08:28 +02:00
Alexey Milovidov
a75c4b9f9f Introduce no-flaky-check tag 2024-08-04 18:07:40 +02:00
Alexey Milovidov
98c791160e
Merge pull request #67746 from ClickHouse/fix-flaky-testss 
Stateless tests: attempt to fix timeouts of `02473_multistep_prewhere* 00411_long_accurate_number_comparison*`
 2024-08-04 15:49:05 +00:00
Alexey Milovidov
fbd3e122e1
Merge pull request #67512 from ClickHouse/divanik/fix_retry_integration_tests_decorator 
[Green CI] Test `test_storage_azure_blob_storage/test.py` is flaky
 2024-08-04 15:46:55 +00:00
Alexey Milovidov
623665dc45
Merge pull request #67739 from azat/system.clusters-mask 
Fill only selected columns from system.clusters
 2024-08-04 15:46:01 +00:00
Alexey Milovidov
592f966c2e
Merge pull request #67740 from azat/bump-nuraft 
Bump NuRaft (to properly catch thread exceptions)
 2024-08-04 15:45:41 +00:00
Alexey Milovidov
7073022f11
Merge pull request #67623 from ClickHouse/fix-logical-error-in-cached-about-expected-read-size 
Fix `Logical error: 'file_offset_of_buffer_end <= read_until_position'` in fs cache
 2024-08-04 15:44:46 +00:00
Azat Khuzhin
eed2edd7db Comment certificateFile/privateKeyFile/dhParamsFile in keeper config 
Otherwise you will get annoying messages at startup:

    2024.07.02 10:03:38.331593 [ 1 ] {} <Error> CertificateReloader: Cannot obtain modification time for certificate file /etc/clickhouse-keeper/server.crt, skipping update. errno: 2, strerror: 0
    2024.07.02 10:03:38.331658 [ 1 ] {} <Error> CertificateReloader: Cannot obtain modification time for key file /etc/clickhouse-keeper/server.key, skipping update. errno: 2, strerror: 0
    2024.07.02 10:03:38.341085 [ 1 ] {} <Error> CertificateReloader: Poco::Exception. Code: 1000, e.code() = 0, SSL context exception: Error loading private key from file /etc/clickhouse-keeper/server.key: error:80000002:system library::No such file or directory

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:20:42 +02:00
Azat Khuzhin
1f2bb6676f Remove outdated comments about workaround for #64086 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:14:05 +02:00
Azat Khuzhin
dea95e6c33 Ensure that llvm-symbolizer is used for symbolizing sanitizer reports 
Since you don't want to fallback to addr2line:

    # addr2line
    $ time /bin/test
    set_flag_impl: Success
    set_flag_if: Success
    /usr/bin/addr2line: DWARF error: invalid or unhandled FORM value: 0x23
    ==================
    WARNING: ThreadSanitizer: data race (pid=18)
    ...
    real    3m8.580s
    user    0m21.967s
    sys     0m40.628s

    # llvm-symbolizer
    $ time ./test
    set_flag_impl: Success
    set_flag_if: Success
    ==================
    WARNING: ThreadSanitizer: data race (pid=24884)
    real    0m0.028s
    user    0m0.003s
    sys     0m0.006s

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:14:05 +02:00
Azat Khuzhin
9f31488e50 Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading 
On CI you can find that 01747_executable_pool_dictionary_implicit_key
can hang [1], it is possible due to after CANNOT_SCHEDULE_TASK the async
loading will hang:

    2024.07.18 03:56:32.365226 [ 6138 ] {6206a18f-668c-4a5c-a5ad-07f577220762} <Trace> ExternalDictionariesLoader: Will load the object 'executable_pool_simple_implicit_key' in background, force = false, loading_id = 2
    2024.07.18 03:56:32.368005 [ 6138 ] {6206a18f-668c-4a5c-a5ad-07f577220762} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=766, jobs=746): In scope SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1)). (CANNOT_SCHEDULE_TASK) (version 24.7.1.2241) (from [::1]:56446) (comment: 01747_executable_pool_dictionary_implicit_key.sql) (in query: SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1));), Stack trace (when copying this message, always include the lines below):
    0. /build/contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000015f8a292
    1. /build/src/Common/Exception.cpp:110: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c3df6b9
    2. /build/contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000006de714c
    3. /build/contrib/llvm-project/libcxx/include/vector:438: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000000c4838eb
    4. /build/src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000000c4832d3
    5. /build/src/Common/ThreadPool.cpp:186: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000000c47e7db
    6. /build/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000c47ec8d
    7. /build/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001114b16e
    8. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ExternalLoader::LoadingDispatcher::startLoading(DB::ExternalLoader::LoadingDispatcher::Info&, bool, unsigned long) @ 0x0000000011147733
    9. /build/src/Interpreters/ExternalLoader.cpp:837: DB::ExternalLoader::LoadingDispatcher::loadImpl(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>, bool, std::unique_lock<std::mutex>&)::'lambda'()::operator()() const @ 0x0000000011158bf9
    10. /build/contrib/llvm-project/libcxx/include/__mutex_base:397: DB::ExternalLoader::LoadingDispatcher::loadImpl(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>, bool, std::unique_lock<std::mutex>&) @ 0x00000000111588bc
    11. /build/src/Interpreters/ExternalLoader.cpp:604: DB::ExternalLoader::LoadResult DB::ExternalLoader::LoadingDispatcher::tryLoad<DB::ExternalLoader::LoadResult>(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>) @ 0x00000000111440bf
    12. /build/src/Interpreters/ExternalLoader.cpp:1381: std::shared_ptr<DB::IExternalLoadable const> DB::ExternalLoader::load<std::shared_ptr<DB::IExternalLoadable const>, void>(String const&) const @ 0x00000000111442f5
    13. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:587: DB::ExternalDictionariesLoader::getDictionary(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000011141028
    14. /build/src/Functions/FunctionsExternalDictionaries.h:76: DB::FunctionDictHelper::getDictionary(String const&) @ 0x00000000071d28ec
    ...
    2024.07.18 03:58:29.000900 [ 48468 ] {8cf63d7e-dcbf-4af6-bd7c-0e1789ddce3b} <Debug> executeQuery: (from [::1]:40410) (comment: 01747_executable_pool_dictionary_implicit_key.sql) SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1)); (stage: Complete)
    # and no more rows for 8cf63d7e-dcbf-4af6-bd7c-0e1789ddce3b

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/66495/bc029ed8207ac75e96e9cb48cb79d27a9ffa4e2f/stress_test__debug_.html

The problem that it should be properly cancelled, otherwise it will not
be loaded in loadImpl(), but will be waited.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 15:48:59 +02:00
Alexey Milovidov
c629e2af50
Merge pull request #67741 from ClickHouse/arrayWithConstant-safety-thresholds 
Better safety thresholds in `arrayWithConstant` function
 2024-08-04 13:22:06 +00:00