thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-29 02:52:13 +00:00
Code Issues Packages Projects Releases Wiki Activity
149,305 Commits 9,188 Branches 1,782 Tags 2.2 GiB
6b4353b910
Commit Graph

149305 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexey Milovidov
6b4353b910
Merge pull request #66495 from azat/ci-images-llvm-cleanup 
Ensure that llvm-symbolizer is used for symbolizing sanitizer reports
 2024-08-05 12:25:32 +00:00
Alexey Milovidov
243eb515d6
Merge pull request #66671 from azat/mongodb-uuid-type 
Support true UUID type for MongoDB engine
 2024-08-05 12:25:17 +00:00
János Benjamin Antal
0da44ad047
Merge pull request #67394 from ClickHouse/add-debug-to-s3-queue-shards-distributed 
Print debug info in `test_storage_s3_queue/test.py::test_shards_distributed`
 2024-08-05 12:23:40 +00:00
Alexey Milovidov
07f039677e
Merge pull request #67771 from ClickHouse/whitespaces 
Whitespaces
 2024-08-05 12:21:20 +00:00
Alexey Milovidov
78e26ac15f
Merge pull request #67784 from ClickHouse/fix-02231_bloom_filter_sizing 
Fix test `02231_bloom_filter_sizing`
 2024-08-05 12:21:10 +00:00
Kseniia Sumarokova
3111f0f32f
Merge pull request #67540 from ClickHouse/fix-busy-loop 
Fix (incorrectly) immediate reschedule of FileCache::freeSpaceRatioKeepingThreadFunc() in case of non-evictable cache
 2024-08-05 11:33:26 +00:00
János Benjamin Antal
8fece64f33
Merge pull request #66214 from sguerrini97/use_same_password_for_base_backup 
Allow to create incremental password protected backups
 2024-08-05 11:18:21 +00:00
Raúl Marín
079b8e1db2
Merge pull request #67405 from rschu1ze/input_row_count_remainders 
Follow-up to #67294
 2024-08-05 11:09:26 +00:00
Kseniia Sumarokova
2f1259dbfd
Merge pull request #67654 from shiyer7474/parameterized_view_serialization_fix 
Fix serialization of parameterized view parameters
 2024-08-05 10:56:13 +00:00
Kruglov Pavel
2c5687f3a4
Merge pull request #67122 from Avogar/variant-aggregate-column 
Fix crash with Variant + AggregateFunction type
 2024-08-05 10:49:22 +00:00
vdimir
2894b4339d
Merge pull request #67538 from ClickHouse/vdimir/join_on_tuple_comparison_elimination_bug 
Fix join on tuple with NULLs
 2024-08-05 10:22:32 +00:00
vdimir
908f3fc937
Merge pull request #66257 from bigo-sg/ch_support_printf 
Add function printf for spark compatiability
 2024-08-05 09:53:22 +00:00
vdimir
7add8ecc7a
Merge pull request #67135 from azat/local/config-from-home 
Read configuration for clickhouse-local from ~/.clickhouse-local
 2024-08-05 09:44:22 +00:00
Alexey Katsman
5006c571f3
Merge pull request #67674 from Blokje5/use-functionargumentdescriptors-in-bitslice 
Use FunctionArgumentDescriptors to check bitSlice function + add test case
 2024-08-05 09:36:27 +00:00
Alexey Milovidov
681eafef79
Merge pull request #67752 from azat/ci/stateless 
ci: better stateless runner (correctly collect artifacts and also some basic errors capturing)
 2024-08-05 02:53:11 +00:00
Alexey Milovidov
5a860fcc3a
Update 02099_tsv_raw_format.sh 2024-08-05 00:25:56 +02:00
Alexey Milovidov
270dddc00f Fix test 02231_bloom_filter_sizing 2024-08-05 00:22:48 +02:00
Azat Khuzhin
43cf85ef28 ci: collect basic issues in stateless tests (dmesg, fatal and similar) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Azat Khuzhin
c200f43774 ci: fix stateless runner to correctly gather artifacts when server failed 
Right now it simply does not work due to "set -e", with it you cannot
use "foo=$(false)" since bash will break execution after, rewrite it to
a plain shell, with "if".

Also use ZSTD everywhere (ugh)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Alexey Milovidov
1e19459610
Merge pull request #67334 from petern48/aliasFeature 
New Feature: Add camelCase aliases denseRank() and percentRank()
 2024-08-04 20:52:31 +00:00
Alexey Milovidov
16e8321d48
Merge pull request #67751 from azat/dict-fix-async-loading 
Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading (also fixes possible stress tests hung)
 2024-08-04 20:46:57 +00:00
Azat Khuzhin
a49924b383 tests: avoid leaving leftovers after test_storage_mongodb (fixes flaky chec) 
CI: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 22:01:08 +02:00
Alexey Milovidov
bd50224329 Whitespaces 2024-08-04 20:57:22 +02:00
Alexey Milovidov
ce58549592
Merge pull request #67557 from rschu1ze/icu-s390x 
Re-enable ICU on s390/x
 2024-08-04 17:51:41 +00:00
Alexey Milovidov
6f9d78d873
Merge pull request #67743 from azat/tests/test_storage_rabbitmq 
Try to fix RabbitMQ test failures
 2024-08-04 19:26:23 +02:00
Nikita Fomichev
c4a0e7665c
Merge pull request #67747 from azat/tests/test_ttl_move 
Fix test_ttl_move::test_alter_with_merge_work flakiness
 2024-08-04 16:47:24 +00:00
Alexey Milovidov
0e013ba62e
Merge pull request #67306 from ucasfl/uniq-crash 
Fix crash of uniq and uniqTheta with empty tuple argument
 2024-08-04 16:12:00 +00:00
Alexey Milovidov
98c791160e
Merge pull request #67746 from ClickHouse/fix-flaky-testss 
Stateless tests: attempt to fix timeouts of `02473_multistep_prewhere* 00411_long_accurate_number_comparison*`
 2024-08-04 15:49:05 +00:00
Alexey Milovidov
fbd3e122e1
Merge pull request #67512 from ClickHouse/divanik/fix_retry_integration_tests_decorator 
[Green CI] Test `test_storage_azure_blob_storage/test.py` is flaky
 2024-08-04 15:46:55 +00:00
Alexey Milovidov
623665dc45
Merge pull request #67739 from azat/system.clusters-mask 
Fill only selected columns from system.clusters
 2024-08-04 15:46:01 +00:00
Alexey Milovidov
592f966c2e
Merge pull request #67740 from azat/bump-nuraft 
Bump NuRaft (to properly catch thread exceptions)
 2024-08-04 15:45:41 +00:00
Alexey Milovidov
7073022f11
Merge pull request #67623 from ClickHouse/fix-logical-error-in-cached-about-expected-read-size 
Fix `Logical error: 'file_offset_of_buffer_end <= read_until_position'` in fs cache
 2024-08-04 15:44:46 +00:00
Azat Khuzhin
1f2bb6676f Remove outdated comments about workaround for #64086 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:14:05 +02:00
Azat Khuzhin
dea95e6c33 Ensure that llvm-symbolizer is used for symbolizing sanitizer reports 
Since you don't want to fallback to addr2line:

    # addr2line
    $ time /bin/test
    set_flag_impl: Success
    set_flag_if: Success
    /usr/bin/addr2line: DWARF error: invalid or unhandled FORM value: 0x23
    ==================
    WARNING: ThreadSanitizer: data race (pid=18)
    ...
    real    3m8.580s
    user    0m21.967s
    sys     0m40.628s

    # llvm-symbolizer
    $ time ./test
    set_flag_impl: Success
    set_flag_if: Success
    ==================
    WARNING: ThreadSanitizer: data race (pid=24884)
    real    0m0.028s
    user    0m0.003s
    sys     0m0.006s

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:14:05 +02:00
Azat Khuzhin
9f31488e50 Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading 
On CI you can find that 01747_executable_pool_dictionary_implicit_key
can hang [1], it is possible due to after CANNOT_SCHEDULE_TASK the async
loading will hang:

    2024.07.18 03:56:32.365226 [ 6138 ] {6206a18f-668c-4a5c-a5ad-07f577220762} <Trace> ExternalDictionariesLoader: Will load the object 'executable_pool_simple_implicit_key' in background, force = false, loading_id = 2
    2024.07.18 03:56:32.368005 [ 6138 ] {6206a18f-668c-4a5c-a5ad-07f577220762} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=766, jobs=746): In scope SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1)). (CANNOT_SCHEDULE_TASK) (version 24.7.1.2241) (from [::1]:56446) (comment: 01747_executable_pool_dictionary_implicit_key.sql) (in query: SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1));), Stack trace (when copying this message, always include the lines below):
    0. /build/contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000015f8a292
    1. /build/src/Common/Exception.cpp:110: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c3df6b9
    2. /build/contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000006de714c
    3. /build/contrib/llvm-project/libcxx/include/vector:438: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000000c4838eb
    4. /build/src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000000c4832d3
    5. /build/src/Common/ThreadPool.cpp:186: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000000c47e7db
    6. /build/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000c47ec8d
    7. /build/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001114b16e
    8. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ExternalLoader::LoadingDispatcher::startLoading(DB::ExternalLoader::LoadingDispatcher::Info&, bool, unsigned long) @ 0x0000000011147733
    9. /build/src/Interpreters/ExternalLoader.cpp:837: DB::ExternalLoader::LoadingDispatcher::loadImpl(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>, bool, std::unique_lock<std::mutex>&)::'lambda'()::operator()() const @ 0x0000000011158bf9
    10. /build/contrib/llvm-project/libcxx/include/__mutex_base:397: DB::ExternalLoader::LoadingDispatcher::loadImpl(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>, bool, std::unique_lock<std::mutex>&) @ 0x00000000111588bc
    11. /build/src/Interpreters/ExternalLoader.cpp:604: DB::ExternalLoader::LoadResult DB::ExternalLoader::LoadingDispatcher::tryLoad<DB::ExternalLoader::LoadResult>(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>) @ 0x00000000111440bf
    12. /build/src/Interpreters/ExternalLoader.cpp:1381: std::shared_ptr<DB::IExternalLoadable const> DB::ExternalLoader::load<std::shared_ptr<DB::IExternalLoadable const>, void>(String const&) const @ 0x00000000111442f5
    13. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:587: DB::ExternalDictionariesLoader::getDictionary(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000011141028
    14. /build/src/Functions/FunctionsExternalDictionaries.h:76: DB::FunctionDictHelper::getDictionary(String const&) @ 0x00000000071d28ec
    ...
    2024.07.18 03:58:29.000900 [ 48468 ] {8cf63d7e-dcbf-4af6-bd7c-0e1789ddce3b} <Debug> executeQuery: (from [::1]:40410) (comment: 01747_executable_pool_dictionary_implicit_key.sql) SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1)); (stage: Complete)
    # and no more rows for 8cf63d7e-dcbf-4af6-bd7c-0e1789ddce3b

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/66495/bc029ed8207ac75e96e9cb48cb79d27a9ffa4e2f/stress_test__debug_.html

The problem that it should be properly cancelled, otherwise it will not
be loaded in loadImpl(), but will be waited.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 15:48:59 +02:00
Alexey Milovidov
c629e2af50
Merge pull request #67741 from ClickHouse/arrayWithConstant-safety-thresholds 
Better safety thresholds in `arrayWithConstant` function
 2024-08-04 13:22:06 +00:00
Alexey Milovidov
06f83022da
Merge pull request #67324 from ClickHouse/fix-json-recursion 
Fix stack overflow in too deep JSON
 2024-08-04 13:21:44 +00:00
Kseniia Sumarokova
881ac7208e
Merge pull request #67361 from ClickHouse/add-no-distributed-cache-tag 
Add no-distributed-cache tag in tests
 2024-08-04 12:36:31 +00:00
Azat Khuzhin
062490e1b4 tests: fix test_ttl_move::test_alter_with_merge_work flakiness 
Increase timeout for TTL DELETE, since otherwise if other routines will
take too long, the part will be removed when it should be still be on
"external" disk:

    2024.08.04 03:48:53.803032 [ 622 ] {} <Information> default.mt_test_alter_with_merge_work_1722743323 (9dc6904a-f082-4f06-be7a-efe4733e811c): Will drop empty part all_1_3_4_4

And this is how part_log looks like:

    SELECT
        event_time,
        event_type,
        rows,
        part_name,
        error,
        database,
        disk_name
    FROM system.part_log
    WHERE `table` = 'mt_test_alter_with_merge_work_1722743323'
    ORDER BY event_time ASC

    Query id: a118b3cd-e4fe-45a5-b675-d73bdd887d79

        ┌──────────event_time─┬─event_type─┬─rows─┬─part_name───┬─error─┬─database─┬─disk_name─┐
     1. │ 2024-08-04 03:48:44 │ NewPart    │    2 │ all_1_1_0   │     0 │ default  │ jbod1     │
     2. │ 2024-08-04 03:48:44 │ NewPart    │    2 │ all_2_2_0   │     0 │ default  │ jbod2     │
     3. │ 2024-08-04 03:48:45 │ NewPart    │    2 │ all_3_3_0   │     0 │ default  │ jbod1     │
     4. │ 2024-08-04 03:48:46 │ MutatePart │    2 │ all_1_1_0_4 │     0 │ default  │ jbod1     │
     5. │ 2024-08-04 03:48:46 │ MutatePart │    2 │ all_2_2_0_4 │     0 │ default  │ jbod2     │
     6. │ 2024-08-04 03:48:46 │ MutatePart │    2 │ all_3_3_0_4 │     0 │ default  │ jbod1     │
     7. │ 2024-08-04 03:48:47 │ MovePart   │    2 │ all_1_1_0_4 │     0 │ default  │ external  │
     8. │ 2024-08-04 03:48:47 │ MovePart   │    2 │ all_3_3_0_4 │     0 │ default  │ jbod2     │
     9. │ 2024-08-04 03:48:47 │ MergeParts │    6 │ all_1_3_1_4 │     0 │ default  │ jbod2     │
    10. │ 2024-08-04 03:48:48 │ MovePart   │    6 │ all_1_3_1_4 │     0 │ default  │ external  │
    11. │ 2024-08-04 03:48:52 │ MergeParts │    4 │ all_1_3_2_4 │     0 │ default  │ external  │
    12. │ 2024-08-04 03:48:53 │ MergeParts │    0 │ all_1_3_3_4 │     0 │ default  │ external  │ # rows==0
    13. │ 2024-08-04 03:48:53 │ MergeParts │    0 │ all_1_3_4_4 │     0 │ default  │ external  │
        └─────────────────────┴────────────┴──────┴─────────────┴───────┴──────────┴───────────┘

CI: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests__tsan__[5_6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 14:25:18 +02:00
Azat Khuzhin
47dbc5e05b tests: add debug info into test_ttl_move::test_alter_with_merge_work 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 14:16:40 +02:00
Azat Khuzhin
f0aaac3bd1 tests: remove useless retries from test_ttl_move::test_alter_with_merge_work 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 14:14:42 +02:00
Robert Schulze
e4c165e919
Merge pull request #67103 from bigo-sg/fix_map_from_arrays 
Function `mapFromArrays`: Allow `Map(K, V)` and `Array(Nullable(T))` as 1st arguments
 2024-08-04 09:54:50 +00:00
Robert Schulze
80b41a3edd
Merge remote-tracking branch 'ClickHouse/master' into input_row_count_remainders 2024-08-04 09:53:00 +00:00
Robert Schulze
27db715761
Incorporate review feedback 2024-08-04 09:52:36 +00:00
Robert Schulze
ee8079b039
Merge remote-tracking branch 'ClickHouse/master' into icu-s390x 2024-08-04 09:38:36 +00:00
Kseniia Sumarokova
4f1c2ad277
Merge pull request #67459 from ClickHouse/write-buffer-from-s3-fix-dtor 
Fix wait for tasks in ~WriteBufferFromS3 in case WriteBuffer was cancelled
 2024-08-04 09:27:24 +00:00
Kseniia Sumarokova
15e1217beb
Merge branch 'master' into fix-busy-loop 2024-08-04 11:26:39 +02:00
Azat Khuzhin
dc527b6fd1 tests: detailed errors for RabbitMQ startup 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 09:07:05 +02:00
Azat Khuzhin
264be9c598 tests: increase timeout for RabbitMQ startup 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 09:06:50 +02:00
Azat Khuzhin
59e5a09b49 Use RabbitMQ without management in tests (attempt to improve startup) 
Sometimes startup can take ~90 seconds [1]:

    2024-08-03 23:11:38.756067+00:00 [info] <0.9.0> Time to start RabbitMQ: 94651980 us

Unlike normally ~10 [2]:

    2024-08-04 00:33:07.016137+00:00 [info] <0.9.0> Time to start RabbitMQ: 10082489 us

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/67737/b4e3bbcb82158bea4f5db1d9f5c28cfb741d1d51/integration_tests__asan__old_analyzer__[4_6].html
  [2]: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests__asan__old_analyzer__[4_6].html

I've tried locally, the difference is very small, 3135665 us
(+management) vs (2740747 us), but still something, and who knows how it
works under pressure.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 09:04:41 +02:00