thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-14 19:45:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
149,261 Commits 9,131 Branches 1,774 Tags 2.1 GiB
5e01e4d6b7
Commit Graph

149261 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Chun-Sheng, Li
5e01e4d6b7
Update aspell-dict.txt 
Add PostLinks to ignore spell check
 2024-09-30 21:07:49 +08:00
avogar
74c2c92a82 Restart CI 2024-09-30 12:34:32 +00:00
peter279k
2b52bce40b
Improve Stack Overflow dataset doc 2024-08-05 18:55:05 +08:00
vdimir
908f3fc937
Merge pull request #66257 from bigo-sg/ch_support_printf 
Add function printf for spark compatiability
 2024-08-05 09:53:22 +00:00
vdimir
7add8ecc7a
Merge pull request #67135 from azat/local/config-from-home 
Read configuration for clickhouse-local from ~/.clickhouse-local
 2024-08-05 09:44:22 +00:00
Alexey Katsman
5006c571f3
Merge pull request #67674 from Blokje5/use-functionargumentdescriptors-in-bitslice 
Use FunctionArgumentDescriptors to check bitSlice function + add test case
 2024-08-05 09:36:27 +00:00
Alexey Milovidov
681eafef79
Merge pull request #67752 from azat/ci/stateless 
ci: better stateless runner (correctly collect artifacts and also some basic errors capturing)
 2024-08-05 02:53:11 +00:00
Azat Khuzhin
43cf85ef28 ci: collect basic issues in stateless tests (dmesg, fatal and similar) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Azat Khuzhin
c200f43774 ci: fix stateless runner to correctly gather artifacts when server failed 
Right now it simply does not work due to "set -e", with it you cannot
use "foo=$(false)" since bash will break execution after, rewrite it to
a plain shell, with "if".

Also use ZSTD everywhere (ugh)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Alexey Milovidov
1e19459610
Merge pull request #67334 from petern48/aliasFeature 
New Feature: Add camelCase aliases denseRank() and percentRank()
 2024-08-04 20:52:31 +00:00
Alexey Milovidov
16e8321d48
Merge pull request #67751 from azat/dict-fix-async-loading 
Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading (also fixes possible stress tests hung)
 2024-08-04 20:46:57 +00:00
Alexey Milovidov
ce58549592
Merge pull request #67557 from rschu1ze/icu-s390x 
Re-enable ICU on s390/x
 2024-08-04 17:51:41 +00:00
Alexey Milovidov
6f9d78d873
Merge pull request #67743 from azat/tests/test_storage_rabbitmq 
Try to fix RabbitMQ test failures
 2024-08-04 19:26:23 +02:00
Nikita Fomichev
c4a0e7665c
Merge pull request #67747 from azat/tests/test_ttl_move 
Fix test_ttl_move::test_alter_with_merge_work flakiness
 2024-08-04 16:47:24 +00:00
Alexey Milovidov
0e013ba62e
Merge pull request #67306 from ucasfl/uniq-crash 
Fix crash of uniq and uniqTheta with empty tuple argument
 2024-08-04 16:12:00 +00:00
Alexey Milovidov
98c791160e
Merge pull request #67746 from ClickHouse/fix-flaky-testss 
Stateless tests: attempt to fix timeouts of `02473_multistep_prewhere* 00411_long_accurate_number_comparison*`
 2024-08-04 15:49:05 +00:00
Alexey Milovidov
fbd3e122e1
Merge pull request #67512 from ClickHouse/divanik/fix_retry_integration_tests_decorator 
[Green CI] Test `test_storage_azure_blob_storage/test.py` is flaky
 2024-08-04 15:46:55 +00:00
Alexey Milovidov
623665dc45
Merge pull request #67739 from azat/system.clusters-mask 
Fill only selected columns from system.clusters
 2024-08-04 15:46:01 +00:00
Alexey Milovidov
592f966c2e
Merge pull request #67740 from azat/bump-nuraft 
Bump NuRaft (to properly catch thread exceptions)
 2024-08-04 15:45:41 +00:00
Alexey Milovidov
7073022f11
Merge pull request #67623 from ClickHouse/fix-logical-error-in-cached-about-expected-read-size 
Fix `Logical error: 'file_offset_of_buffer_end <= read_until_position'` in fs cache
 2024-08-04 15:44:46 +00:00
Azat Khuzhin
9f31488e50 Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading 
On CI you can find that 01747_executable_pool_dictionary_implicit_key
can hang [1], it is possible due to after CANNOT_SCHEDULE_TASK the async
loading will hang:

    2024.07.18 03:56:32.365226 [ 6138 ] {6206a18f-668c-4a5c-a5ad-07f577220762} <Trace> ExternalDictionariesLoader: Will load the object 'executable_pool_simple_implicit_key' in background, force = false, loading_id = 2
    2024.07.18 03:56:32.368005 [ 6138 ] {6206a18f-668c-4a5c-a5ad-07f577220762} <Error> executeQuery: Code: 439. DB::Exception: Cannot schedule a task: fault injected (threads=766, jobs=746): In scope SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1)). (CANNOT_SCHEDULE_TASK) (version 24.7.1.2241) (from [::1]:56446) (comment: 01747_executable_pool_dictionary_implicit_key.sql) (in query: SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1));), Stack trace (when copying this message, always include the lines below):
    0. /build/contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000015f8a292
    1. /build/src/Common/Exception.cpp:110: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c3df6b9
    2. /build/contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000006de714c
    3. /build/contrib/llvm-project/libcxx/include/vector:438: DB::Exception::Exception<String const&, unsigned long, unsigned long&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<unsigned long>::type, std::type_identity<unsigned long&>::type>, String const&, unsigned long&&, unsigned long&) @ 0x000000000c4838eb
    4. /build/src/Common/ThreadPool.cpp:0: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda'(String const&)::operator()(String const&) const @ 0x000000000c4832d3
    5. /build/src/Common/ThreadPool.cpp:186: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool) @ 0x000000000c47e7db
    6. /build/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000000c47ec8d
    7. /build/contrib/llvm-project/libcxx/include/__functional/function.h:818: ? @ 0x000000001114b16e
    8. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::ExternalLoader::LoadingDispatcher::startLoading(DB::ExternalLoader::LoadingDispatcher::Info&, bool, unsigned long) @ 0x0000000011147733
    9. /build/src/Interpreters/ExternalLoader.cpp:837: DB::ExternalLoader::LoadingDispatcher::loadImpl(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>, bool, std::unique_lock<std::mutex>&)::'lambda'()::operator()() const @ 0x0000000011158bf9
    10. /build/contrib/llvm-project/libcxx/include/__mutex_base:397: DB::ExternalLoader::LoadingDispatcher::loadImpl(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>, bool, std::unique_lock<std::mutex>&) @ 0x00000000111588bc
    11. /build/src/Interpreters/ExternalLoader.cpp:604: DB::ExternalLoader::LoadResult DB::ExternalLoader::LoadingDispatcher::tryLoad<DB::ExternalLoader::LoadResult>(String const&, std::chrono::duration<long long, std::ratio<1l, 1000l>>) @ 0x00000000111440bf
    12. /build/src/Interpreters/ExternalLoader.cpp:1381: std::shared_ptr<DB::IExternalLoadable const> DB::ExternalLoader::load<std::shared_ptr<DB::IExternalLoadable const>, void>(String const&) const @ 0x00000000111442f5
    13. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:587: DB::ExternalDictionariesLoader::getDictionary(String const&, std::shared_ptr<DB::Context const>) const @ 0x0000000011141028
    14. /build/src/Functions/FunctionsExternalDictionaries.h:76: DB::FunctionDictHelper::getDictionary(String const&) @ 0x00000000071d28ec
    ...
    2024.07.18 03:58:29.000900 [ 48468 ] {8cf63d7e-dcbf-4af6-bd7c-0e1789ddce3b} <Debug> executeQuery: (from [::1]:40410) (comment: 01747_executable_pool_dictionary_implicit_key.sql) SELECT dictGet('executable_pool_simple_implicit_key', 'a', toUInt64(1)); (stage: Complete)
    # and no more rows for 8cf63d7e-dcbf-4af6-bd7c-0e1789ddce3b

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/66495/bc029ed8207ac75e96e9cb48cb79d27a9ffa4e2f/stress_test__debug_.html

The problem that it should be properly cancelled, otherwise it will not
be loaded in loadImpl(), but will be waited.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 15:48:59 +02:00
Alexey Milovidov
c629e2af50
Merge pull request #67741 from ClickHouse/arrayWithConstant-safety-thresholds 
Better safety thresholds in `arrayWithConstant` function
 2024-08-04 13:22:06 +00:00
Alexey Milovidov
06f83022da
Merge pull request #67324 from ClickHouse/fix-json-recursion 
Fix stack overflow in too deep JSON
 2024-08-04 13:21:44 +00:00
Kseniia Sumarokova
881ac7208e
Merge pull request #67361 from ClickHouse/add-no-distributed-cache-tag 
Add no-distributed-cache tag in tests
 2024-08-04 12:36:31 +00:00
Azat Khuzhin
062490e1b4 tests: fix test_ttl_move::test_alter_with_merge_work flakiness 
Increase timeout for TTL DELETE, since otherwise if other routines will
take too long, the part will be removed when it should be still be on
"external" disk:

    2024.08.04 03:48:53.803032 [ 622 ] {} <Information> default.mt_test_alter_with_merge_work_1722743323 (9dc6904a-f082-4f06-be7a-efe4733e811c): Will drop empty part all_1_3_4_4

And this is how part_log looks like:

    SELECT
        event_time,
        event_type,
        rows,
        part_name,
        error,
        database,
        disk_name
    FROM system.part_log
    WHERE `table` = 'mt_test_alter_with_merge_work_1722743323'
    ORDER BY event_time ASC

    Query id: a118b3cd-e4fe-45a5-b675-d73bdd887d79

        ┌──────────event_time─┬─event_type─┬─rows─┬─part_name───┬─error─┬─database─┬─disk_name─┐
     1. │ 2024-08-04 03:48:44 │ NewPart    │    2 │ all_1_1_0   │     0 │ default  │ jbod1     │
     2. │ 2024-08-04 03:48:44 │ NewPart    │    2 │ all_2_2_0   │     0 │ default  │ jbod2     │
     3. │ 2024-08-04 03:48:45 │ NewPart    │    2 │ all_3_3_0   │     0 │ default  │ jbod1     │
     4. │ 2024-08-04 03:48:46 │ MutatePart │    2 │ all_1_1_0_4 │     0 │ default  │ jbod1     │
     5. │ 2024-08-04 03:48:46 │ MutatePart │    2 │ all_2_2_0_4 │     0 │ default  │ jbod2     │
     6. │ 2024-08-04 03:48:46 │ MutatePart │    2 │ all_3_3_0_4 │     0 │ default  │ jbod1     │
     7. │ 2024-08-04 03:48:47 │ MovePart   │    2 │ all_1_1_0_4 │     0 │ default  │ external  │
     8. │ 2024-08-04 03:48:47 │ MovePart   │    2 │ all_3_3_0_4 │     0 │ default  │ jbod2     │
     9. │ 2024-08-04 03:48:47 │ MergeParts │    6 │ all_1_3_1_4 │     0 │ default  │ jbod2     │
    10. │ 2024-08-04 03:48:48 │ MovePart   │    6 │ all_1_3_1_4 │     0 │ default  │ external  │
    11. │ 2024-08-04 03:48:52 │ MergeParts │    4 │ all_1_3_2_4 │     0 │ default  │ external  │
    12. │ 2024-08-04 03:48:53 │ MergeParts │    0 │ all_1_3_3_4 │     0 │ default  │ external  │ # rows==0
    13. │ 2024-08-04 03:48:53 │ MergeParts │    0 │ all_1_3_4_4 │     0 │ default  │ external  │
        └─────────────────────┴────────────┴──────┴─────────────┴───────┴──────────┴───────────┘

CI: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests__tsan__[5_6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 14:25:18 +02:00
Azat Khuzhin
47dbc5e05b tests: add debug info into test_ttl_move::test_alter_with_merge_work 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 14:16:40 +02:00
Azat Khuzhin
f0aaac3bd1 tests: remove useless retries from test_ttl_move::test_alter_with_merge_work 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 14:14:42 +02:00
Robert Schulze
e4c165e919
Merge pull request #67103 from bigo-sg/fix_map_from_arrays 
Function `mapFromArrays`: Allow `Map(K, V)` and `Array(Nullable(T))` as 1st arguments
 2024-08-04 09:54:50 +00:00
Robert Schulze
ee8079b039
Merge remote-tracking branch 'ClickHouse/master' into icu-s390x 2024-08-04 09:38:36 +00:00
Kseniia Sumarokova
4f1c2ad277
Merge pull request #67459 from ClickHouse/write-buffer-from-s3-fix-dtor 
Fix wait for tasks in ~WriteBufferFromS3 in case WriteBuffer was cancelled
 2024-08-04 09:27:24 +00:00
Azat Khuzhin
dc527b6fd1 tests: detailed errors for RabbitMQ startup 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 09:07:05 +02:00
Azat Khuzhin
264be9c598 tests: increase timeout for RabbitMQ startup 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 09:06:50 +02:00
Azat Khuzhin
59e5a09b49 Use RabbitMQ without management in tests (attempt to improve startup) 
Sometimes startup can take ~90 seconds [1]:

    2024-08-03 23:11:38.756067+00:00 [info] <0.9.0> Time to start RabbitMQ: 94651980 us

Unlike normally ~10 [2]:

    2024-08-04 00:33:07.016137+00:00 [info] <0.9.0> Time to start RabbitMQ: 10082489 us

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/67737/b4e3bbcb82158bea4f5db1d9f5c28cfb741d1d51/integration_tests__asan__old_analyzer__[4_6].html
  [2]: https://s3.amazonaws.com/clickhouse-test-reports/66671/2f00c962711e13ca00af324366421fe4593b4ce6/integration_tests__asan__old_analyzer__[4_6].html

I've tried locally, the difference is very small, 3135665 us
(+management) vs (2740747 us), but still something, and who knows how it
works under pressure.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 09:04:41 +02:00
Nikita Fomichev
c6c4b313e6
Merge pull request #67738 from azat/tests/fix-retries 
Fix test retries
 2024-08-04 06:33:48 +00:00
Peter Nguyen
a31e2a4a5a
Merge branch 'ClickHouse:master' into aliasFeature 2024-08-03 21:38:42 -06:00
Alexey Milovidov
3efa208680
Merge pull request #67729 from ClickHouse/fix-typo 
Fix a typo
 2024-08-03 23:04:41 +00:00
Alexey Milovidov
6a20f777f6
Merge pull request #67428 from ClickHouse/enable-text-log 
Enable `text_log` by default
 2024-08-03 23:02:40 +00:00
Alexey Milovidov
e7686494f5
Merge pull request #67727 from ClickHouse/fix-host-resolver-ubsan 
Fix strange code in HostResolvePool
 2024-08-03 22:41:14 +00:00
Alexey Milovidov
185b6a54da Merge with master 2024-08-04 00:34:58 +02:00
Alexey Milovidov
d37a4567f6 Merge branch 'master' into fix-json-recursion 2024-08-04 00:30:29 +02:00
Alexey Milovidov
b8050cc471
Merge pull request #67702 from ClickHouse/fix-timer-descriptor-locks 
Fix locking inside TimerDescriptor
 2024-08-04 00:28:23 +02:00
Alexey Milovidov
4e3202d906
Merge pull request #67645 from ClickHouse/fix-race-condition-in-settings-processlist 
Fix race condition between ProcessList and Settings
 2024-08-03 22:21:49 +00:00
Alexey Milovidov
aa3b4595fa
Merge pull request #67701 from ClickHouse/improve-dashboard 
Improve the dashboard
 2024-08-03 22:16:07 +00:00
Alexey Milovidov
7a066a6505 Add a test 2024-08-03 23:48:51 +02:00
Alexey Milovidov
8562a6106c Better safety thresholds in arrayWithConstant 2024-08-03 23:45:18 +02:00
Azat Khuzhin
9d0e066cda Bump NuRaft (to properly catch thread exceptions) 
Refs: https://github.com/ClickHouse/NuRaft/pull/75
Refs: https://github.com/eBay/NuRaft/pull/525
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-03 23:41:50 +02:00
Azat Khuzhin
f06ae2f551 Fill only selected columns from system.clusters 
Some of them pretty heavy, i.e. is_active for ReplicatedDatabase

This should fix 02903_rmt_retriable_merge_exception flakiness [1].

 [1]: https://s3.amazonaws.com/clickhouse-test-reports/67687/89c47df559ba23d988f8af3c342e0c8d5531f4b8/fast_test.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-03 23:37:26 +02:00
Azat Khuzhin
a197502341 Fix test retries 
Should fix issues like:
- 02494_zero_copy_projection_cancel_fetch - https://s3.amazonaws.com/clickhouse-test-reports/67719/40cd5467c18d65a6624d273ac1a8fd9cc9257d8c/stateless_tests__tsan__s3_storage__[4_4].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-03 23:11:20 +02:00
Alexey Milovidov
d9d30d1004
Merge pull request #67295 from canhld94/fix_empty_tuple_in_lhs_in 
Fix hung query when using empty tuple as lhs of function IN
 2024-08-03 20:57:59 +00:00
Alexey Milovidov
02b8d563e3
Merge pull request #67720 from vitlibar/fix-completing-restore-on-cluster 
Fix completion of RESTORE ON CLUSTER
 2024-08-03 18:58:32 +00:00