thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 11:22:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
74,246 Commits 9,338 Branches 1,783 Tags 2.1 GiB
a9a7fd94cd
Commit Graph

74246 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
tavplubix
a9a7fd94cd
Merge pull request #29929 from azat/db-fix-lock-order-inversion 
Fix lock-order-inversion between DROP TABLE for DatabaseMemory and LiveView
 2021-10-11 13:23:58 +03:00
Kseniia Sumarokova
20e8074b5e
Merge pull request #29912 from taiyang-li/remove_unused_var 
Remove unused variable in s3cluster
 2021-10-11 12:45:04 +03:00
Nikita Mikhaylov
32d77aafe2
Merge pull request #29921 from azat/copier-fixes 
copier: add ability to configure retries and delays between them
 2021-10-11 12:11:38 +03:00
Maksim Kita
c368b20dce
Merge pull request #29982 from kitaisreal/tests-naming-fix 
Tests naming fix
 2021-10-11 11:35:37 +03:00
Maksim Kita
e05d041537 Tests naming fix 2021-10-11 11:34:58 +03:00
Vladimir C
d2b8d293b4
Merge pull request #29976 from hexiaoting/dev_quantile_fuse2 2021-10-11 10:06:40 +03:00
Vladimir C
6c7fbf0b88
Merge pull request #29956 from azat/fix-tuple-filter 2021-10-11 10:02:47 +03:00
Vitaly Baranov
e74a4a8b6c
Merge pull request #28934 from vitlibar/allow-macros-in-rabbitmq-settings-test 
Add test for expanding macros in RabbitMQ settings.
 2021-10-11 08:32:24 +03:00
Alexey Milovidov
1338f3daf8 Add missing files to libc-headers. 2021-10-11 07:44:12 +03:00
hexiaoting
aabe52e3e9 Fix bug for quantile fusion 2021-10-11 10:39:55 +08:00
alexey-milovidov
0adbcd58c9
Merge pull request #29930 from azat/buffer-fix-data-race 
Fix data-race between flush() and startup() in StorageBuffer
 2021-10-11 01:11:39 +03:00
alexey-milovidov
c64ca7be08
Merge pull request #29947 from azat/stateless-tests-fixes-v21.10 
Stateless flaky tests fixes
 2021-10-11 01:09:57 +03:00
alexey-milovidov
765f7b692c
Merge pull request #29919 from azat/integration-tests-fixes-v21.10 
Fix flaky integration tests (test_backup_restore/test_input_format_parallel_parsing_memory_tracking)
 2021-10-11 00:14:06 +03:00
alexey-milovidov
a70249f8f7
Merge pull request #29932 from ClickHouse/Whitespaces 
Minor changes
 2021-10-11 00:11:51 +03:00
alexey-milovidov
341eb1f9bf
Update adopters.md 2021-10-11 00:05:53 +03:00
Vitaly Baranov
07aea95a75 Fix rabbitmq tests. 2021-10-11 00:00:12 +03:00
Maksim Kita
85f280f017
Merge pull request #29951 from vitlibar/fix-access-control-shutdown 
Fix shutdown of AccessControlManager
 2021-10-10 23:56:40 +03:00
Azat Khuzhin
f37dac03f4 Fix lock-order-inversion between DROP TABLE for DatabaseMemory and LiveView 
CI stress founds [1], TSan report:

WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=509)
  Cycle in lock order graph: M71436201 (0x7b5800a91308) => M237700395169415472 (0x000000000000) => M71436201

  Mutex M237700395169415472 acquired here while holding mutex M71436201 in thread T723:
    3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse+0x154b0db9)
    4 DB::DatabaseWithOwnTablesBase::tryGetTable() const obj-x86_64-linux-gnu/../src/Databases/DatabasesCommon.cpp:37:21 (clickhouse+0x154b0db9)
    5 DB::DatabaseMemory::tryGetTableUUID() const obj-x86_64-linux-gnu/../src/Databases/DatabaseMemory.cpp:95:22 (clickhouse+0x15466bb5)
    6 DB::Context::resolveStorageID() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2672:90 (clickhouse+0x155e6aa1)
    7 DB::JoinedTables::getLeftTableStorage() obj-x86_64-linux-gnu/../src/Interpreters/JoinedTables.cpp:200:29 (clickhouse+0x15eee962)
    8 DB::InterpreterSelectQuery::InterpreterSelectQuery() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:321:33 (clickhouse+0x15b792be)
    9 DB::InterpreterSelectQuery::InterpreterSelectQuery() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:160:7 (clickhouse+0x15b78160)
    10 DB::StorageLiveView::collectMergeableBlocks() obj-x86_64-linux-gnu/../src/Storages/LiveView/StorageLiveView.cpp:113:28 (clickhouse+0x16aae192)
    11 DB::StorageLiveView::getNewBlocks() obj-x86_64-linux-gnu/../src/Storages/LiveView/StorageLiveView.cpp:384:33 (clickhouse+0x16ab393a)
    12 DB::StorageLiveView::refresh(bool) obj-x86_64-linux-gnu/../src/Storages/LiveView/StorageLiveView.cpp:528:13 (clickhouse+0x16ab6395)
    13 DB::StorageLiveView::read() obj-x86_64-linux-gnu/../src/Storages/LiveView/StorageLiveView.cpp:545:9 (clickhouse+0x16ab6395)
    21 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:950:30 (clickhouse+0x16101441)

    Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message

  Mutex M71436201 acquired here while holding mutex M237700395169415472 in thread T723:
    3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse+0x16ab5c29)
    4 DB::StorageLiveView::drop() obj-x86_64-linux-gnu/../src/Storages/LiveView/StorageLiveView.cpp:477:21 (clickhouse+0x16ab5c29)
    5 DB::DatabaseMemory::dropTable() obj-x86_64-linux-gnu/../src/Databases/DatabaseMemory.cpp:44:16 (clickhouse+0x15465cd6)
    8 DB::InterpreterDropQuery::execute() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterDropQuery.cpp:62:16 (clickhouse+0x15afa679)

  [1]: https://clickhouse-test-reports.s3.yandex.net/29856/42ca2b4bb241827edf69bbd6938d6b19c31935f1/stress_test_(thread).html#fail1

v2: fix attachTableUnlocked() (as pointed by @tavplubix)
v3: use SCOPE_EXIT() and so avoid holding a lock during removing from fs
 2021-10-10 23:51:36 +03:00
Azat Khuzhin
6c33eaee32 Fix filtering by tuple (some conditions was lost during analyzing) 
Fixes: #29281
Fixes: test_cluster_copier/test_three_nodes.py::test
 2021-10-10 22:37:52 +03:00
Nikolai Kochetov
a95c28ec4b
Merge pull request #29898 from ClickHouse/remove-native-stream 
Remove some more streams.
 2021-10-10 21:01:16 +03:00
alexey-milovidov
af5150c38b
Merge pull request #29935 from ClickHouse/remove-trash-printf 
Remove `printf` function usage.
 2021-10-10 16:11:00 +03:00
Vitaly Baranov
28459b282d Fix shutdown of AccessControlManager. Now there cannot be configuration reloading after AccessControlManager has been destroyed. 2021-10-10 13:25:24 +03:00
tavplubix
1aeb5d55dd
Grep server log even if it contains binary data (#29903) 
* grep server log even if it contains binary data

* Update cluster.py

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-10-10 12:55:37 +03:00
tavplubix
ca8625f1a3
Merge pull request #29923 from ClickHouse/fix_tests_replication_check 
Fix some flaky tests
 2021-10-10 12:53:54 +03:00
Azat Khuzhin
0c5d26b647 Make 01085_max_distributed_connections less flaky 2021-10-10 09:57:27 +03:00
Azat Khuzhin
38de183379 Make 00984_parser_stack_overflow less flaky 
According to query_log/trace_log the problem is real:

    2021.10.10 07:52:37.702364 [ 3736 ] {c6a219f2-3df8-4474-8324-bb307ee7a7a9} <Debug> executeQuery: (from [::1]:38096) (comment: /usr/share/clickhouse-test/queries/0_stateless/00984_parser_stack_overflow.sh) SELECT [[[...
    2021.10.10 07:53:11.407949 [ 3736 ] {c6a219f2-3df8-4474-8324-bb307ee7a7a9} <Error> executeQuery: Code: 306. DB::Exception: Maximum parse depth (1000) exceeded. Consider rising max_parser_depth ...

From trace_log:

    2021-10-10 07:52:37.958938 DB::Dwarf::findDebugInfoOffset
    2021-10-10 07:53:12.583248 DB::Dwarf::findDebugInfoOffset

CI: https://clickhouse-test-reports.s3.yandex.net/29928/2bab572caf91d2fc8962f409fea34149ecec8c95/functional_stateless_tests_(debug).html#fail1
 2021-10-10 09:57:26 +03:00
Azat Khuzhin
452b3b443d copier: add ability to configure retries and delays between them 
The following options had been introduced:
- max-table-tries
- max-shard-partition-tries
- max-shard-partition-piece-tries-for-alter
- retry-delay-ms
 2021-10-10 08:58:35 +03:00
Alexey Milovidov
103b3c91ba Remove 'printf' function usage. 2021-10-10 04:21:07 +03:00
Alexey Milovidov
daf4cc6aba Whitespaces 2021-10-10 04:09:17 +03:00
Azat Khuzhin
4bc90d1dd7 Fix data-race between flush() and startup() in StorageBuffer 
Stress tests found [1], TSan report:

    ==================
    WARNING: ThreadSanitizer: data race (pid=485)
      Read of size 8 at 0x7b5001280bd8 by thread T567 (mutexes: write M612061890855345680):
        1 std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>::operator bool() const obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2851:62 (clickhouse+0x159140a6)
        2 bool std::__1::operator!=<DB::BackgroundSchedulePoolTaskInfo>() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3447:30 (clickhouse+0x159140a6)
        3 DB::BackgroundSchedulePoolTaskHolder::operator bool() const obj-x86_64-linux-gnu/../src/Core/BackgroundSchedulePool.h:164:46 (clickhouse+0x159140a6)
        4 DB::StorageBuffer::flush() obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:675:10 (clickhouse+0x159140a6)

      Previous write of size 8 at 0x7b5001280bd8 by thread T586 (mutexes: write M191819750614415520):
        2 std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>::operator=(std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>&&) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3243:34 (clickhouse+0x15913e22)
        3 DB::BackgroundSchedulePoolTaskHolder::operator=() obj-x86_64-linux-gnu/../src/Core/BackgroundSchedulePool.h:156:110 (clickhouse+0x15913e22)
        4 DB::StorageBuffer::startup() obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:668:18 (clickhouse+0x15913e22)
        5 DB::InterpreterCreateQuery::doCreateTable() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:1092:10 (clickhouse+0x149bef7b)
        6 DB::InterpreterCreateQuery::createTable() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:952:20 (clickhouse+0x149ba9f5)
        7 DB::InterpreterCreateQuery::execute() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:1302:16 (clickhouse+0x149c1086)

  [1]: https://clickhouse-test-reports.s3.yandex.net/0/1c9778603ff49563d1d3d0d357de0608167e504d/stress_test_(thread).html

Fixes: #29416
 2021-10-10 04:03:36 +03:00
alexey-milovidov
2c3b866ab1
Merge pull request #29931 from azat/fix-build 
Fix build due to conflicts in serverConstants
 2021-10-10 03:56:39 +03:00
Azat Khuzhin
cc1fbe27a7 Fix build due to conflicts in serverConstants 
Refs: #29755
Refs: #29913
 2021-10-10 03:40:51 +03:00
alexey-milovidov
fbbc515a2f
Merge pull request #29925 from ClickHouse/update-cctz-2 
Update CCTZ
 2021-10-10 02:17:09 +03:00
Maksim Kita
84555646ff
Merge pull request #29927 from ClickHouse/better-exception-message-in-reading-arrow 
Better exception message while reading column from Arrow-supported formats
 2021-10-10 01:18:43 +03:00
Maksim Kita
daf9cf12d9
Merge pull request #29755 from Joeywzr/new_func 
add function getOSKernelVersion
 2021-10-10 01:12:54 +03:00
Maksim Kita
b9969232f5
Update 02095_function_get_os_kernel_version.sql 2021-10-10 01:12:36 +03:00
Alexander Tokmakov
c61f09c4dc fix test for replication consistency 2021-10-09 21:12:15 +03:00
alexey-milovidov
ee577e1ab4
Merge pull request #29155 from FArthur-cmd/add_CORS 
Add Cors support
 2021-10-09 20:18:17 +03:00
Maksim Kita
889034f6c2
Merge pull request #29913 from amosbird/refactor1 
Cosmetic refactoring of server constants.
 2021-10-09 19:59:27 +03:00
Alexey Milovidov
0828ecae97 Better exception message while reading column from Arrow-supported formats 2021-10-09 19:46:36 +03:00
Vladimir C
8ca3c7be3a
Merge pull request #29881 from vesslanjin/optimize-column-string-filter-performance 2021-10-09 19:44:15 +03:00
Nikolai Kochetov
102dd33cd8
Merge pull request #29841 from ClickHouse/coro-example 
Add coroutines example.
 2021-10-09 19:17:30 +03:00
Nikolai Kochetov
f908531b76 Fix debug build. 2021-10-09 19:12:47 +03:00
WangZengrui
bc093e2b23 fix test 2021-10-10 00:06:11 +08:00
alexey-milovidov
f0b7e54e83
Update CHANGELOG.md 2021-10-09 18:42:53 +03:00
Alexey Milovidov
f25fbe3740 Update CCTZ 2021-10-09 18:37:38 +03:00
tavplubix
017b3eccbe
Merge pull request #29901 from ClickHouse/logging_in_zk_client 
Add logging in ZooKeeper client
 2021-10-09 18:25:40 +03:00
alexey-milovidov
268c155b7d
Merge pull request #29856 from azat/clickhouse-test-python-client 
Rewrite clickhouse-test to use python clickhouse_driver
 2021-10-09 17:58:40 +03:00
Artur
c973d6d059 Merge branch 'add_CORS' of https://github.com/FArthur-cmd/ClickHouse into add_CORS 2021-10-09 13:55:56 +00:00
Artur
24568c9de5 update test reference 2021-10-09 13:55:30 +00:00