thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-13 19:14:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
74,366 Commits 9,130 Branches 1,774 Tags 2 GiB
56ba0423f1
Commit Graph

74366 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Maksim Kita
85f280f017
Merge pull request #29951 from vitlibar/fix-access-control-shutdown 
Fix shutdown of AccessControlManager
 2021-10-10 23:56:40 +03:00
Azat Khuzhin
f37dac03f4 Fix lock-order-inversion between DROP TABLE for DatabaseMemory and LiveView 
CI stress founds [1], TSan report:

WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=509)
  Cycle in lock order graph: M71436201 (0x7b5800a91308) => M237700395169415472 (0x000000000000) => M71436201

  Mutex M237700395169415472 acquired here while holding mutex M71436201 in thread T723:
    3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse+0x154b0db9)
    4 DB::DatabaseWithOwnTablesBase::tryGetTable() const obj-x86_64-linux-gnu/../src/Databases/DatabasesCommon.cpp:37:21 (clickhouse+0x154b0db9)
    5 DB::DatabaseMemory::tryGetTableUUID() const obj-x86_64-linux-gnu/../src/Databases/DatabaseMemory.cpp:95:22 (clickhouse+0x15466bb5)
    6 DB::Context::resolveStorageID() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2672:90 (clickhouse+0x155e6aa1)
    7 DB::JoinedTables::getLeftTableStorage() obj-x86_64-linux-gnu/../src/Interpreters/JoinedTables.cpp:200:29 (clickhouse+0x15eee962)
    8 DB::InterpreterSelectQuery::InterpreterSelectQuery() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:321:33 (clickhouse+0x15b792be)
    9 DB::InterpreterSelectQuery::InterpreterSelectQuery() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:160:7 (clickhouse+0x15b78160)
    10 DB::StorageLiveView::collectMergeableBlocks() obj-x86_64-linux-gnu/../src/Storages/LiveView/StorageLiveView.cpp:113:28 (clickhouse+0x16aae192)
    11 DB::StorageLiveView::getNewBlocks() obj-x86_64-linux-gnu/../src/Storages/LiveView/StorageLiveView.cpp:384:33 (clickhouse+0x16ab393a)
    12 DB::StorageLiveView::refresh(bool) obj-x86_64-linux-gnu/../src/Storages/LiveView/StorageLiveView.cpp:528:13 (clickhouse+0x16ab6395)
    13 DB::StorageLiveView::read() obj-x86_64-linux-gnu/../src/Storages/LiveView/StorageLiveView.cpp:545:9 (clickhouse+0x16ab6395)
    21 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:950:30 (clickhouse+0x16101441)

    Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message

  Mutex M71436201 acquired here while holding mutex M237700395169415472 in thread T723:
    3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse+0x16ab5c29)
    4 DB::StorageLiveView::drop() obj-x86_64-linux-gnu/../src/Storages/LiveView/StorageLiveView.cpp:477:21 (clickhouse+0x16ab5c29)
    5 DB::DatabaseMemory::dropTable() obj-x86_64-linux-gnu/../src/Databases/DatabaseMemory.cpp:44:16 (clickhouse+0x15465cd6)
    8 DB::InterpreterDropQuery::execute() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterDropQuery.cpp:62:16 (clickhouse+0x15afa679)

  [1]: https://clickhouse-test-reports.s3.yandex.net/29856/42ca2b4bb241827edf69bbd6938d6b19c31935f1/stress_test_(thread).html#fail1

v2: fix attachTableUnlocked() (as pointed by @tavplubix)
v3: use SCOPE_EXIT() and so avoid holding a lock during removing from fs
 2021-10-10 23:51:36 +03:00
Azat Khuzhin
6c33eaee32 Fix filtering by tuple (some conditions was lost during analyzing) 
Fixes: #29281
Fixes: test_cluster_copier/test_three_nodes.py::test
 2021-10-10 22:37:52 +03:00
Nikolai Kochetov
a95c28ec4b
Merge pull request #29898 from ClickHouse/remove-native-stream 
Remove some more streams.
 2021-10-10 21:01:16 +03:00
mikael
934b72ddd1 Add FreeBSD aarch64 bits for cross-builds for CI. 2021-10-10 18:54:38 +02:00
Vitaly Baranov
1dda596689 Fix releasing query ID and session ID at the end of query processing. 2021-10-10 17:34:45 +03:00
alexey-milovidov
af5150c38b
Merge pull request #29935 from ClickHouse/remove-trash-printf 
Remove `printf` function usage.
 2021-10-10 16:11:00 +03:00
mikael
5bd2fd0ba3 Add the missing bits for FreeBSD aarch64 2021-10-10 14:52:42 +02:00
mikael
714ac8efc4 FreeBSD aarch64 doesn't implement sbrk 2021-10-10 14:52:42 +02:00
mikael
689c9cdbb9 Add uc_mcontext for FreeBSD aarch64 2021-10-10 14:52:17 +02:00
Vitaly Baranov
28459b282d Fix shutdown of AccessControlManager. Now there cannot be configuration reloading after AccessControlManager has been destroyed. 2021-10-10 13:25:24 +03:00
tavplubix
1aeb5d55dd
Grep server log even if it contains binary data (#29903) 
* grep server log even if it contains binary data

* Update cluster.py

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-10-10 12:55:37 +03:00
tavplubix
ca8625f1a3
Merge pull request #29923 from ClickHouse/fix_tests_replication_check 
Fix some flaky tests
 2021-10-10 12:53:54 +03:00
Azat Khuzhin
0c5d26b647 Make 01085_max_distributed_connections less flaky 2021-10-10 09:57:27 +03:00
Azat Khuzhin
38de183379 Make 00984_parser_stack_overflow less flaky 
According to query_log/trace_log the problem is real:

    2021.10.10 07:52:37.702364 [ 3736 ] {c6a219f2-3df8-4474-8324-bb307ee7a7a9} <Debug> executeQuery: (from [::1]:38096) (comment: /usr/share/clickhouse-test/queries/0_stateless/00984_parser_stack_overflow.sh) SELECT [[[...
    2021.10.10 07:53:11.407949 [ 3736 ] {c6a219f2-3df8-4474-8324-bb307ee7a7a9} <Error> executeQuery: Code: 306. DB::Exception: Maximum parse depth (1000) exceeded. Consider rising max_parser_depth ...

From trace_log:

    2021-10-10 07:52:37.958938 DB::Dwarf::findDebugInfoOffset
    2021-10-10 07:53:12.583248 DB::Dwarf::findDebugInfoOffset

CI: https://clickhouse-test-reports.s3.yandex.net/29928/2bab572caf91d2fc8962f409fea34149ecec8c95/functional_stateless_tests_(debug).html#fail1
 2021-10-10 09:57:26 +03:00
Azat Khuzhin
452b3b443d copier: add ability to configure retries and delays between them 
The following options had been introduced:
- max-table-tries
- max-shard-partition-tries
- max-shard-partition-piece-tries-for-alter
- retry-delay-ms
 2021-10-10 08:58:35 +03:00
Alexey Milovidov
103b3c91ba Remove 'printf' function usage. 2021-10-10 04:21:07 +03:00
Alexey Milovidov
daf4cc6aba Whitespaces 2021-10-10 04:09:17 +03:00
Azat Khuzhin
4bc90d1dd7 Fix data-race between flush() and startup() in StorageBuffer 
Stress tests found [1], TSan report:

    ==================
    WARNING: ThreadSanitizer: data race (pid=485)
      Read of size 8 at 0x7b5001280bd8 by thread T567 (mutexes: write M612061890855345680):
        1 std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>::operator bool() const obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2851:62 (clickhouse+0x159140a6)
        2 bool std::__1::operator!=<DB::BackgroundSchedulePoolTaskInfo>() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3447:30 (clickhouse+0x159140a6)
        3 DB::BackgroundSchedulePoolTaskHolder::operator bool() const obj-x86_64-linux-gnu/../src/Core/BackgroundSchedulePool.h:164:46 (clickhouse+0x159140a6)
        4 DB::StorageBuffer::flush() obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:675:10 (clickhouse+0x159140a6)

      Previous write of size 8 at 0x7b5001280bd8 by thread T586 (mutexes: write M191819750614415520):
        2 std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>::operator=(std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>&&) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3243:34 (clickhouse+0x15913e22)
        3 DB::BackgroundSchedulePoolTaskHolder::operator=() obj-x86_64-linux-gnu/../src/Core/BackgroundSchedulePool.h:156:110 (clickhouse+0x15913e22)
        4 DB::StorageBuffer::startup() obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:668:18 (clickhouse+0x15913e22)
        5 DB::InterpreterCreateQuery::doCreateTable() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:1092:10 (clickhouse+0x149bef7b)
        6 DB::InterpreterCreateQuery::createTable() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:952:20 (clickhouse+0x149ba9f5)
        7 DB::InterpreterCreateQuery::execute() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:1302:16 (clickhouse+0x149c1086)

  [1]: https://clickhouse-test-reports.s3.yandex.net/0/1c9778603ff49563d1d3d0d357de0608167e504d/stress_test_(thread).html

Fixes: #29416
 2021-10-10 04:03:36 +03:00
alexey-milovidov
2c3b866ab1
Merge pull request #29931 from azat/fix-build 
Fix build due to conflicts in serverConstants
 2021-10-10 03:56:39 +03:00
Azat Khuzhin
cc1fbe27a7 Fix build due to conflicts in serverConstants 
Refs: #29755
Refs: #29913
 2021-10-10 03:40:51 +03:00
alexey-milovidov
fbbc515a2f
Merge pull request #29925 from ClickHouse/update-cctz-2 
Update CCTZ
 2021-10-10 02:17:09 +03:00
Maksim Kita
84555646ff
Merge pull request #29927 from ClickHouse/better-exception-message-in-reading-arrow 
Better exception message while reading column from Arrow-supported formats
 2021-10-10 01:18:43 +03:00
Maksim Kita
daf9cf12d9
Merge pull request #29755 from Joeywzr/new_func 
add function getOSKernelVersion
 2021-10-10 01:12:54 +03:00
Maksim Kita
b9969232f5
Update 02095_function_get_os_kernel_version.sql 2021-10-10 01:12:36 +03:00
Alexander Tokmakov
c61f09c4dc fix test for replication consistency 2021-10-09 21:12:15 +03:00
alexey-milovidov
ee577e1ab4
Merge pull request #29155 from FArthur-cmd/add_CORS 
Add Cors support
 2021-10-09 20:18:17 +03:00
Maksim Kita
889034f6c2
Merge pull request #29913 from amosbird/refactor1 
Cosmetic refactoring of server constants.
 2021-10-09 19:59:27 +03:00
Alexey Milovidov
0828ecae97 Better exception message while reading column from Arrow-supported formats 2021-10-09 19:46:36 +03:00
Vladimir C
8ca3c7be3a
Merge pull request #29881 from vesslanjin/optimize-column-string-filter-performance 2021-10-09 19:44:15 +03:00
Nikolai Kochetov
102dd33cd8
Merge pull request #29841 from ClickHouse/coro-example 
Add coroutines example.
 2021-10-09 19:17:30 +03:00
Nikolai Kochetov
f908531b76 Fix debug build. 2021-10-09 19:12:47 +03:00
WangZengrui
bc093e2b23 fix test 2021-10-10 00:06:11 +08:00
alexey-milovidov
f0b7e54e83
Update CHANGELOG.md 2021-10-09 18:42:53 +03:00
Alexey Milovidov
f25fbe3740 Update CCTZ 2021-10-09 18:37:38 +03:00
tavplubix
017b3eccbe
Merge pull request #29901 from ClickHouse/logging_in_zk_client 
Add logging in ZooKeeper client
 2021-10-09 18:25:40 +03:00
alexey-milovidov
268c155b7d
Merge pull request #29856 from azat/clickhouse-test-python-client 
Rewrite clickhouse-test to use python clickhouse_driver
 2021-10-09 17:58:40 +03:00
Artur
c973d6d059 Merge branch 'add_CORS' of https://github.com/FArthur-cmd/ClickHouse into add_CORS 2021-10-09 13:55:56 +00:00
Artur
24568c9de5 update test reference 2021-10-09 13:55:30 +00:00
Azat Khuzhin
c66942bba5 Fix test_input_format_parallel_parsing_memory_tracking::test_memory_tracking_total 
Server has pretty low memory limit 3GB, so let's use local to generate
the data.
 2021-10-09 16:15:42 +03:00
Filatenkov Artur
7753cc9c28
Update users.xml 2021-10-09 16:01:01 +03:00
Filatenkov Artur
98ecd59ebe
Update users.xml 2021-10-09 16:00:37 +03:00
Artur
a853cd57ca refactoring 2021-10-09 12:56:00 +00:00
Artur
12f59f2dca refactor 2021-10-09 12:52:20 +00:00
Artur
473f7bee3a refactor 2021-10-09 12:16:37 +00:00
Azat Khuzhin
b8bde2d451 Fix test_backup_restore after #29649 
CI: https://clickhouse-test-reports.s3.yandex.net/29856/42ca2b4bb241827edf69bbd6938d6b19c31935f1/integration_tests_(asan).html#fail1
Cc: @CurtizJ
 2021-10-09 15:14:20 +03:00
WangZengrui
96a9d99ab6 add test 2021-10-09 19:53:21 +08:00
Vladimir C
3940382c2f
Remove defined(__POPCNT__) from column filter 2021-10-09 13:34:02 +03:00
Maksim Kita
0358d664b9
Merge pull request #29882 from MaxWk/add-alias-for-name 
Add table alias to system.tables and database alias to system.databases
 2021-10-09 12:44:40 +03:00
WangZengrui
2204597cfe add FunctionOSKernelVersion 2021-10-09 16:49:49 +08:00