thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-18 05:32:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
63,751 Commits 9,136 Branches 1,774 Tags 2 GiB
bb11cde871
Commit Graph

3218 Commits

Author SHA1 Message Date
Amos Bird
8a3b5c1fab
Add _partition_value virtual column 2021-04-27 16:15:59 +08:00
Maksim Kita
8b5a049ca0
Merge pull request #23635 from kitaisreal/aggregate-functions-remove-unused-code 
Aggregator remove unused code
 2021-04-27 00:52:34 +03:00
Nikita Mikhaylov
223c6b4ac5 better 2021-04-26 21:26:55 +03:00
Maksim Kita
c93666e0de Fixed code review issues 2021-04-26 13:52:41 +03:00
Maksim Kita
561b045991 Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
64f49a981b Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
2e3e90b25b Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
6ba83eccd3 Added support for DETACH DICTIONARY PERMANENTLY 2021-04-26 13:52:41 +03:00
Maksim Kita
3cb4d1bdcb Fixed style check 2021-04-26 13:52:41 +03:00
Maksim Kita
a9917a6540 Refactored StorageDictionary 2021-04-26 13:52:41 +03:00
Maksim Kita
5bb263f535 Fixed tests 2021-04-26 13:44:59 +03:00
Alexander Kuzmenkov
fed35445d9
Merge pull request #23546 from ClickHouse/aku/window-distributed 
fix window functions for Distributed tables
 2021-04-26 12:34:46 +03:00
Nikolai Kochetov
092ff0faeb
Merge pull request #23524 from kssenii/insert-union-select 
Allow to insert default instead of Null in insert-select / insert-select-union-all
 2021-04-26 12:25:49 +03:00
Maksim Kita
cac4a85286 Aggregator remove unused code 2021-04-26 01:17:24 +03:00
Alexey Milovidov
649550c5ab Attempt to normalize big integers 2021-04-25 12:30:43 +03:00
Alexey Milovidov
60be962dca Merging with master 2021-04-25 07:08:57 +03:00
Alexey Milovidov
8f01af62d9 Merge branch 'master' into normalize-bigint 2021-04-25 06:57:44 +03:00
alexey-milovidov
bcede10919
Merge pull request #23312 from vdimir/issue-23002 
Correct sample_block in HashJoin for dict join with join_algorithm
 2021-04-24 04:40:43 +03:00
alexey-milovidov
ca230224cf
Merge pull request #23549 from vdimir/issue-17718 
Remove clashed columns from totals for StorageJoin
 2021-04-24 03:19:00 +03:00
kssenii
3a9b11e141 Add check for columns sizes match 2021-04-23 19:09:09 +00:00
kssenii
d032651aba Try fix 2021-04-23 17:29:22 +00:00
Alexander Kuzmenkov
ab75f86ee9 add a comment 2021-04-23 18:45:45 +03:00
Alexander Kuzmenkov
6982ab5324 fix extra distinct 2021-04-23 15:44:46 +03:00
kssenii
7c3b074891 Better 2021-04-23 12:40:34 +00:00
vdimir
9460989721
Remove clashed columns from totals for StorageJoin 2021-04-23 15:37:05 +03:00
kssenii
8f34199538 Fixes 2021-04-23 12:36:40 +00:00
Alexander Kuzmenkov
9d95ac3d66 remove debug 2021-04-23 14:00:17 +03:00
Alexander Kuzmenkov
54ad086753 fix for distributed 2021-04-23 13:59:21 +03:00
kssenii
54a7a6ef0d Allow null to default convertion only if column has default 2021-04-23 09:55:57 +00:00
Alexander Kuzmenkov
7ea5bd4a3f Merge remote-tracking branch 'origin/master' into HEAD 2021-04-23 09:26:43 +03:00
kssenii
459612c895 Use a separate setting 2021-04-22 22:14:14 +00:00
kssenii
3176c24a8a Add test 2021-04-22 21:46:49 +00:00
kssenii
28c4436246 Better 2021-04-22 21:30:31 +00:00
Alexander Kuzmenkov
06ca44a378 fix pvs warnings 2021-04-22 22:54:58 +03:00
kssenii
ac55183d2e Null as default for insert select / with union 2021-04-22 18:11:36 +00:00
Maksim Kita
7357c35197 Fixed code review issues 2021-04-22 17:19:47 +03:00
Maksim Kita
8fde34fc52 Updated tests 2021-04-22 16:23:42 +03:00
Maksim Kita
86ffa9fcfa JoinToSubqueriesVisitor ASTColumnsMatcher fix 2021-04-22 13:14:35 +03:00
mergify[bot]
71bca99bbf
Merge branch 'master' into explain-output 2021-04-22 07:02:47 +00:00
Nikita Mikhaylov
d4e8f9676b
Merge pull request #21304 from amosbird/indexhint 
Resurrect indexHint function.
 2021-04-21 16:14:43 +03:00
alexey-milovidov
9c649932bc
Merge pull request #23375 from ClickHouse/fix-read-in-order-with-collation 
Fix read in order optimization of COLLATE is present #22379
 2021-04-21 15:00:02 +03:00
Maksim Kita
02515ff0a4
Merge pull request #23191 from kitaisreal/use-local-shard-if-no-remote-connections 
Updated QueryAliasVisitor to prefer alias for aliases like _subquery
 2021-04-21 00:58:47 +03:00
Pavel Kovalenko
10cf881a08 Add SYSTEM RESTART DISK command. 2021-04-20 22:23:54 +03:00
Pavel Kovalenko
98f8cbbdb2 DiskRestartProxy refactoring. 2021-04-20 21:58:34 +03:00
Alexey Milovidov
9bc0706d2f Fix read in order optimization of COLLATE is present #22379 2021-04-20 21:20:45 +03:00
tavplubix
738d8a757b
Merge pull request #23189 from ClickHouse/minor_fixes_in_attach_query 
Minor fixes in ATTACH query
 2021-04-20 20:04:26 +03:00
alexey-milovidov
632a5288fa
Merge pull request #23348 from azat/zero-streams 
Fix LOGICAL_ERROR for max_distributed_connections=0
 2021-04-20 19:48:21 +03:00
Nikolai Kochetov
6b264933b2 Fix style. 2021-04-20 17:51:48 +03:00
Nikolai Kochetov
942cd54f23 Fix copy paste thanks to PVS studio. 2021-04-20 16:41:37 +03:00
Anton Popov
8f57141fad
Merge pull request #23195 from CurtizJ/nested-update 
Merging #22503
 2021-04-20 14:57:47 +03:00
Amos Bird
32c84f77c3
Resurrect indexHint function. 2021-04-20 19:27:23 +08:00
Alexander Tokmakov
4108a235f5 make separate functions 2021-04-20 14:22:02 +03:00
Maksim Kita
457b75a196 Updated tests 2021-04-20 13:31:01 +03:00
Maksim Kita
cbc34afd55 Updated QueryAliasVisitor to set prefer_alias_to_column_name for aliases like _subquery 2021-04-20 13:16:29 +03:00
vdimir
3266bbb54c
TableJoin: forceHashJoin for dictionary_reader, add tests 2021-04-20 12:52:52 +03:00
Nikolai Kochetov
0c4e789900 Remove debug output 2021-04-20 10:08:35 +03:00
Azat Khuzhin
86069ffb40 Fix LOGICAL_ERROR for max_distributed_connections=0 2021-04-20 09:05:44 +03:00
Nikolai Kochetov
17f7631878 Fix build. 2021-04-20 08:55:20 +03:00
Anton Popov
0a0c510aa2 minor fixes 2021-04-20 05:51:07 +03:00
alexey-milovidov
62899436db
Revert "add uniqThetaSketch" 2021-04-20 03:34:21 +03:00
alexey-milovidov
5b66086bc9
Merge pull request #23219 from azat/optimize_skip_unused_shards_rewrite_in-fix 
Add type conversion for optimize_skip_unused_shards_rewrite_in
 2021-04-19 22:45:35 +03:00
vdimir
a95c4dcfff
Correct sample_block in HashJoin for dict join with join_algorithm = auto 2021-04-19 18:02:56 +03:00
Nikolai Kochetov
8c2d58333a Merge branch 'master' into explain-output 2021-04-19 17:40:27 +03:00
Nikolai Kochetov
c59628846c
Merge pull request #22352 from ClickHouse/add-read-from-mt-step 
Special query plan step for read from MergeTree
 2021-04-19 17:38:40 +03:00
Alexander Kuzmenkov
2a4bcb6e3f
Merge pull request #23159 from ClickHouse/aku/merge-fusecount 
merging sumCount fusion PR #21337
 2021-04-19 16:47:13 +03:00
Kruglov Pavel
995973bf1f
Merge pull request #22609 from pingyu/datasketches-uniq 
add uniqThetaSketch
 2021-04-19 10:32:29 +03:00
Nikolai Kochetov
4d86f51eff Merge branch 'master' into add-read-from-mt-step 2021-04-19 10:17:21 +03:00
Maksim Kita
ca93987ae9
Merge pull request #23182 from kitaisreal/system-query-reload-model 
Added system query reload model
 2021-04-19 00:39:31 +03:00
Azat Khuzhin
d476a4a9bd Add type conversion for optimize_skip_unused_shards_rewrite_in 
MSan reports [1]:

    2021.04.17 15:20:06.665152 [ 56 ] {2336bf92-0269-4acd-8b3f-f09623223d18} <Debug> executeQuery: (from [::1]:44744, using production parser) SELECT * FROM dist_01757 WHERE dummy IN ('255', 0) FORMAT Null
    ...
    0 0x305af885 in (anonymous namespace)::shardContains() obj-x86_64-linux-gnu/../src/Interpreters/OptimizeShardingKeyRewriteInVisitor.cpp:50:28
    ...
    Uninitialized value was created by an allocation of 'sharding_value' in the stack frame of function '_ZN12_GLOBAL__N_113shardContainsERKN2DB5FieldERKNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEERKNS4_10shared_ptrINS0_17ExpressionActionsEEESC_RKNS0_7Cluster9ShardInfoERKNS4_6vectorImNS8_ImEEEE'
    0 0x305ae260 in (anonymous namespace)::shardContains() obj-x86_64-linux-gnu/../src/Interpreters/OptimizeShardingKeyRewriteInVisitor.cpp:42

  [1]: https://clickhouse-test-reports.s3.yandex.net/23212/d02eb3f6b7a8dc07bbed5605e1afcf7634436b3a/fuzzer_msan/report.html#fail1

P.S. fuzzers are great!
 2021-04-19 00:03:38 +03:00
Azat Khuzhin
3f156b83c9 Add some comments in OptimizeShardingKeyRewriteInVisitor 2021-04-18 09:07:51 +03:00
alexey-milovidov
7f3a40e1b0
Merge pull request #22237 from ClickHouse/protocol-compression-auto 
Autodetect compression #22234
 2021-04-18 03:13:54 +03:00
Alexander Tokmakov
0354312294 minor fixes in attach query 2021-04-17 22:43:02 +03:00
vdimir
826296c799
Convert columns to LC(Nullable(T)) in join 2021-04-17 22:03:32 +03:00
Maksim Kita
abf2a87ce7 Added system query reload model 2021-04-17 20:59:08 +03:00
Alexey Milovidov
77e64b3ebd Merge branch 'master' into protocol-compression-auto 2021-04-17 16:46:51 +03:00
alexey-milovidov
138c6ae228
Merge pull request #23185 from ClickHouse/fix_22573 
Fix data_type_default_nullable on ATTACH
 2021-04-17 16:13:42 +03:00
Alexey Milovidov
ec473a90d2 Fixed missing semicolon in exception message 2021-04-17 02:14:48 +03:00
alexey-milovidov
74a0ac6917
Merge pull request #23160 from azat/logging-v2 
[RFC] Change logging from trace to debug for messages with rows/bytes
 2021-04-16 21:18:32 +03:00
Nikolai Kochetov
1bf420cbcf Update json description for other steps. 2021-04-16 19:36:59 +03:00
Maksim Kita
d88e0b1b44 Use local shard if there are no remote connections 2021-04-16 16:01:53 +03:00
Nikolai Kochetov
8f969ec063 Revert "Ignore output format for explain query on client." 
This reverts commit 3f7cee1149.
 2021-04-16 15:44:27 +03:00
Nikolai Kochetov
a5f8b0f269 Revert "Omit EXPLAIN FORMAT for old TCP clients." 
This reverts commit 4c37d461de.
 2021-04-16 15:44:18 +03:00
Anton Popov
b14798af2a Merge branch 'dev_nested_upadte' of git://github.com/hexiaoting/ClickHouse into nested-update 2021-04-16 14:41:19 +03:00
Nikolai Kochetov
23089a2fec Add special setting to show indexes. 2021-04-16 14:11:45 +03:00
Alexander Kuzmenkov
5e0fb440a5
Update TreeRewriter.cpp 2021-04-16 13:35:02 +03:00
Alexander Tokmakov
ebf0a3119e fix data_type_default_nullable on attach 2021-04-16 13:29:33 +03:00
tavplubix
ae3dc3dfc0
Merge pull request #23161 from ClickHouse/logs_for_debuging_test_failures 
More verbose logs for debuging test failures with Replicated and Keeper
 2021-04-16 12:01:58 +03:00
Alexander Tokmakov
0ad6205fa6 logs for debuging test failures with Replicated and Keeper 2021-04-15 21:34:53 +03:00
Azat Khuzhin
d2cf03ea41 Change logging from trace to debug for messages with rows/bytes 2021-04-15 21:00:16 +03:00
Alexander Kuzmenkov
2489b6af96 cleanup 2021-04-15 19:40:49 +03:00
Alexander Kuzmenkov
3b95b637a5 Merge remote-tracking branch 'origin/master' into HEAD 2021-04-15 18:19:53 +03:00
alexey-milovidov
69eac517b6
Merge pull request #23086 from azat/memory-blocking 
Block all memory tracking limits in dtors/SCOPE_EXIT_*SAFE/tryLogCurrentException
 2021-04-15 14:36:10 +03:00
Vladimir
8c7ffda676
Merge pull request #22753 from ClickHouse/revert-19685-dev_joinon 2021-04-15 12:42:50 +03:00
alexey-milovidov
9b546f3b89
Merge pull request #22657 from amosbird/ctefix3 
Fix CTE usage in view definition
 2021-04-15 02:01:10 +03:00
Alexey Milovidov
17d04cbca9 Merge branch 'master' into protocol-compression-auto 2021-04-15 00:24:07 +03:00
tavplubix
2479c80fb7
Merge branch 'master' into fix_attach_mv 2021-04-15 00:07:48 +03:00
Azat Khuzhin
9b9e0a9bbc Block all memory tracking limits in destructors 2021-04-14 23:38:42 +03:00
Nikolai Kochetov
4c37d461de Omit EXPLAIN FORMAT for old TCP clients. 2021-04-14 18:59:08 +03:00
Nikolai Kochetov
3f7cee1149 Ignore output format for explain query on client. 2021-04-14 18:33:07 +03:00
Nikolai Kochetov
9b86c19836 Use JSONBuilder for explain json result. 2021-04-14 17:07:56 +03:00
alexey-milovidov
f8ce5547aa
Merge pull request #21318 from azat/servers-pool-wait 
Fix SIGSEGV by waiting servers thread pool
 2021-04-14 14:15:07 +03:00
hexiaoting
61e2ffc0dd Fix build error 2021-04-14 16:53:06 +08:00
Alexey Milovidov
6f56c3280f Uncompress data in Distributed sends if needed 2021-04-14 00:53:39 +03:00
Alexey Milovidov
e08c8a3d2c Merge branch 'master' into protocol-compression-auto 2021-04-13 23:43:08 +03:00
Nikita Mikhaylov
024374a2ec review fixes 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
704fb04941 better 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
c333c3dedb review fixes 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
36a8419f60 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
8a4b5a586e fixed uuid 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
7276b40556 better 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4843f86329 use only one connection between initiator and worker 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4e4b383214 added hash of itiator address 2021-04-13 22:39:41 +03:00
Alexander Tokmakov
05e4c8d3ef fix attach mv in atomic db 2021-04-13 22:13:26 +03:00
Azat Khuzhin
d59bdfd45d Fix one more lock-order-inversion 
TSan report [1]:

WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=11314)
  Cycle in lock order graph: M183938897938677368 (0x000000000000) => M2505 (0x7b9000002008) => M183938897938677368

  Mutex M2505 acquired here while holding mutex M183938897938677368 in thread T6:
    0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a327b6)
    1 std::__1::__libcpp_recursive_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:385:10 (clickhouse-tsan+0x17cdb689)
    2 std::__1::recursive_mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:71:14 (clickhouse-tsan+0x17cdb689)
    3 std::__1::unique_lock<std::__1::recursive_mutex>::unique_lock(std::__1::recursive_mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:119:61 (clickhouse-tsan+0x11e3506f)
    4 DB::Context::getLock() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:517:12 (clickhouse-tsan+0x11e3506f)
    5 DB::Context::getSchedulePool() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1517:17 (clickhouse-tsan+0x11e3506f)
    6 DB::IBackgroundJobExecutor::start() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:158:42 (clickhouse-tsan+0x12be1cda)
    7 DB::StorageMergeTree::startup() obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:112:29 (clickhouse-tsan+0x129ed46e)
    8 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_2::operator()(std::__1::shared_ptr<DB::IStorage> const&) const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:230:16 (clickhouse-tsan+0x11d71fba)
    9 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3::operator()() const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:238:56 (clickhouse-tsan+0x11d71fba)
    10 decltype(std::__1::forward<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(fp)()) std::__1::__invoke<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x11d71fba)
    11 void std::__1::__invoke_void_return_wrapper<void>::__call<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x11d71fba)
    12 std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x11d71fba)
    13 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x11d71fba)
    14 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b3b8e0)
    15 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b3b8e0)
    16 ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b3b8e0)
    17 void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3e600)
    18 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(fp0)...)) std::__1::__invoke_constexpr<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3682:1 (clickhouse-tsan+0x8b3e600)
    19 decltype(auto) std::__1::__apply_tuple_impl<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::__tuple_indices<std::__1::tuple<>&...>) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1415:1 (clickhouse-tsan+0x8b3e600)
    20 decltype(auto) std::__1::apply<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1424:1 (clickhouse-tsan+0x8b3e600)
    21 ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()::operator()() obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:178:13 (clickhouse-tsan+0x8b3e600)
    22 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3e561)
    23 void std::__1::__invoke_void_return_wrapper<void>::__call<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x8b3e561)
    24 std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x8b3e561)
    25 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x8b3e561)
    26 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b39205)
    27 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b39205)
    28 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b39205)
    29 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3cea8)
    30 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3cea8)
    31 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>&, std::__1::__tuple_indices<>) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:280:5 (clickhouse-tsan+0x8b3cea8)
    32 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:291:5 (clickhouse-tsan+0x8b3cea8)

  Mutex M183938897938677368 previously acquired by the same thread here:
    0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a327b6)
    1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse-tsan+0x17cdb4f9)
    2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse-tsan+0x17cdb4f9)
    3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse-tsan+0x12be1ca9)
    4 DB::IBackgroundJobExecutor::start() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:155:21 (clickhouse-tsan+0x12be1ca9)
    5 DB::StorageMergeTree::startup() obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:112:29 (clickhouse-tsan+0x129ed46e)
    6 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_2::operator()(std::__1::shared_ptr<DB::IStorage> const&) const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:230:16 (clickhouse-tsan+0x11d71fba)
    7 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3::operator()() const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:238:56 (clickhouse-tsan+0x11d71fba)
    8 decltype(std::__1::forward<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(fp)()) std::__1::__invoke<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x11d71fba)
    9 void std::__1::__invoke_void_return_wrapper<void>::__call<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x11d71fba)
    10 std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x11d71fba)
    11 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x11d71fba)
    12 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b3b8e0)
    13 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b3b8e0)
    14 ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b3b8e0)
    15 void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3e600)
    16 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(fp0)...)) std::__1::__invoke_constexpr<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3682:1 (clickhouse-tsan+0x8b3e600)
    17 decltype(auto) std::__1::__apply_tuple_impl<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::__tuple_indices<std::__1::tuple<>&...>) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1415:1 (clickhouse-tsan+0x8b3e600)
    18 decltype(auto) std::__1::apply<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1424:1 (clickhouse-tsan+0x8b3e600)
    19 ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()::operator()() obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:178:13 (clickhouse-tsan+0x8b3e600)
    20 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3e561)
    21 void std::__1::__invoke_void_return_wrapper<void>::__call<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x8b3e561)
    22 std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x8b3e561)
    23 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x8b3e561)
    24 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b39205)
    25 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b39205)
    26 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b39205)
    27 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3cea8)
    28 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3cea8)
    29 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>&, std::__1::__tuple_indices<>) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:280:5 (clickhouse-tsan+0x8b3cea8)
    30 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:291:5 (clickhouse-tsan+0x8b3cea8)

  Mutex M183938897938677368 acquired here while holding mutex M2505 in main thread:
    0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a327b6)
    1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse-tsan+0x17cdb4f9)
    2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse-tsan+0x17cdb4f9)
    3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse-tsan+0x12be261b)
    4 DB::IBackgroundJobExecutor::finish() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:167:21 (clickhouse-tsan+0x12be261b)
    5 DB::IBackgroundJobExecutor::~IBackgroundJobExecutor() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:185:5 (clickhouse-tsan+0x12be261b)
    6 DB::StorageMergeTree::~StorageMergeTree() obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:174:1 (clickhouse-tsan+0x129ed768)
    7 std::__1::default_delete<DB::StorageMergeTree>::operator()(DB::StorageMergeTree*) const obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1397:5 (clickhouse-tsan+0x12e48b0b)
    8 std::__1::__shared_ptr_pointer<DB::StorageMergeTree*, std::__1::shared_ptr<DB::StorageMergeTree>::__shared_ptr_default_delete<DB::StorageMergeTree, DB::StorageMergeTree>, std::__1::allocator<DB::StorageMergeTree> >::__on_zero_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2565:5 (clickhouse-tsan+0x12e48b0b)
    9 std::__1::__shared_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2475:9 (clickhouse-tsan+0x125b53ea)
    10 std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2517:27 (clickhouse-tsan+0x125b53ea)
    11 std::__1::shared_ptr<DB::IStorage>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19 (clickhouse-tsan+0x125b53ea)
    12 DB::SystemLog<DB::AsynchronousMetricLogElement>::~SystemLog() obj-x86_64-linux-gnu/../src/Interpreters/SystemLog.h:118:7 (clickhouse-tsan+0x125b53ea)
    13 std::__1::allocator<DB::AsynchronousMetricLog>::destroy(DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:891:15 (clickhouse-tsan+0x125b1dd8)
    14 void std::__1::allocator_traits<std::__1::allocator<DB::AsynchronousMetricLog> >::__destroy<DB::AsynchronousMetricLog>(std::__1::integral_constant<bool, true>, std::__1::allocator<DB::AsynchronousMetricLog>&, DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:539:21 (clickhouse-tsan+0x125b1dd8)
    15 void std::__1::allocator_traits<std::__1::allocator<DB::AsynchronousMetricLog> >::destroy<DB::AsynchronousMetricLog>(std::__1::allocator<DB::AsynchronousMetricLog>&, DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:487:14 (clickhouse-tsan+0x125b1dd8)
    16 std::__1::__shared_ptr_emplace<DB::AsynchronousMetricLog, std::__1::allocator<DB::AsynchronousMetricLog> >::__on_zero_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2611:9 (clickhouse-tsan+0x125b1dd8)
    17 std::__1::__shared_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2475:9 (clickhouse-tsan+0x125904ff)
    18 std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2517:27 (clickhouse-tsan+0x125904ff)
    19 std::__1::shared_ptr<DB::AsynchronousMetricLog>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19 (clickhouse-tsan+0x125904ff)
    20 DB::SystemLogs::~SystemLogs() obj-x86_64-linux-gnu/../src/Interpreters/SystemLog.cpp:155:1 (clickhouse-tsan+0x125904ff)
    21 std::__1::__optional_destruct_base<DB::SystemLogs, false>::reset() obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:245:21 (clickhouse-tsan+0x11e43655)
    22 DB::ContextShared::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:443:21 (clickhouse-tsan+0x11e43655)
    23 DB::Context::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2251:13 (clickhouse-tsan+0x11e3be37)
    24 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5::operator()() const obj-x86_64-linux-gnu/../programs/server/Server.cpp:892:5 (clickhouse-tsan+0x8abacc2)
    25 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::invoke() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:97:9 (clickhouse-tsan+0x8abacc2)
    26 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::~basic_scope_guard() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:47:28 (clickhouse-tsan+0x8abacc2)
    27 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:1395:1 (clickhouse-tsan+0x8ab5cba)
    28 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b4ac7b)
    29 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa97be)
    30 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b67093)
    31 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8aa838e)
    32 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8aa68f9)

  Mutex M2505 previously acquired by the same thread here:
    0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a327b6)
    1 std::__1::__libcpp_recursive_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:385:10 (clickhouse-tsan+0x17cdb689)
    2 std::__1::recursive_mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:71:14 (clickhouse-tsan+0x17cdb689)
    3 std::__1::lock_guard<std::__1::recursive_mutex>::lock_guard(std::__1::recursive_mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse-tsan+0x11e4363f)
    4 DB::ContextShared::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:438:21 (clickhouse-tsan+0x11e4363f)
    5 DB::Context::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2251:13 (clickhouse-tsan+0x11e3be37)
    6 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5::operator()() const obj-x86_64-linux-gnu/../programs/server/Server.cpp:892:5 (clickhouse-tsan+0x8abacc2)
    7 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::invoke() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:97:9 (clickhouse-tsan+0x8abacc2)
    8 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::~basic_scope_guard() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:47:28 (clickhouse-tsan+0x8abacc2)
    9 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:1395:1 (clickhouse-tsan+0x8ab5cba)
    10 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b4ac7b)
    11 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa97be)
    12 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b67093)
    13 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8aa838e)
    14 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8aa68f9)

  [1]: https://clickhouse-test-reports.s3.yandex.net/21318/f3b1ad0f5d1024275674e1beac24251ae97c8453/functional_stateful_tests_(thread).html#fail1

v2: Convert ContextSharedPart::system_logs to std::unique_ptr (to avoid copy ctor)
v3: Fix readability-identifier-naming,-warnings-as-errors for system_logs_
v4: fix conflicts
 2021-04-13 21:37:38 +03:00
Azat Khuzhin
a61ae26729 Fix lock-order-inversion during system.*_log shutting down 
As TSan reports [1]:

WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=24429)
  Cycle in lock order graph: M152695175523663992 (0x000000000000) => M2505 (0x7b9000002008) => M152695175523663992

  Mutex M2505 acquired here while holding mutex M152695175523663992 in thread T7:
    0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a301b6)
    1 std::__1::__libcpp_recursive_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:385:10 (clickhouse-tsan+0x17cd6e89)
    2 std::__1::recursive_mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:71:14 (clickhouse-tsan+0x17cd6e89)
    3 std::__1::unique_lock<std::__1::recursive_mutex>::unique_lock(std::__1::recursive_mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:119:61 (clickhouse-tsan+0x11e32a9f)
    4 DB::Context::getLock() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:517:12 (clickhouse-tsan+0x11e32a9f)
    5 DB::Context::getSchedulePool() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1517:17 (clickhouse-tsan+0x11e32a9f)
    6 DB::IBackgroundJobExecutor::start() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:158:42 (clickhouse-tsan+0x12bde50a)
    7 DB::StorageMergeTree::startup() obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:112:29 (clickhouse-tsan+0x129e9e1e)
    8 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_2::operator()(std::__1::shared_ptr<DB::IStorage> const&) const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:230:16 (clickhouse-tsan+0x11d6fa2a)
    9 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3::operator()() const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:238:56 (clickhouse-tsan+0x11d6fa2a)
    10 decltype(std::__1::forward<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(fp)()) std::__1::__invoke<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x11d6fa2a)
    11 void std::__1::__invoke_void_return_wrapper<void>::__call<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x11d6fa2a)
    12 std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x11d6fa2a)
    13 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x11d6fa2a)
    14 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b39350)
    15 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b39350)
    16 ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b39350)
    17 void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3c070)
    18 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(fp0)...)) std::__1::__invoke_constexpr<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3682:1 (clickhouse-tsan+0x8b3c070)
    19 decltype(auto) std::__1::__apply_tuple_impl<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::__tuple_indices<std::__1::tuple<>&...>) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1415:1 (clickhouse-tsan+0x8b3c070)
    20 decltype(auto) std::__1::apply<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1424:1 (clickhouse-tsan+0x8b3c070)
    21 ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()::operator()() obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:178:13 (clickhouse-tsan+0x8b3c070)
    22 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3bfd1)
    23 void std::__1::__invoke_void_return_wrapper<void>::__call<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x8b3bfd1)
    24 std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x8b3bfd1)
    25 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x8b3bfd1)
    26 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b36c75)
    27 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b36c75)
    28 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b36c75)
    29 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3a918)
    30 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3a918)
    31 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>&, std::__1::__tuple_indices<>) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:280:5 (clickhouse-tsan+0x8b3a918)
    32 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:291:5 (clickhouse-tsan+0x8b3a918)

    Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message

  Mutex M152695175523663992 acquired here while holding mutex M2505 in main thread:
    0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a301b6)
    1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse-tsan+0x17cd6cf9)
    2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse-tsan+0x17cd6cf9)
    3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse-tsan+0x12bdee4b)
    4 DB::IBackgroundJobExecutor::finish() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:167:21 (clickhouse-tsan+0x12bdee4b)
    5 DB::IBackgroundJobExecutor::~IBackgroundJobExecutor() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:185:5 (clickhouse-tsan+0x12bdee4b)
    6 DB::StorageMergeTree::~StorageMergeTree() obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:174:1 (clickhouse-tsan+0x129ea118)
    7 std::__1::default_delete<DB::StorageMergeTree>::operator()(DB::StorageMergeTree*) const obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1397:5 (clickhouse-tsan+0x12e4433b)
    8 std::__1::__shared_ptr_pointer<DB::StorageMergeTree*, std::__1::shared_ptr<DB::StorageMergeTree>::__shared_ptr_default_delete<DB::StorageMergeTree, DB::StorageMergeTree>, std::__1::allocator<DB::StorageMergeTree> >::__on_zero_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2565:5 (clickhouse-tsan+0x12e4433b)
    9 std::__1::__shared_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2475:9 (clickhouse-tsan+0x125b355a)
    10 std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2517:27 (clickhouse-tsan+0x125b355a)
    11 std::__1::shared_ptr<DB::IStorage>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19 (clickhouse-tsan+0x125b355a)
    12 DB::SystemLog<DB::AsynchronousMetricLogElement>::~SystemLog() obj-x86_64-linux-gnu/../src/Interpreters/SystemLog.h:118:7 (clickhouse-tsan+0x125b355a)
    13 std::__1::allocator<DB::AsynchronousMetricLog>::destroy(DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:891:15 (clickhouse-tsan+0x125aff68)
    14 void std::__1::allocator_traits<std::__1::allocator<DB::AsynchronousMetricLog> >::__destroy<DB::AsynchronousMetricLog>(std::__1::integral_constant<bool, true>, std::__1::allocator<DB::AsynchronousMetricLog>&, DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:539:21 (clickhouse-tsan+0x125aff68)
    15 void std::__1::allocator_traits<std::__1::allocator<DB::AsynchronousMetricLog> >::destroy<DB::AsynchronousMetricLog>(std::__1::allocator<DB::AsynchronousMetricLog>&, DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:487:14 (clickhouse-tsan+0x125aff68)
    16 std::__1::__shared_ptr_emplace<DB::AsynchronousMetricLog, std::__1::allocator<DB::AsynchronousMetricLog> >::__on_zero_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2611:9 (clickhouse-tsan+0x125aff68)
    17 std::__1::__shared_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2475:9 (clickhouse-tsan+0x1258e74f)
    18 std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2517:27 (clickhouse-tsan+0x1258e74f)
    19 std::__1::shared_ptr<DB::AsynchronousMetricLog>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19 (clickhouse-tsan+0x1258e74f)
    20 DB::SystemLogs::~SystemLogs() obj-x86_64-linux-gnu/../src/Interpreters/SystemLog.cpp:155:1 (clickhouse-tsan+0x1258e74f)
    21 std::__1::__optional_destruct_base<DB::SystemLogs, false>::reset() obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:245:21 (clickhouse-tsan+0x11e41085)
    22 DB::ContextShared::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:443:21 (clickhouse-tsan+0x11e41085)
    23 DB::Context::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2251:13 (clickhouse-tsan+0x11e39867)
    24 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5::operator()() const obj-x86_64-linux-gnu/../programs/server/Server.cpp:892:5 (clickhouse-tsan+0x8ab8732)
    25 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::invoke() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:97:9 (clickhouse-tsan+0x8ab8732)
    26 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::~basic_scope_guard() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:47:28 (clickhouse-tsan+0x8ab8732)
    27 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:1395:1 (clickhouse-tsan+0x8ab3839)
    28 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b464ab)
    29 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa71be)
    30 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b628c3)
    31 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8aa5d8e)
    32 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8aa42f9)

  [1]: https://clickhouse-test-reports.s3.yandex.net/21318/38be9ff43ac4c46ce6e803fc125d910bde1d4c71/functional_stateful_tests_(thread).html#fail1
 2021-04-13 21:37:38 +03:00
Azat Khuzhin
3056d2c5d1 Fix data-race during server shutdown the context 
Found with 01737_clickhouse_server_wait_server_pool:

==================
WARNING: ThreadSanitizer: data race (pid=13248)
  Write of size 1 at 0x7b9000003a38 by main thread:
    0 std::__1::__optional_destruct_base<DB::SystemLogs, false>::reset() obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:246:24 (clickhouse-tsan+0x11e3043e)
    1 DB::ContextShared::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:441:21 (clickhouse-tsan+0x11e3043e)
    2 DB::Context::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2249:13 (clickhouse-tsan+0x11e28c17)
    3 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5::operator()() const obj-x86_64-linux-gnu/../programs/server/Server.cpp:892:5 (clickhouse-tsan+0x8ab1a32)
    4 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::invoke() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:97:9 (clickhouse-tsan+0x8ab1a32)
    5 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::~basic_scope_guard() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:47:28 (clickhouse-tsan+0x8ab1a32)
    6 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:1395:1 (clickhouse-tsan+0x8aacb39)
    7 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
    8 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
    9 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
    10 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
    11 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)

  Previous read of size 1 at 0x7b9000003a38 by thread T2 (mutexes: write M2504):
    0 std::__1::__optional_storage_base<DB::SystemLogs, false>::has_value() const obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:295:22 (clickhouse-tsan+0x11e25348)
    1 std::__1::optional<DB::SystemLogs>::operator bool() const obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:938:64 (clickhouse-tsan+0x11e25348)
    2 DB::Context::getQueryLog() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1875:10 (clickhouse-tsan+0x11e25348)
    3 DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:657:50 (clickhouse-tsan+0x12653751)
    4 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:904:30 (clickhouse-tsan+0x12651308)
    5 DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:289:24 (clickhouse-tsan+0x12f04b45)
    6 DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1500:9 (clickhouse-tsan+0x12f13907)
    7 Poco::Net::TCPServerConnection::start() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse-tsan+0x15b1f722)
    8 Poco::Net::TCPServerDispatcher::run() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:113:19 (clickhouse-tsan+0x15b1fe4e)
    9 Poco::PooledThread::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14 (clickhouse-tsan+0x15c86fe1)
    10 Poco::(anonymous namespace)::RunnableHolder::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:55:11 (clickhouse-tsan+0x15c8557f)
    11 Poco::ThreadImpl::runnableEntry(void*) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27 (clickhouse-tsan+0x15c83d87)

  Location is heap block of size 7296 at 0x7b9000002000 allocated by main thread:
    0 operator new(unsigned long) <null> (clickhouse-tsan+0x8a9aae7)
    1 std::__1::__unique_if<DB::ContextShared>::__unique_single std::__1::make_unique<DB::ContextShared>() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2068:28 (clickhouse-tsan+0x11e15c2c)
    2 DB::Context::createShared() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:503:32 (clickhouse-tsan+0x11e15c2c)
    3 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:426:27 (clickhouse-tsan+0x8aa19ee)
    4 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
    5 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
    6 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
    7 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
    8 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)

  Mutex M2504 (0x7b9000002008) created at:
    0 pthread_mutex_init <null> (clickhouse-tsan+0x8a0d37d)
    1 std::__1::__libcpp_recursive_mutex_init(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:370:10 (clickhouse-tsan+0x17cc4d93)
    2 std::__1::recursive_mutex::recursive_mutex() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:56:14 (clickhouse-tsan+0x17cc4d93)
    3 DB::ContextShared::ContextShared() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:394:5 (clickhouse-tsan+0x11e40bc3)
    4 std::__1::__unique_if<DB::ContextShared>::__unique_single std::__1::make_unique<DB::ContextShared>() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2068:32 (clickhouse-tsan+0x11e15c37)
    5 DB::Context::createShared() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:503:32 (clickhouse-tsan+0x11e15c37)
    6 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:426:27 (clickhouse-tsan+0x8aa19ee)
    7 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
    8 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
    9 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
    10 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
    11 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)

  Thread T2 'TCPHandler' (tid=13643, running) created by main thread at:
    0 pthread_create <null> (clickhouse-tsan+0x8a0bf0b)
    1 Poco::ThreadImpl::startImpl(Poco::SharedPtr<Poco::Runnable, Poco::ReferenceCounter, Poco::ReleasePolicy<Poco::Runnable> >) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:202:6 (clickhouse-tsan+0x15c83827)
    2 Poco::Thread::start(Poco::Runnable&) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:128:2 (clickhouse-tsan+0x15c84f6c)
    3 Poco::PooledThread::start() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:85:10 (clickhouse-tsan+0x15c873e2)
    4 Poco::ThreadPool::ThreadPool(int, int, int, int) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:252:12 (clickhouse-tsan+0x15c873e2)
    5 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:843:22 (clickhouse-tsan+0x8aa8e5f)
    6 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
    7 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
    8 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
    9 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
    10 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)

SUMMARY: ThreadSanitizer: data race obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:246:24 in std::__1::__optional_destruct_base<DB::SystemLogs, false>::reset()

v2: fix deadlock by calling SystemLogs::shutdown w/o Context lock
 2021-04-13 21:37:38 +03:00
Kruglov Pavel
6350f734dc
Merge branch 'master' into datasketches-uniq 2021-04-13 19:34:15 +03:00
hexiaoting
4f36ac7f64 Fix error 2021-04-13 17:31:49 +08:00
Nikolai Kochetov
d1d2b89a51 Use setting for explain json 2021-04-13 10:51:55 +03:00
Nikolai Kochetov
e731dfe650
Merge pull request #22991 from ClickHouse/untuple-and-subquery 
Fix subquery with untuple.
 2021-04-13 09:53:13 +03:00
hexiaoting
77c460e8d1 Merge remote-tracking branch 'origin/master' into dev-sumcount 2021-04-13 11:14:14 +08:00
hexiaoting
52bdec396b Merge remote-tracking branch 'origin/master' into dev_nested_upadte 2021-04-13 09:56:47 +08:00
alexey-milovidov
e2edecc8e2
Merge pull request #22434 from Avogar/catboost-reload 
Update paths to the catboost model configs in config reloading
 2021-04-13 01:21:04 +03:00
Nikolai Kochetov
0448e1415f Fix subquery with untuple. 2021-04-12 15:15:55 +03:00
Pavel Kruglov
e4f3678ae9 Fix build 2021-04-12 12:28:46 +03:00
Azat Khuzhin
79bd8d4d3f Respect optimize_skip_unused_shards_rewrite_in with optimize_skip_unused_shards_limit 2021-04-12 10:37:28 +03:00
Azat Khuzhin
fbb386dca5 Rewrite IN in query for remote shards to exclude values that does not belongs to shard 
v2: fix optimize_skip_unused_shards_rewrite_in for sharding_key wrapped into function
v3: fix column name for optimize_skip_unused_shards_rewrite_in
v4: fix optimize_skip_unused_shards_rewrite_in with Null
v5:
- squash with Remove query argument for IStreamFactory::createForShard()
- use proper column after function execution (using sharding_key_column_name)
- update the test reference since (X) now is tuple(X)
 2021-04-12 10:37:28 +03:00
Azat Khuzhin
773212529b Make Cluster::SlotToShards public 2021-04-12 10:32:04 +03:00
alexey-milovidov
d21ffd83a7
Merge branch 'master' into catboost-reload 2021-04-11 13:29:36 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
alexey-milovidov
5469cc15c1
Trigger CI 2021-04-10 23:43:56 +03:00
Nikolai Kochetov
48bcd5b490 Add tree conversion for expression. 2021-04-09 19:18:45 +03:00
Nikolai Kochetov
66b6d53a93 Add json=1 setting for explain actions 2021-04-09 17:44:58 +03:00
Alexander Kuzmenkov
0264124146
Merge pull request #21942 from ucasFL/distributed_depth 
Add settings max_distributed_depth
 2021-04-09 15:52:58 +03:00
Alexander Kuzmenkov
5db8e4c111
Update SystemLog.h 2021-04-09 15:48:09 +03:00
Maksim Kita
0cacca5b6e
Merge pull request #22833 from ClickHouse/trivial-fix 
Fix what looks like a trivial mistake
 2021-04-09 12:02:16 +03:00
Nikolai Kochetov
28ca191102
Merge pull request #22087 from ClickHouse/better-filter-push-down 
Better filter push down
 2021-04-09 10:22:17 +03:00
alexey-milovidov
b02a82250b
Merge pull request #22858 from vitlibar/fix-hanging-in-temporary-live-view-cleaner 
Fix hanging in TemporaryLiveViewCleaner
 2021-04-09 09:25:38 +03:00
Nikolai Kochetov
3426bc3906
Merge pull request #22763 from CurtizJ/fix-having-push-down 
Fix pushdown of having
 2021-04-08 21:53:50 +03:00
Nikolai Kochetov
3ea186358e Fix fuzz crash. 2021-04-08 18:00:32 +03:00
Vitaly Baranov
5bcb7a0fed Fix hanging in TemporaryLiveViewCleaner 2021-04-08 16:14:49 +03:00
Alexander Kuzmenkov
4a66548c5d
write distributed depth as VarUInt to save space, because it's small 2021-04-08 15:17:47 +03:00
Alexander Kuzmenkov
dc7a2b7f88
Merge pull request #22761 from ClickHouse/aku/flush-log 
add more messages when flushing the logs
 2021-04-08 13:50:17 +03:00
Nikolai Kochetov
118090396f Filter column after filter push down now always the first one.. 2021-04-08 12:54:39 +03:00
Alexey Milovidov
8fb6e6fbf2 Fix mistake 2021-04-08 11:51:22 +03:00
alesapin
1533f9b9aa
Merge pull request #14113 from johnskopis/dynamic-interserver-creds-v20 
Support interserver credential rotation
 2021-04-08 11:01:40 +03:00
alesapin
98871f8c5a More clear messages 2021-04-07 17:05:40 +03:00
alesapin
2a6560b1c6 Better interface 2021-04-07 16:59:18 +03:00
alesapin
2987bbc948 Small improvements 2021-04-07 16:52:11 +03:00
Alexander Kuzmenkov
fecd5f3435 more debug messages 2021-04-07 16:16:19 +03:00
Alexander Kuzmenkov
06bb58cb69
Merge pull request #22704 from ClickHouse/aku/window-multiple 
fix window functions with multiple input streams and no sorting
 2021-04-07 15:15:47 +03:00
Vladimir
9fe20c1628
Revert "Move conditions from JOIN ON to WHERE" 2021-04-07 14:57:20 +03:00
alesapin
feff1175f4 Merge branch 'master' into johnskopis/dynamic-interserver-creds-v20 2021-04-07 13:55:38 +03:00
Vladimir
864ab4a7ab
Merge pull request #21370 from vdimir/fix-merge-engine-join-group-by 2021-04-07 11:52:27 +03:00
hexiaoting
ee24d9a5cc Merge remote-tracking branch 'origin/master' into dev_nested_upadte 2021-04-07 10:42:51 +08:00
hexiaoting
8c60f95551 Fix error 2021-04-07 10:39:01 +08:00
feng lv
78afb5b23c fix 2021-04-07 01:58:57 +00:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT 
It executes the code in the dtor, that should never throw.
 2021-04-06 21:10:37 +03:00
Azat Khuzhin
b25a4e066c Lock MEMORY_LIMIT_EXCEEDED in ThreadStatus::detachQuery() 
Found with fuzzer [1]:

    <Fatal> BaseDaemon: (version 21.5.1.6440, build id: 3B097C902DDAA35688D90750552ED499DC5D10A0) (from thread 8012) Terminate called for uncaught exception:
    Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 153.51 MiB (attempt to allocate chunk of 4194368 bytes), maximum: 150.00 MiB, Stack trace (when copying this message, always include the lines below):

    0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x26fdcdd9 in /usr/bin/clickhouse
    1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:57: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xad0df02 in /usr/bin/clickhouse
    2. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:1444: DB::Exception::Exception<char const*, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*&&, char const*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0xacc7bef in /usr/bin/clickhouse
    3. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:219: MemoryTracker::alloc(long) @ 0xacc65eb in /usr/bin/clickhouse
    4. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
    5. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/Common/AllocatorWithMemoryTracking.h:35: AllocatorWithMemoryTracking<DB::Field>::allocate(unsigned long) @ 0xad0a2fe in /usr/bin/clickhouse
    7. void std::__1::vector<DB::Field, AllocatorWithMemoryTracking<DB::Field> >::__push_back_slow_path<DB::Field>(DB::Field&&) @ 0x11712a51 in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Interpreters/ThreadStatusExt.cpp:356: DB::ThreadStatus::detachQuery(bool, bool) @ 0x1f5d5237 in /usr/bin/clickhouse
    9. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:0: void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0x20c488e6 in /usr/bin/clickhouse
    10. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:0: ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xad9f6cc in /usr/bin/clickhouse
    11. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1655: void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) @ 0xada8264 in /usr/bin/clickhouse
    12. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    13. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
     (version 21.5.1.6440)

  [1]: https://clickhouse-test-reports.s3.yandex.net/22583/69296876005c0fa171c755f8b224e4d58192c402/stress_test_(address).html#fail1
 2021-04-06 21:10:36 +03:00
Alexander Kuzmenkov
d9ad06bd6d fix ubsan 2021-04-06 19:24:56 +03:00
alesapin
b4fc13de8e fix style 2021-04-06 17:02:56 +03:00
alesapin
4633afe962 Some fixes 2021-04-06 16:56:14 +03:00
alesapin
1c8e33414f Merge branch 'master' into johnskopis/dynamic-interserver-creds-v20 2021-04-06 16:28:46 +03:00
Alexander Kuzmenkov
585da2c8d0 Merge remote-tracking branch 'origin/master' into HEAD 2021-04-06 14:58:21 +03:00
Alexey Milovidov
f70c9e8752 Merge branch 'master' into protocol-compression-auto 2021-04-06 14:53:31 +03:00
Alexander Kuzmenkov
06e7a9f8f2 fix window functions with multiple input streams and no sorting 2021-04-06 14:43:27 +03:00
Nikolai Kochetov
6102652c99 Merge branch 'master' into better-filter-push-down 2021-04-06 13:38:03 +03:00
vdimir
735154c81a
Handle not plain where tree in StorageMerge modifySelect 2021-04-06 12:29:29 +03:00
alesapin
86a843bb51
Merge pull request #21509 from kssenii/library-bridge 
clickhouse-library-bridge for library dictionary source
 2021-04-06 12:26:08 +03:00
Amos Bird
46c9affa83
Fix CTE in view. 2021-04-06 11:32:16 +08:00
Anton Popov
c5354885a6
Merge pull request #22654 from azat/log-nested-fix 
[RFC] Fix LOGICAL_ERROR for Log with nested types w/o columns in the SELECT clause
 2021-04-06 00:34:58 +03:00
Maksim Kita
de71553c9d
Merge pull request #22593 from zlobober/introduce_distributed_write 
Introduce IStorage::distributedWrite method for distributed INSERT SELECTS
 2021-04-05 22:59:57 +03:00
feng lv
f6e4e13f89 fix conflict 2021-04-05 14:08:30 +00:00
feng lv
56073db22d max distributed depth 
Add settings max_distributed_depth

fix style

fix

fix

fix

fix test

fix

fix
 2021-04-05 14:00:54 +00:00
kssenii
9dba529494 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-04-05 13:14:51 +00:00
pingyu
e05ee09621 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into datasketches-uniq 2021-04-05 16:47:53 +08:00
Azat Khuzhin
42d858b7e5 Fix LOGICAL_ERROR for Log with nested types w/o columns in the SELECT clause 
Found with fuzzer [1]:

<details>

    2021.04.04 10:25:46.762596 [ 135 ] {4b6b5de3-d242-4267-910a-76b235467356} <Fatal> : Logical error: 'Logical error: no information about file column%2Ename.size0 in StorageLog'.
    ...
    2021.04.04 10:25:46.763563 [ 44 ] {} <Trace> BaseDaemon: Received signal 6
    2021.04.04 10:25:46.765884 [ 165 ] {} <Fatal> BaseDaemon: 5. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.04.04 10:25:46.901540 [ 165 ] {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:51: DB::handle_error_code() @ 0x11cc4f16 in /workspace/clickhouse
    2021.04.04 10:25:47.030755 [ 165 ] {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:58: DB::Exception::Exception() @ 0x11cc5025 in /workspace/clickhouse
    2021.04.04 10:25:47.617201 [ 165 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:183: DB::LogSource::readData()

</details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/22583/69296876005c0fa171c755f8b224e4d58192c402/fuzzer_debug/report.html#fail1

The problem here is that there is no column%2Ename.size0 there is
column.size0 instead, and it seems that reading sizeX file is not enough
in other places, so just filter them out in
ExpressionActions::getSmallestColumn() (fallback method for storages w/o
getColumnSizes() implemented)
 2021-04-05 08:48:04 +03:00
Maxim Akhmedov
725fa17961 Introduce IStorage::distributedWrite method for distributed INSERT SELECT. 2021-04-05 02:14:27 +03:00
alexey-milovidov
b1e7624b3f
Merge pull request #22589 from ClickHouse/fix-comments 
Fix comments
 2021-04-05 00:22:27 +03:00
Anton Popov
0d03b8c0c4
Merge pull request #21889 from CurtizJ/fix-aggregation-in-order-1 
Fix bugs in aggregation by primary key
 2021-04-04 18:17:04 +03:00
pingyu
1ea74a1947 bug fix. add unit tests.# 2021-04-04 22:22:47 +08:00
Alexey Milovidov
54caa61af5 Fix comments 2021-04-04 12:33:06 +03:00
alexey-milovidov
30ece2c28f
Merge pull request #22510 from vdimir/merge-join-lc-bug-fix 
Convert right block to full in MergeJoin
 2021-04-03 12:54:34 +03:00
alexey-milovidov
840ecb5e99
Update MergeJoin.cpp 2021-04-02 21:07:11 +03:00
Alexander Kuzmenkov
d0037329cf
Merge pull request #22459 from ClickHouse/aku/window-frame-offset 
fix window frame offset check and add more tests
 2021-04-02 19:49:59 +03:00
vdimir
c8f51998ba
Remove LowCardinality from right block sample in MergeJoin 2021-04-02 17:51:02 +03:00
vdimir
8cd841c4ac
Handle lowcardinality block in MergeJoin extractMinMax 2021-04-02 16:55:42 +03:00
Anton Popov
a000d6154c fix clang-tidy 2021-04-02 15:10:49 +03:00
hexiaoting
6e8583a028 Add array size check for Nested Type subcolumns when Update 2021-04-02 19:31:33 +08:00
Pavel Kruglov
dff7834af0 Fix config comparison 2021-04-02 13:17:28 +03:00
Vladimir
9b40f384e9
Merge pull request #20622 from vdimir/issue-20497 2021-04-02 11:48:29 +03:00
Anton Popov
56320bcbfd minor refactoring 2021-04-02 01:16:54 +03:00
Anton Popov
88cd775f6a Merge remote-tracking branch 'upstream/master' into HEAD 2021-04-02 00:14:03 +03:00
Alexander Kuzmenkov
d006c5532e fix window frame offset check and add more tests 2021-04-01 18:56:58 +03:00
vdimir
3f464595eb
Accurate removing 'join' part for queries to 'Merge' engine 2021-04-01 14:31:57 +03:00
vdimir
fff8043e55
Revert "Fix joined table access with Merge engine and aggregation" 
This reverts commit 5476e68d6c.
 2021-04-01 14:31:46 +03:00
vdimir
5476e68d6c
Fix joined table access with Merge engine and aggregation 2021-04-01 14:26:28 +03:00
Pavel Kruglov
cc65a1db97 Update paths to the model configs in config reloading 2021-04-01 13:30:05 +03:00
pingyu
86d8bc1c54 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into datasketches-uniq 2021-04-01 12:35:24 +08:00
Alexey Milovidov
f441fd3015 Merge branch 'master' into protocol-compression-auto 2021-04-01 07:18:08 +03:00