Maksim Kita
8d6c282e0a
Fixed test
2021-05-08 17:39:34 +03:00
Maksim Kita
af06f5b68b
Fixed UUID test
2021-05-08 17:39:34 +03:00
Maksim Kita
97d87eb518
Fixed JITMemoryManager
2021-05-08 17:39:34 +03:00
Maksim Kita
e40aa26611
Fixed MSan
2021-05-08 17:39:34 +03:00
Maksim Kita
baf1978124
CHJIT updated thread safe interface
2021-05-08 17:39:34 +03:00
Maksim Kita
2b58357a77
Fixed tests
2021-05-08 17:39:34 +03:00
Maksim Kita
69fdcdcc4a
Fixed tests
2021-05-08 17:39:34 +03:00
Maksim Kita
231433d951
Fixed style check
2021-05-08 17:39:34 +03:00
Maksim Kita
ca9a23abbc
Fix build
2021-05-08 17:39:34 +03:00
Maksim Kita
e58e956092
Enable llvm in fast test
2021-05-08 17:39:34 +03:00
Maksim Kita
3ec4409d52
Added CHJIT
2021-05-08 17:39:34 +03:00
Maksim Kita
ca44ff2ede
Updated example
2021-05-08 17:39:34 +03:00
Maksim Kita
c264e42542
Updated example
2021-05-08 17:39:34 +03:00
Maksim Kita
9f88fea760
Updated test
2021-05-08 17:39:34 +03:00
Maksim Kita
6b70cbd53c
Added jit example
2021-05-08 17:39:34 +03:00
Alexey Milovidov
78867a53c6
Fixed build
2021-05-08 17:39:34 +03:00
Alexey Milovidov
996cdd327c
Addition to prev. revision
2021-05-08 17:39:34 +03:00
Alexey Milovidov
e862580898
Added comment
2021-05-08 17:39:34 +03:00
Alexey Milovidov
f7830eb7b3
Unpoison data generated by JIT-ed functions for MSan
2021-05-08 17:39:34 +03:00
Alexey Milovidov
aa727e941d
Fix bad code (found by PVS-Studio)
2021-05-08 17:21:36 +03:00
alexey-milovidov
680860aae7
Merge pull request #23912 from azat/CU
...
Cleanup IDatabase.h from extra headers
2021-05-08 16:42:56 +03:00
Alexey Milovidov
47a4c101b9
Merge branch 'master' into normalize-bigint
2021-05-08 16:41:15 +03:00
hexiaoting
cca53e59f5
Fix bug
2021-05-07 17:17:43 +08:00
hexiaoting
a1d521902c
Merge remote-tracking branch 'origin/master' into dev-continue-sundyli
2021-05-07 14:22:53 +08:00
Alexey Milovidov
c1bb5bfc2b
BloomFilterHash for UUID
2021-05-07 03:23:32 +03:00
kreuzerkrieg
e86d8950c2
Drop unnecessary ports where applicable
2021-05-06 20:30:24 +03:00
fibersel
cb53bbb7b0
add experimental codecs flag, add integration test for experimental codecs
2021-05-06 14:57:22 +03:00
vdimir
1e4fda08e2
Allow JOIN ON with Join engine
2021-05-06 09:08:41 +03:00
Azat Khuzhin
c757b532c6
Make DatabaseCatalog.cpp self compilable (missing AbstractConfiguration)
2021-05-06 09:07:58 +03:00
vdimir
abdb731ce2
Merge branch 'master' into issue-20309
2021-05-06 09:07:51 +03:00
Alexey Milovidov
2c9ae14cb0
Add 256 bit integers in some places
2021-05-06 03:31:09 +03:00
Azat Khuzhin
eefd67fce5
Disable optimize_distributed_group_by_sharding_key with window functions
2021-05-06 00:44:22 +03:00
Azat Khuzhin
8703b7863d
Fix distributed_group_by_no_merge=2 with GROUP BY
...
Before this patch the following query:
SELECT assumeNotNull(argMax(dummy, 1))
FROM remote('127.1', system.one)
SETTINGS distributed_group_by_no_merge = 2
Leads to:
Code: 10. DB::Exception: Received from localhost:9000. DB::Exception: Not found column argMax(dummy, 1) in block: while executing 'INPUT : 0 -> argMax(dummy, 1) UInt8 : 0'.
Since it tries to execute function one more time, but shards will not
send this column when the query processed with
distributed_group_by_no_merge=2 (i.e. up to
WithMergeableStateAfterAggregation).
v0: no exception
v2: execut window functions
v3: throw exception, since executing window function in this case will
lead to messy output
2021-05-06 00:21:47 +03:00
kssenii
2dabdd0f73
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-05-05 18:42:40 +03:00
kssenii
5ef3ee50fe
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into progress-bar
2021-05-05 09:26:12 +00:00
pingyu
46f809d07f
Revert "Merge pull request #23334 from ClickHouse/revert-22609-datasketches-uniq"
...
This reverts commit af2499359bdb82e9e3d5
2021-05-05 16:42:57 +08:00
Alexey Milovidov
54fb40c041
Fix OpenTelemetry
2021-05-05 01:42:14 +03:00
Alexey Milovidov
eea47a65d2
PVS-Studio
2021-05-04 15:13:47 +03:00
Alexey Milovidov
86169cc36e
Merge branch 'master' into normalize-bigint
2021-05-04 14:53:42 +03:00
Alexey Milovidov
aa9a367d45
Fix issue #7834
2021-05-04 02:27:05 +03:00
Alexey Milovidov
02ce9cc725
Untangle UUID
2021-05-04 01:46:51 +03:00
Alexey Milovidov
8ecd9d36e6
Add UInt128 conversion functions
2021-05-03 18:41:37 +03:00
Alexey Milovidov
9f74565948
PVS-Studio
2021-05-03 01:42:01 +03:00
Alexey Milovidov
1fda866881
Merge branch 'master' into normalize-bigint
2021-05-02 21:24:45 +03:00
Nikolai Kochetov
08f10dced0
Merge pull request #23743 from ClickHouse/refactor-join-step
...
Refactor join step
2021-05-02 21:17:23 +03:00
feng lv
39f68bf5ff
fix conflict
2021-05-02 16:33:45 +00:00
kssenii
ee06936596
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-05-01 17:24:31 +03:00
alexey-milovidov
f360e7c808
Merge pull request #23766 from kitaisreal/remove-unused-compilers
...
Remove unused compilers (fixed for the troublesome "unbundled" build)
2021-05-01 16:15:38 +03:00
Vladimir
3b7c68196b
Merge pull request #23429 from Jokser/disk-s3-restart-possibility
2021-05-01 15:28:14 +03:00
Alexey Milovidov
b879d05ec4
Fix build
2021-05-01 10:06:30 +03:00
Nikolai Kochetov
33fd7206c4
Revert "Try to remove useless code"
...
This reverts commit 10e3cbe63a
2021-04-30 16:07:22 +03:00
Nikolai Kochetov
10e3cbe63a
Try to remove useless code
2021-04-30 13:50:24 +03:00
Nikolai Kochetov
3d5b61d63f
Fix fuild.
2021-04-30 13:18:01 +03:00
Maksim Kita
318c4bb80d
Add examples folder filter to ya.make.in
2021-04-30 11:25:52 +03:00
Nikolai Kochetov
6b79403cb5
Rename and fix fuild.
2021-04-30 11:25:39 +03:00
Maksim Kita
8c84f3fc8b
Merge pull request #23771 from kitaisreal/poco-timespan-reference-fix
...
Poco::Timespan reference fix
2021-04-30 09:49:01 +03:00
Maksim Kita
d0c0054ff0
Merge pull request #23732 from kitaisreal/external-loader-repository-fix-arcadia
...
ExternalLoaderRepository fix arcadia
2021-04-30 01:26:04 +03:00
Pavel Kovalenko
0050dbb75b
Apply disk settings on config reload.
2021-04-29 23:32:19 +03:00
Nikolai Kochetov
2a9ffcf6d5
Fix build.
2021-04-29 20:57:18 +03:00
Maksim Kita
0d0a14a925
Poco::Timespan reference fix
2021-04-29 19:11:20 +03:00
Nikolai Kochetov
f11341af3d
Fix build.
2021-04-29 17:58:58 +03:00
Nikolai Kochetov
bc2375044b
Remove some code
2021-04-29 17:34:52 +03:00
vdimir
8197e1c0c1
Add qualified names for right columns for storage join
2021-04-29 17:30:02 +03:00
Nikolai Kochetov
b500c420b9
Fix build and style.
2021-04-29 17:14:23 +03:00
Vladimir
1a8407e029
Merge pull request #23237 from vdimir/fix-join-lowcard-nullable
2021-04-29 13:18:04 +03:00
Nikolai Kochetov
1f5a0f8842
Merge branch 'master' into refactor-join-step
2021-04-29 12:23:36 +03:00
Nikolai Kochetov
25fe5f63ee
Fix other tests.
2021-04-29 12:08:49 +03:00
kssenii
3386dc32d3
Change in Functions/*
2021-04-29 00:38:58 +03:00
kssenii
c33f358a9a
Better way to track progress, display progress for ch-client
2021-04-28 19:12:28 +00:00
kssenii
deb4903af8
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-04-28 20:57:13 +03:00
kssenii
5e429b1266
Change in Interpreters/*
2021-04-28 20:52:00 +03:00
Nikolai Kochetov
69ddca1d73
Refactor join step, part 3
2021-04-28 20:32:12 +03:00
vdimir
fb88547a04
Call nestedToNullable without copying column in convertColumnToNullable for join
2021-04-28 20:18:03 +03:00
vdimir
6e4b37a45e
Merge branch 'master' into fix-join-lowcard-nullable
2021-04-28 20:06:49 +03:00
Nikolai Kochetov
c508868570
Merge pull request #23082 from ClickHouse/explain-output
...
JSON output for EXPLAIN
2021-04-28 15:39:06 +03:00
Maksim Kita
c7ff95fbd9
ExternalLoaderRepository fix arcadia
2021-04-28 14:40:35 +03:00
Maksim Kita
b544037f03
Merge pull request #23436 from kitaisreal/storage-dictionary-updated
...
Refactored StorageDictionary
2021-04-28 13:11:48 +03:00
alexey-milovidov
fdc5a44202
Merge pull request #23644 from nikitamikhaylov/delete-unit-tests
...
Move non gtest unit tests to /examples folder
2021-04-28 07:28:18 +03:00
Maksim Kita
3747a0c8e5
Fixed TRUNCATE TABLE for Dictionary storage
2021-04-27 22:13:21 +03:00
Maksim Kita
2e5f97c6ff
Fixed code review issues
2021-04-27 22:09:03 +03:00
Nikolai Kochetov
a51a380645
Refactor join step, part 1
2021-04-27 17:00:16 +03:00
kssenii
3a0372535e
Show progress only with --progress option
2021-04-27 13:21:42 +00:00
Nikolai Kochetov
5570b56cc3
A little bit faster merge of aggregating states.
2021-04-27 12:01:58 +03:00
Amos Bird
8a3b5c1fab
Add _partition_value virtual column
2021-04-27 16:15:59 +08:00
Maksim Kita
8b5a049ca0
Merge pull request #23635 from kitaisreal/aggregate-functions-remove-unused-code
...
Aggregator remove unused code
2021-04-27 00:52:34 +03:00
kssenii
ffcfe5f9a1
Progress bar for file table engine
2021-04-26 21:02:52 +00:00
Nikita Mikhaylov
223c6b4ac5
better
2021-04-26 21:26:55 +03:00
Maksim Kita
c93666e0de
Fixed code review issues
2021-04-26 13:52:41 +03:00
Maksim Kita
561b045991
Fixed tests
2021-04-26 13:52:41 +03:00
Maksim Kita
64f49a981b
Fixed tests
2021-04-26 13:52:41 +03:00
Maksim Kita
2e3e90b25b
Fixed tests
2021-04-26 13:52:41 +03:00
Maksim Kita
6ba83eccd3
Added support for DETACH DICTIONARY PERMANENTLY
2021-04-26 13:52:41 +03:00
Maksim Kita
3cb4d1bdcb
Fixed style check
2021-04-26 13:52:41 +03:00
Maksim Kita
a9917a6540
Refactored StorageDictionary
2021-04-26 13:52:41 +03:00
Maksim Kita
5bb263f535
Fixed tests
2021-04-26 13:44:59 +03:00
Alexander Kuzmenkov
fed35445d9
Merge pull request #23546 from ClickHouse/aku/window-distributed
...
fix window functions for Distributed tables
2021-04-26 12:34:46 +03:00
Nikolai Kochetov
092ff0faeb
Merge pull request #23524 from kssenii/insert-union-select
...
Allow to insert default instead of Null in insert-select / insert-select-union-all
2021-04-26 12:25:49 +03:00
Maksim Kita
cac4a85286
Aggregator remove unused code
2021-04-26 01:17:24 +03:00
Alexey Milovidov
649550c5ab
Attempt to normalize big integers
2021-04-25 12:30:43 +03:00
Alexey Milovidov
60be962dca
Merging with master
2021-04-25 07:08:57 +03:00
Alexey Milovidov
8f01af62d9
Merge branch 'master' into normalize-bigint
2021-04-25 06:57:44 +03:00
alexey-milovidov
bcede10919
Merge pull request #23312 from vdimir/issue-23002
...
Correct sample_block in HashJoin for dict join with join_algorithm
2021-04-24 04:40:43 +03:00
alexey-milovidov
ca230224cf
Merge pull request #23549 from vdimir/issue-17718
...
Remove clashed columns from totals for StorageJoin
2021-04-24 03:19:00 +03:00
kssenii
3a9b11e141
Add check for columns sizes match
2021-04-23 19:09:09 +00:00
kssenii
d032651aba
Try fix
2021-04-23 17:29:22 +00:00
Alexander Kuzmenkov
ab75f86ee9
add a comment
2021-04-23 18:45:45 +03:00
Ivan Lezhankin
cdd825b18b
Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2
2021-04-23 18:08:25 +03:00
Alexander Kuzmenkov
6982ab5324
fix extra distinct
2021-04-23 15:44:46 +03:00
kssenii
7c3b074891
Better
2021-04-23 12:40:34 +00:00
vdimir
9460989721
Remove clashed columns from totals for StorageJoin
2021-04-23 15:37:05 +03:00
kssenii
8f34199538
Fixes
2021-04-23 12:36:40 +00:00
feng lv
0f5c05854c
fix conflict
2021-04-23 12:22:51 +00:00
feng lv
4ffe199d39
Implement table comments
2021-04-23 12:18:23 +00:00
Alexander Kuzmenkov
9d95ac3d66
remove debug
2021-04-23 14:00:17 +03:00
Alexander Kuzmenkov
54ad086753
fix for distributed
2021-04-23 13:59:21 +03:00
kssenii
54a7a6ef0d
Allow null to default convertion only if column has default
2021-04-23 09:55:57 +00:00
Alexander Kuzmenkov
7ea5bd4a3f
Merge remote-tracking branch 'origin/master' into HEAD
2021-04-23 09:26:43 +03:00
kssenii
459612c895
Use a separate setting
2021-04-22 22:14:14 +00:00
kssenii
3176c24a8a
Add test
2021-04-22 21:46:49 +00:00
kssenii
28c4436246
Better
2021-04-22 21:30:31 +00:00
Alexander Kuzmenkov
06ca44a378
fix pvs warnings
2021-04-22 22:54:58 +03:00
kssenii
ac55183d2e
Null as default for insert select / with union
2021-04-22 18:11:36 +00:00
Maksim Kita
7357c35197
Fixed code review issues
2021-04-22 17:19:47 +03:00
Maksim Kita
8fde34fc52
Updated tests
2021-04-22 16:23:42 +03:00
Maksim Kita
86ffa9fcfa
JoinToSubqueriesVisitor ASTColumnsMatcher fix
2021-04-22 13:14:35 +03:00
mergify[bot]
71bca99bbf
Merge branch 'master' into explain-output
2021-04-22 07:02:47 +00:00
Nikita Mikhaylov
d4e8f9676b
Merge pull request #21304 from amosbird/indexhint
...
Resurrect indexHint function.
2021-04-21 16:14:43 +03:00
alexey-milovidov
9c649932bc
Merge pull request #23375 from ClickHouse/fix-read-in-order-with-collation
...
Fix read in order optimization of COLLATE is present #22379
2021-04-21 15:00:02 +03:00
Maksim Kita
02515ff0a4
Merge pull request #23191 from kitaisreal/use-local-shard-if-no-remote-connections
...
Updated QueryAliasVisitor to prefer alias for aliases like _subquery
2021-04-21 00:58:47 +03:00
Pavel Kovalenko
10cf881a08
Add SYSTEM RESTART DISK command.
2021-04-20 22:23:54 +03:00
Pavel Kovalenko
98f8cbbdb2
DiskRestartProxy refactoring.
2021-04-20 21:58:34 +03:00
Alexey Milovidov
9bc0706d2f
Fix read in order optimization of COLLATE is present #22379
2021-04-20 21:20:45 +03:00
tavplubix
738d8a757b
Merge pull request #23189 from ClickHouse/minor_fixes_in_attach_query
...
Minor fixes in ATTACH query
2021-04-20 20:04:26 +03:00
alexey-milovidov
632a5288fa
Merge pull request #23348 from azat/zero-streams
...
Fix LOGICAL_ERROR for max_distributed_connections=0
2021-04-20 19:48:21 +03:00
Nikolai Kochetov
6b264933b2
Fix style.
2021-04-20 17:51:48 +03:00
Nikolai Kochetov
942cd54f23
Fix copy paste thanks to PVS studio.
2021-04-20 16:41:37 +03:00
Anton Popov
8f57141fad
Merge pull request #23195 from CurtizJ/nested-update
...
Merging #22503
2021-04-20 14:57:47 +03:00
Amos Bird
32c84f77c3
Resurrect indexHint function.
2021-04-20 19:27:23 +08:00
Alexander Tokmakov
4108a235f5
make separate functions
2021-04-20 14:22:02 +03:00
Maksim Kita
457b75a196
Updated tests
2021-04-20 13:31:01 +03:00
Maksim Kita
cbc34afd55
Updated QueryAliasVisitor to set prefer_alias_to_column_name for aliases like _subquery
2021-04-20 13:16:29 +03:00
vdimir
3266bbb54c
TableJoin: forceHashJoin for dictionary_reader, add tests
2021-04-20 12:52:52 +03:00
Nikolai Kochetov
0c4e789900
Remove debug output
2021-04-20 10:08:35 +03:00
Azat Khuzhin
86069ffb40
Fix LOGICAL_ERROR for max_distributed_connections=0
2021-04-20 09:05:44 +03:00
Nikolai Kochetov
17f7631878
Fix build.
2021-04-20 08:55:20 +03:00
Anton Popov
0a0c510aa2
minor fixes
2021-04-20 05:51:07 +03:00
alexey-milovidov
62899436db
Revert "add uniqThetaSketch"
2021-04-20 03:34:21 +03:00
alexey-milovidov
5b66086bc9
Merge pull request #23219 from azat/optimize_skip_unused_shards_rewrite_in-fix
...
Add type conversion for optimize_skip_unused_shards_rewrite_in
2021-04-19 22:45:35 +03:00
vdimir
a95c4dcfff
Correct sample_block in HashJoin for dict join with join_algorithm = auto
2021-04-19 18:02:56 +03:00
Nikolai Kochetov
8c2d58333a
Merge branch 'master' into explain-output
2021-04-19 17:40:27 +03:00
Nikolai Kochetov
c59628846c
Merge pull request #22352 from ClickHouse/add-read-from-mt-step
...
Special query plan step for read from MergeTree
2021-04-19 17:38:40 +03:00
Alexander Kuzmenkov
2a4bcb6e3f
Merge pull request #23159 from ClickHouse/aku/merge-fusecount
...
merging sumCount fusion PR #21337
2021-04-19 16:47:13 +03:00
Kruglov Pavel
995973bf1f
Merge pull request #22609 from pingyu/datasketches-uniq
...
add uniqThetaSketch
2021-04-19 10:32:29 +03:00
Nikolai Kochetov
4d86f51eff
Merge branch 'master' into add-read-from-mt-step
2021-04-19 10:17:21 +03:00
Maksim Kita
ca93987ae9
Merge pull request #23182 from kitaisreal/system-query-reload-model
...
Added system query reload model
2021-04-19 00:39:31 +03:00
Azat Khuzhin
d476a4a9bd
Add type conversion for optimize_skip_unused_shards_rewrite_in
...
MSan reports [1]:
2021.04.17 15:20:06.665152 [ 56 ] {2336bf92-0269-4acd-8b3f-f09623223d18} <Debug> executeQuery: (from [::1]:44744, using production parser) SELECT * FROM dist_01757 WHERE dummy IN ('255', 0) FORMAT Null
...
0 0x305af885 in (anonymous namespace)::shardContains() obj-x86_64-linux-gnu/../src/Interpreters/OptimizeShardingKeyRewriteInVisitor.cpp:50:28
...
Uninitialized value was created by an allocation of 'sharding_value' in the stack frame of function '_ZN12_GLOBAL__N_113shardContainsERKN2DB5FieldERKNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEERKNS4_10shared_ptrINS0_17ExpressionActionsEEESC_RKNS0_7Cluster9ShardInfoERKNS4_6vectorImNS8_ImEEEE'
0 0x305ae260 in (anonymous namespace)::shardContains() obj-x86_64-linux-gnu/../src/Interpreters/OptimizeShardingKeyRewriteInVisitor.cpp:42
[1]: https://clickhouse-test-reports.s3.yandex.net/23212/d02eb3f6b7a8dc07bbed5605e1afcf7634436b3a/fuzzer_msan/report.html#fail1
P.S. fuzzers are great!
2021-04-19 00:03:38 +03:00
Azat Khuzhin
3f156b83c9
Add some comments in OptimizeShardingKeyRewriteInVisitor
2021-04-18 09:07:51 +03:00
alexey-milovidov
7f3a40e1b0
Merge pull request #22237 from ClickHouse/protocol-compression-auto
...
Autodetect compression #22234
2021-04-18 03:13:54 +03:00
Alexander Tokmakov
0354312294
minor fixes in attach query
2021-04-17 22:43:02 +03:00
vdimir
826296c799
Convert columns to LC(Nullable(T)) in join
2021-04-17 22:03:32 +03:00
Maksim Kita
abf2a87ce7
Added system query reload model
2021-04-17 20:59:08 +03:00
Alexey Milovidov
77e64b3ebd
Merge branch 'master' into protocol-compression-auto
2021-04-17 16:46:51 +03:00
alexey-milovidov
138c6ae228
Merge pull request #23185 from ClickHouse/fix_22573
...
Fix data_type_default_nullable on ATTACH
2021-04-17 16:13:42 +03:00
Alexey Milovidov
ec473a90d2
Fixed missing semicolon in exception message
2021-04-17 02:14:48 +03:00
alexey-milovidov
74a0ac6917
Merge pull request #23160 from azat/logging-v2
...
[RFC] Change logging from trace to debug for messages with rows/bytes
2021-04-16 21:18:32 +03:00
Nikolai Kochetov
1bf420cbcf
Update json description for other steps.
2021-04-16 19:36:59 +03:00
Maksim Kita
d88e0b1b44
Use local shard if there are no remote connections
2021-04-16 16:01:53 +03:00
Nikolai Kochetov
8f969ec063
Revert "Ignore output format for explain query on client."
...
This reverts commit 3f7cee1149
2021-04-16 15:44:27 +03:00
Nikolai Kochetov
a5f8b0f269
Revert "Omit EXPLAIN FORMAT for old TCP clients."
...
This reverts commit 4c37d461de
2021-04-16 15:44:18 +03:00
Anton Popov
b14798af2a
Merge branch 'dev_nested_upadte' of git://github.com/hexiaoting/ClickHouse into nested-update
2021-04-16 14:41:19 +03:00
Nikolai Kochetov
23089a2fec
Add special setting to show indexes.
2021-04-16 14:11:45 +03:00
Alexander Kuzmenkov
5e0fb440a5
Update TreeRewriter.cpp
2021-04-16 13:35:02 +03:00
Alexander Tokmakov
ebf0a3119e
fix data_type_default_nullable on attach
2021-04-16 13:29:33 +03:00
tavplubix
ae3dc3dfc0
Merge pull request #23161 from ClickHouse/logs_for_debuging_test_failures
...
More verbose logs for debuging test failures with Replicated and Keeper
2021-04-16 12:01:58 +03:00
Alexander Tokmakov
0ad6205fa6
logs for debuging test failures with Replicated and Keeper
2021-04-15 21:34:53 +03:00
Azat Khuzhin
d2cf03ea41
Change logging from trace to debug for messages with rows/bytes
2021-04-15 21:00:16 +03:00
Alexander Kuzmenkov
2489b6af96
cleanup
2021-04-15 19:40:49 +03:00
Alexander Kuzmenkov
3b95b637a5
Merge remote-tracking branch 'origin/master' into HEAD
2021-04-15 18:19:53 +03:00
alexey-milovidov
69eac517b6
Merge pull request #23086 from azat/memory-blocking
...
Block all memory tracking limits in dtors/SCOPE_EXIT_*SAFE/tryLogCurrentException
2021-04-15 14:36:10 +03:00
Vladimir
8c7ffda676
Merge pull request #22753 from ClickHouse/revert-19685-dev_joinon
2021-04-15 12:42:50 +03:00
Ivan Lezhankin
06bc435b06
Fix code with params
2021-04-15 10:33:15 +03:00
Ivan
2cec6a2002
Merge branch 'master' into ast-table-identifier-2
2021-04-15 09:30:24 +03:00
alexey-milovidov
9b546f3b89
Merge pull request #22657 from amosbird/ctefix3
...
Fix CTE usage in view definition
2021-04-15 02:01:10 +03:00
Alexey Milovidov
17d04cbca9
Merge branch 'master' into protocol-compression-auto
2021-04-15 00:24:07 +03:00
tavplubix
2479c80fb7
Merge branch 'master' into fix_attach_mv
2021-04-15 00:07:48 +03:00
Azat Khuzhin
9b9e0a9bbc
Block all memory tracking limits in destructors
2021-04-14 23:38:42 +03:00
Nikolai Kochetov
4c37d461de
Omit EXPLAIN FORMAT for old TCP clients.
2021-04-14 18:59:08 +03:00
Ivan
b099ad467d
Merge branch 'master' into ast-table-identifier-2
2021-04-14 18:38:30 +03:00
Ivan Lezhankin
00e8571088
Fix for alter update and IN operator
2021-04-14 18:35:52 +03:00
Nikolai Kochetov
3f7cee1149
Ignore output format for explain query on client.
2021-04-14 18:33:07 +03:00
Nikolai Kochetov
9b86c19836
Use JSONBuilder for explain json result.
2021-04-14 17:07:56 +03:00
alexey-milovidov
f8ce5547aa
Merge pull request #21318 from azat/servers-pool-wait
...
Fix SIGSEGV by waiting servers thread pool
2021-04-14 14:15:07 +03:00
hexiaoting
61e2ffc0dd
Fix build error
2021-04-14 16:53:06 +08:00
Alexey Milovidov
6f56c3280f
Uncompress data in Distributed sends if needed
2021-04-14 00:53:39 +03:00
Alexey Milovidov
e08c8a3d2c
Merge branch 'master' into protocol-compression-auto
2021-04-13 23:43:08 +03:00
Nikita Mikhaylov
024374a2ec
review fixes
2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
704fb04941
better
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
c333c3dedb
review fixes
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
36a8419f60
style
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
8a4b5a586e
fixed uuid
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
7276b40556
better
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4843f86329
use only one connection between initiator and worker
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4e4b383214
added hash of itiator address
2021-04-13 22:39:41 +03:00
Alexander Tokmakov
05e4c8d3ef
fix attach mv in atomic db
2021-04-13 22:13:26 +03:00
Azat Khuzhin
d59bdfd45d
Fix one more lock-order-inversion
...
TSan report [1]:
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=11314)
Cycle in lock order graph: M183938897938677368 (0x000000000000) => M2505 (0x7b9000002008) => M183938897938677368
Mutex M2505 acquired here while holding mutex M183938897938677368 in thread T6:
0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a327b6)
1 std::__1::__libcpp_recursive_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:385:10 (clickhouse-tsan+0x17cdb689)
2 std::__1::recursive_mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:71:14 (clickhouse-tsan+0x17cdb689)
3 std::__1::unique_lock<std::__1::recursive_mutex>::unique_lock(std::__1::recursive_mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:119:61 (clickhouse-tsan+0x11e3506f)
4 DB::Context::getLock() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:517:12 (clickhouse-tsan+0x11e3506f)
5 DB::Context::getSchedulePool() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1517:17 (clickhouse-tsan+0x11e3506f)
6 DB::IBackgroundJobExecutor::start() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:158:42 (clickhouse-tsan+0x12be1cda)
7 DB::StorageMergeTree::startup() obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:112:29 (clickhouse-tsan+0x129ed46e)
8 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_2::operator()(std::__1::shared_ptr<DB::IStorage> const&) const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:230:16 (clickhouse-tsan+0x11d71fba)
9 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3::operator()() const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:238:56 (clickhouse-tsan+0x11d71fba)
10 decltype(std::__1::forward<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(fp)()) std::__1::__invoke<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x11d71fba)
11 void std::__1::__invoke_void_return_wrapper<void>::__call<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x11d71fba)
12 std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x11d71fba)
13 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x11d71fba)
14 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b3b8e0)
15 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b3b8e0)
16 ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b3b8e0)
17 void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3e600)
18 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(fp0)...)) std::__1::__invoke_constexpr<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3682:1 (clickhouse-tsan+0x8b3e600)
19 decltype(auto) std::__1::__apply_tuple_impl<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::__tuple_indices<std::__1::tuple<>&...>) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1415:1 (clickhouse-tsan+0x8b3e600)
20 decltype(auto) std::__1::apply<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1424:1 (clickhouse-tsan+0x8b3e600)
21 ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()::operator()() obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:178:13 (clickhouse-tsan+0x8b3e600)
22 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3e561)
23 void std::__1::__invoke_void_return_wrapper<void>::__call<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x8b3e561)
24 std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x8b3e561)
25 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x8b3e561)
26 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b39205)
27 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b39205)
28 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b39205)
29 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3cea8)
30 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3cea8)
31 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>&, std::__1::__tuple_indices<>) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:280:5 (clickhouse-tsan+0x8b3cea8)
32 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:291:5 (clickhouse-tsan+0x8b3cea8)
Mutex M183938897938677368 previously acquired by the same thread here:
0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a327b6)
1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse-tsan+0x17cdb4f9)
2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse-tsan+0x17cdb4f9)
3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse-tsan+0x12be1ca9)
4 DB::IBackgroundJobExecutor::start() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:155:21 (clickhouse-tsan+0x12be1ca9)
5 DB::StorageMergeTree::startup() obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:112:29 (clickhouse-tsan+0x129ed46e)
6 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_2::operator()(std::__1::shared_ptr<DB::IStorage> const&) const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:230:16 (clickhouse-tsan+0x11d71fba)
7 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3::operator()() const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:238:56 (clickhouse-tsan+0x11d71fba)
8 decltype(std::__1::forward<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(fp)()) std::__1::__invoke<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x11d71fba)
9 void std::__1::__invoke_void_return_wrapper<void>::__call<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x11d71fba)
10 std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x11d71fba)
11 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x11d71fba)
12 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b3b8e0)
13 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b3b8e0)
14 ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b3b8e0)
15 void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3e600)
16 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(fp0)...)) std::__1::__invoke_constexpr<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3682:1 (clickhouse-tsan+0x8b3e600)
17 decltype(auto) std::__1::__apply_tuple_impl<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::__tuple_indices<std::__1::tuple<>&...>) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1415:1 (clickhouse-tsan+0x8b3e600)
18 decltype(auto) std::__1::apply<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1424:1 (clickhouse-tsan+0x8b3e600)
19 ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()::operator()() obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:178:13 (clickhouse-tsan+0x8b3e600)
20 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3e561)
21 void std::__1::__invoke_void_return_wrapper<void>::__call<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x8b3e561)
22 std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x8b3e561)
23 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x8b3e561)
24 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b39205)
25 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b39205)
26 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b39205)
27 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3cea8)
28 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3cea8)
29 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>&, std::__1::__tuple_indices<>) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:280:5 (clickhouse-tsan+0x8b3cea8)
30 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:291:5 (clickhouse-tsan+0x8b3cea8)
Mutex M183938897938677368 acquired here while holding mutex M2505 in main thread:
0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a327b6)
1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse-tsan+0x17cdb4f9)
2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse-tsan+0x17cdb4f9)
3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse-tsan+0x12be261b)
4 DB::IBackgroundJobExecutor::finish() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:167:21 (clickhouse-tsan+0x12be261b)
5 DB::IBackgroundJobExecutor::~IBackgroundJobExecutor() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:185:5 (clickhouse-tsan+0x12be261b)
6 DB::StorageMergeTree::~StorageMergeTree() obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:174:1 (clickhouse-tsan+0x129ed768)
7 std::__1::default_delete<DB::StorageMergeTree>::operator()(DB::StorageMergeTree*) const obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1397:5 (clickhouse-tsan+0x12e48b0b)
8 std::__1::__shared_ptr_pointer<DB::StorageMergeTree*, std::__1::shared_ptr<DB::StorageMergeTree>::__shared_ptr_default_delete<DB::StorageMergeTree, DB::StorageMergeTree>, std::__1::allocator<DB::StorageMergeTree> >::__on_zero_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2565:5 (clickhouse-tsan+0x12e48b0b)
9 std::__1::__shared_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2475:9 (clickhouse-tsan+0x125b53ea)
10 std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2517:27 (clickhouse-tsan+0x125b53ea)
11 std::__1::shared_ptr<DB::IStorage>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19 (clickhouse-tsan+0x125b53ea)
12 DB::SystemLog<DB::AsynchronousMetricLogElement>::~SystemLog() obj-x86_64-linux-gnu/../src/Interpreters/SystemLog.h:118:7 (clickhouse-tsan+0x125b53ea)
13 std::__1::allocator<DB::AsynchronousMetricLog>::destroy(DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:891:15 (clickhouse-tsan+0x125b1dd8)
14 void std::__1::allocator_traits<std::__1::allocator<DB::AsynchronousMetricLog> >::__destroy<DB::AsynchronousMetricLog>(std::__1::integral_constant<bool, true>, std::__1::allocator<DB::AsynchronousMetricLog>&, DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:539:21 (clickhouse-tsan+0x125b1dd8)
15 void std::__1::allocator_traits<std::__1::allocator<DB::AsynchronousMetricLog> >::destroy<DB::AsynchronousMetricLog>(std::__1::allocator<DB::AsynchronousMetricLog>&, DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:487:14 (clickhouse-tsan+0x125b1dd8)
16 std::__1::__shared_ptr_emplace<DB::AsynchronousMetricLog, std::__1::allocator<DB::AsynchronousMetricLog> >::__on_zero_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2611:9 (clickhouse-tsan+0x125b1dd8)
17 std::__1::__shared_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2475:9 (clickhouse-tsan+0x125904ff)
18 std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2517:27 (clickhouse-tsan+0x125904ff)
19 std::__1::shared_ptr<DB::AsynchronousMetricLog>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19 (clickhouse-tsan+0x125904ff)
20 DB::SystemLogs::~SystemLogs() obj-x86_64-linux-gnu/../src/Interpreters/SystemLog.cpp:155:1 (clickhouse-tsan+0x125904ff)
21 std::__1::__optional_destruct_base<DB::SystemLogs, false>::reset() obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:245:21 (clickhouse-tsan+0x11e43655)
22 DB::ContextShared::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:443:21 (clickhouse-tsan+0x11e43655)
23 DB::Context::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2251:13 (clickhouse-tsan+0x11e3be37)
24 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5::operator()() const obj-x86_64-linux-gnu/../programs/server/Server.cpp:892:5 (clickhouse-tsan+0x8abacc2)
25 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::invoke() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:97:9 (clickhouse-tsan+0x8abacc2)
26 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::~basic_scope_guard() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:47:28 (clickhouse-tsan+0x8abacc2)
27 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:1395:1 (clickhouse-tsan+0x8ab5cba)
28 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b4ac7b)
29 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa97be)
30 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b67093)
31 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8aa838e)
32 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8aa68f9)
Mutex M2505 previously acquired by the same thread here:
0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a327b6)
1 std::__1::__libcpp_recursive_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:385:10 (clickhouse-tsan+0x17cdb689)
2 std::__1::recursive_mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:71:14 (clickhouse-tsan+0x17cdb689)
3 std::__1::lock_guard<std::__1::recursive_mutex>::lock_guard(std::__1::recursive_mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse-tsan+0x11e4363f)
4 DB::ContextShared::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:438:21 (clickhouse-tsan+0x11e4363f)
5 DB::Context::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2251:13 (clickhouse-tsan+0x11e3be37)
6 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5::operator()() const obj-x86_64-linux-gnu/../programs/server/Server.cpp:892:5 (clickhouse-tsan+0x8abacc2)
7 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::invoke() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:97:9 (clickhouse-tsan+0x8abacc2)
8 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::~basic_scope_guard() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:47:28 (clickhouse-tsan+0x8abacc2)
9 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:1395:1 (clickhouse-tsan+0x8ab5cba)
10 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b4ac7b)
11 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa97be)
12 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b67093)
13 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8aa838e)
14 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8aa68f9)
[1]: https://clickhouse-test-reports.s3.yandex.net/21318/f3b1ad0f5d1024275674e1beac24251ae97c8453/functional_stateful_tests_(thread).html#fail1
v2: Convert ContextSharedPart::system_logs to std::unique_ptr (to avoid copy ctor)
v3: Fix readability-identifier-naming,-warnings-as-errors for system_logs_
v4: fix conflicts
2021-04-13 21:37:38 +03:00
Azat Khuzhin
a61ae26729
Fix lock-order-inversion during system.*_log shutting down
...
As TSan reports [1]:
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=24429)
Cycle in lock order graph: M152695175523663992 (0x000000000000) => M2505 (0x7b9000002008) => M152695175523663992
Mutex M2505 acquired here while holding mutex M152695175523663992 in thread T7:
0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a301b6)
1 std::__1::__libcpp_recursive_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:385:10 (clickhouse-tsan+0x17cd6e89)
2 std::__1::recursive_mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:71:14 (clickhouse-tsan+0x17cd6e89)
3 std::__1::unique_lock<std::__1::recursive_mutex>::unique_lock(std::__1::recursive_mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:119:61 (clickhouse-tsan+0x11e32a9f)
4 DB::Context::getLock() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:517:12 (clickhouse-tsan+0x11e32a9f)
5 DB::Context::getSchedulePool() const obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1517:17 (clickhouse-tsan+0x11e32a9f)
6 DB::IBackgroundJobExecutor::start() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:158:42 (clickhouse-tsan+0x12bde50a)
7 DB::StorageMergeTree::startup() obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:112:29 (clickhouse-tsan+0x129e9e1e)
8 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_2::operator()(std::__1::shared_ptr<DB::IStorage> const&) const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:230:16 (clickhouse-tsan+0x11d6fa2a)
9 DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3::operator()() const obj-x86_64-linux-gnu/../src/Databases/DatabaseOrdinary.cpp:238:56 (clickhouse-tsan+0x11d6fa2a)
10 decltype(std::__1::forward<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(fp)()) std::__1::__invoke<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x11d6fa2a)
11 void std::__1::__invoke_void_return_wrapper<void>::__call<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&>(DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x11d6fa2a)
12 std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x11d6fa2a)
13 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::DatabaseOrdinary::startupTables(ThreadPoolImpl<ThreadFromGlobalPool>&)::$_3, void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x11d6fa2a)
14 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b39350)
15 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b39350)
16 ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b39350)
17 void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3c070)
18 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(fp0)...)) std::__1::__invoke_constexpr<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3682:1 (clickhouse-tsan+0x8b3c070)
19 decltype(auto) std::__1::__apply_tuple_impl<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::__tuple_indices<std::__1::tuple<>&...>) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1415:1 (clickhouse-tsan+0x8b3c070)
20 decltype(auto) std::__1::apply<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&, std::__1::tuple<>&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&) obj-x86_64-linux-gnu/../contrib/libcxx/include/tuple:1424:1 (clickhouse-tsan+0x8b3c070)
21 ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()::operator()() obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:178:13 (clickhouse-tsan+0x8b3c070)
22 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3bfd1)
23 void std::__1::__invoke_void_return_wrapper<void>::__call<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()&>(void&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:348:9 (clickhouse-tsan+0x8b3bfd1)
24 std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()>::operator()() obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608:12 (clickhouse-tsan+0x8b3bfd1)
25 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2089:16 (clickhouse-tsan+0x8b3bfd1)
26 std::__1::__function::__policy_func<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16 (clickhouse-tsan+0x8b36c75)
27 std::__1::function<void ()>::operator()() const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12 (clickhouse-tsan+0x8b36c75)
28 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:247:17 (clickhouse-tsan+0x8b36c75)
29 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:124:73 (clickhouse-tsan+0x8b3a918)
30 decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...) obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676:1 (clickhouse-tsan+0x8b3a918)
31 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>&, std::__1::__tuple_indices<>) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:280:5 (clickhouse-tsan+0x8b3a918)
32 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:291:5 (clickhouse-tsan+0x8b3a918)
Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message
Mutex M152695175523663992 acquired here while holding mutex M2505 in main thread:
0 pthread_mutex_lock <null> (clickhouse-tsan+0x8a301b6)
1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse-tsan+0x17cd6cf9)
2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse-tsan+0x17cd6cf9)
3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse-tsan+0x12bdee4b)
4 DB::IBackgroundJobExecutor::finish() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:167:21 (clickhouse-tsan+0x12bdee4b)
5 DB::IBackgroundJobExecutor::~IBackgroundJobExecutor() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:185:5 (clickhouse-tsan+0x12bdee4b)
6 DB::StorageMergeTree::~StorageMergeTree() obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:174:1 (clickhouse-tsan+0x129ea118)
7 std::__1::default_delete<DB::StorageMergeTree>::operator()(DB::StorageMergeTree*) const obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1397:5 (clickhouse-tsan+0x12e4433b)
8 std::__1::__shared_ptr_pointer<DB::StorageMergeTree*, std::__1::shared_ptr<DB::StorageMergeTree>::__shared_ptr_default_delete<DB::StorageMergeTree, DB::StorageMergeTree>, std::__1::allocator<DB::StorageMergeTree> >::__on_zero_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2565:5 (clickhouse-tsan+0x12e4433b)
9 std::__1::__shared_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2475:9 (clickhouse-tsan+0x125b355a)
10 std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2517:27 (clickhouse-tsan+0x125b355a)
11 std::__1::shared_ptr<DB::IStorage>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19 (clickhouse-tsan+0x125b355a)
12 DB::SystemLog<DB::AsynchronousMetricLogElement>::~SystemLog() obj-x86_64-linux-gnu/../src/Interpreters/SystemLog.h:118:7 (clickhouse-tsan+0x125b355a)
13 std::__1::allocator<DB::AsynchronousMetricLog>::destroy(DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:891:15 (clickhouse-tsan+0x125aff68)
14 void std::__1::allocator_traits<std::__1::allocator<DB::AsynchronousMetricLog> >::__destroy<DB::AsynchronousMetricLog>(std::__1::integral_constant<bool, true>, std::__1::allocator<DB::AsynchronousMetricLog>&, DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:539:21 (clickhouse-tsan+0x125aff68)
15 void std::__1::allocator_traits<std::__1::allocator<DB::AsynchronousMetricLog> >::destroy<DB::AsynchronousMetricLog>(std::__1::allocator<DB::AsynchronousMetricLog>&, DB::AsynchronousMetricLog*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:487:14 (clickhouse-tsan+0x125aff68)
16 std::__1::__shared_ptr_emplace<DB::AsynchronousMetricLog, std::__1::allocator<DB::AsynchronousMetricLog> >::__on_zero_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2611:9 (clickhouse-tsan+0x125aff68)
17 std::__1::__shared_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2475:9 (clickhouse-tsan+0x1258e74f)
18 std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2517:27 (clickhouse-tsan+0x1258e74f)
19 std::__1::shared_ptr<DB::AsynchronousMetricLog>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19 (clickhouse-tsan+0x1258e74f)
20 DB::SystemLogs::~SystemLogs() obj-x86_64-linux-gnu/../src/Interpreters/SystemLog.cpp:155:1 (clickhouse-tsan+0x1258e74f)
21 std::__1::__optional_destruct_base<DB::SystemLogs, false>::reset() obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:245:21 (clickhouse-tsan+0x11e41085)
22 DB::ContextShared::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:443:21 (clickhouse-tsan+0x11e41085)
23 DB::Context::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2251:13 (clickhouse-tsan+0x11e39867)
24 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5::operator()() const obj-x86_64-linux-gnu/../programs/server/Server.cpp:892:5 (clickhouse-tsan+0x8ab8732)
25 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::invoke() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:97:9 (clickhouse-tsan+0x8ab8732)
26 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::~basic_scope_guard() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:47:28 (clickhouse-tsan+0x8ab8732)
27 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:1395:1 (clickhouse-tsan+0x8ab3839)
28 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b464ab)
29 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa71be)
30 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b628c3)
31 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8aa5d8e)
32 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8aa42f9)
[1]: https://clickhouse-test-reports.s3.yandex.net/21318/38be9ff43ac4c46ce6e803fc125d910bde1d4c71/functional_stateful_tests_(thread).html#fail1
2021-04-13 21:37:38 +03:00
Azat Khuzhin
3056d2c5d1
Fix data-race during server shutdown the context
...
Found with 01737_clickhouse_server_wait_server_pool:
==================
WARNING: ThreadSanitizer: data race (pid=13248)
Write of size 1 at 0x7b9000003a38 by main thread:
0 std::__1::__optional_destruct_base<DB::SystemLogs, false>::reset() obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:246:24 (clickhouse-tsan+0x11e3043e)
1 DB::ContextShared::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:441:21 (clickhouse-tsan+0x11e3043e)
2 DB::Context::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2249:13 (clickhouse-tsan+0x11e28c17)
3 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5::operator()() const obj-x86_64-linux-gnu/../programs/server/Server.cpp:892:5 (clickhouse-tsan+0x8ab1a32)
4 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::invoke() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:97:9 (clickhouse-tsan+0x8ab1a32)
5 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::~basic_scope_guard() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:47:28 (clickhouse-tsan+0x8ab1a32)
6 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:1395:1 (clickhouse-tsan+0x8aacb39)
7 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
8 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
9 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
10 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
11 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)
Previous read of size 1 at 0x7b9000003a38 by thread T2 (mutexes: write M2504):
0 std::__1::__optional_storage_base<DB::SystemLogs, false>::has_value() const obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:295:22 (clickhouse-tsan+0x11e25348)
1 std::__1::optional<DB::SystemLogs>::operator bool() const obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:938:64 (clickhouse-tsan+0x11e25348)
2 DB::Context::getQueryLog() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1875:10 (clickhouse-tsan+0x11e25348)
3 DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:657:50 (clickhouse-tsan+0x12653751)
4 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:904:30 (clickhouse-tsan+0x12651308)
5 DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:289:24 (clickhouse-tsan+0x12f04b45)
6 DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1500:9 (clickhouse-tsan+0x12f13907)
7 Poco::Net::TCPServerConnection::start() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse-tsan+0x15b1f722)
8 Poco::Net::TCPServerDispatcher::run() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:113:19 (clickhouse-tsan+0x15b1fe4e)
9 Poco::PooledThread::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14 (clickhouse-tsan+0x15c86fe1)
10 Poco::(anonymous namespace)::RunnableHolder::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:55:11 (clickhouse-tsan+0x15c8557f)
11 Poco::ThreadImpl::runnableEntry(void*) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27 (clickhouse-tsan+0x15c83d87)
Location is heap block of size 7296 at 0x7b9000002000 allocated by main thread:
0 operator new(unsigned long) <null> (clickhouse-tsan+0x8a9aae7)
1 std::__1::__unique_if<DB::ContextShared>::__unique_single std::__1::make_unique<DB::ContextShared>() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2068:28 (clickhouse-tsan+0x11e15c2c)
2 DB::Context::createShared() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:503:32 (clickhouse-tsan+0x11e15c2c)
3 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:426:27 (clickhouse-tsan+0x8aa19ee)
4 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
5 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
6 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
7 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
8 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)
Mutex M2504 (0x7b9000002008) created at:
0 pthread_mutex_init <null> (clickhouse-tsan+0x8a0d37d)
1 std::__1::__libcpp_recursive_mutex_init(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:370:10 (clickhouse-tsan+0x17cc4d93)
2 std::__1::recursive_mutex::recursive_mutex() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:56:14 (clickhouse-tsan+0x17cc4d93)
3 DB::ContextShared::ContextShared() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:394:5 (clickhouse-tsan+0x11e40bc3)
4 std::__1::__unique_if<DB::ContextShared>::__unique_single std::__1::make_unique<DB::ContextShared>() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2068:32 (clickhouse-tsan+0x11e15c37)
5 DB::Context::createShared() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:503:32 (clickhouse-tsan+0x11e15c37)
6 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:426:27 (clickhouse-tsan+0x8aa19ee)
7 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
8 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
9 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
10 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
11 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)
Thread T2 'TCPHandler' (tid=13643, running) created by main thread at:
0 pthread_create <null> (clickhouse-tsan+0x8a0bf0b)
1 Poco::ThreadImpl::startImpl(Poco::SharedPtr<Poco::Runnable, Poco::ReferenceCounter, Poco::ReleasePolicy<Poco::Runnable> >) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:202:6 (clickhouse-tsan+0x15c83827)
2 Poco::Thread::start(Poco::Runnable&) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:128:2 (clickhouse-tsan+0x15c84f6c)
3 Poco::PooledThread::start() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:85:10 (clickhouse-tsan+0x15c873e2)
4 Poco::ThreadPool::ThreadPool(int, int, int, int) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:252:12 (clickhouse-tsan+0x15c873e2)
5 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:843:22 (clickhouse-tsan+0x8aa8e5f)
6 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
7 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
8 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
9 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
10 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)
SUMMARY: ThreadSanitizer: data race obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:246:24 in std::__1::__optional_destruct_base<DB::SystemLogs, false>::reset()
v2: fix deadlock by calling SystemLogs::shutdown w/o Context lock
2021-04-13 21:37:38 +03:00
Kruglov Pavel
6350f734dc
Merge branch 'master' into datasketches-uniq
2021-04-13 19:34:15 +03:00
hexiaoting
4f36ac7f64
Fix error
2021-04-13 17:31:49 +08:00
Nikolai Kochetov
d1d2b89a51
Use setting for explain json
2021-04-13 10:51:55 +03:00
Nikolai Kochetov
e731dfe650
Merge pull request #22991 from ClickHouse/untuple-and-subquery
...
Fix subquery with untuple.
2021-04-13 09:53:13 +03:00
hexiaoting
77c460e8d1
Merge remote-tracking branch 'origin/master' into dev-sumcount
2021-04-13 11:14:14 +08:00
hexiaoting
52bdec396b
Merge remote-tracking branch 'origin/master' into dev_nested_upadte
2021-04-13 09:56:47 +08:00
alexey-milovidov
e2edecc8e2
Merge pull request #22434 from Avogar/catboost-reload
...
Update paths to the catboost model configs in config reloading
2021-04-13 01:21:04 +03:00
Ivan
a73483751f
Merge branch 'master' into ast-table-identifier-2
2021-04-12 20:23:28 +03:00
Ivan Lezhankin
b559e45d93
Some minor fixes
2021-04-12 20:19:46 +03:00
Nikolai Kochetov
0448e1415f
Fix subquery with untuple.
2021-04-12 15:15:55 +03:00
Pavel Kruglov
e4f3678ae9
Fix build
2021-04-12 12:28:46 +03:00
Azat Khuzhin
79bd8d4d3f
Respect optimize_skip_unused_shards_rewrite_in with optimize_skip_unused_shards_limit
2021-04-12 10:37:28 +03:00
Azat Khuzhin
fbb386dca5
Rewrite IN in query for remote shards to exclude values that does not belongs to shard
...
v2: fix optimize_skip_unused_shards_rewrite_in for sharding_key wrapped into function
v3: fix column name for optimize_skip_unused_shards_rewrite_in
v4: fix optimize_skip_unused_shards_rewrite_in with Null
v5:
- squash with Remove query argument for IStreamFactory::createForShard()
- use proper column after function execution (using sharding_key_column_name)
- update the test reference since (X) now is tuple(X)
2021-04-12 10:37:28 +03:00
Azat Khuzhin
773212529b
Make Cluster::SlotToShards public
2021-04-12 10:32:04 +03:00
alexey-milovidov
d21ffd83a7
Merge branch 'master' into catboost-reload
2021-04-11 13:29:36 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr ( #22297 )
...
* Replace all Context references with std::weak_ptr
* Fix shared context captured by value
* Fix build
* Fix Context with named sessions
* Fix copy context
* Fix gcc build
* Merge with master and fix build
* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
alexey-milovidov
5469cc15c1
Trigger CI
2021-04-10 23:43:56 +03:00
Nikolai Kochetov
48bcd5b490
Add tree conversion for expression.
2021-04-09 19:18:45 +03:00
Ivan Lezhankin
3da3250bfd
Merge branch 'master' into ast-table-identifier-2
2021-04-09 17:56:15 +03:00
Nikolai Kochetov
66b6d53a93
Add json=1 setting for explain actions
2021-04-09 17:44:58 +03:00
Alexander Kuzmenkov
0264124146
Merge pull request #21942 from ucasFL/distributed_depth
...
Add settings max_distributed_depth
2021-04-09 15:52:58 +03:00
Alexander Kuzmenkov
5db8e4c111
Update SystemLog.h
2021-04-09 15:48:09 +03:00
Maksim Kita
0cacca5b6e
Merge pull request #22833 from ClickHouse/trivial-fix
...
Fix what looks like a trivial mistake
2021-04-09 12:02:16 +03:00
Nikolai Kochetov
28ca191102
Merge pull request #22087 from ClickHouse/better-filter-push-down
...
Better filter push down
2021-04-09 10:22:17 +03:00
alexey-milovidov
b02a82250b
Merge pull request #22858 from vitlibar/fix-hanging-in-temporary-live-view-cleaner
...
Fix hanging in TemporaryLiveViewCleaner
2021-04-09 09:25:38 +03:00
Nikolai Kochetov
3426bc3906
Merge pull request #22763 from CurtizJ/fix-having-push-down
...
Fix pushdown of having
2021-04-08 21:53:50 +03:00
Nikolai Kochetov
3ea186358e
Fix fuzz crash.
2021-04-08 18:00:32 +03:00
Vitaly Baranov
5bcb7a0fed
Fix hanging in TemporaryLiveViewCleaner
2021-04-08 16:14:49 +03:00
Alexander Kuzmenkov
4a66548c5d
write distributed depth as VarUInt to save space, because it's small
2021-04-08 15:17:47 +03:00
Alexander Kuzmenkov
dc7a2b7f88
Merge pull request #22761 from ClickHouse/aku/flush-log
...
add more messages when flushing the logs
2021-04-08 13:50:17 +03:00
Nikolai Kochetov
118090396f
Filter column after filter push down now always the first one..
2021-04-08 12:54:39 +03:00
Alexey Milovidov
8fb6e6fbf2
Fix mistake
2021-04-08 11:51:22 +03:00
alesapin
1533f9b9aa
Merge pull request #14113 from johnskopis/dynamic-interserver-creds-v20
...
Support interserver credential rotation
2021-04-08 11:01:40 +03:00
alesapin
98871f8c5a
More clear messages
2021-04-07 17:05:40 +03:00
alesapin
2a6560b1c6
Better interface
2021-04-07 16:59:18 +03:00
alesapin
2987bbc948
Small improvements
2021-04-07 16:52:11 +03:00
Alexander Kuzmenkov
fecd5f3435
more debug messages
2021-04-07 16:16:19 +03:00
Alexander Kuzmenkov
06bb58cb69
Merge pull request #22704 from ClickHouse/aku/window-multiple
...
fix window functions with multiple input streams and no sorting
2021-04-07 15:15:47 +03:00
Vladimir
9fe20c1628
Revert "Move conditions from JOIN ON to WHERE"
2021-04-07 14:57:20 +03:00
alesapin
feff1175f4
Merge branch 'master' into johnskopis/dynamic-interserver-creds-v20
2021-04-07 13:55:38 +03:00
Vladimir
864ab4a7ab
Merge pull request #21370 from vdimir/fix-merge-engine-join-group-by
2021-04-07 11:52:27 +03:00
hexiaoting
ee24d9a5cc
Merge remote-tracking branch 'origin/master' into dev_nested_upadte
2021-04-07 10:42:51 +08:00
hexiaoting
8c60f95551
Fix error
2021-04-07 10:39:01 +08:00
feng lv
78afb5b23c
fix
2021-04-07 01:58:57 +00:00
Azat Khuzhin
f157278b72
Safer SCOPE_EXIT
...
It executes the code in the dtor, that should never throw.
2021-04-06 21:10:37 +03:00
Azat Khuzhin
b25a4e066c
Lock MEMORY_LIMIT_EXCEEDED in ThreadStatus::detachQuery()
...
Found with fuzzer [1]:
<Fatal> BaseDaemon: (version 21.5.1.6440, build id: 3B097C902DDAA35688D90750552ED499DC5D10A0) (from thread 8012) Terminate called for uncaught exception:
Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 153.51 MiB (attempt to allocate chunk of 4194368 bytes), maximum: 150.00 MiB, Stack trace (when copying this message, always include the lines below):
0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x26fdcdd9 in /usr/bin/clickhouse
1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:57: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xad0df02 in /usr/bin/clickhouse
2. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:1444: DB::Exception::Exception<char const*, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*&&, char const*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0xacc7bef in /usr/bin/clickhouse
3. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:219: MemoryTracker::alloc(long) @ 0xacc65eb in /usr/bin/clickhouse
4. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
5. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
6. ./obj-x86_64-linux-gnu/../src/Common/AllocatorWithMemoryTracking.h:35: AllocatorWithMemoryTracking<DB::Field>::allocate(unsigned long) @ 0xad0a2fe in /usr/bin/clickhouse
7. void std::__1::vector<DB::Field, AllocatorWithMemoryTracking<DB::Field> >::__push_back_slow_path<DB::Field>(DB::Field&&) @ 0x11712a51 in /usr/bin/clickhouse
8. ./obj-x86_64-linux-gnu/../src/Interpreters/ThreadStatusExt.cpp:356: DB::ThreadStatus::detachQuery(bool, bool) @ 0x1f5d5237 in /usr/bin/clickhouse
9. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:0: void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0x20c488e6 in /usr/bin/clickhouse
10. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:0: ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xad9f6cc in /usr/bin/clickhouse
11. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1655: void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) @ 0xada8264 in /usr/bin/clickhouse
12. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
13. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
(version 21.5.1.6440)
[1]: https://clickhouse-test-reports.s3.yandex.net/22583/69296876005c0fa171c755f8b224e4d58192c402/stress_test_(address).html#fail1
2021-04-06 21:10:36 +03:00
Alexander Kuzmenkov
d9ad06bd6d
fix ubsan
2021-04-06 19:24:56 +03:00
alesapin
b4fc13de8e
fix style
2021-04-06 17:02:56 +03:00
alesapin
4633afe962
Some fixes
2021-04-06 16:56:14 +03:00
alesapin
1c8e33414f
Merge branch 'master' into johnskopis/dynamic-interserver-creds-v20
2021-04-06 16:28:46 +03:00
Alexander Kuzmenkov
585da2c8d0
Merge remote-tracking branch 'origin/master' into HEAD
2021-04-06 14:58:21 +03:00
Alexey Milovidov
f70c9e8752
Merge branch 'master' into protocol-compression-auto
2021-04-06 14:53:31 +03:00
Alexander Kuzmenkov
06e7a9f8f2
fix window functions with multiple input streams and no sorting
2021-04-06 14:43:27 +03:00
Nikolai Kochetov
6102652c99
Merge branch 'master' into better-filter-push-down
2021-04-06 13:38:03 +03:00
vdimir
735154c81a
Handle not plain where tree in StorageMerge modifySelect
2021-04-06 12:29:29 +03:00
alesapin
86a843bb51
Merge pull request #21509 from kssenii/library-bridge
...
clickhouse-library-bridge for library dictionary source
2021-04-06 12:26:08 +03:00
Amos Bird
46c9affa83
Fix CTE in view.
2021-04-06 11:32:16 +08:00
Anton Popov
c5354885a6
Merge pull request #22654 from azat/log-nested-fix
...
[RFC] Fix LOGICAL_ERROR for Log with nested types w/o columns in the SELECT clause
2021-04-06 00:34:58 +03:00
Maksim Kita
de71553c9d
Merge pull request #22593 from zlobober/introduce_distributed_write
...
Introduce IStorage::distributedWrite method for distributed INSERT SELECTS
2021-04-05 22:59:57 +03:00
feng lv
f6e4e13f89
fix conflict
2021-04-05 14:08:30 +00:00
feng lv
56073db22d
max distributed depth
...
Add settings max_distributed_depth
fix style
fix
fix
fix
fix test
fix
fix
2021-04-05 14:00:54 +00:00
kssenii
9dba529494
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge
2021-04-05 13:14:51 +00:00
pingyu
e05ee09621
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into datasketches-uniq
2021-04-05 16:47:53 +08:00
Azat Khuzhin
42d858b7e5
Fix LOGICAL_ERROR for Log with nested types w/o columns in the SELECT clause
...
Found with fuzzer [1]:
<details>
2021.04.04 10:25:46.762596 [ 135 ] {4b6b5de3-d242-4267-910a-76b235467356} <Fatal> : Logical error: 'Logical error: no information about file column%2Ename.size0 in StorageLog'.
...
2021.04.04 10:25:46.763563 [ 44 ] {} <Trace> BaseDaemon: Received signal 6
2021.04.04 10:25:46.765884 [ 165 ] {} <Fatal> BaseDaemon: 5. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
2021.04.04 10:25:46.901540 [ 165 ] {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:51: DB::handle_error_code() @ 0x11cc4f16 in /workspace/clickhouse
2021.04.04 10:25:47.030755 [ 165 ] {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:58: DB::Exception::Exception() @ 0x11cc5025 in /workspace/clickhouse
2021.04.04 10:25:47.617201 [ 165 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:183: DB::LogSource::readData()
</details>
[1]: https://clickhouse-test-reports.s3.yandex.net/22583/69296876005c0fa171c755f8b224e4d58192c402/fuzzer_debug/report.html#fail1
The problem here is that there is no column%2Ename.size0 there is
column.size0 instead, and it seems that reading sizeX file is not enough
in other places, so just filter them out in
ExpressionActions::getSmallestColumn() (fallback method for storages w/o
getColumnSizes() implemented)
2021-04-05 08:48:04 +03:00
Maxim Akhmedov
725fa17961
Introduce IStorage::distributedWrite method for distributed INSERT SELECT.
2021-04-05 02:14:27 +03:00
alexey-milovidov
b1e7624b3f
Merge pull request #22589 from ClickHouse/fix-comments
...
Fix comments
2021-04-05 00:22:27 +03:00
Anton Popov
0d03b8c0c4
Merge pull request #21889 from CurtizJ/fix-aggregation-in-order-1
...
Fix bugs in aggregation by primary key
2021-04-04 18:17:04 +03:00
pingyu
1ea74a1947
bug fix. add unit tests.#
2021-04-04 22:22:47 +08:00
Alexey Milovidov
54caa61af5
Fix comments
2021-04-04 12:33:06 +03:00
alexey-milovidov
30ece2c28f
Merge pull request #22510 from vdimir/merge-join-lc-bug-fix
...
Convert right block to full in MergeJoin
2021-04-03 12:54:34 +03:00
alexey-milovidov
840ecb5e99
Update MergeJoin.cpp
2021-04-02 21:07:11 +03:00
Alexander Kuzmenkov
d0037329cf
Merge pull request #22459 from ClickHouse/aku/window-frame-offset
...
fix window frame offset check and add more tests
2021-04-02 19:49:59 +03:00
vdimir
c8f51998ba
Remove LowCardinality from right block sample in MergeJoin
2021-04-02 17:51:02 +03:00
vdimir
8cd841c4ac
Handle lowcardinality block in MergeJoin extractMinMax
2021-04-02 16:55:42 +03:00
Anton Popov
a000d6154c
fix clang-tidy
2021-04-02 15:10:49 +03:00
hexiaoting
6e8583a028
Add array size check for Nested Type subcolumns when Update
2021-04-02 19:31:33 +08:00
Pavel Kruglov
dff7834af0
Fix config comparison
2021-04-02 13:17:28 +03:00
Vladimir
9b40f384e9
Merge pull request #20622 from vdimir/issue-20497
2021-04-02 11:48:29 +03:00
Anton Popov
56320bcbfd
minor refactoring
2021-04-02 01:16:54 +03:00
Anton Popov
88cd775f6a
Merge remote-tracking branch 'upstream/master' into HEAD
2021-04-02 00:14:03 +03:00
Alexander Kuzmenkov
d006c5532e
fix window frame offset check and add more tests
2021-04-01 18:56:58 +03:00
vdimir
3f464595eb
Accurate removing 'join' part for queries to 'Merge' engine
2021-04-01 14:31:57 +03:00
vdimir
fff8043e55
Revert "Fix joined table access with Merge engine and aggregation"
...
This reverts commit 5476e68d6c
2021-04-01 14:31:46 +03:00
vdimir
5476e68d6c
Fix joined table access with Merge engine and aggregation
2021-04-01 14:26:28 +03:00
Pavel Kruglov
cc65a1db97
Update paths to the model configs in config reloading
2021-04-01 13:30:05 +03:00
pingyu
86d8bc1c54
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into datasketches-uniq
2021-04-01 12:35:24 +08:00
Alexey Milovidov
f441fd3015
Merge branch 'master' into protocol-compression-auto
2021-04-01 07:18:08 +03:00
Maksim Kita
55984e849c
Merge branch 'master' into hierarchy-dictionaries-updated
2021-03-31 15:55:07 +03:00
tavplubix
6c1e2a0d87
Merge pull request #21690 from ClickHouse/test_multiple_nodes
...
Run three nodes with Replicated database and NuKeeper in CI
2021-03-31 12:31:13 +03:00
vdimir
0353775086
Pass inner_columns to PredicateRewriteVisitorData::rewriteSubquery explicitly
2021-03-31 12:28:25 +03:00
vdimir
7f1b67db06
Merge branch 'master' into issue-20497
2021-03-31 12:16:36 +03:00
Alexey Milovidov
e9bfc7456d
Merge branch 'master' into virtualutil
2021-03-31 08:19:43 +03:00
alexey-milovidov
0c70b06960
Merge branch 'master' into system-querylog-map
2021-03-31 04:54:30 +03:00
alexey-milovidov
59c158300a
Merge pull request #22365 from ClickHouse/aku/processes-database
...
add current database to system.processes
2021-03-31 04:14:17 +03:00
alexey-milovidov
5c15b8a896
Merge branch 'master' into virtualutil
2021-03-30 23:59:44 +03:00
Amos Bird
69204e1d21
Add prefer_column_name_to_alias settings ( #22044 )
2021-03-30 16:51:45 +03:00
Alexander Kuzmenkov
0252f60c23
add current database to system.processes
2021-03-30 16:36:08 +03:00
Alexander Tokmakov
500a20f30d
Merge branch 'master' into test_multiple_nodes
2021-03-30 16:23:33 +03:00
Alexey Milovidov
150faa5e4d
Merge branch 'master' into protocol-compression-auto
2021-03-30 15:59:36 +03:00
Kseniia Sumarokova
39ba3e4024
Merge pull request #20382 from stigsb/materializemysql-version-index
...
MaterializeMySQL: add skipping index for _version column
2021-03-30 14:30:50 +03:00
Alexey Milovidov
9e4e70ddac
Merge branch 'master' into protocol-compression-auto
2021-03-30 13:32:34 +03:00
Alexander Tokmakov
24fc120076
Merge branch 'master' into test_multiple_nodes
2021-03-30 10:40:38 +03:00
alesapin
5e3ebb3e0e
Merge pull request #22274 from ClickHouse/rename_keeper
...
Rename NuKeeper and TestKeeper to Keeper in all places
2021-03-30 10:39:22 +03:00
Alexey Milovidov
21a909bde0
Merge branch 'master' into protocol-compression-auto
2021-03-30 04:15:02 +03:00
Alexey Milovidov
230b2930b7
Merge branch 'master' into virtualutil
2021-03-30 00:43:00 +03:00
alexey-milovidov
25f02f13b0
Merge pull request #22238 from ClickHouse/remove-harmful-code
...
Remove harmful default parameters
2021-03-29 23:46:18 +03:00
kssenii
49fc550d10
Merge branch 'master' of github.com:ClickHouse/ClickHouse into stigsb-materializemysql-version-index
2021-03-29 23:31:47 +03:00
flynn
858d1fe593
Fix table function clusterAllReplicas return wrong _shard_num ( #21498 )
...
* Fix deadlock in delete mutations when predicate contains table itself
* Fix table function clusterAllReplicas return wrong _shard_num
fix build on gcc-10
better
update test
fix
fix
fix
fix
fix
fix cluster config
Co-authored-by: alesapin <alesapin@gmail.com>
2021-03-29 23:21:08 +03:00
Alexander Tokmakov
ccd31d1b72
Merge remote-tracking branch 'ch/rename_keeper' into test_multiple_nodes
2021-03-29 22:44:40 +03:00
Alexander Tokmakov
a5b07f2f03
Merge branch 'master' into test_multiple_nodes
2021-03-29 22:43:25 +03:00
tavplubix
3c0f5a57ec
Merge pull request #21535 from ClickHouse/distributed_ddl_improvements
...
Distributed DDL improvements
2021-03-29 22:40:11 +03:00
alexey-milovidov
9d84f3113a
Merge pull request #21512 from azat/optimize_skip_unused_shards_limit
...
Add optimize_skip_unused_shards_limit
2021-03-29 21:13:18 +03:00
Anton Popov
ea82e7725f
Merge pull request #21562 from CurtizJ/serialization-refactoring-4
...
Refactoring of data types serialization
2021-03-29 16:36:44 +03:00
alesapin
be132a32a2
More renames
2021-03-29 11:24:56 +03:00
pingyu
69ac23e870
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into datasketches-uniq
2021-03-29 14:48:14 +08:00
alexey-milovidov
f90a328568
Merge pull request #22109 from azat/nested-epoll-wait
...
Disable async_socket_for_remote/use_hedged_requests for buggy linux kernels
2021-03-29 05:29:10 +03:00
alexey-milovidov
c762fa2f75
Merge pull request #22129 from vdimir/issue-19303
...
Shrink totals block to 1 row for JOIN with TOTALS and arrayJoin
2021-03-29 05:26:17 +03:00
alexey-milovidov
3bc170e5ec
Merge pull request #22122 from fastio/fix_thread_pool_name_exception
...
The thread pool name should be not longer than 15 chars
2021-03-29 05:24:55 +03:00
Alexey Milovidov
169369e90f
Remove harmful default parameters
2021-03-29 04:08:28 +03:00
Alexey Milovidov
39bb7c39c0
Autodetect compression #22234
2021-03-29 04:04:52 +03:00
Alexey Milovidov
2a8ac01cdb
Rename as suggested by Kita
2021-03-28 22:24:28 +03:00
pingyu
1ac42e003c
wip #14893
2021-03-29 00:46:50 +08:00
Alexey Milovidov
50f712e198
Integrate mmap cache to the infrastructure
2021-03-28 04:10:30 +03:00
Maksim Kita
9f2f0d1095
Refactored hierarchy dictionaries interface
2021-03-26 18:42:32 +03:00
tavplubix
d9ec4090d2
Merge pull request #21936 from abyss7/html-form-fix
...
Fix issues with HTMLForm::MultipartReadBuffer and PeekableReadBuffer
2021-03-26 15:35:34 +03:00
tavplubix
ae62bee939
Merge pull request #21913 from ClickHouse/fix_context_ref
...
Fix possibly dangling reference to Context
2021-03-26 10:39:13 +03:00
Azat Khuzhin
8e445c5e55
Disable async_socket_for_remote/use_hedged_requests for buggy linux kernels
...
async_socket_for_remote/use_hedged_requests uses nested epolls, which
may not be reliable in 5.5+ [1], but it has been fixed in shortly in
5.7+ [2] or
5.6.13+.
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=339ddb53d373
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c54a6a44bf3
2021-03-25 23:39:30 +03:00
Alexander Kuzmenkov
0675f9403c
Merge pull request #21895 from ClickHouse/aku/laglead
...
float RANGE OFFSET and lag/lead_in_frame
2021-03-25 23:00:22 +03:00
Nikolai Kochetov
e9f7213858
Filter push down for LEFT/INNER JOIN, CreatingSets, Union.
2021-03-25 21:11:54 +03:00
vdimir
8f5c8f32df
Shrink totals block to 1 row for JOIN with TOTALS and arrayJoin
2021-03-25 17:32:36 +03:00
Peng Jian
2ede490152
The thread pool name should not longer than 15 chars
2021-03-25 21:57:47 +08:00
alexey-milovidov
37a4f3ab3a
Merge pull request #22063 from ClickHouse/tuple-null-in
...
Allow to search tuple of NULLs in a set
2021-03-25 15:59:41 +03:00
Anton Popov
6a15431be7
Merge remote-tracking branch 'upstream/master' into HEAD
2021-03-25 15:57:35 +03:00
Anton Popov
f58d470dd6
Merge pull request #21830 from foolchi/optimize_select_with_final
...
optimize select final with prewhere primary key
2021-03-25 15:32:13 +03:00
Vitaly Baranov
f6f739eca5
Merge pull request #22027 from vitlibar/external-loader-small-simplification
...
Small simplification in ExternalLoader.
2021-03-25 13:43:31 +03:00
Nikolai Kochetov
0e2f52518f
Extract converting from UnionStep.
2021-03-25 12:57:14 +03:00
Nikolai Kochetov
0afede382b
Use empty header for CreatingSets step.
2021-03-25 11:39:55 +03:00
kssenii
d9326cc229
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge
2021-03-24 18:28:55 +00:00
Nikolai Kochetov
12de01cb48
Remove AddingDelayedSourceStep.
2021-03-24 16:22:51 +03:00
vdimir
bc1b0aed85
Correctly handle table alias in PredicateRewriteVisitor
2021-03-24 15:48:29 +03:00
Nikolai Kochetov
6ca8a66da2
Merge pull request #21550 from amosbird/shardpruningliteral
...
Shard pruning via literals
2021-03-24 13:59:34 +03:00
Alexey Milovidov
eae268f2f0
Allow to search tuple of NULLs in a set
2021-03-24 03:15:48 +03:00
Alexander Kuzmenkov
c9facd631c
Merge remote-tracking branch 'origin/master' into HEAD
2021-03-23 17:40:13 +03:00
Alexander Kuzmenkov
97d5be20bb
Merge pull request #22003 from ClickHouse/aku/field-reinterpret
...
prevent accidental reinterpret_cast in Field::get<>
2021-03-23 15:57:26 +03:00
Vitaly Baranov
46f4c60839
Small simplification in ExternalLoader.
2021-03-23 15:15:44 +03:00
vdimir
ea6b8486e9
Try to fix issue #20497
2021-03-23 15:07:51 +03:00
Vitaly Baranov
090e558da4
Merge pull request #21844 from Avogar/catboost-deadlock
...
Fix deadlock in first catboost model execution
2021-03-23 15:01:04 +03:00
Alexander Tokmakov
c938f4f2fe
fix
2021-03-23 14:29:29 +03:00
alesapin
36898bdc4a
Merge pull request #21142 from Jokser/unfreeze-partitions
...
Add ALTER TABLE UNFREEZE command.
2021-03-23 14:05:38 +03:00
Anton Popov
5de2b8512f
fix destruction of aggregate states
2021-03-23 01:12:14 +03:00
Alexander Kuzmenkov
d40f781b9a
Merge origin/master into tmp (using imerge)
2021-03-22 23:27:18 +03:00
Alexander Kuzmenkov
77a2f03d3b
prevent accidental reinterpret_cast in Field::get<>
2021-03-22 23:23:44 +03:00
Alexander Kuzmenkov
b0284f20c3
Merge branch 'master' into aku/laglead
2021-03-22 21:16:27 +03:00
tavplubix
5dfd0045a6
Merge branch 'master' into distributed_ddl_improvements
2021-03-22 18:13:19 +03:00
Alexander Kuzmenkov
ac5fdda999
Merge pull request #21915 from ClickHouse/aku/window-mt-order
...
disable optimize_read_in_order for window functions
2021-03-22 17:21:00 +03:00
Alexander Tokmakov
fc3e11a068
fix
2021-03-22 17:09:38 +03:00
kssenii
8a650c4408
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge
2021-03-22 13:38:25 +00:00
Maksim Kita
8be263062d
Merge pull request #21910 from kitaisreal/table-function-dictionary
...
Added table function dictionary
2021-03-22 16:08:56 +03:00
Nikolai Kochetov
730785f4c2
Merge pull request #21459 from ClickHouse/refactor-actions-dag
...
Refactor actions dag
2021-03-22 11:58:16 +03:00
Maksim Kita
849a980644
Updated ExternalDictionariesLoader interface with context parameter
2021-03-20 18:02:09 +03:00
Ivan Lezhankin
957c053f7e
Fix segfault
2021-03-19 23:29:01 +03:00
Alexander Kuzmenkov
590259a7a5
disable optimize_read_in_order for window functions
2021-03-19 17:29:09 +03:00
Alexander Tokmakov
1845df25f3
fix possibly dangling reference to Context
2021-03-19 17:02:48 +03:00
Maksim Kita
32831e37ba
Added table function dictionary
2021-03-19 15:47:27 +03:00
Nikolai Kochetov
bc63d4244b
Merge pull request #21856 from ClickHouse/aggregator-no-more-keys-batch
...
Batch aggregation for overflow rows.
2021-03-19 11:52:55 +03:00
Maksim Kita
f340e71913
Merge pull request #21859 from kitaisreal/ddl-dictionary-current-database-name
...
DDL dictionary use current database name
2021-03-19 11:39:51 +03:00
Alexander Kuzmenkov
4bf2e94fa4
clang is too smart
2021-03-19 03:02:35 +03:00
Alexander Kuzmenkov
1beba597ca
Merge remote-tracking branch 'origin/master' into HEAD
2021-03-19 02:11:08 +03:00
Alexander Kuzmenkov
6aa9039f7d
float frames and lag/lead_in_frame
2021-03-19 02:05:43 +03:00
Anton Popov
6f7c8894e9
fix bugs in aggregation by primary key
2021-03-18 23:57:42 +03:00
Alexander Kuzmenkov
c821c0c27a
Merge pull request #21761 from ClickHouse/aku/chunk-no-columns
...
fix incorrect number of rows for Chunks with no columns in PartialSor…
2021-03-18 17:58:09 +03:00
Maksim Kita
a65bfaeab3
Fixed tests
2021-03-18 17:03:22 +03:00
Nikolai Kochetov
07c5a7766c
Fxi test.
2021-03-18 16:23:09 +03:00
Alexander Tokmakov
343b8c906d
Merge branch 'master' into test_multiple_nodes
2021-03-18 16:01:04 +03:00
Anton Popov
33aefa832b
Merge pull request #21818 from amosbird/datarace2
...
Fix arena data race in two level merge
2021-03-18 15:05:25 +03:00
Nikolai Kochetov
f4cbecec27
Remove comment.
2021-03-18 14:37:45 +03:00
Nikolai Kochetov
c3c393a7aa
Merge branch 'master' into refactor-actions-dag
2021-03-18 14:33:07 +03:00
Nikolai Kochetov
66d5968cbc
Batch aggregation for overflow rows.
2021-03-18 12:31:14 +03:00
Nikolai Kochetov
c4dfd97709
Fix ActionsDAG::Index::insert
2021-03-18 10:59:48 +03:00
Pavel Kruglov
bf888b7e5f
Fix deadlock in first catboost model execution
2021-03-17 21:57:40 +03:00
Nikolai Kochetov
45fead9062
Fix typos
2021-03-17 21:42:42 +03:00
Nikolai Kochetov
9fdb0e667f
Add comment.
2021-03-17 21:40:26 +03:00
Nikolai Kochetov
2c1539f641
Fix added input.
2021-03-17 20:08:30 +03:00
Nikolai Kochetov
3eba817a68
Add comments.
2021-03-17 19:08:46 +03:00
Nikolai Kochetov
738cb1af62
Fix filter push down columns order.
2021-03-17 18:33:18 +03:00
Alexander Kuzmenkov
1cd9f28bd4
another way
2021-03-17 18:31:30 +03:00
Alexander Tokmakov
91068b782b
fix
2021-03-17 17:29:24 +03:00
kssenii
34dea63248
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge
2021-03-17 13:32:43 +00:00
Nikita Mikhaylov
2303f1788c
Merge pull request #21807 from 3ldar-nasyrov/master
...
DDLWorker.cpp: fixed exceeded amount of tries typo
2021-03-17 16:04:59 +03:00
fuqi
5e20ea2c33
optimize select final with prewhere primary key
2021-03-17 18:49:24 +08:00
kssenii
9662a492c9
Proper bridge start
2021-03-17 08:21:04 +00:00
alexey-milovidov
d02726bcac
Merge pull request #9404 from Enmk/DateTime64_extended_range
...
Date time64 extended range
2021-03-17 11:06:03 +03:00
Amos Bird
0640bb4cba
Fix arena data race in two level merge
2021-03-17 14:33:57 +08:00
tavplubix
7ae5e51ee9
Update InterpreterAlterQuery.cpp
2021-03-17 01:50:37 +03:00
Alexander Tokmakov
69cbb99f2c
fix
2021-03-17 01:01:48 +03:00
3ldar-nasyrov
37a17749ea
fixed exceeded amount of tries typo
...
'retires' -> 'retries'
2021-03-16 23:17:07 +03:00
Alexander Tokmakov
2d34bbb4b2
Merge branch 'master' into test_multiple_nodes
2021-03-16 21:29:26 +03:00
Anton Popov
6247cd5503
Merge remote-tracking branch 'upstream/master' into HEAD
2021-03-16 16:50:15 +03:00
Nikita Mikhaylov
7d7a2bc724
Merge pull request #21630 from nikitamikhaylov/pf-local
...
Parallel formatting for clickhouse-local
2021-03-16 14:37:40 +03:00
Alexey Milovidov
ea049b2cc0
Merge branch 'master' into DateTime64_extended_range
2021-03-16 14:08:02 +03:00
Pavel Kovalenko
39f3b17041
Merge remote-tracking branch 'origin/master' into unfreeze-partitions
2021-03-16 10:56:24 +03:00
Anton Popov
173d2ea1f4
Merge remote-tracking branch 'upstream/master' into HEAD
2021-03-16 02:50:14 +03:00
Anton Popov
840417c957
Merge pull request #21685 from Avogar/reload-cluster-config
...
Update clusters only if their configs were updated
2021-03-15 20:09:58 +03:00
Alexey Milovidov
eaeb4a3bf0
Merge branch 'master' into DateTime64_extended_range
2021-03-15 16:59:32 +03:00
Vitaly Baranov
6153f52811
Merge pull request #21641 from vitlibar/allowed-grantees
...
Add new clause GRANTEES for CREATE/ALTER USER
2021-03-15 12:47:22 +03:00
tavplubix
6a265f468e
Merge branch 'master' into distributed_ddl_improvements
2021-03-15 11:24:14 +03:00
Vitaly Baranov
37ce6e26d3
Add a new clause GRANTEES to commands CREATE USER and ALTER USER.
2021-03-14 19:12:42 +03:00
Vitaly Baranov
d6e0342c30
Improvements in implementations of the classes AccessRights and GrantedRoles.
2021-03-14 19:12:35 +03:00
alexey-milovidov
53e21443eb
Merge branch 'master' into DateTime64_extended_range
2021-03-14 12:27:17 +03:00
Pavel Kruglov
ee483d89e5
Fix style
2021-03-13 23:18:49 +03:00
alesapin
06eb2d8dfd
Merge branch 'master' into s3_zero_copy_replication
2021-03-13 22:32:54 +03:00
alexey-milovidov
36935a335f
Merge pull request #21102 from Enmk/governance/query_log
...
Added Grant, Revoke and System query_kind for system.query_log
2021-03-13 15:16:32 +03:00
Anton Popov
ed42437219
Merge remote-tracking branch 'upstream/master' into HEAD
2021-03-13 01:41:26 +03:00
Pavel Kruglov
58e415c7d4
Update clusters only if their configs were updated
2021-03-13 00:17:19 +03:00
Vladimir
b2a6a8feb8
Merge pull request #21658 from vdimir/fix-joins_in_memory
2021-03-12 22:48:04 +03:00
Vasily Nemkov
f4246e7be5
Merge branch 'master' into governance/query_log
2021-03-12 18:31:08 +03:00
Anton Ivashkin
ace628d565
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication
2021-03-12 12:58:32 +03:00
vdimir
0eba776fa6
Fix ambigous column error in joins_in_memory
2021-03-12 12:56:43 +03:00
alexey-milovidov
87368befea
Merge pull request #21646 from azat/storage-join-type-conversion
...
Add type conversion for StorageJoin (previously led to SIGSEGV)
2021-03-12 11:21:56 +03:00
alesapin
b98e27ba82
Merge branch 'master' into s3_zero_copy_replication
2021-03-12 11:07:20 +03:00
hexiaoting
ab2aaa7fe7
Rewriter visitor and add performance test
2021-03-12 15:29:38 +08:00
Nikolai Kochetov
b3ea25116f
Fix style
2021-03-12 00:33:06 +03:00
Denis Glazachev
290a6d273e
Add Kerberos support for authenticating existing users when accessing over HTTP
2021-03-11 23:41:10 +03:00
Azat Khuzhin
c2372420ee
Add type conversion for StorageJoin (previously led to SIGSEGV)
...
Found with fuzzer under UBsan [1]:
[1]: https://clickhouse-test-reports.s3.yandex.net/21579/61d40c3600ba6a1c6d6c0cf4919a3cdaebb3a31f/fuzzer_ubsan/report.html#fail1
2021-03-11 23:28:32 +03:00
Nikolai Kochetov
1fb81631ad
Fix style.
2021-03-11 21:00:55 +03:00
Nikolai Kochetov
a5a8a70e6f
Remove commented code.
2021-03-11 20:03:39 +03:00
Nikolai Kochetov
cc60059b50
Merge pull request #21196 from ClickHouse/improve-performance-group-by-multiple-fixed-keys-2
...
Save packed keys for GROUP BY with multiple fixed size keys
2021-03-11 18:28:14 +03:00
Nikolai Kochetov
78b7137877
Try fix last tests.
2021-03-11 17:13:36 +03:00
Nikita Mikhaylov
38e1cb41cc
done
2021-03-11 17:04:59 +03:00
Nikolai Kochetov
7a4dafe106
Fix filter push down columns order.
2021-03-11 13:34:15 +03:00
Nikolai Kochetov
3ee771c600
Remove debug output.
2021-03-10 21:50:34 +03:00
Nikolai Kochetov
7b341c3749
Remove debug output.
2021-03-10 19:05:37 +03:00
Nikolai Kochetov
3a050286f7
Try fix other tests.
2021-03-10 18:12:34 +03:00
Nikolai Kochetov
7eb6b87415
Added comments.
2021-03-10 14:50:51 +03:00
Anton Ivashkin
cef0dda933
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication
2021-03-10 14:10:29 +03:00
Nikolai Kochetov
4c8c45b163
Refactor
2021-03-10 14:00:24 +03:00
Nikolai Kochetov
5ef5ec4e05
Fix build.
2021-03-10 13:48:08 +03:00
Nikolai Kochetov
4f133ae036
Merge branch 'master' into refactor-actions-dag
2021-03-10 12:40:55 +03:00
Nikolai Kochetov
3dbd351c9e
Fix style
2021-03-10 12:33:23 +03:00
Nikolai Kochetov
5ef9d87790
Fix some other tests.
2021-03-10 11:41:24 +03:00
Vladimir
33dce1296b
Merge pull request #20392 from vdimir/cross-to-inner-join-rewrite
2021-03-10 09:53:55 +03:00
alexey-milovidov
2159fb8481
Merge branch 'master' into governance/query_log
2021-03-10 05:29:56 +03:00
Nikolai Kochetov
f3cfc67fa3
Fix other tests.
2021-03-10 00:41:10 +03:00
