Kseniia Sumarokova
d4aa96e262
Merge branch 'master' into better-local-object-storage
2023-04-28 21:05:05 +02:00
Mike Kot
2d20658885
Merge branch 'master' into feature/show-secrets-option
2023-04-28 20:19:35 +03:00
Kseniia Sumarokova
a04025623c
Update FileCache.cpp
2023-04-28 16:47:15 +02:00
Kseniia Sumarokova
b2aa2aa244
Fix
2023-04-28 16:45:00 +02:00
Alexey Milovidov
911d4e9977
Merge branch 'master' into fix_some_tests3
2023-04-28 17:23:46 +03:00
Mike Kot
f1d0ed367d
separate name for format setting
2023-04-28 13:39:38 +00:00
Mike Kot
7bffa89e96
add all 3 settings
2023-04-28 13:22:36 +00:00
Alexey Milovidov
6dcfdbfaf2
Merge branch 'master' into clusters_is_active_column
2023-04-28 16:16:25 +03:00
kssenii
298ed454ce
Merge remote-tracking branch 'upstream/master' into better-local-object-storage
2023-04-28 13:26:55 +02:00
Mike Kot
e428af0b63
replace format settings with server setting
2023-04-27 21:27:18 +00:00
Mike Kot
b24de29c70
reverting mr changes
2023-04-27 20:37:12 +00:00
Mike Kot
357cde95e9
Merge branch 'master' into feature/show-secrets-option
2023-04-27 20:19:40 +00:00
kssenii
f7404d7e72
Fix
2023-04-27 21:40:55 +02:00
Nikolay Degterinsky
df1e56b985
Merge remote-tracking branch 'upstream/master' into bcrypt
2023-04-27 17:01:58 +00:00
Kseniia Sumarokova
45e2d296f9
Merge branch 'master' into better-local-object-storage
2023-04-27 14:54:04 +02:00
Alexey Milovidov
3fa19c5ee2
Merge branch 'master' into fix_some_tests3
2023-04-27 15:04:15 +03:00
kssenii
641b64c4e8
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-27 13:31:41 +02:00
Kseniia Sumarokova
c72bd3eeef
Merge pull request #49260 from ClickHouse/kssenii-patch-7
...
Add a comment in FileCache.cpp
2023-04-27 13:29:19 +02:00
Nikolay Degterinsky
ea55222131
Merge pull request #44674 from evillique/add_default_password_type
...
Add default password type
2023-04-27 13:25:58 +02:00
kssenii
fab84345fd
Corrections after merge with master
2023-04-27 13:18:32 +02:00
kssenii
3d4413183f
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-27 13:16:36 +02:00
Kseniia Sumarokova
9325787884
Update FileCache.cpp
2023-04-27 13:03:41 +02:00
Kseniia Sumarokova
28dee371e0
Merge pull request #49203 from kssenii/fix-system-error-from-cache
...
Catch exceptions from create_directories in fs cache
2023-04-27 12:29:38 +02:00
Nikolay Degterinsky
527b136ddf
Merge pull request #48806 from ucasfl/in-operator
...
IN operator support compare Date and Date32
2023-04-27 01:17:05 +02:00
Kseniia Sumarokova
5a01673118
Merge branch 'master' into fix-after-49110
2023-04-26 22:19:13 +02:00
kssenii
fd432a3ee4
Fix
2023-04-26 17:54:10 +02:00
kssenii
8ba9ab67c2
Fix
2023-04-26 17:13:15 +02:00
kssenii
3939498ef7
Review fixes
2023-04-26 15:43:09 +02:00
Alexey Milovidov
d151e03b04
Merge branch 'master' into rs/show-indexes
2023-04-26 13:55:35 +03:00
Nikolay Degterinsky
a8d124ac0f
Merge branch 'master' into add_default_password_type
2023-04-26 12:53:23 +02:00
Alexey Milovidov
815767a064
Merge branch 'master' into vdimir/mutliple_storage_join
2023-04-26 13:44:22 +03:00
Alexey Milovidov
2a42bcd4ae
Merge pull request #49120 from CurtizJ/add-field-async-insert-log
...
Add field with number of rows to `system.asynchronous_insert_log`
2023-04-26 13:43:09 +03:00
Alexey Milovidov
4dceee0dc3
Merge branch 'master' into rs/qc-totals-extremes2
2023-04-26 13:40:12 +03:00
Robert Schulze
c58e0e347c
Various cleanups
2023-04-26 09:46:35 +00:00
Robert Schulze
403396d0bd
Fix style
2023-04-25 21:04:55 +00:00
Robert Schulze
c4ab1e12f1
Initial version of SHOW INDEXES
2023-04-25 20:48:11 +00:00
kssenii
cbf0b98102
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-25 15:15:22 +02:00
vdimir
7731ea7905
Fix key not found error for queries with multiple StorageJoin
2023-04-25 11:28:54 +00:00
Kseniia Sumarokova
81a53ab5f8
Merge branch 'master' into fix-assertions-from-cache
2023-04-25 11:42:15 +02:00
Anton Popov
cc5acfbe62
add profile event
2023-04-25 01:37:37 +00:00
Anton Popov
c7db58be66
add MergeTree setting async_insert
2023-04-25 01:11:58 +00:00
Anton Popov
aa81937eee
add field with number of rows to async insert log
2023-04-25 00:31:55 +00:00
Robert Schulze
b565257a39
Support totals and extremes in query cache
2023-04-24 22:23:27 +00:00
Alexey Milovidov
dee0b78303
Merge pull request #48745 from Algunenano/arenita
...
Don't count unused arena bytes as read_bytes in AggregateFunction types
2023-04-25 00:53:11 +03:00
Nikolay Degterinsky
2fe3a69ba1
Merge remote-tracking branch 'upstream/master' into add_default_password_type
2023-04-24 18:51:32 +00:00
Nikolay Degterinsky
c90d5ffed8
Fix style
2023-04-24 15:27:24 +00:00
Nikolay Degterinsky
12ad5b710a
Separate AuthenticationData and AuthenticationType, small fixes
2023-04-24 15:12:45 +00:00
pufit
1985fb785e
Merge pull request #48262 from ClickHouse/42497-inherit-grant
...
GRANT CURRENT GRANTS implementation
2023-04-24 11:11:10 -04:00
kssenii
4048756bdb
Fix
2023-04-24 16:50:07 +02:00
Alexander Tokmakov
19f43ee7c7
Merge branch 'master' into fix_some_tests3
2023-04-24 16:13:27 +02:00
Alexander Tokmakov
332a736a59
Merge branch 'master' into clusters_is_active_column
2023-04-24 15:54:03 +02:00
Robert Schulze
ded8eca041
Merge pull request #48284 from ClickHouse/rs/qc-quota
...
Query Cache: Allow per-user quotas
2023-04-24 11:32:32 +02:00
Raúl Marín
f0e045bb3d
Merge remote-tracking branch 'blessed/master' into arenita
2023-04-24 10:42:56 +02:00
Alexey Milovidov
76230be7c8
Merge pull request #49061 from ClickHouse/parameter-exception-context
...
Add the context to the exception message about query paramters
2023-04-23 21:25:01 +03:00
Robert Schulze
38d5f12a32
Query Cache: Implement per-user quotas
2023-04-23 16:13:51 +00:00
Robert Schulze
7fe0562de5
Make the cfg parameter naming more consistent
2023-04-23 14:00:22 +00:00
Alexey Milovidov
e2bb8467d5
Merge pull request #48961 from evillique/default_shard_replica
...
Use default `{replica}`, `{shard}` arguments in Replicated engine
2023-04-23 16:09:57 +03:00
Alexey Milovidov
4f6e8b0b3c
Merge pull request #48999 from liuneng1994/fix_aggregate_empty_string_key
...
Fix aggregate empty string error
2023-04-23 16:08:44 +03:00
kssenii
d8023806a9
Merge remote-tracking branch 'upstream/master' into better-local-object-storage
2023-04-23 12:39:34 +02:00
kssenii
ef4c4e0192
Fix incorrectly resolved conflicts after merge with master
2023-04-23 12:37:04 +02:00
Kseniia Sumarokova
be0cb808de
Merge pull request #48973 from kssenii/randomize-fs-read-settings
...
Randomize prefetch settings in ci
2023-04-23 12:04:30 +02:00
Alexey Milovidov
2ae9dad9b8
Add the context to the exception message about query paramters
2023-04-23 09:34:12 +02:00
kssenii
1fbaaf0445
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-22 17:14:50 +02:00
Alexey Milovidov
48c2884ef0
Update Aggregator.h
2023-04-22 11:55:32 +08:00
liuneng
eea363a8c3
fix aggregate empty string error
2023-04-22 11:55:32 +08:00
Nikolay Degterinsky
61f217be1d
Better solution, add test
2023-04-22 01:55:18 +00:00
Nikolay Degterinsky
b599d0bd01
Better
2023-04-21 22:03:56 +00:00
Dmitry Novik
e23105b322
Merge remote-tracking branch 'origin/master' into revert-47476-revert_46622
2023-04-21 16:40:04 +00:00
Alexey Milovidov
bd64a4a6fc
Merge pull request #48904 from ClickHouse/fix_set_withzk_and_analyzer
...
Fix segfault when set is not built yet
2023-04-21 15:14:23 +03:00
Kruglov Pavel
032b39e50b
Merge pull request #48976 from Avogar/fix-unknown-identifier-row-policy
...
Fix UNKNOWN_IDENTIFIER error while select from table with row policy and column with dots
2023-04-21 13:28:46 +02:00
kssenii
e58c16318f
Merge remote-tracking branch 'upstream/master' into better-local-object-storage
2023-04-21 12:49:40 +02:00
pufit
544972bcda
Solve PR issues
2023-04-20 18:31:35 -04:00
kssenii
0a34f3bcea
Merge remote-tracking branch 'upstream/master' into randomize-fs-read-settings
2023-04-20 19:04:01 +02:00
kssenii
6552ec6afc
Small improvement
2023-04-20 18:15:09 +02:00
avogar
3b72a6821c
Fix UNKNOWN_IDENTIFIER error while select from table with row policy and column with dots
2023-04-20 12:28:39 +00:00
Vladimir C
99ac321f5d
Merge pull request #48664 from ClickHouse/vdimir/http_tmp_buffer_fs_cache
2023-04-20 14:15:30 +02:00
Antonio Andelic
09156ab2b7
Merge branch 'master' into keeper-check-not-exists
2023-04-20 12:07:16 +00:00
Han Fei
6805edf9e6
Merge pull request #48850 from sichenzhao/add_alias_for_parts_name
...
add aliases for parts name and names
2023-04-20 13:04:11 +02:00
Nikolay Degterinsky
75e283323f
Use default parameters in Replicated engine
2023-04-19 20:03:02 +00:00
kssenii
fef179a9ea
Remove redundant code
2023-04-19 19:23:50 +02:00
kssenii
69de3a3c2d
Fix tsan in test test_temporary_data_in_cache
2023-04-19 17:48:56 +02:00
Vladimir C
b31f696aae
Fix iteration in Context::getClusters
2023-04-19 16:44:54 +02:00
vdimir
b1f0e0d62f
Lock clusters_mutex for cluster_discovery
2023-04-19 16:44:54 +02:00
vdimir
d6416a1321
Store clusters from ClusterDiscovery in separate map
2023-04-19 16:44:54 +02:00
Vladimir C
61635eb2d3
Update TemporaryDataOnDisk::createRawStream
2023-04-19 16:44:21 +02:00
vdimir
92d0d9d4ff
Http temporary buffer integration with fs cache
2023-04-19 16:44:21 +02:00
Antonio Andelic
3f00d46785
Update enum for ZooKeeperLog
2023-04-19 14:07:38 +02:00
Alexander Gololobov
2a829d9838
Check getData() result in other places
2023-04-18 17:11:19 +02:00
kssenii
67d695580a
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-18 12:42:25 +02:00
kssenii
c34f277fe7
Fix conflicts after merge
2023-04-18 00:00:38 +02:00
Alexander Gololobov
ba5ca15c40
Merge pull request #46835 from ClickHouse/reduce_mem_in_mutation_with_subquery
...
Reduce memory consumption by mutations with big subqueries used with IN
2023-04-17 23:55:08 +02:00
kssenii
cd25d61795
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-17 19:47:43 +02:00
kssenii
1322ad53b8
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-17 18:22:21 +02:00
kssenii
53181c938a
Fixes
2023-04-17 16:32:08 +02:00
Sichen Zhao
eb4c26d81f
add aliases for parts name and names
2023-04-17 13:18:40 +00:00
Raúl Marín
db71bbb08d
Fix examples build
2023-04-17 10:59:40 +02:00
Raúl Marín
39f8c43a60
Merge remote-tracking branch 'blessed/master' into arenita
2023-04-17 10:33:38 +02:00
Michael Kolupaev
473f212c82
Hopefully fix assertion failure in CachedOnDiskReadBufferFromFile
2023-04-17 04:58:32 +00:00
kssenii
0517de58d1
Simplify FileCacheFactory
2023-04-15 16:15:20 +02:00
flynn
203df96d5c
IN operator support compare Date and Date32
2023-04-15 12:41:42 +00:00
Alexander Gololobov
018f768e7e
Cleanups
2023-04-14 20:07:37 +02:00
Alexander Tokmakov
70c1000c22
fix some tests
2023-04-14 19:42:17 +02:00
Vitaly Baranov
567111f146
Merge pull request #48683 from azat/table-deps-fix
...
Fix table dependencies in case of failed RENAME TABLE
2023-04-14 17:22:14 +02:00
Dmitry Novik
cf5d9a175a
Revert "Merge pull request #48760 from ClickHouse/revert-46089-background-memory-tracker"
...
This reverts commit a61ed33223
, reversing
changes made to 5f01b8a2b5
.
2023-04-14 16:34:19 +02:00
Alexey Milovidov
84cb2ff879
Added setting "use_index_for_in_with_subqueries_max_values" #2571
2023-04-14 16:12:18 +02:00
Alexander Gololobov
28d0cc974a
Cleanups
2023-04-14 16:12:18 +02:00
Alexander Gololobov
0ae74ddf42
Use different types for referencing local set and set that is stored in cache shared between tasks
2023-04-14 16:12:18 +02:00
Alexander Gololobov
05a9d1cbfc
Cleanups
2023-04-14 16:12:18 +02:00
Alexander Gololobov
0aa7979dd5
Unblock tests with ENGINE=Set
2023-04-14 16:12:18 +02:00
Alexander Gololobov
ae2da38549
Check that the set has been built before it is used
2023-04-14 16:12:18 +02:00
Alexander Gololobov
fbf09a1115
Fixes for new analyzer
2023-04-14 16:12:18 +02:00
Alexander Gololobov
adbe87e9d3
Fixes
2023-04-14 16:12:18 +02:00
Alexander Gololobov
0b9579bfb8
Reworked, testing
2023-04-14 16:12:18 +02:00
Alexander Gololobov
02876ed87d
Share PreparedSet-s between task of the same mutation to reduce mem usage
2023-04-14 16:12:18 +02:00
Dmitry Novik
352a240a20
Merge pull request #47984 from ClickHouse/optimize-compilation
...
Optimize compilation
2023-04-14 15:34:46 +02:00
kssenii
ed2a5d25e5
Fix
2023-04-14 15:26:25 +02:00
kssenii
f7232ef537
Fix
2023-04-14 12:46:36 +02:00
Robert Schulze
2378df0a47
Merge pull request #48763 from ClickHouse/rs/fix-squashing
...
Fix squashing in query cache
2023-04-14 11:44:58 +02:00
Robert Schulze
a09db7f4f5
Fix squashing in query cache
2023-04-13 20:09:29 +00:00
Robert Schulze
43b8cbc50c
Merge pull request #48737 from ClickHouse/rs/fix-wshadow
...
Remove `-Wshadow` suppression which leaked into global namespace
2023-04-13 21:18:50 +02:00
Alexander Tokmakov
af1bf08663
Revert "Add MemoryTracker for the background tasks"
2023-04-13 21:05:02 +03:00
Anton Popov
5f01b8a2b5
Merge pull request #48517 from CurtizJ/fix-memory-compressed-tables
...
Fix storage `Memory` with enabled compression
2023-04-13 19:52:49 +02:00
pufit
abe3c9b9db
Merge pull request #48758 from ClickHouse/revert-48679-scheme-infer
...
Revert "Make Schema inference works for CREATE AS SELECT"
2023-04-13 13:46:30 -04:00
Anton Popov
4a1c868ca9
Merge pull request #48663 from CurtizJ/fix-async-insert-no-data
...
Fix async inserts with empty data
2023-04-13 19:44:47 +02:00
pufit
d717372c6e
Revert "Make Schema inference works for CREATE AS SELECT"
2023-04-13 13:35:32 -04:00
pufit
a8c892f925
Merge pull request #48679 from ucasfl/scheme-infer
...
Make Schema inference works for CREATE AS SELECT
2023-04-13 13:02:04 -04:00
kssenii
599a02ca0d
Fix
2023-04-13 18:53:16 +02:00
kssenii
461477dabf
Fix style check
2023-04-13 16:38:38 +02:00
Dmitry Novik
5cc9b46f78
Merge remote-tracking branch 'origin/master' into optimize-compilation
2023-04-13 16:04:09 +02:00
kssenii
ce723ec32d
Fix style check, better priority->iterate
2023-04-13 14:45:28 +02:00
kssenii
58a30213c9
Fix after merge
2023-04-13 13:34:19 +02:00
Dmitry Novik
467ecf45e1
Merge pull request #46089 from ClickHouse/background-memory-tracker
...
Add MemoryTracker for the background tasks
2023-04-13 13:29:27 +02:00
kssenii
ce45105448
Cleanup a bit
2023-04-13 13:27:01 +02:00
kssenii
09c23287aa
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-13 13:25:45 +02:00
Raúl Marín
2b70e08f23
Don't count unreserved bytes in Arenas as read_bytes
2023-04-13 12:43:24 +02:00
Robert Schulze
7a21d5888c
Remove -Wshadow suppression which leaked into global namespace
2023-04-13 08:46:40 +00:00
Raúl Marín
da9a539cf7
Reduce the usage of Arena.h
2023-04-13 10:31:32 +02:00
pufit
aa0f3bc994
Solve pr issues
2023-04-12 20:28:17 -04:00
kssenii
b773b506b2
Better
2023-04-12 23:27:06 +02:00
Anton Popov
05a93ef123
add flush id and time for async insert with empty data
2023-04-12 20:57:55 +00:00
Alexey Milovidov
9a3030803e
Merge pull request #48596 from azat/ci/second_deadlock_stack
...
Add second_deadlock_stack=1 for TSan on CI and fix some lock-order-inversion problems
2023-04-12 22:11:06 +03:00
Anton Popov
08aa629e56
Merge branch 'master' into fix-memory-compressed-tables
2023-04-12 15:38:54 +02:00
Nikolai Kochetov
f66e9c5a26
Merge pull request #48427 from ClickHouse/update_sort_desc_follow_up
...
Update sort desc: more efficient original node search in ActionsDAG
2023-04-12 13:59:37 +02:00
Azat Khuzhin
5277c802f7
Fix table dependencies in case of failed RENAME TABLE
...
CI found this [1]
[ 382620 ] {} <Error> Application: Caught exception while loading metadata: Code: 60. DB::Exception: Table test_25.join doesn\'t exist: While processing _CAST(joinGet(test_25.join, \'m\', CAST(\'42\', \'int\')) AS m_tmp_alter1351896088559986425, \'Int32\') AS m: default expression and column type are incomp
[ 382620 ] {} <Error> Application: Code: 60. DB::Exception: Table test_25.join doesn\'t exist: While processing _CAST(joinGet(test_25.join, \'m\', CAST(\'42\', \'int\')) AS m_tmp_alter1351896088559986425, \'Int32\') AS m: default expression and column type are incompatible.: Cannot attach table `test_25`.`t
[1]: https://s3.amazonaws.com/clickhouse-test-reports/48670/d7f865037266ed87538cf4df7ec7e8165681871b/stress_test__asan_.html
The problem here is that after failed RENAME dependencies got lost:
[4e937f39d866] 2023.04.12 00:51:06.833624 [ 13419 ] {61429225-64cb-4fce-b60d-01e0dac6e52c} <Error> executeQuery: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 34.65 GiB (attempt to allocate chunk of 2097419 bytes), maximum: 34.29 GiB. OvercommitTracker decision: Memory overcommit has freed not enough memory. (MEMORY_LIMIT_EXCEEDED) (version 23.4.1.1) (from [::1]:45710) (comment: 01160_table_dependencies.sh) (in query: create database test_25_1), Stack trace (when copying this message, always include the lines below):
[4e937f39d866] 2023.04.12 00:51:07.351914 [ 5151 ] {66d8bdd4-668e-4239-a8af-6b8f17bb5222} <Error> executeQuery: Code: 81. DB::Exception: Database test_25_1 doesn't exist. (UNKNOWN_DATABASE) (version 23.4.1.1) (from [::1]:45762) (comment: 01160_table_dependencies.sh) (in query: rename table t to test_25_1.t), Stack trace (when copying this message, always include the lines below):
And from the test output:
OK
OK
OK
OK
a [] [] []
data_02344 [] [] []
date_table [] [] []
dict1 [] ['dict_src'] ['join']
dict2 [] ['join'] []
dict_src [] [] ['dict1']
dist_02346 [] [] []
join [] ['dict1'] ['dict2','s']
^^^^^^^^^^^^^ no "t"
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-12 11:11:05 +02:00
flynn
ea9187511e
Make Schema inference works for CREATE AS SELECT
2023-04-12 04:21:03 +00:00
pufit
078b8f5399
Small refactor, additional tests
2023-04-11 19:55:59 -04:00
Dmitry Novik
06e6794fc0
Merge remote-tracking branch 'origin/master' into background-memory-tracker
2023-04-11 15:29:35 +00:00
Anton Popov
09c66c3879
fix async inserts with empty data
2023-04-11 15:13:46 +00:00
Mikhail f. Shiryaev
89e4509e68
Merge branch 'master' into fix-memory-compressed-tables
2023-04-11 16:38:51 +02:00
Alexander Tokmakov
a1fda047f6
add docs
2023-04-11 16:32:40 +02:00
Alexander Tokmakov
dfff30e1a9
Merge branch 'master' into clusters_is_active_column
2023-04-11 16:32:15 +02:00
Anton Popov
3896375247
fix getting subcolumn from in-memory data
2023-04-11 14:26:31 +00:00
Nikolay Degterinsky
0621222737
Fix crashes with incorrect query parameters
2023-04-11 14:22:55 +00:00
Nikolay Degterinsky
f34b304707
Fixes, add test
2023-04-11 12:54:54 +00:00
SmitaRKulkarni
e2010c0257
Merge branch 'master' into zk_retry_timeout
2023-04-11 14:05:52 +02:00
Nikolay Degterinsky
7606336fd8
Merge remote-tracking branch 'upstream/master' into add_default_password_type
2023-04-11 11:59:45 +00:00
vdimir
4605d71947
Fix build src/Interpreters/InterpreterInsertQuery.h
2023-04-11 11:17:02 +00:00
Sema Checherinda
4dd86a406a
Merge pull request #48543 from azat/mv-uniq-thread-group
...
Use one ThreadGroup while pushing to materialized views (and some refactoring for ThreadGroup)
2023-04-11 11:47:46 +02:00
Alexey Milovidov
d259217cf3
Merge pull request #48570 from azat/build/logger_useful
...
Remove superfluous includes of logger_userful.h from headers
2023-04-11 03:56:39 +03:00
Nikolay Degterinsky
908be1e324
Fix tests
2023-04-10 22:04:10 +00:00
Alexey Milovidov
f4d6596eaa
Merge pull request #48577 from ClickHouse/evillique-patch-2
...
Don't use type conversion with String query parameters
2023-04-10 22:18:39 +03:00
Azat Khuzhin
013a3ed269
Fix possible lock order inversion in Context::setTemporaryStorageInCache()
...
TSan report [1]:
WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=1)
Cycle in lock order graph: M0 (0x7b9000000008) => M1 (0x7b90000000a8) => M0
Mutex M1 acquired here while holding mutex M0 in main thread:
0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
3 std::__1::lock_guard<std::__1::mutex>::lock_guard[abi:v15000](std::__1::mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:94:27 (clickhouse+0x1d0ccb3a) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
4 DB::Context::getDisk() const build_docker/./src/Interpreters/Context.cpp:3281:21 (clickhouse+0x1d0ccb3a)
5 DB::Context::setTemporaryStorageInCache() build_docker/./src/Interpreters/Context.cpp:927:21 (clickhouse+0x1d0cc188) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
6 DB::Server::main() build_docker/./programs/server/Server.cpp:1055:25 (clickhouse+0x148504c2) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
Mutex M0 previously acquired by the same thread here:
0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
3 std::__1::unique_lock<std::__1::recursive_mutex>::unique_lock[abi:v15000](std::__1::recursive_mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:122:61 (clickhouse+0x1d0cc148) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
4 DB::Context::getLock() const build_docker/./src/Interpreters/Context.cpp:690:12 (clickhouse+0x1d0cc148)
5 DB::Context::setTemporaryStorageInCache() build_docker/./src/Interpreters/Context.cpp:922:17 (clickhouse+0x1d0cc148)
6 DB::Server::main() build_docker/./programs/server/Server.cpp:1055:25 (clickhouse+0x148504c2) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
Muex M0 acquired here while holding mutex M1 in main thread:
0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
3 std::__1::unique_lock<std::__1::recursive_mutex>::unique_lock[abi:v15000](std::__1::recursive_mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:122:61 (clickhouse+0x1d0c7ce1) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
4 DB::Context::getLock() const build_docker/./src/Interpreters/Context.cpp:690:12 (clickhouse+0x1d0c7ce1)
5 DB::Context::getPath() const build_docker/./src/Interpreters/Context.cpp:713:17 (clickhouse+0x1d0c7ce1)
6 DB::loadDiskLocalConfig() build_docker/./src/Disks/DiskLocal.cpp:75:30 (clickhouse+0x1ce7ed8a) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
7 DB::DiskLocal::applyNewSettings() build_docker/./src/Disks/DiskLocal.cpp:491:5 (clickhouse+0x1ce7e7ef) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
8 DB::DiskSelector::updateFromConfig() const build_docker/./src/Disks/DiskSelector.cpp:92:19 (clickhouse+0x1d14088e) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
9 DB::Context::updateStorageConfiguration() build_docker/./src/Interpreters/Context.cpp:3388:49 (clickhouse+0x1d0ec002) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
..
17 DB::ConfigReloader::reloadIfNewer() build_docker/./src/Common/Config/ConfigReloader.cpp:149:13 (clickhouse+0x1fff4250)
Mutex M1 previously acquired by the same thread here:
0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
3 std::__1::lock_guard<std::__1::mutex>::lock_guard[abi:v15000](std::__1::mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:94:27 (clickhouse+0x1d0ebf25) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
4 DB::Context::updateStorageConfiguration() build_docker/./src/Interpreters/Context.cpp:3384:21 (clickhouse+0x1d0ebf25)
..
12 DB::ConfigReloader::reloadIfNewer() build_docker/./src/Common/Config/ConfigReloader.cpp:149:13 (clickhouse+0x1fff4250)
SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/usr/bin/clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280) in pthread_mutex_lock
==================
[1]: https://s3.amazonaws.com/clickhouse-test-reports/48596/a1272e8536265929255fdf5020836f057859e425/integration_tests__tsan__[6/6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 19:56:52 +02:00
Nikolay Degterinsky
def8611d2c
Fix style
2023-04-10 17:05:31 +00:00
Nikolay Degterinsky
6f57a952d9
Improvements
2023-04-10 16:40:49 +00:00
Azat Khuzhin
79b83c4fd2
Remove superfluous includes of logger_userful.h from headers
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 17:59:30 +02:00
Nikolai Kochetov
28484cd762
Merge pull request #48522 from ClickHouse/many-mutations-memory-limit
...
Reduce memory usage for multiple alter delete mutations.
2023-04-10 12:50:42 +02:00
Raúl Marín
45ad555c39
Merge branch 'master' into zk_retry_timeout
2023-04-10 10:04:16 +02:00
Nikolay Degterinsky
8524f6958a
Update src/Interpreters/ReplaceQueryParameterVisitor.cpp
...
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2023-04-10 09:09:57 +02:00
Nikolay Degterinsky
7957752128
Merge remote-tracking branch 'upstream/master' into add_default_password_type
2023-04-10 02:25:46 +00:00
Nikolay Degterinsky
64dc60a8d1
Better version, introduce ASTAuthenticationData
2023-04-10 02:20:57 +00:00
Nikolay Degterinsky
e71c7fe744
Fix
2023-04-10 03:48:58 +02:00
Nikolay Degterinsky
6b1c86c6ec
Don't use type conversion with String query parameters
2023-04-10 03:39:13 +02:00
Vitaly Baranov
62e9c419de
Merge pull request #48519 from azat/build/fwd-decl-threadpool
...
Use forward declaration of ThreadPool
2023-04-08 00:44:01 +02:00
Alexey Milovidov
e09e80b493
Merge pull request #48500 from CurtizJ/fix-sparse-column-query-cache
...
Fix query cache with sparse columns
2023-04-07 19:34:10 +03:00
Alexander Tokmakov
9ec91acef3
add some columns to system.clusters
2023-04-07 18:26:23 +02:00
Azat Khuzhin
44ae8485f1
Use one ThreadGroup while pushing to materialized views
...
Before this patch only the case when ThreadStatus (current_thread)
already has ThreadGroup works that way, after they will be identical in
this aspect.
But this should not affect anything, but it just make sense.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-07 15:37:35 +02:00
Azat Khuzhin
aacf2a0838
Move ThreadGroupSwitcher to ThreadStatus.h (out from MergeTree code)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-07 15:32:08 +02:00
Azat Khuzhin
5b2b20a0b0
Rename ThreadGroupStatus to ThreadGroup
...
There are methods like getThreadGroup() and ThreadGroupSwitcher class,
so seems that this is logical.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-07 15:31:48 +02:00
Sema Checherinda
0503ed6fb8
Merge pull request #46668 from azat/fix-query_views_log-from-bg
...
Fix system.query_views_log for MVs that are pushed from background threads
2023-04-07 14:47:05 +02:00
Azat Khuzhin
011480924a
Use forward declaration of ThreadPool
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-07 11:25:35 +02:00
Yakov Olkhovskiy
8d1924cc9a
Merge pull request #47962 from ClickHouse/imp-use-structure-from-insertion
...
Improvement and fix for use_structure_from_insertion_table_in_table_functions
2023-04-06 19:01:00 -04:00
Nikolai Kochetov
e8fb135180
Reduce memory usage for multiple alter delete mutations.
2023-04-06 19:07:53 +00:00
Anton Popov
8d1b535efb
fix storage Memory with compression
2023-04-06 18:27:53 +00:00
Sema Checherinda
9d8d9e8ec9
Merge branch 'master' into fix-query_views_log-from-bg
2023-04-06 18:16:47 +02:00
Anton Popov
4757d76fbf
fix query cache with sparse columns
2023-04-06 16:01:23 +00:00
Anton Popov
b5a3723c88
Merge pull request #48395 from ClickHouse/with_fill_clarification
...
WITH FILL clarification and cleanup
2023-04-06 17:02:57 +02:00
pufit
d1fa9596b8
Prevent execution on cluster
2023-04-06 10:56:35 -04:00
Sema Checherinda
b8729be97a
Merge pull request #47104 from CheSema/thead-group-for-merge
...
merge and mutation make thread group for setting memory trackers right
2023-04-06 15:46:56 +02:00
Vitaly Baranov
b45b662fe3
Merge pull request #48431 from Algunenano/system_referential_table
...
Don't check dependencies when renaming system tables automatically
2023-04-06 15:28:37 +02:00
Nikolai Kochetov
6c6335b686
Merge pull request #48357 from ClickHouse/fix-projections-analysis-with-analyzer
...
Batch fix for projections analysis with analyzer.
2023-04-06 15:13:56 +02:00
Yakov Olkhovskiy
d2e897d4b8
Merge branch 'master' into imp-use-structure-from-insertion
2023-04-06 08:52:38 -04:00
Antonio Andelic
28a33f0a47
Merge pull request #48435 from ClickHouse/kv-update-only-affected-rows
...
Update only affected rows in KV storage
2023-04-06 12:33:54 +02:00
Vladimir C
e7a5c96353
Merge pull request #48219 from ClickHouse/vdimir/lock_order_47072
2023-04-06 11:54:33 +02:00
Vladimir C
a2b71a0054
Merge pull request #48386 from ClickHouse/vdimir/reset-downloader-tmp-data-cache
2023-04-06 11:23:16 +02:00
Yakov Olkhovskiy
8a19ac4e73
Merge branch 'master' into imp-use-structure-from-insertion
2023-04-06 00:51:44 -04:00
pufit
144ee7aed4
Fix logic, additional tests
2023-04-05 22:52:31 -04:00
robot-ch-test-poll3
3ad0a6ac18
Merge pull request #48433 from ClickHouse/some-fixes-for-parallel-replicas
...
Some fixes for parallel replicas
2023-04-06 02:00:06 +02:00
Alexander Tokmakov
a154fa73f4
Merge pull request #48401 from ClickHouse/fix_00002_log_messages
...
Find big allocations without memory limits checks
2023-04-05 23:52:39 +03:00
Sema Checherinda
554c928534
Merge branch 'master' into thead-group-for-merge
2023-04-05 21:09:38 +02:00
Raúl Marín
1fa992f015
Merge branch 'master' into system_referential_table
2023-04-05 20:47:53 +02:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling
...
Per-query/per-server throttling for remote IO/local IO/BACKUPs
2023-04-05 19:32:37 +02:00
kssenii
a405d06e8a
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-05 18:46:52 +02:00
Antonio Andelic
0260b84bc3
Update only affected rows in KV storage
2023-04-05 16:12:14 +00:00
Raúl Marín
5d9e006c46
PR comments
2023-04-05 18:04:03 +02:00
Igor Nikonov
46f7016d8c
Add search alias by input name in ActionsDAG
2023-04-05 15:56:06 +00:00
Nikita Mikhaylov
3f44c6d936
Better
2023-04-05 15:19:45 +00:00
Raúl Marín
d65d00f496
Don't check dependencies when renaming system tables automatically
2023-04-05 16:15:24 +02:00
vdimir
d9abcf2f69
Reduce scope storage_policies_mutex in setTemporaryStoragePolicy
2023-04-05 13:42:49 +00:00
vdimir
01ce3baf0f
Rename Context::getTemporaryVolume to getGlobalTemporaryVolume
2023-04-05 13:42:13 +00:00
Igor Nikonov
41f6e0e7b5
More efficient original node search in ActionsDAG
2023-04-05 13:36:07 +00:00
vdimir
6f3fa33641
Add unbound flag to system.filesystem_cache
2023-04-05 13:12:20 +00:00
Nikita Mikhaylov
7b526d3df9
Always communicate with local replica through loopback interface for parallel replicas ( #48246 )
2023-04-05 13:32:46 +02:00
Sema Checherinda
f504cd5fed
avoid race, do not set thread_group.counters.parent from query
2023-04-05 12:36:55 +02:00
Igor Nikonov
47e7e19ca9
Move suffix generation code into one function
2023-04-05 10:12:42 +00:00
vdimir
bae1286b63
Update temp_data_on_disk in shared context
2023-04-05 10:07:44 +00:00
vdimir
7fe06f8435
Do not take lock for shared context in setTempDataOnDisk
2023-04-05 09:22:09 +00:00
Azat Khuzhin
4008155a6e
Add max_ prefix for backup_bandwidth settings
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 10:54:25 +02:00
Azat Khuzhin
61405b827d
Use MAKE_DEPRECATED_BY_SERVER_CONFIG() for deprecated throttle settings
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:40:18 +02:00
Azat Khuzhin
684ecc24e2
Ignore deprecated values max_remote_{read,write}_network_bandwidth_for_server
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:40:18 +02:00
Azat Khuzhin
ad96d3c249
Do not require proper Poco Application for Context
...
$ yes | head | clickhouse-obfuscator --structure 'id String' --seed foo --input-format TSV --output-format TSV
Poco::Exception. Code: 1000, e.code() = 0, Null pointer: _pInstance in file "/src/ch/clickhouse/base/poco/Util/include/Poco/Util/Application.h", line 446, Stack trace (when copying this message, always include the lines below):
0. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x15605622 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
1. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x156055ed in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
2. /src/ch/clickhouse/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x2ebd2d80 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
3. /src/ch/clickhouse/base/poco/Foundation/src/Exception.cpp:132: Poco::LogicException::LogicException(String const&, int) @ 0x2ebd3667 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
4. /src/ch/clickhouse/base/poco/Foundation/src/Exception.cpp:134: Poco::NullPointerException::NullPointerException(String const&, int) @ 0x2ebd3da7 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
5. /src/ch/clickhouse/base/poco/Foundation/src/Bugcheck.cpp:42: Poco::Bugcheck::nullPointer(char const*, char const*, int) @ 0x2ebc5851 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
6. /src/ch/clickhouse/base/poco/Util/include/Poco/Util/Application.h:446: Poco::Util::Application::instance() @ 0x1ee3e6cd in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
7. /src/ch/clickhouse/src/Interpreters/Context.cpp:358: DB::ContextSharedPart::ContextSharedPart() @ 0x263b0cca in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
8. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if<DB::ContextSharedPart>::__unique_single std::make_unique[abi:v15000]<DB::ContextSharedPart>() @ 0x26387b6e in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
9. /src/ch/clickhouse/src/Interpreters/Context.cpp:652: DB::Context::createShared() @ 0x26361a3c in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
10. /src/ch/clickhouse/programs/obfuscator/Obfuscator.cpp:1293: mainEntryClickHouseObfuscator(int, char**) @ 0x1ef6eb26 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
11. /src/ch/clickhouse/programs/main.cpp:481: main @ 0x155e569a in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
12. ? @ 0x7ffff7dd2790 in ?
13. __libc_start_main @ 0x7ffff7dd284a in ?
14. _start @ 0x155e51ae in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
(version 23.3.1.2537)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:40:18 +02:00
Azat Khuzhin
a8c1407bc4
Use existing ServerSettings in Context instead of reading it again
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:40:18 +02:00
Azat Khuzhin
c332d290d8
Keep only one throttler for BACKUPs IO (instead of separate read/write)
...
There is no need in separate read/write throttling, because you cannot
write faster then read anyway, and plus this makes the code less cleaner
(and also it will allow avoid implementing throttling backups to S3,
since it does not use common S3 writer).
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:40:17 +02:00
Azat Khuzhin
218b1f9c29
Add ability to throttle BACKUPs on per-server/backup basis
...
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server
Query settings:
- backup_read_bandwidth
- backup_write_bandwidth
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:48 +02:00
Azat Khuzhin
a25dd1d348
Add ability to throttle local IO on per-query/server basis
...
Server settings:
- max_local_read_bandwidth_for_server
- max_local_write_bandwidth_for_server
Query settings:
- max_local_read_bandwidth
- max_local_write_bandwidth
This is the preparation for adding ability to throttle BACKUPs
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:15 +02:00
Azat Khuzhin
b3406beeb7
Add per-query network throttling
...
Controlled with:
- max_remote_read_network_bandwidth
- max_remote_write_network_bandwidth
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:15 +02:00
Azat Khuzhin
abdb682048
Move some methods for Context into module part
...
This will be required later, for per-query bandwidth limiting.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:15 +02:00
Azat Khuzhin
40cb87c2ac
Make remote bandwidth settings server settings
...
Right now those two settings are applied only at start:
- max_remote_read_network_bandwidth_for_server
- max_remote_write_network_bandwidth_for_server
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:15 +02:00
pufit
8441b0a3e1
Fix comments, tests, docs
2023-04-04 20:43:15 -04:00
Alexander Tokmakov
ba65be22da
fix bad error messages
2023-04-04 20:21:22 +02:00
vdimir
5966281f7c
Test WriteBufferToFileSegment with several threads
2023-04-04 18:15:27 +00:00
vdimir
61c8a9586e
Fix usage of is_unbound in FileSegment
2023-04-04 18:14:54 +00:00
Raúl Marín
0e17b3b147
Merge remote-tracking branch 'blessed/master' into zk_retry_timeout
2023-04-04 18:53:55 +02:00
Igor Nikonov
1fb0292fcb
polishing
2023-04-04 16:13:21 +00:00
Igor Nikonov
5a9acac12f
Clarification and cleanup
2023-04-04 16:00:09 +00:00
Azat Khuzhin
9235d1cde2
Fix system.query_views_log for MVs that are pushed from background threads
...
Some of such places:
- push from Buffer
- push from Distributed sends
- system.*_log workers
Before #47564 it simply does not work, but after it throws LOGICAL_ERROR
in such situation.
v2: remove expired() check after #46709 got merged
v3: use ThreadGroupStatus ctor with ContextPtr (after #47564 )
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-04 17:43:10 +02:00
vdimir
50c8472c57
Reset downloader in WriteBufferToFileSegment::nextImpl
2023-04-04 15:39:25 +00:00
vdimir
999a3889d0
Revert "Reset downloader for cache file segment in TemporaryFileStream"
...
This reverts commit 07147771e6
.
2023-04-04 14:22:33 +00:00
vdimir
07147771e6
Reset downloader for cache file segment in TemporaryFileStream
2023-04-04 13:26:24 +00:00
Kruglov Pavel
5f930aeb26
Merge pull request #48115 from save-my-heart/throw_non_parametric_function
...
throw exception while non-parametric functions having parameters
2023-04-04 12:05:34 +02:00
Antonio Andelic
52e4149c98
Merge pull request #47934 from ClickHouse/alternative-keeper-configs
...
Merge #35113
2023-04-04 09:58:33 +02:00
Antonio Andelic
d0ba1e76bf
Merge pull request #48293 from ClickHouse/keeper-map-strict-mode
...
Add strict mode for KeeperMap
2023-04-04 09:57:01 +02:00
Nikolai Kochetov
b327a93cf7
Add comments.
2023-04-03 18:18:39 +00:00
Nikolai Kochetov
61a484f0bd
Batch fix for projections analysis with analyzer.
2023-04-03 16:44:31 +00:00
Yakov Olkhovskiy
e3457da144
for auto check subset of columns support, fix some minor issues
2023-04-03 16:05:21 +00:00
Antonio Andelic
f21c664744
Add error code
2023-04-03 15:17:06 +00:00
Antonio Andelic
f5e97fbb66
Merge branch 'master' into alternative-keeper-configs
2023-04-03 15:06:52 +00:00
Antonio Andelic
e81c2999a3
Update src/Interpreters/InterpreterAlterQuery.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-04-03 16:56:45 +02:00
Antonio Andelic
12bee0573f
Correctly check table
2023-04-03 14:54:14 +00:00
Antonio Andelic
eacc3752bf
Merge branch 'master' into dont-replicate-mutations-keepermap
2023-04-03 14:38:57 +00:00
Antonio Andelic
950a7d9948
Merge branch 'master' into keeper-map-strict-mode
2023-04-03 14:26:14 +00:00