Sema Checherinda
c1adbb7aa6
transfer counters to the query directly
2023-04-05 19:51:36 +02:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling
...
Per-query/per-server throttling for remote IO/local IO/BACKUPs
2023-04-05 19:32:37 +02:00
kssenii
be13ce76f4
Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes
2023-04-05 18:47:08 +02:00
avogar
1be6bceb2f
Generate valid Decimals and Bools in generateRandom function
2023-04-05 16:23:06 +00:00
Antonio Andelic
0260b84bc3
Update only affected rows in KV storage
2023-04-05 16:12:14 +00:00
Nikita Mikhaylov
3f44c6d936
Better
2023-04-05 15:19:45 +00:00
kssenii
62244ee57d
Fix test
2023-04-05 17:14:41 +02:00
vdimir
01ce3baf0f
Rename Context::getTemporaryVolume to getGlobalTemporaryVolume
2023-04-05 13:42:13 +00:00
vdimir
6f3fa33641
Add unbound flag to system.filesystem_cache
2023-04-05 13:12:20 +00:00
Robert Schulze
62608f74d3
Merge pull request #48396 from ClickHouse/rs/clang-tidy-cleanup
...
Cleanup mess in .clang-tidy
2023-04-05 14:39:28 +02:00
Alexander Tokmakov
3b5ed9242c
Merge pull request #48391 from azat/fix-sync-replica-stuck
...
Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION
2023-04-05 15:20:46 +03:00
kssenii
3fb4cd0f52
Fix s3 test
2023-04-05 14:13:46 +02:00
Nikita Mikhaylov
7b526d3df9
Always communicate with local replica through loopback interface for parallel replicas ( #48246 )
2023-04-05 13:32:46 +02:00
Robert Schulze
1be81db885
Fix build, pt. II
2023-04-05 11:23:09 +00:00
Sema Checherinda
f504cd5fed
avoid race, do not set thread_group.counters.parent from query
2023-04-05 12:36:55 +02:00
Azat Khuzhin
218b1f9c29
Add ability to throttle BACKUPs on per-server/backup basis
...
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server
Query settings:
- backup_read_bandwidth
- backup_write_bandwidth
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:48 +02:00
kssenii
a3d69694f4
Fix build
2023-04-04 23:13:17 +02:00
kssenii
c379eb7c29
Fix style check
2023-04-04 22:59:16 +02:00
kssenii
f44c53b97a
Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes
2023-04-04 22:41:22 +02:00
kssenii
8c0be0c3be
Checkpoints
2023-04-04 22:40:05 +02:00
Alexander Tokmakov
ba65be22da
fix bad error messages
2023-04-04 20:21:22 +02:00
Raúl Marín
0e17b3b147
Merge remote-tracking branch 'blessed/master' into zk_retry_timeout
2023-04-04 18:53:55 +02:00
Nikita Mikhaylov
fa5b2bd4a5
Added Keeper retries for backups operations ( #47224 )
2023-04-04 18:12:08 +02:00
Azat Khuzhin
353434f8f1
Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION
...
In case of DROP/REPLACE PARTITION the entries from the queue will be
removed without notifying the subscribes.
CI: https://s3.amazonaws.com/clickhouse-test-reports/48242/4a315cd0d17bc12edd934fd25663b0119880f207/stress_test__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-04 17:16:04 +02:00
Antonio Andelic
a329d80bfa
Merge pull request #47397 from ClickHouse/enable-env-credentials-default
...
Enable `use_environment_credentials` by default
2023-04-04 10:00:03 +02:00
Antonio Andelic
52e4149c98
Merge pull request #47934 from ClickHouse/alternative-keeper-configs
...
Merge #35113
2023-04-04 09:58:33 +02:00
Antonio Andelic
4efa707fe3
Merge branch 'master' into alternative-keeper-configs
2023-04-04 09:53:19 +02:00
Antonio Andelic
0dda3d13ef
Merge branch 'master' into keeper-map-strict-mode
2023-04-04 09:52:53 +02:00
lizhuoyu5
2232d5ad14
Merge branch 'master' into add_columns_for_system_storage_policies
2023-04-04 12:58:10 +08:00
Yakov Olkhovskiy
35e9e45249
Merge pull request #48062 from Algunenano/unnecessary_alter_checks
...
Only check MV on ALTER when necessary
2023-04-03 17:23:11 -04:00
Sema Checherinda
6d5edc0315
rename fake_query_context to task_context
2023-04-03 22:01:18 +02:00
kssenii
75b11bceb5
Fix style check
2023-04-03 21:27:05 +02:00
kssenii
18a9a670c3
Better
2023-04-03 20:56:10 +02:00
kssenii
9b3d0ec86d
Adjustments after conflicts
2023-04-03 19:53:34 +02:00
kssenii
8915f49b7d
Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes
2023-04-03 17:43:42 +02:00
Antonio Andelic
f5e97fbb66
Merge branch 'master' into alternative-keeper-configs
2023-04-03 15:06:52 +00:00
kssenii
6e1cf1907d
Better
2023-04-03 16:57:49 +02:00
Antonio Andelic
950a7d9948
Merge branch 'master' into keeper-map-strict-mode
2023-04-03 14:26:14 +00:00
kssenii
5578cb08ad
Fix s3 cluster
2023-04-03 14:40:04 +02:00
Anton Popov
207a4c6166
Merge branch 'master' into do-not-build-unnecessary-sets
2023-04-03 14:29:33 +02:00
Robert Schulze
a50e741bea
Merge pull request #45912 from ClickHouse/qc-compression
...
Query cache: Enable compression of cache entries
2023-04-03 13:33:47 +02:00
Robert Schulze
3e124f09d0
Merge remote-tracking branch 'origin/master' into qc-compression
2023-04-03 11:28:17 +00:00
Robert Schulze
e79343c169
Make the column order in system.query_cache more intutitive
2023-04-03 10:36:11 +00:00
kssenii
018df220bf
Remove redundant
2023-04-03 12:18:04 +02:00
kssenii
35a8328922
Fix
2023-04-03 12:11:12 +02:00
Antonio Andelic
2d89057142
Merge branch 'master' into keeper-map-strict-mode
2023-04-03 09:30:52 +00:00
Antonio Andelic
40a0ecf66a
Fix
2023-04-03 09:30:39 +00:00
Raúl Marín
8fdf87982c
Merge remote-tracking branch 'blessed/master' into zk_retry_timeout
2023-04-03 10:26:18 +02:00
Antonio Andelic
da194f082d
Merge branch 'master' into enable-env-credentials-default
2023-04-03 09:45:03 +02:00
Alexey Milovidov
99ed3b627b
Merge branch 'master' into fix-race-storage-s3
2023-04-03 06:25:44 +03:00
Anton Popov
ff1cc5598f
fix clang-tidy
2023-04-02 22:21:10 +00:00
Alexey Milovidov
5ebf66819b
Merge pull request #47617 from ClickHouse/cnf-new-analyzer
...
CNF/Constraint optimizer in new analyzer
2023-04-02 20:09:34 +03:00
kssenii
6d75ca9c6f
Fix
2023-04-02 18:21:45 +02:00
Alexey Milovidov
9c6594a043
Merge pull request #48320 from azat/explain-fix
...
Fix crash in EXPLAIN PIPELINE for Merge over Distributed
2023-04-02 17:11:27 +03:00
Azat Khuzhin
907ed27ae2
Fix crash in EXPLAIN PIPELINE for Merge over Distributed
...
CI: https://s3.amazonaws.com/clickhouse-test-reports/48314/179450542879d11711cd2415c3fa7eeab18188be/fuzzer_astfuzzerasan/report.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-02 10:48:23 +02:00
kssenii
ddf2af598d
Fix
2023-04-01 20:52:22 +02:00
kssenii
8cfd544328
Better
2023-04-01 18:41:42 +02:00
Azat Khuzhin
c64f9e6f07
Use StrongTypedef for ProfileEvents
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-01 16:01:33 +02:00
Azat Khuzhin
ba6ecd2d4e
Fix ThreadPool for DistributedSink
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-01 16:00:03 +02:00
Alexander Tokmakov
48b23dd012
Fix race between DROP MatView and RESTART REPLICAS ( #47863 )
...
* fix race between drop mv and restart replicas
* unrelated: fix bad exception messages
* fix test
* fix
* fix
* fix
* fix
* fix test
* fix
* fix test
* Update 02437_drop_mv_restart_replicas.sh
* fix tests
2023-04-01 15:26:00 +03:00
kssenii
6ff728fb4d
Fix cpu usage in rabbitmq
2023-04-01 11:45:25 +02:00
Anton Popov
cdf9cf2fd4
do not build sets for indexes if they are not used
2023-03-31 16:06:20 +00:00
Anton Popov
3b98a43dc6
Merge branch 'master' into fix-race-storage-s3
2023-03-31 17:54:52 +02:00
Alexander Tokmakov
e39498129c
Merge branch 'master' into sync_replica_lightweight
2023-03-31 18:17:08 +03:00
Antonio Andelic
1e3abc9e84
Add strict mode for KeeperMap
2023-03-31 14:24:32 +00:00
Alexander Tokmakov
75ffc8451b
fix
2023-03-31 16:09:00 +02:00
Anton Popov
f715bd95f1
fix writing to StorageS3
2023-03-31 14:08:28 +00:00
Alexey Milovidov
3de29f5cc1
Merge pull request #48278 from azat/backups/fix-UB
...
Fix UB (signed integer overflow) in StorageMergeTree::backupData()
2023-03-31 16:35:10 +03:00
Alexander Tokmakov
caf00b5968
Merge branch 'master' into sync_replica_lightweight
2023-03-31 14:52:55 +02:00
Alexey Milovidov
dd79bf0a03
Update autogenerated version to 23.4.1.1 and contributors
2023-03-31 12:20:50 +02:00
Azat Khuzhin
bac7def719
Fix UB (signed integer overflow) in StorageMergeTree::backupData()
...
UBsan report:
/build/src/Storages/StorageMergeTree.cpp:2088:80: runtime error: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long'
0 0x30c2ae7b in DB::StorageMergeTree::backupData(DB::BackupEntriesCollector&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::optional<absl::lts_20211102::InlinedVector<std::__1::shared_ptr<DB::IAST>, 7ul, std::__1::allocator<std::__1::shared_ptr<DB::IAST>>>> const&) build_docker/./src/Storages/StorageMergeTree.cpp:2088:80
1 0x2dd641b5 in DB::BackupEntriesCollector::makeBackupEntriesForTableData(DB::QualifiedTableName const&) build_docker/./src/Backups/BackupEntriesCollector.cpp:703:18
2 0x2dd5c2a7 in DB::BackupEntriesCollector::makeBackupEntriesForTablesData() build_docker/./src/Backups/BackupEntriesCollector.cpp:676:9
3 0x2dd5848d in DB::BackupEntriesCollector::run() build_docker/./src/Backups/BackupEntriesCollector.cpp:119:5
4 0x2dd84da3 in DB::BackupsWorker::doBackup(std::__1::shared_ptr<DB::ASTBackupQuery> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, DB::BackupInfo const&, DB::BackupSettings, std::__1::shared_ptr<DB::IBackupCoordination>, std::__1::shared_ptr<DB::Context const> const&, std::__1::shared_ptr<DB::Context>, bool) build_docker/./src/Backups/BackupsWorker.cpp:359:59
5 0x2dd82405 in DB::BackupsWorker::startMakingBackup(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Backups/BackupsWorker.cpp:248:13
6 0x2dd81d0a in DB::BackupsWorker::start(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>) build_docker/./src/Backups/BackupsWorker.cpp:179:16
7 0x2f4d0d5f in DB::InterpreterBackupQuery::execute() build_docker/./src/Interpreters/InterpreterBackupQuery.cpp:39:30
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /build/src/Storages/StorageMergeTree.cpp:2088:80 in
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-31 12:29:31 +03:00
Antonio Andelic
e982f2a67a
Merge branch 'master' into enable-env-credentials-default
2023-03-31 09:11:01 +00:00
Anton Popov
5ceb855e7f
Merge branch 'master' into fix-race-storage-s3
2023-03-31 04:16:35 +02:00
kssenii
f1fe44e43d
Better
2023-03-30 23:09:12 +02:00
Anton Popov
38389d878c
fix one more race in StorageS3
2023-03-30 21:06:53 +00:00
Alexey Milovidov
614be6f718
Merge pull request #47387 from sippy/pg_select_from_only
...
Use "SELECT * FROM ONLY xyz", not "SELECT * FROM xyz" while replicating
2023-03-30 22:57:57 +03:00
kssenii
319417062f
Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes
2023-03-30 18:29:46 +02:00
kssenii
60efa3cae7
Fixes for hudi
2023-03-30 18:25:54 +02:00
Antonio Andelic
80cb121d2a
Merge pull request #48092 from ClickHouse/nosign-keyword-for-s3
...
Add support for `NOSIGN` keyword and `no_sign_request` config for S3
2023-03-30 18:10:56 +02:00
Anton Popov
e72472e71b
Merge branch 'master' into fix-race-storage-s3
2023-03-30 16:19:57 +02:00
kssenii
539414554f
Fix s3
2023-03-30 15:32:38 +02:00
Alexey Milovidov
cb27416d8e
Merge pull request #48102 from ClickHouse/vdimir/25402_storage_merge_explain_graph_crash
...
Fix crash in explain graph with StorageMerge
2023-03-30 13:54:22 +03:00
kssenii
bd407d2d5c
Fix build without s3
2023-03-30 12:13:03 +02:00
Robert Schulze
b0473098c6
Merge branch 'master' into qc-compression
2023-03-30 10:55:42 +02:00
Antonio Andelic
9db58532f4
Clang-tidy fix
2023-03-30 08:41:14 +02:00
Alexey Milovidov
a629fa3ed9
Merge pull request #48162 from ClickHouse/vdimir/remove_unused_from_gtest_transform_query
...
Remove unused mockSystemDatabase from gtest_transform_query_for_exter…
2023-03-30 01:29:42 +03:00
Alexey Milovidov
e982fb9f1c
Merge pull request #47880 from azat/threadpool-introspection
...
ThreadPool metrics introspection
2023-03-30 01:27:31 +03:00
Anton Popov
ed29c141fb
fix race in StorageS3
2023-03-29 22:13:45 +00:00
kssenii
bb698a6ed4
Fix black check
2023-03-29 23:54:44 +02:00
Alexander Gololobov
98b3617ce2
Merge pull request #48150 from DimasKovas/fix_stdlib_compatibility
...
Fix stdlib compatibility issues
2023-03-29 21:29:33 +02:00
kssenii
71d02d76f5
Better
2023-03-29 20:01:21 +02:00
kssenii
0300142888
Fix build without s3
2023-03-29 15:18:47 +02:00
Robert Schulze
b324d0ed5e
Merge branch 'master' into qc-compression
2023-03-29 15:09:22 +02:00
kssenii
fcc8e4223c
Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes
2023-03-29 14:13:27 +02:00
kssenii
c5516266fa
Fix build without s3
2023-03-29 14:09:22 +02:00
lzydmxy
3410082cb2
add columns perform_ttl_move_on_insert, load_balancing for table system.storage_policies
2023-03-29 19:58:25 +08:00
Antonio Andelic
44e95aa65f
Merge branch 'master' into nosign-keyword-for-s3
2023-03-29 11:10:03 +00:00
Antonio Andelic
7b1ad221b2
Address PR comments
2023-03-29 11:08:44 +00:00
robot-ch-test-poll
04d30a2afb
Merge pull request #48119 from Avogar/avro-more-types
...
Support more ClickHouse types in Avro format
2023-03-29 13:07:35 +02:00
vdimir
bdc8a47bcf
Remove unused mockSystemDatabase from gtest_transform_query_for_external_database
2023-03-29 11:02:50 +00:00
Kseniia Sumarokova
9e8bbe4bfe
Merge pull request #47987 from kssenii/try-fix-rabbitmq-test
...
Try fix rabbitmq test
2023-03-29 12:20:05 +02:00
vdimir
588fdcffdf
comment child_plans in ReadFromMerge
2023-03-29 09:42:32 +00:00
Dmitrii Kovalkov
71a6a02e88
fix stdlib compatibility issues
2023-03-29 12:20:27 +03:00
Azat Khuzhin
f38a7aeabe
ThreadPool metrics introspection
...
There are lots of thread pools and simple local-vs-global is not enough
already, it is good to know which one in particular uses threads.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-29 10:46:59 +02:00
Antonio Andelic
50ed205aa0
Merge pull request #47303 from ClickHouse/avoid-merging-partition
...
Skip merged partitions for `min_age_to_force_merge_seconds` merges
2023-03-29 09:31:00 +02:00
Sema Checherinda
1031e2001b
fix build
2023-03-29 07:41:31 +02:00
Robert Schulze
622d1dfadd
Merge pull request #46252 from ClibMouse/feature/hastokencaseinsensitive-index-utilization
...
Implement tokenbf_v1 index utilization for hasTokenCaseInsensitive
2023-03-28 22:38:21 +02:00
kssenii
82b642c9c6
Fix style check
2023-03-28 21:57:14 +02:00
kssenii
1573790fe7
Better
2023-03-28 20:51:25 +02:00
avogar
06a6f5cf76
Support more ClickHouse types in Avro format
2023-03-28 17:36:20 +00:00
robot-ch-test-poll3
19c73afda4
Merge pull request #48098 from kssenii/materialized-pg-small-fix
...
materialized pg small fix
2023-03-28 19:13:10 +02:00
kssenii
13f29a7242
Better
2023-03-28 18:57:24 +02:00
Sema Checherinda
b774310dc9
keep query id for logging when optimize query
2023-03-28 16:44:47 +02:00
kssenii
04b28bf822
Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes
2023-03-28 15:39:24 +02:00
Kruglov Pavel
292539b90e
Merge pull request #47981 from kssenii/better-delegate-disks-handle-mutual-path
...
Fix case when (optional) path is not added to encrypted disk configuration
2023-03-28 13:52:50 +02:00
Raúl Marín
bffde5e476
Merge remote-tracking branch 'blessed/master' into unnecessary_alter_checks
2023-03-28 13:17:36 +02:00
vdimir
7a85974c35
Fix crash in explain graph with StorageMerge
2023-03-28 11:04:07 +00:00
kssenii
943a4f75f8
Catch all exceptions to avoid replication stuck
2023-03-28 12:00:56 +02:00
kssenii
57226fcb25
Review fixes
2023-03-28 11:43:15 +02:00
Vladimir C
5fa519d043
Merge pull request #48047 from ClickHouse/vdimir/max_number_of_mutatuins_for_replica
2023-03-28 10:50:35 +02:00
Antonio Andelic
0b5fd2140a
Merge branch 'master' into cnf-new-analyzer
2023-03-28 07:57:12 +00:00
Antonio Andelic
30e4ee705a
Better
2023-03-28 07:54:23 +00:00
Antonio Andelic
a3b6d43212
Merge branch 'master' into avoid-merging-partition
2023-03-28 09:40:36 +02:00
Antonio Andelic
4aa8c5fe49
Merge branch 'master' into alternative-keeper-configs
2023-03-28 07:23:24 +00:00
Antonio Andelic
160aa186bb
Add support for NOSIGN keyword and no_sign_request config
2023-03-28 07:05:35 +00:00
Alexander Tokmakov
b5fe1ada0b
more options for sync replica
2023-03-28 01:39:36 +02:00
kssenii
f8b45a00ec
Fix
2023-03-27 19:20:48 +02:00
kssenii
314ee12442
Fix checks
2023-03-27 19:20:11 +02:00
Igor Nikonov
c2c7895b82
Merge pull request #47770 from ClickHouse/stop_wait_for_quorum_retries_on_shutdown
...
Stop `wait for quorum` retries on shutdown
2023-03-27 18:27:14 +02:00
Raúl Marín
d1a6c1991a
Only check MV on ALTER when necessary
2023-03-27 17:45:15 +02:00
Sema Checherinda
a6ab33a906
no use query, but storage context
2023-03-27 16:30:32 +02:00
Raúl Marín
83b68caccc
Do not continue retrying to connect to ZK if the query is killed or over limits
2023-03-27 16:01:15 +02:00
Alexander Tokmakov
00cf1efbc1
Merge pull request #48048 from ClickHouse/rename_system_dropped_tables
...
Rename `system.marked_dropped_tables` to `dropped_tables`
2023-03-27 15:09:22 +03:00
vdimir
08fa1846b1
Add merge tree setting max_number_of_mutations_for_replica
2023-03-27 11:56:48 +00:00
Alexander Tokmakov
b01f08ccb8
rename marked_dropped_tables to dropped_tables
2023-03-27 13:23:56 +02:00
Antonio Andelic
93f5920e02
Merge branch 'master' into alternative-keeper-configs
2023-03-27 10:00:21 +02:00
Antonio Andelic
e4a78a982f
Merge branch 'master' into cnf-new-analyzer
2023-03-27 07:43:21 +00:00
Smita Kulkarni
eabcefe452
Fixed issue with reserving vector size
2023-03-26 14:52:08 +02:00
SmitaRKulkarni
f0b724497f
Merge branch 'master' into 47200_Fix_for_index_file_cidx_is_unexpectedly_long
2023-03-26 14:02:56 +02:00
Smita Kulkarni
decdf0c507
Updated to delete added keys if there is an exception during update of metadata cache
2023-03-26 10:35:44 +02:00
Nikolai Kochetov
327674f56d
Merge pull request #46537 from ClickHouse/aggregate-projections-analysis-query-plan
...
Projections analysis from query plan
2023-03-25 13:23:30 +01:00
kssenii
36cc6fee51
Rewrite data lakes (part 1)
2023-03-24 22:35:12 +01:00
Sema Checherinda
aeb8766ad5
adjust after rebase
2023-03-24 19:53:16 +01:00
Sema Checherinda
da3e744405
set context from the master thread
2023-03-24 19:53:16 +01:00
Sema Checherinda
0fcf7c0363
std::optional instead shared_ptr
2023-03-24 19:53:16 +01:00
Sema Checherinda
6a6d45e6e7
set up performance_counters for thread group
2023-03-24 19:53:16 +01:00
Sema Checherinda
da4f2bd923
do not attach empty thread group
2023-03-24 19:53:16 +01:00
Sema Checherinda
bc107c70fa
merge and mutation make thread group for setting memory trackers right
2023-03-24 19:53:16 +01:00
Nikolai Kochetov
82f7c60e38
Merge pull request #47726 from k-morozov/feature/add_SourceStep_optimize
...
Add optimization for MemoryStorageStep
2023-03-24 17:56:10 +01:00
Maksim Kita
5cb2d30563
Merge pull request #47316 from ClickHouse/vdimir/transform_external_query_analyzer
...
Support transformQueryForExternalDatabase for analyzer
2023-03-24 17:53:00 +01:00