thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
115,775 Commits 9,326 Branches 1,783 Tags 2.1 GiB
63225b37a2
Commit Graph

12786 Commits

Author SHA1 Message Date
Sema Checherinda
554c928534
Merge branch 'master' into thead-group-for-merge 2023-04-05 21:09:38 +02:00
Sema Checherinda
c1adbb7aa6 transfer counters to the query directly 2023-04-05 19:51:36 +02:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling 
Per-query/per-server throttling for remote IO/local IO/BACKUPs
 2023-04-05 19:32:37 +02:00
kssenii
be13ce76f4 Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-04-05 18:47:08 +02:00
kssenii
a405d06e8a Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-04-05 18:46:52 +02:00
avogar
1be6bceb2f Generate valid Decimals and Bools in generateRandom function 2023-04-05 16:23:06 +00:00
Antonio Andelic
0260b84bc3 Update only affected rows in KV storage 2023-04-05 16:12:14 +00:00
Nikita Mikhaylov
3f44c6d936 Better 2023-04-05 15:19:45 +00:00
kssenii
62244ee57d Fix test 2023-04-05 17:14:41 +02:00
vdimir
01ce3baf0f
Rename Context::getTemporaryVolume to getGlobalTemporaryVolume 2023-04-05 13:42:13 +00:00
vdimir
6f3fa33641
Add unbound flag to system.filesystem_cache 2023-04-05 13:12:20 +00:00
Robert Schulze
62608f74d3
Merge pull request #48396 from ClickHouse/rs/clang-tidy-cleanup 
Cleanup mess in .clang-tidy
 2023-04-05 14:39:28 +02:00
Alexander Tokmakov
3b5ed9242c
Merge pull request #48391 from azat/fix-sync-replica-stuck 
Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION
 2023-04-05 15:20:46 +03:00
kssenii
3fb4cd0f52 Fix s3 test 2023-04-05 14:13:46 +02:00
Nikita Mikhaylov
7b526d3df9
Always communicate with local replica through loopback interface for parallel replicas (#48246) 2023-04-05 13:32:46 +02:00
Robert Schulze
1be81db885
Fix build, pt. II 2023-04-05 11:23:09 +00:00
Sema Checherinda
f504cd5fed avoid race, do not set thread_group.counters.parent from query 2023-04-05 12:36:55 +02:00
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis 
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:48 +02:00
kssenii
a3d69694f4 Fix build 2023-04-04 23:13:17 +02:00
kssenii
c379eb7c29 Fix style check 2023-04-04 22:59:16 +02:00
kssenii
f44c53b97a Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-04-04 22:41:22 +02:00
kssenii
8c0be0c3be Checkpoints 2023-04-04 22:40:05 +02:00
Alexander Tokmakov
ba65be22da fix bad error messages 2023-04-04 20:21:22 +02:00
Raúl Marín
0e17b3b147 Merge remote-tracking branch 'blessed/master' into zk_retry_timeout 2023-04-04 18:53:55 +02:00
Nikita Mikhaylov
fa5b2bd4a5
Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
Azat Khuzhin
353434f8f1 Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION 
In case of DROP/REPLACE PARTITION the entries from the queue will be
removed without notifying the subscribes.

CI: https://s3.amazonaws.com/clickhouse-test-reports/48242/4a315cd0d17bc12edd934fd25663b0119880f207/stress_test__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-04 17:16:04 +02:00
Antonio Andelic
a329d80bfa
Merge pull request #47397 from ClickHouse/enable-env-credentials-default 
Enable `use_environment_credentials` by default
 2023-04-04 10:00:03 +02:00
Antonio Andelic
52e4149c98
Merge pull request #47934 from ClickHouse/alternative-keeper-configs 
Merge #35113
 2023-04-04 09:58:33 +02:00
Antonio Andelic
4efa707fe3
Merge branch 'master' into alternative-keeper-configs 2023-04-04 09:53:19 +02:00
Antonio Andelic
0dda3d13ef
Merge branch 'master' into keeper-map-strict-mode 2023-04-04 09:52:53 +02:00
lizhuoyu5
2232d5ad14
Merge branch 'master' into add_columns_for_system_storage_policies 2023-04-04 12:58:10 +08:00
Yakov Olkhovskiy
35e9e45249
Merge pull request #48062 from Algunenano/unnecessary_alter_checks 
Only check MV on ALTER when necessary
 2023-04-03 17:23:11 -04:00
Sema Checherinda
6d5edc0315 rename fake_query_context to task_context 2023-04-03 22:01:18 +02:00
kssenii
75b11bceb5 Fix style check 2023-04-03 21:27:05 +02:00
kssenii
18a9a670c3 Better 2023-04-03 20:56:10 +02:00
kssenii
9b3d0ec86d Adjustments after conflicts 2023-04-03 19:53:34 +02:00
kssenii
8915f49b7d Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-04-03 17:43:42 +02:00
Antonio Andelic
f5e97fbb66 Merge branch 'master' into alternative-keeper-configs 2023-04-03 15:06:52 +00:00
kssenii
6e1cf1907d Better 2023-04-03 16:57:49 +02:00
Antonio Andelic
950a7d9948 Merge branch 'master' into keeper-map-strict-mode 2023-04-03 14:26:14 +00:00
kssenii
5578cb08ad Fix s3 cluster 2023-04-03 14:40:04 +02:00
Anton Popov
207a4c6166
Merge branch 'master' into do-not-build-unnecessary-sets 2023-04-03 14:29:33 +02:00
Robert Schulze
a50e741bea
Merge pull request #45912 from ClickHouse/qc-compression 
Query cache: Enable compression of cache entries
 2023-04-03 13:33:47 +02:00
Robert Schulze
3e124f09d0
Merge remote-tracking branch 'origin/master' into qc-compression 2023-04-03 11:28:17 +00:00
Robert Schulze
e79343c169
Make the column order in system.query_cache more intutitive 2023-04-03 10:36:11 +00:00
kssenii
018df220bf Remove redundant 2023-04-03 12:18:04 +02:00
kssenii
35a8328922 Fix 2023-04-03 12:11:12 +02:00
Antonio Andelic
2d89057142 Merge branch 'master' into keeper-map-strict-mode 2023-04-03 09:30:52 +00:00
Antonio Andelic
40a0ecf66a Fix 2023-04-03 09:30:39 +00:00
Raúl Marín
8fdf87982c Merge remote-tracking branch 'blessed/master' into zk_retry_timeout 2023-04-03 10:26:18 +02:00
Antonio Andelic
da194f082d
Merge branch 'master' into enable-env-credentials-default 2023-04-03 09:45:03 +02:00
Alexey Milovidov
ef1e282d4c
Merge branch 'master' into compress-marks-primary-key-by-default 2023-04-03 06:28:48 +03:00
Alexey Milovidov
99ed3b627b
Merge branch 'master' into fix-race-storage-s3 2023-04-03 06:25:44 +03:00
Anton Popov
ff1cc5598f fix clang-tidy 2023-04-02 22:21:10 +00:00
Alexey Milovidov
5ebf66819b
Merge pull request #47617 from ClickHouse/cnf-new-analyzer 
CNF/Constraint optimizer in new analyzer
 2023-04-02 20:09:34 +03:00
kssenii
6d75ca9c6f Fix 2023-04-02 18:21:45 +02:00
Alexey Milovidov
9c6594a043
Merge pull request #48320 from azat/explain-fix 
Fix crash in EXPLAIN PIPELINE for Merge over Distributed
 2023-04-02 17:11:27 +03:00
Azat Khuzhin
907ed27ae2 Fix crash in EXPLAIN PIPELINE for Merge over Distributed 
CI: https://s3.amazonaws.com/clickhouse-test-reports/48314/179450542879d11711cd2415c3fa7eeab18188be/fuzzer_astfuzzerasan/report.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-02 10:48:23 +02:00
kssenii
ddf2af598d Fix 2023-04-01 20:52:22 +02:00
kssenii
8cfd544328 Better 2023-04-01 18:41:42 +02:00
Azat Khuzhin
c64f9e6f07 Use StrongTypedef for ProfileEvents 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-01 16:01:33 +02:00
Azat Khuzhin
ba6ecd2d4e Fix ThreadPool for DistributedSink 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-01 16:00:03 +02:00
Alexander Tokmakov
48b23dd012
Fix race between DROP MatView and RESTART REPLICAS (#47863) 
* fix race between drop mv and restart replicas

* unrelated: fix bad exception messages

* fix test

* fix

* fix

* fix

* fix

* fix test

* fix

* fix test

* Update 02437_drop_mv_restart_replicas.sh

* fix tests
 2023-04-01 15:26:00 +03:00
kssenii
6ff728fb4d Fix cpu usage in rabbitmq 2023-04-01 11:45:25 +02:00
Anton Popov
cdf9cf2fd4 do not build sets for indexes if they are not used 2023-03-31 16:06:20 +00:00
Anton Popov
3b98a43dc6
Merge branch 'master' into fix-race-storage-s3 2023-03-31 17:54:52 +02:00
Alexander Tokmakov
e39498129c
Merge branch 'master' into sync_replica_lightweight 2023-03-31 18:17:08 +03:00
Antonio Andelic
1e3abc9e84 Add strict mode for KeeperMap 2023-03-31 14:24:32 +00:00
Alexander Tokmakov
75ffc8451b fix 2023-03-31 16:09:00 +02:00
Anton Popov
f715bd95f1 fix writing to StorageS3 2023-03-31 14:08:28 +00:00
Alexey Milovidov
3de29f5cc1
Merge pull request #48278 from azat/backups/fix-UB 
Fix UB (signed integer overflow) in StorageMergeTree::backupData()
 2023-03-31 16:35:10 +03:00
Alexander Tokmakov
caf00b5968 Merge branch 'master' into sync_replica_lightweight 2023-03-31 14:52:55 +02:00
Alexey Milovidov
17c2678675
Merge branch 'master' into compress-marks-primary-key-by-default 2023-03-31 15:15:38 +03:00
Alexey Milovidov
dd79bf0a03 Update autogenerated version to 23.4.1.1 and contributors 2023-03-31 12:20:50 +02:00
Azat Khuzhin
bac7def719 Fix UB (signed integer overflow) in StorageMergeTree::backupData() 
UBsan report:

    /build/src/Storages/StorageMergeTree.cpp:2088:80: runtime error: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long'
        0 0x30c2ae7b in DB::StorageMergeTree::backupData(DB::BackupEntriesCollector&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::optional<absl::lts_20211102::InlinedVector<std::__1::shared_ptr<DB::IAST>, 7ul, std::__1::allocator<std::__1::shared_ptr<DB::IAST>>>> const&) build_docker/./src/Storages/StorageMergeTree.cpp:2088:80
        1 0x2dd641b5 in DB::BackupEntriesCollector::makeBackupEntriesForTableData(DB::QualifiedTableName const&) build_docker/./src/Backups/BackupEntriesCollector.cpp:703:18
        2 0x2dd5c2a7 in DB::BackupEntriesCollector::makeBackupEntriesForTablesData() build_docker/./src/Backups/BackupEntriesCollector.cpp:676:9
        3 0x2dd5848d in DB::BackupEntriesCollector::run() build_docker/./src/Backups/BackupEntriesCollector.cpp:119:5
        4 0x2dd84da3 in DB::BackupsWorker::doBackup(std::__1::shared_ptr<DB::ASTBackupQuery> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, DB::BackupInfo const&, DB::BackupSettings, std::__1::shared_ptr<DB::IBackupCoordination>, std::__1::shared_ptr<DB::Context const> const&, std::__1::shared_ptr<DB::Context>, bool) build_docker/./src/Backups/BackupsWorker.cpp:359:59
        5 0x2dd82405 in DB::BackupsWorker::startMakingBackup(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const> const&) build_docker/./src/Backups/BackupsWorker.cpp:248:13
        6 0x2dd81d0a in DB::BackupsWorker::start(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>) build_docker/./src/Backups/BackupsWorker.cpp:179:16
        7 0x2f4d0d5f in DB::InterpreterBackupQuery::execute() build_docker/./src/Interpreters/InterpreterBackupQuery.cpp:39:30

    SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /build/src/Storages/StorageMergeTree.cpp:2088:80 in

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-31 12:29:31 +03:00
Antonio Andelic
e982f2a67a Merge branch 'master' into enable-env-credentials-default 2023-03-31 09:11:01 +00:00
Anton Popov
5ceb855e7f
Merge branch 'master' into fix-race-storage-s3 2023-03-31 04:16:35 +02:00
kssenii
f1fe44e43d Better 2023-03-30 23:09:12 +02:00
Anton Popov
38389d878c fix one more race in StorageS3 2023-03-30 21:06:53 +00:00
Alexey Milovidov
614be6f718
Merge pull request #47387 from sippy/pg_select_from_only 
Use "SELECT * FROM ONLY xyz", not "SELECT * FROM xyz" while replicating
 2023-03-30 22:57:57 +03:00
kssenii
319417062f Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-03-30 18:29:46 +02:00
kssenii
60efa3cae7 Fixes for hudi 2023-03-30 18:25:54 +02:00
Antonio Andelic
80cb121d2a
Merge pull request #48092 from ClickHouse/nosign-keyword-for-s3 
Add support for `NOSIGN` keyword and `no_sign_request` config for S3
 2023-03-30 18:10:56 +02:00
kssenii
e2cbccfbc4 Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-03-30 17:33:46 +02:00
Anton Popov
e72472e71b
Merge branch 'master' into fix-race-storage-s3 2023-03-30 16:19:57 +02:00
kssenii
539414554f Fix s3 2023-03-30 15:32:38 +02:00
Alexey Milovidov
cb27416d8e
Merge pull request #48102 from ClickHouse/vdimir/25402_storage_merge_explain_graph_crash 
Fix crash in explain graph with StorageMerge
 2023-03-30 13:54:22 +03:00
kssenii
bd407d2d5c Fix build without s3 2023-03-30 12:13:03 +02:00
Robert Schulze
b0473098c6
Merge branch 'master' into qc-compression 2023-03-30 10:55:42 +02:00
Antonio Andelic
9db58532f4
Clang-tidy fix 2023-03-30 08:41:14 +02:00
Alexey Milovidov
a629fa3ed9
Merge pull request #48162 from ClickHouse/vdimir/remove_unused_from_gtest_transform_query 
Remove unused mockSystemDatabase from gtest_transform_query_for_exter…
 2023-03-30 01:29:42 +03:00
Alexey Milovidov
e982fb9f1c
Merge pull request #47880 from azat/threadpool-introspection 
ThreadPool metrics introspection
 2023-03-30 01:27:31 +03:00
Anton Popov
ed29c141fb fix race in StorageS3 2023-03-29 22:13:45 +00:00
kssenii
bb698a6ed4 Fix black check 2023-03-29 23:54:44 +02:00
Alexander Gololobov
98b3617ce2
Merge pull request #48150 from DimasKovas/fix_stdlib_compatibility 
Fix stdlib compatibility issues
 2023-03-29 21:29:33 +02:00
kssenii
71d02d76f5 Better 2023-03-29 20:01:21 +02:00
kssenii
0300142888 Fix build without s3 2023-03-29 15:18:47 +02:00
Robert Schulze
b324d0ed5e
Merge branch 'master' into qc-compression 2023-03-29 15:09:22 +02:00
kssenii
fcc8e4223c Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-03-29 14:13:27 +02:00
kssenii
c5516266fa Fix build without s3 2023-03-29 14:09:22 +02:00
lzydmxy
3410082cb2 add columns perform_ttl_move_on_insert, load_balancing for table system.storage_policies 2023-03-29 19:58:25 +08:00
Antonio Andelic
44e95aa65f Merge branch 'master' into nosign-keyword-for-s3 2023-03-29 11:10:03 +00:00
Antonio Andelic
7b1ad221b2 Address PR comments 2023-03-29 11:08:44 +00:00
robot-ch-test-poll
04d30a2afb
Merge pull request #48119 from Avogar/avro-more-types 
Support more ClickHouse types in Avro format
 2023-03-29 13:07:35 +02:00
vdimir
bdc8a47bcf
Remove unused mockSystemDatabase from gtest_transform_query_for_external_database 2023-03-29 11:02:50 +00:00
kssenii
4cf2862a0e Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-03-29 12:50:36 +02:00
Kseniia Sumarokova
9e8bbe4bfe
Merge pull request #47987 from kssenii/try-fix-rabbitmq-test 
Try fix rabbitmq test
 2023-03-29 12:20:05 +02:00
vdimir
588fdcffdf
comment child_plans in ReadFromMerge 2023-03-29 09:42:32 +00:00
Dmitrii Kovalkov
71a6a02e88 fix stdlib compatibility issues 2023-03-29 12:20:27 +03:00
Azat Khuzhin
f38a7aeabe ThreadPool metrics introspection 
There are lots of thread pools and simple local-vs-global is not enough
already, it is good to know which one in particular uses threads.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-29 10:46:59 +02:00
Antonio Andelic
50ed205aa0
Merge pull request #47303 from ClickHouse/avoid-merging-partition 
Skip merged partitions for `min_age_to_force_merge_seconds` merges
 2023-03-29 09:31:00 +02:00
Sema Checherinda
1031e2001b fix build 2023-03-29 07:41:31 +02:00
HarryLeeIBM
47b31c39d1 Fix partition id issue for s390x 2023-03-28 15:31:26 -07:00
Robert Schulze
622d1dfadd
Merge pull request #46252 from ClibMouse/feature/hastokencaseinsensitive-index-utilization 
Implement tokenbf_v1 index utilization for hasTokenCaseInsensitive
 2023-03-28 22:38:21 +02:00
kssenii
82b642c9c6 Fix style check 2023-03-28 21:57:14 +02:00
kssenii
1573790fe7 Better 2023-03-28 20:51:25 +02:00
avogar
06a6f5cf76 Support more ClickHouse types in Avro format 2023-03-28 17:36:20 +00:00
robot-ch-test-poll3
19c73afda4
Merge pull request #48098 from kssenii/materialized-pg-small-fix 
materialized pg small fix
 2023-03-28 19:13:10 +02:00
kssenii
13f29a7242 Better 2023-03-28 18:57:24 +02:00
Sema Checherinda
b774310dc9 keep query id for logging when optimize query 2023-03-28 16:44:47 +02:00
kssenii
04b28bf822 Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-03-28 15:39:24 +02:00
Kruglov Pavel
292539b90e
Merge pull request #47981 from kssenii/better-delegate-disks-handle-mutual-path 
Fix case when (optional) path is not added to encrypted disk configuration
 2023-03-28 13:52:50 +02:00
Raúl Marín
bffde5e476 Merge remote-tracking branch 'blessed/master' into unnecessary_alter_checks 2023-03-28 13:17:36 +02:00
vdimir
7a85974c35
Fix crash in explain graph with StorageMerge 2023-03-28 11:04:07 +00:00
kssenii
943a4f75f8 Catch all exceptions to avoid replication stuck 2023-03-28 12:00:56 +02:00
kssenii
57226fcb25 Review fixes 2023-03-28 11:43:15 +02:00
Vladimir C
5fa519d043
Merge pull request #48047 from ClickHouse/vdimir/max_number_of_mutatuins_for_replica 2023-03-28 10:50:35 +02:00
Antonio Andelic
0b5fd2140a Merge branch 'master' into cnf-new-analyzer 2023-03-28 07:57:12 +00:00
Antonio Andelic
30e4ee705a Better 2023-03-28 07:54:23 +00:00
Antonio Andelic
a3b6d43212
Merge branch 'master' into avoid-merging-partition 2023-03-28 09:40:36 +02:00
Antonio Andelic
4aa8c5fe49 Merge branch 'master' into alternative-keeper-configs 2023-03-28 07:23:24 +00:00
Antonio Andelic
160aa186bb Add support for NOSIGN keyword and no_sign_request config 2023-03-28 07:05:35 +00:00
Alexander Tokmakov
b5fe1ada0b more options for sync replica 2023-03-28 01:39:36 +02:00
kssenii
f8b45a00ec Fix 2023-03-27 19:20:48 +02:00
kssenii
314ee12442 Fix checks 2023-03-27 19:20:11 +02:00
Igor Nikonov
c2c7895b82
Merge pull request #47770 from ClickHouse/stop_wait_for_quorum_retries_on_shutdown 
Stop `wait for quorum` retries on shutdown
 2023-03-27 18:27:14 +02:00
Raúl Marín
d1a6c1991a Only check MV on ALTER when necessary 2023-03-27 17:45:15 +02:00
Sema Checherinda
a6ab33a906 no use query, but storage context 2023-03-27 16:30:32 +02:00
Raúl Marín
83b68caccc Do not continue retrying to connect to ZK if the query is killed or over limits 2023-03-27 16:01:15 +02:00
Alexander Tokmakov
00cf1efbc1
Merge pull request #48048 from ClickHouse/rename_system_dropped_tables 
Rename `system.marked_dropped_tables` to `dropped_tables`
 2023-03-27 15:09:22 +03:00
vdimir
08fa1846b1
Add merge tree setting max_number_of_mutations_for_replica 2023-03-27 11:56:48 +00:00
Kruglov Pavel
bc92e3184e
Merge branch 'master' into non-blocking-connect 2023-03-27 13:50:50 +02:00
Alexander Tokmakov
b01f08ccb8 rename marked_dropped_tables to dropped_tables 2023-03-27 13:23:56 +02:00
Antonio Andelic
93f5920e02
Merge branch 'master' into alternative-keeper-configs 2023-03-27 10:00:21 +02:00
Antonio Andelic
e4a78a982f Merge branch 'master' into cnf-new-analyzer 2023-03-27 07:43:21 +00:00
Smita Kulkarni
eabcefe452 Fixed issue with reserving vector size 2023-03-26 14:52:08 +02:00
SmitaRKulkarni
f0b724497f
Merge branch 'master' into 47200_Fix_for_index_file_cidx_is_unexpectedly_long 2023-03-26 14:02:56 +02:00
Smita Kulkarni
decdf0c507 Updated to delete added keys if there is an exception during update of metadata cache 2023-03-26 10:35:44 +02:00
Nikolai Kochetov
327674f56d
Merge pull request #46537 from ClickHouse/aggregate-projections-analysis-query-plan 
Projections analysis from query plan
 2023-03-25 13:23:30 +01:00
Kruglov Pavel
713ecb2492
Merge branch 'master' into non-blocking-connect 2023-03-24 22:55:45 +01:00
kssenii
36cc6fee51 Rewrite data lakes (part 1) 2023-03-24 22:35:12 +01:00
Sema Checherinda
aeb8766ad5 adjust after rebase 2023-03-24 19:53:16 +01:00
Sema Checherinda
da3e744405 set context from the master thread 2023-03-24 19:53:16 +01:00
Sema Checherinda
0fcf7c0363 std::optional instead shared_ptr 2023-03-24 19:53:16 +01:00
Sema Checherinda
6a6d45e6e7 set up performance_counters for thread group 2023-03-24 19:53:16 +01:00
Sema Checherinda
da4f2bd923 do not attach empty thread group 2023-03-24 19:53:16 +01:00
Sema Checherinda
bc107c70fa merge and mutation make thread group for setting memory trackers right 2023-03-24 19:53:16 +01:00
Nikolai Kochetov
82f7c60e38
Merge pull request #47726 from k-morozov/feature/add_SourceStep_optimize 
Add optimization for MemoryStorageStep
 2023-03-24 17:56:10 +01:00
Maksim Kita
5cb2d30563
Merge pull request #47316 from ClickHouse/vdimir/transform_external_query_analyzer 
Support transformQueryForExternalDatabase for analyzer
 2023-03-24 17:53:00 +01:00
kssenii
16ebfcc3f8 Better delegate disks handle mutual path 2023-03-24 16:21:19 +01:00
Kruglov Pavel
f3f93dd06c
Merge branch 'master' into non-blocking-connect 2023-03-24 15:59:40 +01:00
Nikolai Kochetov
a2182f2659
Merge pull request #47874 from amosbird/correct_exact_rows_before_limit 
Correct `exact_rows_before_limit` in some scenarios
 2023-03-24 15:09:42 +01:00
Dmitry Novik
3488a4d84b Improve file includes 2023-03-24 03:44:52 +01:00
Alexander Tokmakov
4c9b63ccc0
Merge pull request #47916 from ClickHouse/fix_logical_error_in_distributed 
Fix logical error in DistributedSink
 2023-03-24 04:43:45 +03:00
Alexander Tokmakov
4dde73c3eb
Merge pull request #47917 from ClickHouse/fix_terminate_in_check_thread 
Fix terminate in parts check thread
 2023-03-24 04:22:00 +03:00
Nikolai Kochetov
73e98de46d Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-03-23 21:28:36 +01:00
Kruglov Pavel
3ee12e21fb
Merge branch 'master' into non-blocking-connect 2023-03-23 20:53:44 +01:00
ltrk2
f46970671d Implement tokenbf_v1 index utilization for hasTokenCaseInsensitive 2023-03-23 12:24:13 -07:00
Maksim Kita
24b9e5103f
Merge pull request #47748 from ClickHouse/revert-47721-revert-47622-analyzer-planner-fixes-before-enable-by-default 
Revert "Revert "Analyzer planner fixes before enable by default""
 2023-03-23 22:20:34 +03:00
vdimir
c2a3679959
Add buildSelectQueryInfo to Planner/Utils, use in unit tests 2023-03-23 15:53:22 +00:00
Alexander Tokmakov
5b263b9ffe fix 2023-03-23 15:36:17 +01:00
Alexander Tokmakov
54314061ab fix logical error on cancellation 2023-03-23 13:13:16 +01:00
Maksim Kita
7da53bf7e8 Fixed tests 2023-03-23 12:34:23 +01:00
Antonio Andelic
9ea8dc4e98 Merge branch 'master' into alternative-keeper-configs 2023-03-23 09:22:46 +00:00
Antonio Andelic
a4d1cd514d Refactor 2023-03-23 08:58:56 +00:00
robot-ch-test-poll3
a31c0d8eee
Merge pull request #47883 from ClickHouse/drop-column-with-projections 
Fix DROP COLUMN with ReplicatedMergeTree containing projections
 2023-03-22 23:55:16 +01:00
Alexander Tokmakov
e3c798bae5 fix terminate in parts check thread 2023-03-22 23:46:15 +01:00
vdimir
426f4ec233
Better transformQueryForExternalDatabase for analyzer 2023-03-22 16:58:34 +00:00
Maksim Kita
f45dac99c2 Revert "Revert "Analyzer planner fixes before enable by default"" 2023-03-22 17:19:32 +01:00
Nikolai Kochetov
d7d3722b76 Revert analyzer till later. 2023-03-22 15:41:54 +00:00
Amos Bird
02c5d1f364
Correct exact_rows_before_limit in all senarios 2023-03-22 23:26:31 +08:00
Antonio Andelic
65b99ec6a7 Merge branch 'master' into alternative-keeper-configs 2023-03-22 12:22:57 +00:00
Nikita Mikhaylov
301e26b23d
Parallel replicas. Part 2.5 (#47858) 2023-03-22 13:05:38 +01:00
Antonio Andelic
b832840cf3 Recalculate always projections on mutations in ReplicatedMergeTree 2023-03-22 11:21:23 +00:00
vdimir
6919db3d46 Fix test_mysql_protocol 2023-03-22 08:59:04 +01:00
vdimir
023d14a894 Support transformQueryForExternalDatabase for analyzer 2023-03-22 08:59:04 +01:00
kssenii
716bbe1a97 Fix style check, black check 2023-03-21 22:19:05 +01:00
Alexander Tokmakov
c114361879
Update MergeTreeData.h 2023-03-21 23:15:59 +03:00
Nikolay Degterinsky
3a8314c13e
Merge pull request #47576 from HarryLeeIBM/hlee-s390x-storage-gen 
Fix SipHash hashing and byte order in GenerateRandom for s389x
 2023-03-21 21:06:08 +01:00
Alexander Tokmakov
8a71e42f08
Merge pull request #47769 from HarryLeeIBM/hlee-s390x-partid 
Fix partition ID byte order for s390x
 2023-03-21 23:01:36 +03:00
Alexander Tokmakov
0989dd837a
Merge pull request #47802 from ClickHouse/fix_some_tests 
Fix 993 and two other tests
 2023-03-21 22:20:06 +03:00
kssenii
19819f12f3 Better 2023-03-21 20:19:30 +01:00
Nikolai Kochetov
b432fa1122 Attempt to use new analyzer. 2023-03-21 17:56:58 +00:00
avogar
38e44861ae Fix possible race conditions 2023-03-21 16:01:54 +00:00
Kseniia Sumarokova
3c550b4314
Merge pull request #46647 from kssenii/named-collections-finish 
Named collections: finish replacing old code for storages
 2023-03-21 12:36:46 +01:00
Kseniia Sumarokova
a9bcd022d5
Merge pull request #46241 from kssenii/named-collections-granular-access 
Allow separate grants for every named collection
 2023-03-21 12:27:53 +01:00
Antonio Andelic
ed3f51d9b3 Merge branch 'master' into avoid-merging-partition 2023-03-21 10:46:03 +00:00
Vladimir C
5add1e1b2e
Merge pull request #47684 from ClickHouse/vdimir/improve_log_queries_cut 2023-03-21 11:17:54 +01:00
Alexander Tokmakov
d5a561a2ef fix 993 and two other tests 2023-03-20 22:51:32 +01:00
Robert Schulze
595796e63d
Merge branch 'master' into rs/more-llvm-16-preparation 2023-03-20 21:16:27 +01:00
Robert Schulze
d8493780b2
Query cache: Enable compression and squashing of result blocks 
ClickHouse reads table data in blocks of 'max_block_size' rows. Due to
filtering, aggregation, etc., result blocks are typically much smaller
than 'max_block_size' but there are also cases where they are much
bigger. Setting 'query_cache_squash_partial_results' (enabled by
default) now controls if result blocks are squashed (if they are tiny)
or split (if they are large) into blocks of 'max_block_size' size before
insertion into the query result cache. This reduces performance of
writes into the query cache but improves compressability of cache
entries and provides more natural block granularity when query results
are later served from the query cache.

Entries in the query cache are now also compressed by default. This
reduces the overall memory consumption at the cost of slower writes into
/ reads from the query cache. To disable compression, use setting
'query_cache_compress_entries'.
 2023-03-20 20:11:41 +00:00
Robert Schulze
82a4b307ae
Merge branch 'master' into rs/qc-cachebase 2023-03-20 20:44:34 +01:00
HarryLeeIBM
ed1ff02114 Fix partition ID byte order for s390x 2023-03-20 14:19:27 -04:00
robot-ch-test-poll4
91f8901cd6
Merge pull request #47743 from ucasfl/remove 
remove unused code
 2023-03-20 17:43:41 +01:00
Robert Schulze
3b7f929001
Merge branch 'master' into rs/qc-cachebase 2023-03-20 17:08:47 +01:00
kssenii
75624c52f7 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-03-20 16:38:14 +01:00
vdimir
bb7a8ed862
Apply log_queries_cut_to_length in MergeTreeWhereOptimizer 2023-03-20 15:38:09 +00:00
Igor Nikonov
1073c7f3c9 Revert "Actually stop retries in case of storage shutdown" 
This was unnecessary since, as for exceptions, retries done only on
zkutil::KeeperException

This reverts commit 795a1c84e1.
 2023-03-20 13:54:25 +00:00
Igor Nikonov
795a1c84e1 Actually stop retries in case of storage shutdown 2023-03-20 13:48:04 +00:00
Igor Nikonov
1098055409 Stop wait for quorum retries on shutdown 2023-03-20 13:21:10 +00:00
Robert Schulze
5b036a1a3b
More preparation for libcxx(abi), llvm, clang-tidy 16 (follow-up to #47722) 2023-03-20 12:55:03 +00:00
Raúl Marín
fb6ce35ea2 Fix ZK exception error message 2023-03-20 13:00:24 +01:00
flynn
cac37c8a3c
Merge branch 'master' into remove 2023-03-20 18:43:48 +08:00
kssenii
cae3b335d6 Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-03-20 11:23:22 +01:00
kssenii
8537ee400d Fix kafka test 2023-03-20 11:22:25 +01:00
flynn
8a6e4daefa remove unused code 2023-03-20 08:31:38 +00:00
Alexey Milovidov
ddf29b69f9
Merge branch 'master' into fuzzer-of-data-formats 2023-03-20 03:26:51 +03:00
Alexey Milovidov
e788bf0712
Merge pull request #47721 from ClickHouse/revert-47622-analyzer-planner-fixes-before-enable-by-default 
Revert "Analyzer planner fixes before enable by default"
 2023-03-20 03:24:03 +03:00
Konstantin Morozov
dec5972194 add optimization for MemoryStorageStep 2023-03-19 20:35:24 +01:00
Robert Schulze
0e01e912fe
Preparation for libcxx(abi), llvm, clang-tidy 16 2023-03-19 17:25:41 +00:00
Alexey Milovidov
ef5e8462da
Revert "Analyzer planner fixes before enable by default" 2023-03-19 19:25:37 +03:00
Robert Schulze
c4664c70a1
Merge branch 'master' into rs/qc-cachebase 2023-03-19 15:23:24 +01:00
Alexey Milovidov
a99f10e16f Better error messages 2023-03-19 05:34:44 +01:00
Alexander Tokmakov
f1df91cd09
Merge pull request #47659 from ClickHouse/fix_993 
Fixes for 993 and friends
 2023-03-18 14:46:43 +03:00
Maksim Kita
5187138510
Merge pull request #47622 from kitaisreal/analyzer-planner-fixes-before-enable-by-default 
Analyzer planner fixes before enable by default
 2023-03-18 13:11:40 +03:00
Antonio Andelic
a0582a14b9
Merge pull request #47423 from ClickHouse/add-expiration-window-s3 
Add expiration window for S3 credentials
 2023-03-18 10:11:29 +01:00
Sema Checherinda
0b641fcead
Merge pull request #47564 from CheSema/remove-redundant 
remove counters for threads, fix negative counters
 2023-03-18 02:23:52 +01:00
Han Fei
722e6bb2e8
Merge pull request #47680 from ClickHouse/rs/uncruftify-build 
CMake: Remove further cruft from build
 2023-03-17 22:58:46 +01:00
kssenii
f776f4ff46 Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-03-17 19:45:28 +01:00
kssenii
e48d8d12e7 Fixes for hudi 2023-03-17 19:44:30 +01:00
Alexander Tokmakov
d11aee2b0c
Merge branch 'master' into fix_993 2023-03-17 21:28:23 +03:00
alesapin
e45b558a86
Merge pull request #47676 from ClickHouse/fix_zero_copy_lock_check 
Fix zero copy lock check
 2023-03-17 17:51:14 +01:00
alesapin
1022c47421 Review fixes 2023-03-17 17:45:02 +01:00
Antonio Andelic
1b7401b58a
Update src/Storages/StorageS3.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2023-03-17 15:46:15 +01:00
Antonio Andelic
79cff0a14b Merge branch 'master' into cnf-new-analyzer 2023-03-17 14:37:40 +00:00
Robert Schulze
f72a337074
Remove cruft from build 
No need to check compiler flags, clang >= 15 supports all of them.
 2023-03-17 13:44:04 +00:00
Antonio Andelic
0fb9f9ffe6 Add support for substitute column 2023-03-17 13:38:01 +00:00
Maksim Kita
2d2696bc20 Fixed code review issues 2023-03-17 14:34:23 +01:00
kssenii
a2977e373a Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-03-17 14:34:04 +01:00
kssenii
d7b2ea60eb Review fixes 2023-03-17 13:56:02 +01:00
kssenii
bb0beb7449 Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-03-17 13:02:36 +01:00
alesapin
e44fca8bb9 Add test 2023-03-17 12:56:31 +01:00
Sergei Trifonov
1a84203c7f
Merge pull request #47656 from ClickHouse/background_pool_limit_metric 
Add background pools size metrics
 2023-03-17 11:49:53 +01:00
chen
60b7ad819d
Add system.marked_dropped_tables (#47364) 
* add system table marked_dropped_tables

* add test

* add test

* fix

* fix test

* fix style

* fix test

* Update 25400_marked_dropped_tables.sql

* Update 25400_marked_dropped_tables.sql

* Update 25400_marked_dropped_tables.sql

* Update 25400_marked_dropped_tables.sql

* Update 25400_marked_dropped_tables.sql

* fix

* fix test

* Update StorageSystemMarkedDroppedTables.cpp

* fix

* if table_mark_dropped.table is nullptr, engine set empty
 2023-03-17 13:48:26 +03:00
alesapin
4a80197f16 Merge branch 'master' into fix_zero_copy_lock_check 2023-03-17 11:18:51 +01:00
alesapin
efa0622a84
Merge pull request #47631 from ClickHouse/fix_wait_for_zero_copy_lock 
Fix wait for zero copy lock during move
 2023-03-17 11:15:55 +01:00
Sergei Trifonov
2cbd91b852
Merge branch 'master' into background_pool_limit_metric 2023-03-16 23:23:42 +01:00
alesapin
3731567b6a Bad code 2023-03-16 21:48:47 +01:00
Robert Schulze
02a2c73bae
Merge pull request #47607 from save-my-heart/inverted_index 
checksum: do not check inverted index files
 2023-03-16 21:34:17 +01:00
Alexander Tokmakov
d621b2c4ad fix intersecting parts, better fault injections 2023-03-16 21:28:07 +01:00
alesapin
92eb6685c6 Fix lock check 2023-03-16 20:17:29 +01:00
Sema Checherinda
3c6deddd1d work with comments on PR 2023-03-16 19:55:58 +01:00
serxa
3ad7194e31 add pool size metrics 2023-03-16 17:31:41 +00:00
Antonio Andelic
985a0285c7 Merge branch 'master' into cnf-new-analyzer 2023-03-16 16:00:51 +00:00
Azat Khuzhin
8495deb7e3 Remove unused removePartFromZooKeeper() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit dbdb46e906)
 2023-03-16 16:43:23 +01:00
Alexander Tokmakov
8b8215f91a fix some other tests, add logs for further debugging 2023-03-16 16:40:08 +01:00
Alexander Tokmakov
8e36f883d3 add 'strict' mode for sync replica, fix 00993 2023-03-16 16:04:42 +01:00
Antonio Andelic
c7787a6652 Fix column resolution 2023-03-16 14:57:07 +00:00
Robert Schulze
7d7052350c
Merge branch 'master' into rs/qc-cachebase 2023-03-16 10:42:17 +01:00
Antonio Andelic
0f6e4d3934 Start implementing constraint optimizer 2023-03-16 09:30:55 +00:00
alesapin
fc913649cf Check only for interactive moves 2023-03-15 22:51:53 +01:00
alesapin
ef7508469c Fix wait for zero-copy replication move during manual move 2023-03-15 22:29:24 +01:00
Maksim Kita
6c2084052c Fixed tests 2023-03-15 15:21:49 +01:00
Maksim Kita
c102adb728 Analyzer added PREWHERE support 2023-03-15 15:21:49 +01:00
Mike Kot
511ed93512
Update src/Storages/System/StorageSystemDatabases.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2023-03-15 16:38:59 +03:00
Maksim Kita
4337a3161a
Merge pull request #47370 from ClickHouse/fix-grouping-for-grouping-sets 
Fix GROUPING function initialization for grouping sets
 2023-03-15 16:06:49 +03:00
save-my-heart
b7d9d721fb add test 2023-03-15 21:01:12 +08:00
robot-ch-test-poll1
43db7c59d9
Merge pull request #47260 from ClickHouse/vdimir/storage_join_insert_select_deadlock 
Forbid insert select for the same StorageJoin
 2023-03-15 12:52:29 +01:00
Robert Schulze
2f75a6da61
Merge remote-tracking branch 'origin/master' into rs/qc-cachebase 2023-03-15 11:00:53 +00:00
Robert Schulze
75ca2dd0e2
Merge remote-tracking branch 'origin/master' into rs/qc-cachebase 2023-03-15 10:55:53 +00:00
Vitaly Baranov
00c05f7f8b
Merge pull request #47546 from vitlibar/use-server-side-copy-for-restore 
Use server-side copy for restore from S3 to S3.
 2023-03-15 11:28:29 +01:00
Robert Schulze
5cee479f0c
Merge pull request #47424 from ClickHouse/rs/compile-with-cpp23 
Compile with C++23
 2023-03-15 10:55:42 +01:00
save-my-heart
d70c352313 checksum: do not check inverted index files 2023-03-15 16:43:59 +08:00
Han Fei
6cdd3ffb31
Merge pull request #47541 from azat/dist/preserve-error-on-flush 
Preserve error in system.distribution_queue on SYSTEM FLUSH DISTRIBUTED
 2023-03-14 20:25:01 +01:00
HarryLeeIBM
a32d7a00bc Fix SipHash integer hashing and byte order issue in GenerateRandom for s390x 2023-03-14 11:54:17 -04:00
SmitaRKulkarni
52b6976822
Merge pull request #47495 from ClickHouse/Parameterized_view_bug_fix_47287_47247 
Parameterized view bug fix 47287 47247
 2023-03-14 14:33:44 +01:00
Dmitry Novik
cced9cf613 Fix build 2023-03-14 12:04:39 +00:00
Dmitry Novik
ae3d30a736 Merge remote-tracking branch 'origin/master' into fix-grouping-for-grouping-sets 2023-03-14 12:01:51 +00:00
Alexander Tokmakov
5a1ee881f8
Merge pull request #47547 from ClickHouse/filter_too_frequent_logs 
Hide too noisy log messages, fix some tests
 2023-03-14 13:53:21 +03:00
Robert Schulze
73afae2d3f
Introduce TTLCachePolicy 2023-03-14 09:45:20 +00:00
Smita Kulkarni
8a7bc3250d Added space to if expression of replaceQueryParameterWithValue function 2023-03-14 09:26:53 +01:00
Smita Kulkarni
4deeb13c6d Addressed review comments for parameterized view bug fix 2023-03-14 09:23:12 +01:00
Dmitry Novik
49b0bc21ad Resolve as FunctionGrouping 2023-03-14 03:33:31 +00:00
Michael Kolupaev
d3a514d221 Compress marks in memory 2023-03-13 16:29:00 -07:00
Alexander Tokmakov
ba6c6255a3 hide too noisy log messages, fix some tests 2023-03-14 00:23:36 +01:00
Vitaly Baranov
1cf1ce07fe Use server-side copy during restore from S3 to S3. 2023-03-13 23:50:13 +01:00
Azat Khuzhin
3d247b8635 Preserve error in system.distribution_queue on SYSTEM FLUSH DISTRIBUTED 
After refactoring in #45491 this behaviour had been changed, hence
01555_system_distribution_queue_mask became flaky, since if the flush
had not happened before SYSTEM FLUSH DISTRIBUTED the error was not saved
into the system.distribution_queue.

v2: Improve 01555_system_distribution_queue_mask test
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-13 20:07:55 +01:00
Nikolai Kochetov
669a92bae0 Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-03-13 19:55:49 +01:00
Alexey Milovidov
a411ae10b5
Merge pull request #47486 from ClickHouse/revert_25674 
Remove "asynchronous connection drain"
 2023-03-13 21:40:39 +03:00
Robert Schulze
657aa65446
Cleanup: Untangle CacheBase's constructors 
This prepares for the custom TTLCachePolicy which users should not be
able to configure explicitly.
 2023-03-13 15:11:12 +00:00
alesapin
8db0d26519
Merge branch 'master' into revert-46909-revert-45911-mutations_rename_hang 2023-03-13 14:48:20 +01:00
Nikolai Kochetov
67424edbf0
Merge pull request #47180 from k-morozov/indices-storage-memory/update-read 
Update read in StorageMemory
 2023-03-13 14:40:35 +01:00
Anton Popov
739c8988dd
Merge pull request #46830 from ClickHouse/fix-url-progress-bar 
Fix progress bar with URL
 2023-03-13 13:53:40 +01:00
vdimir
9f220054c8
Fix build 2023-03-13 10:49:51 +00:00
Antonio Andelic
4013bc9255 Merge branch 'master' into add-expiration-window-s3 2023-03-13 10:21:39 +00:00
Antonio Andelic
51416da818 Merge branch 'master' into enable-env-credentials-default 2023-03-13 10:20:21 +00:00
Smita Kulkarni
0d45443cb0 Fix for clang-tidy build 2023-03-13 09:18:44 +01:00
Konstantin Morozov
b287e4ae07
Merge branch 'master' into indices-storage-memory/update-read 2023-03-12 22:13:19 +01:00
Alexey Milovidov
f445649845
Merge branch 'master' into rs/compile-with-cpp23 2023-03-12 21:33:20 +03:00
Alexey Milovidov
38612635b0
Update StorageView.cpp 2023-03-12 20:43:47 +03:00
Smita Kulkarni
39a146ab87 Merge branch 'master' into Parameterized_view_bug_fix_47287_47247 2023-03-12 11:05:28 +01:00
Smita Kulkarni
e85096b444 Updated variable name to fix build & call to getSampleBlockForColumns. Added a test for substring parameter name. 2023-03-12 11:00:17 +01:00
Alexander Tokmakov
ed08f8f5c5
Merge branch 'master' into revert_25674 2023-03-12 02:33:25 +03:00
Alexey Milovidov
218ac10425 More detailed error messages on too many parts 2023-03-11 21:44:38 +01:00
Smita Kulkarni
5b11d72ce4 In parameterized view, moved parameter values to SelectQueryInfo from StorageView as it can be used by multiple queries and updated column name check to avoid issues with parameter name being substring of column name. 2023-03-11 20:39:43 +01:00
Maksim Kita
114d30d2bb
Merge pull request #47383 from kitaisreal/analyzer-planner-fixes-before-enable-by-default 
Analyzer planner fixes before enable by default
 2023-03-11 22:09:04 +03:00
Alexander Tokmakov
7b1b238d0b Revert "Merge pull request #25674 from amosbird/distributedreturnconnection" 
This reverts commit 5ffd99dfd4, reversing
changes made to 2796aa333f.
 2023-03-11 19:09:47 +01:00
Maksim Kita
c835fa3958 Fixed tests 2023-03-11 11:51:54 +01:00
Maksim Kita
8bc92c197f Fixed build 2023-03-11 11:51:54 +01:00
Maksim Kita
5d86df963e Analyzer disable StorageWindowView support 2023-03-11 11:51:54 +01:00
Maksim Kita
3d34247998 Fixed tests 2023-03-11 11:51:54 +01:00
Maksim Kita
0358cb36d8 Fixed tests 2023-03-11 11:51:54 +01:00
Maksim Kita
677408e02e Fixed style check 2023-03-11 11:51:54 +01:00
Maksim Kita
a762112e15 Analyzer support distributed JOINS and subqueries in IN functions 2023-03-11 11:51:54 +01:00
Alexander Tokmakov
6e7ffd9a7b
Update ReplicatedMergeTreeAttachThread.cpp 2023-03-11 00:45:09 +03:00
Robert Schulze
348a40aa5d
Compile with C++23 2023-03-10 19:11:58 +00:00
Dmitry Novik
4f24a7bd11 Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-03-10 18:57:45 +00:00
Dmitry Novik
2699ef477f Move visitor 2023-03-10 14:36:56 +00:00
Konstantin Morozov
08fdf42cd8 Merge branch 'master' into indices-storage-memory/update-read 2023-03-10 11:51:06 +01:00
Antonio Andelic
a170a909a4 Add expiration window for S3 credentials 2023-03-10 10:06:32 +00:00
Dmitry Novik
a305c6e7ab Fix distributed GROUPING SETS and GROUPING function 2023-03-09 18:00:23 +00:00
Anton Popov
66a113ca61
Merge pull request #47384 from CurtizJ/fix-clear-column-with-sparse 
Fix `ALTER CLEAR COLUMN` with sparse columns
 2023-03-09 16:57:23 +01:00
alesapin
588cc0f159
Merge branch 'master' into revert-46909-revert-45911-mutations_rename_hang 2023-03-09 14:54:10 +01:00
vdimir
c9e48f37cb
Forbid insert select for the same StorageJoin 2023-03-09 12:28:44 +00:00
Antonio Andelic
5bc21538e5 Enable use_environment_credentials by default 2023-03-09 10:31:55 +00:00
Antonio Andelic
a70ca31884 Merge branch 'master' into fix-url-progress-bar 2023-03-09 10:17:33 +00:00
Konstantin Morozov
255102292c refactoring: remove uselees ctor 2023-03-09 10:02:16 +01:00
Konstantin Morozov
4686eb3cdd refactoring: create new step only once 2023-03-09 09:57:23 +01:00
Konstantin Morozov
82efbf01aa fix empty pipe for read 2023-03-09 08:28:55 +01:00
Maksym Sobolyev
6885863564 Use "SELECT FROM ONLY xyz", not "SELECT FROM xyz" while replicating 
postgresql tables, to properly handle inherited tables. Currently,
it would fetch same data twice - once from the child tables and then
from the parent table.
 2023-03-08 16:18:13 -08:00
Anton Popov
c3b9af96c4 fix ALTER CLEAR COLUMN with sparse columns 2023-03-08 17:19:04 +00:00
Antonio Andelic
a04b38db90
Merge pull request #45108 from ClickHouse/custom-key-parallel-replicas 
Add support for custom key in parallel replicas
 2023-03-08 17:46:11 +01:00
Alexander Tokmakov
38f9f08e1a
Merge pull request #47268 from myrrc/improvement/more-to-base-3 
use std::lerp, constexpr hex.h
 2023-03-08 19:07:34 +03:00
kssenii
0240ad4c68 Add spark to tests, rewrite tests, fix bug 2023-03-08 15:45:35 +01:00
alesapin
4423be8332
Merge branch 'master' into revert-46909-revert-45911-mutations_rename_hang 2023-03-08 12:59:07 +01:00
Antonio Andelic
816a9c1099
Merge branch 'master' into custom-key-parallel-replicas 2023-03-08 11:06:30 +01:00
Vladimir C
2aec0c1cb0
Merge pull request #47309 from ClickHouse/vdimir/fix_live_view_bg_pool_exception 2023-03-08 10:06:17 +01:00
Robert Schulze
2a47326061
Merge pull request #47313 from ClickHouse/rs/doc-cleanup-kafka 
Docs: Small cleanups after Kafka fix #47138
 2023-03-08 09:40:47 +01:00
Alexey Milovidov
e8cdb0c8b1
Merge pull request #46839 from CurtizJ/fix-crash-in-skip-index 
Do not allow const and non-deterministic secondary indexes
 2023-03-08 03:03:09 +03:00
Alexey Milovidov
e2aff59a2d
Merge pull request #47106 from kssenii/allow-nested-disks-in-custom-disks 
Allow nested custom disks
 2023-03-08 03:00:55 +03:00
Mike Kot
9920a52c51 use std::lerp, constexpr hex.h 2023-03-07 22:50:17 +00:00
Robert Schulze
ff8fc0634a
Docs: Small cleanups after Kafka fix #47138 2023-03-07 19:50:42 +00:00
Robert Schulze
5b91631722
Merge pull request #47138 from AVMusorin/disallow-default-columns-kafka-engine 
Prohibit DEFAULT/EPHEMERAL/ALIAS/MATERIALIZED columns in KafkaEngine
 2023-03-07 20:36:46 +01:00
vdimir
4e79df448f
Catch exceptions in LiveViewPeriodicRefreshTask 2023-03-07 16:17:35 +00:00
Antonio Andelic
d098891185 Skip merges if no need 2023-03-07 13:08:56 +00:00
alesapin
c10cb436f4 Fix build 2023-03-07 12:42:42 +01:00
alesapin
1db6b9414e
Update src/Storages/StorageReplicatedMergeTree.cpp 
Co-authored-by: Ilya Yatsishin <2159081+qoega@users.noreply.github.com>
 2023-03-07 12:39:17 +01:00
Antonio Andelic
35c15e6ef8 Merge branch 'master' into custom-key-parallel-replicas 2023-03-07 09:37:38 +00:00
alesapin
b5dffe7417 Fix bug in zero copy replica which can lead to dataloss 2023-03-06 19:57:09 +01:00
Anton Popov
1b0d0c6108 fix skip indexes 2023-03-06 15:29:13 +00:00
Mike Kot
1fca58b694
Merge branch 'master' into feature/show-secrets-option 2023-03-06 15:46:54 +03:00
Han Fei
b7eef62458
Merge pull request #45491 from azat/dist/async-send-refactoring 
[RFC] Rewrite distributed sends to avoid using filesystem as a queue, use in-memory queue instead
 2023-03-06 12:32:33 +01:00
kssenii
8567e3976b Fix kafka 2023-03-06 12:30:25 +01:00
Vladimir C
1455e7602d
Merge pull request #47154 from azat/prewhere-columns-fix 2023-03-06 10:03:31 +01:00
Anton Popov
6f3e4d4137
Merge pull request #46118 from CurtizJ/fix-issues-with-sparse 
Randomize setting `ratio_of_defaults_for_sparse_serialization`
 2023-03-05 22:28:18 +01:00
kssenii
c06af1f1e7 Fix clang-tidy 2023-03-05 22:12:51 +01:00
kssenii
8f2d75cef8 Fix tests 2023-03-05 12:56:00 +01:00
Robert Schulze
10753481f9
Merge pull request #47242 from ClickHouse/rs/docs-index-support-has-hasall-hasany 
Docs: Update index support of has(), hasAny(), hasAll()
 2023-03-04 18:49:24 +01:00
Robert Schulze
11930ad685
Docs: Update index support of has(), hasAny(), hasAll() 2023-03-04 17:27:47 +00:00
Sema Checherinda
85178c9609
Merge pull request #47102 from CheSema/merge-mutate-do-thread-group 
do flushUntrackedMemory when context switches
 2023-03-03 20:23:25 +01:00
Konstantin Morozov
e0ca12dc66 fix failed CI 2023-03-03 18:16:14 +01:00
Anton Popov
9389cc29da better mutations of sparse columns 2023-03-03 16:52:01 +00:00
kssenii
7b369e516c Fix stress test 2023-03-03 17:20:27 +01:00
kssenii
f78da967b3 Fix test 2023-03-03 16:32:23 +01:00
AVMusorin
0f2ae72141
prohibit DEFAULT/EPHEMERAL/ALIAS in KafkaEngine 2023-03-03 16:25:42 +01:00
Antonio Andelic
737cf8e149 Better 2023-03-03 15:14:49 +00:00
Alexander Tokmakov
cadaf06829
Merge pull request #46681 from ClickHouse/fix_insert_cancellation_in_native_protocol 
Fix queries cancellation when a client dies
 2023-03-03 16:27:20 +03:00
Konstantin Bogdanov
1bbf5acd47
Pass headers from StorageURL to WriteBufferFromHTTP (#46996) 
* Pass headers from StorageURL to WriteBufferFromHTTP

* Add a test

* Lint

* `time.sleep(1)`

* Start echo server earlier

* Add proper handling for mock server start

* Automatic style fix

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
 2023-03-03 13:55:52 +01:00
Kseniia Sumarokova
386663953c
Merge branch 'master' into named-collections-finish 2023-03-03 12:23:38 +01:00
Konstantin Morozov
3cb82bfe07 Update read and with ReadFromMemoryStorage 2023-03-03 06:41:48 +01:00
kssenii
cd7cd0526b Fix tests 2023-03-02 21:21:04 +01:00
Maksim Kita
3cd88003dd
Merge pull request #47101 from kitaisreal/analyzer-planner-fixes-before-enable-by-default 
Analyzer Planner fixes before enable by default
 2023-03-02 22:16:39 +03:00
Sema Checherinda
7a9f4aab8e do not call adjustWithUntrackedMemory when untracked_memory is 0 2023-03-02 19:21:03 +01:00
alesapin
63f615cdcd Fix metadata version 2023-03-02 19:17:18 +01:00
kssenii
0c4bb3c80f Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-03-02 19:08:29 +01:00
Azat Khuzhin
167e4903a8 Fix concrete columns PREWHERE support 
This is the fix for the IStorage::supportedPrewhereColumns() API.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-02 18:51:23 +01:00
Alexander Tokmakov
c518f3690e Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-02 18:24:08 +01:00
Anton Popov
fdf8a418f5 better interface 2023-03-02 14:46:26 +00:00
Antonio Andelic
01cf9c94f4 Merge branch 'master' into custom-key-parallel-replicas 2023-03-02 14:28:42 +00:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
kssenii
2883861795 Merge remote-tracking branch 'upstream/master' into allow-nested-disks-in-custom-disks 2023-03-02 14:30:58 +01:00
alesapin
915b4ea56a
Merge branch 'master' into revert-46909-revert-45911-mutations_rename_hang 2023-03-02 14:29:21 +01:00
alesapin
db0f9b6578 Fix one more bug 2023-03-02 14:26:07 +01:00
alesapin
189836307c Merge branch 'revert-46909-revert-45911-mutations_rename_hang' of github.com:ClickHouse/ClickHouse into revert-46909-revert-45911-mutations_rename_hang 2023-03-02 12:49:06 +01:00
Kruglov Pavel
ce8e49a9a0
Merge pull request #45693 from Avogar/fix-kafka-protobuf 
Fix formats parser resetting, test processing bad messages in kafka
 2023-03-02 11:50:45 +01:00
Alexey Milovidov
97c0f8a0e0
Merge pull request #47027 from amosbird/clang16 
Prepare to build with clang 16
 2023-03-02 02:10:03 +03:00
Anton Popov
c4a999ce29 Merge remote-tracking branch 'upstream/master' into HEAD 2023-03-01 19:33:16 +00:00
alesapin
46f25d53b3 Merge branch 'master' into revert-46909-revert-45911-mutations_rename_hang 2023-03-01 20:09:12 +01:00
kssenii
f3e3b916c9 Allow nested custom disks 2023-03-01 19:55:59 +01:00
Maksim Kita
d39be3ac9c Fixed tests 2023-03-01 18:05:07 +01:00
Maksim Kita
51ee007e01 Fixed tests 2023-03-01 18:05:07 +01:00
kssenii
80d017629a Fix test 2023-03-01 17:01:34 +01:00
alesapin
ebba37b18f
Merge pull request #47010 from ClickHouse/zero_copy_fetch 
Fix bug in zero-copy replication disk choice during fetch
 2023-03-01 15:55:00 +01:00
alesapin
b7370865a7
Merge pull request #47032 from ClickHouse/fix_message_for_zero_copy 
Better exception message when move was unseccessful
 2023-03-01 15:26:53 +01:00
Alexander Tokmakov
a97e15e36f Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-01 14:26:41 +01:00
Antonio Andelic
f540f7f6f9 Fix some tests 2023-03-01 12:45:00 +00:00
kssenii
4716273349 Fix test 2023-03-01 13:37:05 +01:00
alesapin
d72c310faa
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:54:22 +01:00
alesapin
8ff77c89eb
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:50:35 +01:00
alesapin
be5ef666ba
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:50:29 +01:00
alesapin
5de8a8e776
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2023-03-01 11:50:23 +01:00
Alexey Milovidov
fe446afb7d Increase max_replicated_merges_in_queue 2023-03-01 06:02:59 +01:00
Amos Bird
c0fb4ddc3e
Make thread analyzer happy 2023-03-01 11:07:36 +08:00
Azat Khuzhin
d06a4b50d6 Latest review fixes (variable naming: s/monitor/queue) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:48:07 +01:00
Azat Khuzhin
591fca57f3 Fix function names for opentelemetry spans in StorageDistributed 
Fixes: 02417_opentelemetry_insert_on_distributed_table
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
7063c20b3c Change noisy "Skipping send data over distributed table." message to test 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
16bfef3c8a Fix processing current_batch.txt on init 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
752d27d663 Fix lossing files during distributed batch send 
v2: do not suppress exceptions in case of errors
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
263c042c6a Fix opentelemetry for distributed batch sends 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
00115c6615 Rename readDistributedAsyncInsertHeader() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
a76d7b22c1 Use existing public methods of StorageDistributed in DistributedSink 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
e83699a8d3 Improve comment for DistributedAsyncInsertDirectoryQueue::initializeFilesFromDisk() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
e10fb142fd Fix race for distributed sends from disk 
Before it was initialized from disk only on startup, but if some INSERT
can create the object before, then, it will lead to the situation when
it will not be initialized.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
b5434eac3b Rename StorageDistributedDirectoryMonitor to DistributedAsyncInsertDirectoryQueue 
Since #44922 it is not a directory monitor anymore.

v2: Remove unused error codes
v3: Contains some header fixes due to conflicts with master
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
1c4659b8e7 Separate out Batch as DistributedAsyncInsertBatch (and also some helpers) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
33b13549ad Separate out DirectoryMonitorSource as DistributedAsyncInsertSource 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
325a7b2305 Separate out DistributedHeader as DistributedAsyncInsertHeader 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
22a39e29f7 Add a comment for StorageDistributedDirectoryMonitor::Batch::recovered 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
0c19a75a1c Add log message for batch restore 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
13a3e03f19 Introduce StorageDistributedDirectoryMonitor::Batch::{de,}serialize() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
f0a2efa630 Always manipulate with absolute file paths in DirectoryMonitor 
Otherwise on batch restore we can get the difference in file paths.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
ef1e642e05 Add log message to StorageDistributedDirectoryMonitor::addAndSchedule() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
16646c0923 Rename DirectoryMonitor::current_batch_file to current_file 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Azat Khuzhin
3f892e52ab Revert "Revert "Merge pull request #44922 from azat/dist/async-INSERT-metrics"" 
This is the revert of revert since there will be follow up patches to
address the issues.

This reverts commit a55798626a.
 2023-02-28 22:33:36 +01:00
alesapin
c0f28ace51 Better 2023-02-28 17:52:21 +01:00
alesapin
abd6500ee3 Better exception message when move was unseccessful 2023-02-28 17:49:16 +01:00
pufit
29296cb004
Merge pull request #46550 from ClickHouse/46231-systerm-server-settings 
Implement `system.server_settings`
 2023-02-28 11:23:15 -05:00
Antonio Andelic
45dc5dc25d No progress bar if no size 2023-02-28 15:25:28 +00:00
Antonio Andelic
d8b72e91ef Merge branch 'master' into fix-url-progress-bar 2023-02-28 14:45:45 +00:00
alesapin
bb7571427b Fix bug in zero-copy replica disk choice during fetch 2023-02-28 14:48:46 +01:00
Kruglov Pavel
a06596bdba
Fix special build 2023-02-28 14:33:41 +01:00
alesapin
712a8309c5
Merge pull request #46976 from ClickHouse/decrease-log-level 
Decrease log level in "disks"
 2023-02-28 12:28:30 +01:00
alesapin
967a47d870 Fix bug with readonly storages 2023-02-28 12:17:43 +01:00
kssenii
d4e6fc4546 Fix test 2023-02-27 21:43:45 +01:00
Nikolai Kochetov
738551dbf1
Merge pull request #46624 from CheSema/detached_mt 
add multitreading for StorageSystemDetachedParts
 2023-02-27 20:54:28 +01:00
Alexey Milovidov
29da7fc965 Decrease log level in "disks" 2023-02-27 17:48:38 +01:00
kssenii
ad88251ee7 Fix tests 2023-02-27 17:42:04 +01:00
Mike Kot
c799bd094b
Merge branch 'master' into feature/show-secrets-option 2023-02-27 18:43:14 +03:00
Anton Popov
7b85b35534 fix skip indexes 2023-02-27 15:11:25 +00:00
Anton Popov
caa1d22759
Merge branch 'master' into fix-issues-with-sparse 2023-02-27 15:25:19 +01:00
alesapin
7adbe93105 Remove strange part 2023-02-27 14:48:47 +01:00
Robert Schulze
6f0709032d
Merge pull request #46921 from ClickHouse/rs/inv-idx-checksum 
A better alternative to #46344
 2023-02-27 14:44:44 +01:00
kssenii
5418dd655e Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-02-27 14:35:30 +01:00
alesapin
9dec64c94c Choose correct version for mutation 2023-02-27 14:35:26 +01:00
Mike Kot
62acc7a6cf setting and some docs 2023-02-27 13:18:21 +00:00
Nikolai Kochetov
3392b96cc3
Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-27 14:07:04 +01:00
kssenii
b19264cf9f Remove redundant 2023-02-27 12:32:13 +01:00
alesapin
e3961c118a
Revert "Revert "Fix several RENAME COLUMN bugs."" 2023-02-27 12:27:57 +01:00
Robert Schulze
4c9b9b362c
Replace dynamic_cast by typeid_cast 2023-02-27 10:01:21 +00:00
zk_kiger
4f05eeeff9
Merge branch 'master' into improve_cluster_table_funcion_skip_unavailable_shards 2023-02-27 17:15:02 +08:00
Robert Schulze
cc0c0c6133
A better alternative to #46344 
The (experimental) inverted index writes/reads files different from the
standard files written by the other skip indexes. The original problem
was that with database engine "ordinary", DROP TABLE of a table with
inverted index finds unknown files in persistence and complains. The
same will happen with engine "atomic" but deferred. As a hotfix, the
error was silenced by explicitly adding the four files created in a
specific test to the deletion code.

This PR tries a cleaner solution where all needed files are provided via
the normal checksum structure. One drawback remains which is that the
affected files were written earlier and we don't have their checksums
available. Therefore, the inverted index is currently excluded from
CHECK TABLE.

Minimal repro:
  SET allow_experimental_inverted_index = 1;
  DROP TABLE IF EXISTS tab;
  CREATE TABLE tab(s String, INDEX af(s) TYPE inverted(2)) ENGINE = MergeTree() ORDER BY s;
  INSERT INTO tab VALUES ('Alick a01');
  CHECK TABLE tab;
  DROP TABLE IF EXISTS tab;

  run ./clickhouse-test with --db-engine Ordinary
 2023-02-26 21:06:38 +00:00
Alexey Milovidov
73aec86f5a Change exception message 2023-02-26 21:36:26 +01:00
Sema Checherinda
1666527dae do listing combined with getting size in one thread 2023-02-26 12:09:05 +01:00
Alexey Milovidov
d448a48f16
Merge pull request #46831 from azat/merge-default-prewhere 
Allow PREWHERE for Merge with different DEFAULT expression for column
 2023-02-26 10:02:34 +03:00
Robert Schulze
5e68d50b39
Merge pull request #46752 from ClickHouse/rs/periods-in-kafka-topics 
Allow configuration of Kafka topics with periods
 2023-02-26 07:49:28 +01:00
Alexey Milovidov
8f40bbd5b4
Revert "Fix several RENAME COLUMN bugs." 2023-02-26 05:47:05 +03:00
Alexey Milovidov
eb89ab471f
Merge pull request #45911 from ClickHouse/mutations_rename_hang 
Fix several `RENAME COLUMN` bugs.
 2023-02-26 01:56:40 +03:00
Alexey Milovidov
656af55831
Merge pull request #46384 from ClickHouse/fixes_for_993 
Fixes for 993
 2023-02-26 01:55:20 +03:00
Robert Schulze
fbba9e319d
Merge branch 'master' into rs/periods-in-kafka-topics 2023-02-25 21:45:20 +01:00
Robert Schulze
3618e37dd2
Reduce indentation by early out 2023-02-25 15:36:59 +00:00
Sema Checherinda
0c9643f1ed make listing in parrallel as well 2023-02-25 16:27:27 +01:00
Nikolai Kochetov
57b5f9e7d7 Fix case with no detached parts for table. 2023-02-25 13:33:20 +01:00
Sema Checherinda
e2bfa83429 cancel workers when exception 2023-02-25 13:33:20 +01:00
Sema Checherinda
08dc874a37 do not run support threads if no tasks left 2023-02-25 13:33:20 +01:00
Sema Checherinda
e5cbe4311e create limites count of support threads for a block 2023-02-25 13:33:20 +01:00
Sema Checherinda
ef0c1841af work with comments on review 2023-02-25 13:33:20 +01:00
Sema Checherinda
3b58249320 pack several files in one thread 2023-02-25 13:33:20 +01:00
Sema Checherinda
8abc1f0d5d fix build and style 2023-02-25 13:33:20 +01:00
Sema Checherinda
dce319e1c5 add multitreading for StorageSystemDetachedParts 2023-02-25 13:33:20 +01:00
Alexander Tokmakov
cad1e0b768 fix 2023-02-25 01:18:34 +01:00
kssenii
c2bcc4119f Better 2023-02-24 20:57:37 +01:00
kssenii
96b57c8a65 Better support for equal keys 2023-02-24 20:38:40 +01:00
Anton Popov
3a947ecddc do not allow const and non-deterministic secondary indexes 2023-02-24 19:17:44 +00:00
Alexander Tokmakov
ca1c793cac
Merge pull request #46820 from Algunenano/avoid_KeeperMultiException_system_errors 
Don't report ZK node exists to system.errors when a block was created concurrently by a different replica
 2023-02-24 19:42:32 +03:00
kssenii
6224ca9cd6 Fix 2023-02-24 17:08:05 +01:00
Robert Schulze
6402050125
Fixup comments 2023-02-24 16:59:38 +01:00
Azat Khuzhin
86e9b131c7 Allow PREWHERE for Merge with different DEFAULT expression for column 
It can be not that special to have Merge engine w/o default expression,
while MergeTree will have it. So improve PREWHERE for such kinds, and
also allow PREWHERE for DEFAULT and MATERIALIZED (since both are
physical columns on disk and this make sense).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-24 16:33:49 +01:00
Antonio Andelic
56a126f7af Fix progress bar with URL 2023-02-24 14:49:14 +00:00
kssenii
f2a9109fe1 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-02-24 15:20:46 +01:00
Alexander Gololobov
b43b98f7c0
Merge pull request #46785 from ClickHouse/fix_combined_prewhere_column 
Fix combined PREWHERE column accumulated from multiple steps
 2023-02-24 14:00:17 +01:00
Kruglov Pavel
2150b836b1
Merge branch 'master' into fix-kafka-protobuf 2023-02-24 12:48:07 +01:00
Raúl Marín
7ff8d2d3d5 Don't report ZK node exists to system.errors 2023-02-24 12:36:29 +01:00
pufit
2fbe75957d Add default column for system.settings and system.server_settings 2023-02-23 18:24:57 -05:00
Anton Popov
d926713cf5 Merge remote-tracking branch 'upstream/master' into HEAD 2023-02-23 23:04:22 +00:00
Anton Popov
5e5e802348 return back optimization for ALTER MODIFY Nullable 2023-02-23 22:55:47 +00:00
Mikhail f. Shiryaev
18a20268d8
Merge pull request #46792 from ClickHouse/23.3-prepare 
Update version after release
 2023-02-23 23:51:28 +01:00
Alexander Tokmakov
fe92fd8a61 Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-02-23 22:35:31 +01:00
Robert Schulze
6906cafa99
Change to S3 cfg syntax 2023-02-23 20:04:41 +00:00
Antonio Andelic
1efba8995d Update version to 23.3.1.2537 2023-02-23 19:35:57 +00:00
Alexander Gololobov
47f51fe187
Merge pull request #46771 from ClickHouse/fix_prewhere_logging 
More concise logging at trace level for PREWHERE steps
 2023-02-23 20:20:30 +01:00
Alexander Gololobov
8b22373488 Apply filter at last step if it has been accumulated from multiple steps 2023-02-23 16:58:28 +01:00
Alexander Gololobov
cbcb1a7852 More concise logging at trace level 2023-02-23 13:31:56 +01:00
Kseniia Sumarokova
7e064ccbd4
Merge pull request #46667 from kssenii/hide-named-collections-under-setting 
Allow to hide only values from system.named_collections while making collection visible
 2023-02-23 11:44:50 +01:00
Kseniia Sumarokova
30a8eb0c2f
Merge pull request #46670 from kssenii/hide-disk-setting 
Hide arguments of custom disk merge tree setting
 2023-02-23 11:42:53 +01:00
kigerzhang
f74f3718a0 add skip_unavailable_shards setting for table function cluste 2023-02-23 17:05:51 +08:00
Alexey Milovidov
c0bc549e77
Merge pull request #46709 from azat/bg-pool-no-query 
BackgroundSchedulePool should not have any query context
 2023-02-23 10:24:46 +03:00
Alexey Milovidov
e3f24b2b68
Merge pull request #46744 from ClickHouse/remove_unused_field 
Remove unused MergeTreeReadTask::remove_prewhere_column
 2023-02-23 08:02:54 +03:00
Robert Schulze
81bf43157f
Allow configuration of Kafka topics with periods 
The Kafka table engine allows global configuration and per-Kafka-topic
configuration. The latter uses syntax <kafka_TOPIC>, e.g. for topic
"football":

  <kafka_football>
      <retry_backoff_ms>250</retry_backoff_ms>
      <fetch_min_bytes>100000</fetch_min_bytes>
  </kafka_football>

Some users had to find out the hard way that such configuration doesn't
take effect if the topic name contains a period, e.g. "sports.football".
The reason is that ClickHouse configuration framework already uses
periods as level separators to descend the configuration hierarchy.
(Besides that, per-topic configuration at the same level as global
configuration could be considered ugly.)

Note that Kafka topics may contain characters "a-zA-Z0-9._-" (*) and
a tree-like topic organization using periods is quite common in
practice.

This PR deprecates the existing per-topic configuration syntax (but
continues to support it for backward compat) and introduces a new
per-topic configuration syntax below the global Kafka configuration of
the form:

<kafka>
   <topic name="football">
       <retry_backoff_ms>250</retry_backoff_ms>
       <fetch_min_bytes>100000</fetch_min_bytes>
   </topic>
</kafka>

The period restriction doesn't apply to XML attributes, so <topic
name="sports.football"> will work. Also, everything Kafka-related is
below <kafka>.

Considered but rejected alternatives:
- Extending Poco ConfigurationView with custom separators (e.g."/"
  instead of "."). Won't work easily because ConfigurationView only
  builds a path but defers descending the configuration tree to the
  normal configuration classes.
- Reloading the configuration file in StorageKafka (instead of reading
  the loaded file) but with a custom separator. This mode is supported
  by XML configuration. Too ugly and error-prone since the true
  configuration is composed from multiple configuration files.

(*) https://stackoverflow.com/a/37067544
 2023-02-22 20:35:09 +00:00
Alexander Tokmakov
e660c0838c fix multipart requests 2023-02-22 17:54:35 +01:00
Alexander Gololobov
67dcd9694c Remove unused MergeTreeReadTask::remove_prewhere_column 2023-02-22 17:49:22 +01:00
pufit
8e7533fa57
Merge pull request #46564 from AVMusorin/update-time-distribution-queue 
Added `last_exception_time` column into distribution_queue table
 2023-02-22 10:43:35 -05:00
kssenii
c4761d6cc6 Fix checks 2023-02-22 14:27:43 +01:00
kssenii
f0ca3ef207 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-02-22 14:02:29 +01:00
kssenii
f518ddd96d Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-02-22 13:43:18 +01:00
Kruglov Pavel
3ba3fdbfa3
Merge pull request #46607 from kssenii/delay-loading-of-named-collections 
Do not load named collections on server startup (on first access instead)
 2023-02-22 13:22:34 +01:00
Azat Khuzhin
2ca47a6eb6 BackgroundSchedulePool should not have any query context 
BackgroundSchedulePool is used for some peridic jobs, not from the query
context, i.e. flush of Buffer table.

And for such jobs there cannot be any query context, and more
importantly it will not work correctly since that query_context will
eventually expires.

And this is the reason of this failures [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/46668/015991bc5e20c787851050c2eaa13f0fef3aac00/stateless_tests_flaky_check__asan_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-22 10:50:51 +01:00
Alexander Tokmakov
592af6d652 fix incomplete interst through http 2023-02-22 02:34:03 +01:00
AVMusorin
0bf0fe488e
added last_exception_time column into distribution_queue table 2023-02-21 23:46:57 +01:00
Alexander Tokmakov
f5c232707d cancel insert queries correctly 2023-02-21 22:04:39 +01:00
Alexander Gololobov
b61bb56a5a
Merge pull request #46611 from ClickHouse/prewhere_progress 
Fix reporting progress when rows are filtered in PREWHERE
 2023-02-21 21:27:56 +01:00
kssenii
b15be14792 Fix table function remote 2023-02-21 18:55:43 +01:00
kssenii
9a7c71b78e Allow to hide only values from system.named_collections 2023-02-21 18:07:57 +01:00
Alexey Milovidov
848286eca7
Merge pull request #40945 from arthurpassos/add_select_final_mt_setting 
Add user setting to force select final at query level
 2023-02-21 19:58:45 +03:00
Nikolai Kochetov
c5f93eb108 Fix more tests. 2023-02-21 15:44:50 +00:00
kssenii
d84215d005 Fix style check 2023-02-21 16:20:57 +01:00
Kseniia Sumarokova
655e7f29f7
Merge pull request #46595 from kssenii/update-rabbit-producer 
Fix rabbitmq test
 2023-02-21 16:18:55 +01:00
Anton Popov
b66a5bdcf0 fix mutations with sparse columns 2023-02-21 15:13:29 +00:00
kssenii
b3ca976f5f Finish for streaming engines 2023-02-21 14:50:55 +01:00
Alexander Gololobov
5354b0d069
Merge pull request #46140 from ClickHouse/split_prewhere_dag 
Split PREWHERE DAG into multiple read steps
 2023-02-21 14:37:09 +01:00
kssenii
68e06ecb99 Replace for table function remote, and external storage 2023-02-21 14:33:37 +01:00
Kseniia Sumarokova
09a979f106
Merge branch 'master' into update-rabbit-producer 2023-02-21 11:50:30 +01:00
Vladimir C
a791250a4a
Merge pull request #38614 from bharatnc/ncb/profile-events-partlog 2023-02-21 11:06:44 +01:00
Kseniia Sumarokova
24d0490ef3
Merge branch 'master' into delay-loading-of-named-collections 2023-02-21 00:24:18 +01:00
Kruglov Pavel
828a06b511
Merge branch 'master' into fix-kafka-protobuf 2023-02-20 21:45:43 +01:00
Kruglov Pavel
b0424c1021
Merge pull request #46171 from Avogar/insert-null-as-default 
Use default of column type in `insert_null_as_default` if column DEFAULT values is not specified
 2023-02-20 21:45:02 +01:00
kssenii
a54b011670 Finish for mysql 2023-02-20 21:37:38 +01:00
Nikolai Kochetov
ee597b939f Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-20 19:46:52 +00:00
Arthur Passos
4f2fcf763b
Merge branch 'master' into add_select_final_mt_setting 2023-02-20 16:22:14 -03:00
Kseniia Sumarokova
47604ad77e
Merge pull request #46106 from kssenii/return-materialized-postrgesql-tests-back 
Enable disabled MaterializedPostgreSQL tests back in CI
 2023-02-20 19:04:58 +01:00
Alexander Gololobov
c84a64d22f Return chunks with 0 rows from MergeTreeSource to report progress when rows are filtered in PREWHERE 2023-02-20 17:35:01 +01:00
kssenii
9d16205c8a Load named collections on first access 2023-02-20 16:33:45 +01:00
Nikolai Kochetov
84e442e620 Fixing build and style. 2023-02-20 14:21:20 +00:00
kssenii
80a0aa1327 Merge remote-tracking branch 'upstream/master' into return-materialized-postrgesql-tests-back 2023-02-20 14:37:39 +01:00
kssenii
361678ad73 rabbitmq-test-fix 2023-02-20 14:16:40 +01:00
Nikolai Kochetov
2a8460d962 Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-02-20 13:54:05 +01:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Arthur Passos
3d9b23e5f3
Merge branch 'master' into add_select_final_mt_setting 2023-02-19 15:01:41 -03:00
Alexey Milovidov
5359dcc014
Merge pull request #46554 from ucasfl/iceberg-fix 
fix build without avro
 2023-02-19 14:35:29 +03:00
Alexander Gololobov
520b381675 Fixes for executing actions on block with unknown row count 2023-02-19 12:14:23 +01:00
Maksim Kita
6dffe52e68
Merge pull request #46471 from kitaisreal/analyzer-planner-fixes-before-enable-by-default 
Analyzer planner fixes before enable by default
 2023-02-19 13:49:18 +03:00
flynn
7b4fbf33b3 fix 2023-02-19 03:47:14 +00:00
Alexander Gololobov
0ed0b191f8 Fix for Nullable step filter column 2023-02-18 19:12:32 +01:00
flynn
177261a5ac fix typo 2023-02-18 17:34:37 +00:00
flynn
830398f0bd fix build without avro 
fix
 2023-02-18 17:32:35 +00:00
Alexander Gololobov
88fb6c3c25 Fix for incorrect combined PREWHERE column 2023-02-18 18:29:16 +01:00
Maksim Kita
cbd961de98 Fixed code review issues 2023-02-18 17:06:00 +01:00
Nikolay Degterinsky
f93aae80b8
Merge pull request #46407 from evillique/parameters-rename-query 
Add parameterized RENAME queries
 2023-02-18 16:04:55 +01:00
Kseniia Sumarokova
b0ea047e76
Merge pull request #46533 from kssenii/fix-disk-setting 
Allow `disk` merge tree setting in query to override config merge tree setting `storage_policy`
 2023-02-18 12:36:54 +01:00
pufit
4496db66b2 Implement system.server_settings 2023-02-17 23:46:09 -05:00
Kseniia Sumarokova
bda4c94f66
Merge pull request #45384 from ucasfl/iceberg 2023-02-17 23:38:24 +01:00