thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-12 17:32:32 +00:00
Code Issues Packages Projects Releases Wiki Activity
128,960 Commits 9,328 Branches 1,783 Tags 2.1 GiB
3319934055
Commit Graph

15015 Commits

Author SHA1 Message Date
Alexander Sapin
c3dcb81a62 Fix race condition in case of memory errors 2023-11-03 16:00:16 +01:00
Alexander Sapin
a9ffb9808e Merge branch 'master' into fix_obscure_segfault 2023-11-03 12:20:52 +01:00
Kseniia Sumarokova
31ae602d37
Merge pull request #56131 from ClickHouse/backup-some-more-retries 
Add missing zookeeper retries in StorageReplicatedMergeTree::backupData
 2023-11-03 12:00:41 +01:00
Alexander Tokmakov
b50c4c9f3b fix 2023-11-03 11:24:10 +01:00
vdimir
86524fce0f
Merge pull request #56246 from ClickHouse/vdimir/analyzer_01019_alter_materialized_view 
Analyzer: fix 01019_alter_materialized_view_consistent
 2023-11-03 11:04:14 +01:00
vdimir
a70d0c5203
Merge pull request #56264 from azat/rocksdb/block-based-table-options 
Add ability to specify BlockBasedTableOptions for rocksdb
 2023-11-03 10:15:19 +01:00
vdimir
55a1baf029
Merge pull request #56022 from ClickHouse/vdimir/check_all_tables 
Add CHECK query for all the tables
 2023-11-03 10:10:42 +01:00
Alexander Tokmakov
480e58b427 fix unexpected parts after drop range 2023-11-03 01:47:50 +01:00
alesapin
2daff2d41e Fxi 2023-11-03 00:32:07 +01:00
Alexey Milovidov
b400a6682e
Merge pull request #56275 from ClickHouse/23.11-prepare 
Update version after release
 2023-11-02 20:20:03 +01:00
Alexey Milovidov
1348d4c696 Update autogenerated version to 23.11.1.1 and contributors 2023-11-02 20:18:15 +01:00
Alexander Tokmakov
e88fa10aa2
Merge pull request #56142 from ClickHouse/make_replicated_start_more_robust 
Better process broken parts on table start for replicated tables
 2023-11-02 20:11:22 +01:00
Alexander Tokmakov
0ce1560e5d retry if table replica being restarted 2023-11-02 18:30:32 +01:00
Robert Schulze
99dd3ed060
Merge pull request #56226 from HarryLeeIBM/hlee-ftsearch-compress 
Add compression of posting lists for inverted index
 2023-11-02 15:56:30 +01:00
Azat Khuzhin
decf85baca Add ability to specify BlockBasedTableOptions for rocksdb 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-02 15:36:00 +01:00
Sema Checherinda
90b64bcdb9 backward compatibility and implementation feature storage_metadata_write_full_object_key 2023-11-02 15:20:55 +01:00
kssenii
d0bc77f084 Fxi 2023-11-02 14:45:15 +01:00
Joe Lynch
67e29d5eb9
Allow repeated adds and removes for delta lake files 2023-11-02 13:40:50 +01:00
alesapin
3ba12b6e3e
Merge pull request #56238 from evillique/remove-less-parts 
Do not remove part if `Too many open files` is thrown
 2023-11-02 12:33:02 +01:00
Alexey Milovidov
1267246785
Merge pull request #55642 from ClickHouse/morton-utils 
Indexing by space-filling curves
 2023-11-02 12:21:33 +01:00
alesapin
5c4915e264
Update src/Storages/MergeTree/MergeTreePartInfo.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-11-02 12:02:04 +01:00
alesapin
7917a2cb40
Update src/Storages/MergeTree/MergeTreePartInfo.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-11-02 12:01:58 +01:00
alesapin
800f2f097c
Update src/Storages/MergeTree/MergeTreePartInfo.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-11-02 12:01:51 +01:00
alesapin
81b93ef12b
Update src/Storages/MergeTree/MergeTreePartInfo.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-11-02 12:01:46 +01:00
vdimir
5703e0995b
Analyzer: fix 01019_alter_materialized_view_consistent 2023-11-02 10:27:22 +00:00
Nikolay Degterinsky
0d6a0d04ae Do not remove parts if 'Too many open files' is thrown 2023-11-02 05:01:53 +00:00
Nikolay Degterinsky
f93c032425 Fix suggestions for user without grants 2023-11-02 03:56:59 +00:00
Nikolay Degterinsky
1af52d3b00 Better 2023-11-02 00:42:57 +00:00
Igor Nikonov
823b62a55c
Merge pull request #56162 from ClickHouse/part-check-scheduling-monotonic-time 
Use monotonic clock for part check scheduling
 2023-11-01 23:15:27 +01:00
HarryLeeIBM
013e4b40c6 Add compression of posting lists for inverted index 2023-11-01 14:14:38 -07:00
Igor Nikonov
447386edd9
Merge branch 'master' into fix-schedule-delayed-part-checks 2023-11-01 21:26:06 +01:00
Duc Canh Le
7d4c97e8f3
print broken part path before throw (#56181) 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-11-01 19:11:23 +01:00
Alexander Sapin
1e0cab93f6 Satisfy tidy 2023-11-01 16:51:20 +01:00
Alexander Sapin
bfcd8b7de5 Better logging and tests 2023-11-01 16:50:02 +01:00
Azat Khuzhin
c25d6cd624
Rename directory monitor concept into background INSERT (#55978) 
* Limit log frequence for "Skipping send data over distributed table" message

After SYSTEM STOP DISTRIBUTED SENDS it will constantly print this
message.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename directory monitor concept into async INSERT

Rename the following query settings (with preserving backward
compatiblity, by keeping old name as an alias):
- distributed_directory_monitor_sleep_time_ms -> distributed_async_insert_sleep_time_ms
- distributed_directory_monitor_max_sleep_time_ms -> distributed_async_insert_max_sleep_time_ms
- distributed_directory_monitor_batch -> distributed_async_insert_batch_inserts
- distributed_directory_monitor_split_batch_on_failure -> distributed_async_insert_split_batch_on_failure

Rename the following table settings (with preserving backward
compatiblity, by keeping old name as an alias):
- monitor_batch_inserts -> async_insert_batch
- monitor_split_batch_on_failure -> async_insert_split_batch_on_failure
- directory_monitor_sleep_time_ms -> async_insert_sleep_time_ms
- directory_monitor_max_sleep_time_ms -> async_insert_max_sleep_time_ms

And also update all the references:

    $ gg -e directory_monitor_ -e monitor_ tests docs | cut -d: -f1 | sort -u | xargs sed -e 's/distributed_directory_monitor_sleep_time_ms/distributed_async_insert_sleep_time_ms/g' -e 's/distributed_directory_monitor_max_sleep_time_ms/distributed_async_insert_max_sleep_time_ms/g' -e 's/distributed_directory_monitor_batch_inserts/distributed_async_insert_batch/g' -e 's/distributed_directory_monitor_split_batch_on_failure/distributed_async_insert_split_batch_on_failure/g' -e 's/monitor_batch_inserts/async_insert_batch/g' -e 's/monitor_split_batch_on_failure/async_insert_split_batch_on_failure/g' -e 's/monitor_sleep_time_ms/async_insert_sleep_time_ms/g' -e 's/monitor_max_sleep_time_ms/async_insert_max_sleep_time_ms/g' -i

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename async_insert for Distributed into background_insert

This will avoid amigibuity between general async INSERT's and INSERT
into Distributed, which are indeed background, so new term express it
even better.

Mostly done with:

    $ git di HEAD^ --name-only | xargs sed -i -e 's/distributed_async_insert/distributed_background_insert/g' -e 's/async_insert_batch/background_insert_batch/g' -e 's/async_insert_split_batch_on_failure/background_insert_split_batch_on_failure/g' -e 's/async_insert_sleep_time_ms/background_insert_sleep_time_ms/g' -e 's/async_insert_max_sleep_time_ms/background_insert_max_sleep_time_ms/g'

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Mark 02417_opentelemetry_insert_on_distributed_table as long

CI: https://s3.amazonaws.com/clickhouse-test-reports/55978/7a6abb03a0b507e29e999cb7e04f246a119c6f28/stateless_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-11-01 15:09:39 +01:00
Alexander Tokmakov
65f3cf81dc
Merge pull request #55987 from ClickHouse/fix_lost_part_count 
Avoid counting lost parts twice
 2023-11-01 15:07:27 +01:00
Kruglov Pavel
5533258091
Merge pull request #55477 from Avogar/streaming-errors 
Allow to save unparsed records and errors in RabbitMQ, NATS and FileLog engines
 2023-11-01 14:35:35 +01:00
vdimir
2ca1a14ac4
Merge pull request #55741 from ClickHouse/vdimir/fix_analyzer_materialized_distributed_join 
Analyzer: fix block stucture mismatch in matview with engine distributed
 2023-11-01 14:16:33 +01:00
Kseniia Sumarokova
4510014014
Update StorageReplicatedMergeTree.cpp 2023-11-01 14:05:12 +01:00
Kseniia Sumarokova
c300297153
Fix style check 2023-11-01 13:27:38 +01:00
Alexander Sapin
b4328ad8dc Fix stupid bug 2023-11-01 13:06:19 +01:00
kssenii
68fd273114 Better 2023-11-01 11:18:58 +01:00
jsc0218
f1c7665feb
Merge pull request #56121 from lithiumlabcompany/fix-auto-configuration-queue-for-librdkafka 
Correctly set queued.min.messages
 2023-10-31 19:59:04 -04:00
Michael Kolupaev
9c79c165b9
Re-fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table (#56140) 
* Fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table

* Fix the test
 2023-10-31 13:41:54 -07:00
Alexander Sapin
3a1182f6c0 Proper implementation 2023-10-31 18:53:33 +01:00
kssenii
ae1a7fffec Fix 2023-10-31 16:19:08 +01:00
Alexander Tokmakov
f39f0c6a74
Merge pull request #56083 from ClickHouse/fix_drop_range_commit_existing_part_race 
Fix race condition between DROP_RANGE and committing existing block
 2023-10-31 14:24:53 +01:00
Alexander Sapin
f4081e3f85 Merge branch 'master' into make_replicated_start_more_robust 2023-10-31 14:16:39 +01:00
Alexander Tokmakov
4d437ca462
Merge pull request #56137 from ClickHouse/fix_unexpected_parts_handling 
Do not try to activate covered parts when handilng unexpected parts
 2023-10-31 14:00:47 +01:00
Igor Nikonov
f8a7bdb8f0 Use monotonic time for part checks scheduling 2023-10-31 11:21:20 +00:00
Alexey Milovidov
016285ea87
Merge branch 'master' into morton-utils 2023-10-31 01:47:16 +01:00
alesapin
20b8954872 Better process broken parts on table start for replicated tables 2023-10-30 23:16:04 +01:00
Alexander Tokmakov
373d8bbb20 do not try to activate covered parts 2023-10-30 20:17:10 +01:00
Igor Nikonov
0958d0b168 Fix style 2023-10-30 16:35:00 +00:00
Nikolai Kochetov
cc3c038394 Fixing test. 2023-10-30 16:29:22 +00:00
kssenii
6fc4c8d332 Add retries 2023-10-30 17:20:17 +01:00
Alexander Tokmakov
ae8d7a557b
Update ReplicatedMergeTreeSink.cpp 2023-10-30 15:43:48 +01:00
Alexander Tokmakov
073a6a6f0e
Update src/Storages/StorageReplicatedMergeTree.cpp 
Co-authored-by: Raúl Marín <git@rmr.ninja>
 2023-10-30 15:33:35 +01:00
Alexander Tokmakov
6635b84a46 fixes 2023-10-30 15:20:04 +01:00
Igor Nikonov
f2acc366f7 Fix: schedule delayed part checks correctly 2023-10-30 12:57:43 +00:00
Nikolai Kochetov
554ceb4e1d Merge branch 'master' into planner-prepare-filters-for-analysis-2 2023-10-30 11:56:30 +01:00
Stas Morozov
5a922014f7
Avoid problem with configuration queued.min.messages automatically for Kafka storage 2023-10-30 13:53:52 +03:00
Kruglov Pavel
e22cfa3d5b
Remove unneeded changge 2023-10-30 11:35:44 +01:00
Kruglov Pavel
53b6a455fc
Remove unused field from .h file 2023-10-30 11:35:09 +01:00
Kruglov Pavel
ba736beaf2
Remove unused field 2023-10-30 11:34:42 +01:00
Alexey Milovidov
3631e476eb
Merge pull request #54947 from amosbird/minmax-combinator 
Introduce -ArgMin/-ArgMax combinators.
 2023-10-30 03:35:28 +01:00
Alexey Milovidov
64b6e68a50
Merge pull request #55683 from amosbird/issue-55653 
Reuse granule during skip index reading
 2023-10-30 00:51:51 +01:00
Alexey Milovidov
8e6a7fdff0 Fix f-string 2023-10-29 23:43:36 +01:00
Alexey Milovidov
9aff0a8872 Revert "Fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table (#55995)" 
This reverts commit b65c498016.
 2023-10-29 04:34:04 +03:00
Alexey Milovidov
1b085b1161
Merge pull request #55900 from ClickHouse/make-test-better 
Make test `system_zookeeper_connection` better
 2023-10-28 15:01:36 +02:00
Michael Kolupaev
b65c498016
Fix 'Block structure mismatch' on concurrent ALTER and INSERTs in Buffer table (#55995) 2023-10-27 10:06:43 -07:00
Alexander Tokmakov
e07bb123d3 fix race condition between DROP_RANGE and committing existing block 2023-10-27 17:50:45 +02:00
Alexander Tokmakov
db5e9f1d1c beter check for duplicate lost parts 2023-10-27 17:33:30 +02:00
vdimir
faffdba8a2
Split source and worker processors in InterpreterCheckQuery 2023-10-27 15:22:10 +00:00
vdimir
c1ae369919
wip CHECK query for all tables 2023-10-27 15:22:10 +00:00
Alexander Tokmakov
9db5f4396c Merge branch 'master' into fix_lost_part_count 2023-10-26 17:40:37 +02:00
Andrey Zvonov
5737369d0b
Merge branch 'ClickHouse:master' into zvonand-datetime-ranges 2023-10-26 11:42:13 +02:00
vdimir
12890eb70c
Merge pull request #55806 from canhld94/rocksdb_trivial_count 
Support trivial count for rocksdb and storage join
 2023-10-26 11:22:38 +02:00
vdimir
1d5f75fd71
Merge pull request #53404 from ClickHouse/vdimir/check_table_improvements2 
Improve CHECK TABLE system query, support checking part
 2023-10-26 10:57:44 +02:00
Nikolay Degterinsky
3c690337ec Improvements 2023-10-26 03:17:30 +00:00
zvonand
0766c73aab Rename date_time_overflow_mode -> date_time_overflow_behavior, moved it to format settings 2023-10-25 23:11:13 +02:00
Igor Nikonov
1a5d485c6c
Merge branch 'master' into pr-progress-bar 2023-10-25 19:53:20 +02:00
Alexander Tokmakov
3c0b8aeb6a avoid counting lost parts twice 2023-10-25 19:29:40 +02:00
Alexander Tokmakov
c0482cb537
Merge pull request #55944 from azat/rmt-retriable-exception-log 
Do not write retriable errors for Replicated mutate/merge into error log
 2023-10-25 18:25:59 +02:00
alesapin
0d68a52c06
Merge pull request #50181 from ClickHouse/repro_49887 
Make exception about `ALTER TABLE ... DROP COLUMN|INDEX|PROJECTION` more clear
 2023-10-25 15:57:17 +02:00
Nikolay Degterinsky
2606b60e66 Fix REPLICA_ALREADY_EXISTS for ReplicatedMergeTree 2023-10-25 04:17:35 +00:00
Igor Nikonov
7b7f100250 Merge remote-tracking branch 'origin/master' into pr-progress-bar 2023-10-24 21:02:29 +00:00
李扬
465962df7f
Support orc filter push down (file + stripe + rowgroup level) (#55330) 
* support orc filter push down

* update orc lib version

* replace setqueryinfo with setkeycondition

* fix issue https://github.com/ClickHouse/ClickHouse/issues/53536

* refactor source with key condition

* fix building error

* remove std::cout

* update orc

* update orc version

* fix bugs

* improve code

* upgrade orc lib

* fix code style

* change as requested

* add performance tests for orc filter push down

* add performance tests for orc filter push down

* fix all bugs

* fix default as null issue

* add uts for null as default issues

* upgrade orc lib

* fix failed orc lib uts and fix typo

* fix failed uts

* fix failed uts

* fix ast fuzzer tests

* fix bug of uint64 overflow in https://s3.amazonaws.com/clickhouse-test-reports/55330/de22fdcaea2e12c96f300e95f59beba84401712d/fuzzer_astfuzzerubsan/report.html

* fix asan fatal caused by reused column vector batch in native orc input format. refer to https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__asan__[4_4].htm

* fix wrong performance tests

* disable 02892_orc_filter_pushdown on aarch64. https://s3.amazonaws.com/clickhouse-test-reports/55330/be39d23af2d7e27f5ec7f168947cf75aeaabf674/stateless_tests__aarch64_.html

* add some comments

* add some comments

* inline range::equals and range::less

* fix data race of key condition

* trigger ci
 2023-10-24 12:08:17 -07:00
Kruglov Pavel
cb63b07e89
Merge pull request #55912 from wxybear/fix/storage_null_supports_subcolumns 
fix: StorageNull supports subcolumns
 2023-10-24 14:03:46 +02:00
Alexander Gololobov
04b82d6463
Merge pull request #55935 from ClickHouse/set_has_lwd 
Set storage.has_lightweight_delete_parts flag when a part has been loaded
 2023-10-24 14:01:11 +02:00
Azat Khuzhin
da408df4a7 Proper ignoring of retriable errors during merge/mutate 
The exception cannot be simply suppressed, since sanity checks in the
WriteBuffer dtor relies on the std::uncaught_exceptions(), and if the
exception will be suppressed, then if the buffer was not finalized, it
may abort (even though it is only in debug/sanitizers build).

So instead, IExecutableTask::printExecutionException() had been
introduced to distinguish when exception should be printed and when not.
 2023-10-24 10:19:18 +02:00
Duc Canh Le
e3aa799ca2 disable approximate count by default + fix tests 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-24 03:18:33 +00:00
Srikanth Chekuri
8876a92b42 Merge remote-tracking branch 'upstream/master' into issue-55023 2023-10-23 21:21:25 +05:30
Robert Schulze
0703c88381
Merge pull request #55915 from rschu1ze/sanity-check-rpnbuildertreenode 
Add sanity check to RPNBuilderFunctionTreeNode
 2023-10-23 16:09:56 +02:00
Robert Schulze
601276aa60
Update src/Storages/MergeTree/RPNBuilder.cpp 
Co-authored-by: Igor Nikonov <954088+devcrafter@users.noreply.github.com>
 2023-10-23 15:23:23 +02:00
Azat Khuzhin
38a4547c05 Do not write retriable errors for Replicated mutate/merge into error log 
Fixes: e3f892f84a ("fix gtest with MemoryWriteBuffer, do not mute exception in ReplicatedMergeMutateTaskBase")
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-23 15:14:58 +02:00
Igor Nikonov
0bbf7ba921
Merge branch 'master' into pr-progress-bar 2023-10-23 14:56:59 +02:00
vdimir
7583394c9c
Support CHECK TABLE ... PART ... query 2023-10-23 12:33:35 +00:00
Kruglov Pavel
32da07b4ee
Merge pull request #55695 from Avogar/restrict-iceberg 
Fixes and improvements for Iceberg storage
 2023-10-23 13:58:23 +02:00
Alexander Gololobov
99a14410ee Set storage.has_lightweight_delete_parts flag when a part has been loaded 2023-10-23 13:36:27 +02:00
vdimir
8f0d7954ff
IStorage::checkDataNext returns optional 2023-10-23 10:12:30 +00:00
Robert Schulze
5cf3c95ea4
Update RPNBuilder.cpp 2023-10-23 11:49:52 +02:00
vdimir
fe95c0d0e4
Improve CHECK TABLE system query 
Resubmit PR #52745
 2023-10-23 09:35:26 +00:00
Robert Schulze
77e7f71e02
Add sanity check to RPNBuilderFunctionTreNode 2023-10-22 19:56:05 +00:00
wxybear
53a49164d6 fix: StorageNull supports subcolumns 2023-10-22 16:24:01 +08:00
Srikanth Chekuri
327386a348 Merge remote-tracking branch 'upstream/master' into issue-55023 2023-10-22 10:22:14 +05:30
Alexey Milovidov
2972dbdc40 Merge branch 'master' into morton-utils 2023-10-22 03:52:30 +02:00
Alexey Milovidov
2b72430e24 Make test system_zookeeper_connection better 2023-10-21 23:56:20 +02:00
Srikanth Chekuri
852d627f6a Introduce setting create_table_empty_primary_key_by_default 
If set to true and the table of MergeTree family does not contain
ORDER BY and PRIMARY KEY, then an empty tuple is used by default
 2023-10-21 21:37:04 +05:30
Igor Nikonov
e378d920c4
Merge branch 'master' into pr-progress-bar 2023-10-21 13:16:12 +02:00
Anton Popov
5819bcd07a
Support asynchronous inserts for native protocol (#54730) 
* support async insert for native protocol

* use separate queue for async inserts via native protocol

* fix test

* better logging for async inserts and more tests

* disable mixed internal and external data in async inserts

* fix tests

* fix quota in async inserts

* disable async insert for secondary query of distributed
 2023-10-20 18:39:48 +02:00
Raúl Marín
d08ce13107
Merge pull request #51692 from Algunenano/parallel_replicas_row_estimation 
Automatic decision of nº of parallel replicas
 2023-10-20 15:50:43 +02:00
alesapin
c3c62d90c8
Update MergeTreeData.cpp 2023-10-20 13:48:23 +02:00
alesapin
88f80078e1
Merge branch 'master' into repro_49887 2023-10-20 13:04:00 +02:00
helifu
57d60dc4f2
Add a new column xid for zookeeper_connection (#50702) 
* Refactor the code

* Add a new column xid for zookeeper_connection

* Support hostnames in the configuration

* Fix a typo

* Fix a typo

* Fix a typo

* Fix a bug about connect_time

* Update test case

* Update test case

* Fix a special build check error

* Resolve conflicts caused by rebase.

* Update failed test case

* Refactor the code according to comment

* Fix two compilation errors
 2023-10-20 11:38:44 +02:00
Igor Nikonov
16c1b64773 Merge remote-tracking branch 'origin/master' into pr-progress-bar 2023-10-20 08:12:28 +00:00
Raúl Marín
4a53943926 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-19 15:20:50 +00:00
Duc Canh Le
d968dc1a15 StorageJoin: supports trivial count() 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-19 06:30:25 +00:00
Duc Canh Le
b708fa21f9 rocksdb: trivial count 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-19 06:28:32 +00:00
Duc Canh Le
d8f73e7240 Merge branch 'master' into rocks_db_serialization 
Fix a mysterious style check fail
 2023-10-19 06:26:16 +00:00
Alexey Milovidov
4306b5a991 Revert unnecessary change 2023-10-19 08:13:23 +02:00
Alexey Milovidov
f6db0a5261 Merge branch 'master' into morton-utils 2023-10-19 08:11:25 +02:00
Alexey Milovidov
98111fad3d Fix error 2023-10-19 08:10:36 +02:00
Kseniia Sumarokova
4e0122a299
Merge pull request #54422 from kssenii/s3-queue-fixes 
Fixes for storage `S3Queue`
 2023-10-18 21:51:51 +02:00
kssenii
f90e31eb31 Enable log by default 2023-10-18 15:23:16 +02:00
Igor Nikonov
38895ad2da Less copying in DefaultCoordinator 2023-10-18 12:59:12 +00:00
kssenii
c4bad255b7 Fix 2023-10-18 14:37:33 +02:00
Igor Nikonov
01fa15c33f Less copying while handling annoucement 2023-10-18 12:20:14 +00:00
Duc Canh Le
2d638d6470 rocksdb: not creating block during writing 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-18 09:43:54 +00:00
Amos Bird
602f01f651
Reuse granule during skip index reading 2023-10-18 14:40:34 +08:00
Igor Nikonov
135e4c8412 Merge remote-tracking branch 'origin/master' into pr-progress-bar 2023-10-17 20:42:39 +00:00
Igor Nikonov
fb3d3a37ba Add progress bar for in order coordinator 2023-10-17 20:09:39 +00:00
Alexey Milovidov
79eccfb642
Merge pull request #55678 from azat/fix-OR-resubmit 
Fix filtering by virtual columns with OR filter in query (resubmit)
 2023-10-17 20:33:19 +02:00
avogar
323486f9e8 Add tests 2023-10-17 18:10:47 +00:00
kssenii
eb4519b9d6 Fix 2023-10-17 19:02:04 +02:00
Kseniia Sumarokova
7cedfeff12
Merge pull request #55145 from kssenii/materialized-pg-allow-unique-identifier 
Allow to have several MaterializedPostgreSQL tables following the same Postgres table
 2023-10-17 18:47:32 +02:00
vdimir
c657e500ac
Analyzer: fix block stucture mismatch in matview with engine distributed 2023-10-17 14:02:30 +00:00
vdimir
1d2ac48261
Merge pull request #55681 from ClickHouse/vdimir/fix_analyzer_materialized_distributed_join 
Fix StorageMaterializedView::isRemote
 2023-10-17 16:00:15 +02:00
alesapin
abe6df6fbc
Merge pull request #55645 from ClickHouse/restore_top_level_parts 
Restore only top-level parts instead of unexpected
 2023-10-17 15:45:21 +02:00
kssenii
c549083e2c Fix 2023-10-17 14:23:59 +02:00
avogar
032d82e004 Fixes 2023-10-17 12:12:10 +00:00
Alexander Gololobov
11a62bc751
Merge pull request #55621 from ClickHouse/merge-52352 
Merging #52352
 2023-10-17 13:58:53 +02:00
avogar
527e96db3f Fix 2023-10-17 11:43:51 +00:00
avogar
4892618aa9 Update with master, fix comments 2023-10-17 11:42:19 +00:00
avogar
bb46aa4d57 Clean uo 2023-10-17 11:07:41 +00:00
avogar
f87938735a Fixes and improvements for Iceberg storage 2023-10-17 11:07:37 +00:00
Duc Canh Le
f20d055e1e do not re-create serialization for each row 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2023-10-17 10:12:27 +00:00
kssenii
42ed249954 Fix build 2023-10-17 12:03:49 +02:00
Robert Schulze
96a82df75a
Merge pull request #55616 from HarryLeeIBM/hlee-fts-max-rows 
Inverted index: Replace setting density with `max_rows_in_postings_list`
 2023-10-17 11:36:46 +02:00
Raúl Marín
3eaf752284 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-17 11:36:39 +02:00
Kseniia Sumarokova
550a658236
Merge pull request #55144 from kssenii/fix-iceberg-metadata-parsing 
Fix storage Iceberg files retrieval
 2023-10-17 11:31:07 +02:00
kssenii
4464c86895 Merge remote-tracking branch 'origin/master' into s3-queue-fixes 2023-10-17 11:16:52 +02:00
kssenii
8744cd9c0d Fix 2023-10-17 11:15:53 +02:00
Igor Nikonov
4b36761f89
Merge branch 'master' into pr-progress-bar 2023-10-17 10:36:32 +02:00
Alexey Milovidov
de8215f5dd Merge remote-tracking branch 'origin/master' into morton-utils 2023-10-17 04:55:45 +02:00
HarryLeeIBM
fb8cd298d1 Code refactoring according to code review 2023-10-16 17:52:39 -07:00
Michael Kolupaev
ce7eca0615
DWARF input format (#55450) 
* Add ReadBufferFromFileBase::isRegularLocalFile()

* DWARF input format

* Review comments

* Changed things around ENABLE_EMBEDDED_COMPILER build setting

* Added 'ranges' column

* no-msan no-ubsan
 2023-10-16 17:00:07 -07:00
robot-clickhouse
89eed73fb9
Merge pull request #55519 from azat/zk-retry-fix 
Avoid retrying keeper operations on INSERT after table shutdown
 2023-10-16 23:32:13 +02:00
Kseniia Sumarokova
96c518be5b
Merge branch 'master' into s3-queue-fixes 2023-10-16 22:19:13 +02:00
Kseniia Sumarokova
3e9a1cf69f
Merge branch 'master' into fix-iceberg-metadata-parsing 2023-10-16 22:18:52 +02:00
Kseniia Sumarokova
c31299d86c
Merge branch 'master' into materialized-pg-allow-unique-identifier 2023-10-16 22:18:23 +02:00
robot-ch-test-poll1
aca98cf725
Merge pull request #53774 from helifu/master13 
Display reloaded values of server settings
 2023-10-16 21:17:17 +02:00
Kruglov Pavel
cbdb62d389
Merge pull request #55666 from ClickHouse/Avogar-patch-1 
Fix progress bar for s3 and azure Cluster functions with url without globs
 2023-10-16 18:25:00 +02:00
kssenii
0a6a4b3894 Review fixes 2023-10-16 18:15:17 +02:00
Kseniia Sumarokova
8f3974d289
00002_log_and_exception_messages_formatting ... 2023-10-16 17:30:02 +02:00
vdimir
c9a2a4922d
Fix StorageMaterializedView::isRemote 2023-10-16 15:18:47 +00:00
Igor Nikonov
6abd1a6462
Merge branch 'master' into zk-retry-fix 2023-10-16 16:56:07 +02:00
Azat Khuzhin
96f2a46a66 Fix filtering by virtual columns with OR filter in query 
The problem with the initial implementation #52653 was:
- OR can have multiple arguments
- It simply not correct to assume that if there are two arguments this is OK.
  Consider the following example:

    "WHERE (column_not_from_partition_by = 1) OR false OR false"

  Will be converted to:

    "WHERE false OR false"

And it will simply read nothing.

Yes, we could apply some optimization for bool, but this will not always
work, since to optimize things like "0 = 1" we need to execute it.

And the only way to make handle this correctly (with ability to ignore
some commands during filtering) is to make is_constant() function return
has it use something from the input block, so that we can be sure, that
we have some sensible, and not just "false".

Plus we cannot simply ignore the difference of the input and output
arguments of handling OR, we need to add always-true (1/true) if the
size is different, since otherwise it could break invariants (see
comment in the code).

This includes (but not limited to):
- _part* filtering for MergeTree
- _path/_file for various File/HDFS/... engines
- _table for Merge
- ...

P.S. analyzer does not have this bug, since it execute expression as
whole, and this is what filterBlockWithQuery() should do actually
instead, but this will be a more complex patch.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit b107712e0c)
 2023-10-16 15:42:10 +02:00
Raúl Marín
f799f5d7a1 Merge remote-tracking branch 'blessed/master' into parallel_replicas_row_estimation 2023-10-16 15:07:11 +02:00
Igor Nikonov
fdd7bfbb83 Merge remote-tracking branch 'origin/master' into pr-progress-bar 2023-10-16 12:59:00 +00:00
alesapin
9a925a0802 Fix typo 2023-10-16 13:35:20 +02:00
Raúl Marín
478f635d60
Merge pull request #53792 from ClickHouse/revert-53782-revert-52762-planner-prepare-filters-for-analysis 
Revert "Revert "Planner prepare filters for analysis""
 2023-10-16 13:21:54 +02:00
helifu
dc0f8ed793 Add a new column 'is_hot_reloadable' to indicate whether hot loading is supported 2023-10-16 19:10:35 +08:00
alesapin
31dc46a889 Better test 2023-10-16 13:04:45 +02:00
Kruglov Pavel
6dcc756948
Fix for azure 2023-10-16 12:38:55 +02:00
Kruglov Pavel
836e35b6c4
Fix progress bar for s3 and azure Cluster functions with url without globs 2023-10-16 12:38:10 +02:00
alesapin
dbebdc3dca Fix comment 2023-10-16 11:05:59 +02:00
Antonio Andelic
6c543c730b
Merge pull request #55657 from ClickHouse/revert-55418-fix-OR 
Revert "Fix filtering by virtual columns with OR filter in query"
 2023-10-16 10:51:06 +02:00
alesapin
2d9812b9e9 Fix headers 2023-10-16 10:43:25 +02:00
alesapin
692fcd8422 Merge remote-tracking branch 'origin/master' into repro_49887 2023-10-16 10:39:16 +02:00
alesapin
697c1e3bb0
Merge pull request #55604 from ClickHouse/evillique-partition-parameters 
Implement query paramenters support in partition expression
 2023-10-16 10:26:02 +02:00
alesapin
26a8aa441c Better shutdown handling 2023-10-16 10:24:54 +02:00
Antonio Andelic
fb901c24a1
Revert "Fix filtering by virtual columns with OR filter in query" 2023-10-16 09:45:49 +02:00
Alexey Milovidov
f5aabbecfc Add a comment 2023-10-16 02:24:34 +02:00
Alexey Milovidov
26c0c0b86d More tests 2023-10-16 02:19:24 +02:00
Alexey Milovidov
8e8471f5db Fix error 2023-10-16 00:25:00 +02:00
Alexey Milovidov
724f04e6f6 Allow conditions on part of arguments 2023-10-15 22:21:27 +02:00
Alexey Milovidov
c932a5e0a1 Allow conditions on part of arguments 2023-10-15 22:21:10 +02:00
Alexey Milovidov
0bf6fd3671 Merge remote-tracking branch 'origin/master' into morton-utils 2023-10-15 21:28:00 +02:00
alesapin
bdafe17e9a
Update src/Storages/MergeTree/ActiveDataPartSet.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-10-15 20:17:55 +02:00
alesapin
13ca296497
Update src/Storages/MergeTree/ActiveDataPartSet.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-10-15 20:17:50 +02:00
alesapin
6ac790e918
Update src/Storages/MergeTree/ActiveDataPartSet.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-10-15 20:17:46 +02:00
alesapin
8a865cdfb2
Merge pull request #55646 from ClickHouse/fix_some_typos 
Fix some typos
 2023-10-15 18:28:33 +02:00
Alexey Milovidov
0f89b4aad6
Merge pull request #55640 from ClickHouse/preparation-key-condition-2 
KeyCondition: small changes
 2023-10-15 18:23:53 +02:00
Kseniia Sumarokova
2015a0fc17
Merge branch 'master' into fix-iceberg-metadata-parsing 2023-10-15 18:03:53 +02:00
kssenii
f3056b1643 Add a test 2023-10-15 18:02:34 +02:00
alesapin
3b02748cb6 Fix some typos 2023-10-15 15:43:02 +02:00
alesapin
41479b21e6 Restore only top-level parts instead of unexpected 2023-10-15 15:33:33 +02:00
kssenii
0ddee6ad6c Fix 2023-10-15 14:44:57 +02:00
Kseniia Sumarokova
b26115eabe
Merge branch 'master' into materialized-pg-allow-unique-identifier 2023-10-15 14:17:32 +02:00