Commit Graph

337 Commits

Author SHA1 Message Date
Chen Lixiang
9e95fe5d29 conflict fixed. 2023-12-07 18:10:45 +08:00
Chen Lixiang
88920f08f1 add tests and fix whitespace issue 2023-11-29 17:26:11 +08:00
Han Fei
7bd2ea3add Merge branch 'master' into hanfei/statistic 2023-11-27 23:31:23 +01:00
Chen Lixiang
2b35e98ef8 add comments and doc 2023-11-24 16:40:19 +08:00
Chen Lixiang
89fa729235 show uncompressed size in system.tables 2023-11-24 15:54:27 +08:00
Sema Checherinda
b9cc1580e0
Merge pull request #56367 from canhld94/ch_table_reinit_new_disk
Adding new disk to storage configuration without restart
2023-11-14 15:54:22 +01:00
kssenii
f2c0434c4d Merge remote-tracking branch 'origin/master' into minor-improvements-for-s3-queue 2023-11-06 15:51:11 +01:00
kssenii
da21413354 Better shutdown 2023-11-06 15:47:57 +01:00
Alexander Tokmakov
eba52b318d
Merge pull request #56272 from ClickHouse/rdb_fix_rare_logical_error
Fix rare logical error in Replicated database
2023-11-06 14:14:55 +01:00
Duc Canh Le
4c21ba7b6f tables auto initialize new disks without restart
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-11-06 10:26:48 +00:00
Alexander Tokmakov
0ce1560e5d retry if table replica being restarted 2023-11-02 18:30:32 +01:00
Han Fei
1237cef26d address comments 2023-10-31 00:53:46 +01:00
Han Fei
19cdc6dfe2 Merge branch 'master' into hanfei/statistic 2023-10-30 04:53:18 +01:00
vdimir
faffdba8a2
Split source and worker processors in InterpreterCheckQuery 2023-10-27 15:22:10 +00:00
vdimir
c1ae369919
wip CHECK query for all tables 2023-10-27 15:22:10 +00:00
vdimir
8f0d7954ff
IStorage::checkDataNext returns optional 2023-10-23 10:12:30 +00:00
vdimir
fe95c0d0e4
Improve CHECK TABLE system query
Resubmit PR #52745
2023-10-23 09:35:26 +00:00
Han Fei
4b23142775
Merge branch 'master' into hanfei/statistic 2023-10-17 16:40:31 +02:00
alesapin
47d48d81e0 Implement feature 2023-10-13 16:22:18 +02:00
Han Fei
8cb9fe0154 refine code 2023-09-26 19:16:01 +02:00
Han Fei
17243b84ab Merge branch 'master' into hanfei/statistic 2023-09-19 17:09:17 +02:00
avogar
2d8f33bfa2 Fix parsing error in WithNames formats while reading subset of columns with disabled input_format_with_names_use_header 2023-09-11 14:55:37 +00:00
flynn
6ec783a746 Refactor IHints 2023-09-10 03:48:12 +00:00
Han Fei
f60dad0598 Merge branch 'master' into hanfei/statistic 2023-09-04 16:17:07 +02:00
Nikita Mikhaylov
96cabcae3a Change interface 2023-08-29 16:26:48 +02:00
Han Fei
10c622fa08 Merge branch 'master' into hanfei/statistic 2023-08-24 15:56:06 +02:00
avogar
68e3af56d4 Address comments 2023-08-23 13:19:15 +00:00
Han Fei
2b1f2f1de2 Merge branch 'master' into hanfei/statistic 2023-08-10 00:52:24 +02:00
Han Fei
f5c9d278ad use statistic to order prewhere conditions better 2023-08-09 22:57:49 +02:00
Alexander Tokmakov
6bbed6262e
Merge branch 'master' into add_delay_for_replicated 2023-07-26 12:48:48 +03:00
Azat Khuzhin
d7f7f16fbc Introduce IStorage::supportsTrivialCountOptimization()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-24 05:50:44 +02:00
alesapin
6416fb6eed
Merge branch 'master' into add_delay_for_replicated 2023-07-22 12:11:39 +02:00
Alexander Sapin
b70349852c Better message on timeout and fix test 2023-07-17 19:02:29 +02:00
alesapin
baee73fd96 Make shutdown of replicated tables softer 2023-07-05 18:11:25 +02:00
Azat Khuzhin
e9c9db9335 Disable parallelize_output_from_storages for all system tables
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-06-14 19:11:23 +03:00
Antonio Andelic
b11f744252
Correctly disable async insert with deduplication when it's not needed (#50663)
* Correctly disable async insert when it's not used

* Better

* Add comment

* Better

* Fix tests

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2023-06-07 20:33:08 +02:00
Anton Popov
612173e734 refactoring near alter conversions 2023-05-25 22:54:54 +00:00
Alexey Milovidov
3556c26075
Merge pull request #49434 from ClickHouse/ins
Make INSERT do more things in parallel to avoid getting bottlenecked on one thread
2023-05-09 08:10:15 +03:00
Michael Kolupaev
3bd1489f18 Propagate input_format_parquet_preserve_order to parallelizeOutputAfterReading() 2023-05-05 04:20:27 +00:00
Michael Kolupaev
0fa311ee38 Make INSERT do more things in parallel to avoid getting bottlenecked on one thread 2023-05-05 00:49:27 +00:00
Anton Popov
36d53e0d6c
Merge branch 'master' into add-async-insert-mt-setting 2023-05-01 16:13:23 +02:00
Anton Popov
c7db58be66 add MergeTree setting async_insert 2023-04-25 01:11:58 +00:00
Alexander Tokmakov
19f43ee7c7 Merge branch 'master' into fix_some_tests3 2023-04-24 16:13:27 +02:00
Alexey Milovidov
8a92eb0cf1
Update src/Storages/IStorage.h 2023-04-21 11:50:07 +03:00
Alexander Tokmakov
70c1000c22 fix some tests 2023-04-14 19:42:17 +02:00
Igor Nikonov
7c84dc48c6 Better way to define for which storage output is parallelized 2023-04-14 12:00:02 +00:00
Artem Pershin
2b00c8cb9b Fix typo in word acquired, in comment for lockForAlter method 2023-04-08 18:39:04 +03:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Azat Khuzhin
beff5ac795 Fix PREWHERE for Merge with different default types
In case of underlying table has an ALIAS for this column, while in Merge
table it is not marked as an alias, there will NOT_FOUND_COLUMN_IN_BLOCK
error.

Further more, when underlying tables has different default type for the
column, i.e. one has ALIAS and another has real column, then you will
also get NOT_FOUND_COLUMN_IN_BLOCK, because Merge engine should take
care of this.

Also this patch reworks how PREWHERE is handled for Merge table, and now
if you use PREWHERE on the column that has the same type and default
type (ALIAS, ...) then it will be possible, and only if the type
differs, it will be prohibited and throw ILLEGAL_PREWHERE error.

And last, but not least, also respect this restrictions for
optimize_move_to_prewhere.

v2: introduce IStorage::supportedPrewhereColumns()
v3: Remove excessive condition for PREWHERE in StorageMerge::read()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-16 17:21:53 +01:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine (#41005)
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree

* Add new test for the new ReplacingMergeTree engine

* Limit sign value to -1/1

* Replace sign column(Int8) by is_deleted(UInt8)

* Add keyword 'CLEANUP' when OPTIMIZE

* Cleanup uniquely when it's a replacingMergeTree

* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'

* Cleanup data flagged as 'is_deleted'

* Fix merge when optimize and add a test

* Fix OPTIMIZE and INSERT + add tests

* New fix for cleanup at the merge

* Cleanup debug logs

* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'

* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'

* Add Replicated tests

* Disable tag 'long' for our test and cleanup some white spaces

* Update tests

* Fix tests and remove additional useless whitespace

* Fix replica test

* Style clean && add condition check for is_deleted values

* clean_deleted_rows settings is nom an enum

* Add valid default value to the clean_deleted_rows settings

* Update cleanup checkers to use the enum and fix typos in the test

* Fix submodule contrib/AMQP-CPP pointer

* Add missing messages in test reference and remove a print with non derterministic order

* fix replica test reference

* Fix edge case

* Fix a typo for the spell checker

* Fix reference

* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup

* Change tests file name and update number

* This should fix the ReplacingMergeTree parameter set

* Fix replicated parameters

* Disable allow_deprecated_syntax_for_merge_tree for our new column

* Fix a test

* Remove non deterministic order print in the test

* Test on replicas

* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe

* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"

This reverts commit b65d64c05e.

* Fix replica management and limit the number of argument to two maximum, due to the possiblity of deprecated table create/attach failing otherwise

* Test a fix for replicated log information error

* Try to add sync to have consistent results

* Change path of replicas that should cause one issue and add few prints in case it's not that

* Get cleanup info on replicas only if information found

* Fix style issues

* Try to avoid replication error 'cannot select parts...' and and replica read/write field order

* Cleanup according to PR reviews
 and add tests on error raised.

* Update src/Storages/MergeTree/registerStorageMergeTree.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>

* Select ... FINAL don't show rows with is_deleted = true

* Update and fix SELECT ... FINAL merge parameter

* Remove is_deleted rows only on the version inserted when merge

* Fix (master) updates issues

* Revert changes that should not be commited

* Add changes according to review

* Revert changes that should not be commited - part 2

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-02-16 16:03:16 +03:00