Nikolai Kochetov
84e442e620
Fixing build and style.
2023-02-20 14:21:20 +00:00
Nikolai Kochetov
2a8460d962
Merge branch 'master' into aggregate-projections-analysis-query-plan
2023-02-20 13:54:05 +01:00
Alexey Milovidov
d8cda3dbb8
Remove PVS-Studio
2023-02-19 23:30:05 +01:00
Arthur Passos
3d9b23e5f3
Merge branch 'master' into add_select_final_mt_setting
2023-02-19 15:01:41 -03:00
Alexander Gololobov
520b381675
Fixes for executing actions on block with unknown row count
2023-02-19 12:14:23 +01:00
Maksim Kita
6dffe52e68
Merge pull request #46471 from kitaisreal/analyzer-planner-fixes-before-enable-by-default
...
Analyzer planner fixes before enable by default
2023-02-19 13:49:18 +03:00
Alexander Gololobov
0ed0b191f8
Fix for Nullable step filter column
2023-02-18 19:12:32 +01:00
Alexander Gololobov
88fb6c3c25
Fix for incorrect combined PREWHERE column
2023-02-18 18:29:16 +01:00
Kseniia Sumarokova
b0ea047e76
Merge pull request #46533 from kssenii/fix-disk-setting
...
Allow `disk` merge tree setting in query to override config merge tree setting `storage_policy`
2023-02-18 12:36:54 +01:00
kssenii
afb36dd120
Allow disk setting to override storage_policy from config merge tree settings insteaf of exception
2023-02-17 17:11:29 +01:00
Antonio Andelic
ab51c1d975
Merge branch 'master' into custom-key-parallel-replicas
2023-02-17 15:06:41 +01:00
kssenii
bae85c50e9
Fix stress test
2023-02-17 12:40:17 +01:00
Alexander Tokmakov
9a664efeb6
Merge branch 'master' into fixes_for_993
2023-02-17 14:17:08 +03:00
Sema Checherinda
b551b9fb3a
Merge pull request #46452 from CheSema/resolve-race-get-csn
...
resolve race in getCSNAndAssert
2023-02-17 11:31:58 +01:00
Dmitry Novik
0336f0d8cc
Merge branch 'master' into background-memory-tracker
2023-02-16 20:57:44 +01:00
Arthur Passos
ba6b7b1610
fix test_alias_column
2023-02-16 16:01:25 -03:00
Alexander Gololobov
a6061e022e
Merge branch 'master' into split_prewhere_dag
2023-02-16 14:43:58 +01:00
youennL-cs
6526c2a8ab
[RFC] Replacing merge tree new engine ( #41005 )
...
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree
* Add new test for the new ReplacingMergeTree engine
* Limit sign value to -1/1
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree
* Add new test for the new ReplacingMergeTree engine
* Limit sign value to -1/1
* Replace sign column(Int8) by is_deleted(UInt8)
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree
* Add new test for the new ReplacingMergeTree engine
* Limit sign value to -1/1
* Replace sign column(Int8) by is_deleted(UInt8)
* Add new engine to ReplacingMergeTree corresponding to the ReplacingCollapsingMergeTree
* Add new test for the new ReplacingMergeTree engine
* Limit sign value to -1/1
* Replace sign column(Int8) by is_deleted(UInt8)
* Add keyword 'CLEANUP' when OPTIMIZE
* Cleanup uniquely when it's a replacingMergeTree
* Propagate CLEANUP information and change from 'with_cleanup' to 'cleanup'
* Cleanup data flagged as 'is_deleted'
* Fix merge when optimize and add a test
* Fix OPTIMIZE and INSERT + add tests
* New fix for cleanup at the merge
* Cleanup debug logs
* Add the SETTINGS option 'clean_deleted_rows' that can be 'never' or 'always'
* Fix regression bug; Now REplicatedMergeTree can be called as before without 'is_deleted'
* Add Replicated tests
* Disable tag 'long' for our test and cleanup some white spaces
* Update tests
* Fix tests and remove additional useless whitespace
* Fix replica test
* Style clean && add condition check for is_deleted values
* clean_deleted_rows settings is nom an enum
* Add valid default value to the clean_deleted_rows settings
* Update cleanup checkers to use the enum and fix typos in the test
* Fix submodule contrib/AMQP-CPP pointer
* Add missing messages in test reference and remove a print with non derterministic order
* fix replica test reference
* Fix edge case
* Fix a typo for the spell checker
* Fix reference
* Fix a condition to raise an error if is_deleted differ from 0/1 and cleanup
* Change tests file name and update number
* This should fix the ReplacingMergeTree parameter set
* Fix replicated parameters
* Disable allow_deprecated_syntax_for_merge_tree for our new column
* Fix a test
* Remove non deterministic order print in the test
* Test on replicas
* Remove a condition, when checking optional parameters, that should not be sueful since we disabled the deprected_syntaxe
* Revert "Remove a condition, when checking optional parameters, that should not be useful since we disabled the deprected_syntaxe"
This reverts commit b65d64c05e
2023-02-16 16:03:16 +03:00
Dmitry Novik
728434821a
Merge remote-tracking branch 'origin/master' into background-memory-tracker
2023-02-16 12:35:07 +00:00
Maksim Kita
b72ea982b0
Remove unnecessary includes of InterpreterSelectQuery header
2023-02-16 12:17:03 +01:00
Maksim Kita
0fe3ba36c4
Analyzer added ParitionPruner support
2023-02-16 12:17:02 +01:00
Maksim Kita
bb56af0e2b
MergeTreeIndexGin crash fix
2023-02-16 12:17:02 +01:00
Maksim Kita
77fee97705
MergeTreeIndexFullText fix prepared set index analysis
2023-02-16 12:17:02 +01:00
Kseniia Sumarokova
947753bd4c
Merge pull request #46409 from kssenii/follow-up-to-46168
...
Follow-up to #46168
2023-02-16 11:26:54 +01:00
Sergei Trifonov
3ad41b29ec
Merge pull request #46247 from ClickHouse/round-robin-merge-scheduler
2023-02-16 08:03:22 +01:00
Alexander Gololobov
cdf3828575
Removed unneeded default value for enable_multiple_prewhere_read_steps
2023-02-15 22:41:55 +01:00
Alexander Gololobov
6936471150
fix include
...
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-02-15 22:37:40 +01:00
Alexander Gololobov
8b528739c1
fix typo
...
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-02-15 22:37:18 +01:00
Alexander Gololobov
3ff0c7edcc
fix typo
...
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-02-15 22:37:04 +01:00
Nikolai Kochetov
f524dae924
Fix more tests.
2023-02-15 19:17:24 +00:00
Dmitry Novik
e7a94a28ee
Merge branch 'master' into background-memory-tracker
2023-02-15 19:39:28 +01:00
Sema Checherinda
dd1b509637
resolve race in getCSNAndAssert
2023-02-15 18:16:39 +01:00
Han Fei
5458d4216f
Merge branch 'master' into dev-prewhere
2023-02-15 17:59:44 +01:00
Alexey Milovidov
77f60311da
Merge branch 'master' into add_select_final_mt_setting
2023-02-15 15:35:37 +03:00
Antonio Andelic
4dd4aac125
Merge branch 'master' into custom-key-parallel-replicas
2023-02-15 11:02:37 +01:00
Robert Schulze
ec33204c05
Merge pull request #46344 from ClickHouse/rs/stabilize-full_text_seaerch_test
...
Try to stabilize test 02346_full_text_search.sql
2023-02-15 10:47:00 +01:00
Alexander Tokmakov
e96340e183
Merge pull request #46410 from ClickHouse/fix_noisy_log_messages
...
Fix noisy log messages
2023-02-15 01:22:42 +03:00
Alexander Tokmakov
8ddbc85919
Merge branch 'master' into followup_revert_39737
2023-02-15 01:15:17 +03:00
Sergei Trifonov
b7a4fa8bd9
Merge branch 'master' into round-robin-merge-scheduler
2023-02-14 20:39:02 +01:00
serxa
a7bbf02bac
fix possible deadlock
2023-02-14 19:37:09 +00:00
Han Fei
937fade982
clean up
2023-02-14 18:54:19 +01:00
Han Fei
86fda9bd22
some clean up
2023-02-14 18:52:09 +01:00
Han Fei
32050acde5
Merge branch 'master' into dev-prewhere
2023-02-14 18:39:13 +01:00
Kseniia Sumarokova
fe155bb210
Merge branch 'master' into follow-up-to-46168
2023-02-14 18:28:46 +01:00
Anton Popov
ecc6ff707b
Merge pull request #46387 from CurtizJ/fix-update-from-empty-nested
...
Fix `ALTER UPDATE` of empty `Nested` columns
2023-02-14 16:28:32 +01:00
kssenii
5bda358fb7
Follow-up to #46168
2023-02-14 15:45:38 +01:00
Alexander Tokmakov
1bf3735ca9
fix noisy log messages
2023-02-14 15:33:27 +01:00
vdimir
abd9119583
Remove unnecessary ProfileEventsScope
2023-02-14 14:11:12 +00:00
vdimir
4c1aeb7972
Use overload of PartLog::addNewPart with profile_events
2023-02-14 14:11:12 +00:00
vdimir
07d7478bc7
wip
2023-02-14 14:11:10 +00:00
vdimir
994dcb93f9
Rename ScopedProfileEvents -> ProfileEventsScope
2023-02-14 14:11:10 +00:00
vdimir
a228f7f419
[wip] ProfileCounters for each part
2023-02-14 14:11:09 +00:00
Alexey Milovidov
f554ff9d99
Remove trash
2023-02-14 14:11:08 +00:00
Nicolae Vartolomei
96e4411694
Fix style
2023-02-14 14:11:07 +00:00
John Skopis
f7604cc686
[feat] Add ProfileEvents map to PartLog
...
closes #10316
2023-02-14 14:11:02 +00:00
Antonio Andelic
adde580756
Merge branch 'master' into custom-key-parallel-replicas
2023-02-14 14:09:12 +01:00
Nikita Taranov
cbd10c4493
Fix mbm with aio on top of query plan ( #45892 )
2023-02-14 14:02:54 +01:00
Robert Schulze
95c50b8653
Address the actual problem (not an optimal approach yet ...)
2023-02-14 12:32:09 +00:00
Sema Checherinda
fba7ee6332
Merge pull request #46347 from CheSema/fix-select-candite-to-check
...
fix candidate selection
2023-02-14 11:18:42 +01:00
Anton Popov
1ab37ad5ed
fix update of empty Nested columns
2023-02-14 02:31:42 +00:00
Alexander Gololobov
d76e4f60a8
Fix for combined node for condition group not found
2023-02-13 23:14:18 +01:00
Nikolai Kochetov
1c0c662fe1
Merge pull request #46275 from ClickHouse/less-mem-usage-for-vertical-merge
...
Less memory usage for vertical merge (non-remote disk)
2023-02-13 21:10:43 +01:00
Alexander Tokmakov
a4d9688775
fix 'Directory not empty after drop' with zero copy replication
2023-02-13 19:35:07 +01:00
Alexander Gololobov
0fe080cc6f
Logger instead of debug prints
2023-02-13 17:43:41 +01:00
Alexander Gololobov
18a29e81fc
Better formatting in PrewhereExprInfo::dump()
2023-02-13 17:42:27 +01:00
Alexander Gololobov
f0011862a9
Move prewhere splitting logic into a separate file
2023-02-13 16:00:03 +01:00
Antonio Andelic
95db6fb1a6
Merge pull request #45659 from ClickHouse/add-support-for-settings-alias
...
Add support for settings aliases/Rename `replication_alter_partitions_sync` to `alter_sync`
2023-02-13 15:31:12 +01:00
Sema Checherinda
a48b6ec612
fix candidate selection
2023-02-13 14:52:55 +01:00
Alexander Tokmakov
69f579b8ec
remove an unused argument
2023-02-13 14:46:46 +01:00
Alexander Gololobov
39e4ca9850
Cleanup
2023-02-13 14:40:54 +01:00
Alexander Gololobov
c39cfee954
Properly set "remove" flag for step prewhere_columns when they are used in futher steps
2023-02-13 14:11:18 +01:00
Alexander Gololobov
3060bee18a
Properly handle constants in multiple steps
2023-02-13 12:58:33 +01:00
Alexey Milovidov
9be1bba92f
Merge pull request #46310 from ClickHouse/replicas_status_remove_mutex
...
Remove mutex from replicas_status
2023-02-13 13:36:11 +03:00
Antonio Andelic
8d16fe5793
Merge branch 'master' into add-support-for-settings-alias
2023-02-13 08:46:00 +01:00
Alexander Gololobov
a89465d1cb
Fix build after merge
2023-02-12 22:48:28 +01:00
Alexander Gololobov
3ad0683de9
Refactor DAG splitting into steps
2023-02-12 21:55:53 +01:00
Alexander Gololobov
12e576cddb
Slight refactoring
2023-02-12 21:55:53 +01:00
Alexander Gololobov
8fc08c74f0
Pass action_settings to getReadTaskColumns()
2023-02-12 21:55:53 +01:00
Alexander Gololobov
e460d53619
Use different name when filter column is CASTed to UInt8
2023-02-12 21:55:41 +01:00
Alexander Gololobov
7634b9f56e
Added settings to enable the new logic
2023-02-12 21:54:23 +01:00
Alexander Gololobov
120b112743
Cleanup
2023-02-12 21:54:23 +01:00
Alexander Gololobov
31f5dbe2a3
Cast intermediate step result to Bool if needed
2023-02-12 21:54:23 +01:00
Alexander Gololobov
1b207d9fa0
Fix for overwritten row level filter step
2023-02-12 21:54:23 +01:00
Alexander Gololobov
249e670c9f
Properly preserve columns that are required by next steps
2023-02-12 21:54:23 +01:00
Alexander Gololobov
45ef2f6d60
Split prewhere actions into separate conjuctive steps
2023-02-12 21:54:23 +01:00
alesapin
1683cd5d6f
Merge branch 'master' into mutations_rename_hang
2023-02-12 13:06:48 +01:00
Alexey Milovidov
b8cbc947ce
Remove mutex from replicas_status
2023-02-12 07:39:16 +01:00
Alexey Milovidov
5f00bc49f8
Merge branch 'master' into addParts
2023-02-12 05:55:18 +03:00
Alexey Milovidov
a97f9b2caf
Update MergeTreeData.cpp
2023-02-12 05:55:07 +03:00
Alexey Milovidov
395d6b6bd5
Merge pull request #46282 from CurtizJ/return-back-vertical-merges
...
Allow vertical merges from compact to wide parts [2]
2023-02-12 05:51:12 +03:00
serxa
df6b4c4372
fix typo
2023-02-11 20:18:19 +00:00
serxa
9a7f70f595
fix comment
2023-02-11 16:54:57 +00:00
serxa
eb8cf6f816
Merge branch 'round-robin-merge-scheduler' of github.com:ClickHouse/ClickHouse into round-robin-merge-scheduler
2023-02-11 16:45:55 +00:00
serxa
d2f8377c47
fix check
2023-02-11 16:45:45 +00:00
Sergei Trifonov
297587531b
Merge branch 'master' into round-robin-merge-scheduler
2023-02-11 17:19:27 +01:00
serxa
c58b165b0f
add config option to select scheduling policy
2023-02-11 16:18:42 +00:00
alesapin
5c83ddabf9
Merge branch 'master' into mutations_rename_hang
2023-02-11 12:18:58 +01:00
kssenii
3067c1d723
Merge remote-tracking branch 'upstream/master' into resubmit-prefetches
2023-02-11 11:36:23 +01:00
Sergei Trifonov
fbedf4d520
Merge branch 'master' into round-robin-merge-scheduler
2023-02-11 10:30:26 +01:00
kssenii
3545568440
Fix build
2023-02-10 22:15:44 +01:00
Kseniia Sumarokova
3be36c7979
Merge pull request #41976 from kssenii/allow-single-disk-instead-of-storage-policy
...
Allow configuring storage as `SETTINGS disk='<disk_name>'` (instead of `storage_policy`) and explicit disk creation `SETTINGS disk=disk(<disk_configuration>)`
2023-02-10 21:38:50 +01:00
Anton Popov
1205932a19
add setting to disable vertical merges from compact parts
2023-02-10 16:33:46 +00:00
kssenii
468d5216da
Remove logging because of test 00002_log_and_exception_messages_formatting
2023-02-10 17:25:26 +01:00
Anton Popov
a252d3a139
Revert "Merge pull request #46236 from ClickHouse/revert-45681-compact-parts-vertical-merge"
...
This reverts commit 27bc317ef9ccf7b6f4c1
2023-02-10 14:13:29 +00:00
Anton Popov
27bc317ef9
Merge pull request #46236 from ClickHouse/revert-45681-compact-parts-vertical-merge
...
Revert "Allow vertical merges from compact to wide parts"
2023-02-10 14:31:33 +01:00
alesapin
808a939ad2
Small rehacktoing
2023-02-10 14:25:19 +01:00
KitKatKKK
d31457de44
Update src/Storages/MergeTree/MergeTreeData.cpp
...
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-02-10 21:14:42 +08:00
KitKatKKK
7664114349
Update src/Storages/MergeTree/MergeTreeData.cpp
...
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-02-10 21:14:33 +08:00
kssenii
571aabdb69
Fix clang-tidy
2023-02-10 12:39:05 +01:00
Nikolai Kochetov
8c22c740a4
Merge pull request #46205 from ClickHouse/fix-write-buffer-destruction-order-for-vertical-merge
...
Fix write buffer destruction order for vertical merge.
2023-02-10 11:47:23 +01:00
KitKatKKK
ea300ce13b
Merge branch 'master' into addParts
2023-02-10 16:48:04 +08:00
KitKatKKK
fa73b00a66
Update src/Storages/MergeTree/MergeTreeData.h
...
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-02-10 16:46:57 +08:00
attack204
7f41173a1f
rename API
2023-02-10 16:24:10 +08:00
attack204
93e92dab3b
rename API
2023-02-10 16:05:50 +08:00
attack204
52960d4e7d
add API to MergeTreeData
2023-02-10 11:00:11 +08:00
Nikolai Kochetov
4aa6e4d683
Revert "Add return flag for preFinalize."
...
This reverts commit 3261378ef8
2023-02-09 19:51:09 +00:00
Nikolai Kochetov
3261378ef8
Add return flag for preFinalize.
2023-02-09 19:50:54 +00:00
Nikolai Kochetov
98765cc20b
Disable delayed streams for Vertical merges for non-remote disks.
2023-02-09 19:39:34 +00:00
serxa
bf299b6696
Merge branch 'round-robin-merge-scheduler' of github.com:ClickHouse/ClickHouse into round-robin-merge-scheduler
2023-02-09 19:33:17 +00:00
serxa
cc3c76944b
remove redundant code
2023-02-09 19:31:54 +00:00
Sergei Trifonov
e3727d9b0e
Merge branch 'master' into round-robin-merge-scheduler
2023-02-09 20:04:59 +01:00
serxa
f61f95fcec
Replace background executor scheduler merges and mutations with round-robin
2023-02-09 18:54:15 +00:00
Antonio Andelic
f96d480563
Merge branch 'master' into add-support-for-settings-alias
2023-02-09 16:07:45 +01:00
Anton Popov
612fe0bd00
Revert "Allow vertical merges from compact to wide parts"
2023-02-09 15:42:46 +01:00
alesapin
9b23068d5e
Merge pull request #45821 from ClickHouse/explicit_drop_part_entry_type
...
Make separate DROP_PART log entry type
2023-02-09 15:11:13 +01:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy
2023-02-09 13:16:55 +01:00
kssenii
2a48a96f83
Fix tests
2023-02-09 12:26:15 +01:00
Nikolai Kochetov
65ec9bd7d0
Fix write buffer destruction order for vertical merge.
2023-02-09 10:58:49 +00:00
alesapin
859f528fe1
Merge pull request #45648 from ClickHouse/45508_Update_strategy_for_system_sync_replica
...
Updated checking of SYSTEM SYNC REPLICA
2023-02-09 11:51:05 +01:00
kssenii
b0b865c32e
Resubmit prefetches
2023-02-08 21:26:24 +01:00
avogar
c3e8dd8984
Fix low cardinality case
2023-02-08 19:14:28 +00:00
alesapin
a28d10a810
Fxi
2023-02-08 14:50:30 +01:00
Smita Kulkarni
87ec32f235
Updated to use znode_name instead of log_entry_id - Updated checking of SYSTEM SYNC REPLICA
2023-02-08 14:07:06 +01:00
Azat Khuzhin
151ba92a2e
Fix reading of non existing nested columns with multiple level in compact parts
...
Consider the following example:
CREATE TABLE data (root.array_str Array(UInt8)) ENGINE = MergeTree() ORDER BY tuple();
INSERT INTO data VALUES ([]);
ALTER TABLE data ADD COLUMN root.nested_array Array(Array(UInt8));
In this case the first part will not have data for root.nested_array,
and thanks to #37152 it will simply read offsets column from
root.array_str, however since root.nested_array is a nested array, it
will try to read elements from the same offsets stream and if you are
lucky enough you will get one of the following errors:
- Cannot read all data. Bytes read: 1. Bytes expected: 8.: (while reading column root.nested_array): While executing MergeTreeInOrder. (CANNOT_READ_ALL_DATA)
- DB::Exception: Array size is too large: 8233460228287709730: (while reading column serp.serp_features): While executing MergeTreeInOrder.
So to address this, findColumnForOffsets() had been changed to return
the level of the column too, to allow to read only up to this level.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-08 10:21:40 +01:00
alesapin
bbf044f477
Fxi
2023-02-07 22:30:38 +01:00
Smita Kulkarni
8c9d994fd7
Extended scope of state_mutex while adding subscriber - Updated checking of SYSTEM SYNC REPLICA
2023-02-07 22:04:17 +01:00
alesapin
bc58fd156b
Merge remote-tracking branch 'origin' into mutations_rename_hang
2023-02-07 20:53:04 +01:00
alesapin
36f41da6e0
One more bugfix
2023-02-07 18:53:47 +01:00
Smita Kulkarni
ecea28a832
Fixed style check - Updated checking of SYSTEM SYNC REPLICA
2023-02-07 17:27:34 +01:00
Nikolai Kochetov
3912f5a333
Merge pull request #45681 from CurtizJ/compact-parts-vertical-merge
...
Allow vertical merges from compact to wide parts
2023-02-07 17:09:17 +01:00
Smita Kulkarni
34341ddabe
Updated locks and removed getLogEntryIds function - Updated checking of SYSTEM SYNC REPLICA
2023-02-07 16:57:58 +01:00
Sema Checherinda
52d4e78b7f
Merge pull request #45907 from CheSema/forbid-wal-on-s3
...
do not run wal on remote disks
2023-02-07 14:48:00 +01:00
Yakov Olkhovskiy
4e7dffafe4
Merge pull request #46087 from ClickHouse/fix-ip-bloom
...
Allow bloom filter for IPv4 and IPv6
2023-02-07 08:29:28 -05:00
alesapin
1fd9baa3f3
Fix fetch:
2023-02-07 12:49:51 +01:00
Smita Kulkarni
fb765696b6
Removed state_mutex lock from addSubscriber - Updated checking of SYSTEM SYNC REPLICA
2023-02-07 12:19:58 +01:00
Vitaly Baranov
f021c0eab6
Merge pull request #46058 from vitlibar/remove-unnecessary-execute-for-const-expr
...
Remove unnecessary execute() while evaluating a constant expression.
2023-02-07 12:09:37 +01:00
Smita Kulkarni
ef29b61786
Updated callback to return log entry ids - Updated checking of SYSTEM SYNC REPLICA
2023-02-07 10:02:38 +01:00
Anton Popov
7dfea59267
remove an assertion
2023-02-07 00:50:41 +00:00
alesapin
dd3a98e88d
Fixes
2023-02-06 22:30:31 +01:00
alesapin
aac5da77e8
fix stupid bug
2023-02-06 21:32:40 +01:00
Nikolai Kochetov
b2cc2332f5
Fix 01710_force_use_projection.
2023-02-06 19:20:03 +00:00
Dmitry Novik
1e065b32f3
Add MemoryTracker for the background tasks
2023-02-06 17:25:58 +00:00
kssenii
6cea67d58b
Cleanup
2023-02-06 18:11:33 +01:00
alesapin
2a0484ca41
Add commnents, fix bugs
2023-02-06 18:06:14 +01:00
Yakov Olkhovskiy
e433c26cf9
allow bloom filter for IPv4 and IPv6
2023-02-06 16:30:18 +00:00
Alexander Tokmakov
98a4e1ab2c
better comment
2023-02-06 15:08:30 +01:00
alesapin
30a7198411
Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
2023-02-06 15:06:28 +01:00
alesapin
bd4a3ce2d5
Update src/Storages/MergeTree/DataPartsExchange.cpp
2023-02-06 15:05:13 +01:00
Alexander Tokmakov
86feaae253
Merge branch 'master' into explicit_drop_part_entry_type
2023-02-06 14:59:59 +01:00
alesapin
e51e385017
fix style
2023-02-06 14:16:17 +01:00
Alexander Tokmakov
e9f770d89d
fix
2023-02-06 14:03:14 +01:00
alesapin
b94f9f8006
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica
2023-02-06 12:38:39 +01:00
alesapin
b265300a19
Merge branch 'master' into mutations_rename_hang
2023-02-06 12:25:29 +01:00
Antonio Andelic
f67e5505ab
Merge branch 'master' into custom-key-parallel-replicas
2023-02-06 11:12:39 +00:00
Sema Checherinda
734d3a318b
const auto ref in the loop
2023-02-06 12:09:33 +01:00
Vitaly Baranov
a3423ce196
Remove unnecessary execute() while evaluating a constant expression.
2023-02-05 20:26:34 +01:00
Han Fei
532b341de9
Merge pull request #45975 from ucasfl/_part
...
use LowCardnality for _part and _partition_id virtual column
2023-02-05 18:00:46 +01:00
Antonio Andelic
12569da984
Merge branch 'master' into add-support-for-settings-alias
2023-02-05 16:08:57 +00:00
kssenii
a7eb1b1419
Fix integration test
2023-02-04 22:57:30 +01:00
kssenii
5abbc7cad6
Cleanup
2023-02-04 19:41:41 +01:00
kssenii
1cbff88a7c
Fix integration test
2023-02-04 19:31:19 +01:00
Alexey Milovidov
22bd0b6f69
Merge pull request #38983 from CurtizJ/randomize-mt-settings
...
Allow to randomize merge tree settings in tests
2023-02-04 02:59:52 +01:00
alesapin
fd4ae0990b
Tiny fix
2023-02-03 21:07:34 +01:00
kssenii
5f124263a6
Revert changes with complex SettingsChanges
2023-02-03 21:04:45 +01:00
alesapin
9416401406
Fix
2023-02-03 20:52:10 +01:00
alesapin
f91c10d09d
Initial support for version
2023-02-03 20:49:07 +01:00
Nikolai Kochetov
310a4c69af
Aggregate Projections analysis using query plan [In progress]
2023-02-03 18:30:06 +00:00
alesapin
7a0d0f0c53
Some code which doesn't work
2023-02-03 19:14:49 +01:00
Antonio Andelic
8929838fcc
Merge branch 'master' into add-support-for-settings-alias
2023-02-03 17:47:00 +00:00
Alexander Tokmakov
3f11948bb0
Merge branch 'master' into stack_trace_in_part_log
2023-02-03 20:05:00 +03:00
Alexander Tokmakov
7e6f7c79f2
Merge pull request #45457 from FrankChen021/exception_time
...
Add last_exception_time to replication_queue
2023-02-03 20:00:15 +03:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] ( #43772 )
2023-02-03 14:34:18 +01:00
Alexey Milovidov
3e3df376c0
Merge pull request #45995 from CurtizJ/check-dynamic-columns
...
Check dynamic columns of part before its commit
2023-02-03 15:39:54 +03:00
alesapin
4a3277b1fd
Merge branch 'master' into mutations_rename_hang
2023-02-03 12:35:37 +01:00
Smita Kulkarni
123f3cfe1f
Used unordered_set for log entries and fixed the callback function - Updated checking of SYSTEM SYNC REPLICA
2023-02-03 08:48:16 +01:00
Frank Chen
d38adfab30
Merge two overridden functions as one
2023-02-03 15:27:45 +08:00
flynn
d119367d7d
fix
2023-02-03 07:06:58 +00:00
Frank Chen
d3a05a11da
Merge remote-tracking branch 'remotes/github/master' into stack_trace_in_part_log
2023-02-03 11:39:43 +08:00
Frank Chen
7af5ae7f2e
Merge branch 'master' into better_postpone_reason
2023-02-03 11:20:37 +08:00
Anton Popov
8ca602a148
check dynamic columns of part before its commit
2023-02-03 02:58:28 +00:00
alesapin
b8b0f1c3d3
Make RENAME COLUMN barrier
2023-02-02 21:11:19 +01:00
kssenii
a391de18a1
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2023-02-02 20:49:41 +01:00
alesapin
fc6c62e2ff
Add comment
2023-02-02 17:41:36 +01:00
alesapin
bdc530dead
Fix style
2023-02-02 17:30:38 +01:00
flynn
bc38ebaf52
use LowCardnality for _part and _partition_id virtual column
...
fix
2023-02-02 16:20:29 +00:00
alesapin
dea46e58fb
Fixes for ordinary merge tree
2023-02-02 17:03:24 +01:00
alesapin
174bdba912
Fix bugs in alter rename column
2023-02-02 16:57:46 +01:00
Alexander Gololobov
ec3bb0c04e
Merge pull request #45794 from ClickHouse/parallelize_is_storage_touched_by_mutations
...
Do not restrict count() query to 1 thread in isStorageTouchedByMutations()
2023-02-02 16:44:11 +01:00
Antonio Andelic
f613bfd8d2
Merge branch 'master' into add-support-for-settings-alias
2023-02-02 11:15:39 +00:00
Sema Checherinda
7c416477ac
do not run wal on remote disks
2023-02-01 20:03:45 +01:00
Nikolai Kochetov
ae420b30e5
Aggregate Projections analysis using query plan [In progress]
2023-02-01 17:55:06 +00:00
Sema Checherinda
3e1b17e4cf
fix build
2023-02-01 17:09:43 +01:00
Anton Popov
9f26e16963
loose the assertion
2023-02-01 15:31:06 +00:00
Sema Checherinda
41a1dddc7a
Merge branch 'master' into better_postpone_reason
2023-02-01 15:16:34 +01:00
Antonio Andelic
85cfee4bb9
Better alias definition
2023-02-01 13:54:03 +00:00
Alexander Tokmakov
aba4a00a17
fix
2023-02-01 14:47:57 +01:00
Antonio Andelic
ec04cf6876
Merge branch 'master' into add-support-for-settings-alias
2023-02-01 09:01:01 +00:00
Nikolai Kochetov
e2c32ccbca
Aggregate Projections analysis using query plan [In progress]
2023-01-31 20:33:01 +00:00
Anton Popov
1c4174750d
fix mistake after refactoring
2023-01-31 15:48:14 +00:00
Alexander Tokmakov
7862990721
Merge branch 'master' into explicit_drop_part_entry_type
2023-01-31 13:49:04 +01:00
Alexander Tokmakov
2e8100b6e4
make separate DROP_PART log entry type
2023-01-31 13:37:56 +01:00
Alexander Tokmakov
51b0f66e6b
Merge pull request #45793 from ClickHouse/fix_part_check_cancellation
...
Fix race condition on a part check cancellation
2023-01-31 14:15:17 +03:00
Antonio Andelic
c99efa75b7
Merge branch 'master' into custom-key-parallel-replicas
2023-01-31 11:58:30 +01:00
Alexander Gololobov
4eea3de654
Do not restrict count() query to 1 thread in isStorageTouchedByMutations()
2023-01-30 22:29:28 +01:00
Alexander Tokmakov
e42cd3163f
fix race condition on a part check cancellation
2023-01-30 18:48:20 +01:00
Arthur Passos
48e5211454
Merge branch 'ClickHouse:master' into add_select_final_mt_setting
2023-01-30 13:01:36 -03:00
alesapin
be58d5d1af
Fix bug in tables drop which can lead to potential query hung
2023-01-30 17:00:28 +01:00
Smita Kulkarni
6cf2498609
Fixed clang-tidy build for comparing strings and updated to notify after adding subscriber if queue is already empty- Updated checking of SYSTEM SYNC REPLICA
2023-01-30 08:18:20 +01:00
SmitaRKulkarni
809354dc30
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica
2023-01-29 11:28:21 +01:00
Alexey Milovidov
49a59ff9bd
Fix bad comment
2023-01-29 03:57:59 +01:00
Smita Kulkarni
9b457a57a9
Fixed clang-tidy build - Updated checking of SYSTEM SYNC REPLICA
2023-01-28 21:43:41 +01:00
Smita Kulkarni
2bbe08ce8b
Updated destructor to notify even when queue empty - Updated checking of SYSTEM SYNC REPLICA
2023-01-27 22:37:42 +01:00
Anton Popov
d8fe2bcbaa
Merge branch 'master' into randomize-mt-settings
2023-01-27 16:03:00 +01:00
Anton Popov
3f1db52d51
Merge remote-tracking branch 'upstream/master' into compact-parts-vertical-merge
2023-01-27 14:38:37 +00:00
Nikolai Kochetov
313947e0b0
Merge branch 'master' into issue_45590
2023-01-27 15:08:28 +01:00
SmitaRKulkarni
b22fd40d92
Merge branch 'master' into 45508_Update_strategy_for_system_sync_replica
2023-01-27 15:00:47 +01:00
Anton Popov
f10e82355e
Merge pull request #45630 from CurtizJ/fix-performance-columns-description
...
Fix performance of short queries with `Array` columns
2023-01-27 14:59:55 +01:00
Anton Popov
9c1717b42d
fix computation of granularity in vertical merges
2023-01-27 13:35:04 +00:00
Nikolai Kochetov
f6ad6296ba
Restore.
2023-01-27 12:45:41 +00:00
Alexander Tokmakov
1494262a38
Update DataPartStorageOnDiskFull.cpp
2023-01-27 15:42:41 +03:00
Nikolai Kochetov
b716e2d754
A bit cleaner fix.
2023-01-27 12:37:23 +00:00
alesapin
716d25c33f
Merge pull request #45619 from CurtizJ/refactoring-merge-tree-parts
...
Refactoring of code near merge tree parts
2023-01-27 13:36:33 +01:00
Smita Kulkarni
a95609e0bc
Updated to validate & remove each element instead of just last element - Updated checking of SYSTEM SYNC REPLICA
2023-01-27 10:35:12 +01:00
Amos Bird
716516f84a
Fix test
2023-01-27 16:58:41 +08:00
Amos Bird
fd60954081
Fix key description on duplicate primary keys
...
This can happen in projections. See
https://github.com/ClickHouse/ClickHouse/issues/45590 for details.
2023-01-27 13:01:36 +08:00
Anton Popov
fb3f7409fa
allow vertical merges from compact to wide parts
2023-01-26 22:18:04 +00:00
Alexander Tokmakov
061026c307
fix
2023-01-26 19:01:29 +01:00
Alexander Tokmakov
067b1f5f13
Merge branch 'master' into exception_message_patterns4
2023-01-26 15:20:58 +01:00
Antonio Andelic
714fad1529
Add support for settings alias
2023-01-26 14:06:46 +00:00
Alexander Tokmakov
f676e9eaff
Merge pull request #45613 from azat/replicated-improve-log
...
Improve "at least part X is missing" error message
2023-01-26 16:48:18 +03:00
Anton Popov
92fa39798b
Merge remote-tracking branch 'upstream/master' into HEAD
2023-01-26 13:29:23 +00:00
Anton Popov
87af86e9a3
fix projections
2023-01-26 13:11:31 +00:00
Alexander Tokmakov
14db798191
fix
2023-01-26 13:56:16 +01:00
Antonio Andelic
07b2c5ce0f
Merge branch 'master' into custom-key-parallel-replicas
2023-01-26 12:42:04 +01:00
Anton Popov
b52cf17fa6
Merge pull request #45612 from ClickHouse/CurtizJ-patch-7
...
Remove useless code
2023-01-26 12:38:52 +01:00
Smita Kulkarni
8c885ffeae
Updated checking of SYSTEM SYNC REPLICA
...
Implementation:
* Updated to wait for current last entry to be processed (after pulling shared log) instead of queue size becoming 0.
* Updated Subscriber to notify both queue size and removed log_entry_id.
2023-01-26 11:40:09 +01:00
Alexander Tokmakov
a584ad0eb1
forbid runtime strings
2023-01-26 10:52:47 +01:00
Antonio Andelic
6fe9e9a67f
Merge pull request #45617 from ClickHouse/fix-bfngram-index
...
Correctly check types when using N-gram bloom filter index
2023-01-26 09:05:56 +01:00
Alexander Tokmakov
9b670946db
Merge branch 'master' into exception_message_patterns5
2023-01-26 00:41:32 +01:00
Anton Popov
79ea6a1c9c
fix performance of short queries
2023-01-25 22:20:01 +00:00
Han Fei
0221b50028
Merge pull request #45601 from hanfei1991/hanfei/async-cache-conflict
...
fix race of async-insert-blockids-cache
2023-01-25 22:28:42 +01:00
Alexander Tokmakov
3744fa2c63
format more messages
2023-01-25 21:16:42 +01:00
Anton Popov
0f46c1ecc6
fix build
2023-01-25 20:13:54 +00:00
Azat Khuzhin
00c341850f
Improve "at least part X is missing" error message
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-25 19:53:45 +01:00
Anton Popov
a455083b87
Merge remote-tracking branch 'upstream/master' into refactoring-merge-tree-parts
2023-01-25 18:01:12 +00:00
Kruglov Pavel
83a212a128
Merge branch 'master' into more-fsync-fixes
2023-01-25 18:40:41 +01:00
Anton Popov
8e3698c91f
refactoring of code near merge tree parts
2023-01-25 17:34:09 +00:00
Antonio Andelic
7fa906487c
Fix ngram index with null
2023-01-25 16:40:04 +00:00
Anton Popov
d9aabb48c3
remove useless code
2023-01-25 16:11:15 +00:00
Alexander Tokmakov
6eb557b2ba
Merge branch 'master' into exception_message_patterns4
2023-01-25 13:49:17 +01:00
Robert Schulze
59528cfca0
Merge pull request #45460 from ClickHouse/inv-index-cleanup
...
Cleanup of inverted index
2023-01-25 13:23:38 +01:00
Han Fei
8c4bbcd0fc
fix race of async-insert-blockids-cache
2023-01-25 12:41:01 +01:00
Sergei Trifonov
0d1ea05ff6
Merge pull request #45007 from ClickHouse/cancellable-mutex-integration
...
Fast shared mutex integration
2023-01-25 11:15:46 +01:00
Kseniia Sumarokova
2b8b1ad5d4
Additional check in MergeTreeReadPool ( #45515 )
...
* Check ranges
* Check equality just in case
* Check under ndebug
2023-01-24 23:56:26 +01:00
Azat Khuzhin
54ef885a61
Fsync all small files at once after mutation
...
Everything else is handled in MergedBlockOutputStream::finalizePart()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-24 14:08:37 +01:00
Azat Khuzhin
4b775ccdd5
Do fsync all files at once for fetched parts to decrease latency
...
For filessystems like ext4, fsync of one file will handle all operations
before, so this can be pretty time consuming.
And in case of you write multiple files in a loop, and at the end of
each iteration sync each file, then during writing of this file there
can be other operations in journal, and hence more work for fsync.
Let's call fsync for all files at once instead, like
MergedBlockOutputStream does.
Hope that keeping all file buffers till the end, will not cause troubles
(buffering and so forth).
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-24 14:08:37 +01:00
Azat Khuzhin
b370e3faa5
Move fsync inside transaction callback in DataPartStorageOnDisk::rename()
...
Otherwise, it is useless.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-24 14:08:37 +01:00
Antonio Andelic
95853af459
Merge branch 'master' into custom-key-parallel-replicas
2023-01-24 10:49:40 +00:00
Robert Schulze
9ff2bfcbf5
Merge remote-tracking branch 'origin/master' into inv-index-cleanup
...
src/Interpreters/GinFilter.cpp
src/Interpreters/InterpreterCreateQuery.cpp
src/Storages/MergeTree/MergeTreeData.cpp
src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp
src/Storages/MergeTree/MergeTreeIndexInverted.cpp
2023-01-24 10:09:42 +00:00
Alexander Tokmakov
bb4c8e169f
check number of parameters in format string
2023-01-23 23:16:16 +01:00
Alexander Tokmakov
3f6594f4c6
forbid old ctor of Exception
2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages ( #45449 )
...
* save format string for NetException
* format exceptions
* format exceptions 2
* format exceptions 3
* format exceptions 4
* format exceptions 5
* format exceptions 6
* fix
* format exceptions 7
* format exceptions 8
* Update MergeTreeIndexGin.cpp
* Update AggregateFunctionMap.cpp
* Update AggregateFunctionMap.cpp
* fix
2023-01-24 00:13:58 +03:00
Robert Schulze
7adec3d8a4
Update src/Storages/MergeTree/MergeTreeIndexInverted.cpp
...
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
2023-01-23 20:53:25 +01:00
Robert Schulze
d8af5bc2ef
Update src/Storages/MergeTree/MergeTreeIndexInverted.cpp
...
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
2023-01-23 20:53:17 +01:00
Maksim Kita
7b48c75e82
Merge pull request #45485 from kitaisreal/remove-unnecessary-get-total-row-count-function-calls
...
Remove unnecessary getTotalRowCount function calls
2023-01-23 21:02:51 +03:00
Anton Popov
08b2e524c7
Merge remote-tracking branch 'upstream/master' into HEAD
2023-01-23 15:34:53 +00:00
Kseniia Sumarokova
8ec39ab6f3
Merge pull request #45442 from kssenii/minor-changes
...
Minor improvements around reading from remote
2023-01-23 15:46:30 +01:00
Sergei Trifonov
0fbfa17863
Merge branch 'master' into cancellable-mutex-integration
2023-01-23 12:44:09 +01:00
Antonio Andelic
44ef00dc05
Merge branch 'master' into custom-key-parallel-replicas
2023-01-22 14:20:58 +00:00
Robert Schulze
531e64dc39
Merge branch 'master' into inv-index-cleanup
2023-01-22 13:20:38 +01:00
Nikolai Kochetov
b877c484d2
Merge pull request #45481 from ClickHouse/fix-deadlock-with-allow_asynchronous_read_from_io_pool_for_merge_tree
...
Fix possible deadlock with allow_asynchronous_read_from_io_pool_for_merge_tree in case of exception from ThreadPool::schedule
2023-01-21 12:05:34 +01:00
Sema Checherinda
962894afc8
Merge pull request #44909 from CheSema/intersect-prev-part
...
Do not merge over a gap with outdated undeleted parts
2023-01-21 11:51:21 +01:00
Maksim Kita
47385a19e7
Remove unnecessary getTotalRowCount function calls
2023-01-21 11:27:07 +01:00
Nikolai Kochetov
dcd84c152a
Fix possible deadlock with allow_asynchronous_read_from_io_pool_for_merge_tree in case of exception from ThreadPool::schedule
2023-01-20 18:57:47 +00:00
Alexander Tokmakov
910d6dc0ce
Merge pull request #45342 from ClickHouse/exception_message_patterns
...
Save message format strings for DB::Exception
2023-01-20 18:46:52 +03:00
Robert Schulze
1a966a9590
Fix bad comparison
2023-01-20 13:05:06 +00:00
kssenii
8d20af8127
Fix
2023-01-20 13:34:23 +01:00
Robert Schulze
bfc3b4f5ca
Suffix "GinFilter" --> "Inverted"
2023-01-20 12:02:35 +00:00
Robert Schulze
0738b2499c
Use GinFilters typedef where possible
2023-01-20 11:52:04 +00:00
Robert Schulze
0b77f07f67
Remove superfluous check (the same is checked in MergeTreeIndices.cpp)
2023-01-20 11:50:35 +00:00
Robert Schulze
d2c830ec39
Cosmetics
2023-01-20 11:49:08 +00:00
Robert Schulze
72973076c9
Rename MergeTreeIndexGin.h/cpp to MergeTreeIndexInverted.h/cpp
2023-01-20 11:42:36 +00:00
Robert Schulze
1ef2704539
Cosmetics
2023-01-20 11:39:23 +00:00
Robert Schulze
463cc843de
"segment file" --> "segment metadata file"
2023-01-20 11:26:22 +00:00
Robert Schulze
58df3953bb
Move some code around (no other changes)
2023-01-20 11:24:23 +00:00
Robert Schulze
3267ac2787
Prefix more typedefs in DB namespace with "Gin"
2023-01-20 11:19:07 +00:00
Antonio Andelic
fc9c53f963
Merge branch 'master' into custom-key-parallel-replicas
2023-01-20 11:18:30 +00:00
Robert Schulze
919b67f117
Cosmetics
2023-01-20 11:15:28 +00:00
Robert Schulze
98e117dca6
SegmentDictionary --> GinSegmentDictionary, also move typedef
2023-01-20 11:09:49 +00:00
Robert Schulze
908fa83f72
Move some typedefs around
2023-01-20 11:08:19 +00:00
Robert Schulze
44618927f9
Inline two short methods + uppercase
2023-01-20 11:04:35 +00:00
Robert Schulze
5c3cc5283f
"term dictionary" --> "dictionary"
2023-01-20 10:53:41 +00:00
Robert Schulze
be936b257c
Make version enum private
2023-01-20 10:48:43 +00:00
Robert Schulze
0653f86de9
Various cosmetic cleanups
2023-01-20 10:45:35 +00:00
Frank Chen
f867c4c206
Add an overriden fromCurrentException to simply the code
2023-01-20 16:43:10 +08:00
Frank Chen
38799eb852
Merge two methods as one
2023-01-20 16:35:28 +08:00
Frank Chen
be8ceeb4bb
Save stack trace in part_log
2023-01-20 16:22:40 +08:00
Frank Chen
1fe9bde594
Set replica name to zero copy exclusive lock
2023-01-20 15:48:33 +08:00
Frank Chen
fb8452aa49
Add last_exception_time to replication_queue
2023-01-20 15:20:57 +08:00
kssenii
4ce8950712
Minor changes
2023-01-19 17:53:10 +01:00
larryluogit
52ae33dba7
Merge branch 'master' into ftsearch
2023-01-19 11:34:11 -05:00
Sema Checherinda
ae1dfb9ce5
Update src/Storages/MergeTree/MergeTreeData.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-01-18 15:21:11 +01:00
Han Fei
e51123c9b0
fix data race
2023-01-18 13:11:07 +01:00
Antonio Andelic
a37ca4b961
Merge branch 'master' into custom-key-parallel-replicas-with-shard
2023-01-18 11:53:04 +00:00
Sergei Trifonov
c443c1ece0
Merge branch 'master' into hanfei/async-cache
2023-01-18 00:19:49 +01:00
Alexander Tokmakov
5cd90c1a3e
Merge branch 'master' into exception_message_patterns
2023-01-17 20:04:04 +01:00
Alexander Tokmakov
72e8615bec
formatting of some exception messages
2023-01-17 20:03:56 +01:00
Alexander Tokmakov
8b13b85ea0
Merge pull request #44543 from ClickHouse/text_log_add_pattern
...
Add a column with a message pattern to text_log
2023-01-17 20:19:32 +03:00
Han Fei
8a74238fe0
improve
2023-01-17 15:47:52 +01:00
HarryLeeIBM
e7add8218f
Addressed more review comments and ClangTidy errors
2023-01-17 06:29:13 -08:00
Antonio Andelic
a85565c4a6
Merge branch 'master' into custom-key-parallel-replicas
2023-01-17 07:37:18 +00:00
Alexander Tokmakov
522686f78b
less empty patterns
2023-01-17 01:19:44 +01:00
Alexander Tokmakov
870cfcc36a
less fmt::runtime usages
2023-01-17 00:11:59 +01:00
Sema Checherinda
90fa1ecd49
make that old_parts_lifetime=0 deletes files instantly at drop/truncate
2023-01-16 16:47:12 +01:00
Sema Checherinda
c51f4d7be1
do not merge over a gap with uotdate parts, delete empty parts with respect to old_parts_lifetime
2023-01-16 16:47:11 +01:00
Sema Checherinda
25e16388d7
better message in MergeTreeDataMergerMutator when parts intersect
2023-01-16 16:47:11 +01:00
Anton Popov
6863cd152f
Merge pull request #42181 from CurtizJ/optimize-loading-parts
...
Do not load inactive parts at startup
2023-01-16 14:38:50 +01:00
Han Fei
30a798182a
Merge branch 'master' into hanfei/async-cache
2023-01-16 14:07:36 +01:00
Antonio Andelic
36cf9debae
Merge branch 'master' into custom-key-parallel-replicas
2023-01-16 08:11:08 +00:00
Han Fei
5617f7f616
address comments
2023-01-15 22:51:10 +01:00
Han Fei
701dc88d6f
Merge branch 'master' into hanfei/support-advance-dedup
2023-01-15 19:46:28 +01:00
Han Fei
c859f8dbe5
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp
...
Co-authored-by: alesapin <alesapin@gmail.com>
2023-01-15 19:46:16 +01:00
Han Fei
bb2c0914e9
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp
...
Co-authored-by: alesapin <alesapin@gmail.com>
2023-01-15 19:46:09 +01:00
Alexander Tokmakov
d857d62a03
remove another set of macros
2023-01-13 20:34:31 +01:00
Alexander Tokmakov
2d7773fccc
Merge branch 'master' into text_log_add_pattern
2023-01-13 20:33:46 +01:00
Han Fei
ed49ebf01a
update setting explain
2023-01-13 20:26:08 +01:00
Han Fei
2fb2f503e3
Update src/Storages/MergeTree/MergeTreeSettings.h
...
Co-authored-by: Sergei Trifonov <svtrifonov@gmail.com>
2023-01-13 20:20:08 +01:00
Han Fei
9e99c7e116
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp
...
Co-authored-by: Sergei Trifonov <svtrifonov@gmail.com>
2023-01-13 20:19:13 +01:00
Han Fei
a258a39eb1
Merge branch 'master' into hanfei/async-cache
2023-01-13 20:17:58 +01:00
Anton Popov
487de70d01
fix locking at loading outdated data parts
2023-01-13 17:05:32 +00:00
Anton Popov
71188c22ee
fix race on 'relative_data_path'
2023-01-13 12:19:41 +00:00
Alexander Tokmakov
51d94314d6
Merge pull request #45235 from ClickHouse/more_verbose_logs_about_replication_log_entries
...
More verbose logs about replication log entries
2023-01-13 15:05:21 +03:00
Antonio Andelic
e64f07e72b
Merge branch 'master' into custom-key-parallel-replicas
2023-01-13 08:44:53 +00:00
Alexander Tokmakov
d9b8cf9c04
more verbose logs about replication log entries
2023-01-12 20:32:39 +01:00
serxa
693489a8ad
review fixes
2023-01-12 15:51:04 +00:00
Sergei Trifonov
12d8543578
Merge branch 'master' into cancellable-mutex-integration
2023-01-12 16:03:49 +01:00
Arthur Passos
301b684933
remove trialing whitespace?
2023-01-12 10:00:18 -03:00
Anton Popov
ca80240d84
Merge remote-tracking branch 'upstream/master' into HEAD
2023-01-12 12:13:45 +00:00
Arthur Passos
1b3c7766f8
Merge branch 'master' into add_select_final_mt_setting
2023-01-12 08:31:04 -03:00
Anton Popov
8bdf63f8e5
Merge pull request #45191 from ClickHouse/fix-ip-part-name
...
Fix part ID generation for IP types for backward compatibility
2023-01-12 12:15:06 +01:00
Igor Nikonov
edbcb24da0
Merge pull request #44954 from ClickHouse/fix-insert-delay
...
Fix: insert delay calculation
2023-01-12 09:43:00 +01:00
Antonio Andelic
ecbffa80b6
Add READ_TASKS mode
2023-01-12 07:56:15 +00:00
HarryLeeIBM
6d576cf157
Added more documentation
2023-01-11 13:40:20 -08:00
Yakov Olkhovskiy
9c4ea5a16b
fix part ID generation for IP types for backward compatibility
2023-01-11 18:51:37 +00:00
Alexander Tokmakov
0fba3d6d81
Merge pull request #44468 from ClickHouse/fix_get_part_name
...
Fix incorrect usages of `getPartName()`
2023-01-11 16:24:02 +03:00
Igor Nikonov
1d002e45a9
Fix test & review comments
2023-01-11 12:49:01 +00:00
Nikolai Kochetov
5e7a6ac619
Merge pull request #45122 from ClickHouse/revert-45121-revert-44653-custom-reading-for-mutation
...
Revert "Revert "Custom reading for mutation""
2023-01-11 12:37:32 +01:00
Nikolai Kochetov
d0c1079242
Merge pull request #45133 from ClickHouse/fix-additinal-tables-filter-with-count-projection
...
Fix additional_table_filters with minmax/count projection.
2023-01-11 12:25:40 +01:00
Antonio Andelic
19e5988d33
Better
2023-01-11 09:57:13 +00:00
Anton Popov
85734bb321
fix build
2023-01-11 08:32:33 +00:00
Anton Popov
6c4f53bdca
Merge remote-tracking branch 'upstream/master' into HEAD
2023-01-11 01:00:57 +00:00
HarryLeeIBM
150f9b48a4
Fix functional test for full text search
2023-01-10 14:35:06 -08:00
Igor Nikonov
543221b8b9
Merge remote-tracking branch 'origin/master' into fix-insert-delay
2023-01-10 22:15:41 +00:00
Igor Nikonov
e75df5e76b
Test
...
+ fix some review comments
2023-01-10 22:14:54 +00:00
Alexander Tokmakov
881b17492f
Merge branch 'master' into fix_get_part_name
2023-01-10 21:39:35 +01:00
Alexander Tokmakov
255881a2ff
Merge pull request #44466 from ClickHouse/parallel_parts_cleanup_with_zero_copy
...
Parallel parts cleanup with zero copy replication
2023-01-10 22:44:39 +03:00
Ilya Yatsishin
0efceae6ce
Merge pull request #44917 from azat/mt-fix-log-message
...
Fix logging message in MergeTreeDataMergerMutator (about merged parts)
2023-01-10 19:49:31 +01:00
Nikolai Kochetov
9a81f27fb2
Fix additional_table_filters with minmax/count projection.
2023-01-10 17:32:26 +00:00
HarryLeeIBM
bc48b17817
Addressed more review comments.
2023-01-10 08:26:27 -08:00
Antonio Andelic
347ccab8f9
Merge branch 'master' into custom-key-parallel-replicas
2023-01-10 16:31:40 +01:00
Nikolai Kochetov
4673b3fe1d
Revert "Revert "Custom reading for mutation""
2023-01-10 16:31:01 +01:00
Alexander Tokmakov
c8ec130be4
Revert "Custom reading for mutation"
2023-01-10 17:51:30 +03:00
Antonio Andelic
fbe4808b6b
Add support for custom key in parallel replicas
2023-01-10 13:13:43 +00:00
Han Fei
bef6168c05
support cache for async inserts block ids
2023-01-10 13:19:12 +01:00
Anton Popov
b9bf92fa5d
check for network errors while loading parts
2023-01-10 11:55:08 +00:00
Nikolai Kochetov
11418963c0
Merge pull request #44653 from ClickHouse/custom-reading-for-mutation
...
Custom reading for mutation
2023-01-10 12:16:24 +01:00
alesapin
616e37ba43
Merge pull request #44847 from ClickHouse/fix_rare_race
...
Fix rare race which can lead to queue hang
2023-01-10 12:09:10 +01:00
Nikolai Kochetov
3e430b1b6e
Add comments. Remove unused flag from Context.
2023-01-09 19:52:11 +00:00
Anton Popov
1e5d6e44a5
fix deadlock at shutdown
2023-01-09 13:07:46 +00:00
Sergei Trifonov
81d2ea30ba
Merge branch 'master' into cancellable-mutex-integration
2023-01-07 19:37:46 +01:00
Anton Popov
1f32ffedf8
Merge pull request #43221 from ClickHouse/refactoring-ip-types
...
Replace domain IP types (IPv4, IPv6) with native
2023-01-07 12:01:21 +01:00
Alexey Milovidov
9fcd111724
Merge pull request #45003 from amosbird/issue_44979
...
Fix minmax_count_projection with _partition_value
2023-01-07 03:36:56 +03:00
serxa
15bb127b01
replace every std::shared_mutex with DB::FastSharedMutex
2023-01-06 23:35:38 +00:00
Amos Bird
200d40c53d
Fix minmax_count_projection with _partition_value
...
Check if virtual column `_partition_value` can be used when building
minmax_count_projection block.
2023-01-07 05:38:28 +08:00
Dmitry Novik
9d44c7a737
Merge pull request #44541 from ClickHouse/cleanup-resolved-function
...
Followup #43761
2023-01-06 18:09:32 +01:00
kssenii
8fde606768
Resolve some review comments
2023-01-06 16:10:00 +01:00
kssenii
2a030c1dc0
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2023-01-06 15:58:27 +01:00
alesapin
da83e97b4c
Fix startup issue
2023-01-06 15:36:53 +01:00
Anton Popov
b25f875674
Merge pull request #44875 from ClickHouse/fix-another-one-cannot-read-all-data-for-lc-dict-error
...
Fix right offset for reading LowCardinality dictionary from remote fs
2023-01-06 15:24:36 +01:00
alesapin
b20f07b2cc
Merge branch 'master' into fix_rare_race
2023-01-06 14:20:08 +01:00
Anton Popov
fcdbc8123d
better loading of outdated parts
2023-01-06 11:59:54 +00:00
Alexey Milovidov
b88be9df8c
Merge pull request #44956 from ClickHouse/fix_read_result_consistency_check_on_exception
...
Do not check read result consistency when unwinding
2023-01-06 06:22:42 +03:00
Alexey Milovidov
a205b41c56
Merge pull request #44947 from Algunenano/background_pool_size_logical
...
Change error code on invalid background_pool_size config
2023-01-06 03:19:40 +03:00
Alexander Gololobov
1b31b8bb91
Do not check read result consistency when unwinding
2023-01-06 00:26:08 +01:00
Igor Nikonov
a67afdff6a
Fix: insert delay calculation
2023-01-05 22:43:41 +00:00
Dmitry Novik
ebead611b6
Merge branch 'master' into cleanup-resolved-function
2023-01-05 19:27:08 +01:00
Raúl Marín
b0715ec5eb
Change error code on invalid background_pool_size config
2023-01-05 18:47:02 +01:00
Anton Popov
ed00399eb6
Merge remote-tracking branch 'upstream/master' into HEAD
2023-01-05 14:12:02 +00:00
Larry Luo
f2bc5bb335
Added initializer
2023-01-05 05:41:29 -08:00
Igor Nikonov
2a105d1862
Fix stupid mistake - max() -> min()
2023-01-05 12:38:27 +00:00
Igor Nikonov
db46267a95
Fast fix: force upper bound for time to delay INSERT
2023-01-05 12:38:27 +00:00
Larry Luo
fd0c6cccc8
Fixed style error
2023-01-04 20:08:28 -08:00
Larry Luo
7a4d499cb1
Added density to control Inverted index size
2023-01-04 19:42:45 -08:00
Azat Khuzhin
d06ac2d468
Fix logging message in MergeTreeDataMergerMutator (about merged parts)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-04 19:07:01 +01:00
Yakov Olkhovskiy
7a5a36cbed
Merge branch 'master' into refactoring-ip-types
2023-01-04 11:11:06 -05:00
alesapin
2cd7247619
Merge branch 'master' into fix_rare_race
2023-01-04 15:06:20 +01:00
Kruglov Pavel
9056925c7d
Merge pull request #44874 from Algunenano/time_fix
...
Do not throw DB::Exception when folders do not exist
2023-01-04 01:17:57 +01:00
Nikolai Kochetov
c51f5e6fac
Fix typo.
2023-01-03 18:41:37 +00:00
Nikolai Kochetov
da26f62a9b
Fix right offset for reading LowCardinality dictionary from remote fs in case if right mark was in the middle of compressed block.
2023-01-03 18:19:51 +00:00
Dmitry Novik
a84318af5f
Merge branch 'master' into cleanup-resolved-function
2023-01-03 17:52:29 +01:00
Raúl Marín
0785bf6b29
Do not throw DB::Exception when folders do not exist
2023-01-03 17:05:54 +01:00
Nikolai Kochetov
6b1a697b12
Merge pull request #44850 from ClickHouse/respect-context-in-projection-optimization
...
Save context created from projection analysis.
2023-01-03 13:30:51 +01:00
Han Fei
7d7de3833b
Merge pull request #44651 from hanfei1991/hanfei/fix-async-insert-cleanup
...
fix bug that async blocks cleanup not work
2023-01-02 21:58:27 +01:00
Dmitry Novik
929300a721
Update MergeTreeData.cpp
2023-01-02 21:18:16 +01:00
Dmitry Novik
04fb410a40
Merge branch 'master' into parallel_parts_cleanup_with_zero_copy
2023-01-02 20:48:39 +01:00
Dmitry Novik
c646048af9
Merge pull request #44461 from ClickHouse/parts-removal-limit
...
Add part removal limit for one CleanupThread iteration
2023-01-02 20:04:19 +01:00
Alexey Milovidov
e855d3519a
Merge branch 'master' into refactoring-ip-types
2023-01-02 21:58:53 +03:00
Nikolai Kochetov
09325ff17e
Store context created from projection analysis.
2023-01-02 17:00:48 +00:00
alesapin
6fc63878b4
Fix rare race which can lead to queue hang
2023-01-02 16:57:25 +01:00
Han Fei
eef0136415
add debug log
2023-01-02 11:59:44 +01:00
Larry Luo
7cc84c6743
refactor filter functions support
2022-12-31 18:05:07 -08:00
Larry Luo
71da1b1379
Updated inverted index to use ActionsDAG
2022-12-31 18:05:06 -08:00
Larry Luo
f0a2c06261
Fixed errors caused by merging.
2022-12-31 18:05:06 -08:00
HarryLeeIBM
d7a7de9601
Minor fixes for addressing reviews
2022-12-31 18:05:06 -08:00
HarryLeeIBM
6a5fef632d
Added more tests and experimental flag, etc.
2022-12-31 18:05:01 -08:00
Harry-Lee
b43ddde50a
Refactor full text search according to reviews
2022-12-31 18:03:14 -08:00
Larry Luo
00e4411740
Created FST namespace to hide FST details
2022-12-31 18:03:14 -08:00
Larry Luo
64f807f31a
fixed an edge case that index column has no data
2022-12-31 18:03:14 -08:00
Larry Luo
da93210cc6
Updated MergeTreeConditionGinFilter due to SelectQueryInfo changes.
2022-12-31 18:03:14 -08:00
Larry Luo
a33232cb0a
Clean up code
2022-12-31 18:03:14 -08:00
Larry Luo
7e5f311da9
Fix MemorySanitizer error
2022-12-31 18:03:14 -08:00
Larry Luo
0a47378e5f
Added initializer for current segment
2022-12-31 18:03:14 -08:00
Larry Luo
514e17d037
Code clean up
2022-12-31 18:03:14 -08:00
Larry Luo
1d9bf9962c
Code clean up
2022-12-31 18:03:14 -08:00
Larry Luo
657ce7c614
Cleanup code and tests
2022-12-31 18:03:13 -08:00
Larry Luo
3652740535
Code clean up
2022-12-31 18:03:13 -08:00
Larry Luo
0b63d6b5be
Resolve conflicts and style errors.
2022-12-31 18:03:13 -08:00
HarryLeeIBM
bc8be368bb
Fix coding styles
2022-12-31 18:03:13 -08:00
Larry Luo
91b79b26e3
Use DataPartStorage in GinIndexStore
2022-12-31 18:03:02 -08:00
Harry-Lee
4b29c02f35
Remove useless FUNCTION_LIKE
2022-12-31 18:00:17 -08:00
HarryLeeIBM
e58f0ef3d2
Remove roaring bmp cleanup code which can cause memory leak
2022-12-31 18:00:17 -08:00
HarryLeeIBM
0839ff60ee
Initial Checkin of full text search
2022-12-31 18:00:17 -08:00
Alexey Milovidov
50bb7f90d8
Merge pull request #44809 from ClickHouse/fix-race-in-mergetreedatapart
...
Fix race in `system.parts` and `system.parts_columns`
2023-01-01 01:09:46 +03:00
Han Fei
a036c6bc01
Merge branch 'master' into hanfei/fix-async-insert-cleanup
2022-12-31 20:52:16 +01:00
Alexey Milovidov
3a5cea8281
Fix race in system.parts and system.parts_columns
2022-12-31 15:07:26 +01:00
Alexey Milovidov
26eb2f57e1
Fix race in system.parts and system.parts_columns
2022-12-31 15:06:57 +01:00
Azat Khuzhin
e7bf2cc959
Add a comment for lack of fsync in MergedBlockOutputStream::finalizePartOnDisk()
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-30 22:39:13 +01:00
Azat Khuzhin
9fb6004c92
Add missing fsync for small files (ttl.txt, columns.txt, ...) in mutations
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-30 22:22:39 +01:00
Nikolai Kochetov
758f30cc17
Cleanup.
2022-12-30 17:53:46 +00:00
Han Fei
454f40ab17
fix test
2022-12-30 16:30:33 +01:00
