alesapin
79ef72178e
Less garbage
2020-09-04 17:18:49 +03:00
alesapin
f48d654d35
Less garbage
2020-09-04 17:15:34 +03:00
alesapin
fecb2f1311
Less copypaste
2020-09-04 17:08:43 +03:00
alesapin
ea7168580b
Fixes
2020-09-04 16:55:07 +03:00
Anton Popov
68913eab62
better reading from compact parts with differents codecs
2020-09-04 15:48:55 +03:00
alesapin
f8aa6cfe8a
Merge pull request #14438 from ClickHouse/apply_ttl_if_not_calculated
...
Apply TTL if it's not calculated for part
2020-09-04 15:32:50 +03:00
alesapin
f9dd4cc98d
Merge branch 'control_ttl_merges_in_pool' into recompression_in_background
2020-09-04 14:32:25 +03:00
alesapin
e42d0f60da
Fix several bugs
2020-09-04 14:27:27 +03:00
alesapin
4079636f81
Merge
2020-09-04 13:52:51 +03:00
alesapin
69b31ab90d
More comments
2020-09-04 13:29:55 +03:00
alesapin
82c56349a5
Some comments
2020-09-04 13:08:09 +03:00
Nikolai Kochetov
c761970bc5
Merge branch 'master' into pipes-and-creating-sets
2020-09-04 11:38:35 +03:00
alesapin
61ecaebcb1
Simplify settings for TTL merges
2020-09-04 09:55:19 +03:00
Anton Popov
6883ee7eea
create less compressed streams while writing compact parts
2020-09-04 01:38:17 +03:00
Anton Popov
d5da58918e
create less compressed streams while writing compact parts
2020-09-04 01:04:46 +03:00
Anton Popov
98011f6fc3
Merge remote-tracking branch 'upstream/master' into HEAD
2020-09-03 17:53:05 +03:00
alesapin
0dab4d9485
Merge branch 'fix_select_parts_for_merge' into control_ttl_merges_in_pool
2020-09-03 17:41:23 +03:00
alesapin
15a02321bf
Merge branch 'fix_select_parts_for_merge' into control_ttl_merges_in_pool
2020-09-03 17:16:29 +03:00
alesapin
55adb6f9f2
Update src/Storages/MergeTree/MergeSelector.h
...
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2020-09-03 16:55:34 +03:00
alesapin
e48e4309b2
Better names
2020-09-03 16:29:18 +03:00
alesapin
565555e55c
Fix bug in parts selection
2020-09-03 16:04:28 +03:00
alesapin
13248a744b
Fix bug in parts selection
2020-09-03 16:02:24 +03:00
alesapin
f4c7ff0376
Add fixed size of Merge TTLS
2020-09-03 16:00:13 +03:00
alesapin
aa47d0aabc
Merge branch 'apply_ttl_if_not_calculated' into recompression_in_background
2020-09-03 12:07:03 +03:00
alesapin
acc0ee0657
Apply TTL if it's not calculated for part
2020-09-03 11:59:41 +03:00
alesapin
dcbddbb8d0
Add recompression ttls tests and fix bugs
2020-09-02 19:15:41 +03:00
Nikolai Kochetov
92c937db8b
Remove CreatingSetsBlockInputStream
2020-09-02 16:13:13 +03:00
alesapin
128cb7ce22
Don't select already selected parts
2020-09-02 15:16:12 +03:00
alesapin
aac466ab1c
Merge branch 'better_ttl_merges_selection' into recompression_in_background
2020-09-02 13:30:04 +03:00
alesapin
fbb37c37df
Simplier interface
2020-09-02 11:28:46 +03:00
alesapin
b08056fa8c
Better selection of Merges with TTL
2020-09-02 11:18:50 +03:00
Anton Popov
4fe57fc169
Merge remote-tracking branch 'upstream/master' into HEAD
2020-09-02 01:29:44 +03:00
Anton Popov
26d75f7602
do fsync for WAL
2020-09-02 01:25:10 +03:00
Anton Popov
3cadc9033a
fsyncs for metadata files of part
2020-09-01 18:53:22 +03:00
alesapin
c37a19f7b0
Better
2020-09-01 14:23:38 +03:00
alesapin
f0dc5a3085
First working test
2020-09-01 13:49:53 +03:00
alesapin
b751319f9e
Merge branch 'master' into recompression_in_background
2020-09-01 10:31:10 +03:00
alesapin
05b10048a6
Merge pull request #14116 from ClickHouse/parts_default_compression
...
Save parts default compression
2020-09-01 09:43:42 +03:00
Anton Popov
25140b9bd5
fsync MergeTree format file
2020-09-01 04:39:36 +03:00
alesapin
46f833b7df
Some changes
2020-08-31 22:50:42 +03:00
alesapin
b20a0bc254
Add recompression flag in ReplicatedEntry
2020-08-31 16:42:42 +03:00
alesapin
067eb4599d
Fix empty columns case
2020-08-31 16:39:27 +03:00
alesapin
adc18f4d3f
Write with recompression TTL
2020-08-31 16:29:31 +03:00
alesapin
42c210fcba
Recompress TTLs in memory metadata
2020-08-31 15:12:51 +03:00
alesapin
034f1a895d
Merge branch 'master' into recompression_in_background
2020-08-31 11:10:14 +03:00
alexey-milovidov
b1efc5df01
Merge pull request #14225 from amosbird/bf1
...
Fix bug in mark inclusion search.
2020-08-30 02:58:22 +03:00
alesapin
2d33a4029b
Merge pull request #14220 from ClickHouse/remove_redundant_flag
...
Disable force TTL on optimize final
2020-08-29 10:40:36 +03:00
Amos Bird
591a4d60d4
Fix bug in mark inclusion search.
2020-08-29 09:46:46 +08:00
alesapin
ebc163c924
Merge branch 'master' into parts_default_compression
2020-08-28 23:11:42 +03:00
alesapin
65c33f0802
Merge pull request #13280 from amosbird/as
...
ALTER MODIFY SAMPLE BY
2020-08-28 19:32:09 +03:00
alesapin
232c264d7d
Disable force TTL on optimise
2020-08-28 16:45:42 +03:00
alesapin
0896b49533
Small fixes
2020-08-28 13:18:56 +03:00
alesapin
77faf9587f
Better interface
2020-08-28 12:07:20 +03:00
Alexey Milovidov
5763737d97
Fixups
2020-08-28 03:53:22 +03:00
Alexey Milovidov
ed1d120de0
Added another test; fixup
2020-08-28 03:28:37 +03:00
Alexey Milovidov
c4e8aaac16
Fixups
2020-08-28 03:07:51 +03:00
Alexey Milovidov
6082697c4d
Support for quorum
2020-08-28 02:39:12 +03:00
Alexey Milovidov
cdba5e727c
Fix mistake
2020-08-28 02:30:07 +03:00
Alexey Milovidov
88db4938f5
Fix error; refinements
2020-08-28 02:22:00 +03:00
Alexey Milovidov
c04dc4e722
Merge branch 'master' into manual-write-duplicate-parts-to-replicas
2020-08-28 01:56:11 +03:00
Amos Bird
05a5a13e08
specific ReplicatedMergeTree settings
2020-08-27 22:37:41 +08:00
Amos Bird
078b14610d
ALTER MODIFY SAMPLE BY
2020-08-27 22:31:30 +08:00
alesapin
32db38b4d2
Fix compression codec in WAL
2020-08-27 17:06:14 +03:00
alesapin
d426751e26
Default codec if part is not stored on disk
2020-08-27 16:32:23 +03:00
alexey-milovidov
9baa0fbf81
Merge pull request #14129 from CurtizJ/fix-performance
...
Slightly optimize very short queries with LowCardinality
2020-08-27 15:07:19 +03:00
alesapin
1225c09a02
Fix default codec for in-memory parts
2020-08-27 11:35:55 +03:00
alesapin
886a7b485e
Fixed codec in clone part
2020-08-27 09:49:41 +03:00
bharatnc
ffb73adb11
fix style check
2020-08-26 15:31:48 -07:00
bharatnc
593f054b10
minor fix exception message
2020-08-26 15:31:48 -07:00
bharatnc
3a35aedb56
add comments explaining index_granularity_bytes safe guard
2020-08-26 15:31:48 -07:00
bharatnc
4ac9b59cec
format file
2020-08-26 15:31:48 -07:00
bharatnc
37416045ba
setting min_index_granularity_bytes
...
This PR adds a new setting called `min_index_granularity_bytes`.
If the `index_granularity_bytes` is > 0 and is lesser than the
`min_index_granularity_bytes`, throw an exception inside the
`registerStorageMergeTree.cpp` file.
2020-08-26 15:31:48 -07:00
alesapin
44eb702fbe
Remove redundant code
2020-08-26 22:41:57 +03:00
alesapin
3a680dc8fb
Fix removal bug
2020-08-26 22:40:04 +03:00
Anton Popov
eeb78bf291
slightly optimize very short queries with LowCardinality
2020-08-26 21:46:18 +03:00
alesapin
1fb93a8bf2
Remove read helpers
2020-08-26 18:57:06 +03:00
alesapin
18eb0dbcaa
Better remove
2020-08-26 18:39:11 +03:00
alesapin
2fc80189af
Add default compression codec to merge tree data part
2020-08-26 18:29:46 +03:00
Alexey Milovidov
0e021c6151
Merge branch 'master' into manual-write-duplicate-parts-to-replicas
2020-08-26 03:24:35 +03:00
Anton Popov
4331158d30
merge with master
2020-08-24 16:09:23 +03:00
Anton Popov
4df5c72878
Merge remote-tracking branch 'upstream/master' into HEAD
2020-08-24 12:57:27 +03:00
alexey-milovidov
7b53a0ef33
Revert "Added allow_merges option for volumes in multi-disk configuration ( #13402 )"
...
This reverts commit 1e2616542a
2020-08-21 18:44:29 +03:00
Vladimir Chebotarev
1e2616542a
Added allow_merges option for volumes in multi-disk configuration ( #13402 )
2020-08-21 12:04:13 +03:00
Artem Zuikov
becc186c91
Add support for extended precision integers and decimals ( #13097 )
2020-08-19 14:52:17 +03:00
alexey-milovidov
23ccb0b6be
Merge pull request #13677 from hagen1778/merge-tree-fail-fast-on-rows-limit
...
[mergeTree]: fail fast if max_rows_to_read limit exceeded on parts scan
2020-08-18 22:24:39 +03:00
roman
35e28b4c6b
[mergeTree]: make exception message more clear
2020-08-17 09:52:04 +01:00
Alexey Milovidov
5059d1f441
Merge branch 'master' into manual-write-duplicate-parts-to-replicas
2020-08-16 15:58:57 +03:00
alexey-milovidov
f3755fecb7
Merge pull request #13793 from amosbird/fnc
...
Fix set index with const column pred
2020-08-16 15:18:31 +03:00
Amos Bird
ae2c50e9e8
Fix set index with const column pred
2020-08-16 15:16:37 +08:00
Alexey Milovidov
9e6d4802ac
Do not drop cache if DROP/RENAME in Atomic database
2020-08-15 09:56:47 +03:00
roman
b637699ccd
[mergeTree]: fail fast if max_rows_to_read limit exceeded on parts scan
...
The motivation behind this change is to skip ranges scan for all selected parts
if it is clear that `max_rows_to_read` is already exceeded. The change is quite
noticeable for queries over big number of parts.
2020-08-14 13:53:48 +01:00
tavplubix
e6df1f4868
Merge pull request #13676 from amosbird/bd
...
Make drop table a little bit more robust
2020-08-14 14:14:58 +03:00
Amos Bird
de4a070d6a
log error message
2020-08-14 09:54:51 +08:00
alexey-milovidov
c06cfda3cb
Merge pull request #13678 from ClickHouse/fix-typo-2
...
Fix typo due to copy-paste
2020-08-14 04:15:23 +03:00
alexey-milovidov
eeefcd6493
Merge pull request #13337 from ClickHouse/refactor-pipes-3
...
Refactor Pipe
2020-08-14 04:12:22 +03:00
Alexey Milovidov
bcd29dc9f8
Fix typo due to copy-paste
2020-08-13 18:41:26 +03:00
Amos Bird
3b92692458
Make drop table a little bit more robust
2020-08-13 22:22:04 +08:00
Alexander Tokmakov
e4da1d8e51
fix deadlock and logical error in replication queue
2020-08-12 00:36:02 +03:00
Nikolai Kochetov
9b67cd9faf
Merge branch 'master' into refactor-pipes-3
2020-08-10 10:50:17 +03:00
alexey-milovidov
fa1c4d1eef
Merge pull request #13538 from CurtizJ/fix-skip-indices
...
Fix secondary indices corruption in compact parts
2020-08-09 20:13:13 +03:00
Anton Popov
2280619ad0
fix secondary indices corruption in compact parts
2020-08-09 04:45:10 +03:00
alexey-milovidov
aefa9b297b
Merge pull request #13449 from ClickHouse/mixed_granularity_parts_by_default
...
Enable mixed granularity parts by default
2020-08-09 01:36:59 +03:00
alexey-milovidov
c0fef71507
Merge pull request #13511 from ClickHouse/codespell
...
Fix typos in code with codespell.
2020-08-08 17:09:16 +03:00
Alexey Milovidov
306aefd33e
Change error code #13416
2020-08-08 17:05:46 +03:00
Alexey Milovidov
12f66fa82c
Fix 99% of typos
2020-08-08 04:01:47 +03:00
Alexey Milovidov
edd89a8610
Fix half of typos
2020-08-08 03:47:03 +03:00
Alexey Milovidov
bbbc52362b
Remove strange file
2020-08-08 03:13:51 +03:00
alesapin
5de2491eda
Enable mixed granularity parts by default
2020-08-07 11:55:51 +03:00
Alexey Milovidov
0ac3f0481f
Probably fix error
2020-08-07 04:27:29 +03:00
Nikolai Kochetov
d83c7cc0e6
Merge branch 'master' into refactor-pipes-3
2020-08-06 17:32:50 +03:00
Nikolai Kochetov
20e63d2271
Refactor Pipe [part 6]
2020-08-06 15:24:05 +03:00
Anton Popov
33699d8d4a
Merge pull request #13282 from amosbird/mha
...
Report broken parts in compact MergeTree
2020-08-05 14:27:32 +03:00
Alexey Milovidov
d91471ea00
Merge branch 'master' into stack-frams-size-limit
2020-08-04 15:28:47 +03:00
alesapin
943eb2f990
bump CI
2020-08-04 12:42:51 +03:00
Amos Bird
019f6d3a76
Report broken parts in compact MergeTree
2020-08-04 16:45:52 +08:00
Nikolai Kochetov
09fbce1b1e
Merge branch 'master' into refactor-pipes-3
2020-08-04 11:32:34 +03:00
alexey-milovidov
c43a27782e
Merge pull request #13153 from ClickHouse/merge-tree-settings-sanity-check
...
Sanity checks for MergeTreeSettings
2020-08-04 01:27:19 +03:00
Alexey Milovidov
1d091738dd
Merge branch 'master' into stack-frams-size-limit
2020-08-03 20:41:47 +03:00
Nikolai Kochetov
2cca4d5fcf
Refactor Pipe [part 2].
2020-08-03 16:54:14 +03:00
Alexander Tokmakov
5b92f43f12
Merge branch 'master' into database_atomic_improvements
2020-08-03 14:35:40 +03:00
Nikolai Kochetov
e411916bde
Refactor Pipe [part 1].
2020-08-03 14:33:11 +03:00
Nikolai Kochetov
cf49a83980
Merge pull request #13201 from azat/fix-logging-in-MergeTreeDataSelectExecutor
...
Fix logging in MergeTreeDataSelectExecutor for multiple threads (attach to thread group)
2020-08-03 10:08:58 +03:00
Alexey Milovidov
77516b48fa
Merge branch 'master' into merge-tree-settings-sanity-check
2020-08-03 00:30:56 +03:00
Alexey Milovidov
0fd7af8a63
Merge branch 'master' into stack-frams-size-limit
2020-08-03 00:06:00 +03:00
Alexey Milovidov
4ed0bf3af1
Better code
2020-08-03 00:01:39 +03:00
Alexey Milovidov
3c489ce159
Fix assertion in KeyCondition
2020-08-02 23:55:20 +03:00
Alexey Milovidov
5f808aa503
Fix bad code
2020-08-02 23:41:52 +03:00
alesapin
75a9053511
Merge pull request #13150 from nvartolomei/nv/freeze-verbose-absolute-path
...
Use absolute paths to backup locations (shadow)
2020-08-02 19:25:24 +03:00
Alexey Milovidov
41111d1e57
Merge branch 'master' into stack-frams-size-limit
2020-08-02 19:18:33 +03:00
Alexey Milovidov
778abb346f
Merge branch 'master' into merge-tree-settings-sanity-check
2020-08-02 17:13:17 +03:00
Azat Khuzhin
e37c42c56c
Fix logging in MergeTreeDataSelectExecutor for multiple threads (attach to thread group)
2020-08-02 13:40:01 +03:00
Azat Khuzhin
101217470e
Use "Not using primary index on part" over "Not using index on part" (add "primary")
2020-08-02 13:40:01 +03:00
Vitaly Baranov
18b21511a9
Merge pull request #13013 from vitlibar/implement-custom-settings
...
Implement custom settings
2020-08-02 05:01:14 +03:00
Alexey Milovidov
d3c0635fc0
Merge branch 'master' into stack-frams-size-limit
2020-08-02 04:35:09 +03:00
Alexey Milovidov
8fe201f1b9
Merge branch 'master' into merge-tree-settings-sanity-check
2020-08-01 19:20:11 +03:00
Nikolai Kochetov
c786f05388
Merge pull request #13189 from ClickHouse/refactor-pipes-2
...
Remove TreeExecutorBIS
2020-08-01 11:56:50 +03:00
Vitaly Baranov
56665a15f7
Rework and rename the template class SettingsCollection => BaseSettings.
2020-07-31 20:54:18 +03:00
Vitaly Baranov
90602b869a
Make SettingsChanges a class.
2020-07-31 19:11:27 +03:00
Vitaly Baranov
4ad99f1472
Make type names of settings shorter: SettingUInt64 => UInt64.
2020-07-31 19:11:27 +03:00
Nikolai Kochetov
39530f837e
Remove TreeExecutorBlockInputStream.
2020-07-31 16:23:19 +03:00
alesapin
45894fcf99
Better comments
2020-07-31 15:22:32 +03:00
alexey-milovidov
9c9ef0db9b
Merge branch 'master' into database_atomic_improvements
2020-07-31 14:56:49 +03:00
alesapin
14928874a1
Fix mutations status check
2020-07-31 14:37:16 +03:00
Alexey Milovidov
2c40539df6
Check ALTERs; update test
2020-07-31 00:42:55 +03:00
Alexey Milovidov
c491b2c153
Lower stack frame size
2020-07-30 23:58:08 +03:00
Alexey Milovidov
b9f49d31df
Sanity checks for MergeTreeSettings
2020-07-30 22:08:13 +03:00
alexey-milovidov
fc35ce69fa
Merge pull request #12666 from ClickHouse/forward_alter_error_to_client
...
Forward exceptions to client if exception happened during ALTER
2020-07-30 21:36:06 +03:00
Nicolae Vartolomei
85174d64a0
Use absolute paths to backup locations (shadow)
...
Tests are frankenstain. Had to hide absolute path `/tmp/clickhouse.test..IP69I/data/shadow/test_01417_single_part_7/data/default/table_for_freeze/7_8_8_0` to make them deterministic and independend of the method they are run (local vs remote target)
2020-07-30 18:03:26 +01:00
alexey-milovidov
4c3241808d
Merge pull request #12728 from ClickHouse/prevent-inconsistent-merge-tree-settings
...
Safe logic for incorrect MergeTreeSettings
2020-07-30 17:55:39 +03:00
alexey-milovidov
e6265bf299
Merge pull request #13100 from ClickHouse/keep-less-amount-of-logs-in-zookeeper
...
Keep less amount of logs in ZooKeeper
2020-07-30 15:41:55 +03:00
alexey-milovidov
a22bdb1989
Merge pull request #13081 from CurtizJ/fix-index_analysis
...
Fix wrong index analysis with functions
2020-07-30 15:30:55 +03:00
Alexey Milovidov
6517ddcbd4
Keep less amount of logs in ZooKeeper
2020-07-30 05:47:05 +03:00
Alexander Kuzmenkov
881220eec2
Merge pull request #12755 from bobrik/ivan/obvious-skip-overall
...
Show marks before applying skipping indices
2020-07-29 20:59:08 +03:00
Anton Popov
4c266d1e5d
fix wrong index analysis with functions
2020-07-29 19:09:38 +03:00
alesapin
7a806eb90f
Merge pull request #13017 from ClickHouse/freeze_partition_with_name
...
Alter table freeze with verbose output
2020-07-29 18:42:48 +03:00
Alexander Kuzmenkov
1b9269ae0c
fixup
2020-07-28 19:58:19 +03:00
Alexander Kuzmenkov
297cf65f1f
Merge remote-tracking branch 'origin/master' into HEAD
2020-07-28 19:56:35 +03:00
alesapin
c75830e878
Alter table freeze with verbose output
2020-07-28 18:10:36 +03:00
alesapin
585d9247ec
Merge pull request #12959 from ClickHouse/test_for_backup
...
Check backup restore in integration test.
2020-07-28 17:59:52 +03:00
Alexander Kazakov
2bde393499
Merging #12548 - Correction to merge_with_ttl_timeout logic by @excitoon ( #12982 )
...
* Fixed `merge_with_ttl_timeout` logic.
* Separate TTL-merge timers for each partition
Co-authored-by: Vladimir Chebotarev <vladimir.chebotarev@gmail.com>
2020-07-28 17:38:34 +03:00
Alexander Kuzmenkov
ba7c33f806
Merge pull request #12754 from bobrik/ivan/obvious-skip
...
Show total granules examined by skipping indices
2020-07-28 17:14:25 +03:00
alexey-milovidov
7a22717180
Merge pull request #12760 from ClickHouse/sticky_mutations_bug
...
Sticking mutations bug
2020-07-28 14:42:54 +03:00
alesapin
1bece3de0a
Remove strange code and test
2020-07-28 12:36:08 +03:00
Alexey Milovidov
798ef1b9f9
Adjustments; added test result
2020-07-27 23:18:23 +03:00
Alexey Milovidov
ec84ed30cb
Merge branch 'master' into prevent-inconsistent-merge-tree-settings
2020-07-27 21:39:25 +03:00
Nikolai Kochetov
abdd160db1
Merge pull request #12589 from bobrik/ivan/parallel-ranges
...
Parallelize PK range and skipping index stages
2020-07-27 19:23:00 +03:00
alexey-milovidov
7dac65ce0f
Merge pull request #12659 from zhang2014/fix/ISSUES-10572
...
ISSUES-10572 fix bloom filter with const column
2020-07-27 14:38:56 +03:00
alesapin
1707f84a44
Less strict check and rare rename bug
2020-07-27 12:42:37 +03:00
alesapin
e0bf5913e7
Fix sticking mutations
2020-07-26 17:21:57 +03:00
Alexander Tokmakov
6bfe88e83f
Merge branch 'master' into database_atomic_improvements
2020-07-26 13:40:05 +03:00
alesapin
c66dc23a47
Fix error style
2020-07-25 23:56:29 +03:00
alesapin
eae7dc852e
Fix build
2020-07-25 23:54:33 +03:00
alesapin
393a9195ca
Better exception message
2020-07-25 17:45:46 +03:00
alesapin
78d357f0d2
Add a test for sticking mutations bug
2020-07-25 17:42:20 +03:00
Ivan Babrou
e835ec0b56
Show marks before applying skipping indices
...
This change makes skipping index efficiency more obvious, changing this:
```
Selected 30 parts by date, 30 parts by key, 592 marks to read from 541 ranges
```
Into this:
```
Selected 30 parts by date, 30 parts by key, 48324 marks by primary key, 592 marks to read from 541 ranges
```
2020-07-24 15:45:38 -07:00
Ivan Babrou
67d4529783
Show total granules examined by skipping indices
...
This change makes skipping index efficiency more obvious, changing this:
```
Index `idx_duration` has dropped 59 granules.
```
Into this:
```
Index `idx_duration` has dropped 59 / 61 granules.
```
2020-07-24 14:50:32 -07:00
Anton Popov
d3fbeb296e
write compressed blocks for every column in granule
2020-07-23 19:56:12 +03:00
Anton Popov
61018c275d
Merge remote-tracking branch 'upstream/master' into polymorphic-parts-4
2020-07-23 18:38:29 +03:00
Anton Popov
1d0493cfcc
better code in ReaderCompact and perf test added
2020-07-23 18:37:44 +03:00
Nikolai Kochetov
dad9d369a1
Merge branch 'master' into bobrik-parallel-randes
2020-07-23 16:21:32 +03:00
Alexey Milovidov
6b4be2b9fe
Safe logic for incorrect MergeTreeSettings #10897
2020-07-23 15:31:09 +03:00
Azat Khuzhin
c97fb4b0e5
Remove unused bits in MergeTreeRangeReader ( #12671 )
...
Refs: #7769
Cc: @amosbird
2020-07-23 11:20:09 +03:00
alesapin
978f286625
Trying to fix FreeBSD build
2020-07-23 11:00:18 +03:00
alesapin
6fdcb59df7
Merge branch 'master' into forward_alter_error_to_client
2020-07-23 10:48:58 +03:00
Alexander Tokmakov
237accd154
Merge branch 'master' into database_atomic_improvements
2020-07-23 06:33:20 +03:00
alesapin
355150afce
Fix deadlock and better code
2020-07-22 22:29:54 +03:00
Artem Zuikov
2afd123eda
Refactoring: extract TreeOptimizer from SyntaxAnalyzer ( #12645 )
2020-07-22 20:13:05 +03:00
alesapin
8c8bdd5070
Fix tests and work with multiple mutations
2020-07-22 18:19:54 +03:00
Nikolai Kochetov
76d784be9c
Merge pull request #12613 from ClickHouse/remove-sort-description-from-streams
...
Remove sort description from streams
2020-07-22 17:35:05 +03:00
alesapin
de846e5e3c
Forward exception to client during alter
2020-07-22 15:36:19 +03:00
Nikolai Kochetov
b27066389a
Do not create ThreadPool for single thread.
2020-07-22 14:51:35 +03:00
zhang2014
580889f1e4
ISSUES-10572 fix bloom filter with const column
2020-07-22 17:42:12 +08:00
Alexander Tokmakov
9bcaaea3e0
Merge branch 'master' into database_atomic_improvements
2020-07-22 05:24:48 +03:00
alesapin
be106b662e
Comment and style
2020-07-21 18:19:41 +03:00
Nikolai Kochetov
486a4932c3
Fix tests.
2020-07-21 17:08:18 +03:00
alesapin
daaa5cc382
Add setting allow_non_metadata_alters
2020-07-21 17:05:30 +03:00
Nikolai Kochetov
755f15def3
Make MergeTreeSetIndex::checkInRange const.
2020-07-21 14:22:45 +03:00
alesapin
379a5a750a
Fix error message about granularity
2020-07-21 14:10:23 +03:00
Nikolai Kochetov
12c5e376c6
Remove mutable from RPNElement.
2020-07-21 14:02:58 +03:00
Nikolai Kochetov
cab9146ff3
Remove SortDescription from IBlockInputStream.
2020-07-21 11:05:52 +03:00
Nikolai Kochetov
d6583698a9
Merge branch 'master' into bobrik-parallel-randes
2020-07-20 19:05:28 +03:00
Nikolai Kochetov
0cc55781d8
Try fix tests.
2020-07-20 18:09:00 +03:00
Alexander Kuzmenkov
40ee6eba19
Merge pull request #12545 from ClickHouse/fix_load_part_without_checksums
...
Fix loading parts without checksums
2020-07-20 14:40:53 +03:00
Ivan Babrou
72622a9b00
Parallelize PK range and skipping index stages
...
This runs PK lookup and skipping index stages on parts
in parallel, as described in #11564 .
While #12277 sped up PK lookups, skipping index stage
may still be a bottleneck in a select query. Here we
parallelize both stages between parts.
On a query that uses a bloom filter skipping index to pick
2,688 rows out of 8,273,114,994 on a two day time span,
this change reduces latency from 10.5s to 1.5s.
2020-07-19 21:49:41 -07:00
Alexander Tokmakov
c27de9b424
fixes
2020-07-17 16:11:44 +03:00
alesapin
ac802a8e96
Merge pull request #12543 from ClickHouse/fix_marks_type_on_mutation
...
Fix incorrect adaptive granularity by default
2020-07-17 11:09:54 +03:00
Alexander Tokmakov
a612080df1
Merge branch 'master' into database_atomic_improvements
2020-07-17 00:43:59 +03:00
Anton Popov
97e8a88b30
Merge pull request #12277 from bobrik/ivan/exact-range-speedup
...
WIP: Optimize PK lookup for queries that match exact PK range
2020-07-16 19:17:50 +03:00
alesapin
b3ec3b1ed1
Fix loading parts without checksums
2020-07-16 13:54:49 +03:00
alesapin
42983c5416
Fix incorrect adaptive granularity by default
2020-07-16 12:10:15 +03:00
Anton Popov
c6423b2b28
lower read buffer size, while reading from compact parts
2020-07-15 14:53:23 +03:00
Alexander Tokmakov
09abe71b60
Merge branch 'master' into database_atomic_improvements
2020-07-15 13:07:08 +03:00
alesapin
614540eddf
Merge pull request #12382 from ClickHouse/clear-all-columns
...
Better errors for CLEAR/DROP columns (possibly in partitions)
2020-07-15 12:52:06 +03:00
alesapin
9e41fbca55
Remove check for drop detached partition
2020-07-14 16:56:30 +03:00
alesapin
014bb070ec
Fix tests
2020-07-14 11:19:39 +03:00
alesapin
1f576ee039
Some intermediate solution
2020-07-13 20:27:52 +03:00
alesapin
4a53264a86
Remove redundant and duplicated code
2020-07-13 19:19:08 +03:00
Alexander Tokmakov
fdb02edc7c
Merge branch 'master' into database_atomic_improvements
2020-07-13 17:28:48 +03:00
Alexander Tokmakov
04c3e7cab6
add implicit uuid macro
2020-07-13 16:18:38 +03:00
Anton Popov
a9530d2883
in-memory parts: fix reading from nested
2020-07-13 12:10:55 +03:00
alexey-milovidov
ae7eff98ed
Merge pull request #12433 from amosbird/np
...
Allow nullable key in MergeTree
2020-07-13 04:36:00 +03:00
Amos Bird
cac5a89169
Allow nullable key in MergeTree
2020-07-12 22:21:51 +08:00
Ivan Babrou
8784994d65
Allow conditions outside of PK with exact range
...
Conditions that are outside of PK are marked as `unknown` in `KeyCondition`,
so it's safe to allow them, as long as they are always combined by `AND`.
2020-07-11 18:59:26 -07:00
Ivan Babrou
d9d8d0242e
Optimize PK lookup for queries that match exact PK range
...
Existing code that looks up marks that match the query has a pathological
case, when most of the part does in fact match the query.
The code works by recursively splitting a part into ranges and then discarding
the ranges that definitely do not match the query, based on primary key.
The problem is that it requires visiting every mark that matches the query,
making the complexity of this sort of look up O(n).
For queries that match exact range on the primary key, we can find
both left and right parts of the range with O(log 2) complexity.
This change implements exactly that.
To engage this optimization, the query must:
* Have a prefix list of the primary key.
* Have only range or single set element constraints for columns.
* Have only AND as a boolean operator.
Consider a table with `(service, timestamp)` as the primary key.
The following conditions will be optimized:
* `service = 'foo'`
* `service = 'foo' and timestamp >= now() - 3600`
* `service in ('foo')`
* `service in ('foo') and timestamp >= now() - 3600 and timestamp <= now`
The following will fall back to previous lookup algorithm:
* `timestamp >= now() - 3600`
* `service in ('foo', 'bar') and timestamp >= now() - 3600`
* `service = 'foo'`
Note that the optimization won't engage when PK has a range expression
followed by a point expression, since in that case the range is not continuous.
Trace query logging provides the following messages types of messages,
each representing a different kind of PK usage for a part:
```
Used optimized inclusion search over index for part 20200711_5710108_5710108_0 with 9 steps
Used generic exclusion search over index for part 20200711_5710118_5710228_5 with 1495 steps
Not using index on part 20200710_5710473_5710473_0
```
Number of steps translates to computational complexity.
Here's a comparison for before and after for a query over 24h of data:
```
Read 4562944 rows, 148.05 MiB in 45.19249672 sec., 100966 rows/sec., 3.28 MiB/sec.
Read 4183040 rows, 135.78 MiB in 0.196279627 sec., 21311636 rows/sec., 691.75 MiB/sec.
```
This is especially useful for queries that read data in order
and terminate early to return "last X things" matching a query.
See #11564 for more thoughts on this.
2020-07-11 12:26:54 -07:00
Anton Popov
fbec940e0f
fix reading and check query for compact parts with different codecs
2020-07-11 02:33:36 +03:00
alexey-milovidov
e22547c29d
Merge pull request #12388 from ClickHouse/bloom-filter-arg-check
...
Check arguments of bloom filter index
2020-07-10 20:54:16 +03:00
alexey-milovidov
caef1d8e24
Update MergeTreeIndexFullText.cpp
2020-07-10 20:53:58 +03:00
Anton Popov
24f627e52c
fix reading from compact parts with different codecs
2020-07-10 18:57:10 +03:00
Alexey Milovidov
47eaffbe63
Additional checks
2020-07-10 11:21:40 +03:00
Alexey Milovidov
4b86f36d37
Check arguments of bloom filter index
2020-07-10 11:13:21 +03:00
Alexey Milovidov
276b3a0215
Avoid exception when negative or floating point constant is used in WHERE condition for indexed tables #11905
2020-07-10 09:30:49 +03:00
alexey-milovidov
c16d8e094b
Merge pull request #12308 from ClickHouse/fix-codec-bad-exception-code
...
Fix wrong exception code in codecs Delta, DoubleDelta #12110
2020-07-10 08:40:46 +03:00
Anton Popov
80a62977f1
fix hashing in DataPartWriterCompact
2020-07-09 21:26:54 +03:00
Anton Popov
9f72baad1f
Merge remote-tracking branch 'upstream/master' into HEAD
2020-07-09 20:51:36 +03:00
Alexey Milovidov
6c458a6287
Fix race condition in ReplicatedMergeTreeQueue
2020-07-09 07:43:28 +03:00
Alexey Milovidov
ea970fd57c
Remove bad ugliness
2020-07-09 04:00:16 +03:00
alesapin
84f8bf1e14
Merge pull request #10697 from CurtizJ/polymorphic-parts
...
Polymorphic parts (in-memory format)
2020-07-08 13:25:45 +03:00
Anton Popov
416d9648c2
remove questionable functionality
2020-07-07 12:58:14 +03:00
Anton Popov
d6434f61dc
support codecs in compact parts
2020-07-07 03:15:02 +03:00
alesapin
3f90f65678
Merge remote-tracking branch 'origin' into CurtizJ-polymorphic-parts
2020-07-06 21:34:17 +03:00
Nikita Mikhaylov
2b539e6ace
fix style
2020-07-06 20:26:15 +03:00
Nikita Mikhaylov
d31ed58f01
done
2020-07-06 17:33:31 +03:00
myrrc
8c3417fbf7
ILIKE operator ( #12125 )
...
* Integrated CachingAllocator into MarkCache
* fixed build errors
* reset func hotfix
* upd: Fixing build
* updated submodules links
* fix 2
* updating grabber allocator proto
* updating lost work
* updating CMake to use concepts
* some other changes to get it building (integration into MarkCache)
* further integration into caches
* updated Async metrics, fixed some build errors
* and some other errors revealing
* added perfect forwarding to some functions
* fix: forward template
* fix: constexpr modifier
* fix: FakePODAllocator missing member func
* updated PODArray constructor taking alloc params
* fix: PODArray overload with n restored
* fix: FakePODAlloc duplicating alloc() func
* added constexpr variable for alloc_tag_t
* split cache values by allocators, provided updates
* fix: memcpy
* fix: constexpr modifier
* fix: noexcept modifier
* fix: alloc_tag_t for PODArray constructor
* fix: PODArray copy ctor with different alloc
* fix: resize() signature
* updating to lastest working master
* syncing with 273267
* first draft version
* fix: update Searcher to case-insensitive
* added ILIKE test
* fixed style errors, updated test, split like and ilike, added notILike
* replaced inconsistent comments
* fixed show tables ilike
* updated missing test cases
* regenerated ya.make
* Update 01355_ilike.sql
Co-authored-by: myrrc <me-clickhouse@myrrec.space>
Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
2020-07-05 18:57:59 +03:00
alexey-milovidov
e05283d5d3
Merge pull request #12060 from ClickHouse/fix-12053
...
Check wrong type for filter.
2020-07-04 23:46:34 +03:00
Anton Popov
73676f5022
Improve performace of reading in order of sorting key. ( #11696 )
...
* simplify reading in order of sorting key
* add perf test for reading many parts
* Revert "simplify reading in order of sorting key"
This reverts commit 7267d7c46e
2020-07-04 15:48:51 +03:00
alesapin
8dc204350f
Fix version column in replicated version collapsing merge tree ( #12121 )
2020-07-04 10:35:17 +03:00
Anton Popov
ca346ea13c
rename fsync-related settings
2020-07-03 02:41:37 +03:00
Anton Popov
4422df2e37
Merge remote-tracking branch 'upstream/master' into HEAD
2020-07-02 20:18:21 +03:00
Anton Popov
71059e4dff
Merge pull request #12062 from nvartolomei/nv/set-index-tuple-types
...
Try fix pk in tuple performance
2020-07-02 15:51:03 +03:00
Nikolai Kochetov
79cd33a567
Fix tests.
2020-07-02 10:44:47 +03:00
Nicolae Vartolomei
3854ce6d84
Rewrite Set lookup to make it more readable
2020-07-01 15:05:54 +01:00
Anton Popov
d45697a1b5
in-memory-parts: lazy creation of WAL
2020-06-30 21:47:12 +03:00
Nikolai Kochetov
3585700506
Fix tests.
2020-06-30 19:41:43 +03:00
Nicolae Vartolomei
8f1845185e
Try fix pk in tuple performance
...
Possible approach for fixing #10574
The problem is that prepared sets are built correctly, it is a hash map of key -> set
where key is a hash of AST and list of data types (when we a list of
tuples of literals).
However, when the key is built from the index to try and find if there
exists a prepared set that would match it looks for data types of the
primary key (see how data_types is populated) because the primary key
has only one field (v in my example) it can not find the prepared set.
The patch looks for any prepared indexes where data types match for the
subset of fields found in primary key, we are not interested in other
fields anyway for the purpose of primary key pruning.
2020-06-30 16:33:38 +01:00
Nikolai Kochetov
66da0733ab
Check type of filter for prewhere.
2020-06-30 17:20:27 +03:00
Vladimir Chebotarev
bb8da71eff
Moves task shall be started if new storage policy needs them.
2020-06-30 15:36:44 +03:00
Anton Popov
53e955c6dd
several fixes
2020-06-29 23:36:18 +03:00
Anton Popov
a43cb93be5
remove questionable functionality
2020-06-29 18:46:50 +03:00
Alexey Milovidov
cc597bb30b
Implement for ReplicatedMergeTree
2020-06-28 22:39:31 +03:00
Alexey Milovidov
1f9c27de34
Fix clang-tidy
2020-06-28 22:39:31 +03:00
Alexey Milovidov
b6e31f0a5e
Only allow Enum extension within the same width
2020-06-28 22:39:31 +03:00
Alexey Milovidov
e4d8cc2b44
Slightly more safe
2020-06-28 22:39:31 +03:00
Alexey Milovidov
97ad23b905
Allow to ALTER partition key in some cases
2020-06-28 22:39:31 +03:00
Alexey Milovidov
678d127503
Whitespace
2020-06-28 22:39:31 +03:00
Alexey Milovidov
c4d5ca485e
Remove the usage of obsolete setting
2020-06-28 22:39:31 +03:00
Alexey Milovidov
1462a66d1e
Fix typos
2020-06-27 22:05:00 +03:00
Anton Popov
b2aa565a37
allow to turn on fsync on inserts, merges and fetches
2020-06-27 17:52:16 +03:00
Alexey Milovidov
46caa211d5
Better diagnostics of "Replica {} appears to be already active" message
2020-06-27 16:55:00 +03:00
alesapin
9710a67704
Fix style check
2020-06-26 15:27:12 +03:00
alesapin
6caf2e7a3f
Fix protocol check
2020-06-26 14:38:37 +03:00
alesapin
6f1824f0ea
Correct merge with master
2020-06-26 14:30:23 +03:00
alesapin
e9c47dc89c
Merge branch 'master' into CurtizJ-polymorphic-parts
2020-06-26 14:27:19 +03:00
Alexey Milovidov
8eed47857b
Fix estimation of the number of marks for various thresholds
2020-06-25 23:20:22 +03:00
Alexey Milovidov
8872417d00
Respect direct_io/mmap settings while reading secondary indices
2020-06-25 22:31:54 +03:00
Alexey Milovidov
5608f15749
Revive mmap IO
2020-06-25 22:15:41 +03:00
Anton Popov
f8f67a788e
allow to turn on fsync on inserts and merges
2020-06-25 19:55:45 +03:00
alesapin
63f8f89ca0
bump CI
2020-06-24 19:06:01 +03:00
alesapin
08f5ed6da3
Fix unitialized memory
2020-06-24 17:12:44 +03:00
Nikolai Kochetov
f7f16fdbf7
Merge pull request #11895 from ClickHouse/fix-nullable-prewhere-type-3
...
Fix nullable prewhere type 3
2020-06-24 14:32:08 +03:00
Anton Popov
8c3f496826
add test with alters and in-memory parts
2020-06-24 01:16:29 +03:00
Nikolai Kochetov
658a2d0dc0
Fix anotehr one nullable prewhere column.
2020-06-23 21:52:32 +03:00
alesapin
4185fe9975
Merge branch 'master' into atomic_metadata5
2020-06-23 11:13:05 +03:00
alesapin
292016e045
Add missed lock
2020-06-23 11:04:43 +03:00
Anton Popov
78d28be8cf
check in-memory parts, comments and style-fixes
2020-06-22 21:57:50 +03:00
Nikolai Kochetov
ae21aca3a3
Fix header for nullable prewhere column.
2020-06-22 19:01:59 +03:00
alesapin
c76b4b3481
Clang-tidy fixes
2020-06-22 18:51:11 +03:00
alesapin
b1e8976df4
Merge with master
2020-06-22 12:04:27 +03:00
alesapin
b9e74f4e82
Merge branch 'master' into atomic_metadata5
2020-06-22 12:03:53 +03:00
Alexey Milovidov
0e6fb617aa
Merge branch 'master' into manual-write-duplicate-parts-to-replicas
2020-06-22 01:54:03 +03:00
Alexey Milovidov
ad672f7ce2
Whitespace
2020-06-20 10:26:33 +03:00
Alexey Milovidov
b8769e1ada
Fix style
2020-06-20 10:24:51 +03:00
alesapin
4c0879ae30
Better logging in storages
2020-06-19 20:17:13 +03:00
alesapin
a2e7e9f232
Remove unused constructor and rename method
2020-06-19 18:21:48 +03:00
alesapin
3c47faa9da
Move partition key initialization into registerMergeTree
2020-06-19 18:14:08 +03:00
Alexey Milovidov
5211a42c04
Remove leader election, step 3
2020-06-19 17:18:58 +03:00
alesapin
c666763cc0
Remove unused method better
2020-06-18 20:09:06 +03:00
alesapin
e888dafdc2
Remove unused method
2020-06-18 20:03:42 +03:00
alesapin
d79982f497
Better locks in Storages
2020-06-18 19:10:47 +03:00
alesapin
d4c49816ab
Really atomic metadata
2020-06-18 14:02:31 +03:00
alesapin
ebb36bec8a
Merge branch 'master' into atomic_metadata5
2020-06-18 11:57:16 +03:00
Azat Khuzhin
caa88777be
Parse metadata stored in zookeeper before checking for equality
...
After #11325 trailing whitespaces has been removed for data skipping
indicies, and it may be different, if you have multiple skip indices,
and in this case new server will not load such tables, because metadata
will be different.
Fix this by re-parse metadata in zookeeper.
2020-06-17 21:03:22 +03:00
alesapin
dffdece350
getColumns in StorageInMemoryMetadta (only compilable)
2020-06-17 19:39:58 +03:00
alesapin
33c27de54d
Check methods in metadata
2020-06-17 17:32:25 +03:00
alesapin
eaaef83742
Settings changes in StorageInMemoryMetadata
2020-06-17 16:46:01 +03:00
alesapin
ed8f3b2fc4
TTL in storage in memory metadata
2020-06-17 16:39:26 +03:00
alesapin
1afdebeebd
Primary key in storage metadata
2020-06-17 15:39:20 +03:00
alesapin
1da393b218
Sampling key in StorageInMemoryMetadata
2020-06-17 15:07:09 +03:00
alesapin
eca6e9087e
Fix race condition
2020-06-17 14:52:19 +03:00
alesapin
ba04d02f1e
Compilable sorting key in metadata
2020-06-17 14:05:11 +03:00
alesapin
ab61abccc1
Partition key in StorageInMemoryMetadata
2020-06-17 13:34:23 +03:00
alesapin
62f2c17a66
Secondary indices in StorageInMemoryMetadata
2020-06-17 12:38:47 +03:00
alesapin
2fac3290f0
Merge branch 'master' into atomic_metadata5
2020-06-17 11:14:03 +03:00
alexey-milovidov
e07d02015c
Merge pull request #11661 from ClickHouse/return-not-nullable-from-count-distinct-2
...
Return non-Nullable results from COUNT(DISTINCT), more complete.
2020-06-17 09:25:45 +03:00
alexey-milovidov
598fd75caf
Merge pull request #11639 from ClickHouse/remove-leader-election-2
...
Remove leader election, step 2: allow multiple leaders
2020-06-17 09:25:23 +03:00
Alexey Milovidov
0a6054eece
Merge branch 'master' into return-not-nullable-from-count-distinct-2
2020-06-16 22:31:33 +03:00
alesapin
d214a504d7
Merge pull request #11681 from ClickHouse/fix_mutations_premature_finish
...
Fix mutations premature "finish"
2020-06-16 21:26:14 +03:00
alesapin
ccc2bda666
getConstraints() in StorageInMemoryMetadata (suspicious commit, but pretend to work)
2020-06-16 19:55:04 +03:00
alesapin
1ddeb3d149
Buildable getSampleBlock in StorageInMemoryMetadata
2020-06-16 18:51:29 +03:00
alesapin
71f99a274d
Compileable getSampleBlockWithColumns in StorageInMemoryMetadata
2020-06-16 17:25:08 +03:00
alesapin
0bcd22008a
Get column dependencies in StorageInMemoryMetadata
2020-06-16 15:19:21 +03:00
alesapin
eae901f58d
Merge branch 'master' into atomic_metadata5
2020-06-16 14:44:45 +03:00
alesapin
0a33ccb77b
Fix remove method
2020-06-16 14:12:52 +03:00
alesapin
d1052f2dfe
Better logic
2020-06-16 13:50:47 +03:00
alesapin
d5b76b2fe1
Fix parts count in mutations
2020-06-16 13:34:59 +03:00
alesapin
0cb14c836c
Merge pull request #11691 from ClickHouse/fix-race-condition-system-sync-replicas
...
Fix race condition in SYSTEM SYNC REPLICA
2020-06-16 13:29:18 +03:00
alesapin
362aaf1385
Merge pull request #11516 from ClickHouse/consistent_metadata4
...
Storage metadata as a single struct.
2020-06-16 13:28:22 +03:00
alesapin
605c904fc6
Add method for compatibility
2020-06-16 11:39:12 +03:00
Anton Popov
b19d48a11c
Merge remote-tracking branch 'upstream/master' into HEAD
2020-06-16 06:37:55 +03:00
Alexey Milovidov
125b007360
Merge branch 'master' into remove-leader-election-2
2020-06-16 05:56:29 +03:00
Alexey Milovidov
22a92faab6
Avoid connection to replica when fetches are cancelled
2020-06-16 05:14:53 +03:00
Alexey Milovidov
4360e326ed
Fix clang-static-analyzer
2020-06-16 04:17:02 +03:00
Alexey Milovidov
6da1a0f153
Fix style
2020-06-16 04:13:45 +03:00
Alexey Milovidov
53d985909e
Fix race condition in SYSTEM SYNC REPLICA
2020-06-16 01:54:19 +03:00
Anton Popov
a3ac224ae4
in-memory parts: sync insert to replicated
2020-06-15 22:57:12 +03:00
Alexey Milovidov
d10109dc38
Remove excessive statement #11131
2020-06-15 22:18:33 +03:00
alesapin
36ba0192df
Metadata in read and write methods of IStorage
2020-06-15 22:08:58 +03:00
Alexey Milovidov
e44e1ad0d4
Don't ignore duplicate parts written to replicas
2020-06-15 21:57:38 +03:00
alesapin
b47a7327fd
All set methods in metadata
2020-06-15 21:08:05 +03:00
alesapin
33a74a3ea0
TTL methods in StorageInMemoryMetadata
2020-06-15 20:50:53 +03:00
alesapin
e667eb57b2
Working set*Keys methods
2020-06-15 20:24:49 +03:00
alesapin
5fc41c7ecc
Move set*Key methods to StorageInMemoryMetadata
2020-06-15 20:17:06 +03:00
alesapin
aa30649ce5
Working setColumns, setConstraints, setIndices
2020-06-15 20:10:14 +03:00
alesapin
af2fe2ba55
Compilable setColumns, setConstraints, setIndices
2020-06-15 19:55:33 +03:00
alesapin
8945d0073a
Fix misunderstanding bug in mutations finalization
2020-06-15 16:37:40 +03:00
alesapin
bc9a3c0458
Merge branch 'master' into consistent_metadata4
2020-06-15 11:20:30 +03:00
alexey-milovidov
fd6115cccf
Merge pull request #11648 from ClickHouse/simplify-code-functions-comparison-3
...
Allow comparison with String in index analysis; simplify code.
2020-06-15 05:47:03 +03:00
Alexey Milovidov
5866401f60
Less noise in cleanup thread
2020-06-15 05:12:06 +03:00
Alexey Milovidov
689b6901f8
Fix typo
2020-06-15 04:04:42 +03:00
Alexey Milovidov
1c438a133e
Leader election both backward and forward compatible
2020-06-15 03:59:12 +03:00
Alexey Milovidov
b596831342
Merge branch 'master' into remove-leader-election-2
2020-06-15 03:16:41 +03:00
Alexey Milovidov
8dac30ae95
Split file for better build times
2020-06-14 21:42:10 +03:00
Alexey Milovidov
f6c52fe1c2
Allow comparison with String in index analysis; simplify code #11630
2020-06-14 21:31:42 +03:00
Alexey Milovidov
394fb64a9c
Better way of implementation
2020-06-14 20:42:11 +03:00
Alexey Milovidov
2cd82a25f5
Remove trivial count query optimization if row-level security is set #11352
2020-06-13 23:59:20 +03:00
alexey-milovidov
f53da4d36b
Merge pull request #11608 from ClickHouse/fix-nullable-prewhere-type
...
Fix nullable prewhere type
2020-06-13 20:29:12 +03:00
alesapin
18b58e8483
Revert too strict lock
2020-06-13 12:12:45 +03:00
alesapin
5d3de747cd
Merge remote-tracking branch 'origin/master' into consistent_metadata4
2020-06-13 11:50:27 +03:00
Alexey Milovidov
6f0db5ef10
Fix error
2020-06-13 04:35:49 +03:00
Alexey Milovidov
6ff671b092
Improvement
2020-06-13 04:35:49 +03:00
Alexey Milovidov
ab00e34305
Miscellaneous
2020-06-13 04:35:49 +03:00
Alexey Milovidov
85c0706901
Step 2: allow multiple leaders
2020-06-13 04:35:49 +03:00
Alexey Milovidov
b8b55a5b99
More LeaderElection to Storage/MergeTree
2020-06-13 04:35:49 +03:00
Alexey Milovidov
52ac009754
Update versions on merge and mutation
2020-06-13 04:34:05 +03:00
Alexey Milovidov
6869c122a8
Step 1 towards removing leader election: check and modify version of the "log" node
2020-06-13 04:34:05 +03:00
Alexey Milovidov
72257061d5
Avoid errors due to implicit int<->bool conversions when using ZK API
2020-06-12 18:09:12 +03:00
Nikolai Kochetov
83155e139c
Try fix tests.
2020-06-12 17:59:14 +03:00
alesapin
0edf5ff7a2
Fix race condition
2020-06-12 17:32:47 +03:00
alesapin
465c4b65b7
Slightly better interfaces and comments
2020-06-12 12:37:52 +03:00
Nikolai Kochetov
7b8c1657a5
Fix nullable prewhere column type.
2020-06-11 19:34:36 +03:00
alesapin
a39d010f94
Merge branch 'master' into consistent_metadata4
2020-06-11 18:47:41 +03:00
Alexey Milovidov
b72a095e99
Fix race conditions in CREATE/DROP of different replicas of ReplicatedMergeTree
2020-06-11 06:24:52 +03:00
alesapin
8be957ecb5
Better checks around metadata
2020-06-10 14:16:31 +03:00
alesapin
ec933d9d03
Better naming
2020-06-10 12:09:51 +03:00
alesapin
75977bd939
Gcc fixed
2020-06-10 00:22:01 +03:00
alesapin
d2fcf5aea5
Fixes for gcc
2020-06-09 20:28:29 +03:00
alesapin
4c2e0561f5
Fix primary key update
2020-06-09 20:21:38 +03:00
alesapin
f7a397242f
Merge branch 'master' into consistent_metadata4
2020-06-09 18:59:59 +03:00
alesapin
de59629b38
Fix benign race condition during shutdown
2020-06-09 04:48:30 +03:00
alesapin
0d375e4522
Fix constraints
2020-06-08 21:49:54 +03:00
alesapin
7b63a88120
Remove redundant error code
2020-06-08 18:19:41 +03:00
alesapin
d625162fce
Better comment
2020-06-08 17:23:32 +03:00
alesapin
37a1d128c9
Better
2020-06-08 17:22:11 +03:00
alesapin
a30b72a128
Less debug
2020-06-08 17:21:12 +03:00
alesapin
2354939994
Remove log debug
2020-06-08 17:19:20 +03:00
alesapin
638cd50ce4
Fix several bugs
2020-06-08 17:18:38 +03:00
alesapin
2226f79f16
Fix some bugs
2020-06-08 15:57:25 +03:00
alesapin
ac67d0485c
Merge branch 'master' into consistent_metadata4
2020-06-08 15:06:51 +03:00
Alexey Milovidov
f2cfb7d132
Whitespaces
2020-06-08 00:05:36 +03:00
Ivan
657ce426d3
Fix build
...
Checked locally - no PR
2020-06-06 22:27:48 +03:00
BayoNet
e9b7919e12
DOCS-271: Updated the MergeTree() ORDER BY description ( #11433 )
...
* CLICKHOUSEDOCS-271: Updated the MergeTree() ORDER BY description.
* CLICKHOUSEDOCS-271: Fixes grammar.
* CLICKHOUSEDOCS-271: Updated by comments.
Co-authored-by: Sergei Shtykov <bayonet@yandex-team.ru>
2020-06-06 20:44:48 +03:00
Anton Popov
5c42408add
Merge pull request #9113 from dimarub2000/group_by_in_order_optimization
...
[WIP] Optimization of GROUP BY with respect to table sorting key.
2020-06-06 14:25:59 +03:00
Anton Popov
66e31d4311
in-memory parts: several fixes
2020-06-05 23:49:24 +03:00
alesapin
abaf47f0cd
Make metadata single structure
2020-06-05 20:29:40 +03:00
alesapin
5bc7f67e61
Better metadata for select query and renames
2020-06-05 14:54:54 +03:00
Nikita Mikhaylov
4e4218e10d
Merge pull request #11385 from ClickHouse/akz/bg-processing-pool-update1
...
Extended BackgroundProcessingPool::startTask() to control task execution concurrency
2020-06-04 16:19:29 +04:00
Alexander Kuzmenkov
30d7d41ebc
Merge pull request #11381 from ClickHouse/aku/fuzz-related
...
Fuzzing-related changes.
2020-06-04 15:06:11 +03:00
Anton Popov
4a3d3c6e54
Merge pull request #11419 from CurtizJ/polymorphic-parts-2
...
Return lost comments and default values
2020-06-04 13:32:38 +03:00
Anton Popov
b312ac9786
in-memory parts: fix columns sizes
2020-06-04 01:53:04 +03:00
Anton Popov
df3dfd5b81
fix clang-tidy build
2020-06-04 01:00:02 +03:00
Anton Popov
1980d58ba6
add lost comments and default values
2020-06-04 00:30:45 +03:00
Anton Popov
11c4e9dde3
in-memory parts: fix 'check' query
2020-06-03 22:19:49 +03:00
Alexander Kuzmenkov
1ab3201454
Merge remote-tracking branch 'origin/master' into HEAD
2020-06-03 16:36:22 +03:00
Anton Popov
1ce09e1faa
Merge remote-tracking branch 'upstream/master' into polymorphic-parts
2020-06-03 16:27:54 +03:00
Anton Popov
caf1e4e8cc
in-memory-parts: fixes
2020-06-03 12:51:23 +03:00
Alexander Kuzmenkov
c7d9094a7a
Merge pull request #11259 from ClickHouse/consistent_metadata3
...
More consistent metadata for secondary indices
2020-06-03 12:23:21 +03:00
Alexander Kuzmenkov
f7bb6d57a4
build
2020-06-03 11:50:27 +03:00
Alexander Kazakov
f999021ee1
Submit to clang10's demands of syntactic purity
2020-06-03 11:12:56 +03:00
Alexander Kazakov
0cd4e6e5ba
Trying to fix clang10 build
2020-06-03 00:52:29 +03:00
Alexander Kazakov
9d025337f5
Fixed minor inconsistency with master branch
2020-06-02 21:56:36 +03:00
Alexander Kuzmenkov
a859afa174
macos build fix
2020-06-02 19:57:19 +03:00
Alexander Kazakov
8986983392
insignificant typo
2020-06-02 18:22:05 +03:00
Alexander Kazakov
9fbd72ab34
Renamed method
2020-06-02 18:18:36 +03:00
Alexander Kazakov
013dc2d7c2
Updated startTask() method + reworked some of the worker loop
2020-06-02 17:37:10 +03:00
alexey-milovidov
ff7262968a
Merge pull request #11354 from ClickHouse/get_rid_of_lock_during_send
...
Remove structure lock on parts send
2020-06-02 17:05:24 +03:00
Alexander Kuzmenkov
0a5cc96b67
Fuzzing-related changes.
...
* More LOGICAL_ERROR
* Proper cloning of some Asts
* Field::safeGet for user-supplied values
2020-06-02 16:56:10 +03:00
alesapin
42b6370b67
Remove structure lock on parts send
2020-06-02 11:10:21 +03:00
alesapin
ad6447398f
Fix indices calculation during merge
2020-06-01 22:58:30 +03:00
Anton Popov
e1970f6d28
in-memory parts: fix reading of nested
2020-06-01 20:53:18 +03:00
Nikolai Kochetov
53d12f5ab8
Try fix build.
2020-06-01 20:06:21 +03:00
Nikolai Kochetov
d25326e75c
Try fix build.
2020-06-01 20:03:57 +03:00
Nikolai Kochetov
325393d340
Try fix build.
2020-06-01 19:58:36 +03:00
alesapin
e0e77194f4
Remove redundant parts from MergeTree
2020-06-01 15:39:20 +03:00
alesapin
b49be4c7f2
Better names
2020-06-01 15:11:23 +03:00
alesapin
254d55f2ab
Better method name
2020-06-01 15:02:36 +03:00
alesapin
663e92b1c5
Rename to methods
2020-06-01 14:29:11 +03:00
alesapin
d9a817f578
Less virtual methods in IStorage
2020-06-01 14:17:18 +03:00
alesapin
69a9c856a6
Better variable name
2020-06-01 13:54:31 +03:00
alesapin
3847ea892d
Merge branch 'master' into consistent_metadata3
2020-06-01 13:17:59 +03:00
alexey-milovidov
9984e989c1
Merge pull request #11317 from ClickHouse/logger-fmt-2
...
Preparation for structured logging
2020-06-01 00:11:29 +03:00
Alexey Milovidov
07c14e12dd
Fix build after merge of #11162 with master
2020-05-31 19:53:30 +03:00
alexey-milovidov
8accde79b5
Merge pull request #11162 from azat/data-skip-index-merging-params-fix
...
[RFC] Fix data skipping indexes for columns with additional actions during merge
2020-05-31 19:10:53 +03:00
alexey-milovidov
9416e82b75
Update MergeTreeDataMergerMutator.cpp
2020-05-31 19:06:59 +03:00
Alexey Milovidov
25f941020b
Remove namespace pollution
2020-05-31 00:57:37 +03:00
Alexey Milovidov
5aff138956
Preparation for structured logging
2020-05-31 00:35:52 +03:00
Dmitry
4b0d32f026
Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization
2020-05-31 00:21:02 +03:00
alexey-milovidov
3eea042d16
Merge pull request #11243 from ClickHouse/remove-experimental-use-processors-flag-4
...
Remove some code.
2020-05-30 21:05:14 +03:00
alexey-milovidov
1dbaf80a5d
Merge pull request #11303 from ClickHouse/fix-index-analysis-empty-array
...
Fix issue #11286 ; add a test
2020-05-30 17:28:00 +03:00
Alexey Milovidov
b78c48310b
Whitespaces
2020-05-30 01:37:44 +03:00
Alexey Milovidov
8c8821475c
Fix issue #11286 ; add a test
2020-05-30 00:32:35 +03:00
Anton Popov
fa05641b06
in-memory parts: partition commands
2020-05-29 19:58:08 +03:00
Anton Popov
c919840722
in-memory parts: partition commands
2020-05-29 18:02:12 +03:00
alesapin
a429760641
Fix debug build
2020-05-29 17:10:09 +03:00
Anton Popov
4769ce7271
in-memory parts: several fixes
2020-05-29 02:59:13 +03:00
alesapin
66ccb30299
Fix style check
2020-05-28 20:17:05 +03:00
alesapin
860dc2c814
Merge with consistent_metadata2
2020-05-28 19:45:51 +03:00
alesapin
61aff798d3
Compileable code
2020-05-28 18:33:44 +03:00
alesapin
5f8b69547b
More readable code
2020-05-28 16:45:08 +03:00
alesapin
f5e5290c0a
Better naming
2020-05-28 16:09:03 +03:00
alesapin
26a9829e04
Indices description as vector
2020-05-28 15:47:17 +03:00
alesapin
52ca6b2051
I'm able to build it
2020-05-28 15:37:05 +03:00
Nikolai Kochetov
da0052858d
Fix build.
2020-05-28 13:57:04 +03:00
Anton Popov
d8342e5b12
in-memory parts: better restore and clear stale wal files
2020-05-27 23:05:55 +03:00
alesapin
380c60a33f
Intermediate stage (doesn't compile)
2020-05-27 21:38:34 +03:00
Nikolai Sorokin
57555dbabf
Fix after rebase
2020-05-27 14:00:17 +03:00
Anton Popov
f62adfd588
Fix Arcadia build
2020-05-27 12:53:15 +03:00
Nikolai Sorokin
141ed88751
Allow functions in group by keys; Add default aggregate function; Add more tests
2020-05-27 12:53:15 +03:00
Nikolai Sorokin
66496dc7e3
Fix trailing spaces
2020-05-27 12:53:15 +03:00
Nikolai Sorokin
eeaf608aa9
Refactor code; support prefix of PK as GROUP BY key
2020-05-27 12:53:15 +03:00
Nikolai Sorokin
61974e0047
TTL Expressions WHERE and GROUP BY draft
2020-05-27 12:53:15 +03:00
alesapin
c7cda399c7
Merge branch 'master' into consistent_metadata2
2020-05-27 12:51:56 +03:00
Dmitry
41d1cd1c9b
fix bad merge
2020-05-27 01:17:32 +03:00
Dmitry
38c585f867
Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization
2020-05-26 21:27:50 +03:00
alesapin
9d0c28d96c
Add missing methods
2020-05-26 17:14:08 +03:00
alesapin
8614487a5a
Fix primary key check
2020-05-26 16:46:19 +03:00
alesapin
6626ef08e8
Fix PVS studio errors
2020-05-26 15:14:50 +03:00
alesapin
5780240e88
Fix primry key check
2020-05-26 12:46:04 +03:00
alesapin
7bbb064b34
Fix metadata update
2020-05-26 11:58:32 +03:00
Alexey Milovidov
c858f4d89c
Clarifications
2020-05-26 02:47:21 +03:00
alesapin
355fbadb31
Fix style
2020-05-25 22:18:01 +03:00
alesapin
211d7e229f
Better using
2020-05-25 21:03:04 +03:00
alesapin
adc7439a4c
Comments and refactoring
2020-05-25 20:57:08 +03:00
alesapin
c3a6571036
Compilable code
2020-05-25 20:22:20 +03:00
alesapin
9b3cc9e525
Almost working ttl in IStorage
2020-05-25 20:07:14 +03:00
alesapin
6281dd6893
Merge pull request #11115 from ClickHouse/consistent_metadata
...
Refactoring in storage metadata (more consistent keys)
2020-05-25 13:09:56 +03:00
Azat Khuzhin
c36708caf9
Split data skipping indexes from the order by to evaluate order by before merge
2020-05-24 16:12:08 +03:00
Azat Khuzhin
71731b2e99
Fix data skipping indexes for columns with additional actions during merge
...
For example for AggregatingMergeTree if the column in the data skipping
index is under some aggregate function, then data skipping index will be
incorrect.
Fix this by calculating data skipping indexes after aggregation has been
done.
2020-05-24 16:12:08 +03:00
Alexey Milovidov
7e1813825b
Return old names of macros
2020-05-24 01:24:01 +03:00
Alexey Milovidov
af2a59e49b
Progress on task
2020-05-24 00:50:34 +03:00
Alexey Milovidov
c87fb91532
Progress on task
2020-05-24 00:41:35 +03:00
Alexey Milovidov
2d7d5a1547
Apply all transformations again
2020-05-24 00:16:27 +03:00
Alexey Milovidov
bab24879e9
Progress on task
2020-05-24 00:16:05 +03:00
Alexey Milovidov
e1695feb7f
Apply all transformations again
2020-05-23 23:40:32 +03:00
Alexey Milovidov
85f84550ba
Progress on task
2020-05-23 23:37:37 +03:00
Alexey Milovidov
f75587e562
Progress on task
2020-05-23 23:13:34 +03:00
Alexey Milovidov
18febd7b97
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | while read file; do perl -pne 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/${1}_FORMATTED(${2}, "${3}{}${5}{}${7}{}${9}{}${11}", ${4}, ${6}, ${8}, ${10});/' $file > ${file}.tmp; mv ${file}.tmp $file; done
2020-05-23 22:56:05 +03:00
Alexey Milovidov
cd6d3c7509
Progress on task
2020-05-23 22:55:06 +03:00
Alexey Milovidov
d5b7883451
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7{}\9", \4, \6, \8);/'
2020-05-23 22:41:08 +03:00
Alexey Milovidov
7e2fb9ad65
Apply all transformations again
2020-05-23 22:38:30 +03:00
Alexey Milovidov
eacff92d0e
Progress on task
2020-05-23 22:35:08 +03:00
Alexey Milovidov
29762240de
Remove duplicate whitespaces (preparation)
2020-05-23 22:31:54 +03:00
Alexey Milovidov
9d2a0d2dd7
Apply all transformations again
2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f
Remove duplicate whitespaces (preparation)
2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65
Make all LOG in single line (preparation)
2020-05-23 21:31:37 +03:00
Alexey Milovidov
3384f3b77e
Progress on task
2020-05-23 21:07:38 +03:00
Alexey Milovidov
fc412f2d9d
Progress on task
2020-05-23 20:59:03 +03:00
Alexey Milovidov
9d24908e53
Progress on task
2020-05-23 20:52:11 +03:00
Alexey Milovidov
3a09f9ca1c
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\W_]+\([^,]+, [^<+)"]+\);' | xargs sed -i -r -e 's/(LOG_[^\W_]+)\(([^,]+), ([^<+)"]+)\);/\1_FORMATTED(\2, \3);/'
2020-05-23 20:29:56 +03:00
Alexey Milovidov
241f8c5431
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+ \+ "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+) \+ "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/'
2020-05-23 20:10:21 +03:00
Alexey Milovidov
f69cbdcbfc
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+)\);/\1_FORMATTED(\2, "\3{}", \4);/'
2020-05-23 20:09:37 +03:00
Alexey Milovidov
f68d1ceb4f
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}\7{}", \4, \6, \8);/'
2020-05-23 20:02:09 +03:00
