Commit Graph

172 Commits

Author SHA1 Message Date
Nikita Mikhaylov
9756b8dc33
Added an ability to execute more merges and mutations than threads, added new settings (#29140) 2021-10-01 00:26:24 +03:00
Anton Popov
914781052e Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-29 17:37:07 +03:00
Azat Khuzhin
b108c5e1b0 Implement max_suspicious_broken_parts_bytes
Usually broken parts are found from just created parts, which are
usually small enough, so it make sense to have a limit in bytes for
broken parts.

Default value was set to 1G.
2021-09-17 22:06:32 +03:00
Anton Popov
8203bd1ac6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-09 14:04:37 +03:00
Azat Khuzhin
db0767a194 Implement detach_not_byte_identical_parts
Maybe useful for further analysis of non byte-identical parts.
2021-09-07 23:29:57 +03:00
alesapin
9a3bbf44e3 Better description 2021-08-31 15:28:17 +03:00
alesapin
921e51e061 Remove some obsolete settings for replicated fetches 2021-08-31 15:22:56 +03:00
lthaooo
314ba1e682
Merge branch 'master' into materialize_ttl_recalculate_only 2021-08-25 10:56:24 +08:00
mergify[bot]
ae22bab2c4
Merge branch 'master' into materialize_ttl_recalculate_only 2021-08-24 08:49:19 +00:00
Vitaly Baranov
89166a655a Init SettingFieldBool with true or false, not with 1 or 0. 2021-08-23 09:47:36 +03:00
Anton Popov
c3c3a06078 Merge remote-tracking branch 'upstream/master' into HEAD 2021-08-20 01:45:38 +03:00
terrylin
bd3d9a4518 materialize ttl recalculate only (optional) 2021-08-05 14:17:48 +08:00
Anton Ivashkin
61016da2b1 Set allow_remote_fs_zero_copy_replication to true by default 2021-08-04 17:08:09 +03:00
Anton Popov
c4b454494f Merge remote-tracking branch 'upstream/master' into HEAD 2021-07-20 15:41:01 +03:00
Filatenkov Artur
f1702d356e
Merge pull request #26120 from lthaooo/settings-merge_selecting_sleep_ms
add merge_selecting_sleep_ms setting
2021-07-14 18:28:47 +03:00
Anton Popov
e2249bf1e5
Merge pull request #26256 from zxc111/issue-822
Check hash function at table creation
2021-07-14 14:16:02 +03:00
Zhichang Yu
5047c758f4 fix per review 2021-07-13 07:20:20 +00:00
Zhichang Yu
fbd5eee8a1 hdfs zero copy 2021-07-13 07:19:12 +00:00
zxc111
e0261866a4 add settings && add test case 2021-07-13 01:15:35 +08:00
terrylin
faed6263bb add merge_selecting_sleep_ms setting 2021-07-09 17:29:17 +08:00
Anton Popov
5d64df1e39 disable sparse columns by default 2021-06-08 12:35:58 +03:00
Anton Popov
b3b29be02f Merge remote-tracking branch 'upstream/master' into HEAD 2021-06-07 19:03:24 +03:00
Anton Popov
e48922d320 temorarly make all columns sparse 2021-06-02 20:45:18 +03:00
Anton Popov
018a303387 Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-31 23:08:04 +03:00
alesapin
6924dd51a2 Merge branch 'master' into limited_fetches 2021-05-27 11:17:36 +03:00
alesapin
42b2383e01 Basic throttler implementation 2021-05-26 23:37:44 +03:00
alesapin
8ba6ed3fc2 Simplier test 2021-05-26 16:53:05 +03:00
Anton Popov
3e92c7f61a Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-25 21:45:19 +03:00
alesapin
6e29404a75 Add max_parts_to_merge_at_once setting 2021-05-25 16:21:17 +03:00
Anton Popov
c5d290cddb remove unused setting 2021-05-24 17:49:03 +03:00
Anton Popov
de4afb0cb9 ColumnSparse: add comments 2021-05-21 03:57:11 +03:00
Anton Popov
8ae1533f8f better serialization in native format 2021-05-14 23:29:48 +03:00
Anton Popov
e1e1435b00 ColumnSparse: use varint for offsets 2021-05-12 04:36:47 +03:00
Nicolae Vartolomei
1fa5871ff7 Fix bad rebase and introduce part_moves_between_shards_enable setting 2021-04-27 14:20:13 +01:00
Nicolae Vartolomei
53d57ffb52 Part movement between shards
Integrate query deduplication from #17348
2021-04-27 14:20:12 +01:00
Anton Popov
6ce875175b Merge remote-tracking branch 'upstream/master' into HEAD 2021-04-16 02:08:20 +03:00
Anton Popov
1df3e26d15 temporarily make all columns sparse 2021-04-08 05:09:41 +03:00
alesapin
6456a1507c Support alter setting 2021-04-06 13:14:44 +03:00
Anton Popov
d46958a8d2 Merge remote-tracking branch 'upstream/master' into HEAD 2021-04-06 00:54:49 +03:00
alesapin
156ba8a7fc Disable setting by default and ban for old-syntax MergeTree 2021-04-05 12:16:38 +03:00
alesapin
548ec6d2cb Trying enabled setting 2021-04-04 11:02:09 +03:00
alesapin
14634689e5 Idea code 2021-04-02 14:46:42 +03:00
Anton Popov
372a1b1fe7 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-29 19:57:49 +03:00
Anton Popov
577d571300 ColumnSparse: initial implementation 2021-03-29 19:54:24 +03:00
alexey-milovidov
1512ff832e
Merge branch 'master' into nv/replicated-fetches-timeouts 2021-03-29 19:25:13 +03:00
Anton Ivashkin
ace628d565 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-03-12 12:58:32 +03:00
alesapin
5704405f8c
Disable new setting by default 2021-03-11 12:29:27 +03:00
Anton Ivashkin
265d293934 Use 'merge on single replica' option instead of zookeeper lock 2021-03-09 20:39:55 +03:00
Amos Bird
e6522e1ebe
JBOD data balancer 2021-03-08 11:10:35 +08:00
Anton Ivashkin
5b267b7eec Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-02-26 13:15:18 +03:00
Nicolae Vartolomei
135f82ce94 Correctly document that settings apply only for fetch part requests 2021-02-08 19:44:03 +00:00
Nicolae Vartolomei
b153e8c190 Add support for custom fetchPart timeouts 2021-02-08 19:44:02 +00:00
Amos Bird
ab5c7b75a4
Delay or throw insertion when too many inactive parts 2021-02-07 19:03:55 +08:00
Anton Ivashkin
e64c63c611 Merge master 2021-02-05 20:10:06 +03:00
Anton Ivashkin
df6c882aab Fix build after merge 2021-02-05 18:52:40 +03:00
Alexey Milovidov
474f151707 More docs for MergeTreeSettings 2021-02-04 08:26:22 +03:00
Amos Bird
66fe97d8bd
Per MergeTree table query limit 2021-01-26 14:03:31 +08:00
Anton Ivashkin
357d98eb36 Merge master 2021-01-20 12:23:03 +03:00
alexey-milovidov
2c71b997de
Merge pull request #18464 from hexiaoting/dev_fp_as_pk
Disallow floating point as partition key
2021-01-12 13:08:41 +03:00
Amos Bird
a3d19fa64d
Correctly override default settings remotely 2021-01-08 12:28:09 +08:00
alexey-milovidov
5b98767652
Update MergeTreeSettings.h 2021-01-06 21:29:45 +03:00
Amos Bird
a157a5b3b3
add max_partitions_to_read setting 2021-01-04 12:40:48 +08:00
hexiaoting
e6c9361559 Add a MergeTree setting: allow_floating_point_partition_key 2020-12-29 11:12:02 +08:00
alesapin
f3f5466d23
Merge pull request #18306 from amosbird/restrictsettings
Extend mergetree setting alter restriction
2020-12-23 23:09:59 +03:00
alesapin
fb1c6c4eb4
Update MergeTreeSettings.h 2020-12-23 15:03:55 +03:00
Anton Popov
371050a16e remove unused code 2020-12-22 15:22:54 +03:00
Amos Bird
7a81e0a0f5
Extend mergetree setting alter restriction 2020-12-21 18:45:39 +08:00
Anton Ivashkin
0f0500ca0c Merge master 2020-12-16 18:31:13 +03:00
alexey-milovidov
f1ee1ef16a
Update MergeTreeSettings.h 2020-12-10 22:50:54 +03:00
feng lv
50b64ba1f8 fix 2020-12-07 13:18:31 +00:00
feng lv
d7b30f74d1 add min-max_compress_block_size to mergetree settings 2020-12-07 12:47:51 +00:00
Nicolae Vartolomei
94293ca3ce Assign UUIDs to parts only when configured to do so
Avoid breaking backwards compatibility by default for now.
2020-11-20 13:49:17 +00:00
Mikhail Filimonov
97fef77ed1
execute_merges_on_single_replica 2020-11-18 08:45:33 +01:00
Anton Popov
de5ead0c40 add setting remove_empty_parts 2020-11-15 05:24:47 +03:00
Anton Ivashkin
e3879afa69 S3 zero copy replication: fix virtual method default parameter 2020-10-23 16:27:50 +03:00
Pavel Kruglov
89fdeb4e15 Fix style, move setting and add checking level>0 2020-10-21 20:35:31 +03:00
Pavel Kruglov
8200bab859 Add setting do_not_merge_across_partitions 2020-10-13 17:54:52 +03:00
Alexander Tokmakov
ea16370024 improvements in lost replica recovery 2020-10-07 02:29:33 +03:00
Anton Popov
3e855daefd Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-15 15:31:29 +03:00
Anton Popov
f2a5216e97 add waiting for fsync in WAL 2020-09-11 02:28:50 +03:00
Anton Popov
af4089c606 Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-11 00:39:21 +03:00
Anton Popov
b9bf67b6ac allow to randomize part type in MergeTree 2020-09-10 04:27:36 +03:00
alesapin
8ff0aedebe Merge branch 'control_ttl_merges_in_pool' into recompression_in_background 2020-09-08 09:34:03 +03:00
alesapin
405a204514 Lower timeout for TTL merges 2020-09-08 09:33:00 +03:00
alesapin
fecb2f1311 Less copypaste 2020-09-04 17:08:43 +03:00
alesapin
ea7168580b Fixes 2020-09-04 16:55:07 +03:00
alesapin
4079636f81 Merge 2020-09-04 13:52:51 +03:00
alesapin
61ecaebcb1 Simplify settings for TTL merges 2020-09-04 09:55:19 +03:00
alesapin
f4c7ff0376 Add fixed size of Merge TTLS 2020-09-03 16:00:13 +03:00
alesapin
dcbddbb8d0 Add recompression ttls tests and fix bugs 2020-09-02 19:15:41 +03:00
alesapin
b08056fa8c Better selection of Merges with TTL 2020-09-02 11:18:50 +03:00
Anton Popov
4fe57fc169 Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-02 01:29:44 +03:00
Anton Popov
26d75f7602 do fsync for WAL 2020-09-02 01:25:10 +03:00
bharatnc
37416045ba setting min_index_granularity_bytes
This PR adds a new setting called `min_index_granularity_bytes`.

If the `index_granularity_bytes` is > 0 and is lesser than the
`min_index_granularity_bytes`, throw an exception inside the
`registerStorageMergeTree.cpp` file.
2020-08-26 15:31:48 -07:00
Anton Popov
4df5c72878 Merge remote-tracking branch 'upstream/master' into HEAD 2020-08-24 12:57:27 +03:00
alesapin
5de2491eda Enable mixed granularity parts by default 2020-08-07 11:55:51 +03:00
Alexey Milovidov
778abb346f Merge branch 'master' into merge-tree-settings-sanity-check 2020-08-02 17:13:17 +03:00
Vitaly Baranov
56665a15f7 Rework and rename the template class SettingsCollection => BaseSettings. 2020-07-31 20:54:18 +03:00
Vitaly Baranov
90602b869a Make SettingsChanges a class. 2020-07-31 19:11:27 +03:00
Vitaly Baranov
4ad99f1472 Make type names of settings shorter: SettingUInt64 => UInt64. 2020-07-31 19:11:27 +03:00