Commit Graph

182 Commits

Author SHA1 Message Date
Azat Khuzhin
6965ac26c3 Distributed: Add ability to delay/throttle INSERT until pending data will be reduced
Add two new settings for the Distributed engine:
- bytes_to_delay_insert
- max_delay_to_insert

If at the beginning of INSERT there will be too much pending data, more
then bytes_to_delay_insert, then the INSERT will wait until it will be
shrinked, and not more then max_delay_to_insert seconds.

If after this there will be still too much pending, it will throw an
exception.

Also new profile events were added (by analogy to the MergeTree):
- DistributedDelayedInserts (although you can use system.errors instead
  of this, but still)
- DistributedRejectedInserts
- DistributedDelayedInsertsMilliseconds
2021-03-03 23:30:23 +03:00
Azat Khuzhin
b5a5778589 Distributed: Add ability to limit amount of pending bytes for async INSERT
Right now with distributed_directory_monitor_batch_inserts=1 and
insert_distributed_sync=0 INSERT into Distributed table will store
blocks that should be sent to remote (and in case of
prefer_localhost_replica=0 to the localhost too) on the local
filesystem, and sent it in background.

However there is no limit for this storage, and if the remote is
unavailable (or some other error), these pending blocks may take
significant space, and this is not always desired behaviour.

Add new Distributed setting - bytes_to_throw_insert, that will set the
limit for how much pending bytes is allowed, if the limit will be
reached an exception will be throw.

By default was set to 0, to avoid surprises.
2021-03-03 23:30:00 +03:00
Nikita Mikhaylov
cb2a26308f
Merge pull request #20347 from gyuton/gyuton-DOCSUP-6549-Edit_and_translate
DOCSUP-6549: Edited and translated to Russian
2021-03-01 16:49:06 +03:00
Vladimir
9038fb3861
Merge pull request #20540 from lehasm/alexey-sm-DOCSUP-4328-document-xz-compression 2021-02-26 15:44:47 +03:00
lehasm
bb2ee9609f Compression mentioned in File engine 2021-02-23 16:43:40 +03:00
gyuton
0a58fd2eb2
Merge branch 'master' into gyuton-DOCSUP-6549-Edit_and_translate 2021-02-21 15:28:06 +03:00
Kseniia Sumarokova
20ff183549
Merge branch 'master' into rabbit-format-settings 2021-02-17 19:01:53 +03:00
kssenii
d79f7c04f6 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into rabbit-format-settings 2021-02-17 06:03:19 +00:00
kssenii
29362bb483 Support vhost 2021-02-16 22:05:14 +00:00
kssenii
d9f66d8d30 Better doc 2021-02-15 23:25:19 +00:00
kssenii
2a887b9772 Add missing format factory settings 2021-02-15 22:06:10 +00:00
alexey-milovidov
a299790522
Merge pull request #19748 from excitoon-favorites/s3ssec
Added Server Side Encryption Customer Keys support in S3 client
2021-02-15 15:23:14 +03:00
George
838dab7564 Edit and translated Kafka 2021-02-11 21:07:38 +03:00
Kruglov Pavel
df7cc6b195
Merge pull request #20026 from gyuton/gyuton-DOCSUP-5788-Document_the_optimize_on_insert_setting
DOCSUP-5788: Documented the optimize_on_insert setting
2021-02-10 19:55:32 +03:00
Nikita Mikhaylov
39c73cb448
Merge pull request #19954 from gyuton/gyuton-DOCSUP-5978-document_max_partitions_to_read
DOCUP-5978: Documented max_partitions_to_read
2021-02-09 15:47:17 +03:00
alesapin
f34bec4be4
Update mongodb.md 2021-02-08 17:30:15 +03:00
alesapin
313d17bf96
Update mongodb.md 2021-02-08 10:10:07 +03:00
Olga Revyakina
f910f00285 Description and link (en, ru) 2021-02-07 17:29:54 +03:00
George
a23853c141 minor fixes 2021-02-06 12:42:07 +03:00
gyuton
29c01ed9d0
Update docs/en/engines/table-engines/mergetree-family/mergetree.md
Co-authored-by: Anna <42538400+adevyatova@users.noreply.github.com>
2021-02-05 11:41:57 +03:00
George
932589744c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into gyuton-DOCSUP-5788-Document_the_optimize_on_insert_setting 2021-02-05 11:36:25 +03:00
George
b1a43b9d85 fixed link 2021-02-03 11:20:32 +03:00
George
678f53eaef Fixed a link 2021-02-03 11:15:25 +03:00
George
3d522e5da9 Updated description 2021-02-03 10:56:53 +03:00
Anton Popov
fb02d56534
Merge pull request #19519 from olgarev/revolg-DOCSUP-4962-WHERE_and_GROUP_BY_in_TTL_expressions
DOCSUP-4962: Documented support of WHERE and GROUP BY in TTL expressions
2021-02-02 14:39:45 +03:00
George
e6367de2d9 Added documentation 2021-02-02 05:23:02 +03:00
Olga Revyakina
fb98aeeae1 Fixes 2021-02-02 02:44:40 +03:00
Kruglov Pavel
742b740753
Merge pull request #18490 from damozhaeva/dmozhaeva-DOCSUP-4705-edit-and-translate
DOCSUP-4705 edit and translate
2021-01-29 14:46:42 +03:00
Vladimir Chebotarev
579f8da573 Added SSE-C support in S3 client. 2021-01-28 09:32:41 +03:00
Daria Mozhaeva
f4448ef01e links 2021-01-26 22:24:06 +03:00
vdimir
fc8b8d23b2
Merge pull request #18812 from excitoon-favorites/betters3endpoints
Added prefix-based S3 endpoint settings
2021-01-26 20:47:57 +03:00
tavplubix
27ddf78ba5
Merge pull request #19521 from adevyatova/annadevyatova-DOCSUP-5634-replace
DOCSUP-5634: REPLACE TABLE
2021-01-26 11:41:47 +03:00
Vladimir Chebotarev
68a671f009 Fixed documentation. 2021-01-26 11:33:21 +03:00
Anna
cec83bb76a Update 2021-01-25 21:49:13 +03:00
Anna
2a66eab876
Merge branch 'master' into annadevyatova-DOCSUP-5634-replace 2021-01-25 20:10:42 +03:00
Daria Mozhaeva
39c99edfe5 edit eng ver 2021-01-25 17:13:29 +03:00
Vladimir Chebotarev
7a675c996e Added documentation for S3 endpoints. 2021-01-25 09:33:53 +03:00
damozhaeva
9d91855099
Merge branch 'master' into dmozhaeva-DOCSUP-4705-edit-and-translate 2021-01-24 22:47:06 +03:00
Olga Revyakina
ab8b78d19d Warning added 2021-01-24 11:32:59 +03:00
Olga Revyakina
a541241017 Global MySQL Variables Support 2021-01-24 01:52:50 +03:00
Anna
ae4a2e9c6f Add REPLACE query desc 2021-01-23 22:46:24 +03:00
Olga Revyakina
135426d3cb Syntax updated, examples added. 2021-01-23 21:16:59 +03:00
Daria Mozhaeva
f13a075797 Fixed notes. 2021-01-22 19:40:58 +03:00
alexey-milovidov
8642ba107a
Merge pull request #19246 from sevirov/sevirov-DOCSUP-5840-edit_translate_russian_materializemysql
DOCSUP-5840: Edit and translate to Russian
2021-01-22 17:31:54 +03:00
Dmitriy
0cdf184242 Fix materialize-mysql.md and index.md
Поправил согласно комментариям разработчика.
2021-01-21 23:14:49 +03:00
Pavel Kovalenko
6db51965cd Add S3 disk documentation [EN] 2021-01-19 12:03:48 +03:00
Dmitriy
dcf1c1a071 Update roadmap.md and materialize-mysql.md
Правлю ссылки.
2021-01-18 18:36:14 +03:00
Dmitriy
e495284a0a Edit and translate to Russian
Внес поправки в английскую версию и выполнил перевод на русский язык
2021-01-18 16:43:00 +03:00
sevirov
33cf7351b0
DOCSUP-5169: Document the min_compress_block_size and max_compress_block_size settings (#18248)
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2021-01-10 23:21:32 +03:00
Azat Khuzhin
471deab63a Rename fsync_tmp_directory to fsync_directories for Distributed engine 2021-01-09 17:51:30 +03:00