Commit Graph

58537 Commits

Author SHA1 Message Date
Nikolai Kochetov
4d697c27e4 Fix some tests. 2021-03-09 19:54:06 +03:00
Nikolai Kochetov
a669f7d641 Merge branch 'master' into refactor-actions-dag 2021-03-05 18:21:14 +03:00
Nikolai Kochetov
fc415952ce Try fix test. 2021-03-05 16:56:44 +03:00
Kruglov Pavel
5597da968c
Merge pull request #21441 from DIAOZHAFENG/patch-1
Update in.md
2021-03-05 16:55:35 +03:00
Nikolai Kochetov
c481401b0b Fix build. 2021-03-05 12:54:17 +03:00
Nikolai Kochetov
5f6c8eb1d0
Merge pull request #21447 from ClickHouse/try-fix-perftests
Try fix perftests.
2021-03-05 12:48:28 +03:00
alesapin
9a9d5a74ba
Merge pull request #21439 from ClickHouse/fix_nukeeper_default_path
Use path as default prefix for coordination logs
2021-03-05 10:11:44 +03:00
alexey-milovidov
777731562d
Merge pull request #21458 from amosbird/trivialfix
Tiny fix
2021-03-05 05:21:52 +03:00
Alexander Kuzmenkov
81dc94bca7
Merge pull request #19836 from sevirov/sevirov-DOCSUP-4707-edit_translate_russian_date_time
DOCSUP-4707: Edit and translate to Russian
2021-03-05 05:11:26 +03:00
alexey-milovidov
643b065849
Update datetime.md 2021-03-05 02:55:12 +03:00
alexey-milovidov
1985364c8f
Update datetime.md 2021-03-05 02:35:54 +03:00
Nikita Mikhaylov
37d4c69d3a
Merge pull request #21433 from azat/parsing-exception-message-fix
Fix ParsingException::displayText()
2021-03-05 01:59:06 +03:00
tavplubix
9244bc43ee
Merge pull request #21021 from zhang2014/fix/materialize_mysql_integration_test
Try fix MaterializeMySQL integration test
2021-03-04 22:22:20 +03:00
Nikolai Kochetov
9a39459888 Refactor ActionsDAG 2021-03-04 20:38:12 +03:00
Dmitriy
fbc4791db7 Add the links
Добавил ссылки на продукты, убрал кавычки со стандарта.
2021-03-04 20:13:22 +03:00
Amos Bird
de94f33b05
Tiny fix 2021-03-05 00:21:52 +08:00
Kseniia Sumarokova
08cb02c872
Merge pull request #21416 from excitoon-favorites/caseinsensitivecompression
Case-insensitive compression methods for table functions
2021-03-04 17:37:28 +03:00
alesapin
39de525c74 Simplify config 2021-03-04 16:49:22 +03:00
Ivan
218542589a
Merge pull request #19673 from azat/distributed-bytes_to_throw_insert
Add ability to throttle INSERT into Distributed
2021-03-04 13:28:23 +03:00
Nikolai Kochetov
a195e783d2 Try fix perftests. 2021-03-04 13:25:49 +03:00
DIAOZHAFENG
35aaac84b8
Update in.md 2021-03-04 16:58:09 +08:00
DIAOZHAFENG
587014217c
Update in.md
update untranslated line
2021-03-04 16:47:59 +08:00
Nikolai Kochetov
b985e33294
Merge pull request #21434 from azat/dist-query-SIGSEGV-on-exception-fix
Fix SIGSEGV for distributed queries on failures
2021-03-04 11:32:17 +03:00
alesapin
d28463d70b Use path as default prefix for coordination logs 2021-03-04 11:29:24 +03:00
olgarev
134eaa55e5
DOCSUP-6258: FAQ edit and translate (#21109)
Co-authored-by: ana-uvarova <ana-uvarova@yandex-team.ru>
Co-authored-by: AnaUvarova <64017504+AnaUvarova@users.noreply.github.com>
Co-authored-by: vdimir <vdimir@yandex-team.ru>
Co-authored-by: Olga Revyakina <revolg@yandex-team.ru>
Co-authored-by: Anna <42538400+adevyatova@users.noreply.github.com>
2021-03-04 10:50:14 +03:00
Azat Khuzhin
e858dae187 Fix SIGSEGV for distributed queries on failures
Since after pull(), cancel() will be called, and this will lead to
SIGSEGV, since there is no exception, but has_exception was not reseted
in pull()
2021-03-04 00:42:49 +03:00
Azat Khuzhin
e7948819f9 Fix ParsingException::displayText()
Before it was silently try-catched for messages with additional {}, and
it is very easy to trigger, i.e.:

    SELECT toDateTime(format('{}-{}-01 00:00:00', '2021', '1'))

Will print:

    Code: 41. DB::Exception: Received from localhost:9000. DB::Exception: Cannot parse datetime 2021-1-01 00:00:00{}: Cannot parse DateTime from String: while executing 'FUNCTION toDateTime(format('{}-{}-01 00:00:00', '2021', '1') :: 3) -> toDateTime(format('{}-{}-01 00:00:00', '2021', '1')) DateTime : 2'.
2021-03-03 23:44:51 +03:00
Azat Khuzhin
79ed35876e DirectoryMonitor: Remove const qualifier and lots of mutable qualifiers 2021-03-03 23:30:24 +03:00
Azat Khuzhin
45ee650e26 Distributed: check for bytes_to_throw/delay_insert only before INSERT
Before it was checked for each block.
2021-03-03 23:30:24 +03:00
Azat Khuzhin
a57c646467 Fix 01293_system_distribution_queue flakiness 2021-03-03 23:30:24 +03:00
Azat Khuzhin
9a4f469a38 Fix 00753_distributed_system_columns_and_system_tables flakiness 2021-03-03 23:30:24 +03:00
Azat Khuzhin
6965ac26c3 Distributed: Add ability to delay/throttle INSERT until pending data will be reduced
Add two new settings for the Distributed engine:
- bytes_to_delay_insert
- max_delay_to_insert

If at the beginning of INSERT there will be too much pending data, more
then bytes_to_delay_insert, then the INSERT will wait until it will be
shrinked, and not more then max_delay_to_insert seconds.

If after this there will be still too much pending, it will throw an
exception.

Also new profile events were added (by analogy to the MergeTree):
- DistributedDelayedInserts (although you can use system.errors instead
  of this, but still)
- DistributedRejectedInserts
- DistributedDelayedInsertsMilliseconds
2021-03-03 23:30:23 +03:00
Azat Khuzhin
15f7459cae Distributed/DirectoryMonitor: protect metric_pending_files with metrics_lock
Since there is local value, that is not atomic, anyway we already have
lock for metrics, so it is fine.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
cabe4ca1bb tests: split 00753_system_columns_and_system_tables (to disable Distributed part for arcadia) 2021-03-03 23:30:03 +03:00
Azat Khuzhin
3102835d56 tests: Use total_bytes>0 for Distributed engine in 00753_system_columns_and_system_tables 2021-03-03 23:30:03 +03:00
Azat Khuzhin
70049db143 CurrentMetrics/Increment: Introduce add() 2021-03-03 23:30:03 +03:00
Azat Khuzhin
017c054a35 Distributed/DirectoryMonitor: Use std::lock_guard over std::unique_lock
It is more natural, since we do not need lazy locking.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
fcf49a4914 Distributed: Calculate counters for async INSERT at INSERT time
Previous patch fixes the inaccuracy, but it's done using iterating over
directory on each request (to system.distribution_queue or to check
bytes_to_throw_insert), and like previous patch alredy stated, it may
have pretty huge overhead (especially when you have lots of distributed
files pending).

This patch remove that recalculation (but it will still be done, and
if there is different, there will be a log message), and replace it with
proper account at INSERT time (and after file has been sent, or marked
as broken).
2021-03-03 23:30:03 +03:00
Azat Khuzhin
b43046ba06 Distributed: More accurate distribution_queue counters
So now system.distribution_queue will show accurate statistics, so tests
does not requires sleep anymore.

But note that with too much distributed pending this will iterate over
all directories.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
b5a5778589 Distributed: Add ability to limit amount of pending bytes for async INSERT
Right now with distributed_directory_monitor_batch_inserts=1 and
insert_distributed_sync=0 INSERT into Distributed table will store
blocks that should be sent to remote (and in case of
prefer_localhost_replica=0 to the localhost too) on the local
filesystem, and sent it in background.

However there is no limit for this storage, and if the remote is
unavailable (or some other error), these pending blocks may take
significant space, and this is not always desired behaviour.

Add new Distributed setting - bytes_to_throw_insert, that will set the
limit for how much pending bytes is allowed, if the limit will be
reached an exception will be throw.

By default was set to 0, to avoid surprises.
2021-03-03 23:30:00 +03:00
Azat Khuzhin
ce09b7ff89 Distributed: Implement totalBytes() (system.tables.total_bytes) 2021-03-03 23:29:11 +03:00
Azat Khuzhin
456cbaf747 Distributed: Hide private part of the interface 2021-03-03 23:29:11 +03:00
Azat Khuzhin
8531431bbc getClusterName: Use std::string over String (since it includes only <string>) 2021-03-03 23:29:11 +03:00
Nikolai Kochetov
15921fbfcb Remove index by name from ActionsDAG 2021-03-03 23:01:07 +03:00
Alexey Milovidov
e8df9971f1 Fix Arcadia 2021-03-03 18:12:39 +03:00
robot-clickhouse
6a6bc3924d Update version_date.tsv after release 20.8.14.4 2021-03-03 16:53:26 +03:00
Nikolai Kochetov
991f0e4bca
Merge pull request #21418 from Avogar/hedged-fix
Fix typo in setting name
2021-03-03 16:27:52 +03:00
Pavel Kruglov
dc9dd8ccdc Fix typo 2021-03-03 16:16:17 +03:00
Alexander Kuzmenkov
f169be740a
Merge pull request #21381 from ClickHouse/aku/perf-negative
fix a rare false negative in perf tests
2021-03-03 15:55:14 +03:00
Vladimir Chebotarev
4022418ff3 Case-insensitive compression methods. 2021-03-03 14:51:09 +03:00