thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-22 23:52:03 +00:00
Code Issues Packages Projects Releases Wiki Activity
58,496 Commits 9,163 Branches 1,781 Tags 2.1 GiB
bd8c3eb117
Commit Graph

58496 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
filimonov
bd8c3eb117
avoid race in librdkafka 
see https://github.com/edenhill/librdkafka/issues/3279
 2021-03-04 15:01:10 +01:00
Ivan
218542589a
Merge pull request #19673 from azat/distributed-bytes_to_throw_insert 
Add ability to throttle INSERT into Distributed
 2021-03-04 13:28:23 +03:00
Nikolai Kochetov
b985e33294
Merge pull request #21434 from azat/dist-query-SIGSEGV-on-exception-fix 
Fix SIGSEGV for distributed queries on failures
 2021-03-04 11:32:17 +03:00
olgarev
134eaa55e5
DOCSUP-6258: FAQ edit and translate (#21109) 
Co-authored-by: ana-uvarova <ana-uvarova@yandex-team.ru>
Co-authored-by: AnaUvarova <64017504+AnaUvarova@users.noreply.github.com>
Co-authored-by: vdimir <vdimir@yandex-team.ru>
Co-authored-by: Olga Revyakina <revolg@yandex-team.ru>
Co-authored-by: Anna <42538400+adevyatova@users.noreply.github.com>
 2021-03-04 10:50:14 +03:00
Azat Khuzhin
e858dae187 Fix SIGSEGV for distributed queries on failures 
Since after pull(), cancel() will be called, and this will lead to
SIGSEGV, since there is no exception, but has_exception was not reseted
in pull()
 2021-03-04 00:42:49 +03:00
Azat Khuzhin
79ed35876e DirectoryMonitor: Remove const qualifier and lots of mutable qualifiers 2021-03-03 23:30:24 +03:00
Azat Khuzhin
45ee650e26 Distributed: check for bytes_to_throw/delay_insert only before INSERT 
Before it was checked for each block.
 2021-03-03 23:30:24 +03:00
Azat Khuzhin
a57c646467 Fix 01293_system_distribution_queue flakiness 2021-03-03 23:30:24 +03:00
Azat Khuzhin
9a4f469a38 Fix 00753_distributed_system_columns_and_system_tables flakiness 2021-03-03 23:30:24 +03:00
Azat Khuzhin
6965ac26c3 Distributed: Add ability to delay/throttle INSERT until pending data will be reduced 
Add two new settings for the Distributed engine:
- bytes_to_delay_insert
- max_delay_to_insert

If at the beginning of INSERT there will be too much pending data, more
then bytes_to_delay_insert, then the INSERT will wait until it will be
shrinked, and not more then max_delay_to_insert seconds.

If after this there will be still too much pending, it will throw an
exception.

Also new profile events were added (by analogy to the MergeTree):
- DistributedDelayedInserts (although you can use system.errors instead
  of this, but still)
- DistributedRejectedInserts
- DistributedDelayedInsertsMilliseconds
 2021-03-03 23:30:23 +03:00
Azat Khuzhin
15f7459cae Distributed/DirectoryMonitor: protect metric_pending_files with metrics_lock 
Since there is local value, that is not atomic, anyway we already have
lock for metrics, so it is fine.
 2021-03-03 23:30:03 +03:00
Azat Khuzhin
cabe4ca1bb tests: split 00753_system_columns_and_system_tables (to disable Distributed part for arcadia) 2021-03-03 23:30:03 +03:00
Azat Khuzhin
3102835d56 tests: Use total_bytes>0 for Distributed engine in 00753_system_columns_and_system_tables 2021-03-03 23:30:03 +03:00
Azat Khuzhin
70049db143 CurrentMetrics/Increment: Introduce add() 2021-03-03 23:30:03 +03:00
Azat Khuzhin
017c054a35 Distributed/DirectoryMonitor: Use std::lock_guard over std::unique_lock 
It is more natural, since we do not need lazy locking.
 2021-03-03 23:30:03 +03:00
Azat Khuzhin
fcf49a4914 Distributed: Calculate counters for async INSERT at INSERT time 
Previous patch fixes the inaccuracy, but it's done using iterating over
directory on each request (to system.distribution_queue or to check
bytes_to_throw_insert), and like previous patch alredy stated, it may
have pretty huge overhead (especially when you have lots of distributed
files pending).

This patch remove that recalculation (but it will still be done, and
if there is different, there will be a log message), and replace it with
proper account at INSERT time (and after file has been sent, or marked
as broken).
 2021-03-03 23:30:03 +03:00
Azat Khuzhin
b43046ba06 Distributed: More accurate distribution_queue counters 
So now system.distribution_queue will show accurate statistics, so tests
does not requires sleep anymore.

But note that with too much distributed pending this will iterate over
all directories.
 2021-03-03 23:30:03 +03:00
Azat Khuzhin
b5a5778589 Distributed: Add ability to limit amount of pending bytes for async INSERT 
Right now with distributed_directory_monitor_batch_inserts=1 and
insert_distributed_sync=0 INSERT into Distributed table will store
blocks that should be sent to remote (and in case of
prefer_localhost_replica=0 to the localhost too) on the local
filesystem, and sent it in background.

However there is no limit for this storage, and if the remote is
unavailable (or some other error), these pending blocks may take
significant space, and this is not always desired behaviour.

Add new Distributed setting - bytes_to_throw_insert, that will set the
limit for how much pending bytes is allowed, if the limit will be
reached an exception will be throw.

By default was set to 0, to avoid surprises.
 2021-03-03 23:30:00 +03:00
Azat Khuzhin
ce09b7ff89 Distributed: Implement totalBytes() (system.tables.total_bytes) 2021-03-03 23:29:11 +03:00
Azat Khuzhin
456cbaf747 Distributed: Hide private part of the interface 2021-03-03 23:29:11 +03:00
Azat Khuzhin
8531431bbc getClusterName: Use std::string over String (since it includes only <string>) 2021-03-03 23:29:11 +03:00
Alexey Milovidov
e8df9971f1 Fix Arcadia 2021-03-03 18:12:39 +03:00
robot-clickhouse
6a6bc3924d Update version_date.tsv after release 20.8.14.4 2021-03-03 16:53:26 +03:00
Nikolai Kochetov
991f0e4bca
Merge pull request #21418 from Avogar/hedged-fix 
Fix typo in setting name
 2021-03-03 16:27:52 +03:00
Pavel Kruglov
dc9dd8ccdc Fix typo 2021-03-03 16:16:17 +03:00
Alexander Kuzmenkov
f169be740a
Merge pull request #21381 from ClickHouse/aku/perf-negative 
fix a rare false negative in perf tests
 2021-03-03 15:55:14 +03:00
Nikolai Kochetov
019513d869 Try fix perftests. 2021-03-03 13:42:44 +03:00
Nikolai Kochetov
cb122160ae
Merge pull request #19291 from Avogar/hedged-requests 
Implementation of HedgedRequests
 2021-03-03 12:58:37 +03:00
alesapin
6b1005aea5
Merge pull request #21334 from ClickHouse/fix_alter_partition_key 
Fix alter modify query for partition key and other metadata fields
 2021-03-03 10:15:11 +03:00
alexey-milovidov
36c5fd840b
Merge pull request #21392 from ClickHouse/add-test-14740 
Add a test for #14740
 2021-03-03 06:33:18 +03:00
alexey-milovidov
bd7b540b82
Merge pull request #21307 from amosbird/moduleopt 
Specialize intDiv/module
 2021-03-03 04:49:29 +03:00
alexey-milovidov
9092bc3b85
Merge pull request #21387 from den-crane/tests/issue21369 
test for issue#21369
 2021-03-03 03:05:01 +03:00
Anton Popov
a4c00ab5dc
Merge pull request #21303 from ucasFL/forbid 
Forbid to drop a column if it's referenced by materialized view
 2021-03-03 02:55:06 +03:00
alexey-milovidov
33bb77f777
Merge pull request #21367 from amosbird/mysqlfix 
mysql client compatibility
 2021-03-03 02:13:58 +03:00
alexey-milovidov
a1424bc675
Merge pull request #21365 from azat/clickhouse-stop-kill-wait 
clickhouse stop: wait for the server to be killed (process exited)
 2021-03-03 02:11:17 +03:00
Alexey Milovidov
a93e29ef4a Add a test for #14740 2021-03-03 02:00:19 +03:00
alexey-milovidov
6e5638cee8
Merge pull request #21366 from songenjie/shutdown_timeout_log 
[ClickHouse][LOG]correct shutdown timeout log
 2021-03-03 01:55:15 +03:00
Pavel Kruglov
d1524b749e Fix tests 2021-03-03 01:24:41 +03:00
alexey-milovidov
c78256ad65
Merge pull request #21327 from amosbird/hasequalvalue 
IColunm::hasEqualValues()
 2021-03-03 01:19:26 +03:00
Pavel Kruglov
82a6be44cf Disable changing replica when we have made progress 2021-03-03 01:03:03 +03:00
Kruglov Pavel
fbe0a14a2f
Merge pull request #21035 from anmolarora/system-parts-requested-columns-fill 
Fill only requested columns when querying system.parts & system.parts…
 2021-03-03 00:39:45 +03:00
Denis Zhuravlev
1278b5c71a test for issue#21369 2021-03-02 16:29:11 -04:00
Nikolai Kochetov
8e6fa404c4
Merge pull request #21246 from ucasFL/union-distinct-improve 
Improve Normalization of ASTSelectWithUnionQuery
 2021-03-02 21:37:07 +03:00
Alexander Kuzmenkov
08148e062f Fix a rare false negative in perf tests 2021-03-02 19:21:30 +03:00
alesapin
c29d7c7f49 Shutup clang tidy 2021-03-02 19:13:36 +03:00
Amos Bird
9411d496d9
Fix build 2021-03-02 23:17:02 +08:00
Pavel Kruglov
4269eaa252 Change tests 2021-03-02 17:05:33 +03:00
Amos Bird
134216dc4a Specialize intDiv/module 2021-03-02 16:27:28 +03:00
robot-clickhouse
d9e22ba646 Update version_date.tsv after release 20.12.8.5 2021-03-02 15:53:39 +03:00
Pavel Kruglov
4166ae1f45 Restart timeout when we make progress 2021-03-02 15:40:24 +03:00