Azat Khuzhin
15f7459cae
Distributed/DirectoryMonitor: protect metric_pending_files with metrics_lock
...
Since there is local value, that is not atomic, anyway we already have
lock for metrics, so it is fine.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
cabe4ca1bb
tests: split 00753_system_columns_and_system_tables (to disable Distributed part for arcadia)
2021-03-03 23:30:03 +03:00
Azat Khuzhin
3102835d56
tests: Use total_bytes>0 for Distributed engine in 00753_system_columns_and_system_tables
2021-03-03 23:30:03 +03:00
Azat Khuzhin
70049db143
CurrentMetrics/Increment: Introduce add()
2021-03-03 23:30:03 +03:00
Azat Khuzhin
017c054a35
Distributed/DirectoryMonitor: Use std::lock_guard over std::unique_lock
...
It is more natural, since we do not need lazy locking.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
fcf49a4914
Distributed: Calculate counters for async INSERT at INSERT time
...
Previous patch fixes the inaccuracy, but it's done using iterating over
directory on each request (to system.distribution_queue or to check
bytes_to_throw_insert), and like previous patch alredy stated, it may
have pretty huge overhead (especially when you have lots of distributed
files pending).
This patch remove that recalculation (but it will still be done, and
if there is different, there will be a log message), and replace it with
proper account at INSERT time (and after file has been sent, or marked
as broken).
2021-03-03 23:30:03 +03:00
Azat Khuzhin
b43046ba06
Distributed: More accurate distribution_queue counters
...
So now system.distribution_queue will show accurate statistics, so tests
does not requires sleep anymore.
But note that with too much distributed pending this will iterate over
all directories.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
b5a5778589
Distributed: Add ability to limit amount of pending bytes for async INSERT
...
Right now with distributed_directory_monitor_batch_inserts=1 and
insert_distributed_sync=0 INSERT into Distributed table will store
blocks that should be sent to remote (and in case of
prefer_localhost_replica=0 to the localhost too) on the local
filesystem, and sent it in background.
However there is no limit for this storage, and if the remote is
unavailable (or some other error), these pending blocks may take
significant space, and this is not always desired behaviour.
Add new Distributed setting - bytes_to_throw_insert, that will set the
limit for how much pending bytes is allowed, if the limit will be
reached an exception will be throw.
By default was set to 0, to avoid surprises.
2021-03-03 23:30:00 +03:00
Azat Khuzhin
ce09b7ff89
Distributed: Implement totalBytes() (system.tables.total_bytes)
2021-03-03 23:29:11 +03:00
Azat Khuzhin
456cbaf747
Distributed: Hide private part of the interface
2021-03-03 23:29:11 +03:00
Azat Khuzhin
8531431bbc
getClusterName: Use std::string over String (since it includes only <string>)
2021-03-03 23:29:11 +03:00
alesapin
76e10a4fb1
Missed files for tests
2021-03-03 23:05:33 +03:00
alesapin
0f733828c2
Missed config
2021-03-03 19:44:06 +03:00
alesapin
15a29dd3e6
Stop yelling
2021-03-03 19:28:46 +03:00
alesapin
4e175bd988
Useful comments
2021-03-03 19:27:08 +03:00
alesapin
e6083c4dc4
Fix style
2021-03-03 19:25:47 +03:00
alesapin
eb5e416fd5
Merge branch 'master' into persistent_nukeeper_snapshot_storage
2021-03-03 19:19:02 +03:00
alesapin
e6f4ce1741
Better initial batch callback
2021-03-03 19:16:55 +03:00
alesapin
e77f7e57e2
Better initial batch callback
2021-03-03 19:13:29 +03:00
alesapin
7e28bfeb9e
Fix log store start
2021-03-03 18:37:31 +03:00
Alexey Milovidov
e8df9971f1
Fix Arcadia
2021-03-03 18:12:39 +03:00
robot-clickhouse
6a6bc3924d
Update version_date.tsv after release 20.8.14.4
2021-03-03 16:53:26 +03:00
Nikolai Kochetov
991f0e4bca
Merge pull request #21418 from Avogar/hedged-fix
...
Fix typo in setting name
2021-03-03 16:27:52 +03:00
alesapin
869818d4ff
Better test
2021-03-03 16:18:28 +03:00
Pavel Kruglov
dc9dd8ccdc
Fix typo
2021-03-03 16:16:17 +03:00
Alexander Kuzmenkov
f169be740a
Merge pull request #21381 from ClickHouse/aku/perf-negative
...
fix a rare false negative in perf tests
2021-03-03 15:55:14 +03:00
alesapin
186b39f0d5
Followup fix
2021-03-03 15:31:21 +03:00
alesapin
0136d7d51a
Get rid of unused field
2021-03-03 15:29:00 +03:00
alesapin
37fc5faa6f
Fix nasty serialization bug
2021-03-03 15:21:21 +03:00
Vladimir Chebotarev
4022418ff3
Case-insensitive compression methods.
2021-03-03 14:51:09 +03:00
alesapin
1707e7f1c3
Rename file
2021-03-03 14:22:38 +03:00
alesapin
e615299ae6
Fix some bugs and add snapshots validation
2021-03-03 14:10:24 +03:00
Nikolai Kochetov
019513d869
Try fix perftests.
2021-03-03 13:42:44 +03:00
Nikolai Kochetov
cb122160ae
Merge pull request #19291 from Avogar/hedged-requests
...
Implementation of HedgedRequests
2021-03-03 12:58:37 +03:00
alesapin
07e39ef47a
Merge branch 'master' into persistent_nukeeper_snapshot_storage
2021-03-03 11:12:01 +03:00
alesapin
6b1005aea5
Merge pull request #21334 from ClickHouse/fix_alter_partition_key
...
Fix alter modify query for partition key and other metadata fields
2021-03-03 10:15:11 +03:00
alexey-milovidov
36c5fd840b
Merge pull request #21392 from ClickHouse/add-test-14740
...
Add a test for #14740
2021-03-03 06:33:18 +03:00
alexey-milovidov
bd7b540b82
Merge pull request #21307 from amosbird/moduleopt
...
Specialize intDiv/module
2021-03-03 04:49:29 +03:00
Alexey Milovidov
71b52542d9
Add to skip list for parallel runs
2021-03-03 04:37:11 +03:00
Alexey Milovidov
8f45ed5117
Add a test for #15469
2021-03-03 04:35:43 +03:00
alexey-milovidov
9092bc3b85
Merge pull request #21387 from den-crane/tests/issue21369
...
test for issue#21369
2021-03-03 03:05:01 +03:00
Anton Popov
a4c00ab5dc
Merge pull request #21303 from ucasFL/forbid
...
Forbid to drop a column if it's referenced by materialized view
2021-03-03 02:55:06 +03:00
alexey-milovidov
33bb77f777
Merge pull request #21367 from amosbird/mysqlfix
...
mysql client compatibility
2021-03-03 02:13:58 +03:00
alexey-milovidov
a1424bc675
Merge pull request #21365 from azat/clickhouse-stop-kill-wait
...
clickhouse stop: wait for the server to be killed (process exited)
2021-03-03 02:11:17 +03:00
Alexey Milovidov
a93e29ef4a
Add a test for #14740
2021-03-03 02:00:19 +03:00
alexey-milovidov
6e5638cee8
Merge pull request #21366 from songenjie/shutdown_timeout_log
...
[ClickHouse][LOG]correct shutdown timeout log
2021-03-03 01:55:15 +03:00
Pavel Kruglov
d1524b749e
Fix tests
2021-03-03 01:24:41 +03:00
alexey-milovidov
c78256ad65
Merge pull request #21327 from amosbird/hasequalvalue
...
IColunm::hasEqualValues()
2021-03-03 01:19:26 +03:00
Pavel Kruglov
82a6be44cf
Disable changing replica when we have made progress
2021-03-03 01:03:03 +03:00
Kruglov Pavel
fbe0a14a2f
Merge pull request #21035 from anmolarora/system-parts-requested-columns-fill
...
Fill only requested columns when querying system.parts & system.parts…
2021-03-03 00:39:45 +03:00
