kssenii
5ee2925123
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge
2021-03-08 19:53:48 +00:00
kssenii
61d8e27ea7
Common base for bridges part 2
2021-03-07 11:21:49 +00:00
robot-clickhouse
4e61f45dbb
Auto version update to [21.4.1.1] [54449]
2021-03-06 14:43:30 +03:00
Azat Khuzhin
79ed35876e
DirectoryMonitor: Remove const qualifier and lots of mutable qualifiers
2021-03-03 23:30:24 +03:00
Azat Khuzhin
45ee650e26
Distributed: check for bytes_to_throw/delay_insert only before INSERT
...
Before it was checked for each block.
2021-03-03 23:30:24 +03:00
Azat Khuzhin
6965ac26c3
Distributed: Add ability to delay/throttle INSERT until pending data will be reduced
...
Add two new settings for the Distributed engine:
- bytes_to_delay_insert
- max_delay_to_insert
If at the beginning of INSERT there will be too much pending data, more
then bytes_to_delay_insert, then the INSERT will wait until it will be
shrinked, and not more then max_delay_to_insert seconds.
If after this there will be still too much pending, it will throw an
exception.
Also new profile events were added (by analogy to the MergeTree):
- DistributedDelayedInserts (although you can use system.errors instead
of this, but still)
- DistributedRejectedInserts
- DistributedDelayedInsertsMilliseconds
2021-03-03 23:30:23 +03:00
Azat Khuzhin
15f7459cae
Distributed/DirectoryMonitor: protect metric_pending_files with metrics_lock
...
Since there is local value, that is not atomic, anyway we already have
lock for metrics, so it is fine.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
70049db143
CurrentMetrics/Increment: Introduce add()
2021-03-03 23:30:03 +03:00
Azat Khuzhin
017c054a35
Distributed/DirectoryMonitor: Use std::lock_guard over std::unique_lock
...
It is more natural, since we do not need lazy locking.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
fcf49a4914
Distributed: Calculate counters for async INSERT at INSERT time
...
Previous patch fixes the inaccuracy, but it's done using iterating over
directory on each request (to system.distribution_queue or to check
bytes_to_throw_insert), and like previous patch alredy stated, it may
have pretty huge overhead (especially when you have lots of distributed
files pending).
This patch remove that recalculation (but it will still be done, and
if there is different, there will be a log message), and replace it with
proper account at INSERT time (and after file has been sent, or marked
as broken).
2021-03-03 23:30:03 +03:00
Azat Khuzhin
b43046ba06
Distributed: More accurate distribution_queue counters
...
So now system.distribution_queue will show accurate statistics, so tests
does not requires sleep anymore.
But note that with too much distributed pending this will iterate over
all directories.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
b5a5778589
Distributed: Add ability to limit amount of pending bytes for async INSERT
...
Right now with distributed_directory_monitor_batch_inserts=1 and
insert_distributed_sync=0 INSERT into Distributed table will store
blocks that should be sent to remote (and in case of
prefer_localhost_replica=0 to the localhost too) on the local
filesystem, and sent it in background.
However there is no limit for this storage, and if the remote is
unavailable (or some other error), these pending blocks may take
significant space, and this is not always desired behaviour.
Add new Distributed setting - bytes_to_throw_insert, that will set the
limit for how much pending bytes is allowed, if the limit will be
reached an exception will be throw.
By default was set to 0, to avoid surprises.
2021-03-03 23:30:00 +03:00
Azat Khuzhin
ce09b7ff89
Distributed: Implement totalBytes() (system.tables.total_bytes)
2021-03-03 23:29:11 +03:00
Azat Khuzhin
456cbaf747
Distributed: Hide private part of the interface
2021-03-03 23:29:11 +03:00
alesapin
6b1005aea5
Merge pull request #21334 from ClickHouse/fix_alter_partition_key
...
Fix alter modify query for partition key and other metadata fields
2021-03-03 10:15:11 +03:00
Anton Popov
a4c00ab5dc
Merge pull request #21303 from ucasFL/forbid
...
Forbid to drop a column if it's referenced by materialized view
2021-03-03 02:55:06 +03:00
Kruglov Pavel
fbe0a14a2f
Merge pull request #21035 from anmolarora/system-parts-requested-columns-fill
...
Fill only requested columns when querying system.parts & system.parts…
2021-03-03 00:39:45 +03:00
alesapin
c29d7c7f49
Shutup clang tidy
2021-03-02 19:13:36 +03:00
alesapin
4e33587043
Comments
2021-03-02 13:57:09 +03:00
Nikolai Kochetov
ffbea93b10
Merge pull request #19576 from traceon/row-policy-with-prewhere
...
Allow row policies with PREWHERE
2021-03-02 13:51:37 +03:00
alesapin
06678d650d
Merge branch 'master' into fix_alter_partition_key
2021-03-02 13:43:41 +03:00
alesapin
9ebf1b4fad
Get rid of separate minmax index fields
2021-03-02 13:33:54 +03:00
Pavel Kruglov
780a65e3a0
Merge branch 'master' of github.com:ClickHouse/ClickHouse into system-parts-requested-columns-fill
2021-03-02 10:29:26 +03:00
feng lv
a26c9e64a9
fix
...
fix
2021-03-02 03:20:03 +00:00
Kruglov Pavel
caaa0c5e86
Restart tests.
2021-03-01 13:25:20 +03:00
alesapin
9c8afbeb53
Fix alter modify query for partition key and other metadata fields
2021-03-01 12:59:19 +03:00
Nikolai Kochetov
976dbe8077
Merge pull request #20341 from ClickHouse/filter-push-down
...
Filter push down
2021-03-01 12:35:06 +03:00
alexey-milovidov
174f49a9f1
Merge pull request #21308 from fuwhu/refine-wal-restore-code
...
remove unused code in MergeTreeWriteAheadLog::restore
2021-03-01 01:44:23 +03:00
Denis Glazachev
eb80c189e4
Merge branch 'master' into row-policy-with-prewhere
2021-02-28 20:14:31 +04:00
fuwhu
2df33be7c2
Remove unused code in MergeTreeWriteAheadLog::restore
2021-02-28 23:17:31 +08:00
Azat Khuzhin
8b58dba094
Drop unused HexWriteBuffer
2021-02-28 10:52:09 +03:00
feng lv
51021c1164
forbid to drop a column if it's referenced by materialized view
2021-02-28 05:24:39 +00:00
Anmol Arora
2d9b524bdd
Fill only requested columns when querying system.parts & system.parts_columns
2021-02-28 01:58:38 +00:00
alexey-milovidov
b8fba768e5
Merge pull request #21264 from ClickHouse/fix_zookeeper_update
...
Fix several bugs with ZooKeeper client
2021-02-28 01:57:04 +03:00
alesapin
9e93d7f507
Fix tidy and add comments
2021-02-27 11:07:14 +03:00
Nikolai Kochetov
d91b8a3acb
Merge branch 'master' into filter-push-down
2021-02-26 19:33:27 +03:00
Nikolai Kochetov
d328bfa41f
Review fixes. Add setting max_optimizations_to_apply.
2021-02-26 19:29:56 +03:00
tavplubix
09f62447af
Merge pull request #21060 from mxzlxy/fix-bug-for-replicated-engine-parameter
...
fix default_replica_path and default_replica_name values are useless …
2021-02-26 17:18:56 +03:00
Vladimir
78024ae2d6
Merge pull request #21009 from vdimir/fix-race-storage-join
2021-02-26 14:26:10 +03:00
Maksim Kita
749dd8d555
Merge pull request #21193 from kitaisreal/storage-rabbit-mq-added-uvloop
...
StorageRabbitMQ added UVLoop
2021-02-26 11:04:02 +03:00
alesapin
5b72cd231f
Merge pull request #21183 from ClickHouse/fix_mutation_for_storages
...
Restrict mutations for engines which doesn't support them
2021-02-26 09:54:09 +03:00
kssenii
1247c0f36c
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into rabbit-fix
2021-02-25 16:49:29 +00:00
vdimir
9c1ba71416
Merge remote-tracking branch 'upstream/master' into fix-race-storage-join
2021-02-25 19:13:49 +03:00
alesapin
653d962cdd
Fix style and add low-level check
2021-02-25 17:43:58 +03:00
Maksim Kita
29e783999c
Fixed typo
2021-02-25 17:43:32 +03:00
Maksim Kita
f077a25a7a
StorageRabbitMQ added UVLoop
2021-02-25 17:02:34 +03:00
vdimir
6f864ddb42
Minor changes in Hash/StorageJoin, add TableLockHolder to ExecutableFunctionJoinGet
2021-02-25 14:21:06 +03:00
alesapin
5c6c318737
Restrict mutations for engines which doesn't support them
2021-02-25 13:07:48 +03:00
vdimir
2f70e895fc
Update StorageJoin locking
...
Move joinGet into StorageJoin
Protect JoinSource with lock, add test
Add comments about locking logic
2021-02-25 12:31:22 +03:00
fastio
fea2836673
get zookeeper from global context
2021-02-25 14:19:58 +08:00
