Commit Graph

327 Commits

Author SHA1 Message Date
alexey-milovidov
c9334d3fde
Merge pull request #10491 from azat/dist-shutdown
Proper Distributed shutdown (fixes UAF, avoid waiting for sending all batches)
2020-04-25 23:47:59 +03:00
Azat Khuzhin
747a74215f Avoid processing all batches before Distributed shutdown 2020-04-25 02:03:27 +03:00
Azat Khuzhin
8ad6b37913 Proper StorageDistributed shutdown to avoid UAF in DistributedMonitor
StorageDistributed::shutdown() does not acquire the lock, that controls
access to the cluster_nodes_data, thus it does not synced with the
requireDirectoryMonitor(), hence some monitors can be untracked that
will trigger UAF (use-after-free) after DROP TABLE dist:

This is for the SIGSEGV from the DirectoryMonitor (with already destroyed storage):
    0  0x0000000008e9f760 in std::__1::__cxx_atomic_load<int> (__order=std::__1::memory_order::seq_cst, __a=0x0)
    1  std::__1::__atomic_base<int, false>::load (__m=std::__1::memory_order::seq_cst, this=0x0) <-- this is nullptr
    2  std::__1::__atomic_base<int, false>::operator int (this=0x0)
    3  DB::ActionBlocker::isCancelled (this=0x7f85e31c9bb8) at ../src/Common/ActionBlocker.h:18
    4  DB::StorageDistributedDirectoryMonitor::run (this=0x7f85f93b2a00) at ../src/Storages/Distributed/DirectoryMonitor.cpp:140
2020-04-25 02:03:26 +03:00
alesapin
793f4b734a Remove obsolete comment 2020-04-24 13:31:03 +03:00
alesapin
dc2dd77d2e Remove redundant overrides from IStorage 2020-04-24 12:20:09 +03:00
Alexander Tokmakov
04d6b59ac0 Merge branch 'master' into database_atomic 2020-04-23 17:31:37 +03:00
Alexey Milovidov
1e325a9fd9 Checkpoint 2020-04-22 09:22:14 +03:00
Alexander Tokmakov
b29bddac12 Merge branch 'master' into database_atomic 2020-04-20 14:09:09 +03:00
Alexey Milovidov
d7264b292d Merge branch 'master' into sorting-processors 2020-04-20 09:29:41 +03:00
alexey-milovidov
1577d771df
Merge pull request #10341 from azat/auto_distributed_group_by_no_merge
Auto distributed_group_by_no_merge on GROUP BY sharding key
2020-04-20 08:30:27 +03:00
Azat Khuzhin
e44d5c5749 Fix clang readability-container-size-empty warning in StorageDistributed::canForceGroupByNoMerge() 2020-04-20 01:12:22 +03:00
Azat Khuzhin
be1dec9239 Fix distributed_group_by_no_merge optimization for Distributed-over-Distributed 2020-04-19 21:11:14 +03:00
Azat Khuzhin
93d049fe64 Allow auto distributed_group_by_no_merge for DISTINCT of sharding key 2020-04-19 18:53:37 +03:00
Azat Khuzhin
de4a723264 Auto distributed_group_by_no_merge on GROUP BY injective function of sharding key 2020-04-19 18:33:49 +03:00
Azat Khuzhin
5d11118cc9 Use thread pool (background_distributed_schedule_pool_size) for distributed sends
After #8756 the problem with 1 thread for each (distributed table, disk)
for distributed sends became even worse (since there can be multiple
disks), so use predefined thread pool for this tasks, that can be
controlled with background_distributed_schedule_pool_size knob.
2020-04-19 12:01:56 +03:00
Nikolai Kochetov
153f795ebe Merge branch 'master' into sorting-processors 2020-04-15 12:07:05 +03:00
Alexander Tokmakov
5e6d4b9449 Merge branch 'master' into database_atomic 2020-04-12 16:35:44 +03:00
Nikolai Kochetov
71c72a75d7 Move Graphite params to separate file. 2020-04-10 12:24:16 +03:00
Alexander Kazakov
497df3086f Merge branch 'master' into timed_rwlock
Change-Id: I620bfde2121ff013773b001d514b40b1e796a58b
2020-04-10 11:38:20 +03:00
Alexander Kazakov
26dd6140b2 Added new config settings to control timeouts
* "lock_acquire_timeout" controls for how long a query will continue to
acquire each lock on its argument tables
 * "lock_acquire_timeout_for_background_operations" is a per-table
setting for storages of *MergeTree family
2020-04-09 21:10:27 +03:00
Alexander Tokmakov
dd1590830b Merge branch 'master' into database_atomic 2020-04-08 22:00:46 +03:00
alexey-milovidov
6d80ab1eed
Merge pull request #9811 from vitlibar/RBAC-8
RBAC-8
2020-04-08 05:47:55 +03:00
Vitaly Baranov
e573549945 Rework access rights for table functions. 2020-04-07 23:31:59 +03:00
Alexander Tokmakov
4c48b7dd80 better rename 2020-04-07 18:31:33 +03:00
Alexander Tokmakov
08bae4668d Merge branch 'master' into database_atomic 2020-04-06 16:18:07 +03:00
Azat Khuzhin
1232760f78 Fix Distributed-over-Distributed when nested table has only one shard 2020-04-04 13:47:35 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00