Commit Graph

227 Commits

Author SHA1 Message Date
Anton Popov
cbbd8e521c fix usage of primary key wrapped into a function with 'FINAL' modifier and 'ORDER BY' optimization 2020-05-07 01:27:35 +03:00
alexey-milovidov
114f09cfca
Merge pull request #10666 from NanoBjorn/refactor-volume
Volumes and storages refactoring
2020-05-05 16:25:26 +03:00
alexey-milovidov
c54f6d5a66
Merge pull request #10546 from kyprizel/master
Lexer, ast, checksum and column specs fuzzers added
2020-05-05 14:50:55 +03:00
alexey-milovidov
09756a036a
Merge pull request #10645 from ClickHouse/check-suspicious-codecs
Check for suspicious codecs #4966
2020-05-04 23:33:59 +03:00
Gleb Novikov
c637d99e07 Volumes and storages refactoring:
1. Moved Volume to separate file
  2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
  3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
  4. Removed old Volume completely
  5. Moved StoragePolicy to separated files
  6. Moved DiskSelector to separated files
  7. Removed DiskSpaceMonitor file
2020-05-04 23:15:38 +03:00
alesapin
d47d28d51f Fix mutations assignment 2020-05-04 19:30:39 +03:00
Alexey Milovidov
99c18c5a09 Check for suspicious codecs #4966 2020-05-04 03:11:49 +03:00
alexey-milovidov
884c2aa631
Merge pull request #10641 from ClickHouse/storage-buffer-nullptr-dereference
Fix nullptr dereference in StorageBuffer
2020-05-04 02:01:57 +03:00
Alexey Milovidov
a5bdc375ca Fix Arcadia #10569 2020-05-04 00:36:11 +03:00
Alexey Milovidov
f8866af0c1 Fix nullptr dereference in StorageBuffer 2020-05-03 23:53:54 +03:00
alexey-milovidov
db4c2352e1
Merge pull request #10569 from zhang2014/fix/ISSUES-10551
ISSUES-10551 add backward compatibility for create bloom filter index
2020-05-02 22:25:29 +03:00
alexey-milovidov
229f666dea
Merge pull request #10611 from azat/optimize_skip_unused_shards-LowCardinality
Fix optimize_skip_unused_shards with LowCardinality
2020-05-02 22:14:33 +03:00
alexey-milovidov
4f1b115e76
Merge pull request #10610 from azat/optimize_skip_unused_shards-fixes
Fix logging for optimize_skip_unused_shards
2020-05-02 22:13:01 +03:00
tavplubix
a073cd6c79
Fix segfault in StorageBuffer when exception on server startup (#10609)
* fix segfault in StorageBuffer when exception on server startup

* Trigger CI

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
2020-05-02 19:54:20 +03:00
Alexey Milovidov
81adcc9058 Remove obsolete records from system.build_options 2020-05-02 17:00:03 +03:00
alexey-milovidov
404452bad3
Update MergeTreeIndices.cpp 2020-05-01 21:40:56 +03:00
alesapin
4d8660f270 Remove obsolete comment 2020-05-01 13:05:37 +03:00
Azat Khuzhin
63d8ab8f03 Make createSelector() static (in storage) and const (in stream) 2020-05-01 11:31:05 +03:00
Azat Khuzhin
f22ba15b4a Reduce copy-paste of DistributedBlockOutputStream::createSelector
This will make it less error prone.
2020-05-01 02:59:40 +03:00
Azat Khuzhin
cdd7013438 Drop superfluous "Skipping irrelevant shards" messages
Before this patch it printed 3 times:
- from StorageDistributed::getProcessingStageImpl()
- from StorageDistributed::read()
- from StorageDistributed::getProcessingStageImpl() (from StorageDistributed::read() -> getSampleBlock())
(But this should be optimized)
2020-05-01 02:56:13 +03:00
Azat Khuzhin
c648c300bf Fix optimize_skip_unused_shards with LowCardinality 2020-05-01 02:39:58 +03:00
Azat Khuzhin
4cbe625567 Fix shard numbers output in logs (full cluster had been printed over optimized) 2020-05-01 02:13:07 +03:00
alesapin
15e8f37839
Merge pull request #10531 from CurtizJ/polymorphic-parts-2
Fix index corruption in merges with compact parts.
2020-04-30 12:15:25 +03:00
alesapin
5e2afad549
Merge pull request #10477 from ClickHouse/refactor_istorage
Remove getColumn and hasColumn from IStorage, better virtuals.
2020-04-30 10:42:52 +03:00
Anton Popov
6ed3092c8a make fillIndexGranularity less complicated 2020-04-30 00:57:58 +03:00
alesapin
69fdf2a6a3 Review fixes and better test 2020-04-29 15:15:23 +03:00
zhang2014
8ac709b5d3 ISSUES-10551 add backward compatibility for create bloom filter index 2020-04-29 11:57:28 +08:00
robot-clickhouse
de4e88ff97 Auto version update to [20.5.1.1] [54435] 2020-04-28 20:12:16 +03:00
alesapin
f981649213 Fix pushing to views stream and refactor virtuals 2020-04-28 13:38:57 +03:00
Eldar Zaitov
670ed4bf1b lexer, ast, checksum and column specs fuzzers added 2020-04-28 02:12:18 +03:00
Anton Popov
032a0aa282 fix clang-tidy 2020-04-27 21:12:17 +03:00
alesapin
7c2f4d121a Fix func 2020-04-27 20:48:53 +03:00
alesapin
4badd0fd28 Better code 2020-04-27 20:46:51 +03:00
alesapin
228dbf475e
Finalize mutations in background (#10526) 2020-04-27 19:19:04 +03:00
alesapin
01db4877f6 Fix style check 2020-04-27 18:44:33 +03:00
alesapin
c24ab563d7 Fix several wrong usages 2020-04-27 18:38:35 +03:00
alesapin
b0c5c1d685 Fix style 2020-04-27 18:21:37 +03:00
alesapin
6cfe0f7046 Fix after refactoring 2020-04-27 18:20:03 +03:00
alesapin
cefc58e77c Fix style 2020-04-27 18:17:08 +03:00
alesapin
5c6316afbc Fix bug 2020-04-27 18:10:50 +03:00
alesapin
4c1ac83112 Fix in IStorage 2020-04-27 17:44:32 +03:00
alesapin
e85b50c911 Revert "Strange fix, how it was possible earlier?"
This reverts commit 8d20644c99.
2020-04-27 17:17:12 +03:00
alesapin
8d20644c99 Strange fix, how it was possible earlier? 2020-04-27 17:12:54 +03:00
alesapin
18c550df15 Better virtuals logic 2020-04-27 16:55:30 +03:00
Anton Popov
bee343dae1 fix writing of index in compact parts 2020-04-27 16:36:50 +03:00
alesapin
2829774105 Merge branch 'master' into refactor_istorage 2020-04-27 15:34:21 +03:00
alexey-milovidov
c9334d3fde
Merge pull request #10491 from azat/dist-shutdown
Proper Distributed shutdown (fixes UAF, avoid waiting for sending all batches)
2020-04-25 23:47:59 +03:00
Azat Khuzhin
747a74215f Avoid processing all batches before Distributed shutdown 2020-04-25 02:03:27 +03:00
Azat Khuzhin
8ad6b37913 Proper StorageDistributed shutdown to avoid UAF in DistributedMonitor
StorageDistributed::shutdown() does not acquire the lock, that controls
access to the cluster_nodes_data, thus it does not synced with the
requireDirectoryMonitor(), hence some monitors can be untracked that
will trigger UAF (use-after-free) after DROP TABLE dist:

This is for the SIGSEGV from the DirectoryMonitor (with already destroyed storage):
    0  0x0000000008e9f760 in std::__1::__cxx_atomic_load<int> (__order=std::__1::memory_order::seq_cst, __a=0x0)
    1  std::__1::__atomic_base<int, false>::load (__m=std::__1::memory_order::seq_cst, this=0x0) <-- this is nullptr
    2  std::__1::__atomic_base<int, false>::operator int (this=0x0)
    3  DB::ActionBlocker::isCancelled (this=0x7f85e31c9bb8) at ../src/Common/ActionBlocker.h:18
    4  DB::StorageDistributedDirectoryMonitor::run (this=0x7f85f93b2a00) at ../src/Storages/Distributed/DirectoryMonitor.cpp:140
2020-04-25 02:03:26 +03:00
tavplubix
7928f2eaa1
Merge pull request #7512 from ClickHouse/database_atomic
DatabaseAtomic
2020-04-24 22:28:18 +03:00