thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
40,354 Commits 9,328 Branches 1,783 Tags 2.1 GiB
4151cf7ac8
Commit Graph

295 Commits

Author SHA1 Message Date
Nikolai Kochetov
c22f7a0864 Try fix header for GenerateRandom. 2020-05-07 17:14:56 +03:00
Vitaliy Zakaznikov
90e52e7fea Adding support for ALTER RENAME COLUMN query to Distributed table engine. 2020-05-07 14:54:35 +02:00
Anton Popov
cbbd8e521c fix usage of primary key wrapped into a function with 'FINAL' modifier and 'ORDER BY' optimization 2020-05-07 01:27:35 +03:00
alexey-milovidov
4df7dd8c2b
Merge pull request #10656 from azat/kafka-consumer-hang 
Kafka cleanup
 2020-05-07 00:20:31 +03:00
bobrovskij artemij
a2916c23a5 fixes/style 2020-05-06 02:42:44 +03:00
alexey-milovidov
114f09cfca
Merge pull request #10666 from NanoBjorn/refactor-volume 
Volumes and storages refactoring
 2020-05-05 16:25:26 +03:00
alexey-milovidov
c54f6d5a66
Merge pull request #10546 from kyprizel/master 
Lexer, ast, checksum and column specs fuzzers added
 2020-05-05 14:50:55 +03:00
alexey-milovidov
09756a036a
Merge pull request #10645 from ClickHouse/check-suspicious-codecs 
Check for suspicious codecs #4966
 2020-05-04 23:33:59 +03:00
Gleb Novikov
c637d99e07 Volumes and storages refactoring: 
1. Moved Volume to separate file
  2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
  3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
  4. Removed old Volume completely
  5. Moved StoragePolicy to separated files
  6. Moved DiskSelector to separated files
  7. Removed DiskSpaceMonitor file
 2020-05-04 23:15:38 +03:00
alesapin
d47d28d51f Fix mutations assignment 2020-05-04 19:30:39 +03:00
Azat Khuzhin
61c5df5662 Drop claimed from the KafkaBlockInputStream 2020-05-04 17:23:09 +03:00
Azat Khuzhin
a10cfc3ae5 Pass stream_cancelled to the copyData as stub in StorageKafka 2020-05-04 17:23:09 +03:00
Azat Khuzhin
c69217e9f3 Force destrying of kafka buffers from the shutdown in StorageKafka 2020-05-04 17:23:09 +03:00
Azat Khuzhin
9e14db2091 Add missing kafka settings into docs 2020-05-04 17:23:05 +03:00
Alexey Milovidov
99c18c5a09 Check for suspicious codecs #4966 2020-05-04 03:11:49 +03:00
alexey-milovidov
884c2aa631
Merge pull request #10641 from ClickHouse/storage-buffer-nullptr-dereference 
Fix nullptr dereference in StorageBuffer
 2020-05-04 02:01:57 +03:00
Alexey Milovidov
a5bdc375ca Fix Arcadia #10569 2020-05-04 00:36:11 +03:00
Alexey Milovidov
f8866af0c1 Fix nullptr dereference in StorageBuffer 2020-05-03 23:53:54 +03:00
Azat Khuzhin
6ffdd53b6a Share auto-increment for first batch and tmp file in DistributedBlockOutputStream 2020-05-03 14:47:59 +03:00
Azat Khuzhin
53c470cab4 Fix directory monitor initialization from INSERT into Distributed 
This also fixes hardlink code (when one file should be sent to multiple
servers, i.e. internal_replication == false) of writeToShard() with
distributed_storage_policy (i.e. when StorageDistributed::getPath() will
path to different filesystems).

Plus also cleanup DistributedBlockOutputStream::writeToShard() a little.
 2020-05-03 14:47:51 +03:00
Azat Khuzhin
e97e1f06db Do not schedule distributed send if there were no error 
Since in this case it will be scheduled from the
DistributedBlockOutputStream with the
distributed_directory_monitor_max_sleep_time_ms, and this will overwrite
timer that was set by the DistributedBlockOutputStream, not good.
 2020-05-03 14:46:44 +03:00
Azat Khuzhin
947b3942dd Schedule distributed sends after the file has been written 2020-05-03 14:46:43 +03:00
Azat Khuzhin
0157fd5d93 Fix distributed send that are scheduled by INSERT query 
Before this patch each INSERT query re-schedule distributed send, thus
each time it resets the timer, while this is not the expected behaviour,
since in on frequent INSERT distributed sends will not be triggered at
all.

Fix this by not resetting the timer.
 2020-05-03 14:46:42 +03:00
Azat Khuzhin
6bb39dafc3 Drop decreated code (cond var and note for thread) in DirectoryMonitor 2020-05-03 14:46:41 +03:00
alexey-milovidov
db4c2352e1
Merge pull request #10569 from zhang2014/fix/ISSUES-10551 
ISSUES-10551 add backward compatibility for create bloom filter index
 2020-05-02 22:25:29 +03:00
alexey-milovidov
229f666dea
Merge pull request #10611 from azat/optimize_skip_unused_shards-LowCardinality 
Fix optimize_skip_unused_shards with LowCardinality
 2020-05-02 22:14:33 +03:00
alexey-milovidov
4f1b115e76
Merge pull request #10610 from azat/optimize_skip_unused_shards-fixes 
Fix logging for optimize_skip_unused_shards
 2020-05-02 22:13:01 +03:00
tavplubix
a073cd6c79
Fix segfault in StorageBuffer when exception on server startup (#10609) 
* fix segfault in StorageBuffer when exception on server startup

* Trigger CI

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-05-02 19:54:20 +03:00
Alexey Milovidov
81adcc9058 Remove obsolete records from system.build_options 2020-05-02 17:00:03 +03:00
alexey-milovidov
404452bad3
Update MergeTreeIndices.cpp 2020-05-01 21:40:56 +03:00
alesapin
4d8660f270 Remove obsolete comment 2020-05-01 13:05:37 +03:00
Azat Khuzhin
63d8ab8f03 Make createSelector() static (in storage) and const (in stream) 2020-05-01 11:31:05 +03:00
Azat Khuzhin
f22ba15b4a Reduce copy-paste of DistributedBlockOutputStream::createSelector 
This will make it less error prone.
 2020-05-01 02:59:40 +03:00
Azat Khuzhin
cdd7013438 Drop superfluous "Skipping irrelevant shards" messages 
Before this patch it printed 3 times:
- from StorageDistributed::getProcessingStageImpl()
- from StorageDistributed::read()
- from StorageDistributed::getProcessingStageImpl() (from StorageDistributed::read() -> getSampleBlock())
(But this should be optimized)
 2020-05-01 02:56:13 +03:00
Azat Khuzhin
c648c300bf Fix optimize_skip_unused_shards with LowCardinality 2020-05-01 02:39:58 +03:00
Azat Khuzhin
4cbe625567 Fix shard numbers output in logs (full cluster had been printed over optimized) 2020-05-01 02:13:07 +03:00
alesapin
15e8f37839
Merge pull request #10531 from CurtizJ/polymorphic-parts-2 
Fix index corruption in merges with compact parts.
 2020-04-30 12:15:25 +03:00
alesapin
5e2afad549
Merge pull request #10477 from ClickHouse/refactor_istorage 
Remove getColumn and hasColumn from IStorage, better virtuals.
 2020-04-30 10:42:52 +03:00
Anton Popov
6ed3092c8a make fillIndexGranularity less complicated 2020-04-30 00:57:58 +03:00
alesapin
69fdf2a6a3 Review fixes and better test 2020-04-29 15:15:23 +03:00
zhang2014
8ac709b5d3 ISSUES-10551 add backward compatibility for create bloom filter index 2020-04-29 11:57:28 +08:00
Azat Khuzhin
038235684d Add optimize_distributed_group_by_sharding_key and disable it by default 
I know at least one way to fool that optimization, by using as sharding
key something like `if(col1>0, col1, col2)` (although this is not common
sharding key I would say, but can be useful if this will work
correctly), so let's disable it by default.
 2020-04-29 00:09:25 +03:00
robot-clickhouse
de4e88ff97 Auto version update to [20.5.1.1] [54435] 2020-04-28 20:12:16 +03:00
bobrovskij artemij
0cc3d780b3 Support write for XDBCStorage 2020-04-28 14:03:24 +03:00
alesapin
f981649213 Fix pushing to views stream and refactor virtuals 2020-04-28 13:38:57 +03:00
bobrovskij artemij
1bcd077fbf Support write for XDBCStorage 2020-04-28 03:56:44 +03:00
Eldar Zaitov
670ed4bf1b lexer, ast, checksum and column specs fuzzers added 2020-04-28 02:12:18 +03:00
Anton Popov
032a0aa282 fix clang-tidy 2020-04-27 21:12:17 +03:00
alesapin
7c2f4d121a Fix func 2020-04-27 20:48:53 +03:00
alesapin
4badd0fd28 Better code 2020-04-27 20:46:51 +03:00
alesapin
228dbf475e
Finalize mutations in background (#10526) 2020-04-27 19:19:04 +03:00
alesapin
01db4877f6 Fix style check 2020-04-27 18:44:33 +03:00
alesapin
c24ab563d7 Fix several wrong usages 2020-04-27 18:38:35 +03:00
alesapin
b0c5c1d685 Fix style 2020-04-27 18:21:37 +03:00
alesapin
6cfe0f7046 Fix after refactoring 2020-04-27 18:20:03 +03:00
alesapin
cefc58e77c Fix style 2020-04-27 18:17:08 +03:00
alesapin
5c6316afbc Fix bug 2020-04-27 18:10:50 +03:00
alesapin
4c1ac83112 Fix in IStorage 2020-04-27 17:44:32 +03:00
alesapin
e85b50c911 Revert "Strange fix, how it was possible earlier?" 
This reverts commit 8d20644c99.
 2020-04-27 17:17:12 +03:00
alesapin
8d20644c99 Strange fix, how it was possible earlier? 2020-04-27 17:12:54 +03:00
alesapin
18c550df15 Better virtuals logic 2020-04-27 16:55:30 +03:00
Anton Popov
bee343dae1 fix writing of index in compact parts 2020-04-27 16:36:50 +03:00
alesapin
2829774105 Merge branch 'master' into refactor_istorage 2020-04-27 15:34:21 +03:00
Azat Khuzhin
20b4eed9a1 Disable GROUP BY sharding_key optimization for WITH ROLLUP/CUBE/TOTALS 2020-04-27 01:30:54 +03:00
alexey-milovidov
c9334d3fde
Merge pull request #10491 from azat/dist-shutdown 
Proper Distributed shutdown (fixes UAF, avoid waiting for sending all batches)
 2020-04-25 23:47:59 +03:00
Azat Khuzhin
747a74215f Avoid processing all batches before Distributed shutdown 2020-04-25 02:03:27 +03:00
Azat Khuzhin
8ad6b37913 Proper StorageDistributed shutdown to avoid UAF in DistributedMonitor 
StorageDistributed::shutdown() does not acquire the lock, that controls
access to the cluster_nodes_data, thus it does not synced with the
requireDirectoryMonitor(), hence some monitors can be untracked that
will trigger UAF (use-after-free) after DROP TABLE dist:

This is for the SIGSEGV from the DirectoryMonitor (with already destroyed storage):
    0  0x0000000008e9f760 in std::__1::__cxx_atomic_load<int> (__order=std::__1::memory_order::seq_cst, __a=0x0)
    1  std::__1::__atomic_base<int, false>::load (__m=std::__1::memory_order::seq_cst, this=0x0) <-- this is nullptr
    2  std::__1::__atomic_base<int, false>::operator int (this=0x0)
    3  DB::ActionBlocker::isCancelled (this=0x7f85e31c9bb8) at ../src/Common/ActionBlocker.h:18
    4  DB::StorageDistributedDirectoryMonitor::run (this=0x7f85f93b2a00) at ../src/Storages/Distributed/DirectoryMonitor.cpp:140
 2020-04-25 02:03:26 +03:00
tavplubix
7928f2eaa1
Merge pull request #7512 from ClickHouse/database_atomic 
DatabaseAtomic
 2020-04-24 22:28:18 +03:00
alesapin
793f4b734a Remove obsolete comment 2020-04-24 13:31:03 +03:00
alesapin
b4425c2dc4 Remove getColumn and hasColumn methods from IStorage 2020-04-24 13:20:03 +03:00
alesapin
0c622370c5 Better 2020-04-24 12:21:38 +03:00
alesapin
dc2dd77d2e Remove redundant overrides from IStorage 2020-04-24 12:20:09 +03:00
Alexander Tokmakov
f413061e0d remove Context from some IDatabase methods 2020-04-23 19:51:48 +03:00
alesapin
5ae25b68e4 Remove redundant virtual from function 2020-04-23 18:18:33 +03:00
alesapin
b0a73589b0 Remove virtual columns as separate field from IStorage 2020-04-23 18:09:00 +03:00
Alexander Tokmakov
04d6b59ac0 Merge branch 'master' into database_atomic 2020-04-23 17:31:37 +03:00
Nikita Mikhaylov
3520755ab5 fix c++20 usage 2020-04-23 12:00:43 +03:00
alexey-milovidov
4f6bb00424
Merge pull request #10420 from ClickHouse/clang-tidy-8 
Fixes for clang-tidy on clang-10
 2020-04-23 00:33:41 +03:00
Nikita Mikhaylov
3cf2016867
Merge pull request #10188 from nikitamikhaylov/quorum_bug 
insert_quorum vs drop partition or ttl
 2020-04-23 00:11:57 +04:00
Alexey Milovidov
73dd78bac1 Merge branch 'master' into clang-tidy-8 2020-04-22 21:04:47 +03:00
Alexander Tokmakov
fefbbd37df Merge branch 'master' into database_atomic 2020-04-22 17:02:30 +03:00
zhang2014
0a0555d4d7 ISSUES-9623 fix failure test case 2020-04-22 18:34:42 +08:00
zhang2014
bfb87f0eab ISSUES-9623 check the number and type of arguments when creating BloomFilter index 2020-04-22 16:17:09 +08:00
Alexey Milovidov
03e17405cd Checkpoint 2020-04-22 10:03:43 +03:00
Alexey Milovidov
603d132d45 Checkpoint 2020-04-22 09:34:20 +03:00
Alexey Milovidov
1e325a9fd9 Checkpoint 2020-04-22 09:22:14 +03:00
Alexey Milovidov
be22a4b94e Checkpoint 2020-04-22 08:39:31 +03:00
Ivan
555488a458
Fix build in Arcadia (#10413) 2020-04-21 23:23:44 +03:00
Nikita Mikhaylov
2173bcf28e Merge branch 'master' of github.com:ClickHouse/ClickHouse into quorum_bug 2020-04-21 14:35:06 +03:00
Alexey Milovidov
4cc0269de0 Fix flattening of Array(Tuple(...)) #10259 2020-04-21 02:44:51 +03:00
alexey-milovidov
8c2839d3c9
Merge pull request #10308 from ClickHouse/shared-context-lifetime 
Shared context lifetime
 2020-04-21 00:12:45 +03:00
alexey-milovidov
ab922d44d5
Merge pull request #10379 from ClickHouse/only_fetch_repliac 
Add setting always_fetch_merged_part
 2020-04-21 00:07:00 +03:00
alexey-milovidov
69b6f135e7
Merge pull request #10382 from excitoon-favorites/altercodec 
Allowed to alter column in non-modifying data mode when the same type is specified
 2020-04-20 22:54:53 +03:00
alexey-milovidov
f6de9c6ffd
Update AlterCommands.cpp 2020-04-20 22:54:08 +03:00
alexey-milovidov
1eca5088ec
Update AlterCommands.cpp 2020-04-20 22:53:19 +03:00
Vladimir Chebotarev
82985ff594 Allowed to alter column in non-modifying data mode when the same type is specified. 2020-04-20 14:15:06 +03:00
Alexander Tokmakov
b29bddac12 Merge branch 'master' into database_atomic 2020-04-20 14:09:09 +03:00
Nikita Mikhaylov
05206df88c better 2020-04-20 13:56:59 +03:00
alesapin
1bc84ec8b4 Fix comments 2020-04-20 13:56:23 +03:00
alesapin
8767ea7efa Add setting which restrict replica to merge parts by itself and always prefer dowloading from other replicas. 2020-04-20 13:53:29 +03:00