thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 11:22:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
40,253 Commits 9,338 Branches 1,783 Tags 2.1 GiB
485d37b043
Commit Graph

1007 Commits

Author SHA1 Message Date
alexey-milovidov
09756a036a
Merge pull request #10645 from ClickHouse/check-suspicious-codecs 
Check for suspicious codecs #4966
 2020-05-04 23:33:59 +03:00
Maxim Serebryakov
3691400a60 fix order of params in AggregatingTransform 2020-05-04 23:26:33 +03:00
Gleb Novikov
6e6a195c13 Style fixes, removed redundant includes 2020-05-04 23:20:51 +03:00
Gleb Novikov
c637d99e07 Volumes and storages refactoring: 
1. Moved Volume to separate file
  2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
  3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
  4. Removed old Volume completely
  5. Moved StoragePolicy to separated files
  6. Moved DiskSelector to separated files
  7. Removed DiskSpaceMonitor file
 2020-05-04 23:15:38 +03:00
Nikolai Kochetov
b23b02094f Added comments. 2020-05-04 22:17:52 +03:00
Nikolai Kochetov
ee65f3eedc Init all input ports in GroupingAggregatedTransform. 2020-05-04 21:07:56 +03:00
Nikolai Kochetov
4e27c33451 Init all input ports in GroupingAggregatedTransform. 2020-05-04 20:59:07 +03:00
Nikolai Kochetov
97dde2bcdf Fix limit for MergingSorted with fast-forward optimization. 2020-05-04 20:16:16 +03:00
alesapin
d47d28d51f Fix mutations assignment 2020-05-04 19:30:39 +03:00
FawnD2
f0201f7aca Move ORC to Input only formats 2020-05-04 18:04:20 +03:00
Alexey Milovidov
a198dce505 Fix double space 2020-05-04 17:55:37 +03:00
FawnD2
2f8435813c Remove unused stream 2020-05-04 17:49:13 +03:00
FawnD2
da0a908d27 Remove unused stream 2020-05-04 17:47:09 +03:00
Alexey Milovidov
87fb6bf081 Better exception message 2020-05-04 17:46:32 +03:00
FawnD2
b492e92d41 Remove extra check 2020-05-04 17:32:36 +03:00
FawnD2
182de47012 Use file reader for Arrow format 2020-05-04 17:30:42 +03:00
Alexey Milovidov
301c6137e4 Better exception messages; fixed error; updated tests 2020-05-04 17:25:54 +03:00
Azat Khuzhin
61c5df5662 Drop claimed from the KafkaBlockInputStream 2020-05-04 17:23:09 +03:00
Azat Khuzhin
a10cfc3ae5 Pass stream_cancelled to the copyData as stub in StorageKafka 2020-05-04 17:23:09 +03:00
Azat Khuzhin
c69217e9f3 Force destrying of kafka buffers from the shutdown in StorageKafka 2020-05-04 17:23:09 +03:00
Azat Khuzhin
9e14db2091 Add missing kafka settings into docs 2020-05-04 17:23:05 +03:00
Anton Okhitin
7366227115 Fix arcadia build 2020-05-04 16:47:08 +03:00
FawnD2
6d6d04903f Remove check 2020-05-04 16:33:20 +03:00
FawnD2
a044766d58 Remove check 2020-05-04 16:26:20 +03:00
FawnD2
17450811d4 Move getHeader at the beginning of generate() 2020-05-04 16:19:25 +03:00
FawnD2
83148289d8 Rename arg 2020-05-04 16:01:37 +03:00
Артем Стрельцов
0f23713488 fixed bug with dictIsIn 2020-05-04 12:53:06 +03:00
alexey-milovidov
443c64abc0
Merge pull request #10638 from ClickHouse/fix-ubsan-hashtable 
Fix UBSan report in HashTable
 2020-05-04 04:50:25 +03:00
Артем Стрельцов
05d4e41949 minor fix 2020-05-04 04:28:54 +03:00
FawnD2
7b23844b32 Common order of precompiled checks 2020-05-04 03:58:10 +03:00
FawnD2
02e12215e7 Apply reducing memory usage optimization for seekable files to ORC format 2020-05-04 03:52:28 +03:00
alexey-milovidov
6b0bffcdab
Update CompressionFactory.cpp 2020-05-04 03:47:41 +03:00
Alexey Milovidov
99c18c5a09 Check for suspicious codecs #4966 2020-05-04 03:11:49 +03:00
FawnD2
a554177724 Simplify ORC format 2020-05-04 02:23:20 +03:00
alexey-milovidov
884c2aa631
Merge pull request #10641 from ClickHouse/storage-buffer-nullptr-dereference 
Fix nullptr dereference in StorageBuffer
 2020-05-04 02:01:57 +03:00
alexey-milovidov
788b436275
Merge pull request #10640 from ClickHouse/fix-http-code-parse-error 
Fix http code in case of some parse errors.
 2020-05-04 02:00:47 +03:00
FawnD2
112758b99d Merge branch 'master' into arrow-io-format 2020-05-04 00:53:17 +03:00
Alexey Milovidov
a5bdc375ca Fix Arcadia #10569 2020-05-04 00:36:11 +03:00
Alexey Milovidov
f8866af0c1 Fix nullptr dereference in StorageBuffer 2020-05-03 23:53:54 +03:00
FawnD2
2359ba3497 Correct stop condition 2020-05-03 22:59:19 +03:00
FawnD2
6b2e97fd56 Increate total read length while reading 2020-05-03 22:54:28 +03:00
FawnD2
b4898bdd19 Read single batch 2020-05-03 22:49:06 +03:00
Alexey Milovidov
bb6eabd82d Fix HTTP response code for some parse errors #10636 2020-05-03 22:47:07 +03:00
FawnD2
a8d914a811 Read full table 2020-05-03 22:22:41 +03:00
Артем Стрельцов
5bce553d91 performance optimization 2020-05-03 22:08:45 +03:00
Alexey Milovidov
8f7374c6de Fix another UBSan warning 2020-05-03 21:39:52 +03:00
FawnD2
b86562b8bc Style fixes 2020-05-03 21:29:04 +03:00
Alexey Milovidov
9a344e681f Fix UBSan report in FixedHashTable 2020-05-03 21:24:11 +03:00
Alexey Milovidov
626b9a3ae9 Fix UBSan report in HashTable 2020-05-03 21:21:18 +03:00
FawnD2
7cc7a87f9f Simplify interfaces 2020-05-03 21:12:14 +03:00
Артем Стрельцов
ff7b5db800 fixed bug with dictGetString 2020-05-03 21:03:41 +03:00
Артем Стрельцов
31fb42cf01 removed unused arena 2020-05-03 19:12:17 +03:00
Артем Стрельцов
f8d3999fa2 Merge branch 'master' of github.com:yandex/ClickHouse into layout_direct 2020-05-03 16:23:10 +03:00
FawnD2
a590826fbb Format settings for Arrow 2020-05-03 15:26:39 +03:00
FawnD2
e5b59616cd Style fix 2020-05-03 15:25:53 +03:00
alexey-milovidov
0cdd48e468
Merge pull request #10598 from ClickHouse/sampling-memory-profiler 
Add sampling memory profiler
 2020-05-03 15:17:03 +03:00
Azat Khuzhin
6ffdd53b6a Share auto-increment for first batch and tmp file in DistributedBlockOutputStream 2020-05-03 14:47:59 +03:00
Azat Khuzhin
53c470cab4 Fix directory monitor initialization from INSERT into Distributed 
This also fixes hardlink code (when one file should be sent to multiple
servers, i.e. internal_replication == false) of writeToShard() with
distributed_storage_policy (i.e. when StorageDistributed::getPath() will
path to different filesystems).

Plus also cleanup DistributedBlockOutputStream::writeToShard() a little.
 2020-05-03 14:47:51 +03:00
Azat Khuzhin
e97e1f06db Do not schedule distributed send if there were no error 
Since in this case it will be scheduled from the
DistributedBlockOutputStream with the
distributed_directory_monitor_max_sleep_time_ms, and this will overwrite
timer that was set by the DistributedBlockOutputStream, not good.
 2020-05-03 14:46:44 +03:00
Azat Khuzhin
947b3942dd Schedule distributed sends after the file has been written 2020-05-03 14:46:43 +03:00
Azat Khuzhin
0157fd5d93 Fix distributed send that are scheduled by INSERT query 
Before this patch each INSERT query re-schedule distributed send, thus
each time it resets the timer, while this is not the expected behaviour,
since in on frequent INSERT distributed sends will not be triggered at
all.

Fix this by not resetting the timer.
 2020-05-03 14:46:42 +03:00
Azat Khuzhin
6bb39dafc3 Drop decreated code (cond var and note for thread) in DirectoryMonitor 2020-05-03 14:46:41 +03:00
Alexey Milovidov
469cdc315b Added suggestion from @azat #10607 2020-05-03 14:40:37 +03:00
FawnD2
f62f1a5924 Add missed include 2020-05-03 06:08:37 +03:00
FawnD2
6e0b31fb2a Arrow output format 2020-05-03 05:46:36 +03:00
FawnD2
aeed62d5e7 Remove unused error codes 2020-05-03 04:20:03 +03:00
FawnD2
4d0225f5a9 std::string -> String 2020-05-03 04:15:09 +03:00
FawnD2
f4fe23e3fe Add ptr wrapper. Fix style 2020-05-03 04:01:05 +03:00
FawnD2
fa1c35753b Move CH -> Arrow routine converter into separated class 2020-05-03 03:54:39 +03:00
Alexey Milovidov
abe40e2127 Added required comment #10576 2020-05-03 01:34:50 +03:00
alexey-milovidov
a235533d5f
Merge pull request #10576 from Jokser/s3-with-dynamic-proxies 
S3 with dynamic proxy configuration
 2020-05-03 01:25:13 +03:00
alexey-milovidov
9011d18b80
Update S3Common.h 2020-05-03 01:25:02 +03:00
alexey-milovidov
d9400473f9
Update ThreadStatusExt.cpp 2020-05-03 01:20:14 +03:00
alexey-milovidov
67efc7f066
Merge pull request #10560 from Enmk/DateTime64_fixes 
Fixed comparing DateTime64 in WHERE against String value
 2020-05-03 00:54:11 +03:00
Артем Стрельцов
359de11fee removed redundant lines and comments 2020-05-03 00:49:19 +03:00
alexey-milovidov
342ed093de
Update S3Common.cpp 2020-05-02 23:46:39 +03:00
Артем Стрельцов
c82b1fc0fe fixed (hope) test flapping 2020-05-02 23:26:22 +03:00
Alexey Milovidov
554e7a0dd2 Preparations 2020-05-02 22:59:07 +03:00
Alexey Milovidov
e6ab4d655b Fix bad code 2020-05-02 22:54:29 +03:00
Alexey Milovidov
f0e80fec68 Whitespaces 2020-05-02 22:41:01 +03:00
alexey-milovidov
330627f597
Update ArrowBlockInputFormat.h 2020-05-02 22:35:58 +03:00
alexey-milovidov
47e3f83bc8
Update ArrowBlockInputFormat.h 2020-05-02 22:34:34 +03:00
alexey-milovidov
07f824c746
Update ArrowBlockInputFormat.cpp 2020-05-02 22:33:47 +03:00
alexey-milovidov
dfeb76038d
Update ArrowBlockInputFormat.h 2020-05-02 22:32:33 +03:00
alexey-milovidov
fbb280b5ac
Update ArrowBlockInputFormat.cpp 2020-05-02 22:32:21 +03:00
alexey-milovidov
db4c2352e1
Merge pull request #10569 from zhang2014/fix/ISSUES-10551 
ISSUES-10551 add backward compatibility for create bloom filter index
 2020-05-02 22:25:29 +03:00
alexey-milovidov
250a44a47f
Merge pull request #10607 from Avogar/json_as_string_format 
Add JSONAsStringRowInputFormat
 2020-05-02 22:17:22 +03:00
alexey-milovidov
229f666dea
Merge pull request #10611 from azat/optimize_skip_unused_shards-LowCardinality 
Fix optimize_skip_unused_shards with LowCardinality
 2020-05-02 22:14:33 +03:00
alexey-milovidov
4f1b115e76
Merge pull request #10610 from azat/optimize_skip_unused_shards-fixes 
Fix logging for optimize_skip_unused_shards
 2020-05-02 22:13:01 +03:00
alexey-milovidov
d38ea6fc1c
Merge pull request #10608 from ClickHouse/fix_drop_dns_cache 
Improve DROP DNS CACHE
 2020-05-02 22:10:58 +03:00
Артем Стрельцов
5ba6572210 fixed problem with dictGetString and Float32 2020-05-02 20:58:24 +03:00
tavplubix
a073cd6c79
Fix segfault in StorageBuffer when exception on server startup (#10609) 
* fix segfault in StorageBuffer when exception on server startup

* Trigger CI

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-05-02 19:54:20 +03:00
Alexey Milovidov
81adcc9058 Remove obsolete records from system.build_options 2020-05-02 17:00:03 +03:00
alexey-milovidov
d763d0019b
Merge pull request #10625 from amosbird/unwind 
Partial fix of clang build stack unwinding (omit-frame-pointer)
 2020-05-02 16:43:04 +03:00
Артем Стрельцов
2a4ce4f41c style fix 2020-05-02 15:02:57 +03:00
Amos Bird
014a1233ff
Partial fix of clang build stack unwinding 2020-05-02 18:52:29 +08:00
Артем Стрельцов
25d7f76e4f fixed bug & created functional test for new layout 2020-05-02 04:03:29 +03:00
Артем Стрельцов
97496048d6 created direct dictionary (has bugs, doesn't work properly) 2020-05-02 02:35:54 +03:00
Matwey V. Kornilov
d7627fa4fb Fix capnproto version check for capnp::UnalignedFlatArrayMessageReader 
capnp::UnalignedFlatArrayMessageReader was introduced in
3aa2b2aa02
(which is a part of 0.7.0 release). Unfortunately,
capnp::UnalignedFlatArrayMessageReader was removed in
3f0fee61c6
(which is a part of 0.8.0 release)

So change CAPNP_VERSION check accordingly.
 2020-05-01 22:32:06 +03:00
Alexey Milovidov
439777ef23 Allow to set memory_profiler_sample_probability per query 2020-05-01 22:17:56 +03:00
Alexey Milovidov
93c3a83af2 Remove useless code 2020-05-01 22:01:02 +03:00
Alexey Milovidov
600b396f1b Merge remote-tracking branch 'origin/master' into sampling-memory-profiler 2020-05-01 21:48:20 +03:00
Alexander Kazakov
e9baaa439b
Implementation of new system metrics provider (Procfs) (#10544) 
* New metrics provider (Procfs) + Refactored TasksStatsCounters

* Trivial statless test that ProcFS is provided

* Trivial perf test for ProcfsMetricsProvider

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-05-01 21:47:41 +03:00
alexey-milovidov
404452bad3
Update MergeTreeIndices.cpp 2020-05-01 21:40:56 +03:00
Avogar
3de0bca856 Remove double space 2020-05-01 19:50:18 +03:00
Avogar
9670e74885 Update makeContinuousMemoryFromCheckpointToPos in PeekableReadBuffer. 2020-05-01 19:42:15 +03:00
Avogar
697d5b07ce Fix build and style errors. Update test. 2020-05-01 19:41:14 +03:00
Alexey Milovidov
c4b64edc14 Merge branch 'master' into sampling-memory-profiler 2020-05-01 16:47:10 +03:00
Alexey Milovidov
6ba6d5fc69 Revert unrelated modification 2020-05-01 16:46:42 +03:00
Vasily Nemkov
86edceeb81 Comparison between DateTime64 and String values (just like for DateTime) 2020-05-01 15:52:18 +03:00
Vitaly Baranov
2528e72199
Merge pull request #10587 from vitlibar/database-with-dictionary-init-in-constructor 
Move initialization of DatabaseWithDictionaries to constructor.
 2020-05-01 14:08:53 +03:00
alesapin
4d8660f270 Remove obsolete comment 2020-05-01 13:05:37 +03:00
Azat Khuzhin
63d8ab8f03 Make createSelector() static (in storage) and const (in stream) 2020-05-01 11:31:05 +03:00
Alexander Kuzmenkov
2ac26bd3a5 Disable mremap under MemorySanitizer. 2020-05-01 10:38:05 +03:00
Azat Khuzhin
f22ba15b4a Reduce copy-paste of DistributedBlockOutputStream::createSelector 
This will make it less error prone.
 2020-05-01 02:59:40 +03:00
Azat Khuzhin
cdd7013438 Drop superfluous "Skipping irrelevant shards" messages 
Before this patch it printed 3 times:
- from StorageDistributed::getProcessingStageImpl()
- from StorageDistributed::read()
- from StorageDistributed::getProcessingStageImpl() (from StorageDistributed::read() -> getSampleBlock())
(But this should be optimized)
 2020-05-01 02:56:13 +03:00
Azat Khuzhin
c648c300bf Fix optimize_skip_unused_shards with LowCardinality 2020-05-01 02:39:58 +03:00
Azat Khuzhin
4cbe625567 Fix shard numbers output in logs (full cluster had been printed over optimized) 2020-05-01 02:13:07 +03:00
Alexander Tokmakov
9ca144cab9 drop access cache on DROP DNS CACHE 2020-05-01 01:29:47 +03:00
Avogar
6464a1a902 Add JSONAsStringRowInputFormat 2020-04-30 23:16:09 +03:00
Vitaly Baranov
dd34cd73e2 Move initialization of DatabaseWithDictionaries to constructor. 2020-04-30 20:22:20 +03:00
Alexey Milovidov
db3241ccc7 Add sampling memory profiler 2020-04-30 16:30:10 +03:00
Alexey Milovidov
293ae88e7f Add sampling memory profiler 2020-04-30 16:25:17 +03:00
alesapin
15e8f37839
Merge pull request #10531 from CurtizJ/polymorphic-parts-2 
Fix index corruption in merges with compact parts.
 2020-04-30 12:15:25 +03:00
Alexander Kuzmenkov
7f5b57c6c5
Merge pull request #10559 from ClickHouse/aku/summap-experiment 
Merge pull request #10000 (sumMap for Tuple argument)
 2020-04-30 11:56:04 +03:00
alesapin
5e2afad549
Merge pull request #10477 from ClickHouse/refactor_istorage 
Remove getColumn and hasColumn from IStorage, better virtuals.
 2020-04-30 10:42:52 +03:00
Anton Popov
6ed3092c8a make fillIndexGranularity less complicated 2020-04-30 00:57:58 +03:00
Pavel Kovalenko
0b70c2ca08 Fixed ya.make for S3. 2020-04-29 21:38:35 +03:00
alesapin
47e32aaaa1
bump CI 2020-04-29 21:17:15 +03:00
alesapin
7a8380a682 Fix build 2020-04-29 19:26:01 +03:00
alesapin
69fdf2a6a3 Review fixes and better test 2020-04-29 15:15:23 +03:00
Pavel Kovalenko
f17e022fa4 Code style fixes. 2020-04-29 15:05:46 +03:00
Nikolai Kochetov
a9d63d4520
Merge pull request #10498 from vzakaznikov/insert_into_watch_from_live_view 
Adding support for INSERT INTO table WATCH query to build streaming systems using LIVE VIEW tables
 2020-04-29 12:15:54 +03:00
Pavel Kovalenko
a1acbcb019 Code style fixes. 2020-04-29 11:45:13 +03:00
Pavel Kovalenko
e5e84960ea S3 with proxy integration test and fixes. 2020-04-29 11:40:29 +03:00
Pavel Kovalenko
2ecf4b5a23 Dynamic proxy configuration for DiskS3. 2020-04-29 11:40:29 +03:00
Alexander Kuzmenkov
c5a7796d16 fixup 2020-04-29 09:35:02 +03:00
Alexander Kuzmenkov
31bf2b3be4 fixup 2020-04-29 09:05:52 +03:00
zhang2014
8ac709b5d3 ISSUES-10551 add backward compatibility for create bloom filter index 2020-04-29 11:57:28 +08:00
Azat Khuzhin
038235684d Add optimize_distributed_group_by_sharding_key and disable it by default 
I know at least one way to fool that optimization, by using as sharding
key something like `if(col1>0, col1, col2)` (although this is not common
sharding key I would say, but can be useful if this will work
correctly), so let's disable it by default.
 2020-04-29 00:09:25 +03:00
FawnD2
538e6c39da Arrow input format 2020-04-28 22:52:22 +03:00
Azat Khuzhin
53aea23de0 Fix SELECT of column ALIAS which default expression type different from column type 2020-04-28 22:43:10 +03:00
robot-clickhouse
de4e88ff97 Auto version update to [20.5.1.1] [54435] 2020-04-28 20:12:16 +03:00
Vitaliy Zakaznikov
664a09e55d Simplifying code related to auto set output_format_enable_streaming option. 2020-04-28 18:37:15 +02:00
Vitaliy Zakaznikov
4749122734 Adding support to auto set output_format_enable_streaming option 
based on the query type.
 2020-04-28 17:50:50 +02:00
Artem Zuikov
0b18b6104f add file to ya.make 2020-04-28 18:38:07 +03:00
Alexander Kuzmenkov
73398a3e17 fixup 2020-04-28 17:30:45 +03:00
Artem Zuikov
e34a9457e6
Improve right table insert for Partial MergeJoin on disk (#10467) 2020-04-28 16:55:50 +03:00
alesapin
30052ea46c Better 2020-04-28 16:00:41 +03:00
alesapin
80443af5dc Fix 2020-04-28 15:57:44 +03:00
alesapin
7c25a9bcbd Fix schema of storage 2020-04-28 15:33:17 +03:00
alesapin
f981649213 Fix pushing to views stream and refactor virtuals 2020-04-28 13:38:57 +03:00
Eldar Zaitov
043be916cb Add miniman fuzzing documentation, fix code style. 2020-04-28 13:35:44 +03:00
Eldar Zaitov
f40603441b Add miniman fuzzing documentation, fix code style. 2020-04-28 13:34:57 +03:00
tavplubix
11c3493676
Merge pull request #10547 from ClickHouse/zhang2014/feature/ISSUES-5436 
Merging #7572
 2020-04-28 12:40:48 +03:00
Eldar Zaitov
670ed4bf1b lexer, ast, checksum and column specs fuzzers added 2020-04-28 02:12:18 +03:00
Alexander Tokmakov
e72a484beb minor fixes 2020-04-28 02:05:15 +03:00
Nikolai Kochetov
de6d2d01b9 Try to fix tests. 2020-04-27 21:15:55 +03:00
Anton Popov
032a0aa282 fix clang-tidy 2020-04-27 21:12:17 +03:00
alesapin
7c2f4d121a Fix func 2020-04-27 20:48:53 +03:00
alesapin
4badd0fd28 Better code 2020-04-27 20:46:51 +03:00
alesapin
228dbf475e
Finalize mutations in background (#10526) 2020-04-27 19:19:04 +03:00
alesapin
01db4877f6 Fix style check 2020-04-27 18:44:33 +03:00
alesapin
c24ab563d7 Fix several wrong usages 2020-04-27 18:38:35 +03:00
Nikolai Kochetov
2f06180c5e Revert changes for CSVRowOutputFormat. 2020-04-27 18:21:53 +03:00
alesapin
b0c5c1d685 Fix style 2020-04-27 18:21:37 +03:00
alesapin
6cfe0f7046 Fix after refactoring 2020-04-27 18:20:03 +03:00
alesapin
cefc58e77c Fix style 2020-04-27 18:17:08 +03:00
alesapin
5c6316afbc Fix bug 2020-04-27 18:10:50 +03:00
Nikolai Kochetov
201a184d31 Enable auto flust for output formats if enable_streaming was set. 2020-04-27 18:00:10 +03:00
Nikolai Kochetov
32baa073b5 Merge remote-tracking branch 'upstream/master' into insert_into_watch_from_live_view 2020-04-27 17:49:50 +03:00
alesapin
4c1ac83112 Fix in IStorage 2020-04-27 17:44:32 +03:00
alesapin
e85b50c911 Revert "Strange fix, how it was possible earlier?" 
This reverts commit 8d20644c99.
 2020-04-27 17:17:12 +03:00
alesapin
8d20644c99 Strange fix, how it was possible earlier? 2020-04-27 17:12:54 +03:00
alesapin
18c550df15 Better virtuals logic 2020-04-27 16:55:30 +03:00
Anton Popov
bee343dae1 fix writing of index in compact parts 2020-04-27 16:36:50 +03:00
alesapin
2829774105 Merge branch 'master' into refactor_istorage 2020-04-27 15:34:21 +03:00
alexey-milovidov
436e4e3f5e
Merge pull request #10502 from oandrew/avro-nested2 
Avro nested fields - support complex types
 2020-04-27 13:44:58 +03:00
Andrew Onyshchuk
570be94a48 Use correct type for DateTime64 (ubsan) 2020-04-26 21:10:30 -05:00
Azat Khuzhin
20b4eed9a1 Disable GROUP BY sharding_key optimization for WITH ROLLUP/CUBE/TOTALS 2020-04-27 01:30:54 +03:00
Alexey Milovidov
cac130ad3b Use time zone when comparing DateTime with string literal 2020-04-27 01:13:51 +03:00
Alexey Milovidov
a7d7dc5034 Fix some bad code 2020-04-26 20:34:36 +03:00
alexey-milovidov
c6ae6a7ba8
Merge pull request #10418 from ClickHouse/fix_logical_error_in_diagnostic_info 
Better diagnostic info in input formats
 2020-04-26 20:26:59 +03:00
alexey-milovidov
4c17542096
Merge pull request #10465 from kekekekule/ddl_settings 
Add custom settings support in DDL-queries for CREATE DICTIONARY
 2020-04-26 20:25:29 +03:00
alexey-milovidov
7d58dc154c
Update ASTDictionary.h 2020-04-26 20:24:18 +03:00
alexey-milovidov
0a9bcae0ed
Merge pull request #10511 from ClickHouse/fix-ubsan-datetime64 
Fixed UBSan for DateTime64; added a test
 2020-04-26 20:17:01 +03:00
alexey-milovidov
378d73d477
Merge pull request #10449 from azat/metrics-for-new-bg-schedule-pools 
Add tasks/memory metrics for distributed/buffer schedule pools
 2020-04-26 20:16:02 +03:00
Alexey Milovidov
a70637ab4b Fixed UBSan; added a test 2020-04-26 17:57:45 +03:00
alexey-milovidov
d1464a441f
Merge pull request #10503 from oandrew/read-columns-perf-fix 
IRowInputFormat performance fix
 2020-04-26 17:36:54 +03:00
alexey-milovidov
62e2f10959
Merge pull request #10474 from hczhcz/patch-0422 
Fix overflow in simpleLinearRegression
 2020-04-26 17:34:57 +03:00
Vitaliy Zakaznikov
626468d255 Adding support for streaming mode to JSONEachRowRowOutputFormat 2020-04-26 15:56:45 +02:00
Vitaliy Zakaznikov
369b4d53ef Adding support for output_format_enable_streaming format setting. 2020-04-26 15:44:11 +02:00
Andrew Onyshchuk
4a5497498e IRowInputFormat performance fix 2020-04-25 19:33:36 -05:00
Alexey Milovidov
d01bd44f14 Merge branch 'master' into execute-query-flush-format-2 2020-04-26 03:31:46 +03:00
alexey-milovidov
8820b43606
Merge pull request #10479 from Avogar/markdown_format 
Markdown format
 2020-04-26 03:25:48 +03:00
Andrew Onyshchuk
e7ba75ce98 Avro nested fields - support complex types 2020-04-25 17:42:49 -05:00
alexey-milovidov
8b1e504b19
Merge pull request #10497 from azat/kafka 
Tiny kafka fixes
 2020-04-25 23:50:22 +03:00
alexey-milovidov
092efc53ba
Merge pull request #10496 from ClickHouse/fix-visible-drift-memory-tracking 
Fix visible drift of MemoryTracking metric
 2020-04-25 23:50:08 +03:00
alexey-milovidov
1e25991c62
Merge pull request #10494 from kyprizel/master 
Split fuzzers and sanitizers
 2020-04-25 23:49:45 +03:00
alexey-milovidov
c9334d3fde
Merge pull request #10491 from azat/dist-shutdown 
Proper Distributed shutdown (fixes UAF, avoid waiting for sending all batches)
 2020-04-25 23:47:59 +03:00
Vitaliy Zakaznikov
71a40d6ec9 Revert "Working on adding AUTO REFRESH to LIVE VIEW tables." 
This reverts commit 855e06b031.
 2020-04-25 09:56:57 -04:00
Vitaliy Zakaznikov
1fc345f5a0 Revert "Reverting all the changes in LIVE VIEW table." 
This reverts commit 684c7f943a.
 2020-04-25 09:56:41 -04:00
Vitaliy Zakaznikov
e73166c78d Removing SquashingBlockOutputStream from the stream to 
allow blocks from WATCH query to propagate to target table
as is.
 2020-04-25 09:46:46 -04:00
Alexey Milovidov
c75c2fac29 Fix visible drift of MemoryTracking metric 2020-04-25 15:36:01 +03:00
Azat Khuzhin
40c65f7bee IRowInputFormat: omit "Skipped 0 rows with errors while reading the input stream" 
This will just pollute the output log.

And this is significant for kafka streaming, since it will print this
message for each message (due to how
ReadBufferFromKafkaConsumer/KafkaBlockInputStream works).

You can see this in the server log of the test_storage_kafka
(integration test), for the table with kafka_skip_broken_messages > 0,
it will have 52 lines of such message, 2 with "Skipped 1 rows" and 50
with "Skipped 0 rows" (since there are 52 messages in summary, 2
malformed and 50 correct).
 2020-04-25 15:16:29 +03:00
Vitaliy Zakaznikov
684c7f943a Reverting all the changes in LIVE VIEW table. 2020-04-25 07:39:15 -04:00
Vitaliy Zakaznikov
8621e93460 Starting to remove changes to LIVE VIEW table. 2020-04-25 07:37:33 -04:00
Vitaliy Zakaznikov
34a03939ea First changes to try to support INSERT INTO table WATCH query. 2020-04-25 07:33:47 -04:00
Eldar Zaitov
d7a037c26b split fuzzers and sanitizers 2020-04-25 10:12:38 +03:00
Azat Khuzhin
747a74215f Avoid processing all batches before Distributed shutdown 2020-04-25 02:03:27 +03:00
Azat Khuzhin
8ad6b37913 Proper StorageDistributed shutdown to avoid UAF in DistributedMonitor 
StorageDistributed::shutdown() does not acquire the lock, that controls
access to the cluster_nodes_data, thus it does not synced with the
requireDirectoryMonitor(), hence some monitors can be untracked that
will trigger UAF (use-after-free) after DROP TABLE dist:

This is for the SIGSEGV from the DirectoryMonitor (with already destroyed storage):
    0  0x0000000008e9f760 in std::__1::__cxx_atomic_load<int> (__order=std::__1::memory_order::seq_cst, __a=0x0)
    1  std::__1::__atomic_base<int, false>::load (__m=std::__1::memory_order::seq_cst, this=0x0) <-- this is nullptr
    2  std::__1::__atomic_base<int, false>::operator int (this=0x0)
    3  DB::ActionBlocker::isCancelled (this=0x7f85e31c9bb8) at ../src/Common/ActionBlocker.h:18
    4  DB::StorageDistributedDirectoryMonitor::run (this=0x7f85f93b2a00) at ../src/Storages/Distributed/DirectoryMonitor.cpp:140
 2020-04-25 02:03:26 +03:00
Vitaliy Zakaznikov
855e06b031 Working on adding AUTO REFRESH to LIVE VIEW tables. 2020-04-24 17:15:17 -04:00
tavplubix
7928f2eaa1
Merge pull request #7512 from ClickHouse/database_atomic 
DatabaseAtomic
 2020-04-24 22:28:18 +03:00
Alexander Tokmakov
04d4130b0d update yamake 2020-04-24 20:14:10 +03:00
Артем Стрельцов
5423506393 sure this will exactly fix the fail with build check 2020-04-24 19:04:54 +03:00
Alexander Tokmakov
53dcce55e9 better diagnostic info in input formats 2020-04-24 15:03:48 +03:00
Avogar
778f1aa1e7 Merge branch 'markdown_format' of github.com:avogar/ClickHouse into markdown_format 2020-04-24 14:07:04 +03:00
Avogar
2458da802c Update MarkdownRowOutputFormat 2020-04-24 14:06:41 +03:00
alesapin
793f4b734a Remove obsolete comment 2020-04-24 13:31:03 +03:00
alesapin
b4425c2dc4 Remove getColumn and hasColumn methods from IStorage 2020-04-24 13:20:03 +03:00
Артем Стрельцов
26a6353284 tried to fix build fail & fixed codestyle 2020-04-24 12:59:49 +03:00
alesapin
0c622370c5 Better 2020-04-24 12:21:38 +03:00
alesapin
dc2dd77d2e Remove redundant overrides from IStorage 2020-04-24 12:20:09 +03:00
alesapin
a2cb67c542
Merge pull request #10459 from ClickHouse/refactor_istorage 
Remove virtuals from IStorage as separate column
 2020-04-24 11:42:04 +03:00
hcz
541cd638ba Fix overflow in simpleLinearRegression 2020-04-24 16:33:09 +08:00
alexey-milovidov
79d531af87
Merge pull request #10466 from ClickHouse/aku/arena-infinite-loop 
Work around a bug leading to an infinite loop in addressToLine
 2020-04-24 10:38:07 +03:00
Alexander Kuzmenkov
2bedc82c18 Work around a bug leading to an infinite loop in addressToLine. 
Fixing this is hopeless and would entail rewriting the entire
WriteBuffer thing, and also this hack is something we can backport.
 2020-04-24 02:42:04 +03:00
Артем Стрельцов
aab2e8f9c1 unsignificant change 2020-04-24 02:21:51 +03:00
tavplubix
7d6ab32eb6
Update DatabaseCatalog.cpp 2020-04-24 01:08:26 +03:00
Артем Стрельцов
d7e27d4326 added SETTINGS section to DDL-query for CREATE DICTIONARY 2020-04-24 00:57:40 +03:00
Артем Стрельцов
f39fdf7182 some progress in parsing settings (i did this commit for having something to backup to) 2020-04-23 23:50:01 +03:00
Alexander Tokmakov
8be4d51111 fix build 2020-04-23 22:50:05 +03:00
Alexey Milovidov
2b569cf260 Added clarification in exception message #10405 2020-04-23 21:31:37 +03:00
Alexander Tokmakov
15e7ce0053 fixes 2020-04-23 21:00:43 +03:00
Alexander Tokmakov
f413061e0d remove Context from some IDatabase methods 2020-04-23 19:51:48 +03:00
alesapin
5ae25b68e4 Remove redundant virtual from function 2020-04-23 18:18:33 +03:00
alesapin
b0a73589b0 Remove virtual columns as separate field from IStorage 2020-04-23 18:09:00 +03:00
Alexander Tokmakov
04d6b59ac0 Merge branch 'master' into database_atomic 2020-04-23 17:31:37 +03:00
Артем Стрельцов
5da371b9c0 added settings parsing from DDL-query (they aren't applied now) 2020-04-23 15:39:27 +03:00
alexey-milovidov
1ecfa27cd3
Merge pull request #10441 from ClickHouse/remove_casts_from_defaults 
Remove CAST from DEFAULT expressions during table creation.
 2020-04-23 12:36:12 +03:00
Nikita Mikhaylov
3520755ab5 fix c++20 usage 2020-04-23 12:00:43 +03:00
Azat Khuzhin
4f5ec14f27 Configure thread names for buffer/distributed schedule pools 2020-04-23 03:16:14 +03:00
Azat Khuzhin
d854049d21 Add tasks/memory metrics for distributed/buffer schedule pools 
Follow-up-for: #10315
Follow-up-for: #10263
 2020-04-23 02:18:37 +03:00
Azat Khuzhin
c04c73b4c4 Drop unused MemoryTrackingForMerges from Context 
It set in src/Storages/MergeTree/MergeList.cpp anyway.
 2020-04-23 01:57:33 +03:00
alexey-milovidov
77969d11a5
Merge pull request #10442 from zhang2014/fix/create_database_failure 
Not remove metadata directory when attach database fails
 2020-04-23 01:07:13 +03:00
alexey-milovidov
b8e3655f0d
Merge pull request #10445 from azat/logger-source-coloring-client 
Make logger name colored in client too
 2020-04-23 00:35:40 +03:00
alexey-milovidov
fd27ce619b
Merge pull request #10444 from ClickHouse/server-memory-profiler 
Simple server wide memory profiler
 2020-04-23 00:34:23 +03:00
alexey-milovidov
4f6bb00424
Merge pull request #10420 from ClickHouse/clang-tidy-8 
Fixes for clang-tidy on clang-10
 2020-04-23 00:33:41 +03:00
Alexander Tokmakov
e02b5790af fix 2020-04-22 23:49:13 +03:00
Alexander Tokmakov
1833ac7f16 fixes 2020-04-22 23:43:10 +03:00