thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-18 05:32:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
40,839 Commits 9,136 Branches 1,774 Tags 2 GiB
d0c0a0aed6
Commit Graph

1347 Commits

Author SHA1 Message Date
Alexey Milovidov
6a3055a924 Added a comment 2020-05-11 03:38:35 +03:00
Alexey Milovidov
f8d2e599a6 Added a comment 2020-05-11 03:37:31 +03:00
Alexey Milovidov
fe19ca9955 Improve performance two times 2020-05-11 03:36:14 +03:00
Alexey Milovidov
75860eb674 Improve performance 20% 2020-05-11 03:23:47 +03:00
Alexey Milovidov
49e57c555d Fixed code; added a test 2020-05-11 03:16:50 +03:00
Alexey Milovidov
2438a510b0 Merge branch 'fuzz_functions' of https://github.com/xolm/ClickHouse into xolm-fuzz_functions 2020-05-11 02:42:55 +03:00
Alexey Milovidov
20a7889909 Integrate a change from Vitaly Stoyan #10788 2020-05-11 02:39:49 +03:00
alexey-milovidov
f359d75bbc
Merge pull request #10788 from vitstn/patch-1 
Update ya.make
 2020-05-11 02:38:43 +03:00
Andrew Onyshchuk
330f06328f Use src_type for convertion in KeyCondition 2020-05-10 18:17:54 -05:00
Andrei Nekrashevich
84ed5b118c fix 2020-05-10 22:31:47 +03:00
Alexey Milovidov
20808de08c Merge remote-tracking branch 'origin/master' into even-more-warnings 2020-05-10 22:28:16 +03:00
Andrei Nekrashevich
c3873495c9 fix and performance test template 2020-05-10 22:07:02 +03:00
vitstn
be6fba82f9
Update ya.make 2020-05-10 20:51:32 +03:00
Gleb Novikov
1235c36f3e data_part instead of volume in IMergeTreeDataPartWriter (as it done in IMergeTreeReader) 2020-05-10 16:33:27 +03:00
Azat Khuzhin
7342f46394 Fix setting type for the memory_profiler_sample_probability 2020-05-10 15:49:31 +03:00
Gleb Novikov
57b6fb6d80 Merge branch 'master' into refactor-reservations 2020-05-10 13:01:45 +03:00
Azat Khuzhin
5c89cdbe61 Fix distributed send retries on distributed_directory_monitor_{max_,}sleep_time_ms > 5min 
In this case error_count can be decreased before checking it for
rescheduling send.

And actually this can be a problem not only when
distributed_directory_monitor_{max_,}sleep_time_ms > 5min, because all
threads can be occupated and the real timeout between sends will be > 5min.
 2020-05-10 12:37:38 +03:00
Gleb Novikov
fbd00d6af3 removed default value from overriden functions in IReservation children 2020-05-10 12:33:52 +03:00
Gleb Novikov
a961e049a2 VolumeType as class with enum member class 2020-05-10 09:26:33 +03:00
Alexey Milovidov
36d9d646a9 Fix race condition in ReplicatedMergeTreeQueue 2020-05-10 05:18:10 +03:00
Alexey Milovidov
ce59ad8e04 Merge branch 'master' into even-more-warnings 2020-05-10 03:49:12 +03:00
alexey-milovidov
21fe2bdc92
Merge pull request #10780 from cw9/cw9-patch-1 
Fix comment for merge_tree_max_bytes_to_use_cache
 2020-05-10 03:47:52 +03:00
Alexey Milovidov
fd26550a70 Remove commented code before merge #10580 2020-05-10 03:41:52 +03:00
Alexey Milovidov
d9257a8682 Merge branch 'arrow-io-format' of https://github.com/FawnD2/ClickHouse into FawnD2-arrow-io-format 2020-05-10 03:26:10 +03:00
alexey-milovidov
992e58923a
Merge pull request #10588 from excitoon-favorites/fixmutations 
Fixed handling condition variable for synchronous mutations
 2020-05-10 03:20:41 +03:00
alexey-milovidov
ad26306131
Fix clang-tidy. 2020-05-10 03:09:51 +03:00
Alexey Milovidov
cb08d2b910 Fix build 2020-05-10 03:04:14 +03:00
Alexey Milovidov
9ab19f98ee Fix build 2020-05-10 02:54:04 +03:00
Артем Стрельцов
de8bf3f78c Draft of ComplexKeyDirectDictionary layout 2020-05-10 02:53:01 +03:00
Alexey Milovidov
dda2151f71 Fix build 2020-05-10 02:35:20 +03:00
alexey-milovidov
63c4a7661f
Merge pull request #10715 from CurtizJ/pk-function 
Fix usage of primary key wrapped into a function.
 2020-05-10 02:08:59 +03:00
Alexey Milovidov
a576a4fbcd Enable extra warnings for base, utils, programs 2020-05-10 01:59:34 +03:00
Alexey Milovidov
fb32dbe6ee Enable more warnings 2020-05-10 00:44:33 +03:00
Gleb Novikov
390b39b272 VolumePtr instead of DiskPtr in MergeTreeData* 2020-05-10 00:24:15 +03:00
alexey-milovidov
d459f07eee
Update randomString.cpp 2020-05-09 21:54:23 +03:00
alexey-milovidov
4a66b10669
Update randomString.cpp 2020-05-09 21:52:27 +03:00
Nikolai Kochetov
540e1004ca Remove useless code. 2020-05-09 21:43:16 +03:00
Nikolai Kochetov
8d6413accb try fix tests. 2020-05-09 21:43:16 +03:00
Nikolai Kochetov
5cf705c9df Try fix tests. 2020-05-09 21:43:16 +03:00
Nikolai Kochetov
66111a6744 Try nut update scalars for only_analyze. 2020-05-09 21:43:16 +03:00
Nikolai Kochetov
5cc0d5e537 Do not calculate subqueries when only_analyze enabled. 2020-05-09 21:43:16 +03:00
Nikolai Kochetov
884c93ed93 Do not calculate subqueries when only_analyze enabled. 2020-05-09 21:43:16 +03:00
Nikolai Kochetov
18e4df1e85 Create local context for query analyzing in PushingToViewsBlockOutputStream. 2020-05-09 21:43:16 +03:00
Nikolai Kochetov
c6b9580e9e Create local context for query analyzing in PushingToViewsBlockOutputStream. 2020-05-09 21:43:16 +03:00
Nikolai Kochetov
df17eda9c8 Try disable scalars with only_analyse flag. 2020-05-09 21:43:16 +03:00
Nikolai Kochetov
086831e74c Fix tests. 2020-05-09 21:43:16 +03:00
Nikolai Kochetov
abef634d1f Fix scalars for MV. 2020-05-09 21:43:16 +03:00
alexey-milovidov
e41e998f4a
Merge pull request #10758 from bgiard/master 
Fix serialize/deserialize of denominator of Average functions
 2020-05-09 21:22:16 +03:00
alexey-milovidov
42de5192fd
Merge pull request #10772 from azat/fix-system-logs-init 
Avoid std::terminate in case of exception from SystemLogs::SystemLogs
 2020-05-09 19:35:26 +03:00
alexey-milovidov
8f7ee19a2d
Merge pull request #10760 from amosbird/shared-build-fix 
Shared build fix
 2020-05-09 19:33:31 +03:00
Gleb Novikov
b54209e43e Modified reservation to work with multiple disks 2020-05-09 19:03:22 +03:00
alexey-milovidov
f9db45fcc2
Merge pull request #10763 from azat/unbundled-build-poco-fix 
Fix build with unbundled Poco
 2020-05-09 18:27:00 +03:00
Azat Khuzhin
c42943850f Avoid std::terminate in case of exception from SystemLogs::SystemLogs 
Since, at least, this will hide the real exception
 2020-05-09 16:25:58 +03:00
alexey-milovidov
699ef4ff52
Merge pull request #10741 from hczhcz/patch-0422 
Fix OrNull and OrDefault
 2020-05-09 04:38:32 +03:00
alexey-milovidov
5992bae69a
Merge pull request #10736 from ClickHouse/extract-groups-range-check 
Added range check for extractGroups function
 2020-05-09 04:21:56 +03:00
Azat Khuzhin
150dd81e3b Fix build with unbundled Poco 2020-05-09 01:22:12 +03:00
Amos Bird
9d908f48e1
Shared build fix 2020-05-09 04:55:08 +08:00
Chao Wang
bc5f9d5322
Fix comment for merge_tree_max_bytes_to_use_cache 2020-05-08 16:39:00 -04:00
Baudouin Giard
be2cf70da8 Fix serialize/deserialize of denominator of Average functions 
Use writeBinary/readBinary to serialize/deserialize the denominator of Average functions.

Previously it would be transmitted as an unsigned int, which means its value would get corrupted when doing an avgWeighted over a floating point number.

Note: this commit is not backwards compatible!

Signed-off-by: Baudouin Giard <bgiard@bloomberg.net>
 2020-05-08 15:10:47 -04:00
Nikolai Kochetov
e365bf422e Delay exception for PushingToViewsBlockOutputStream mvs. 2020-05-08 17:48:10 +03:00
Ivan
85d783c247
Poco contrib refactoring (#10396) 
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
 2020-05-08 17:11:19 +03:00
Pavel Kovalenko
8d8dc3035c Fix naming. 2020-05-08 16:10:21 +03:00
Pavel Kovalenko
8602f7b47c Minor updates in proxy resolver parameters naming. 2020-05-08 13:58:56 +03:00
Pavel Kovalenko
6165c2aa99 Add possibility to use proxy resolver in DiskS3. 2020-05-08 13:53:12 +03:00
hcz
8a60e22223 Fix OrNull and OrDefault 2020-05-08 17:52:54 +08:00
Vladimir Chebotarev
28631f461e Fixed handling of condition variable for synchronous mutations. 2020-05-08 12:01:06 +03:00
Alexey Milovidov
1f0d95e5c3 Added range check for extractGroups function 2020-05-08 02:31:15 +03:00
Alexey Milovidov
048e26dabf Fix "Arcadia" build 2020-05-07 23:40:18 +03:00
Alexey Milovidov
8690b6e702 Fix "Arcadia" build 2020-05-07 23:17:57 +03:00
alexey-milovidov
07f7f3f33b
Merge pull request #10734 from ClickHouse/fix-generate-random-with-nested 
Fix generate random with nested
 2020-05-07 22:02:43 +03:00
Alexey Milovidov
8d3e68a30b Fix "Arcadia" build 2020-05-07 21:57:04 +03:00
Andrei Nekrashevich
43bc55ab5f randomString function 2020-05-07 18:36:11 +03:00
Andrei Nekrashevich
28685a80f5 ClickHouse: functions for fuzzing 2020-05-07 17:54:33 +03:00
Nikolai Kochetov
c22f7a0864 Try fix header for GenerateRandom. 2020-05-07 17:14:56 +03:00
Guillaume Tassery
2e719314a3 push functional draft 2020-05-07 20:40:50 +07:00
alexey-milovidov
6edb0ea962
Merge pull request #10721 from ClickHouse/fix-10489 
Fix reading from field.
 2020-05-07 15:55:40 +03:00
Vitaliy Zakaznikov
90e52e7fea Adding support for ALTER RENAME COLUMN query to Distributed table engine. 2020-05-07 14:54:35 +02:00
Anton Popov
5fe74b1a3e
Merge pull request #10686 from CurtizJ/fix-nultiple-sets 
Fix multiple usages of 'IN' operator
 2020-05-07 15:39:41 +03:00
Alexey Milovidov
2a3e7f8076 Fix "Arcadia" build 2020-05-07 15:36:56 +03:00
alexey-milovidov
3e583ef45e
Merge pull request #10712 from ClickHouse/merging-extract-groups 
Merging #10534
 2020-05-07 15:34:41 +03:00
Alexey Milovidov
4a1f9f41fb Update ya.make for Arcadia build 2020-05-07 15:33:50 +03:00
Alexey Milovidov
fce23f4535 Update ya.make for Arcadia build 2020-05-07 15:32:33 +03:00
Alexey Milovidov
c85fec6c67 Update ya.make for Arcadia build 2020-05-07 15:31:28 +03:00
Alexey Milovidov
0e00e3049e Regenerate ya.make file 2020-05-07 15:27:42 +03:00
Alexey Milovidov
efeb2acb6b Fix build 2020-05-07 15:19:42 +03:00
tavplubix
04a24c59dc
Add tests with DatabaseAtomic (#10714) 
* add tests with DatabaseAtomic

* fix
 2020-05-07 14:29:58 +03:00
Alexey Milovidov
6efe0376de Fix build 2020-05-07 12:06:11 +03:00
Nikolai Kochetov
0d57b9ded0 Fix reading from field. 2020-05-07 11:58:06 +03:00
Alexey Milovidov
dea8d366c9 Loose some limitation 2020-05-07 04:29:31 +03:00
Alexey Milovidov
b81b4832fe Fix style 2020-05-07 04:25:46 +03:00
Alexey Milovidov
c73453fa86 Less indirection 2020-05-07 04:25:06 +03:00
Alexey Milovidov
2642d23861 Fixed error 2020-05-07 04:06:23 +03:00
Alexey Milovidov
495d18bb64 Added missing function "extractGroups" 2020-05-07 03:55:54 +03:00
Alexey Milovidov
da2c2c3edf Great simplification 2020-05-07 03:40:25 +03:00
Alexey Milovidov
5d1f36872b Fixed style 2020-05-07 02:51:41 +03:00
Alexey Milovidov
1e690510b6 Fixed style 2020-05-07 02:51:09 +03:00
Alexey Milovidov
466da445e1 Every function in its own file, part 13 2020-05-07 02:21:13 +03:00
Anton Popov
cbbd8e521c fix usage of primary key wrapped into a function with 'FINAL' modifier and 'ORDER BY' optimization 2020-05-07 01:27:35 +03:00
alexey-milovidov
eeac7aa313
Merge pull request #10622 from kekekekule/layout_direct 
Add direct layout for dictionaries
 2020-05-07 01:05:29 +03:00
alexey-milovidov
d0b61f9d61
Merge pull request #10711 from ClickHouse/h3-range-check 
Range check in function h3EdgeAngle
 2020-05-07 01:00:57 +03:00
Артем Стрельцов
b2993c314c fixes relevant to review 2020-05-07 00:59:31 +03:00
alexey-milovidov
4df7dd8c2b
Merge pull request #10656 from azat/kafka-consumer-hang 
Kafka cleanup
 2020-05-07 00:20:31 +03:00
Alexey Milovidov
7d332ab705 Minor improvements 2020-05-06 23:00:54 +03:00
Alexey Milovidov
5f498ad9d9 Range check in h3EdgeAngle function 2020-05-06 22:35:59 +03:00
alexey-milovidov
01e7c6bbcf
Merge pull request #10534 from Enmk/multiExtractAll 
extractAllGroups(haystack, re_needle) function
 2020-05-06 22:23:36 +03:00
Nikolai Kochetov
152923aaa4
Merge pull request #10667 from palasonic1/palasonic-fix-params-order 
fix order of params in AggregatingTransform constructor
 2020-05-06 21:47:41 +03:00
alexey-milovidov
e4b3234df9
Merge pull request #10664 from ClickHouse/fix-10655 
Fix 10655
 2020-05-06 21:34:10 +03:00
alexey-milovidov
f2705cf608
Merge pull request #10698 from ClickHouse/fix-10674 
Do not filter totals
 2020-05-06 21:30:11 +03:00
Артем Стрельцов
9088f220ee style fix 2020-05-06 16:59:48 +03:00
Nikolai Kochetov
c72b0a45bc Do not filter totals. 2020-05-06 14:30:17 +03:00
Nikolai Kochetov
bd9df858ed
Merge pull request #10660 from ClickHouse/fix-10566 
Fix 10566
 2020-05-06 11:56:44 +03:00
Guillaume Tassery
8ce606571e write first draft of offset clause 2020-05-06 13:50:55 +07:00
Anton Popov
19528007fd fix multiple usages of 'IN' operator with the same set 2020-05-06 03:57:44 +03:00
Артем Стрельцов
1428188e21 fixed test for direct layout & fixed data load for FileDictionarySource 2020-05-06 03:52:13 +03:00
Артем Стрельцов
50d2b4c26f code review fixes & added exception in case lifetime parameter presence in config 2020-05-06 02:43:06 +03:00
bobrovskij artemij
a2916c23a5 fixes/style 2020-05-06 02:42:44 +03:00
Артем Стрельцов
a51440cc31 significantly improved performance 2020-05-05 21:49:22 +03:00
Артем Стрельцов
eb5488faeb std::unordered_map -> HashMap 2020-05-05 19:18:21 +03:00
FawnD2
ea87bd1c64 Merge branch 'master' into arrow-io-format 2020-05-05 18:52:44 +03:00
Vasily Nemkov
87b2d90725 extractAllGroups(haystack, re_needle) function 
Function that extracts from haystack all matching non-overlapping groups with regular expressions,
and put those into Array(Array(String)) column.

Example:
:) SELECT extractAllGroups('abc=111, def=222, "jkl mno"="444 foo bar"', '("[^"]+"|\\w+)=("[^"]+"|\\w+)');

┌─extractAllGroups('abc=111, def=222, "jkl mno"="444 foo bar"', '("[^"]+"|\\w+)=("[^"]+"|\\w+)')─┐
│ [['abc','def','"jkl mno"'],['111','222','"444 foo bar"']]                                      │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
 2020-05-05 17:41:31 +03:00
alexey-milovidov
114f09cfca
Merge pull request #10666 from NanoBjorn/refactor-volume 
Volumes and storages refactoring
 2020-05-05 16:25:26 +03:00
FawnD2
19022cb670 Remove redundant includes. Style fix 2020-05-05 15:56:54 +03:00
alexey-milovidov
c54f6d5a66
Merge pull request #10546 from kyprizel/master 
Lexer, ast, checksum and column specs fuzzers added
 2020-05-05 14:50:55 +03:00
alexey-milovidov
e908d1cd01
Revert "Partial fix of clang build stack unwinding (omit-frame-pointer)" 2020-05-05 14:36:32 +03:00
Alexey Ilyukhov
6a9355c95d Do not use grid for non const 2020-05-05 13:34:58 +03:00
FawnD2
aaf4f4ad61 Add missing Float16 for parquet format 2020-05-05 04:46:04 +03:00
alexey-milovidov
09756a036a
Merge pull request #10645 from ClickHouse/check-suspicious-codecs 
Check for suspicious codecs #4966
 2020-05-04 23:33:59 +03:00
Maxim Serebryakov
3691400a60 fix order of params in AggregatingTransform 2020-05-04 23:26:33 +03:00
Gleb Novikov
6e6a195c13 Style fixes, removed redundant includes 2020-05-04 23:20:51 +03:00
Gleb Novikov
c637d99e07 Volumes and storages refactoring: 
1. Moved Volume to separate file
  2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
  3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
  4. Removed old Volume completely
  5. Moved StoragePolicy to separated files
  6. Moved DiskSelector to separated files
  7. Removed DiskSpaceMonitor file
 2020-05-04 23:15:38 +03:00
Nikolai Kochetov
b23b02094f Added comments. 2020-05-04 22:17:52 +03:00
Nikolai Kochetov
ee65f3eedc Init all input ports in GroupingAggregatedTransform. 2020-05-04 21:07:56 +03:00
Nikolai Kochetov
4e27c33451 Init all input ports in GroupingAggregatedTransform. 2020-05-04 20:59:07 +03:00
Nikolai Kochetov
97dde2bcdf Fix limit for MergingSorted with fast-forward optimization. 2020-05-04 20:16:16 +03:00
alesapin
d47d28d51f Fix mutations assignment 2020-05-04 19:30:39 +03:00
FawnD2
f0201f7aca Move ORC to Input only formats 2020-05-04 18:04:20 +03:00
Alexey Milovidov
a198dce505 Fix double space 2020-05-04 17:55:37 +03:00
FawnD2
2f8435813c Remove unused stream 2020-05-04 17:49:13 +03:00
FawnD2
da0a908d27 Remove unused stream 2020-05-04 17:47:09 +03:00
Alexey Milovidov
87fb6bf081 Better exception message 2020-05-04 17:46:32 +03:00
FawnD2
b492e92d41 Remove extra check 2020-05-04 17:32:36 +03:00
FawnD2
182de47012 Use file reader for Arrow format 2020-05-04 17:30:42 +03:00
Alexey Milovidov
301c6137e4 Better exception messages; fixed error; updated tests 2020-05-04 17:25:54 +03:00
Azat Khuzhin
61c5df5662 Drop claimed from the KafkaBlockInputStream 2020-05-04 17:23:09 +03:00
Azat Khuzhin
a10cfc3ae5 Pass stream_cancelled to the copyData as stub in StorageKafka 2020-05-04 17:23:09 +03:00
Azat Khuzhin
c69217e9f3 Force destrying of kafka buffers from the shutdown in StorageKafka 2020-05-04 17:23:09 +03:00
Azat Khuzhin
9e14db2091 Add missing kafka settings into docs 2020-05-04 17:23:05 +03:00
Anton Okhitin
7366227115 Fix arcadia build 2020-05-04 16:47:08 +03:00
FawnD2
6d6d04903f Remove check 2020-05-04 16:33:20 +03:00
FawnD2
a044766d58 Remove check 2020-05-04 16:26:20 +03:00
FawnD2
17450811d4 Move getHeader at the beginning of generate() 2020-05-04 16:19:25 +03:00
FawnD2
83148289d8 Rename arg 2020-05-04 16:01:37 +03:00
Guillaume Tassery
0ab6936645 Add parser for handle offset without a limit clause 2020-05-04 19:52:38 +07:00
Артем Стрельцов
0f23713488 fixed bug with dictIsIn 2020-05-04 12:53:06 +03:00
alexey-milovidov
443c64abc0
Merge pull request #10638 from ClickHouse/fix-ubsan-hashtable 
Fix UBSan report in HashTable
 2020-05-04 04:50:25 +03:00
Артем Стрельцов
05d4e41949 minor fix 2020-05-04 04:28:54 +03:00
FawnD2
7b23844b32 Common order of precompiled checks 2020-05-04 03:58:10 +03:00
FawnD2
02e12215e7 Apply reducing memory usage optimization for seekable files to ORC format 2020-05-04 03:52:28 +03:00
alexey-milovidov
6b0bffcdab
Update CompressionFactory.cpp 2020-05-04 03:47:41 +03:00
Alexey Milovidov
99c18c5a09 Check for suspicious codecs #4966 2020-05-04 03:11:49 +03:00
FawnD2
a554177724 Simplify ORC format 2020-05-04 02:23:20 +03:00
alexey-milovidov
884c2aa631
Merge pull request #10641 from ClickHouse/storage-buffer-nullptr-dereference 
Fix nullptr dereference in StorageBuffer
 2020-05-04 02:01:57 +03:00
alexey-milovidov
788b436275
Merge pull request #10640 from ClickHouse/fix-http-code-parse-error 
Fix http code in case of some parse errors.
 2020-05-04 02:00:47 +03:00
FawnD2
112758b99d Merge branch 'master' into arrow-io-format 2020-05-04 00:53:17 +03:00
Alexey Milovidov
a5bdc375ca Fix Arcadia #10569 2020-05-04 00:36:11 +03:00
Alexey Milovidov
f8866af0c1 Fix nullptr dereference in StorageBuffer 2020-05-03 23:53:54 +03:00
FawnD2
2359ba3497 Correct stop condition 2020-05-03 22:59:19 +03:00
FawnD2
6b2e97fd56 Increate total read length while reading 2020-05-03 22:54:28 +03:00
FawnD2
b4898bdd19 Read single batch 2020-05-03 22:49:06 +03:00
Alexey Milovidov
bb6eabd82d Fix HTTP response code for some parse errors #10636 2020-05-03 22:47:07 +03:00
FawnD2
a8d914a811 Read full table 2020-05-03 22:22:41 +03:00
Артем Стрельцов
5bce553d91 performance optimization 2020-05-03 22:08:45 +03:00
Alexey Milovidov
8f7374c6de Fix another UBSan warning 2020-05-03 21:39:52 +03:00
FawnD2
b86562b8bc Style fixes 2020-05-03 21:29:04 +03:00
Alexey Milovidov
9a344e681f Fix UBSan report in FixedHashTable 2020-05-03 21:24:11 +03:00
Alexey Milovidov
626b9a3ae9 Fix UBSan report in HashTable 2020-05-03 21:21:18 +03:00
FawnD2
7cc7a87f9f Simplify interfaces 2020-05-03 21:12:14 +03:00
Артем Стрельцов
ff7b5db800 fixed bug with dictGetString 2020-05-03 21:03:41 +03:00
Артем Стрельцов
31fb42cf01 removed unused arena 2020-05-03 19:12:17 +03:00
Артем Стрельцов
f8d3999fa2 Merge branch 'master' of github.com:yandex/ClickHouse into layout_direct 2020-05-03 16:23:10 +03:00
FawnD2
a590826fbb Format settings for Arrow 2020-05-03 15:26:39 +03:00
FawnD2
e5b59616cd Style fix 2020-05-03 15:25:53 +03:00
alexey-milovidov
0cdd48e468
Merge pull request #10598 from ClickHouse/sampling-memory-profiler 
Add sampling memory profiler
 2020-05-03 15:17:03 +03:00
Azat Khuzhin
6ffdd53b6a Share auto-increment for first batch and tmp file in DistributedBlockOutputStream 2020-05-03 14:47:59 +03:00
Azat Khuzhin
53c470cab4 Fix directory monitor initialization from INSERT into Distributed 
This also fixes hardlink code (when one file should be sent to multiple
servers, i.e. internal_replication == false) of writeToShard() with
distributed_storage_policy (i.e. when StorageDistributed::getPath() will
path to different filesystems).

Plus also cleanup DistributedBlockOutputStream::writeToShard() a little.
 2020-05-03 14:47:51 +03:00
Azat Khuzhin
e97e1f06db Do not schedule distributed send if there were no error 
Since in this case it will be scheduled from the
DistributedBlockOutputStream with the
distributed_directory_monitor_max_sleep_time_ms, and this will overwrite
timer that was set by the DistributedBlockOutputStream, not good.
 2020-05-03 14:46:44 +03:00
Azat Khuzhin
947b3942dd Schedule distributed sends after the file has been written 2020-05-03 14:46:43 +03:00
Azat Khuzhin
0157fd5d93 Fix distributed send that are scheduled by INSERT query 
Before this patch each INSERT query re-schedule distributed send, thus
each time it resets the timer, while this is not the expected behaviour,
since in on frequent INSERT distributed sends will not be triggered at
all.

Fix this by not resetting the timer.
 2020-05-03 14:46:42 +03:00
Azat Khuzhin
6bb39dafc3 Drop decreated code (cond var and note for thread) in DirectoryMonitor 2020-05-03 14:46:41 +03:00
Alexey Milovidov
469cdc315b Added suggestion from @azat #10607 2020-05-03 14:40:37 +03:00
FawnD2
f62f1a5924 Add missed include 2020-05-03 06:08:37 +03:00
FawnD2
6e0b31fb2a Arrow output format 2020-05-03 05:46:36 +03:00
FawnD2
aeed62d5e7 Remove unused error codes 2020-05-03 04:20:03 +03:00
FawnD2
4d0225f5a9 std::string -> String 2020-05-03 04:15:09 +03:00
FawnD2
f4fe23e3fe Add ptr wrapper. Fix style 2020-05-03 04:01:05 +03:00
FawnD2
fa1c35753b Move CH -> Arrow routine converter into separated class 2020-05-03 03:54:39 +03:00
Alexey Milovidov
abe40e2127 Added required comment #10576 2020-05-03 01:34:50 +03:00
alexey-milovidov
a235533d5f
Merge pull request #10576 from Jokser/s3-with-dynamic-proxies 
S3 with dynamic proxy configuration
 2020-05-03 01:25:13 +03:00
alexey-milovidov
9011d18b80
Update S3Common.h 2020-05-03 01:25:02 +03:00
alexey-milovidov
d9400473f9
Update ThreadStatusExt.cpp 2020-05-03 01:20:14 +03:00
alexey-milovidov
67efc7f066
Merge pull request #10560 from Enmk/DateTime64_fixes 
Fixed comparing DateTime64 in WHERE against String value
 2020-05-03 00:54:11 +03:00
Артем Стрельцов
359de11fee removed redundant lines and comments 2020-05-03 00:49:19 +03:00
alexey-milovidov
342ed093de
Update S3Common.cpp 2020-05-02 23:46:39 +03:00
Артем Стрельцов
c82b1fc0fe fixed (hope) test flapping 2020-05-02 23:26:22 +03:00
Alexey Milovidov
554e7a0dd2 Preparations 2020-05-02 22:59:07 +03:00
Alexey Milovidov
e6ab4d655b Fix bad code 2020-05-02 22:54:29 +03:00
Alexey Milovidov
f0e80fec68 Whitespaces 2020-05-02 22:41:01 +03:00
alexey-milovidov
330627f597
Update ArrowBlockInputFormat.h 2020-05-02 22:35:58 +03:00
alexey-milovidov
47e3f83bc8
Update ArrowBlockInputFormat.h 2020-05-02 22:34:34 +03:00
alexey-milovidov
07f824c746
Update ArrowBlockInputFormat.cpp 2020-05-02 22:33:47 +03:00
alexey-milovidov
dfeb76038d
Update ArrowBlockInputFormat.h 2020-05-02 22:32:33 +03:00
alexey-milovidov
fbb280b5ac
Update ArrowBlockInputFormat.cpp 2020-05-02 22:32:21 +03:00
alexey-milovidov
db4c2352e1
Merge pull request #10569 from zhang2014/fix/ISSUES-10551 
ISSUES-10551 add backward compatibility for create bloom filter index
 2020-05-02 22:25:29 +03:00
alexey-milovidov
250a44a47f
Merge pull request #10607 from Avogar/json_as_string_format 
Add JSONAsStringRowInputFormat
 2020-05-02 22:17:22 +03:00
alexey-milovidov
229f666dea
Merge pull request #10611 from azat/optimize_skip_unused_shards-LowCardinality 
Fix optimize_skip_unused_shards with LowCardinality
 2020-05-02 22:14:33 +03:00
alexey-milovidov
4f1b115e76
Merge pull request #10610 from azat/optimize_skip_unused_shards-fixes 
Fix logging for optimize_skip_unused_shards
 2020-05-02 22:13:01 +03:00
alexey-milovidov
d38ea6fc1c
Merge pull request #10608 from ClickHouse/fix_drop_dns_cache 
Improve DROP DNS CACHE
 2020-05-02 22:10:58 +03:00
Артем Стрельцов
5ba6572210 fixed problem with dictGetString and Float32 2020-05-02 20:58:24 +03:00
tavplubix
a073cd6c79
Fix segfault in StorageBuffer when exception on server startup (#10609) 
* fix segfault in StorageBuffer when exception on server startup

* Trigger CI

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-05-02 19:54:20 +03:00
Alexey Ilyukhov
a03da4eb34 Fix style and array index 2020-05-02 19:48:36 +03:00
Alexey Milovidov
81adcc9058 Remove obsolete records from system.build_options 2020-05-02 17:00:03 +03:00
alexey-milovidov
d763d0019b
Merge pull request #10625 from amosbird/unwind 
Partial fix of clang build stack unwinding (omit-frame-pointer)
 2020-05-02 16:43:04 +03:00
Артем Стрельцов
2a4ce4f41c style fix 2020-05-02 15:02:57 +03:00
Amos Bird
014a1233ff
Partial fix of clang build stack unwinding 2020-05-02 18:52:29 +08:00
Alexey Ilyukhov
c12cd35099 Add point in polygon for non-const polygons 2020-05-02 11:46:18 +03:00
Артем Стрельцов
25d7f76e4f fixed bug & created functional test for new layout 2020-05-02 04:03:29 +03:00
Артем Стрельцов
97496048d6 created direct dictionary (has bugs, doesn't work properly) 2020-05-02 02:35:54 +03:00
Matwey V. Kornilov
d7627fa4fb Fix capnproto version check for capnp::UnalignedFlatArrayMessageReader 
capnp::UnalignedFlatArrayMessageReader was introduced in
3aa2b2aa02
(which is a part of 0.7.0 release). Unfortunately,
capnp::UnalignedFlatArrayMessageReader was removed in
3f0fee61c6
(which is a part of 0.8.0 release)

So change CAPNP_VERSION check accordingly.
 2020-05-01 22:32:06 +03:00
Alexey Milovidov
439777ef23 Allow to set memory_profiler_sample_probability per query 2020-05-01 22:17:56 +03:00
Alexey Milovidov
93c3a83af2 Remove useless code 2020-05-01 22:01:02 +03:00
Alexey Milovidov
600b396f1b Merge remote-tracking branch 'origin/master' into sampling-memory-profiler 2020-05-01 21:48:20 +03:00
Alexander Kazakov
e9baaa439b
Implementation of new system metrics provider (Procfs) (#10544) 
* New metrics provider (Procfs) + Refactored TasksStatsCounters

* Trivial statless test that ProcFS is provided

* Trivial perf test for ProcfsMetricsProvider

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-05-01 21:47:41 +03:00
alexey-milovidov
404452bad3
Update MergeTreeIndices.cpp 2020-05-01 21:40:56 +03:00
Avogar
3de0bca856 Remove double space 2020-05-01 19:50:18 +03:00
Avogar
9670e74885 Update makeContinuousMemoryFromCheckpointToPos in PeekableReadBuffer. 2020-05-01 19:42:15 +03:00
Avogar
697d5b07ce Fix build and style errors. Update test. 2020-05-01 19:41:14 +03:00
Alexey Milovidov
c4b64edc14 Merge branch 'master' into sampling-memory-profiler 2020-05-01 16:47:10 +03:00
Alexey Milovidov
6ba6d5fc69 Revert unrelated modification 2020-05-01 16:46:42 +03:00
Vasily Nemkov
86edceeb81 Comparison between DateTime64 and String values (just like for DateTime) 2020-05-01 15:52:18 +03:00
Vitaly Baranov
2528e72199
Merge pull request #10587 from vitlibar/database-with-dictionary-init-in-constructor 
Move initialization of DatabaseWithDictionaries to constructor.
 2020-05-01 14:08:53 +03:00
alesapin
4d8660f270 Remove obsolete comment 2020-05-01 13:05:37 +03:00
Azat Khuzhin
63d8ab8f03 Make createSelector() static (in storage) and const (in stream) 2020-05-01 11:31:05 +03:00
Alexander Kuzmenkov
2ac26bd3a5 Disable mremap under MemorySanitizer. 2020-05-01 10:38:05 +03:00
Azat Khuzhin
f22ba15b4a Reduce copy-paste of DistributedBlockOutputStream::createSelector 
This will make it less error prone.
 2020-05-01 02:59:40 +03:00
Azat Khuzhin
cdd7013438 Drop superfluous "Skipping irrelevant shards" messages 
Before this patch it printed 3 times:
- from StorageDistributed::getProcessingStageImpl()
- from StorageDistributed::read()
- from StorageDistributed::getProcessingStageImpl() (from StorageDistributed::read() -> getSampleBlock())
(But this should be optimized)
 2020-05-01 02:56:13 +03:00
Azat Khuzhin
c648c300bf Fix optimize_skip_unused_shards with LowCardinality 2020-05-01 02:39:58 +03:00
Azat Khuzhin
4cbe625567 Fix shard numbers output in logs (full cluster had been printed over optimized) 2020-05-01 02:13:07 +03:00
Alexander Tokmakov
9ca144cab9 drop access cache on DROP DNS CACHE 2020-05-01 01:29:47 +03:00
Avogar
6464a1a902 Add JSONAsStringRowInputFormat 2020-04-30 23:16:09 +03:00
Vitaly Baranov
dd34cd73e2 Move initialization of DatabaseWithDictionaries to constructor. 2020-04-30 20:22:20 +03:00
Nikolai Kochetov
10afbc22b5 Fix build. 2020-04-30 16:36:20 +03:00
Alexey Milovidov
db3241ccc7 Add sampling memory profiler 2020-04-30 16:30:10 +03:00
Alexey Milovidov
293ae88e7f Add sampling memory profiler 2020-04-30 16:25:17 +03:00
Nikolai Kochetov
62e9be7ba8 Fix build. 2020-04-30 14:33:27 +03:00
Nikolai Kochetov
19dadb8c2d Add parallel final. 2020-04-30 12:59:08 +03:00
alesapin
15e8f37839
Merge pull request #10531 from CurtizJ/polymorphic-parts-2 
Fix index corruption in merges with compact parts.
 2020-04-30 12:15:25 +03:00
Alexander Kuzmenkov
7f5b57c6c5
Merge pull request #10559 from ClickHouse/aku/summap-experiment 
Merge pull request #10000 (sumMap for Tuple argument)
 2020-04-30 11:56:04 +03:00
alesapin
5e2afad549
Merge pull request #10477 from ClickHouse/refactor_istorage 
Remove getColumn and hasColumn from IStorage, better virtuals.
 2020-04-30 10:42:52 +03:00
Anton Popov
6ed3092c8a make fillIndexGranularity less complicated 2020-04-30 00:57:58 +03:00
Pavel Kovalenko
0b70c2ca08 Fixed ya.make for S3. 2020-04-29 21:38:35 +03:00
alesapin
47e32aaaa1
bump CI 2020-04-29 21:17:15 +03:00
alesapin
7a8380a682 Fix build 2020-04-29 19:26:01 +03:00
Potya
351d62aa83 Nothing interesting 2020-04-29 15:17:40 +03:00
alesapin
69fdf2a6a3 Review fixes and better test 2020-04-29 15:15:23 +03:00
Pavel Kovalenko
f17e022fa4 Code style fixes. 2020-04-29 15:05:46 +03:00
Nikolai Kochetov
a9d63d4520
Merge pull request #10498 from vzakaznikov/insert_into_watch_from_live_view 
Adding support for INSERT INTO table WATCH query to build streaming systems using LIVE VIEW tables
 2020-04-29 12:15:54 +03:00
Pavel Kovalenko
a1acbcb019 Code style fixes. 2020-04-29 11:45:13 +03:00
Pavel Kovalenko
e5e84960ea S3 with proxy integration test and fixes. 2020-04-29 11:40:29 +03:00
Pavel Kovalenko
2ecf4b5a23 Dynamic proxy configuration for DiskS3. 2020-04-29 11:40:29 +03:00
Alexander Kuzmenkov
c5a7796d16 fixup 2020-04-29 09:35:02 +03:00
Alexander Kuzmenkov
31bf2b3be4 fixup 2020-04-29 09:05:52 +03:00
zhang2014
8ac709b5d3 ISSUES-10551 add backward compatibility for create bloom filter index 2020-04-29 11:57:28 +08:00
Azat Khuzhin
038235684d Add optimize_distributed_group_by_sharding_key and disable it by default 
I know at least one way to fool that optimization, by using as sharding
key something like `if(col1>0, col1, col2)` (although this is not common
sharding key I would say, but can be useful if this will work
correctly), so let's disable it by default.
 2020-04-29 00:09:25 +03:00
FawnD2
538e6c39da Arrow input format 2020-04-28 22:52:22 +03:00
Azat Khuzhin
53aea23de0 Fix SELECT of column ALIAS which default expression type different from column type 2020-04-28 22:43:10 +03:00
robot-clickhouse
de4e88ff97 Auto version update to [20.5.1.1] [54435] 2020-04-28 20:12:16 +03:00
Vitaliy Zakaznikov
664a09e55d Simplifying code related to auto set output_format_enable_streaming option. 2020-04-28 18:37:15 +02:00
Vitaliy Zakaznikov
4749122734 Adding support to auto set output_format_enable_streaming option 
based on the query type.
 2020-04-28 17:50:50 +02:00
Artem Zuikov
0b18b6104f add file to ya.make 2020-04-28 18:38:07 +03:00
Alexander Kuzmenkov
73398a3e17 fixup 2020-04-28 17:30:45 +03:00
Artem Zuikov
e34a9457e6
Improve right table insert for Partial MergeJoin on disk (#10467) 2020-04-28 16:55:50 +03:00
alesapin
30052ea46c Better 2020-04-28 16:00:41 +03:00
alesapin
80443af5dc Fix 2020-04-28 15:57:44 +03:00
alesapin
7c25a9bcbd Fix schema of storage 2020-04-28 15:33:17 +03:00
bobrovskij artemij
0cc3d780b3 Support write for XDBCStorage 2020-04-28 14:03:24 +03:00
alesapin
f981649213 Fix pushing to views stream and refactor virtuals 2020-04-28 13:38:57 +03:00
Eldar Zaitov
043be916cb Add miniman fuzzing documentation, fix code style. 2020-04-28 13:35:44 +03:00
Eldar Zaitov
f40603441b Add miniman fuzzing documentation, fix code style. 2020-04-28 13:34:57 +03:00
tavplubix
11c3493676
Merge pull request #10547 from ClickHouse/zhang2014/feature/ISSUES-5436 
Merging #7572
 2020-04-28 12:40:48 +03:00
bobrovskij artemij
1bcd077fbf Support write for XDBCStorage 2020-04-28 03:56:44 +03:00
Eldar Zaitov
670ed4bf1b lexer, ast, checksum and column specs fuzzers added 2020-04-28 02:12:18 +03:00
Alexander Tokmakov
e72a484beb minor fixes 2020-04-28 02:05:15 +03:00
Nikolai Kochetov
de6d2d01b9 Try to fix tests. 2020-04-27 21:15:55 +03:00
Anton Popov
032a0aa282 fix clang-tidy 2020-04-27 21:12:17 +03:00
alesapin
7c2f4d121a Fix func 2020-04-27 20:48:53 +03:00
alesapin
4badd0fd28 Better code 2020-04-27 20:46:51 +03:00
alesapin
228dbf475e
Finalize mutations in background (#10526) 2020-04-27 19:19:04 +03:00
alesapin
01db4877f6 Fix style check 2020-04-27 18:44:33 +03:00
alesapin
c24ab563d7 Fix several wrong usages 2020-04-27 18:38:35 +03:00
Nikolai Kochetov
2f06180c5e Revert changes for CSVRowOutputFormat. 2020-04-27 18:21:53 +03:00
alesapin
b0c5c1d685 Fix style 2020-04-27 18:21:37 +03:00
alesapin
6cfe0f7046 Fix after refactoring 2020-04-27 18:20:03 +03:00
alesapin
cefc58e77c Fix style 2020-04-27 18:17:08 +03:00
alesapin
5c6316afbc Fix bug 2020-04-27 18:10:50 +03:00
Nikolai Kochetov
201a184d31 Enable auto flust for output formats if enable_streaming was set. 2020-04-27 18:00:10 +03:00
Nikolai Kochetov
32baa073b5 Merge remote-tracking branch 'upstream/master' into insert_into_watch_from_live_view 2020-04-27 17:49:50 +03:00
alesapin
4c1ac83112 Fix in IStorage 2020-04-27 17:44:32 +03:00
alesapin
e85b50c911 Revert "Strange fix, how it was possible earlier?" 
This reverts commit 8d20644c99.
 2020-04-27 17:17:12 +03:00
alesapin
8d20644c99 Strange fix, how it was possible earlier? 2020-04-27 17:12:54 +03:00
alesapin
18c550df15 Better virtuals logic 2020-04-27 16:55:30 +03:00
Anton Popov
bee343dae1 fix writing of index in compact parts 2020-04-27 16:36:50 +03:00
alesapin
2829774105 Merge branch 'master' into refactor_istorage 2020-04-27 15:34:21 +03:00
alexey-milovidov
436e4e3f5e
Merge pull request #10502 from oandrew/avro-nested2 
Avro nested fields - support complex types
 2020-04-27 13:44:58 +03:00
Andrew Onyshchuk
570be94a48 Use correct type for DateTime64 (ubsan) 2020-04-26 21:10:30 -05:00
Azat Khuzhin
20b4eed9a1 Disable GROUP BY sharding_key optimization for WITH ROLLUP/CUBE/TOTALS 2020-04-27 01:30:54 +03:00
Alexey Milovidov
cac130ad3b Use time zone when comparing DateTime with string literal 2020-04-27 01:13:51 +03:00
Alexey Milovidov
a7d7dc5034 Fix some bad code 2020-04-26 20:34:36 +03:00
alexey-milovidov
c6ae6a7ba8
Merge pull request #10418 from ClickHouse/fix_logical_error_in_diagnostic_info 
Better diagnostic info in input formats
 2020-04-26 20:26:59 +03:00
alexey-milovidov
4c17542096
Merge pull request #10465 from kekekekule/ddl_settings 
Add custom settings support in DDL-queries for CREATE DICTIONARY
 2020-04-26 20:25:29 +03:00
alexey-milovidov
7d58dc154c
Update ASTDictionary.h 2020-04-26 20:24:18 +03:00
alexey-milovidov
0a9bcae0ed
Merge pull request #10511 from ClickHouse/fix-ubsan-datetime64 
Fixed UBSan for DateTime64; added a test
 2020-04-26 20:17:01 +03:00
alexey-milovidov
378d73d477
Merge pull request #10449 from azat/metrics-for-new-bg-schedule-pools 
Add tasks/memory metrics for distributed/buffer schedule pools
 2020-04-26 20:16:02 +03:00
Alexey Milovidov
79af5cd0ae Fix UBSan report in Decimal parse 2020-04-26 18:48:30 +03:00
Alexey Milovidov
a70637ab4b Fixed UBSan; added a test 2020-04-26 17:57:45 +03:00
alexey-milovidov
d1464a441f
Merge pull request #10503 from oandrew/read-columns-perf-fix 
IRowInputFormat performance fix
 2020-04-26 17:36:54 +03:00
alexey-milovidov
62e2f10959
Merge pull request #10474 from hczhcz/patch-0422 
Fix overflow in simpleLinearRegression
 2020-04-26 17:34:57 +03:00
Vitaliy Zakaznikov
626468d255 Adding support for streaming mode to JSONEachRowRowOutputFormat 2020-04-26 15:56:45 +02:00
Vitaliy Zakaznikov
369b4d53ef Adding support for output_format_enable_streaming format setting. 2020-04-26 15:44:11 +02:00
Andrew Onyshchuk
4a5497498e IRowInputFormat performance fix 2020-04-25 19:33:36 -05:00
Alexey Milovidov
d01bd44f14 Merge branch 'master' into execute-query-flush-format-2 2020-04-26 03:31:46 +03:00
alexey-milovidov
8820b43606
Merge pull request #10479 from Avogar/markdown_format 
Markdown format
 2020-04-26 03:25:48 +03:00
Andrew Onyshchuk
e7ba75ce98 Avro nested fields - support complex types 2020-04-25 17:42:49 -05:00
alexey-milovidov
8b1e504b19
Merge pull request #10497 from azat/kafka 
Tiny kafka fixes
 2020-04-25 23:50:22 +03:00
alexey-milovidov
092efc53ba
Merge pull request #10496 from ClickHouse/fix-visible-drift-memory-tracking 
Fix visible drift of MemoryTracking metric
 2020-04-25 23:50:08 +03:00
alexey-milovidov
1e25991c62
Merge pull request #10494 from kyprizel/master 
Split fuzzers and sanitizers
 2020-04-25 23:49:45 +03:00
alexey-milovidov
c9334d3fde
Merge pull request #10491 from azat/dist-shutdown 
Proper Distributed shutdown (fixes UAF, avoid waiting for sending all batches)
 2020-04-25 23:47:59 +03:00
Vitaliy Zakaznikov
71a40d6ec9 Revert "Working on adding AUTO REFRESH to LIVE VIEW tables." 
This reverts commit 855e06b031.
 2020-04-25 09:56:57 -04:00
Vitaliy Zakaznikov
1fc345f5a0 Revert "Reverting all the changes in LIVE VIEW table." 
This reverts commit 684c7f943a.
 2020-04-25 09:56:41 -04:00
Vitaliy Zakaznikov
e73166c78d Removing SquashingBlockOutputStream from the stream to 
allow blocks from WATCH query to propagate to target table
as is.
 2020-04-25 09:46:46 -04:00
Alexey Milovidov
c75c2fac29 Fix visible drift of MemoryTracking metric 2020-04-25 15:36:01 +03:00
Azat Khuzhin
40c65f7bee IRowInputFormat: omit "Skipped 0 rows with errors while reading the input stream" 
This will just pollute the output log.

And this is significant for kafka streaming, since it will print this
message for each message (due to how
ReadBufferFromKafkaConsumer/KafkaBlockInputStream works).

You can see this in the server log of the test_storage_kafka
(integration test), for the table with kafka_skip_broken_messages > 0,
it will have 52 lines of such message, 2 with "Skipped 1 rows" and 50
with "Skipped 0 rows" (since there are 52 messages in summary, 2
malformed and 50 correct).
 2020-04-25 15:16:29 +03:00
Vitaliy Zakaznikov
684c7f943a Reverting all the changes in LIVE VIEW table. 2020-04-25 07:39:15 -04:00
Vitaliy Zakaznikov
8621e93460 Starting to remove changes to LIVE VIEW table. 2020-04-25 07:37:33 -04:00
Vitaliy Zakaznikov
34a03939ea First changes to try to support INSERT INTO table WATCH query. 2020-04-25 07:33:47 -04:00
Eldar Zaitov
d7a037c26b split fuzzers and sanitizers 2020-04-25 10:12:38 +03:00
Azat Khuzhin
747a74215f Avoid processing all batches before Distributed shutdown 2020-04-25 02:03:27 +03:00
Azat Khuzhin
8ad6b37913 Proper StorageDistributed shutdown to avoid UAF in DistributedMonitor 
StorageDistributed::shutdown() does not acquire the lock, that controls
access to the cluster_nodes_data, thus it does not synced with the
requireDirectoryMonitor(), hence some monitors can be untracked that
will trigger UAF (use-after-free) after DROP TABLE dist:

This is for the SIGSEGV from the DirectoryMonitor (with already destroyed storage):
    0  0x0000000008e9f760 in std::__1::__cxx_atomic_load<int> (__order=std::__1::memory_order::seq_cst, __a=0x0)
    1  std::__1::__atomic_base<int, false>::load (__m=std::__1::memory_order::seq_cst, this=0x0) <-- this is nullptr
    2  std::__1::__atomic_base<int, false>::operator int (this=0x0)
    3  DB::ActionBlocker::isCancelled (this=0x7f85e31c9bb8) at ../src/Common/ActionBlocker.h:18
    4  DB::StorageDistributedDirectoryMonitor::run (this=0x7f85f93b2a00) at ../src/Storages/Distributed/DirectoryMonitor.cpp:140
 2020-04-25 02:03:26 +03:00
Vitaliy Zakaznikov
855e06b031 Working on adding AUTO REFRESH to LIVE VIEW tables. 2020-04-24 17:15:17 -04:00
tavplubix
7928f2eaa1
Merge pull request #7512 from ClickHouse/database_atomic 
DatabaseAtomic
 2020-04-24 22:28:18 +03:00
Alexander Tokmakov
04d4130b0d update yamake 2020-04-24 20:14:10 +03:00
Артем Стрельцов
5423506393 sure this will exactly fix the fail with build check 2020-04-24 19:04:54 +03:00
Alexander Tokmakov
53dcce55e9 better diagnostic info in input formats 2020-04-24 15:03:48 +03:00
Avogar
778f1aa1e7 Merge branch 'markdown_format' of github.com:avogar/ClickHouse into markdown_format 2020-04-24 14:07:04 +03:00
Avogar
2458da802c Update MarkdownRowOutputFormat 2020-04-24 14:06:41 +03:00
alesapin
793f4b734a Remove obsolete comment 2020-04-24 13:31:03 +03:00
alesapin
b4425c2dc4 Remove getColumn and hasColumn methods from IStorage 2020-04-24 13:20:03 +03:00
Артем Стрельцов
26a6353284 tried to fix build fail & fixed codestyle 2020-04-24 12:59:49 +03:00
alesapin
0c622370c5 Better 2020-04-24 12:21:38 +03:00
alesapin
dc2dd77d2e Remove redundant overrides from IStorage 2020-04-24 12:20:09 +03:00
alesapin
a2cb67c542
Merge pull request #10459 from ClickHouse/refactor_istorage 
Remove virtuals from IStorage as separate column
 2020-04-24 11:42:04 +03:00
hcz
541cd638ba Fix overflow in simpleLinearRegression 2020-04-24 16:33:09 +08:00
alexey-milovidov
79d531af87
Merge pull request #10466 from ClickHouse/aku/arena-infinite-loop 
Work around a bug leading to an infinite loop in addressToLine
 2020-04-24 10:38:07 +03:00
Alexander Kuzmenkov
2bedc82c18 Work around a bug leading to an infinite loop in addressToLine. 
Fixing this is hopeless and would entail rewriting the entire
WriteBuffer thing, and also this hack is something we can backport.
 2020-04-24 02:42:04 +03:00
Артем Стрельцов
aab2e8f9c1 unsignificant change 2020-04-24 02:21:51 +03:00
tavplubix
7d6ab32eb6
Update DatabaseCatalog.cpp 2020-04-24 01:08:26 +03:00
Артем Стрельцов
d7e27d4326 added SETTINGS section to DDL-query for CREATE DICTIONARY 2020-04-24 00:57:40 +03:00
Артем Стрельцов
f39fdf7182 some progress in parsing settings (i did this commit for having something to backup to) 2020-04-23 23:50:01 +03:00
Alexander Tokmakov
8be4d51111 fix build 2020-04-23 22:50:05 +03:00
Alexey Milovidov
2b569cf260 Added clarification in exception message #10405 2020-04-23 21:31:37 +03:00
Alexander Tokmakov
15e7ce0053 fixes 2020-04-23 21:00:43 +03:00
Alexander Tokmakov
f413061e0d remove Context from some IDatabase methods 2020-04-23 19:51:48 +03:00
alesapin
5ae25b68e4 Remove redundant virtual from function 2020-04-23 18:18:33 +03:00
alesapin
b0a73589b0 Remove virtual columns as separate field from IStorage 2020-04-23 18:09:00 +03:00
Alexander Tokmakov
04d6b59ac0 Merge branch 'master' into database_atomic 2020-04-23 17:31:37 +03:00
Артем Стрельцов
5da371b9c0 added settings parsing from DDL-query (they aren't applied now) 2020-04-23 15:39:27 +03:00
alexey-milovidov
1ecfa27cd3
Merge pull request #10441 from ClickHouse/remove_casts_from_defaults 
Remove CAST from DEFAULT expressions during table creation.
 2020-04-23 12:36:12 +03:00
Nikita Mikhaylov
3520755ab5 fix c++20 usage 2020-04-23 12:00:43 +03:00
Azat Khuzhin
4f5ec14f27 Configure thread names for buffer/distributed schedule pools 2020-04-23 03:16:14 +03:00
Azat Khuzhin
d854049d21 Add tasks/memory metrics for distributed/buffer schedule pools 
Follow-up-for: #10315
Follow-up-for: #10263
 2020-04-23 02:18:37 +03:00
Azat Khuzhin
c04c73b4c4 Drop unused MemoryTrackingForMerges from Context 
It set in src/Storages/MergeTree/MergeList.cpp anyway.
 2020-04-23 01:57:33 +03:00
alexey-milovidov
77969d11a5
Merge pull request #10442 from zhang2014/fix/create_database_failure 
Not remove metadata directory when attach database fails
 2020-04-23 01:07:13 +03:00
alexey-milovidov
b8e3655f0d
Merge pull request #10445 from azat/logger-source-coloring-client 
Make logger name colored in client too
 2020-04-23 00:35:40 +03:00
alexey-milovidov
fd27ce619b
Merge pull request #10444 from ClickHouse/server-memory-profiler 
Simple server wide memory profiler
 2020-04-23 00:34:23 +03:00
alexey-milovidov
4f6bb00424
Merge pull request #10420 from ClickHouse/clang-tidy-8 
Fixes for clang-tidy on clang-10
 2020-04-23 00:33:41 +03:00
Alexander Tokmakov
e02b5790af fix 2020-04-22 23:49:13 +03:00
Alexander Tokmakov
1833ac7f16 fixes 2020-04-22 23:43:10 +03:00
Nikita Mikhaylov
3cf2016867
Merge pull request #10188 from nikitamikhaylov/quorum_bug 
insert_quorum vs drop partition or ttl
 2020-04-23 00:11:57 +04:00
alexey-milovidov
9e14603ae5
Update DatabaseFactory.cpp 2020-04-22 22:37:30 +03:00
Azat Khuzhin
8715cef94c Make logger name colored in client too 
Follow-up-for: #10343
 2020-04-22 22:34:35 +03:00
Alexey Milovidov
282b5785c1 Fixed error 2020-04-22 21:41:19 +03:00
Alexander Kuzmenkov
3d5809a4ac
Merge pull request #10340 from ClickHouse/aku/unstable-codec-test 
Copy less memory in squashing transform.
 2020-04-22 21:18:42 +03:00
Alexey Milovidov
73dd78bac1 Merge branch 'master' into clang-tidy-8 2020-04-22 21:04:47 +03:00
Alexey Milovidov
07dcf40a6a Simple server wide memory profiler 2020-04-22 20:52:21 +03:00
zhang2014
a622208284 Not remove metadata directory when attach database fails 2020-04-23 00:03:58 +08:00
Alexey Milovidov
d189375051 Fix Mac OS build 2020-04-22 17:56:31 +03:00
alesapin
45b8cb5b37 Remove casts from table creation by default 2020-04-22 17:29:38 +03:00
Alexander Tokmakov
fefbbd37df Merge branch 'master' into database_atomic 2020-04-22 17:02:30 +03:00
zhang2014
0a0555d4d7 ISSUES-9623 fix failure test case 2020-04-22 18:34:42 +08:00
Alexey Milovidov
d3418bbb02 Fix clang-tidy 2020-04-22 11:45:14 +03:00
Alexey Milovidov
8c9408f927 Checkpoint 2020-04-22 11:31:10 +03:00
zhang2014
bfb87f0eab ISSUES-9623 check the number and type of arguments when creating BloomFilter index 2020-04-22 16:17:09 +08:00
Alexey Milovidov
03e17405cd Checkpoint 2020-04-22 10:03:43 +03:00
Alexey Milovidov
603d132d45 Checkpoint 2020-04-22 09:34:20 +03:00
Alexey Milovidov
1e325a9fd9 Checkpoint 2020-04-22 09:22:14 +03:00
alexey-milovidov
7065a8ba9c
Merge pull request #10422 from azat/fallback_to_stale_replicas-over-skip_unavailable_shards 
Prefer fallback_to_stale_replicas over skip_unavailable_shards
 2020-04-22 09:20:25 +03:00
Alexey Milovidov
0a7edce036 Checkpoint 2020-04-22 09:01:33 +03:00
Alexey Milovidov
be22a4b94e Checkpoint 2020-04-22 08:39:31 +03:00
Alexey Milovidov
b9b5036749 Merge branch 'master' into clang-tidy-8 2020-04-22 07:54:27 +03:00
Alexey Milovidov
e6acb35d95 Added a fix from Vadim Plakhtinskiy @VadimPlh 2020-04-22 07:32:05 +03:00
alexey-milovidov
eb0a1ec755
Merge pull request #10415 from azat/dict-load-after-detach-attach 
Fix dictionary name (RELOAD and system.dictionaries) after DETACH/ATTACH
 2020-04-22 06:57:08 +03:00
alexey-milovidov
7690c34e5b
Merge pull request #10424 from ClickHouse/tsv-bom-extra-case 
Allow to parse BOM in TSV if the first column cannot contain BOM in its value
 2020-04-22 06:56:43 +03:00
Alexander Kuzmenkov
d1c5e38a21
Merge pull request #10385 from CurtizJ/tuple-literal 
Parse tuples of tuples as literals
 2020-04-22 05:49:14 +03:00
alexey-milovidov
98294dc9fa
Merge pull request #10421 from ClickHouse/max_server_memory_usage_as_server_level_setting 
Make max_server_memory_usage server level setting.
 2020-04-22 03:34:17 +03:00
Alexey Milovidov
c805ed4dea Fixes for clang-tidy on clang-10 2020-04-22 03:29:38 +03:00
Alexey Milovidov
e8a9c7d518 Allow to parse BOM in TSV if the first column cannot contain BOM in its value 2020-04-22 02:46:42 +03:00
Alexander Kuzmenkov
c232f641b4
Merge branch 'master' into aku/unstable-codec-test 2020-04-22 02:40:53 +03:00
alexey-milovidov
0754eb7e78
Merge pull request #10416 from 4ertus2/perf 
Improve HashJoin memory usage
 2020-04-22 01:44:46 +03:00
Azat Khuzhin
0cc9dbd35b Prefer fallback_to_stale_replicas over skip_unavailable_shards 
Patch from: @alex-zaitsev
 2020-04-22 01:42:23 +03:00
Alexey Milovidov
0482e2e3ea Implement suggestion from Sergey Veletsky 2020-04-22 01:40:18 +03:00
alexey-milovidov
fcf5c3c6e1
Merge pull request #10399 from ClickHouse/fix-distributed_group_by_no_merge-segfault 
Fix segfault with distributed_group_by_no_merge
 2020-04-22 01:28:02 +03:00
Ivan
555488a458
Fix build in Arcadia (#10413) 2020-04-21 23:23:44 +03:00
Artem Zuikov
96b9edae29 fix comment 2020-04-21 22:09:18 +03:00
Artem Zuikov
17b0641b02 improve HashJoin memory usage 2020-04-21 22:01:34 +03:00
Alexey Milovidov
33338b722a Fix Mac OS #10246 2020-04-21 21:49:28 +03:00
Azat Khuzhin
9173895f85 Fix dictionary name (RELOAD and system.dictionaries) after DETACH/ATTACH 
After DETACH/ATTACH (or at server startup) the following will be broken:
- SYSTEM RELOAD DICTIONARY db_name.dict_name
- system.dictionaries will not have database for those dictionaries

The problem is that after DETACH/ATTACH the dictionary information is
obtained from metadata/$db/$dict.sql, but it does not have database
name.

Fix this by passing database name to the getDictionaryConfigurationFromAST().

Introduced-in: #10234
 2020-04-21 21:03:12 +03:00
Alexander Kuzmenkov
fcfbe4bfea
Update SquashingTransform.h 2020-04-21 19:27:56 +03:00
Alexander Kuzmenkov
a32ef00bf1 less copying 2020-04-21 17:59:57 +03:00
Anton Popov
3ccff833c1 fix ConstantExpressionTemplate with arrays 2020-04-21 15:54:36 +03:00
Nikita Mikhaylov
2173bcf28e Merge branch 'master' of github.com:ClickHouse/ClickHouse into quorum_bug 2020-04-21 14:35:06 +03:00
Nikolai Kochetov
87a0b40ace Do not create extra totals and extremes ports. 2020-04-21 12:31:40 +03:00
Nikolai Kochetov
1d2388de39 Drop currently calculated totals and extremes before aggregation. 2020-04-21 11:45:28 +03:00
Alexey Milovidov
4cc0269de0 Fix flattening of Array(Tuple(...)) #10259 2020-04-21 02:44:51 +03:00
alexey-milovidov
4ecc86beca
Merge pull request #9950 from ClickHouse/aku/literal-columns 
Create unique columns for literals
 2020-04-21 02:30:17 +03:00
alexey-milovidov
2645dede2c
Merge pull request #10386 from palasonic1/palasonic-fix-read-hash-table 
Fix reading HashTable from buffer
 2020-04-21 02:26:47 +03:00
tavplubix
1a067484c8
Merge pull request #10137 from kekekekule/direct_queries 
Add custom settings support for certain dictionary sources
 2020-04-21 01:21:32 +03:00
Alexander Tokmakov
17c0317b80 minor fixes 2020-04-21 01:17:53 +03:00
alexey-milovidov
8c2839d3c9
Merge pull request #10308 from ClickHouse/shared-context-lifetime 
Shared context lifetime
 2020-04-21 00:12:45 +03:00
alexey-milovidov
fe1309e575
Merge pull request #10381 from ClickHouse/fix-concat-prepare 
Fix concat prepare
 2020-04-21 00:07:32 +03:00
alexey-milovidov
ab922d44d5
Merge pull request #10379 from ClickHouse/only_fetch_repliac 
Add setting always_fetch_merged_part
 2020-04-21 00:07:00 +03:00
alexey-milovidov
2c8576798b
Merge pull request #10354 from oandrew/avro-nested 
Avro nested fields insert support
 2020-04-20 23:36:02 +03:00
alexey-milovidov
3d4696d4ef
Update AvroRowInputFormat.cpp 2020-04-20 23:34:48 +03:00
alexey-milovidov
c4758b69a2
Merge pull request #10378 from hczhcz/patch-0420 
Add JSONExtractKeysAndValuesRaw
 2020-04-20 23:30:40 +03:00
alexey-milovidov
4912940009
Update FunctionsJSON.h 2020-04-20 23:28:54 +03:00
alexey-milovidov
2d799cf096
Merge pull request #10375 from azat/aggregator-drop-disk-reservation 
Drop disks reservation in Aggregator
 2020-04-20 23:22:14 +03:00
Maxim Serebryakov
a99a340b0b fix reading HashTable and HashSet 2020-04-20 23:19:03 +03:00
Anton Popov
e1b7e2de2f parse tuples of tuples as literals 2020-04-20 23:14:56 +03:00
alexey-milovidov
69b6f135e7
Merge pull request #10382 from excitoon-favorites/altercodec 
Allowed to alter column in non-modifying data mode when the same type is specified
 2020-04-20 22:54:53 +03:00
alexey-milovidov
f6de9c6ffd
Update AlterCommands.cpp 2020-04-20 22:54:08 +03:00
alexey-milovidov
1eca5088ec
Update AlterCommands.cpp 2020-04-20 22:53:19 +03:00
alexey-milovidov
faeab5f649
Merge pull request #10370 from amosbird/clang-10 
clang-10 fix
 2020-04-20 22:47:12 +03:00