thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-22 23:52:03 +00:00
Code Issues Packages Projects Releases Wiki Activity
59,902 Commits 9,163 Branches 1,781 Tags 2.1 GiB
7c580d8a01
Commit Graph

4523 Commits

Author SHA1 Message Date
Azat Khuzhin
6965ac26c3 Distributed: Add ability to delay/throttle INSERT until pending data will be reduced 
Add two new settings for the Distributed engine:
- bytes_to_delay_insert
- max_delay_to_insert

If at the beginning of INSERT there will be too much pending data, more
then bytes_to_delay_insert, then the INSERT will wait until it will be
shrinked, and not more then max_delay_to_insert seconds.

If after this there will be still too much pending, it will throw an
exception.

Also new profile events were added (by analogy to the MergeTree):
- DistributedDelayedInserts (although you can use system.errors instead
  of this, but still)
- DistributedRejectedInserts
- DistributedDelayedInsertsMilliseconds
 2021-03-03 23:30:23 +03:00
Azat Khuzhin
cabe4ca1bb tests: split 00753_system_columns_and_system_tables (to disable Distributed part for arcadia) 2021-03-03 23:30:03 +03:00
Azat Khuzhin
3102835d56 tests: Use total_bytes>0 for Distributed engine in 00753_system_columns_and_system_tables 2021-03-03 23:30:03 +03:00
Azat Khuzhin
b43046ba06 Distributed: More accurate distribution_queue counters 
So now system.distribution_queue will show accurate statistics, so tests
does not requires sleep anymore.

But note that with too much distributed pending this will iterate over
all directories.
 2021-03-03 23:30:03 +03:00
Azat Khuzhin
b5a5778589 Distributed: Add ability to limit amount of pending bytes for async INSERT 
Right now with distributed_directory_monitor_batch_inserts=1 and
insert_distributed_sync=0 INSERT into Distributed table will store
blocks that should be sent to remote (and in case of
prefer_localhost_replica=0 to the localhost too) on the local
filesystem, and sent it in background.

However there is no limit for this storage, and if the remote is
unavailable (or some other error), these pending blocks may take
significant space, and this is not always desired behaviour.

Add new Distributed setting - bytes_to_throw_insert, that will set the
limit for how much pending bytes is allowed, if the limit will be
reached an exception will be throw.

By default was set to 0, to avoid surprises.
 2021-03-03 23:30:00 +03:00
Azat Khuzhin
ce09b7ff89 Distributed: Implement totalBytes() (system.tables.total_bytes) 2021-03-03 23:29:11 +03:00
Alexey Milovidov
e8df9971f1 Fix Arcadia 2021-03-03 18:12:39 +03:00
alesapin
6b1005aea5
Merge pull request #21334 from ClickHouse/fix_alter_partition_key 
Fix alter modify query for partition key and other metadata fields
 2021-03-03 10:15:11 +03:00
alexey-milovidov
36c5fd840b
Merge pull request #21392 from ClickHouse/add-test-14740 
Add a test for #14740
 2021-03-03 06:33:18 +03:00
Alexey Milovidov
8f45ed5117 Add a test for #15469 2021-03-03 04:35:43 +03:00
alexey-milovidov
9092bc3b85
Merge pull request #21387 from den-crane/tests/issue21369 
test for issue#21369
 2021-03-03 03:05:01 +03:00
Anton Popov
a4c00ab5dc
Merge pull request #21303 from ucasFL/forbid 
Forbid to drop a column if it's referenced by materialized view
 2021-03-03 02:55:06 +03:00
Alexey Milovidov
a93e29ef4a Add a test for #14740 2021-03-03 02:00:19 +03:00
Pavel Kovalenko
2d957b0f76 Test fixed 2021-03-03 00:27:20 +03:00
Denis Zhuravlev
1278b5c71a test for issue#21369 2021-03-02 16:29:11 -04:00
Nikolai Kochetov
8e6fa404c4
Merge pull request #21246 from ucasFL/union-distinct-improve 
Improve Normalization of ASTSelectWithUnionQuery
 2021-03-02 21:37:07 +03:00
Vasily Nemkov
f4b2cbc30f Fixed tests 2021-03-02 15:55:17 +02:00
ikarishinjieva
6fb68cd454 add functional test on fix #21170 2021-03-02 16:28:34 +03:00
Vasily Nemkov
bfc5cb47cc Fix tests 2021-03-02 15:21:13 +02:00
alesapin
06678d650d Merge branch 'master' into fix_alter_partition_key 2021-03-02 13:43:41 +03:00
Vasily Nemkov
6af4d06dff Added Grant, Revoke and System query_kind for system.query_log 
Also some tests to ensure that all queries are logged
 2021-03-02 12:27:39 +02:00
feng lv
a26c9e64a9 fix 
fix
 2021-03-02 03:20:03 +00:00
Nikolai Kochetov
f82e00ff3c Fix inconsistent column type for Values input format. 2021-03-01 23:10:50 +03:00
Alexander Kuzmenkov
977b98ad69
Merge pull request #21094 from den-crane/tests/window_functions 
test for window functions
 2021-03-01 19:00:45 +03:00
Alexander Kuzmenkov
366fba4b04
Update 01592_long_window_functions1.sql 2021-03-01 16:06:44 +03:00
Anton Popov
c98e6bb1ad
Merge pull request #20550 from ClickHouse/fix-number-of-threads-in-subqueries 
Fix number of threads in subqueries
 2021-03-01 15:49:38 +03:00
alesapin
9c8afbeb53 Fix alter modify query for partition key and other metadata fields 2021-03-01 12:59:19 +03:00
Nikolai Kochetov
976dbe8077
Merge pull request #20341 from ClickHouse/filter-push-down 
Filter push down
 2021-03-01 12:35:06 +03:00
vdimir
b7c7c97d10
Merge branch 'master' into join-cast-types-v2 2021-03-01 11:42:21 +03:00
Nikolai Kochetov
eb42896e26 Merge branch 'master' into ucasFL-union-distinct-improve 2021-03-01 10:43:39 +03:00
alexey-milovidov
3bd180c416
Merge pull request #20149 from keen-wolf/timezone_about 
fix toMinute function to handle special timezone
 2021-02-28 21:53:51 +03:00
feng lv
51021c1164 forbid to drop a column if it's referenced by materialized view 2021-02-28 05:24:39 +00:00
Alexey Milovidov
107d8ec811 Merge branch 'master' into extract-text-from-html 2021-02-28 04:03:32 +03:00
alexey-milovidov
470bc2b01b
Merge pull request #21279 from kssenii/add-tests 
Add tests for zstd and zlib http compression
 2021-02-28 01:33:21 +03:00
Alexey Milovidov
0ab4afeeed Tests and documentation 2021-02-28 00:55:56 +03:00
Alexey Milovidov
2ac673b12a Update logic and tests 2021-02-27 22:33:55 +03:00
Alexey Milovidov
fd1cf49e92 Rewrite extractTextFromHTML function 2021-02-27 22:22:38 +03:00
kssenii
4f61ef3099 Tests for zstd and zlib 2021-02-27 14:12:45 +00:00
alexey-milovidov
56af09891b
Merge pull request #20991 from kssenii/brotli 
Fix Brotli http compression issue
 2021-02-27 16:39:49 +03:00
Maksim Kita
9996f68160
Merge pull request #20990 from kitaisreal/constraints-complex-type-support 
Constraints complex types support
 2021-02-27 13:16:29 +03:00
Vladimir
df728dc4ba
Merge pull request #21248 from vdimir/fix-join-totals-nullable 
Convert columns to nullable for totals in join
 2021-02-27 11:31:02 +03:00
feng lv
4ac5b62f19 Merge branch 'master' of github.com:ClickHouse/ClickHouse into union-distinct-improve 2021-02-27 08:02:41 +00:00
Nikolai Kochetov
d91b8a3acb
Merge branch 'master' into filter-push-down 2021-02-26 19:33:27 +03:00
Nikolai Kochetov
d328bfa41f Review fixes. Add setting max_optimizations_to_apply. 2021-02-26 19:29:56 +03:00
kssenii
991718d459 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into brotli 2021-02-26 14:31:24 +00:00
vdimir
932286df1f
Convert columns to nullable for totals in join 2021-02-26 16:32:34 +03:00
feng lv
4c30c10092 add test 
fix
 2021-02-26 12:14:57 +00:00
Anton Popov
6ed3cf5511 Merge remote-tracking branch 'upstream/master' into HEAD 2021-02-26 15:10:52 +03:00
Vladimir
78024ae2d6
Merge pull request #21009 from vdimir/fix-race-storage-join 2021-02-26 14:26:10 +03:00
Maksim Kita
c51773d2b3 Updated test 2021-02-26 12:11:30 +03:00
Nikolai Kochetov
f63d7b45ce Merge branch 'master' into fix-number-of-threads-in-subqueries 2021-02-26 11:15:03 +03:00
Alexander Tokmakov
d46fee0f04 Merge branch 'master' into stress_test_results 2021-02-26 00:24:22 +03:00
vdimir
9c1ba71416
Merge remote-tracking branch 'upstream/master' into fix-race-storage-join 2021-02-25 19:13:49 +03:00
Alexander Tokmakov
fbbb0c4b69 fix 2021-02-25 19:11:43 +03:00
vdimir
4c6cde4d53
Merge remote-tracking branch 'upstream/master' into join-cast-types-v2 2021-02-25 16:24:28 +03:00
Pavel Kovalenko
cf8b105d42 Merge remote-tracking branch 'origin/master' into unfreeze-partitions 2021-02-25 15:24:43 +03:00
Maksim Kita
23af53067d Updated support for Nullable column 2021-02-25 14:27:46 +03:00
Maksim Kita
8fec34af12 Constraints complex types support 2021-02-25 14:27:46 +03:00
alesapin
5c6c318737 Restrict mutations for engines which doesn't support them 2021-02-25 13:07:48 +03:00
vdimir
2f70e895fc
Update StorageJoin locking 
Move joinGet into StorageJoin

Protect JoinSource with lock, add test

Add comments about locking logic
 2021-02-25 12:31:22 +03:00
Vladimir
36c178c023
Merge pull request #21117 from amosbird/joingetlc 2021-02-25 10:06:57 +03:00
Vitaly Baranov
8601b2f3e7
Merge pull request #21148 from vitlibar/protobuf-format-better-tests-2 
Better tests for protobuf format #2.
 2021-02-25 06:01:47 +03:00
Pavel Kovalenko
3663011f9d Merge remote-tracking branch 'origin/master' into unfreeze-partitions 2021-02-24 22:34:25 +03:00
Anton Popov
94156986bf add test for map with integer keys 2021-02-24 21:15:42 +03:00
Vitaly Baranov
2480e4ee3d Better tests for protobuf format #2. 2021-02-24 21:06:29 +03:00
Pavel Kovalenko
0062e459a3 Fixed tests 2021-02-24 19:56:07 +03:00
Maksim Kita
ae79fb22be
Merge pull request #21121 from amosbird/enumdefaultfix 
disable null_as_default when type is Nullable
 2021-02-24 19:31:30 +03:00
vdimir
f93e5b891e
Rename 01731_race_condition_storage.sh -> 01732_race_condition_storage_long 2021-02-24 19:22:40 +03:00
Vasily Nemkov
d568ba5ec7 Marked 00921_datetime64_compatibility as long test 2021-02-24 17:08:43 +02:00
Vasily Nemkov
2d03d330bc Extended range of DateTime64 to years 1925 - 2238 
The Year 1925 is a starting point because most of the timezones
switched to saner (mostly 15-minutes based) offsets somewhere
during 1924 or before. And that significantly simplifies implementation.

2238 is to simplify arithmetics for sanitizing LUT index access;
there are less than 0x1ffff days from 1925.

* Extended DateLUTImpl internal LUT to 0x1ffff items, some of which
  represent negative (pre-1970) time values.
  As a collateral benefit, Date now correctly supports dates up to 2149
  (instead of 2106).
* Added a new strong typedef ExtendedDayNum, which represents dates
  pre-1970 and post 2149.
* Functions that used to return DayNum now return ExtendedDayNum.
* Refactored DateLUTImpl to untie DayNum from the dual role of being
  a value and an index (due to negative time). Index is now a different
  type LUTIndex with explicit conversion functions from DatNum, time_t,
  and ExtendedDayNum.
* Updated DateLUTImpl to properly support values close to epoch start
  (1970-01-01 00:00), including negative ones.
* Reduced resolution of DateLUTImpl::Values::time_at_offset_change
  to multiple of 15-minutes to allow storing 64-bits of time_t in
  DateLUTImpl::Value while keeping same size.
* Minor performance updates to DateLUTImpl when building month LUT
  by skipping non-start-of-month days.
* Fixed extractTimeZoneFromFunctionArguments to work correctly
  with DateTime64.
* New unit-tests and stateless integration tests for both DateTime
  and DateTime64.
 2021-02-24 17:08:35 +02:00
alexey-milovidov
29a29f5060
Merge pull request #19765 from kssenii/fffix 
Fix #19371
 2021-02-24 17:40:15 +03:00
Maksim Kita
395526cb80
Merge pull request #21120 from hexiaoting/dev-map-bug 
Bug fix: Tuple cast to Map
 2021-02-24 17:21:00 +03:00
Nikolai Kochetov
9ded4dd396 Update test. 2021-02-24 17:13:29 +03:00
Vitaly Baranov
92bf2cfb50
Merge pull request #20968 from vitlibar/protobuf-format-better-tests 
Better tests for protobuf format.
 2021-02-24 15:47:39 +03:00
Maksim Kita
7bbc4e013a
Merge pull request #20541 from kitaisreal/replace-null-fields-in-tuple-during-parsing-with-default-values 
Replace null fields in tuple during parsing with default values
 2021-02-24 12:00:48 +03:00
alesapin
427e5f295c
Merge pull request #21007 from CurtizJ/alter-bad-conversions 
Do not start mutation for alters with wrong type conversion
 2021-02-24 10:08:15 +03:00
Amos Bird
abf26d4935
disable null_as_default when type is Nullable 2021-02-24 14:45:14 +08:00
Amos Bird
0f7f2a4962
Support LC type for joinGet 2021-02-24 11:35:45 +08:00
kssenii
2d31019add Update .reference file 2021-02-23 16:54:50 +00:00
vdimir
e4fdbbfab6
Add test race_condition_storage.sh 2021-02-23 19:27:44 +03:00
hexiaoting
e123c6ef9d Bug fix: Tuple cast to Map 2021-02-23 18:42:33 +08:00
Maksim Kita
fb932ce290 Updated tests for map type 2021-02-23 12:54:38 +03:00
Denis Zhuravlev
2ebae14f12 mark test as long 2021-02-22 23:40:38 -04:00
Denis Zhuravlev
f2b8f81e40 test for window functions 2021-02-22 21:05:25 -04:00
Maksim Kita
f001a6755e Updated for map 2021-02-22 23:32:43 +03:00
Maksim Kita
992747e5ec Updated for Array 2021-02-22 23:32:43 +03:00
Maksim Kita
d6b53d2b02 Added test 2021-02-22 23:32:43 +03:00
alexey-milovidov
cb8cfdba77
Merge branch 'master' into fffix 2021-02-22 23:08:44 +03:00
Alexey Milovidov
df3e70c352 Merge branch 'master' into add-test-9781 2021-02-22 20:53:38 +03:00
Alexey Milovidov
871f153433 Fix tests 2021-02-22 20:44:24 +03:00
Alexey Milovidov
f8501a2595 Add a test for #9781 2021-02-22 17:11:06 +03:00
alexey-milovidov
073ee6c2c3
Merge pull request #21053 from ClickHouse/more-consistent-datetime64-parsing 
More consistent DateTime64 parsing
 2021-02-22 13:13:21 +03:00
alexey-milovidov
8b82c7ea0c
Merge pull request #21050 from azat/datetime64-float-fix 
Fix DateTime64 from Float
 2021-02-22 13:12:09 +03:00
alexey-milovidov
9f9b862a47
Merge pull request #21051 from ClickHouse/add-test-19376 
Add test for #19376
 2021-02-22 13:11:01 +03:00
alexey-milovidov
50639b7dba
Merge pull request #21052 from ClickHouse/file-engine-settings 
Merging #20620
 2021-02-22 13:10:31 +03:00
Alexey Milovidov
29730425d6 Add a test 2021-02-21 23:34:38 +03:00
Alexey Milovidov
46ac365629 Merge branch 'master' into topvisor/file-engine-settings 2021-02-21 23:10:26 +03:00
Azat Khuzhin
d220fe1a4e Add cases with floats into 01691_DateTime64_clamp 2021-02-21 22:36:40 +03:00
Azat Khuzhin
521537eed8 Fix DateTime64 from Float 2021-02-21 22:36:40 +03:00
Alexey Milovidov
7dd49b69ea Add test for #19376 2021-02-21 22:36:39 +03:00
alexey-milovidov
adfac060dc Update 01702_toDateTime_from_string_clamping.sql 2021-02-21 21:01:46 +03:00
Alexey Milovidov
d2a9e5842a Fix wrong formatting of overflowed DateTime64 2021-02-21 21:01:46 +03:00
alexey-milovidov
bf9af598be
Merge pull request #21025 from ClickHouse/fix-ubsan-bigint-arithmetic 
Fix UBSan report in Decimal arithmetic
 2021-02-21 21:00:17 +03:00
alexey-milovidov
125bf1d0d9
Merge pull request #20479 from ucasFL/tr 
fix transform with floating point key
 2021-02-21 17:37:20 +03:00
alexey-milovidov
20a7ccf5b1
Merge pull request #20174 from amosbird/alias 
Normalized function names (lower-case case-insensitive names)
 2021-02-21 16:23:51 +03:00
Alexey Milovidov
9cb972e9ed Fix UBSan report in Decimal arithmetic #19432 2021-02-21 13:23:18 +03:00
feng lv
5d3b8d99e7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into tr 2021-02-21 07:34:09 +00:00
feng lv
a33183ee1e fix transform with floating point key 
better

update test
 2021-02-21 05:20:42 +00:00
Vitaly Baranov
2eecaee08d Better tests for protobuf format. 2021-02-20 23:13:32 +03:00
Nikolai Kochetov
00e0dbc3e5 Fix test. 2021-02-20 20:42:06 +03:00
tavplubix
a697b578bc
Merge pull request #16193 from ClickHouse/database_replicated 
DatabaseReplicated: continuation of #10485
 2021-02-20 19:39:34 +03:00
Anton Popov
48e188681c do not start mutation for alters with wrong type conversion 2021-02-20 19:11:08 +03:00
alexey-milovidov
316cdf5039
Merge pull request #20978 from ClickHouse/little-better 
Just little better
 2021-02-20 17:53:07 +03:00
Alexander Tokmakov
863c099254 fix 2021-02-20 15:57:07 +03:00
alexey-milovidov
04f5b72391
Merge pull request #20995 from amosbird/ubsanfix2 
Suppress signed overflow in AggregateFunctionGroupArrayMoving 2
 2021-02-20 14:25:36 +03:00
Maksim Kita
c5643c47d0
Merge pull request #20970 from kitaisreal/function-sum-map-decimal-fix 
Function sumMap decimal fix
 2021-02-20 13:43:12 +03:00
Alexey Milovidov
d947dbc185 Add test to skip list 2021-02-20 11:44:35 +03:00
alexey-milovidov
a79ccce074
Merge pull request #20980 from den-crane/tests/test_17761 
test for decimal ( p , s) in dictionaries
 2021-02-20 10:56:28 +03:00
Nikolai Kochetov
2f67cc02c1
Merge pull request #20881 from azat/async-task-queue-wait-fix 
Fix 'Empty task was returned from async task queue' on query cancellation
 2021-02-20 10:56:04 +03:00
alexey-milovidov
9c48fcfdef
Merge pull request #20882 from azat/distributed_group_by_no_merge-improvements 
Do only merging of sorted blocks on initiator with distributed_group_by_no_merge=2
 2021-02-20 10:36:32 +03:00
kssenii
0d88366b27 Add forgotten .reference file update 2021-02-20 07:07:50 +00:00
Amos Bird
487fb09ff6
Suppress signed overflow in AggregateFunctionGroupArrayMoving 2 2021-02-20 14:11:01 +08:00
kssenii
e1868d1392 Move test into separate file 2021-02-20 05:13:31 +00:00
M0r64n
2f7d0ba926 Replace direct truncate with O_TRUNC flag 2021-02-20 03:27:23 +04:00
kssenii
21ee685ef0 Fix brotly 2021-02-19 22:18:14 +00:00
Azat Khuzhin
d0fe8900f9 Fix bash syntax in 01731_async_task_queue_wait 2021-02-20 00:04:28 +03:00
Azat Khuzhin
657bb33e73 Merge remote-tracking branch 'upstream/master' into async-task-queue-wait-fix 
Merge upstream/master to debug possible uncaught exceptions with gdb
diagnostics merged.
 2021-02-20 00:01:21 +03:00
Azat Khuzhin
f5893778cb Do not use view() in 01731_async_task_queue_wait to fix ANTLR parser 2021-02-20 00:01:13 +03:00
Denis Zhuravlev
66e775ef88 test for decimal ( p , s) in dictionaries 2021-02-19 14:53:34 -04:00
alexey-milovidov
91de5628f0
Merge pull request #20866 from CurtizJ/fix-distinct-state 
Fix usage of 'distinct' combinator with 'state' combinator
 2021-02-19 20:22:53 +03:00
tavplubix
3509fe88d8
Merge pull request #20878 from ClickHouse/fix_18926 
Fix COM_FIELD_LIST response in MySQL protocol
 2021-02-19 19:25:54 +03:00
Maksim Kita
b84112a603 Function sumMap decimal fix 2021-02-19 19:25:50 +03:00
vdimir
946576017f
Update CrossToInnerJoinVisitor, add tests to cross_to_inner_join 2021-02-19 17:06:57 +03:00
Nikolai Kochetov
866dfaec79
Update 01731_async_task_queue_wait.sh 2021-02-19 15:25:22 +03:00
vdimir
7b8b77f258
Add tests to join_implicit_cast checks column name clash 2021-02-19 14:53:06 +03:00
Maksim Kita
71c65b5d82
Merge pull request #20611 from kitaisreal/rename-untyped-function-reinterpret-as 
Rename untyped function reinterpretAs into reinterpret
 2021-02-19 14:35:00 +03:00
vdimir
1a26310d27
Split test join_implicit_cast (due to timeouts in flaky checks) 2021-02-19 14:28:41 +03:00
vdimir
dc9e660522
Remove 'error' from comment from 01710_join_use_nulls 2021-02-19 14:28:41 +03:00
vdimir
ab0719caf0
More tests for join on key type convert 2021-02-19 14:28:41 +03:00
vdimir
3a7eddcf3a
Remove addRequiredLeftColumn, some tests for join on different types 2021-02-19 14:28:40 +03:00
vdimir
a378bd08aa
Perform implicit type conversion for JOIN ON keys 2021-02-19 14:28:40 +03:00
vdimir
cd7d9584bd
Add tests to 01720_join_implicit_cast - switch, use_nulls 2021-02-19 14:28:40 +03:00
vdimir
435f63f42b
Calculate common type for join using in TreeRewriter 
Split before_join and converting_join_columns dags

Add more detailed comments for column type conversion for join using
 2021-02-19 14:28:32 +03:00
vdimir
d15c1a203b
Fix TableJoin, upd 01675_join_implicit_cast 2021-02-19 14:24:34 +03:00
vdimir
f5b98015a8
Fix converting right key type in join using 2021-02-19 14:24:33 +03:00
vdimir
4c36cd1737
Add converting step for 'join using' 2021-02-19 14:24:28 +03:00
Nikolai Kochetov
df1cf481cf
Update 01731_async_task_queue_wait.sh 2021-02-19 14:14:31 +03:00
Nikolai Kochetov
8f8a4f6423
Update 01731_async_task_queue_wait.sh 2021-02-19 13:59:38 +03:00
Amos Bird
fc1885ea9b
Try fixing flaky tests 2021-02-19 17:28:01 +08:00
Nikolai Kochetov
f4b0b1110c Fix test. 2021-02-18 23:53:40 +03:00
Azat Khuzhin
7bcfe92cd7 Mark 01730_distributed_group_by_no_merge_order_by as long 
https://clickhouse-test-reports.s3.yandex.net/20882/af660140c320ca45bca0edfd89000b3c6da8ee6a/functional_stateless_tests_flaky_check_(address).html#fail1
 2021-02-18 23:29:38 +03:00
Azat Khuzhin
9c01869090 Fix 'Empty task was returned from async task queue' on query cancellation 2021-02-18 22:15:21 +03:00
Azat Khuzhin
af660140c3 Do only merging of sorted blocks on initiator with distributed_group_by_no_merge=2 
When distributed_group_by_no_merge=2 is used (or when
optimize_distributed_group_by_sharding_key takes place), remote servers
will do full ORDER BY, so initiator can skip this step and do only merge
of ordered blocks.
 2021-02-18 21:43:27 +03:00
Alexander Tokmakov
8097532cb7 Merge branch 'master' into database_replicated 2021-02-18 17:42:06 +03:00
Alexander Tokmakov
97f4c457ec fix MySQL COMM_FIELD_LIST response 2021-02-18 16:27:51 +03:00
Nikolai Kochetov
0449546bca Support TotalsHaving. Update test. 2021-02-18 16:13:09 +03:00
Amos Bird
cac9c7fc07
Fix tests 2021-02-18 20:20:28 +08:00
Amos Bird
2dc7ba160a
Better 2021-02-18 20:20:28 +08:00
Amos Bird
77fd060665
Normalize function names 2021-02-18 20:20:28 +08:00
zlx19950903
c92e613b82
Add a function htmlOrXmlCoarseParse to extract content from html or xml format string. (#19600) 
* add html and xml coarse parse

* add test file

* add conditional check: hyperscan

* fix style error

* add conditional check

* bug fix

* delete unit

* typos check fix

* add unit test

* style check fix

* fix build error: case style

* acradis_skip test fix

* LINT error fix

* Remove comments

Co-authored-by: guojiantao <guojiantao15@mails.ucas.ac.cn>
Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
Co-authored-by: Ivan Lezhankin <ilezhankin@yandex-team.ru>
 2021-02-18 15:05:55 +03:00
Maksim Kita
4278098f9a Reinterpret function added Decimal, DateTim64 support 2021-02-18 13:09:01 +03:00
Maksim Kita
dcabbc95ed
Merge pull request #20623 from kitaisreal/dictionary-create-source-with-functions-fix 
Dictionary create source with functions crash fix
 2021-02-18 11:50:20 +03:00
Vitaly Baranov
3cbb325913
Merge pull request #20506 from vitlibar/refactor-protobuf-format-io 
Improved serialization in Protobuf format.
 2021-02-18 11:31:37 +03:00
Nikolai Kochetov
d582160af4 Merge branch 'master' into filter-push-down 2021-02-18 10:58:27 +03:00
M0r64n
1ce9570fcb Fix 01721_engine_file_truncate_on_insert.reference 2021-02-18 07:50:15 +00:00
M0r64n
adf5d24177 Correct file engine settings tests 2021-02-18 11:36:17 +04:00
Anton Popov
dcba99f4b1 fix usage of 'distinct' combinator with 'state' combinator 2021-02-18 02:19:58 +03:00
alexey-milovidov
80e6ad7aae
Merge pull request #20610 from ucasFL/fix-union-distinct-limit 
fix subquery with limit
 2021-02-17 21:24:10 +03:00
Vitaly Baranov
18e036d19b Improved serialization for data types combined of Arrays and Tuples. 
Improved matching enum data types to protobuf enum type.
Fixed serialization of the Map data type.
Omitted values are now set by default.
 2021-02-17 20:50:09 +03:00
Nikolai Kochetov
e5b9c42860 Update test. 2021-02-17 19:54:37 +03:00
Nikita Mikhaylov
d15bac31ff
Merge pull request #20399 from nikitamikhaylov/keen-wolf-filefuncwithreadbuf 
Merging #19204
 2021-02-17 16:11:22 +03:00
Maksim Kita
b2c09f002f Dictionary create source with functions crash fix 2021-02-17 15:26:00 +03:00
Vladimir
2c99a36228
Merge pull request #19685 from hexiaoting/dev_joinon 2021-02-17 13:22:55 +03:00
Maksim Kita
c608fa1e6a Added error reinterpretation tests 2021-02-17 12:53:12 +03:00
feng lv
3b40099578 fix subquery with limit 2021-02-17 08:26:52 +00:00
Maksim Kita
5f88f5817f Rename untyped function reinterpretAs into reinterpret 2021-02-17 11:23:24 +03:00
Alexander Tokmakov
1aac7b3471 Merge branch 'master' into database_replicated 2021-02-17 00:39:56 +03:00
vdimir
1680c8e73e
Merge remote-tracking branch 'upstream/master' into cross-to-inner-join-rewrite 2021-02-16 20:17:54 +03:00
tavplubix
68b427a99d
Merge pull request #20448 from ClickHouse/better_ddl_queue_cleanup 
Better distributed DDL queue cleanup
 2021-02-16 19:29:50 +03:00
M0r64n
75117389ec Add a couple of QOL file engine settings 2021-02-16 18:50:11 +04:00
vdimir
3e2c9ad011
Merge remote-tracking branch 'origin/master' into dev_joinon 2021-02-16 17:08:10 +03:00
Nikita Mikhailov
b404fea18d better 2021-02-16 15:40:09 +03:00
Nikita Mikhailov
6ee98ead05 Merge branch 'master' of github.com:ClickHouse/ClickHouse into keen-wolf-filefuncwithreadbuf 2021-02-16 15:38:03 +03:00
Nikita Mikhailov
7c5d845866 refactor function 2021-02-16 15:37:49 +03:00
vdimir
3d19d0644e
Update join on associativity in some tests 2021-02-16 13:46:25 +03:00
tavplubix
d4b2a37393
Merge pull request #20350 from amosbird/tviewf 
Disable table function view in expression
 2021-02-16 13:01:13 +03:00
Alexander Tokmakov
bf6f64a3fb Merge branch 'master' into database_replicated 2021-02-16 01:28:19 +03:00
vdimir
ed9d49abc3
Update cross to inner rewrite 2021-02-15 22:31:11 +03:00
Nikita Mikhailov
812641f5a7 add test to arcadia skip list 2021-02-15 16:38:31 +03:00
Nikita Mikhailov
d41b2a4d06 Merge branch 'master' of github.com:ClickHouse/ClickHouse into keen-wolf-filefuncwithreadbuf 2021-02-15 16:15:52 +03:00
Anton Popov
b67b16cdf3
Merge pull request #20476 from amosbird/ctefix2 
Fix global-with with subqueries
 2021-02-15 16:14:50 +03:00
alexey-milovidov
3429925338
Merge pull request #20492 from ClickHouse/add-test-8654 
Add a test for #8654
 2021-02-15 15:43:48 +03:00
alexey-milovidov
b75d9a09bd
Merge pull request #20493 from ClickHouse/add-test-10893 
Add a test for #10893
 2021-02-15 15:36:30 +03:00
Nikita Mikhaylov
92e13d76d3
Merge pull request #18508 from depressed-pho/runningConcurrency 
Add function runningConcurrency()
 2021-02-15 14:39:38 +03:00
Nikita Mikhaylov
a77b740a7c
Merge pull request #20286 from nikitamikhaylov/json-import-bugfix 
Error from allocator on JSON import
 2021-02-15 12:40:09 +03:00
Alexey Milovidov
4c7923e6df Add a test for #10893 2021-02-15 05:17:30 +03:00
Alexey Milovidov
c5b96a522e Add a test for #8654 2021-02-15 04:57:34 +03:00
alexey-milovidov
71acb2ffda
Merge pull request #20475 from ClickHouse/intdiv-ubsan-2 
Fix UBSan report in intDiv
 2021-02-15 04:05:16 +03:00
Alexey Milovidov
d529db5498 Adjustments 2021-02-14 23:57:25 +03:00
alexey-milovidov
8be8746643
Merge pull request #20461 from azat/join_use_nulls-logical-error-fix 
Fix LOGICAL_ERROR for join_use_nulls=1 when JOIN contains const from SELECT
 2021-02-14 23:28:50 +03:00
Amos Bird
ed49367fc7
Fix global-with with subqueries 2021-02-14 14:20:23 +08:00
Alexey Milovidov
55c17ac93f Fix UBSan report in intDiv 2021-02-14 06:11:12 +03:00
alexey-milovidov
9db6f4a452
Merge pull request #20466 from azat/mt-logging 
Improve logging during reading from MergeTree
 2021-02-14 04:30:31 +03:00
alexey-milovidov
a0fbc0656b
Merge pull request #20206 from ClickHouse/tavplubix-patch-3 
Suppress signed overflow in AggregateFunctionGroupArrayMoving
 2021-02-14 03:32:16 +03:00
Alexander Tokmakov
b0f2a84306 fix bad test 2021-02-14 01:12:10 +03:00
Alexander Tokmakov
652ede5af9 add test 2021-02-14 00:06:40 +03:00
Azat Khuzhin
790c210e51 Mark 01508_partition_pruning as long 
https://clickhouse-test-reports.s3.yandex.net/20466/ff647ad176ad32718fcfc87677effa0ab37e3f10/functional_stateless_tests_flaky_check_(address).html#fail1
 2021-02-13 20:33:03 +03:00
alexey-milovidov
3e9f1dc83b
Merge pull request #20458 from ClickHouse/array-difference-ubsan 
Fix UBSan report in arrayDifference
 2021-02-13 19:34:44 +03:00
Azat Khuzhin
ff647ad176 Update 01508_partition_pruning expectations 2021-02-13 18:08:55 +03:00
Amos Bird
a551edd8d6
Do not parse view function in expression 2021-02-13 22:12:44 +08:00
Amos Bird
69d4120982
Disable table function view in expression 2021-02-13 21:39:37 +08:00
Ilya Yatsishin
8ff458a2bf
Merge pull request #20459 from ClickHouse/decimal-comparison-ubsan 2021-02-13 12:34:40 +03:00
Azat Khuzhin
4aa46ce3d6 More tests for join_use_nulls 
All of them already works, but just in case
 2021-02-13 11:05:25 +03:00
Azat Khuzhin
cbeda6c60e Fix LOGICAL_ERROR for join_use_nulls=1 when JOIN contains const from SELECT 2021-02-13 11:05:25 +03:00
alesapin
2e010bffc8
Merge pull request #20398 from ClickHouse/test_for_alredy_working_code 
Add test for sign column drop
 2021-02-13 10:17:48 +03:00
Alexey Milovidov
1546f5bcb9 Suppress UBSan report in Decimal comparison 2021-02-13 03:54:38 +03:00
Alexey Milovidov
1c55be261c Fix UBSan report in arrayDifference 2021-02-13 03:45:06 +03:00
Alexander Kuzmenkov
716a3df6eb
Merge pull request #20337 from ClickHouse/aku/window-prototype 
window function rank() and friends
 2021-02-13 03:03:49 +03:00
alexey-milovidov
04701376ac
Merge pull request #20232 from ClickHouse/fix_int_field_to_decimal_conversion 
Fix integer Field to Decimal conversion
 2021-02-13 02:22:32 +03:00
alexey-milovidov
371e84957a
Merge pull request #20303 from azat/constant-folding-fixes 
Do not allow early constant folding of explicitly forbidden functions
 2021-02-13 02:20:06 +03:00
alesapin
051c9533b9 Fix dependent test 2021-02-12 22:42:20 +03:00
alesapin
939a3e9555 Fix tests for better parallel run 2021-02-12 22:28:00 +03:00
Nikita Mikhailov
2a52aa8ca3 fix test 2021-02-12 20:25:40 +03:00
alesapin
9cbda346bb Add test for already working code 2021-02-12 19:49:27 +03:00
Nikolai Kochetov
5fd80555aa Update test. 2021-02-12 19:06:18 +03:00
Nikolai Kochetov
f1c32c4690 Merge branch 'master' into filter-push-down 2021-02-12 18:26:08 +03:00
Nikita Mikhailov
fcd080631f Merge branch 'master' of github.com:ClickHouse/ClickHouse into keen-wolf-filefuncwithreadbuf 2021-02-12 17:53:18 +03:00
Pavel Kruglov
f0163c2acf Don't create empty parts on INSERT 2021-02-12 17:02:04 +03:00
Nikita Mikhaylov
7d02d58390
bump CI 2021-02-12 16:14:34 +03:00
Nikolai Kochetov
683d793cc2 Update test. 2021-02-12 14:31:14 +03:00
Nikita Mikhailov
999062e926 fix test 2021-02-12 13:45:18 +03:00
Alexander Kuzmenkov
824475b224 cleanup 2021-02-12 13:37:27 +03:00
alesapin
a94e223cee
Merge pull request #20344 from azat/join_use_nulls-fix 
Fix null dereference with join_use_nulls=1
 2021-02-12 12:48:53 +03:00
alesapin
9aadbde86c
Merge pull request #20339 from kitaisreal/decimal-binary-operation-constants-fix 
Decimal binary operation constants fix
 2021-02-12 12:32:58 +03:00
Maksim Kita
e141367bcf
Merge pull request #20345 from azat/RANGE_HASHED-dict-fix 
Avoid invalid dereference in RANGE_HASHED() dictionary
 2021-02-12 11:05:10 +03:00
alesapin
a2bb586170
Merge pull request #20335 from ClickHouse/fix_scheduling_of_non_successful_background_tasks 
Fix backoff for failed background tasks in replicated merge tree
 2021-02-12 10:06:04 +03:00
Azat Khuzhin
f6cfcd4da9 Fix null dereference with join_use_nulls=1 
Found with MSan [1], the following query triggers null dereference:

```sql
SELECT
    Y.id - 1
FROM X
RIGHT JOIN Y ON (X.id + 1) = Y.id
SETTINGS join_use_nulls=1; -- { serverError 53 }
```

```
Received signal 11
(version 21.3.1.5916, build id: 2E9E84AA32AEAAC7C8B6EB45DA3EC0B4F15E9ED4) (from thread 100) (query_id: 9ab8cb0d-be8d-445e-8498-930a7268488b) Received signal Segmentation fault (11)
Address: 0x10 Access: read. Address not mapped to object.
Stack trace: 0x2d079d65 0x29bf1f30 0x12b12220 0x12b13098 0x12b17b08 0x12b20459 0x2ae37913 0x2ae352d9 0x2c746072 0x2c7585dd 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d 0x2c7483e3 0x2c74e63d
4. ./obj-x86_64-linux-gnu/../contrib/boost/boost/smart_ptr/intrusive_ptr.hpp:0: DB::ColumnConst::ColumnConst(COW<DB::IColumn>::immutable_ptr<DB::IColumn> const&, unsigned long) @ 0x2d079d65 in /workspace/clickhouse
5. ./obj-x86_64-linux-gnu/../src/Common/COW.h:0: DB::createBlockWithNestedColumns(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) @ 0x29bf1f30 in /workspace/clickhouse
6. DB::FunctionOverloadResolverAdaptor::getReturnTypeDefaultImplementationForNulls(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::function<std::__1::shared_ptr<DB::IDataType const> (std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&)> const&) @ 0x12b12220 in /workspace/clickhouse
7. DB::FunctionOverloadResolverAdaptor::getReturnTypeWithoutLowCardinality(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x12b13098 in /workspace/clickhouse
8. DB::FunctionOverloadResolverAdaptor::getReturnType(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x12b17b08 in /workspace/clickhouse
9. DB::FunctionOverloadResolverAdaptor::build(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x12b20459 in /workspace/clickhouse
```

  [1]: https://clickhouse-test-reports.s3.yandex.net/19451/64c0bf98290362fa216c05b070aa122a12af3c25/fuzzer_msan/report.html#fail1
 2021-02-11 21:30:05 +03:00
Azat Khuzhin
2907385400 Avoid invalid dereference in RANGE_HASHED() dictionary 
UBsan report the following [1], when query does not contains any columns
from the dictionary:

```sql
SELECT
    toUInt32(toUInt32(NULL, toUInt32(NULL, inf, NULL), NULL)),
    toUInt32(toUInt32(toUInt32(toUInt32(toUInt32(NULL, 1., NULL)), toUInt32(toUInt32(NULL, 0.5, NULL)), toUInt32(NULL, NULL)), toUInt32(toUInt32(NULL, 1., NULL)), toUInt32(NULL, NULL)), toUInt32(toUInt32(toUInt32(toUInt32(NULL, 1000.0001220703125, NULL)), toUInt32(toUInt32(NULL, 10.000100135803223, NULL)), toUInt32(NULL, NULL)), NULL, NULL, NULL))
FROM somedict
```

```
std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> >::back() @ 0x128c07a6 in /workspace/clickhouse
./obj-x86_64-linux-gnu/../src/Dictionaries/RangeDictionaryBlockInputStream.h:0: DB::RangeDictionaryBlockInputStream<DB::RangeHashedDictionary, unsigned short, unsigned long>::fillBlock(DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul> const&, DB::PODArray<unsigned short, 4096ul, Allocator<false, false>, 15ul, 16ul> const&, DB::PODArray<unsigned short, 4096ul, Allocator<false, false>, 15ul, 16ul> const&) const @ 0x1692335e in /workspace/clickhouse
./obj-x86_64-linux-gnu/../src/Dictionaries/RangeDictionaryBlockInputStream.h:0: DB::RangeDictionaryBlockInputStream<DB::RangeHashedDictionary, unsigned short, unsigned long>::getBlock(unsigned long, unsigned long) const @ 0x16922f96 in /workspace/clickhouse
./obj-x86_64-linux-gnu/../src/Dictionaries/DictionaryBlockInputStreamBase.cpp:23: DB::DictionaryBlockInputStreamBase::getHeader() const @ 0x166ab57c in /workspace/clickhouse
```

  [1]: https://clickhouse-test-reports.s3.yandex.net/19451/64c0bf98290362fa216c05b070aa122a12af3c25/fuzzer_ubsan/report.html#fail1
 2021-02-11 21:07:37 +03:00
alexey-milovidov
7b55eb8c58
Merge pull request #20178 from amosbird/delayinactive 
Delay or throw insertion when too many inactive parts
 2021-02-11 20:52:04 +03:00
alexey-milovidov
ac707f7165
Merge pull request #20305 from azat/fix-flaky-tests 
Fix 00738_lock_for_inner_table flakiness
 2021-02-11 20:46:42 +03:00
alexey-milovidov
d15d340691
Merge pull request #20301 from azat/numbers-numeric-limit 
Accept arbitrary numeric types for numbers() arguments (for scientific notation)
 2021-02-11 20:41:15 +03:00
Alexander Kuzmenkov
1275be58bf Merge remote-tracking branch 'origin/master' into HEAD 2021-02-11 19:49:37 +03:00
Alexander Kuzmenkov
ebb47595aa
Merge pull request #20293 from ClickHouse/aku/window-prototype-3 
make window functions faster
 2021-02-11 19:49:02 +03:00
Maksim Kita
248a06f930 Added test 2021-02-11 19:11:07 +03:00
Alexander Kuzmenkov
d8f9a8d3cd first_value and last_value 2021-02-11 18:41:54 +03:00
alesapin
bf4af9713c Fix tests 2021-02-11 18:29:08 +03:00
Alexander Kuzmenkov
ecbcf47f28 lag/lead stubs + cleanup 2021-02-11 18:07:42 +03:00
alexey-milovidov
263d751d64
Merge pull request #20169 from ClickHouse/no-excessive-squash-memory 
Disable excessive squashing of blocks for StorageMemory #13052
 2021-02-11 16:38:24 +03:00
Alexander Kuzmenkov
525400bc41 window function rank() and friends 2021-02-11 16:29:30 +03:00
Nikita Mikhailov
e325ab2538 fix test 2021-02-11 15:00:14 +03:00