Commit Graph

1003 Commits

Author SHA1 Message Date
Azat Khuzhin
fb30629ff2 Add settings to control nesting level for shards skipping optimization
- optimize_skip_unused_shards_nesting (allows control nesting level for
  shards skipping optimization)
- force_skip_optimize_shards_nesting (allows control nesting level for
  checking was shards skipped or not)
- deprecates force_skip_optimize_shards_no_nested
2020-06-21 15:27:01 +03:00
Alexey Milovidov
fe71a8eb8c Merge branch 'master' into send-logs-on-fatal-errors 2020-06-21 13:55:11 +03:00
Alexey Milovidov
fd885e86f7 Merge branch 'master' into mysql-vars 2020-06-21 13:53:37 +03:00
Alexey Milovidov
227afea032 Merge branch 'master' into decimal-cut-extra-digits-after-point 2020-06-21 13:50:06 +03:00
Alexey Milovidov
7f05d2bac0 Merge branch 'master' into print-build-id 2020-06-21 13:44:32 +03:00
alexey-milovidov
31dd02c45b
Merge pull request #11833 from ClickHouse/disable-bad-optimization
Disable bad optimization #10047
2020-06-21 02:03:50 +03:00
alexey-milovidov
485c4c9055
Merge pull request #11828 from ClickHouse/fix-fixed-string-comparison
Fix FixedString comparison #11393
2020-06-21 00:21:13 +03:00
alexey-milovidov
cd3d7c9155
Merge pull request #11806 from azat/optimize_aggregators_of_group_by_keys-fix-alias
Preserve column alias with optimize_aggregators_of_group_by_keys
2020-06-21 00:13:19 +03:00
alexey-milovidov
017f55a715
Merge pull request #11821 from ClickHouse/hostname-alias
Added "hostname" as an alias to function "hostName"
2020-06-20 21:46:21 +03:00
Alexey Milovidov
147cda2581 Merge branch 'master' into print-build-id 2020-06-20 21:05:31 +03:00
Alexey Milovidov
14e29f2c11 Disable bad optimization #10047 2020-06-20 21:00:00 +03:00
Alexey Milovidov
7c4fb669db Update tests 2020-06-20 20:57:02 +03:00
alexey-milovidov
5d68dda35b
Merge pull request #11820 from ClickHouse/added-test-for-mutations-after-freeze
Added a test for #6549
2020-06-20 19:17:32 +03:00
Alexey Milovidov
e6a969e2e6 Added support for MySQL style global variables syntax 2020-06-20 19:05:49 +03:00
Alexey Milovidov
408fc241d6 Make reading of Decimal more compatible with other DBMS 2020-06-20 17:43:01 +03:00
alexey-milovidov
df0b66c75b
Merge pull request #11817 from ClickHouse/check-for-tests-with-fail-in-their-names
Check for tests with "fail" in their names
2020-06-20 16:50:42 +03:00
Alexey Milovidov
3658ea6e55 Fix FixedString comparison #11393 2020-06-20 16:13:51 +03:00
Alexey Milovidov
a02fdbcad0 Update tests 2020-06-20 14:29:01 +03:00
Alexey Milovidov
d0b948dc3f Added a test 2020-06-20 12:18:53 +03:00
Alexey Milovidov
f25416beb8 Added "hostname" as an alias to function "hostName" 2020-06-20 10:43:35 +03:00
Alexey Milovidov
a7cc800138 Added a test for #6549 2020-06-20 10:36:17 +03:00
Alexey Milovidov
f2296ba313 Added a test 2020-06-20 09:56:37 +03:00
Alexey Milovidov
9f26ac06ea Fix another 10% of bad code #11529 2020-06-20 09:54:05 +03:00
Alexey Milovidov
dec883e21b Check for tests with "fail" in their names 2020-06-20 09:28:19 +03:00
alexey-milovidov
f27ae6a12a
Merge pull request #11682 from ClickHouse/fix-parallel-final-stuck
Fix parallel final stuck
2020-06-20 09:05:32 +03:00
alexey-milovidov
4129300fdb
Merge pull request #11788 from ClickHouse/fix-too-many-threads
Fix too many threads
2020-06-20 08:24:47 +03:00
tavplubix
b8685322b7
Merge pull request #11767 from amosbird/fs1
add scalars in time
2020-06-20 01:01:20 +03:00
Anton Popov
7a76abeb2a distinct combinator for function of multiuple arguments 2020-06-19 23:13:23 +03:00
Azat Khuzhin
5f14622bfd Preserve column alias with optimize_aggregators_of_group_by_keys
optimize_aggregators_of_group_by_keys has been introduced in #11667
2020-06-19 23:04:19 +03:00
alexey-milovidov
85974dd699
Update 00816_long_concurrent_alter_column.sh 2020-06-19 22:56:18 +03:00
alesapin
b504fc6a90 Fix error code 2020-06-19 16:37:27 +03:00
Nikolai Kochetov
6e38d9ec10 Update test. 2020-06-19 15:26:03 +03:00
Nikolai Kochetov
f7b4039058 Added test. 2020-06-19 14:13:03 +03:00
Anton Popov
36b9847d9a
Merge pull request #11768 from CurtizJ/any-input-optimize
Sligtly better code after #11529.
2020-06-19 03:36:49 +03:00
tavplubix
bed1f37790
Merge pull request #11214 from Potya/multiword_data_type
Add some multiword data types
2020-06-18 23:47:37 +03:00
Ruslan
f44dbcd2ab
Fix crash in min(multiply(1)) with optimize_arithmetic_operations_in_aggregate_functions (#11756) 2020-06-18 22:25:28 +03:00
Azat Khuzhin
0e218b0f15 Improve 01319_optimize_skip_unused_shards_no_nested
Before there is no check that optimize_skip_unused_shards was working
for the first level, use cluster with unavalable shard to guarantee
this.
2020-06-18 21:49:29 +03:00
Azat Khuzhin
724c09a22c Add missing DROP TABLE in 01319_mv_constants_bug 2020-06-18 21:49:29 +03:00
Azat Khuzhin
6507247328 optimize_skip_unused_shards=2 will disable it for nested distributed queries
P.S. Looks like settings can be converted between SettingUInt64 and
SettingBool without breaking binary protocol.

FWIW maybe it is a good idea to change the semantics of the settings as
follow (but I guess that changing semantic is not a good idea, better to
add new settings and deprecate old ones):
- optimize_skip_unused_shards -- accept nesting level on which the
  optimization will work
- force_skip_optimize_shards_nesting -- accept nesting level on which
  the optimization will work
2020-06-18 21:49:29 +03:00
Azat Khuzhin
041533eae2 Disable optimize_skip_unused_shards if sharding_key has non-deterministic func
Example of such functions is rand()

And this patch disables only optimize_skip_unused_shards, i.e. INSERT
code path does not changed, so it will work as before.
2020-06-18 21:49:29 +03:00
Anton Popov
bd28b7e1c2 better code 2020-06-18 20:05:26 +03:00
Ruslan
23cd919681
Any input optimize (#11529)
* workig commit №1

* working commit №2

* first version of ArithmeticOperationsInAgrFuncOptimize (min, max, sum)

* fix conflicts

* fix №2

* attempt №3

* Description + little mistake

* fix style + expanded optimization + add performance test

* style + merge with new version

* one more branch

* bug fix + merge

* just merge

* some code style and logic fixes

* little changes

* tests

* test(2)

* style

* seems to have fixed everything that was required

* bug fix

* style

* build fix

* fix typo

* fix clang-10 warning

* fix clang-10 warning

* nolint

* add new files to ya.make

* extract all functions from Any

* add last new lines.

* extract all functions from Any

* add last new lines.

* build + requested changes

* code style

* build + little perf test + anyLast

* minor changes

* bug fix

* minor changes

* another minor changes

* fix + experement

* experement 2.0

* experement 3.0

* experement 3.1

* experement 4.0

* last experement(think so)

* just another attempt to fix UB

* minor changes

* I think I won

* ya.make

* fix requested changes

Co-authored-by: Artem Zuikov <chertus@gmail.com>
2020-06-18 19:34:29 +03:00
Amos Bird
d524301038
add scalars in time 2020-06-19 00:14:19 +08:00
Alexander Tokmakov
5e42441a72 add test 2020-06-18 17:19:14 +03:00
Nikolai Kochetov
35ab9ad051 Merge branch 'master' into fix-parallel-final-stuck 2020-06-18 16:04:14 +03:00
Artem Zuikov
6af36cb703
CAST keep nullable (#11733) 2020-06-18 13:18:28 +03:00
alexey-milovidov
c9924b8aa1
Merge pull request #11697 from CurtizJ/fix-with-fill
Fix 'ORDER BY ... WITH FILL' over const columns.
2020-06-18 06:38:24 +03:00
Anton Popov
8ba5bd8530 Merge remote-tracking branch 'upstream/master' into distinct-combinator 2020-06-18 01:44:36 +03:00
tavplubix
bb16544bee
Merge pull request #11732 from ClickHouse/fix_type_duduction_in_values_format
Fix type duduction in Values format
2020-06-18 00:21:03 +03:00
Anton Popov
88b325dcdc rework distinct combinator 2020-06-17 22:36:27 +03:00
alesapin
ac93f2795c
Merge pull request #11706 from ClickHouse/add_column_check
Add column if not exists tests
2020-06-17 20:09:57 +03:00
Vitaly Baranov
7011401cd4
Merge pull request #11670 from vitlibar/rbac-improve-parser-multiple-entities
RBAC: improve syntax
2020-06-17 16:26:02 +03:00
Alexander Tokmakov
3938b051c2 fix type duduction in values format 2020-06-17 16:21:06 +03:00
Anton Popov
5abbaeecf5 distinct combinator for single numeric arguments 2020-06-17 15:01:47 +03:00
Nikita Mikhaylov
d2be6c036b
Merge pull request #11602 from ClickHouse/fix-buffer-sample
Fix buffer sample
2020-06-17 14:44:48 +04:00
xPoSx
2d8110ce45
Eliminate min/max/any aggregators of GROUP BY keys in SELECT section (#11667) 2020-06-17 12:32:43 +03:00
alesapin
aaa1063275 Fix test and additional case 2020-06-17 11:29:56 +03:00
alexey-milovidov
e07d02015c
Merge pull request #11661 from ClickHouse/return-not-nullable-from-count-distinct-2
Return non-Nullable results from COUNT(DISTINCT), more complete.
2020-06-17 09:25:45 +03:00
alexey-milovidov
598fd75caf
Merge pull request #11639 from ClickHouse/remove-leader-election-2
Remove leader election, step 2: allow multiple leaders
2020-06-17 09:25:23 +03:00
Alexey Milovidov
634829bccb Merge branch 'return-not-nullable-from-count-distinct-2' of github.com:yandex/ClickHouse into return-not-nullable-from-count-distinct-2 2020-06-17 02:01:49 +03:00
Alexey Milovidov
69067efe7a Update test 2020-06-17 02:01:23 +03:00
Alexey Milovidov
a655765040 Merge branch 'master' into return-not-nullable-from-count-distinct-2 2020-06-17 01:15:57 +03:00
Alexander Kuzmenkov
9cab822718
Merge pull request #11683 from ClickHouse/aku/merge-minmap-11603
Merge minMap, maxMap (#11603)
2020-06-17 00:02:33 +03:00
Artem Zuikov
857d84bf7c
Buffer left blocks optimisation for MergeJoin (#10601) 2020-06-16 23:13:18 +03:00
alexey-milovidov
f387752cf1
Merge pull request #11702 from ClickHouse/fix-mv-header
Fix mv header
2020-06-16 22:43:38 +03:00
Alexey Milovidov
0a6054eece Merge branch 'master' into return-not-nullable-from-count-distinct-2 2020-06-16 22:31:33 +03:00
alexey-milovidov
ad53ef2c14
Merge pull request #11686 from ClickHouse/proper-query-formatting-in-log
Don't break multiline queries when they are printed in server log
2020-06-16 22:17:30 +03:00
alexey-milovidov
47bd8df9de
Merge pull request #11692 from ClickHouse/fix-flaky-test-incremental-streaming
Fix flaky test 01064_incremental_streaming_from_2_src_with_feedback
2020-06-16 22:12:46 +03:00
alexey-milovidov
d55155502f
Merge pull request #11694 from ClickHouse/maybe-fix-max-parser-depth-test
Maybe fix max_parser_depth flaky test
2020-06-16 22:11:36 +03:00
alesapin
d214a504d7
Merge pull request #11681 from ClickHouse/fix_mutations_premature_finish
Fix mutations premature "finish"
2020-06-16 21:26:14 +03:00
Anton Popov
d92ba0b216 Merge remote-tracking branch 'upstream/master' into distinct-combinator 2020-06-16 21:13:20 +03:00
Artem Zuikov
a3d6e047fe
Fix lambdas with multiple_joins_rewriter v2 (#11587) 2020-06-16 19:32:32 +03:00
Ivan Blinkov
7e2bb1fd83 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-16 16:02:55 +03:00
alesapin
0ac5132eb0 Add some tests for alter add column if not exists 2020-06-16 14:36:39 +03:00
alesapin
c403dd6b2f Add test for alter add column if not exists 2020-06-16 14:31:54 +03:00
alesapin
d5b76b2fe1 Fix parts count in mutations 2020-06-16 13:34:59 +03:00
alesapin
0cb14c836c
Merge pull request #11691 from ClickHouse/fix-race-condition-system-sync-replicas
Fix race condition in SYSTEM SYNC REPLICA
2020-06-16 13:29:18 +03:00
alesapin
362aaf1385
Merge pull request #11516 from ClickHouse/consistent_metadata4
Storage metadata as a single struct.
2020-06-16 13:28:22 +03:00
xPoSx
9fc37e1f75
Remove functions of other keys in GROUP BY section (#10051) 2020-06-16 12:14:25 +03:00
Vitaly Baranov
c5229301fc
Merge pull request #11671 from vitlibar/fix-minor-bugs-in-rbac
Fix minor bugs in RBAC
2020-06-16 12:09:54 +03:00
Nikolai Kochetov
86bafbe6c3 Added test from #9043 2020-06-16 11:25:59 +03:00
Alexey Milovidov
125b007360 Merge branch 'master' into remove-leader-election-2 2020-06-16 05:56:29 +03:00
Anton Popov
03058c3c25 fix 'ORDER BY WITH FILL' over const columns 2020-06-16 05:56:16 +03:00
alexey-milovidov
679c3f8ff3
Update 00446_clear_column_in_partition_zookeeper.sql 2020-06-16 04:50:22 +03:00
Alexey Milovidov
edff54e3ab Maybe fix max_parser_depth test 2020-06-16 04:08:01 +03:00
Alexey Milovidov
c85b9ae151 Fix test 2020-06-16 02:43:05 +03:00
Alexey Milovidov
97d5897d2b Fix test 2020-06-16 02:42:10 +03:00
Alexey Milovidov
a753ba9c6a Checked and corrected all other tests 2020-06-16 02:24:32 +03:00
Alexey Milovidov
e0f4c64acf Fix flaky test 2020-06-16 02:18:15 +03:00
Alexey Milovidov
00224ee94f Added a test 2020-06-16 02:04:12 +03:00
Anton Popov
eabbabed04 fix 'LIMIT WITH TIES' with aliases 2020-06-16 01:24:00 +03:00
Alexey Milovidov
ba82441989 Added a test 2020-06-15 23:13:57 +03:00
alesapin
bd17aa8a0e Increase timeout in test 2020-06-15 22:11:17 +03:00
Vitaly Baranov
a5b70fbdda Add tests. 2020-06-15 22:07:47 +03:00
Alexander Tokmakov
d6e69211b1 fix test 2020-06-15 22:05:36 +03:00
Alexander Kuzmenkov
82b5ada130
Merge pull request #11650 from ksvladimir/master
Allow using groupArrayArray and groupUniqArrayArray as SimpleAggregateFunction
2020-06-15 21:29:57 +03:00
Vitaly Baranov
4bd00b02e2 Improve syntax of CREATE QUOTA. Now resource types and key types could be written with underscores.
Also rename columns key_type=>keys and source=>storage in table system.quotas.
2020-06-15 20:10:34 +03:00
Alexey Milovidov
a5e9d793b0 Merge branch 'master' into avoid-trailing-whitespaces-in-some-cases 2020-06-15 19:48:07 +03:00
Alexander Kuzmenkov
0743e8c15d Merge remote-tracking branch 'origin/master' into minmap 2020-06-15 19:39:57 +03:00
alexey-milovidov
3878254e0c
Trigger CI 2020-06-15 19:21:52 +03:00
Alexander Kuzmenkov
e460d7c2bd
Merge pull request #11638 from azat/skip-idx-bloom-filter-fix
Fix bloom filters for String (data skipping indices)
2020-06-15 18:41:04 +03:00
alesapin
8f66c1b22c Merge with master 2020-06-15 17:31:37 +03:00
alesapin
8945d0073a Fix misunderstanding bug in mutations finalization 2020-06-15 16:37:40 +03:00
Vitaly Baranov
c7140724a8 Fix that ALTER USER RENAME could change allowed hosts. 2020-06-15 16:34:16 +03:00
Artem Zuikov
24059efad5
Change push down logic in VIEW (#11513) 2020-06-15 15:36:10 +03:00
Nikolai Kochetov
f9c4721ba2 Added test. 2020-06-15 15:04:30 +03:00
Mikhail Malafeev
10566e2b43
Remove duplicate ORDER BY and DISTINCT from subqueries (#10067) 2020-06-15 15:03:01 +03:00
tavplubix
20b8e1e9ec
Merge pull request #11057 from Potya/null_data_mods
Null data mods
2020-06-15 13:04:51 +03:00
alesapin
bc9a3c0458 Merge branch 'master' into consistent_metadata4 2020-06-15 11:20:30 +03:00
Alexey Milovidov
65a8fe7cf0 Update tests 2020-06-15 09:14:58 +03:00
Alexey Milovidov
6ddc6d7f08 Make the test faster #11637 2020-06-15 07:51:27 +03:00
Alexey Milovidov
e99c6d9143 Update tests 2020-06-15 07:42:29 +03:00
Alexey Milovidov
94d55abfd1 Update tests 2020-06-15 07:40:03 +03:00
Alexey Milovidov
d5e3e7ff76 Update tests 2020-06-15 07:36:55 +03:00
Alexey Milovidov
e2607f005c Fix error with ALTER CONSTRAINT formatting; added a test 2020-06-15 07:36:20 +03:00
Alexey Milovidov
0c1b2d48a3 Update test 2020-06-15 06:58:07 +03:00
Alexey Milovidov
fdc8f7ad14 Merge branch 'master' into avoid-trailing-whitespaces-in-some-cases 2020-06-15 06:52:14 +03:00
alexey-milovidov
fd6115cccf
Merge pull request #11648 from ClickHouse/simplify-code-functions-comparison-3
Allow comparison with String in index analysis; simplify code.
2020-06-15 05:47:03 +03:00
Alexey Milovidov
b51cbbdf15 Update test 2020-06-15 04:08:56 +03:00
alexey-milovidov
550838adbb
Merge pull request #11647 from ClickHouse/simplify-code-functions-comparison-2
Allow comparison of numbers with constant string in IN and VALUES
2020-06-15 03:33:45 +03:00
Alexey Milovidov
b596831342 Merge branch 'master' into remove-leader-election-2 2020-06-15 03:16:41 +03:00
alexey-milovidov
814265b66e
Merge pull request #11665 from ClickHouse/clear-password-from-command-line
Clear password from command line
2020-06-15 02:17:21 +03:00
alexey-milovidov
71dd46bd64
Merge pull request #11664 from ClickHouse/fix-syntax-hilite-create-user
Fix syntax hilite in CREATE USER query
2020-06-15 02:17:11 +03:00
alexey-milovidov
9b6942aa46
Merge pull request #11663 from ClickHouse/fix-timeout-in-test-sql-fuzzy
Fix timeout in sql_fuzzy test
2020-06-15 01:18:34 +03:00
Alexey Milovidov
303d1ebdaf Fix error; more tests 2020-06-15 01:17:06 +03:00
Alexey Milovidov
c4f18d2896 Fix tests 2020-06-15 00:39:22 +03:00
Alexey Milovidov
22366471d0 Added a test 2020-06-14 22:31:45 +03:00
Alexey Milovidov
3663e2a47e Fix syntax hilite in CREATE USER query 2020-06-14 22:11:32 +03:00
Alexey Milovidov
3aedef99ce Added a test 2020-06-14 21:31:42 +03:00
Alexey Milovidov
db0fc6c9a6 Fix tests 2020-06-14 21:24:25 +03:00
Alexey Milovidov
5d891f6c87 Fix tests 2020-06-14 21:13:39 +03:00
alexey-milovidov
561d4b5b01
Merge pull request #11655 from ClickHouse/position-in-system-columns
Added column "position" to system.columns and "column_position" to system.parts_columns
2020-06-14 21:01:41 +03:00
Alexey Milovidov
fcd23d02ee Fix timeout in sql_fuzzy test 2020-06-14 20:57:50 +03:00
Alexey Milovidov
394fb64a9c Better way of implementation 2020-06-14 20:42:11 +03:00
alexey-milovidov
d990b98b90
Merge pull request #11593 from ClickHouse/return-not-nullable-from-count-distinct
Return non-Nullable results from COUNT(DISTINCT)
2020-06-14 20:39:19 +03:00
Alexander Tokmakov
6a439a5eb5 fixes 2020-06-14 19:39:06 +03:00
Alexander Tokmakov
5f9739fbf9 Merge branch 'master' into fork-potya/null_data_mods 2020-06-14 18:11:55 +03:00
Ivan Blinkov
b8611cf46c experiment 2020-06-14 18:05:15 +03:00
alexey-milovidov
88cf3f7083
Merge pull request #11653 from ClickHouse/parse-date-time-best-effort-incompletely-parsed-value
Return NULL or zero when value is not parsed completely in parseDateTimeBestEffortOr* functions
2020-06-14 10:59:10 +03:00
alexey-milovidov
56b8d81ef6
Merge pull request #11651 from ClickHouse/empty-parameters-url
Skip empty URL parameters
2020-06-14 09:00:52 +03:00
Alexey Milovidov
3958a032ac Added a test 2020-06-14 08:15:29 +03:00
alexey-milovidov
b9b725a39f
Merge pull request #11649 from ClickHouse/case-insensitive-regexp
Allow case-insensitive regexps; added a test
2020-06-14 07:55:09 +03:00
Alexey Milovidov
e2f7a41a1a Fix error; clarify more results in test 2020-06-14 07:16:52 +03:00
Alexey Milovidov
4024840795 Add a test 2020-06-14 07:16:52 +03:00
Alexey Milovidov
e24576c56c Update test 2020-06-14 07:13:27 +03:00
Alexey Milovidov
fb040ef09f Update test result (now it is better) 2020-06-14 07:09:02 +03:00
Alexey Milovidov
59d4df19f0 Update test 2020-06-14 07:05:31 +03:00
Alexey Milovidov
cb395ff099 Update test 2020-06-14 07:04:19 +03:00
Alexey Milovidov
d6cf62e587 Added column "position" to system.columns and "column_position" to system.parts_columns 2020-06-14 04:56:22 +03:00
Alexey Milovidov
fae12d5e42 Return NULL or zero when value is not parsed completely in parseDateTimeBestEffortOr* functions 2020-06-14 04:45:03 +03:00