thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 12:32:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
90,805 Commits 9,194 Branches 1,782 Tags 2.2 GiB
dfa5d0f83e
Commit Graph

18391 Commits

Author SHA1 Message Date
Yakov Olkhovskiy
d47141b03d tests are fixed 2022-06-01 17:34:19 -04:00
Yakov Olkhovskiy
d076ef841b number's parser refactoring, no octal support 2022-06-01 16:55:03 -04:00
Alexey Milovidov
b5f48a7d3f Merge branch 'master' of github.com:ClickHouse/ClickHouse into llvm-14 2022-06-01 22:09:58 +02:00
Robert Schulze
366f368d06
Disallow LIKE patterns with trailing escape 
Trailing escape ('ab\') is disallowed in SQL, in standardese:

  "If an escape character is specified, then [...] If there is not a
  partitioning of the string PVC into substrings such that each substring
  has length 1 (one) or 2, no substring of length 1 (one) is the escape
  character ECV, and each substring of length 2 is the escape character
  ECV followed by either the escape character ECV, an <underscore>
  character, or the <percent> character, then an exception condition is
  raised: data exception - invalid escape sequence."

I first thought this is checked already higher up in the stack, at least
for const needles, as single trailing backslashes ('ab\') are rejected,
but then I realized that ClickHouse quotes by default. I.e., double
trailing backslashes ('ab\\') are not rejected but when interpreted as
LIKE needle ('ab\') they should.
 2022-06-01 21:38:46 +02:00
Robert Schulze
b3b0716b32
Merge pull request #37544 from ClickHouse/cached_patterns 
Cache compiled regexps when evaluating non-const needles
 2022-06-01 19:55:25 +02:00
avogar
4abfd54dd6 Fix possible segfault in schema inference 2022-06-01 16:53:37 +00:00
alesapin
b7e8bbb154
Merge pull request #37679 from ClickHouse/fix-keeper-recovery-test 
Fix `test_keeper_force_recovery*` tests
 2022-06-01 18:45:32 +02:00
Alexey Milovidov
89638de521
Merge pull request #37738 from ClickHouse/fix-intersect-with-const 
Fix `Intersect` with constant strings
 2022-06-01 19:31:55 +03:00
Yakov Olkhovskiy
e23cec01d5
Merge pull request #37581 from ClickHouse/http-named-collection 
Support for HTTP source for Data Dictionaries in Named Collections
 2022-06-01 11:55:04 -04:00
avogar
7ef02a2e44 Fix possible logical error in values table function 2022-06-01 15:32:33 +00:00
Dmitry Novik
7fbe91ca81
Merge pull request #37460 from ClickHouse/memory-overcommit-improvement 
Memory Overcommit: update defaults, exception message and add ProfileEvent
 2022-06-01 17:06:33 +02:00
Amos Bird
139264d881
Less parallelism instead 2022-06-01 23:02:31 +08:00
Alexander Tokmakov
002f95741a fix 2022-06-01 16:58:24 +02:00
alesapin
b3b3d7a459 Fix test 2022-06-01 16:58:07 +02:00
Sema Checherinda
16dc3ed97d FR: Expose what triggered the merge in system.part_log #26255 2022-06-01 16:58:07 +02:00
Sema Checherinda
2626a49616 FR: Expose what triggered the merge in system.part_log #26255 2022-06-01 16:58:06 +02:00
Alexander Tokmakov
ca4f7287ff Merge branch 'master' into try_fix_tests2 2022-06-01 16:52:54 +02:00
Vladimir C
2622791b9e
upd 2022-06-01 16:17:17 +02:00
Vladimir C
3a824ef9a4
Add no-backward-compatibility-check to 02315_pmj_union_ubsan_35857 2022-06-01 16:00:30 +02:00
Anton Popov
035f829a28 fix test 2022-06-01 13:29:03 +00:00
Kseniia Sumarokova
7afcfcbaaf
Merge pull request #37691 from kssenii/fix-rabbitmq-restart-with-no-settings 
Fix rabbitmq restart with empty settings
 2022-06-01 14:59:34 +02:00
vdimir
6a5f5997ca
Add test 02315_pmj_union_ubsan_35857 2022-06-01 12:55:47 +00:00
Maksim Kita
4413f0c7c1 Fixed tests 2022-06-01 14:52:01 +02:00
Kseniia Sumarokova
503d94f73b
Update test.py 2022-06-01 14:44:46 +02:00
flynn
ac10a6dc28 update test 2022-06-01 12:41:36 +00:00
Vladimir C
773eb15afc
Try mound instead of volume docker arg in stress_check 2022-06-01 14:36:51 +02:00
Alexander Tokmakov
75f49a48e1 Merge branch 'master' into fix_trash 2022-06-01 14:20:46 +02:00
Anton Popov
011131198c Merge remote-tracking branch 'upstream/master' into fix-mutations-again 2022-06-01 12:04:18 +00:00
Nikolai Kochetov
e401ab8169 Fix more tests. 2022-06-01 11:51:56 +00:00
Alexander Tokmakov
153862dc31 fix tests 
(cherry picked from commit ed52f463d95ac49431600f167a08507d7fcaaaf0)
 2022-06-01 13:47:52 +02:00
Kruglov Pavel
251be860e7
Merge pull request #37428 from loyd/fix/37420-rowbinary-bom 
Stop removing UTF-8 BOM in RowBinary format
 2022-06-01 13:36:55 +02:00
Vladimir C
8c0dba7302
Merge pull request #37650 from amosbird/joinget-fix 
Fix joinGet with  join_use_nulls = 1 and Array type
 2022-06-01 13:30:29 +02:00
Vladimir C
c466cdebf4
Merge pull request #37530 from vdimir/join_cond_dict_issue_37386 2022-06-01 13:29:01 +02:00
Antonio Andelic
6c31d06b2e Add test for const string intersect 2022-06-01 11:17:56 +00:00
Maksim Kita
9aee3f3156 Fixed tests 2022-06-01 13:06:44 +02:00
Anton Popov
cae2478b3f Revert "Merge pull request #37355 from ClickHouse/revert-37266-fix-mutations-with-object" 
This reverts commit a53cfa9fca, reversing
changes made to 9acb42fcdb.
 2022-06-01 10:57:20 +00:00
Maksim Kita
d84a21aea5 Fixed tests 2022-06-01 12:57:19 +02:00
Antonio Andelic
6c2db00f1f
Merge pull request #37568 from ClickHouse/tiny_fixes_for_jepsen 
Slightly better jepsen tests
 2022-06-01 12:16:09 +02:00
Robert Schulze
81318e07d6
Try to fix performance test results 2022-06-01 11:53:37 +02:00
Han Fei
dd7e609585 add zk_write.xml 2022-06-01 17:08:19 +08:00
Han Fei
f6ecf2522b refine tests 2022-06-01 17:02:55 +08:00
Alexey Milovidov
154cae4356
Merge pull request #37704 from vdimir/failed_test_default_sort 
Display entires for failed tests at the top of report by default
 2022-06-01 12:00:55 +03:00
Alexey Milovidov
31b3350749
Merge pull request #37710 from ClickHouse/fix-grouping-function 
Make GROUPING function skip constant folding
 2022-06-01 12:00:14 +03:00
Alexey Milovidov
a0020cb55c
Merge pull request #37724 from CurtizJ/fix-ast-optimizations-remote 
Fix `optimize_monotonous_functions_in_order_by` in distributed queries
 2022-06-01 11:54:45 +03:00
Vladimir C
42feae8940
fix stress_check.py 2022-06-01 10:44:44 +02:00
Han Fei
ea693dd0c2 add config and change test logic 2022-06-01 14:57:07 +08:00
Antonio Andelic
df0a1d523e add comment for overwrite 2022-06-01 06:32:52 +00:00
Paul Loyd
32d267ec6c
Stop removing UTF-8 BOM in RowBinary* formats 
Fixes #37420
 2022-06-01 13:12:55 +08:00
Anton Popov
6cf9405f09 fix optimize_monotonous_functions_in_order_by in distributed queries 2022-06-01 00:50:28 +00:00
Alexander Tokmakov
77c06447d5
Merge pull request #37622 from ClickHouse/try_fix_tests 
Try fix tests
 2022-06-01 01:11:28 +03:00
Anton Popov
20e319d67a
Merge pull request #37666 from CurtizJ/optimize-coalesce 
Optimize function `COALESCE` with two arguments
 2022-05-31 23:48:13 +02:00
Nikolai Kochetov
e993343c89 Fix another one test. 2022-05-31 21:38:56 +00:00
Nikolai Kochetov
04c14e9c5d Fix tests and add comment. 2022-05-31 20:59:50 +00:00
Nikolai Kochetov
9954c59dc1 Update test. 2022-05-31 19:40:50 +00:00
vdimir
cf82df6ce7
Use CLICKHOUSE_TEST_ZOOKEEPER_PREFIX in 00814_replicated_minimalistic_part_header 2022-05-31 19:26:44 +00:00
Nikolai Kochetov
157760ac85 Fix test. 2022-05-31 19:01:25 +00:00
vdimir
fbfab8558a
Add retries to 00814_replicated_minimalistic_part_header_zookeeper 2022-05-31 18:53:36 +00:00
vdimir
fdd190d010
Fix typo in 00814_replicated_minimalistic_part_header_zookeeper.sql 2022-05-31 18:40:14 +00:00
Nikolai Kochetov
86fbb74703 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-31 18:07:47 +00:00
Nikolai Kochetov
32010f0ba8 Add a test. 2022-05-31 17:56:48 +00:00
vdimir
4d336d0228
Get rid of duplicate download_previous_release 2022-05-31 17:50:29 +00:00
mergify[bot]
d57d987a02
Merge branch 'master' into sql-user-defined-functions-readonly-fix 2022-05-31 16:50:00 +00:00
Maksim Kita
66f43b9ad3 Fix executable user default functions execution with Nullable arguments 2022-05-31 18:46:33 +02:00
Dmitry Novik
b11749ca2c Make GROUPING function skip constant folding 2022-05-31 16:45:29 +00:00
vdimir
c5ac6294ae
Display entires for failed tests at the top of report 2022-05-31 16:18:16 +00:00
vdimir
ca0bd754b5
Add no-backward-compatibility-check to 01391_join_on_dict_crash.sql 2022-05-31 16:02:58 +00:00
vdimir
673bc84bfc
Reformat 02244_lowcardinality_hash_join 2022-05-31 16:02:57 +00:00
vdimir
2476c6a988
Fix error on joining with dictionary on some conditions 2022-05-31 16:02:57 +00:00
Vladimir C
2a38fdb796
Merge pull request #37653 from vdimir/cross_join_dup_col_names 2022-05-31 17:50:19 +02:00
Maksim Kita
d1a4550b4f Fix create or drop of sql user defined functions in readonly mode 2022-05-31 17:23:41 +02:00
Antonio Andelic
49f815060a Use tar for logs 2022-05-31 15:18:44 +00:00
Han Fei
5693e6212d add config and fix style check 2022-05-31 23:18:05 +08:00
Maksim Kita
bacee7f19c
Merge pull request #37195 from kitaisreal/merging-sorted-algorithm-single-column-specialization 
MergingSortedAlgorithm single column specialization
 2022-05-31 16:46:18 +02:00
Anton Popov
00f87b0f57 replace multiIf to if in case of one condition 2022-05-31 14:45:12 +00:00
Antonio Andelic
737d6ab1e3 Merge branch 'master' into tiny_fixes_for_jepsen 2022-05-31 13:53:55 +00:00
Antonio Andelic
792adb0576 Update jepsen and scp 2022-05-31 13:53:45 +00:00
Yakov Olkhovskiy
4b427336e3 tests with overridden and appended parameters 2022-05-31 09:37:34 -04:00
Dmitry Novik
b41fe00f31
Merge pull request #37542 from azat/grouping-sets-fix-optimize_aggregation_in_order 
Prohibit optimize_aggregation_in_order with GROUPING SETS (fixes LOGICAL_ERROR)
 2022-05-31 15:31:45 +02:00
Dmitry Novik
f58623a375
Merge pull request #37593 from azat/union-type-cast-resubmit 
Fix converting types for UNION queries (may produce LOGICAL_ERROR)
 2022-05-31 15:27:50 +02:00
mergify[bot]
ba49c6bb46
Merge branch 'master' into memory-overcommit-improvement 2022-05-31 13:17:06 +00:00
alesapin
473b0bd0db
Merge pull request #37604 from ClickHouse/turn_on_s3_tests 
Turn on s3 tests to red mode
 2022-05-31 15:01:24 +02:00
kssenii
c2087b3145 Fix 2022-05-31 14:38:11 +02:00
Kruglov Pavel
7cc87d9a65
Merge pull request #37537 from Avogar/skip-first-lines 
Allow to skip some of the first lines in CSV/TSV formats
 2022-05-31 14:26:21 +02:00
kssenii
69cd3a2b10 Fix 2022-05-31 14:20:31 +02:00
mergify[bot]
d85c3ec69e
Merge branch 'master' into turn_on_s3_tests 2022-05-31 11:58:16 +00:00
Antonio Andelic
582be42329 Wait for leader election 2022-05-31 11:53:46 +00:00
Alexander Tokmakov
30a7b07d97
Merge pull request #37658 from vitlibar/fix-flaky-test_row_policy 
Fix flaky test test_row_policy
 2022-05-31 12:59:50 +03:00
Kseniia Sumarokova
73ed9c3977
Merge pull request #37619 from Vxider/wv-fix-table-identifier 
Fix bugs in WindowView when using table identifier
 2022-05-31 11:07:11 +02:00
Han Fei
7870e02fdf add user access tests 2022-05-31 11:16:06 +08:00
Anton Popov
30f8eb800a optimize function coalesce with two arguments 2022-05-30 22:29:35 +00:00
mergify[bot]
55913cf8e1
Merge branch 'master' into turn_on_s3_tests 2022-05-30 20:52:40 +00:00
mergify[bot]
b43cfd056f
Merge branch 'master' into floating_seconds 2022-05-30 19:18:35 +00:00
Nikolai Kochetov
77b07dd0a8
Merge pull request #37163 from ClickHouse/grouping-function 
Add GROUPING function
 2022-05-30 20:45:04 +02:00
Robert Schulze
ad12adc31c
Measure and rework internal re2 caching 
This commit is based on local benchmarks of ClickHouse's re2 caching.

Question 1: -----------------------------------------------------------
Is pattern caching useful for queries with const LIKE/REGEX
patterns? E.g. SELECT LIKE(col_haystack, '%HelloWorld') FROM T;

The short answer is: no. Runtime is (unsurprisingly) dominated by
pattern evaluation + other stuff going on in queries, but definitely not
pattern compilation. For space reasons, I omit details of the local
experiments.

(Side note: the current caching scheme is unbounded in size which poses
a DoS risk (think of multi-tenancy). This risk is more pronounced when
unbounded caching is used with non-const patterns ..., see next
question)

Question 2: -----------------------------------------------------------
Is pattern caching useful for queries with non-const LIKE/REGEX
patterns? E.g. SELECT LIKE(col_haystack, col_needle) FROM T;

I benchmarked five caching strategies:
1. no caching as a baseline (= recompile for each row)
2. unbounded cache (= threadsafe global hash-map)
3. LRU cache (= threadsafe global hash-map + LRU queue)
4. lightweight local cache 1 (= not threadsafe local hashmap with
   collision list which grows to a certain size (here: 10 elements) and
   afterwards never changes)
5. lightweight local cache 2 (not threadsafe local hashmap without
   collision list in which a collision replaces the stored element, idea
   by Alexey)

... using a haystack of 2 mio strings and
A). 2 mio distinct simple patterns
B). 10 simple patterns
C)  2 mio distinct complex patterns
D)  10 complex patterns

Fo A) and C), caching does not help but these queries still allow to
judge the static overhead of caching on query runtimes.

B) and D) are extreme but common cases in practice. They include
queries like "SELECT ... WHERE LIKE (col_haystack, flag ? '%pattern1%' :
'%pattern2%'). Caching should help significantly.

Because LIKE patterns are internally translated to re2 expressions, I
show only measurements for MATCH queries.

Results in sec, averaged over on multiple measurements;

1.A): 2.12
  B): 1.68
  C): 9.75
  D): 9.45

2.A): 2.17
  B): 1.73
  C): 9.78
  D): 9.47

3.A): 9.8
  B): 0.63
  C): 31.8
  D): 0.98

4.A): 2.14
  B): 0.29
  C): 9.82
  D): 0.41

5.A) 2.12 / 2.15 / 2.26
  B) 1.51 / 0.43 / 0.30
  C) 9.97 / 9.88 / 10.13
  D) 5.70 / 0.42 / 0.43
(10/100/1000 buckets, resp. 10/1/0.1% collision rate)

Evaluation:

1. This is the baseline. It was surprised that complex patterns (C, D)
   slow down the queries so badly compared to simple patterns (A, B).
   The runtime includes evaluation costs, but as caching only helps with
   compilation, and looking at 4.D and 5.D, compilation makes up over 90%
   of the runtime!

2. No speedup compared to 1, probably due to locking overhead. The cache
   is unbounded, and in experiments with data sets > 2 mio rows, 2. is
   the only scheme to throw OOM exceptions which is not acceptable.

3. Unique patterns (A and C) lead to thrashing of the LRU cache and very
   bad runtimes due to LRU queue maintenance and locking. Works pretty
   well however with few distinct patterns (B and D).

4. This scheme is tailored to queries B and D where it performs pretty
   good. More importantly, the caching is lightweight enough to not
   deteriorate performance on datasets A and C.

5. After some tuning of the hash map size, 100 buckets seem optimal to
   be in the same ballpark with 10 distinct patterns as 4. Performance
   also does not deteriorate on A and C compared to the baseline.
   Unlike 4., this scheme behaves LRU-like and can adjust to changing
   pattern distributions.

As a conclusion, this commit implementes two things:

1. Based on Q1, pattern search with const needle no longer uses
   caching. This applies to LIKE and MATCH + a few (exotic) other SQL
   functions. The code for the unbounded caching was removed.

2. Based on Q2, pattern search with non-const needles now use method 5.
 2022-05-30 20:00:35 +02:00
Han Fei
e15cdec39c address comments 2022-05-31 01:46:31 +08:00
Anton Popov
52d3791eb9
Merge pull request #37600 from CurtizJ/fix-with-fill-interval 
Fix `WITH FILL` with negative intervals in `STEP` clause
 2022-05-30 19:43:12 +02:00
alesapin
60b910a4de Fix 2022-05-30 19:04:25 +02:00
mergify[bot]
d4e722bbfa
Merge branch 'master' into http-named-collection 2022-05-30 16:40:18 +00:00
Vitaly Baranov
486a11a5e2 Fix flaky test test_row_policy. 2022-05-30 18:34:28 +02:00
vdimir
8a3f4bda62
Fix columns number mismatch in cross join 2022-05-30 15:40:15 +00:00
zvonand
a7840ef7f9 update tests(2) 2022-05-30 18:34:00 +03:00
Amos Bird
217f492264
Fix test 2022-05-30 21:38:24 +08:00
Nikolai Kochetov
5ef51ed27b Fix more tests. 2022-05-30 13:10:30 +00:00
Amos Bird
b68e8efaf0
Fix joinGet with cannot be null type 2022-05-30 21:01:27 +08:00
zvonand
d44bb61dec updated tests 2022-05-30 15:49:15 +03:00
mergify[bot]
87e896ae05
Merge branch 'master' into try_fix_tests 2022-05-30 12:44:35 +00:00
Kruglov Pavel
0615866aea
Merge pull request #37450 from Avogar/check-format-on-storage-creation 
Check format name on storage creation
 2022-05-30 14:23:20 +02:00
alesapin
362fa745e6 Ignore broken metadata 2022-05-30 13:32:12 +02:00
Nikolai Kochetov
5b4658aa5e Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-30 09:47:35 +00:00
Amos Bird
030fe6e272
Less flaky jbod rebalancer test 2022-05-30 16:53:55 +08:00
Maksim Kita
53d848f187 Fixed tests 2022-05-30 10:53:04 +02:00
Alexey Milovidov
9a5dd75a68 Merge branch 'master' into llvm-14 2022-05-30 05:34:38 +02:00
Alexey Milovidov
a67fb05ea0 Add test 2022-05-30 05:03:16 +02:00
alesapin
753bcee954 Fix lazy mark load 2022-05-29 18:38:09 +02:00
mergify[bot]
df9f7c59a2
Merge branch 'master' into try_fix_tests 2022-05-29 15:04:48 +00:00
alesapin
59a070778d More quite logging for S3 tests 2022-05-29 14:48:04 +02:00
alesapin
0e8ab36913 Merge branch 'master' into turn_on_s3_tests 2022-05-29 14:37:10 +02:00
Andrey Zvonov
2dbbf14de5
Merge branch 'master' into non-neg-deriv 2022-05-29 10:09:51 +03:00
Alexander Tokmakov
579b0e3323
Merge pull request #37627 from ClickHouse/revert-37416-fix_ReplicatedMergeTree_comments 
Revert "Implemented changing comment to a ReplicatedMergeTree table"
 2022-05-29 09:57:12 +03:00
Alexey Milovidov
9e3242f186
Merge pull request #37617 from CurtizJ/aggregation-sparse-columns 
Better performance with sparse columns in aggregate functions
 2022-05-29 09:36:07 +03:00
Alexander Tokmakov
562eec591e
Revert "Implemented changing comment to a ReplicatedMergeTree table" 2022-05-29 09:28:47 +03:00
Alexey Milovidov
975f16049e
Merge pull request #37597 from azat/fix-01317_no_password_in_command_line 
tests: fix 01317_no_password_in_command_line flakiness (and make it race free)
 2022-05-29 07:59:10 +03:00
zvonand
295a0f9ec2 added tests 2022-05-29 03:38:42 +03:00
Alexey Milovidov
c1169019d2 Merge branch 'master' into llvm-14 2022-05-29 02:29:02 +02:00
Alexey Milovidov
b2ba2533b1
Merge pull request #35231 from ClickHouse/time-zone-in-stateless-tests 
Change timezone in stateless tests
 2022-05-29 03:09:12 +03:00
Alexander Tokmakov
313f4833d6 fix 2022-05-28 23:21:55 +02:00
Alexander Tokmakov
9e3e795353 fix 2022-05-28 21:56:10 +02:00
mergify[bot]
c273568e45
Merge branch 'master' into fix-01317_no_password_in_command_line 2022-05-28 18:27:45 +00:00
Alexey Milovidov
8d0c46a105
Merge pull request #37614 from ClickHouse/test-reports-markup 
Remove margin in test reports and change the font
 2022-05-28 21:25:54 +03:00
Alexey Milovidov
e48af2efe4
Update report.py 2022-05-28 21:25:42 +03:00
Alexander Tokmakov
4e52f45695 Merge branch 'master' into fix_trash 2022-05-28 19:43:19 +02:00
Alexander Tokmakov
be498b0658 fix test 2022-05-28 19:40:26 +02:00
Alexander Tokmakov
f3c2ca6a13 Revert "Merge pull request #36634 from azat/01502_long_log_tinylog_deadlock_race-test" 
This reverts commit c04c62795e, reversing
changes made to 88f05ac14a.
 2022-05-28 19:28:38 +02:00
Alexander Tokmakov
70f39e0849 Revert "Merge pull request #36731 from azat/fix-async_inserts_stress_long-test" 
This reverts commit d1d2f2c1a4, reversing
changes made to 1ddb04b992.
 2022-05-28 19:27:40 +02:00
Alexander Tokmakov
9af6e237e1 remove retries 2022-05-28 19:25:56 +02:00
Alexander Tokmakov
1ae919a18a Revert "Merge pull request #35865 from azat/clickhouse-test-left-queries" 
This reverts commit 18d094d79d, reversing
changes made to 224f4dc620.
 2022-05-28 19:23:58 +02:00
Alexander Tokmakov
e33d986cf1 Revert "Merge pull request #34924 from azat/tests-overlaps" 
This reverts commit f13e436d40, reversing
changes made to aa3c05e9d4.
 2022-05-28 19:20:32 +02:00
Yakov Olkhovskiy
4f07c684da style fix 2022-05-28 12:45:53 -04:00
Yakov Olkhovskiy
3ff32fe81d test is added 2022-05-28 12:30:05 -04:00
Han Fei
d883194eb2 try to make test stable 2022-05-28 22:20:09 +08:00
Anton Popov
c39d95e2e6 add perf test 2022-05-28 12:56:38 +00:00
Anton Popov
1d9b3be7da
Merge pull request #37536 from CurtizJ/profile-events-for-part-types 
Add profile events for introspection of part types
 2022-05-28 14:25:21 +02:00
Han Fei
0f71231574 try fix flaky tests and refine code style 2022-05-28 17:25:33 +08:00
Vxider
b24346328d fix parser when using table identifer 2022-05-28 08:22:34 +00:00
Alexey Milovidov
15f2912dc9
Fix Python. 2022-05-28 05:10:29 +03:00
mergify[bot]
503eb3e9e6
Merge branch 'master' into time-zone-in-stateless-tests 2022-05-28 01:28:59 +00:00
Alexey Milovidov
f7a5b1fdaf Remove margin in test reports and change the font 2022-05-28 03:24:56 +02:00
Alexander Gololobov
6a57e1a970
Merge pull request #37601 from ClickHouse/array_norm_dist_fixes 
Added LpNorm and LpDistance functions for arrays
 2022-05-27 23:40:38 +02:00
Nikolai Kochetov
b80b1940ce Fix some tests. 2022-05-27 20:47:35 +00:00
Alexey Milovidov
8c94b75516 Merge branch 'master' into llvm-14 2022-05-27 22:24:20 +02:00
Alexey Milovidov
fa31d758d6 Update tests 2022-05-27 22:22:59 +02:00
Dmitry Novik
db2fe33926
Update test.py 2022-05-27 20:43:59 +02:00
Alexey Milovidov
bf1ad458da Merge branch 'master' into time-zone-in-stateless-tests 2022-05-27 19:38:13 +02:00
Han Fei
0d5448a621 fix flaky tests 2022-05-28 00:53:07 +08:00
mergify[bot]
923ad2e905
Merge branch 'master' into turn_on_s3_tests 2022-05-27 16:31:43 +00:00
Dmitry Novik
60b9d81773 Remove global_memory_usage_overcommit_max_wait_microseconds 2022-05-27 16:30:29 +00:00
alesapin
f63fa9bcc6
Merge pull request #37416 from Enmk/fix_ReplicatedMergeTree_comments 
Implemented changing comment to a ReplicatedMergeTree table
 2022-05-27 18:29:34 +02:00
Han Fei
dda86b49a6 update tests 2022-05-28 00:10:24 +08:00
mergify[bot]
cd602b20cd
Merge branch 'master' into tiny_fixes_for_jepsen 2022-05-27 16:06:08 +00:00
alesapin
0f5032c737
Merge pull request #37527 from ClickHouse/build-report 
Add failed builds to the build report
 2022-05-27 17:45:25 +02:00
Alexander Gololobov
540353566c Added LpNorm and LpDistance functions for arrays 2022-05-27 17:17:08 +02:00
Azat Khuzhin
8a224239c1 Prohibit optimize_aggregation_in_order with GROUPING SETS 
AggregatingStep ignores it anyway, and it leads to the following error
in getSortDescriptionFromGroupBy(), like in [1]:

    2022.05.24 04:29:29.279431 [ 3395 ] {26543564-8bc8-4a3a-b984-70a2adf0245d} <Fatal> : Logical error: 'Trying to get name of not a column: ExpressionList'.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36914/67d3ac72d26ab74d69f03c03422349d4faae9e19/stateless_tests__ubsan__actions_.html

v2: revert change to getSortDescriptionFromGroupBy() after
    GroupingSetsRewriterVisitor had been introduced
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 17:44:57 +03:00
Azat Khuzhin
2fb735ffc1 tests: add echo for 01883_grouping_sets_crash 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 17:44:50 +03:00
alesapin
be1c3c132b Fix some trash 2022-05-27 16:08:49 +02:00
Anton Popov
abc90fad8d fix WITH FILL with negative itervals 2022-05-27 12:42:51 +00:00
Han Fei
d7b372b73a fix style check 2022-05-27 20:40:55 +08:00
alesapin
32167cb6fb Turn on s3 tests 2022-05-27 14:36:12 +02:00
Azat Khuzhin
735d2dfebd tests: fix 01317_no_password_in_command_line flakiness (and make it race free) 
Before it was possible not to check if the query was gone already, also
it checks all processes not only the process of the client for the
possible password.

v2: make it parallel aware
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 15:27:48 +03:00
Anton Popov
d68c30a92e fix tests 2022-05-27 12:27:25 +00:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
Han Fei
447c5eb6ff add zookeeper tag 2022-05-27 19:48:14 +08:00
alesapin
461bb42fb8 Fix flaky test 2022-05-27 13:40:07 +02:00
Han Fei
c66fbacf39 update test 2022-05-27 19:23:48 +08:00
Azat Khuzhin
2613149f6b Fix converting types for UNION queries (may produce LOGICAL_ERROR) 
CI founds [1]:

    2022.02.20 15:14:23.969247 [ 492 ] {} <Fatal> BaseDaemon: (version 22.3.1.1, build id: 6082C357CFA6FF99) (from thread 472) (query_id: a5187ff9-962a-4e7c-86f6-8d48850a47d6) (query: SELECT 0., round(avgWeighted(x, y)) FROM (SELECT toDate(toDate('214748364.8', '-922337203.6854775808', '-0.1', NULL) - NULL, 10.000100135803223, '-2147483647'), 255 AS x, -2147483647 AS y UNION ALL SELECT y, NULL AS x, 2147483646 AS y)) Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/26d0e5438c86e52a145aaaf4cb523c399989a878/fuzzer_astfuzzerdebug,actions//report.html

The problem is that subqueries returns different headers:
- first query  -- x, y
- second query -- y, x

v2: Make order of columns strict only for UNION
    https://s3.amazonaws.com/clickhouse-test-reports/34775/9cc8c01a463d18c471853568b2f0af659a4e643f/stateless_tests__address__actions__[2/2].html
    Fixes: 00597_push_down_predicate_long
v3: add no-backward-compatibility-check for the test
Fixes: #37569
Resubmit: #34775
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit a813f5996e)
 2022-05-27 14:11:57 +03:00
Han Fei
2ea027ffcb Support insert into system.zookeeper 2022-05-27 18:53:12 +08:00
Kseniia Sumarokova
f5d69506b4
Merge pull request #37516 from KinderRiven/improve_local_cache 
Control cache downloads to avoid negative optimization of local caches
 2022-05-27 11:53:17 +02:00
Antonio Andelic
e1772c571c Compress clickhouse logs 2022-05-27 07:10:20 +00:00
Alexey Milovidov
d74333424b Fix test 2022-05-27 04:33:47 +02:00
Alexey Milovidov
393846e1ee Fix test 2022-05-27 04:08:04 +02:00
Dmitry Novik
3a9239b79f
Revert "RFC: Fix converting types for UNION queries (may produce LOGICAL_ERROR)" 2022-05-27 04:05:32 +02:00
Alexey Milovidov
faa519300e Fix more tests 2022-05-27 02:20:16 +02:00
Alexey Milovidov
d71780d34c Fix some tests 2022-05-27 02:18:38 +02:00
Alexey Milovidov
69a1a0c6d7
Merge branch 'master' into time-zone-in-stateless-tests 2022-05-27 03:04:42 +03:00
Alexey Milovidov
5b9bf7eb15 Fix tests 2022-05-27 01:59:45 +02:00
Alexey Milovidov
207ffc0f92 Merge branch 'master' into llvm-14 2022-05-27 01:18:51 +02:00
Alexey Milovidov
86afa3a245
Merge pull request #37502 from ClickHouse/array_norm_dist_fixes 
Renamed arrayXXNorm/arrayXXDistance functions to XXNorm/XXDistance and fixed some overflow cases
 2022-05-27 00:56:29 +03:00
Alexander Tokmakov
eb71dd4c78
Merge pull request #37547 from ClickHouse/followup_37398 
Follow-up to #37398
 2022-05-26 20:29:41 +03:00
Azat Khuzhin
4288d09a85 Do not write expired columns by TTL after merge w/o TTL 
Usually second merge do not perform TTL, since everything is up to date,
however in this case TTLTransform is not used, and hence expired_columns
will not be filled for new part, and so those columns will be written
with default values.

Avoid this, by manually filling expired_columns.

Here is a simpler reproducer:

Simple reproducer:

    ```sql
    create table ttl_02262 (date Date, key Int, value String TTL date + interval 1 month) engine=MergeTree order by key settings min_bytes_for_wide_part=0, min_rows_for_wide_part=0;
    insert into ttl_02262 values ('2010-01-01', 2010, 'foo');
    ```

    ```sh
    # ls -l .server/data/default/ttl_02262/all_*
    .server/data/default/ttl_02262/all_1_1_0:
    total 48
    -rw-r----- 1 root root 335 May 26 14:19 checksums.txt
    -rw-r----- 1 root root  76 May 26 14:19 columns.txt
    -rw-r----- 1 root root   1 May 26 14:19 count.txt
    -rw-r----- 1 root root  28 May 26 14:19 date.bin
    -rw-r----- 1 root root  48 May 26 14:19 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:19 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:19 key.bin
    -rw-r----- 1 root root  48 May 26 14:19 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:19 primary.idx
    -rw-r----- 1 root root  99 May 26 14:19 ttl.txt
    -rw-r----- 1 root root  30 May 26 14:19 value.bin
    -rw-r----- 1 root root  48 May 26 14:19 value.mrk2
    ```

    ```sql
    optimize table ttl_02262 final;
    ```

    ```sh
    .server/data/default/ttl_02262/all_1_1_1:
    total 40
    -rw-r----- 1 root root 279 May 26 14:19 checksums.txt
    -rw-r----- 1 root root  61 May 26 14:19 columns.txt
    -rw-r----- 1 root root   1 May 26 14:19 count.txt
    -rw-r----- 1 root root  28 May 26 14:19 date.bin
    -rw-r----- 1 root root  48 May 26 14:19 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:19 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:19 key.bin
    -rw-r----- 1 root root  48 May 26 14:19 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:19 primary.idx
    -rw-r----- 1 root root  81 May 26 14:19 ttl.txt
    ```

    ```sql
    optimize table ttl_02262 final;
    ```

    ```sh
    .server/data/default/ttl_02262/all_1_1_2:
    total 48
    -rw-r----- 1 root root 349 May 26 14:20 checksums.txt
    -rw-r----- 1 root root  76 May 26 14:20 columns.txt
    -rw-r----- 1 root root   1 May 26 14:20 count.txt
    -rw-r----- 1 root root  28 May 26 14:20 date.bin
    -rw-r----- 1 root root  48 May 26 14:20 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:20 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:20 key.bin
    -rw-r----- 1 root root  48 May 26 14:20 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:20 primary.idx
    -rw-r----- 1 root root  81 May 26 14:20 ttl.txt
    -rw-r----- 1 root root  27 May 26 14:20 value.bin
    -rw-r----- 1 root root  48 May 26 14:20 value.mrk2
    ```

And now we have `value.*` for all_1_1_2, this should not happen.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
mergify[bot]
8f7b9623c6
Merge branch 'master' into build-report 2022-05-26 16:59:22 +00:00
mergify[bot]
5e6d22055d
Merge branch 'master' into tiny_fixes_for_jepsen 2022-05-26 16:50:18 +00:00
Dmitry Novik
673a521d0b
Merge pull request #34775 from azat/union-type-cast 
RFC: Fix converting types for UNION queries (may produce LOGICAL_ERROR)
 2022-05-26 17:28:23 +02:00
alesapin
85c7b55b81 Slightly better jepsen tests 2022-05-26 16:54:46 +02:00
alesapin
8ae277a1c4 Fix test 2022-05-26 15:25:48 +02:00
KinderRiven
55379f0d03 fix config 2022-05-26 20:59:31 +08:00
Alexander Tokmakov
e8f33fb0d9 fix flaky tests 2022-05-26 14:17:05 +02:00
Maksim Kita
3a92e61827
Merge pull request #37148 from kitaisreal/dictionary-get-descendants-performance-improvement 
Dictionary getDescendants performance improvement
 2022-05-26 12:29:17 +02:00
KinderRiven
286c13317f fix stateless test 2022-05-26 17:38:42 +08:00
KinderRiven
ab4614dee5 fix 2022-05-26 16:39:50 +08:00
KinderRiven
f5a186c774 add test for local cache download 2022-05-26 16:36:26 +08:00
Dmitry Novik
5c3c994d2a
Merge pull request #37493 from ClickHouse/grouping-sets-optimization-fix 
Fix ORDER BY optimization in case of GROUPING SETS
 2022-05-26 02:25:02 +02:00