thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-20 08:40:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
90,769 Commits 8,927 Branches 1,765 Tags 2 GiB
88dfcaa892
Commit Graph

18383 Commits

Author SHA1 Message Date
Nikolai Kochetov
86fbb74703 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-31 18:07:47 +00:00
Nikolai Kochetov
32010f0ba8 Add a test. 2022-05-31 17:56:48 +00:00
vdimir
4d336d0228
Get rid of duplicate download_previous_release 2022-05-31 17:50:29 +00:00
mergify[bot]
d57d987a02
Merge branch 'master' into sql-user-defined-functions-readonly-fix 2022-05-31 16:50:00 +00:00
Maksim Kita
66f43b9ad3 Fix executable user default functions execution with Nullable arguments 2022-05-31 18:46:33 +02:00
Dmitry Novik
b11749ca2c Make GROUPING function skip constant folding 2022-05-31 16:45:29 +00:00
vdimir
c5ac6294ae
Display entires for failed tests at the top of report 2022-05-31 16:18:16 +00:00
vdimir
ca0bd754b5
Add no-backward-compatibility-check to 01391_join_on_dict_crash.sql 2022-05-31 16:02:58 +00:00
vdimir
673bc84bfc
Reformat 02244_lowcardinality_hash_join 2022-05-31 16:02:57 +00:00
vdimir
2476c6a988
Fix error on joining with dictionary on some conditions 2022-05-31 16:02:57 +00:00
Vladimir C
2a38fdb796
Merge pull request #37653 from vdimir/cross_join_dup_col_names 2022-05-31 17:50:19 +02:00
Maksim Kita
d1a4550b4f Fix create or drop of sql user defined functions in readonly mode 2022-05-31 17:23:41 +02:00
Antonio Andelic
49f815060a Use tar for logs 2022-05-31 15:18:44 +00:00
Han Fei
5693e6212d add config and fix style check 2022-05-31 23:18:05 +08:00
Maksim Kita
bacee7f19c
Merge pull request #37195 from kitaisreal/merging-sorted-algorithm-single-column-specialization 
MergingSortedAlgorithm single column specialization
 2022-05-31 16:46:18 +02:00
Anton Popov
00f87b0f57 replace multiIf to if in case of one condition 2022-05-31 14:45:12 +00:00
Antonio Andelic
737d6ab1e3 Merge branch 'master' into tiny_fixes_for_jepsen 2022-05-31 13:53:55 +00:00
Antonio Andelic
792adb0576 Update jepsen and scp 2022-05-31 13:53:45 +00:00
Yakov Olkhovskiy
4b427336e3 tests with overridden and appended parameters 2022-05-31 09:37:34 -04:00
Dmitry Novik
b41fe00f31
Merge pull request #37542 from azat/grouping-sets-fix-optimize_aggregation_in_order 
Prohibit optimize_aggregation_in_order with GROUPING SETS (fixes LOGICAL_ERROR)
 2022-05-31 15:31:45 +02:00
Dmitry Novik
f58623a375
Merge pull request #37593 from azat/union-type-cast-resubmit 
Fix converting types for UNION queries (may produce LOGICAL_ERROR)
 2022-05-31 15:27:50 +02:00
mergify[bot]
ba49c6bb46
Merge branch 'master' into memory-overcommit-improvement 2022-05-31 13:17:06 +00:00
alesapin
473b0bd0db
Merge pull request #37604 from ClickHouse/turn_on_s3_tests 
Turn on s3 tests to red mode
 2022-05-31 15:01:24 +02:00
kssenii
c2087b3145 Fix 2022-05-31 14:38:11 +02:00
Kruglov Pavel
7cc87d9a65
Merge pull request #37537 from Avogar/skip-first-lines 
Allow to skip some of the first lines in CSV/TSV formats
 2022-05-31 14:26:21 +02:00
kssenii
69cd3a2b10 Fix 2022-05-31 14:20:31 +02:00
mergify[bot]
d85c3ec69e
Merge branch 'master' into turn_on_s3_tests 2022-05-31 11:58:16 +00:00
Antonio Andelic
582be42329 Wait for leader election 2022-05-31 11:53:46 +00:00
Alexander Tokmakov
30a7b07d97
Merge pull request #37658 from vitlibar/fix-flaky-test_row_policy 
Fix flaky test test_row_policy
 2022-05-31 12:59:50 +03:00
Kseniia Sumarokova
73ed9c3977
Merge pull request #37619 from Vxider/wv-fix-table-identifier 
Fix bugs in WindowView when using table identifier
 2022-05-31 11:07:11 +02:00
Han Fei
7870e02fdf add user access tests 2022-05-31 11:16:06 +08:00
Anton Popov
30f8eb800a optimize function coalesce with two arguments 2022-05-30 22:29:35 +00:00
mergify[bot]
55913cf8e1
Merge branch 'master' into turn_on_s3_tests 2022-05-30 20:52:40 +00:00
mergify[bot]
b43cfd056f
Merge branch 'master' into floating_seconds 2022-05-30 19:18:35 +00:00
Nikolai Kochetov
77b07dd0a8
Merge pull request #37163 from ClickHouse/grouping-function 
Add GROUPING function
 2022-05-30 20:45:04 +02:00
Robert Schulze
ad12adc31c
Measure and rework internal re2 caching 
This commit is based on local benchmarks of ClickHouse's re2 caching.

Question 1: -----------------------------------------------------------
Is pattern caching useful for queries with const LIKE/REGEX
patterns? E.g. SELECT LIKE(col_haystack, '%HelloWorld') FROM T;

The short answer is: no. Runtime is (unsurprisingly) dominated by
pattern evaluation + other stuff going on in queries, but definitely not
pattern compilation. For space reasons, I omit details of the local
experiments.

(Side note: the current caching scheme is unbounded in size which poses
a DoS risk (think of multi-tenancy). This risk is more pronounced when
unbounded caching is used with non-const patterns ..., see next
question)

Question 2: -----------------------------------------------------------
Is pattern caching useful for queries with non-const LIKE/REGEX
patterns? E.g. SELECT LIKE(col_haystack, col_needle) FROM T;

I benchmarked five caching strategies:
1. no caching as a baseline (= recompile for each row)
2. unbounded cache (= threadsafe global hash-map)
3. LRU cache (= threadsafe global hash-map + LRU queue)
4. lightweight local cache 1 (= not threadsafe local hashmap with
   collision list which grows to a certain size (here: 10 elements) and
   afterwards never changes)
5. lightweight local cache 2 (not threadsafe local hashmap without
   collision list in which a collision replaces the stored element, idea
   by Alexey)

... using a haystack of 2 mio strings and
A). 2 mio distinct simple patterns
B). 10 simple patterns
C)  2 mio distinct complex patterns
D)  10 complex patterns

Fo A) and C), caching does not help but these queries still allow to
judge the static overhead of caching on query runtimes.

B) and D) are extreme but common cases in practice. They include
queries like "SELECT ... WHERE LIKE (col_haystack, flag ? '%pattern1%' :
'%pattern2%'). Caching should help significantly.

Because LIKE patterns are internally translated to re2 expressions, I
show only measurements for MATCH queries.

Results in sec, averaged over on multiple measurements;

1.A): 2.12
  B): 1.68
  C): 9.75
  D): 9.45

2.A): 2.17
  B): 1.73
  C): 9.78
  D): 9.47

3.A): 9.8
  B): 0.63
  C): 31.8
  D): 0.98

4.A): 2.14
  B): 0.29
  C): 9.82
  D): 0.41

5.A) 2.12 / 2.15 / 2.26
  B) 1.51 / 0.43 / 0.30
  C) 9.97 / 9.88 / 10.13
  D) 5.70 / 0.42 / 0.43
(10/100/1000 buckets, resp. 10/1/0.1% collision rate)

Evaluation:

1. This is the baseline. It was surprised that complex patterns (C, D)
   slow down the queries so badly compared to simple patterns (A, B).
   The runtime includes evaluation costs, but as caching only helps with
   compilation, and looking at 4.D and 5.D, compilation makes up over 90%
   of the runtime!

2. No speedup compared to 1, probably due to locking overhead. The cache
   is unbounded, and in experiments with data sets > 2 mio rows, 2. is
   the only scheme to throw OOM exceptions which is not acceptable.

3. Unique patterns (A and C) lead to thrashing of the LRU cache and very
   bad runtimes due to LRU queue maintenance and locking. Works pretty
   well however with few distinct patterns (B and D).

4. This scheme is tailored to queries B and D where it performs pretty
   good. More importantly, the caching is lightweight enough to not
   deteriorate performance on datasets A and C.

5. After some tuning of the hash map size, 100 buckets seem optimal to
   be in the same ballpark with 10 distinct patterns as 4. Performance
   also does not deteriorate on A and C compared to the baseline.
   Unlike 4., this scheme behaves LRU-like and can adjust to changing
   pattern distributions.

As a conclusion, this commit implementes two things:

1. Based on Q1, pattern search with const needle no longer uses
   caching. This applies to LIKE and MATCH + a few (exotic) other SQL
   functions. The code for the unbounded caching was removed.

2. Based on Q2, pattern search with non-const needles now use method 5.
 2022-05-30 20:00:35 +02:00
Han Fei
e15cdec39c address comments 2022-05-31 01:46:31 +08:00
Anton Popov
52d3791eb9
Merge pull request #37600 from CurtizJ/fix-with-fill-interval 
Fix `WITH FILL` with negative intervals in `STEP` clause
 2022-05-30 19:43:12 +02:00
alesapin
60b910a4de Fix 2022-05-30 19:04:25 +02:00
mergify[bot]
d4e722bbfa
Merge branch 'master' into http-named-collection 2022-05-30 16:40:18 +00:00
Vitaly Baranov
486a11a5e2 Fix flaky test test_row_policy. 2022-05-30 18:34:28 +02:00
vdimir
8a3f4bda62
Fix columns number mismatch in cross join 2022-05-30 15:40:15 +00:00
zvonand
a7840ef7f9 update tests(2) 2022-05-30 18:34:00 +03:00
Amos Bird
217f492264
Fix test 2022-05-30 21:38:24 +08:00
Nikolai Kochetov
5ef51ed27b Fix more tests. 2022-05-30 13:10:30 +00:00
Amos Bird
b68e8efaf0
Fix joinGet with cannot be null type 2022-05-30 21:01:27 +08:00
zvonand
d44bb61dec updated tests 2022-05-30 15:49:15 +03:00
mergify[bot]
87e896ae05
Merge branch 'master' into try_fix_tests 2022-05-30 12:44:35 +00:00
Kruglov Pavel
0615866aea
Merge pull request #37450 from Avogar/check-format-on-storage-creation 
Check format name on storage creation
 2022-05-30 14:23:20 +02:00
alesapin
362fa745e6 Ignore broken metadata 2022-05-30 13:32:12 +02:00
Nikolai Kochetov
5b4658aa5e Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-30 09:47:35 +00:00
Amos Bird
030fe6e272
Less flaky jbod rebalancer test 2022-05-30 16:53:55 +08:00
Maksim Kita
53d848f187 Fixed tests 2022-05-30 10:53:04 +02:00
Alexey Milovidov
9a5dd75a68 Merge branch 'master' into llvm-14 2022-05-30 05:34:38 +02:00
Alexey Milovidov
a67fb05ea0 Add test 2022-05-30 05:03:16 +02:00
alesapin
753bcee954 Fix lazy mark load 2022-05-29 18:38:09 +02:00
mergify[bot]
df9f7c59a2
Merge branch 'master' into try_fix_tests 2022-05-29 15:04:48 +00:00
alesapin
59a070778d More quite logging for S3 tests 2022-05-29 14:48:04 +02:00
alesapin
0e8ab36913 Merge branch 'master' into turn_on_s3_tests 2022-05-29 14:37:10 +02:00
Andrey Zvonov
2dbbf14de5
Merge branch 'master' into non-neg-deriv 2022-05-29 10:09:51 +03:00
Alexander Tokmakov
579b0e3323
Merge pull request #37627 from ClickHouse/revert-37416-fix_ReplicatedMergeTree_comments 
Revert "Implemented changing comment to a ReplicatedMergeTree table"
 2022-05-29 09:57:12 +03:00
Alexey Milovidov
9e3242f186
Merge pull request #37617 from CurtizJ/aggregation-sparse-columns 
Better performance with sparse columns in aggregate functions
 2022-05-29 09:36:07 +03:00
Alexander Tokmakov
562eec591e
Revert "Implemented changing comment to a ReplicatedMergeTree table" 2022-05-29 09:28:47 +03:00
Alexey Milovidov
975f16049e
Merge pull request #37597 from azat/fix-01317_no_password_in_command_line 
tests: fix 01317_no_password_in_command_line flakiness (and make it race free)
 2022-05-29 07:59:10 +03:00
zvonand
295a0f9ec2 added tests 2022-05-29 03:38:42 +03:00
Alexey Milovidov
c1169019d2 Merge branch 'master' into llvm-14 2022-05-29 02:29:02 +02:00
Alexey Milovidov
b2ba2533b1
Merge pull request #35231 from ClickHouse/time-zone-in-stateless-tests 
Change timezone in stateless tests
 2022-05-29 03:09:12 +03:00
Alexander Tokmakov
313f4833d6 fix 2022-05-28 23:21:55 +02:00
Alexander Tokmakov
9e3e795353 fix 2022-05-28 21:56:10 +02:00
mergify[bot]
c273568e45
Merge branch 'master' into fix-01317_no_password_in_command_line 2022-05-28 18:27:45 +00:00
Alexey Milovidov
8d0c46a105
Merge pull request #37614 from ClickHouse/test-reports-markup 
Remove margin in test reports and change the font
 2022-05-28 21:25:54 +03:00
Alexey Milovidov
e48af2efe4
Update report.py 2022-05-28 21:25:42 +03:00
Alexander Tokmakov
4e52f45695 Merge branch 'master' into fix_trash 2022-05-28 19:43:19 +02:00
Alexander Tokmakov
be498b0658 fix test 2022-05-28 19:40:26 +02:00
Alexander Tokmakov
f3c2ca6a13 Revert "Merge pull request #36634 from azat/01502_long_log_tinylog_deadlock_race-test" 
This reverts commit c04c62795e, reversing
changes made to 88f05ac14a.
 2022-05-28 19:28:38 +02:00
Alexander Tokmakov
70f39e0849 Revert "Merge pull request #36731 from azat/fix-async_inserts_stress_long-test" 
This reverts commit d1d2f2c1a4, reversing
changes made to 1ddb04b992.
 2022-05-28 19:27:40 +02:00
Alexander Tokmakov
9af6e237e1 remove retries 2022-05-28 19:25:56 +02:00
Alexander Tokmakov
1ae919a18a Revert "Merge pull request #35865 from azat/clickhouse-test-left-queries" 
This reverts commit 18d094d79d, reversing
changes made to 224f4dc620.
 2022-05-28 19:23:58 +02:00
Alexander Tokmakov
e33d986cf1 Revert "Merge pull request #34924 from azat/tests-overlaps" 
This reverts commit f13e436d40, reversing
changes made to aa3c05e9d4.
 2022-05-28 19:20:32 +02:00
Yakov Olkhovskiy
4f07c684da style fix 2022-05-28 12:45:53 -04:00
Yakov Olkhovskiy
3ff32fe81d test is added 2022-05-28 12:30:05 -04:00
Han Fei
d883194eb2 try to make test stable 2022-05-28 22:20:09 +08:00
Anton Popov
c39d95e2e6 add perf test 2022-05-28 12:56:38 +00:00
Anton Popov
1d9b3be7da
Merge pull request #37536 from CurtizJ/profile-events-for-part-types 
Add profile events for introspection of part types
 2022-05-28 14:25:21 +02:00
Han Fei
0f71231574 try fix flaky tests and refine code style 2022-05-28 17:25:33 +08:00
Vxider
b24346328d fix parser when using table identifer 2022-05-28 08:22:34 +00:00
Alexey Milovidov
15f2912dc9
Fix Python. 2022-05-28 05:10:29 +03:00
mergify[bot]
503eb3e9e6
Merge branch 'master' into time-zone-in-stateless-tests 2022-05-28 01:28:59 +00:00
Alexey Milovidov
f7a5b1fdaf Remove margin in test reports and change the font 2022-05-28 03:24:56 +02:00
Alexander Gololobov
6a57e1a970
Merge pull request #37601 from ClickHouse/array_norm_dist_fixes 
Added LpNorm and LpDistance functions for arrays
 2022-05-27 23:40:38 +02:00
Nikolai Kochetov
b80b1940ce Fix some tests. 2022-05-27 20:47:35 +00:00
Alexey Milovidov
8c94b75516 Merge branch 'master' into llvm-14 2022-05-27 22:24:20 +02:00
Alexey Milovidov
fa31d758d6 Update tests 2022-05-27 22:22:59 +02:00
Dmitry Novik
db2fe33926
Update test.py 2022-05-27 20:43:59 +02:00
Alexey Milovidov
bf1ad458da Merge branch 'master' into time-zone-in-stateless-tests 2022-05-27 19:38:13 +02:00
Han Fei
0d5448a621 fix flaky tests 2022-05-28 00:53:07 +08:00
mergify[bot]
923ad2e905
Merge branch 'master' into turn_on_s3_tests 2022-05-27 16:31:43 +00:00
Dmitry Novik
60b9d81773 Remove global_memory_usage_overcommit_max_wait_microseconds 2022-05-27 16:30:29 +00:00
alesapin
f63fa9bcc6
Merge pull request #37416 from Enmk/fix_ReplicatedMergeTree_comments 
Implemented changing comment to a ReplicatedMergeTree table
 2022-05-27 18:29:34 +02:00
Han Fei
dda86b49a6 update tests 2022-05-28 00:10:24 +08:00
mergify[bot]
cd602b20cd
Merge branch 'master' into tiny_fixes_for_jepsen 2022-05-27 16:06:08 +00:00
alesapin
0f5032c737
Merge pull request #37527 from ClickHouse/build-report 
Add failed builds to the build report
 2022-05-27 17:45:25 +02:00
Alexander Gololobov
540353566c Added LpNorm and LpDistance functions for arrays 2022-05-27 17:17:08 +02:00
Azat Khuzhin
8a224239c1 Prohibit optimize_aggregation_in_order with GROUPING SETS 
AggregatingStep ignores it anyway, and it leads to the following error
in getSortDescriptionFromGroupBy(), like in [1]:

    2022.05.24 04:29:29.279431 [ 3395 ] {26543564-8bc8-4a3a-b984-70a2adf0245d} <Fatal> : Logical error: 'Trying to get name of not a column: ExpressionList'.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36914/67d3ac72d26ab74d69f03c03422349d4faae9e19/stateless_tests__ubsan__actions_.html

v2: revert change to getSortDescriptionFromGroupBy() after
    GroupingSetsRewriterVisitor had been introduced
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 17:44:57 +03:00
Azat Khuzhin
2fb735ffc1 tests: add echo for 01883_grouping_sets_crash 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 17:44:50 +03:00
alesapin
be1c3c132b Fix some trash 2022-05-27 16:08:49 +02:00
Anton Popov
abc90fad8d fix WITH FILL with negative itervals 2022-05-27 12:42:51 +00:00
Han Fei
d7b372b73a fix style check 2022-05-27 20:40:55 +08:00
alesapin
32167cb6fb Turn on s3 tests 2022-05-27 14:36:12 +02:00
Azat Khuzhin
735d2dfebd tests: fix 01317_no_password_in_command_line flakiness (and make it race free) 
Before it was possible not to check if the query was gone already, also
it checks all processes not only the process of the client for the
possible password.

v2: make it parallel aware
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 15:27:48 +03:00
Anton Popov
d68c30a92e fix tests 2022-05-27 12:27:25 +00:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
Han Fei
447c5eb6ff add zookeeper tag 2022-05-27 19:48:14 +08:00
alesapin
461bb42fb8 Fix flaky test 2022-05-27 13:40:07 +02:00
Han Fei
c66fbacf39 update test 2022-05-27 19:23:48 +08:00
Azat Khuzhin
2613149f6b Fix converting types for UNION queries (may produce LOGICAL_ERROR) 
CI founds [1]:

    2022.02.20 15:14:23.969247 [ 492 ] {} <Fatal> BaseDaemon: (version 22.3.1.1, build id: 6082C357CFA6FF99) (from thread 472) (query_id: a5187ff9-962a-4e7c-86f6-8d48850a47d6) (query: SELECT 0., round(avgWeighted(x, y)) FROM (SELECT toDate(toDate('214748364.8', '-922337203.6854775808', '-0.1', NULL) - NULL, 10.000100135803223, '-2147483647'), 255 AS x, -2147483647 AS y UNION ALL SELECT y, NULL AS x, 2147483646 AS y)) Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/26d0e5438c86e52a145aaaf4cb523c399989a878/fuzzer_astfuzzerdebug,actions//report.html

The problem is that subqueries returns different headers:
- first query  -- x, y
- second query -- y, x

v2: Make order of columns strict only for UNION
    https://s3.amazonaws.com/clickhouse-test-reports/34775/9cc8c01a463d18c471853568b2f0af659a4e643f/stateless_tests__address__actions__[2/2].html
    Fixes: 00597_push_down_predicate_long
v3: add no-backward-compatibility-check for the test
Fixes: #37569
Resubmit: #34775
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit a813f5996e)
 2022-05-27 14:11:57 +03:00
Han Fei
2ea027ffcb Support insert into system.zookeeper 2022-05-27 18:53:12 +08:00
Kseniia Sumarokova
f5d69506b4
Merge pull request #37516 from KinderRiven/improve_local_cache 
Control cache downloads to avoid negative optimization of local caches
 2022-05-27 11:53:17 +02:00
Antonio Andelic
e1772c571c Compress clickhouse logs 2022-05-27 07:10:20 +00:00
Alexey Milovidov
d74333424b Fix test 2022-05-27 04:33:47 +02:00
Alexey Milovidov
393846e1ee Fix test 2022-05-27 04:08:04 +02:00
Dmitry Novik
3a9239b79f
Revert "RFC: Fix converting types for UNION queries (may produce LOGICAL_ERROR)" 2022-05-27 04:05:32 +02:00
Alexey Milovidov
faa519300e Fix more tests 2022-05-27 02:20:16 +02:00
Alexey Milovidov
d71780d34c Fix some tests 2022-05-27 02:18:38 +02:00
Alexey Milovidov
69a1a0c6d7
Merge branch 'master' into time-zone-in-stateless-tests 2022-05-27 03:04:42 +03:00
Alexey Milovidov
5b9bf7eb15 Fix tests 2022-05-27 01:59:45 +02:00
Alexey Milovidov
207ffc0f92 Merge branch 'master' into llvm-14 2022-05-27 01:18:51 +02:00
Alexey Milovidov
86afa3a245
Merge pull request #37502 from ClickHouse/array_norm_dist_fixes 
Renamed arrayXXNorm/arrayXXDistance functions to XXNorm/XXDistance and fixed some overflow cases
 2022-05-27 00:56:29 +03:00
Alexander Tokmakov
eb71dd4c78
Merge pull request #37547 from ClickHouse/followup_37398 
Follow-up to #37398
 2022-05-26 20:29:41 +03:00
Azat Khuzhin
4288d09a85 Do not write expired columns by TTL after merge w/o TTL 
Usually second merge do not perform TTL, since everything is up to date,
however in this case TTLTransform is not used, and hence expired_columns
will not be filled for new part, and so those columns will be written
with default values.

Avoid this, by manually filling expired_columns.

Here is a simpler reproducer:

Simple reproducer:

    ```sql
    create table ttl_02262 (date Date, key Int, value String TTL date + interval 1 month) engine=MergeTree order by key settings min_bytes_for_wide_part=0, min_rows_for_wide_part=0;
    insert into ttl_02262 values ('2010-01-01', 2010, 'foo');
    ```

    ```sh
    # ls -l .server/data/default/ttl_02262/all_*
    .server/data/default/ttl_02262/all_1_1_0:
    total 48
    -rw-r----- 1 root root 335 May 26 14:19 checksums.txt
    -rw-r----- 1 root root  76 May 26 14:19 columns.txt
    -rw-r----- 1 root root   1 May 26 14:19 count.txt
    -rw-r----- 1 root root  28 May 26 14:19 date.bin
    -rw-r----- 1 root root  48 May 26 14:19 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:19 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:19 key.bin
    -rw-r----- 1 root root  48 May 26 14:19 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:19 primary.idx
    -rw-r----- 1 root root  99 May 26 14:19 ttl.txt
    -rw-r----- 1 root root  30 May 26 14:19 value.bin
    -rw-r----- 1 root root  48 May 26 14:19 value.mrk2
    ```

    ```sql
    optimize table ttl_02262 final;
    ```

    ```sh
    .server/data/default/ttl_02262/all_1_1_1:
    total 40
    -rw-r----- 1 root root 279 May 26 14:19 checksums.txt
    -rw-r----- 1 root root  61 May 26 14:19 columns.txt
    -rw-r----- 1 root root   1 May 26 14:19 count.txt
    -rw-r----- 1 root root  28 May 26 14:19 date.bin
    -rw-r----- 1 root root  48 May 26 14:19 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:19 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:19 key.bin
    -rw-r----- 1 root root  48 May 26 14:19 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:19 primary.idx
    -rw-r----- 1 root root  81 May 26 14:19 ttl.txt
    ```

    ```sql
    optimize table ttl_02262 final;
    ```

    ```sh
    .server/data/default/ttl_02262/all_1_1_2:
    total 48
    -rw-r----- 1 root root 349 May 26 14:20 checksums.txt
    -rw-r----- 1 root root  76 May 26 14:20 columns.txt
    -rw-r----- 1 root root   1 May 26 14:20 count.txt
    -rw-r----- 1 root root  28 May 26 14:20 date.bin
    -rw-r----- 1 root root  48 May 26 14:20 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:20 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:20 key.bin
    -rw-r----- 1 root root  48 May 26 14:20 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:20 primary.idx
    -rw-r----- 1 root root  81 May 26 14:20 ttl.txt
    -rw-r----- 1 root root  27 May 26 14:20 value.bin
    -rw-r----- 1 root root  48 May 26 14:20 value.mrk2
    ```

And now we have `value.*` for all_1_1_2, this should not happen.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
mergify[bot]
8f7b9623c6
Merge branch 'master' into build-report 2022-05-26 16:59:22 +00:00
mergify[bot]
5e6d22055d
Merge branch 'master' into tiny_fixes_for_jepsen 2022-05-26 16:50:18 +00:00
Dmitry Novik
673a521d0b
Merge pull request #34775 from azat/union-type-cast 
RFC: Fix converting types for UNION queries (may produce LOGICAL_ERROR)
 2022-05-26 17:28:23 +02:00
alesapin
85c7b55b81 Slightly better jepsen tests 2022-05-26 16:54:46 +02:00
alesapin
8ae277a1c4 Fix test 2022-05-26 15:25:48 +02:00
KinderRiven
55379f0d03 fix config 2022-05-26 20:59:31 +08:00
Alexander Tokmakov
e8f33fb0d9 fix flaky tests 2022-05-26 14:17:05 +02:00
Maksim Kita
3a92e61827
Merge pull request #37148 from kitaisreal/dictionary-get-descendants-performance-improvement 
Dictionary getDescendants performance improvement
 2022-05-26 12:29:17 +02:00
KinderRiven
286c13317f fix stateless test 2022-05-26 17:38:42 +08:00
KinderRiven
ab4614dee5 fix 2022-05-26 16:39:50 +08:00
KinderRiven
f5a186c774 add test for local cache download 2022-05-26 16:36:26 +08:00
Dmitry Novik
5c3c994d2a
Merge pull request #37493 from ClickHouse/grouping-sets-optimization-fix 
Fix ORDER BY optimization in case of GROUPING SETS
 2022-05-26 02:25:02 +02:00
Anton Popov
f488efd27e fix tests 2022-05-26 00:03:31 +00:00
Dmitry Novik
3c1b6609ae Add comments and make tests more verbose 2022-05-25 21:23:35 +00:00
mergify[bot]
49cce189e3
Merge branch 'master' into floating_seconds 2022-05-25 21:08:55 +00:00
Mikhail f. Shiryaev
e1ed2aa3dc
Add failed builds to the build report 2022-05-25 23:00:33 +02:00
alesapin
1db9cf480b Merge remote-tracking branch 'origin/master' into i_object_storage 2022-05-25 22:50:22 +02:00
alesapin
4410d3d15f Better test 2022-05-25 22:49:54 +02:00
Maksim Kita
58cd1bd3ec
Merge pull request #36843 from bharatnc/ncb/h3-unidirectionaledges-funcs 
add h3 unidirectional edge functions
 2022-05-25 22:46:40 +02:00
Maksim Kita
bee3c30f66
Merge pull request #37524 from kitaisreal/geo-distance-functions-improve-performance 
Geo distance functions improve performance
 2022-05-25 22:40:40 +02:00
Maksim Kita
b12b363158 Fixed build of hierarchical index for HashedArrayDictionary 2022-05-25 22:40:19 +02:00
Alexander Gololobov
168b47d0ad Use same norm and distance function names for tuples and arrays 2022-05-25 22:39:59 +02:00
Alexander Gololobov
b065839f44 always return Float64 2022-05-25 22:27:00 +02:00
alesapin
6f5c86e55e Merge branch 'master' into i_object_storage 2022-05-25 20:49:01 +02:00
Alexander Tokmakov
779e6ea0b9 make it better, fix on cluster queries 2022-05-25 20:17:49 +02:00
Azat Khuzhin
a813f5996e Fix converting types for UNION queries (may produce LOGICAL_ERROR) 
CI founds [1]:

    2022.02.20 15:14:23.969247 [ 492 ] {} <Fatal> BaseDaemon: (version 22.3.1.1, build id: 6082C357CFA6FF99) (from thread 472) (query_id: a5187ff9-962a-4e7c-86f6-8d48850a47d6) (query: SELECT 0., round(avgWeighted(x, y)) FROM (SELECT toDate(toDate('214748364.8', '-922337203.6854775808', '-0.1', NULL) - NULL, 10.000100135803223, '-2147483647'), 255 AS x, -2147483647 AS y UNION ALL SELECT y, NULL AS x, 2147483646 AS y)) Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/26d0e5438c86e52a145aaaf4cb523c399989a878/fuzzer_astfuzzerdebug,actions//report.html

The problem is that subqueries returns different headers:
- first query  -- x, y
- second query -- y, x

v2: Make order of columns strict only for UNION
    https://s3.amazonaws.com/clickhouse-test-reports/34775/9cc8c01a463d18c471853568b2f0af659a4e643f/stateless_tests__address__actions__[2/2].html
    Fixes: 00597_push_down_predicate_long
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-25 20:31:47 +03:00
Nikolai Kochetov
ff98c24d44
Merge pull request #37048 from Avogar/fix-array-map-nothing 
Add default implementation for Nothing in functions
 2022-05-25 19:10:40 +02:00
alesapin
0a3597da72
Merge pull request #34915 from ianton-ru/MDB-16962 
Fix collision of S3 operation log revision
 2022-05-25 18:15:31 +02:00
avogar
4c9812d4c1 Allow to skip some of the first rows in CSV/TSV formats 2022-05-25 15:00:11 +00:00
Anton Popov
16e839ac71 add profile events for introspection of part types 2022-05-25 14:54:49 +00:00
mergify[bot]
f49552d48e
Merge branch 'master' into grouping-sets-optimization-fix 2022-05-25 13:03:54 +00:00
Maksim Kita
45da28ecae Improve performance of geo distance functions 2022-05-25 14:22:22 +02:00
Maksim Kita
9a9df26eec Fixed tests 2022-05-25 11:44:37 +02:00
Maksim Kita
6c033f340b Fixed tests 2022-05-25 11:44:37 +02:00
Kseniia Sumarokova
b50d4549c9
Merge pull request #37356 from amosbird/partition-prune-for-s3 
"Partition pruning" for s3
 2022-05-25 11:03:07 +02:00
avogar
9518c41dda Try to fix tests 2022-05-25 09:01:12 +00:00
avogar
f782fa31c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into check-format-on-storage-creation 2022-05-25 08:42:54 +00:00
Bharat Nallan Chakravarthy
c586c91c09 add more tests 2022-05-25 07:16:45 +05:30
Bharat Nallan Chakravarthy
57cfc0bd04 check for validity of h3 index 2022-05-25 06:17:15 +05:30
Alexey Milovidov
a5541dc2d5 Merge branch 'master' into llvm-14 2022-05-25 01:56:24 +02:00
Ilya Yatsishin
cda88ebf9c Try to run aarch64 performance tests 2022-05-24 21:47:49 +02:00
Igor Nikonov
04e2737a57
Merge pull request #37337 from ClickHouse/column_decl_null_before_default_value 
Column declaration: [NOT] NULL right after type 

+ fixed: data_type_default_nullable=true, it didn't make columns nullable
         if the column declaration contains default expression w/o type

Issue #37229
 2022-05-24 21:16:25 +02:00
Alexander Gololobov
2ff747785e
Merge pull request #37394 from ClickHouse/array_norm_dist_fixes 
Do computations on the raw input data without copying to Eigen::Matrix
 2022-05-24 20:59:04 +02:00
Dmitry Novik
c4dc0f7cda Fix ORDER BY optimization in case of GROUPING SETS 2022-05-24 18:56:22 +00:00
Robert Schulze
7348a0eb28
Merge pull request #37251 from ClickHouse/non_const_like 
Support non-constant SQL functions (NOT) (I)LIKE and MATCH
 2022-05-24 20:28:31 +02:00
Alexander Tokmakov
13bdabce3a
Update 02302_join_auto_lc_nullable_bug.sql 2022-05-24 21:21:12 +03:00
Mikhail f. Shiryaev
749eac120e
Merge pull request #37477 from ClickHouse/sync-workflows-paths 
Sync workflows paths for PR and DocsCheck
 2022-05-24 19:35:45 +02:00
Alexander Tokmakov
4618429201 fixes 2022-05-24 18:53:52 +02:00
Kruglov Pavel
0c3bcfa122
Merge pull request #36884 from Algunenano/http_proper_summary_and_exception 
HTTP: Always return summary data and exception (when possible)
 2022-05-24 18:33:24 +02:00
Maksim Kita
3c0c322d7c
Merge pull request #37480 from kitaisreal/dynamic-dispatch-infrastructure-improvements 
Dynamic dispatch infrastructure style fixes
 2022-05-24 18:13:53 +02:00
Alexander Tokmakov
bb5f04efaf Merge branch 'master' into fix_trash 2022-05-24 17:45:51 +02:00
Raúl Marín
cd4020fa2f Reduce flakiness 2022-05-24 17:13:53 +02:00
Raúl Marín
a86aa43baa Merge remote-tracking branch 'blessed/master' into floating_seconds_2 2022-05-24 17:10:45 +02:00
Vladimir C
0eaf22b370
Merge pull request #37453 from vdimir/join_auto_lc_nullable_bug 2022-05-24 16:28:23 +02:00
Alexander Tokmakov
229d35408b
Merge pull request #37398 from ClickHouse/fixes_for_transactions 
Fixes for transactions
 2022-05-24 15:28:01 +03:00
Maksim Kita
e6e4b2826d Dynamic dispatch infrastructure style fixes 2022-05-24 14:25:29 +02:00
Alexander Tokmakov
dbde63d275 fixes 2022-05-24 14:24:03 +02:00
Kruglov Pavel
6c9a524f6b
Merge pull request #37192 from Avogar/formats-with-names 
Improve performance and memory usage for select of subset of columns for some formats
 2022-05-24 13:28:14 +02:00
Kruglov Pavel
9c87424cd7
Merge pull request #37458 from Avogar/fix-parquet-nested 
Fix named tuples output in ORC/Arrow/Parquet formats
 2022-05-24 13:24:05 +02:00
alesapin
ea19f1f18c
Merge pull request #37440 from ClickHouse/fix-keeper-force-recovery-one-node 
Fix `clickhouse-keeper` force recovery for single node cluster
 2022-05-24 13:03:56 +02:00
Amos Bird
1ee02a4ac5
No parallel testing 2022-05-24 18:57:56 +08:00
Amos Bird
c25ef92139
Fix tests 2022-05-24 18:57:55 +08:00
Amos Bird
093d315756
partition pruning for s3 2022-05-24 18:57:55 +08:00
Mikhail f. Shiryaev
b4c0f7a621
Simplify labels check 2022-05-24 12:57:06 +02:00
Mikhail f. Shiryaev
90425dedd1
Fail release on status == failure 2022-05-24 11:14:17 +02:00
Mikhail f. Shiryaev
3af9a699d5
Fix ssh-agent socket in docker image 2022-05-24 11:14:16 +02:00
Mikhail f. Shiryaev
249af9c69a
Run container as user, print stdout 2022-05-24 11:14:16 +02:00
Mikhail f. Shiryaev
21d6fc54d5
Do not check docs_release.py for changed files 2022-05-24 11:14:16 +02:00
Mikhail f. Shiryaev
501a8158a5
Don't fail docker images on dispatch or other events 2022-05-24 11:14:14 +02:00
Maksim Kita
712b000f2a
Merge pull request #37443 from kitaisreal/functions-normalize-utf8-fix 
Functions normalize utf8 fix
 2022-05-24 11:11:15 +02:00
李扬
e33cfc889c
Fix bug of datetime64 parsed from string '1969-12-31 23:59:59.123' (#37039) 2022-05-24 10:47:17 +02:00
Alexander Gololobov
65fbda436a Do computations on the raw input data without copying to Eigen::Matrix 2022-05-24 10:24:50 +02:00
Alexander Gololobov
6094000d7c
Merge pull request #37437 from ClickHouse/norm_distance_perf 
Perf test for Norm and Distance functions for arrays and tuples
 2022-05-24 10:17:35 +02:00
Igor Nikonov
838a6c6f61 Column declaration: [NOT] NULL right after type 
+ fixed: data_type_default_nullable=true, it didn't make columns nullable
         if the column declaration contains default expression w/o type

Issue #37229
 2022-05-23 22:03:45 +00:00
Maksim Kita
293295815c Fixed tests 2022-05-23 23:44:05 +02:00
mergify[bot]
51ff49a0ee
Merge branch 'master' into i_object_storage 2022-05-23 20:29:49 +00:00
Azat Khuzhin
1b3df39311 tests: fix table in 01710_projection_aggregation_in_order 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Follow-up for: #37342
 2022-05-23 23:16:37 +03:00
Alexander Tokmakov
d0f998fb88 Merge branch 'master' into fix_trash 2022-05-23 21:25:56 +02:00
avogar
3651ef93fe Fix performance test 2022-05-23 17:42:13 +00:00
Alexander Tokmakov
6bc68c0cbc Merge branch 'master' into fixes_for_transactions 2022-05-23 18:49:21 +02:00
Alexander Gololobov
d0f5551c9f Parameterized with norm kind 2022-05-23 18:27:41 +02:00
Alexander Gololobov
2658a9eeeb Test with max_threads=1 2022-05-23 18:06:07 +02:00
Antonio Andelic
e91e7fdba7 Fix style 2022-05-23 15:33:29 +00:00
avogar
c4f3d8bce1 Fix test 2022-05-23 15:24:58 +00:00
Vladimir C
14a820c2a9
Merge pull request #37336 from vdimir/fix_clash_const_aggegate_join 2022-05-23 17:09:30 +02:00
Kruglov Pavel
b73d49158d
Fix test 2022-05-23 17:01:45 +02:00
Maksim Kita
9de54040ad
Merge pull request #37449 from kitaisreal/range-hashed-dictionary-added-range-cast-test 
RangeHashedDictionary added test
 2022-05-23 16:49:10 +02:00
avogar
d40b0461c5 Update test 2022-05-23 14:25:43 +00:00
avogar
ce4adb447f Fix named tuples output in ORC/Arrow/Parquet formats 2022-05-23 14:21:08 +00:00
vdimir
dabb150a95
Fix cast lowcard of nullable in JoinSwitcher 2022-05-23 13:05:59 +00:00
avogar
37b66c8a9e Check format name on storage creation 2022-05-23 12:48:48 +00:00
Maksim Kita
94772f9cfc Added performance tests 2022-05-23 14:43:13 +02:00
Maksim Kita
41cb5b4afc RangeHashedDictionary added test 2022-05-23 14:35:26 +02:00
Alexander Gololobov
70cc27ecac Test with different element types 2022-05-23 14:08:15 +02:00
Mikhail f. Shiryaev
bc145294a6
Merge pull request #37441 from ClickHouse/website-content 
Release website content
 2022-05-23 12:51:14 +02:00
Maksim Kita
fa76f9af34 Update tests 2022-05-23 12:42:00 +02:00
Maksim Kita
d35654226a Fixed tests 2022-05-23 12:42:00 +02:00
Maksim Kita
585b86446e Added hierarchical_index_bytes_allocated column in system.dictionaries 2022-05-23 12:42:00 +02:00
Antonio Andelic
87d445295f
Merge pull request #37342 from azat/projections-optimize_aggregation_in_order-fix 
Fix projections with GROUP/ORDER BY in query and optimize_aggregation_in_order
 2022-05-23 12:29:16 +02:00
Maksim Kita
e44fec0127 Added tests 2022-05-23 12:21:59 +02:00
Kruglov Pavel
f539fb835d
Merge branch 'master' into formats-with-names 2022-05-23 12:14:20 +02:00
Kruglov Pavel
ce48e8e102
Merge pull request #36975 from Avogar/json-columns-formats 
Add columnar JSON formats
 2022-05-23 12:11:28 +02:00
Kruglov Pavel
754e675ec3
Merge pull request #37253 from Avogar/fix-defaults 
Fix inserting defaults for missing values in columnar formats
 2022-05-23 12:10:14 +02:00
Kruglov Pavel
9bc74439c1
Merge pull request #37327 from Avogar/arrow-strings 
Allow to use String type instead of Binary in Arrow/Parquet/ORC formats
 2022-05-23 12:05:33 +02:00
Antonio Andelic
c268296fc6 Fix single node force recovery and add tests 2022-05-23 09:53:46 +00:00
Mikhail f. Shiryaev
cfe98c4aba
Fix publishing of content.clickhouse.com 
- Publish only benchmarks and data
- Minimize clickhouse/docs-release
- Run it as a normal user
- Speed up running by not redownload virtual env deps
- Clean out docs and blog buildging
- Minimize docs/tools/requirements.txt
 2022-05-23 11:50:02 +02:00
Vasily Nemkov
e481a707aa Fixed test 2022-05-23 11:51:57 +03:00
Alexander Gololobov
7897a5bac7 Perf test for Norm and Distance fuctions for arrays and tuples 2022-05-23 10:18:24 +02:00
Robert Schulze
e25ca139cd
Implement SQL functions (NOT) (I)LIKE() + MATCH() with non-const needles 
With this commit, SQL functions LIKE and MATCH and their variants can
work with non-const needle arguments. E.g.

  create table tab
    (id UInt32, haystack String, needle String)
    engine = MergeTree()
    order by id;

  insert into tab values
  (1, 'Hello', '%ell%')
  (2, 'World', '%orl%')

  select id, haystack, needle, like(haystack, needle)
  from tab;

For that, methods vectorVector() and vectorFixedVector() were added to
MatchImpl. The existing code for const needles has an optimization where
the compiled regexp is cached. The new code expects a different needle
per row and consequently does not cache the regexp.
 2022-05-23 09:41:28 +02:00
Alexey Milovidov
698e5e5352
Merge pull request #37415 from Joeywzr/gen_uuid 
Generate multiple columns with UUID
 2022-05-23 00:29:42 +03:00
Alexey Milovidov
6fe9231ee8 Update some tests 2022-05-22 22:58:31 +02:00
Alexey Milovidov
a021cff703 Fix test 2022-05-22 22:48:29 +02:00
Alexey Milovidov
b832d3641f Fix test 2022-05-22 22:46:14 +02:00
Alexey Milovidov
dce7a86ca9 Merge branch 'master' into llvm-14 2022-05-22 22:38:07 +02:00
mergify[bot]
747aa5575c
Merge branch 'master' into remove-useless-code-2 2022-05-22 17:41:57 +00:00
Alexey Milovidov
d32d45e12e
Merge pull request #37400 from ndchikin/window_frame_fix 
WindowTransform::moveRowNumber fix
 2022-05-22 20:35:00 +03:00
Alexey Milovidov
3b046472fe
Update 02306_window_move_row_number_fix.sql 2022-05-22 20:33:07 +03:00
alesapin
c8d92b87c8 Merge branch 'master' into i_object_storage 2022-05-22 12:16:10 +02:00
Memo
aa503f699a fix tests 2022-05-22 13:43:05 +08:00
Memo
3a17180a4f fix tests 2022-05-22 13:42:36 +08:00