thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-01 12:01:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
156,662 Commits 9,191 Branches 1,782 Tags 2.2 GiB
16ef3d7511
Commit Graph

1360 Commits

Author SHA1 Message Date
Kruglov Pavel
d694723d5f
Merge branch 'master' into json-data-type 2024-07-24 21:26:20 +02:00
avogar
2381c3dbca Fix and rewrite tests 2024-07-24 19:11:51 +00:00
alesapin
68c53764f2
Update tests/performance/decimal_aggregates.xml 2024-07-24 17:50:42 +02:00
alesapin
a03a59a3bf Use less data overall 2024-07-24 15:14:02 +02:00
alesapin
80304b5962 Less memory-intensive fill queries for decimal aggregates 2024-07-24 15:13:10 +02:00
avogar
f37f228af9 Improve columns squashing for String/Array/Map/Variant/Dynamic types 2024-07-24 13:10:20 +00:00
Michael Kolupaev
e09d984fec Merge remote-tracking branch 'origin/master' into mv4 2024-07-23 17:52:40 +00:00
Alexey Milovidov
abb6025e72 Fix performance test about the generateRandom table function, supposedly 2024-07-23 02:30:45 +02:00
Robert Schulze
67f5ffc592
Fixups 2024-07-16 20:21:21 +00:00
János Benjamin Antal
e5804ae5cd
Use numbers_mt to reduce duration of tests 2024-07-10 17:28:10 +02:00
János Benjamin Antal
211994333f
Fix invalid XML 2024-07-10 14:56:28 +02:00
Robert Schulze
948565f554
Speed up replaceRegexp(All|One) if the pattern is trivial 2024-07-07 11:09:01 +00:00
Michael Kolupaev
4ac37682e3 Merge remote-tracking branch 'origin/master' into mv4 2024-06-27 20:45:08 +00:00
Antonio Andelic
f49c194ad0 Fix server restarts in performance tests 2024-06-26 13:48:35 +02:00
Michael Kolupaev
78a3d0d4f7 Fix parallel_mv perf test 2024-06-25 08:44:36 +00:00
Nikita Taranov
cb0a692ba0 fix read_from_comp_parts 2024-06-21 23:42:56 +01:00
Nikita Taranov
91dc9a69d8 fix final_big_column 2024-06-21 22:03:38 +01:00
Nikita Taranov
6f1f416700 one more test 2024-06-20 17:37:05 +01:00
Nikita Taranov
b125f8166f impl 2024-06-19 16:41:50 +01:00
Nikita Taranov
57d036e589 impl 2024-06-15 22:04:24 +01:00
kevinyhzou
2a14168075 improve least/greatest by allow nullable numberic fields compare 2024-06-05 15:47:42 +08:00
Alexey Milovidov
2f1fb6c5d8
Merge pull request #64426 from jiebinn/SparseColumnIteratorIncrease 
Reduce the redundant `isDefault()` of `ColumnSparse::filter` to improve performance
 2024-05-29 02:23:27 +02:00
Jiebin Sun
bc4d96481c Add isolated performance test 2024-05-29 01:12:28 +08:00
Robert Schulze
9d63095db9
Revert "Revert "Speed up splitByRegexp"" 
This reverts commit 08e5c2ba4d.
 2024-05-23 10:01:10 +00:00
Alexey Milovidov
500475f2b8 Add a test 2024-05-19 10:45:05 +02:00
Tomer Shafir
2a87f01890
fix io_uring.xml 2024-05-15 19:15:53 +03:00
tomershafir
7de812000c io_uring: add basic io_uring clickhouse perf test 2024-05-14 18:50:51 +03:00
Michael Kolupaev
0fd52c36c6
Merge pull request #61632 from bigo-sg/opt_string_ascii 
Process string functions XXXUTF8 'asciily' if input strings are all ascii chars
 2024-05-13 22:13:01 +00:00
Alexey Milovidov
989a880230
Merge pull request #62404 from Avogar/trivial-insert-select-from-files 
Improve trivial insert select from files, add separate max_parsing_threads setting
 2024-04-30 01:57:56 +02:00
Maksim Kita
e42d00e4b8 Added performance tests 2024-04-28 12:52:04 +03:00
Alexander Tokmakov
7fb78aa4d4
Merge pull request #60856 from azat/alter-select-throughtput-v2 
[RFC] Reduce overhead of the mutations for SELECTs (v2)
 2024-04-26 17:46:57 +00:00
Azat Khuzhin
be6777bc86 Reduce overhead of the mutations for SELECTs (v2) 
SELECTs are affected by the mutations, since it tries to apply them on
fly, and scanning over existing mutations can take significant amount of
time (for simple queries, i.e. count())

And also even after mutation had been finished, it still a problem,
because mutations are not removed instantly.

So instead introduce an atomic counter alter_conversions_mutations, that
is incremented for new mutations and decremented once mutation
finished/killed, that way once the mutation finished they will not
affect queries.

Here are some numbers for non-RENAME mutations:

    rmt vanilla w/o mutations | queries: 3693, QPS: 494.813
    rmt vanilla w/ mutations  | queries: 2190, QPS: 388.256
    rmt patched w/o mutations | queries: 3168, QPS: 620.061
    rmt patched w/ mutations  | queries: 3155, QPS: 614.424
    mt vanilla w/o mutations  | queries: 3498, QPS: 656.399
    mt vanilla w/ mutations   | queries: 3821, QPS: 600.425
    mt patched w/o mutations  | queries: 5732, QPS: 745.585
    mt patched w/ mutations   | queries: 4719, QPS: 715.034

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-04-25 14:35:21 +02:00
Robert Schulze
8d7221fe35
Add test for bug 33446 2024-04-23 13:20:27 +00:00
Robert Schulze
7c0c3168c1
Revert "Add test for bug #33446" 2024-04-22 16:13:13 +02:00
Robert Schulze
b2d76b1dea
Merge pull request #62511 from rschu1ze/one_thousand_joins 
Add test for bug #33446
 2024-04-22 09:41:47 +00:00
JackyWoo
f1ffcf01eb Merge branch 'master' into support_minmax_index_for_pointInPolygon 2024-04-22 10:12:05 +08:00
János Benjamin Antal
6189cd49e1
Merge pull request #62210 from bigo-sg/json_format_early_skip 
Improve `JSONEachRowRowInputFormat` by skipping all remaining fields when all required fields are read
 2024-04-17 13:32:58 +00:00
Robert Schulze
5cad63b9d5
Add test for bug 33446 2024-04-17 13:31:35 +00:00
lgbo-ustc
d4773ef1bb Merge remote-tracking branch 'origin/master' into json_format_early_skip 2024-04-17 08:56:54 +08:00
Dmitry Novik
7c63d99625
Merge pull request #62564 from ClickHouse/analyzer-perf-tests 
Always use new analyzer in performance tests
 2024-04-16 20:45:16 +00:00
Robert Schulze
3272d75ec8
Merge pull request #62583 from KevinyhZou/improve_utc_timestamp_transform 
Improve performance of `(to|from)_utc_timestamp` functions
 2024-04-16 16:45:45 +00:00
Robert Schulze
b6af948eb4
Reduce performance test size 2024-04-16 10:09:17 +00:00
JackyWoo
eb07f8c7e9 Merge branch 'master' into support_minmax_index_for_pointInPolygon 2024-04-16 17:05:36 +08:00
Robert Schulze
3257780ce3
Merge pull request #62392 from bigo-sg/opt_memchr 
Speed up `splitByRegexp`
 2024-04-16 08:07:49 +00:00
JackyWoo
46eb4b71a4 Merge branch 'master' into support_minmax_index_for_pointInPolygon 2024-04-16 15:08:39 +08:00
liuneng
92116dc8e5 change test max_threads 2024-04-16 11:08:47 +08:00
Robert Schulze
fad21c9958
Add test back 2024-04-15 10:29:05 +00:00
taiyang-li
47bcc854fc Merge remote-tracking branch 'origin/master' into opt_string_ascii 2024-04-15 15:16:54 +08:00
Kruglov Pavel
ce7432424e
Merge branch 'master' into trivial-insert-select-from-files 2024-04-12 14:26:48 +02:00
kevinyhzou
a9711c4bec modify test 2024-04-12 16:57:34 +08:00
kevinyhzou
878e0ed26b add performance test 2024-04-12 16:37:35 +08:00
lgbo-ustc
feddc40c45 update test 2024-04-12 10:00:52 +08:00
lgbo-ustc
e9635189d2 Merge remote-tracking branch 'origin/master' into json_format_early_skip 2024-04-12 08:53:38 +08:00
Dmitry Novik
5d7882bb60
Always use new analyzer in perf tests 2024-04-11 16:50:01 +02:00
Maksim Kita
1218cf1568 Updated implementation 2024-04-11 11:09:08 +03:00
Maksim Kita
0dcd70b5a5 Added performance tests 2024-04-11 11:09:08 +03:00
lgbo-ustc
d2e840f60a update 2024-04-11 15:59:04 +08:00
lgbo-ustc
a87fb7dc84 Merge remote-tracking branch 'origin/master' into json_format_early_skip 2024-04-11 14:37:12 +08:00
taiyang-li
4f4fa79ce7 last commit 2024-04-10 16:19:34 +08:00
Maksim Kita
35e1e5aff7 Perf script update path in documentation 2024-04-09 14:42:29 +03:00
Kruglov Pavel
19f00e7c42
Restart CI 2024-04-09 12:33:18 +02:00
avogar
ed6e4fbe16 Improve trivial insert select from files, add separate max_parsing_threads setting 2024-04-08 13:56:15 +00:00
taiyang-li
07815d93de add perf tests 2024-04-08 18:31:11 +08:00
lgbo-ustc
28de6d8b25 Merge remote-tracking branch 'origin/master' into json_format_early_skip 2024-04-08 10:15:33 +08:00
JackyWoo
7ddbd1ac16 Merge branch 'master' into support_minmax_index_for_pointInPolygon 2024-04-07 10:11:24 +08:00
JackyWoo
b58e1bbbfc Support NaN number 2024-04-03 15:06:27 +08:00
lgbo-ustc
cbedf8c5d2 improve by skip to line end when all fields are read 2024-04-03 10:30:32 +08:00
Robert Schulze
d729b63293
Merge pull request #61144 from bigo-sg/inline_poaarray_details 
Inline small but frequently invoked functions in PODArrayDetails
 2024-04-02 13:25:25 +02:00
Raúl Marín
b2e764f633 Revert "Merge pull request #61564 from liuneng1994/optimize_in_single_value" 
This reverts commit 20a45b4073, reversing
changes made to a642f4d3ec.
 2024-04-01 13:51:56 +02:00
JackyWoo
2f6e70d395 Add performance test 2024-03-29 15:20:13 +08:00
János Benjamin Antal
e288528ede
Merge pull request #61931 from ClickHouse/fix-aggregating-merge-tree-simple-aggregate-function-string 
Fix performance test `aggregating_merge_tree_simple_aggregate_function_string`
 2024-03-28 08:39:45 +01:00
Dmitry Novik
65121b712e Merge remote-tracking branch 'origin/master' into enable-analyzer 2024-03-26 19:02:30 +01:00
János Benjamin Antal
1ec7258d1e Enable allow_suspicious_primary_key for perf tests that need it 2024-03-26 11:09:03 +00:00
Dmitry Novik
8bd49aa303 Try to fix perf tests 2024-03-25 17:06:42 +01:00
Yarik Briukhovetskyi
20a45b4073
Merge pull request #61564 from liuneng1994/optimize_in_single_value 
New analyzer pass to optimize in single value
 2024-03-25 12:53:01 +01:00
李扬
be68b21de4
Merge branch 'ClickHouse:master' into opt_memcpy_small 2024-03-21 11:27:10 +08:00
taiyang-li
c350991005 add more tests 2024-03-21 10:29:35 +08:00
李扬
7c44505b49
Merge branch 'ClickHouse:master' into inline_poaarray_details 2024-03-21 10:20:45 +08:00
Raúl Marín
6c7be89c61
Merge pull request #61100 from Algunenano/experiment_int128 
30x faster printing for 256-bit integers
 2024-03-20 12:32:26 +01:00
taiyang-li
49422debaf add perf tests 2024-03-20 18:59:22 +08:00
liuneng
a29d4f5db2 fix bug 2024-03-20 10:24:57 +08:00
liuneng
532d72a7d8 add test case 2024-03-19 17:30:16 +08:00
Raúl Marín
4626a0b3a1 Merge remote-tracking branch 'blessed/master' into experiment_int128 2024-03-18 16:10:32 +01:00
Robert Schulze
f02db275c5
Merge remote-tracking branch 'rschu1ze/master' into dotProduct-memcpy 2024-03-18 09:26:29 +00:00
李扬
4da1dbd2c2
Merge branch 'ClickHouse:master' into inline_poaarray_details 2024-03-18 10:57:48 +08:00
李扬
9296be3747
Merge branch 'ClickHouse:master' into opt_memcpy_small 2024-03-18 10:35:09 +08:00
Sema Checherinda
025ad9791c
Merge pull request #58879 from azat/small-block-lock-contention 
Improve query performance in case of very small blocks
 2024-03-13 17:08:37 +01:00
Raúl Marín
cc92b8ffc5 Merge remote-tracking branch 'blessed/master' into experiment_int128 2024-03-13 13:57:41 +01:00
taiyang-li
dc9548d31b add perf test 2024-03-13 18:57:33 +08:00
taiyang-li
d17212616c add perf tests 2024-03-13 17:26:45 +08:00
Raúl Marín
45efa69189 Add perf tests 2024-03-08 15:17:59 +01:00
Robert Schulze
245ea0c186
Implement const/non-const shortcut 2024-03-06 16:56:01 +00:00
taiyang-li
8b2e163d76 Merge remote-tracking branch 'origin/master' into opt_multiif_decimal 2024-03-05 17:17:10 +08:00
Nikita Taranov
bcd3ad01de
Merge pull request #55809 from amosbird/alt-51399 
Improve the performance of serialized aggregation method when involving multiple [nullable] columns
 2024-03-01 22:44:56 +01:00
Amos Bird
3b04f5c605
Improve the performance of serialized aggregation method when involving multiple [nullable] columns. 2024-02-29 21:48:15 +08:00
taiyang-li
f7d1739808 fix failed uts 2024-02-29 15:00:09 +08:00
Raúl Marín
c1c4a8c02a Merge remote-tracking branch 'blessed/master' into argmin_optimization 2024-02-27 12:49:18 +01:00
Azat Khuzhin
a34f42ca22 Remove lock from the ReadProgressCallback 
It looks redundant (added in 5ef51ed), though it has "fix tests" in the
log message, but CI reports is not available for the commits from that
PR [1], so let's try.

  [1]: https://github.com/ClickHouse/ClickHouse/pull/37543

Also this can be a big problem, since the code under that lock
(throttling or quotas with previous implementation that uses
boost::atomic_shared_ptr) may sleep.

Some numbers:

    run                     | time
    ------------------------|------
    max_threads=100 before  | 23.1
    max_threads=100 after   | 15.1
    max_threads=4500 before | 4.5
    max_threads=4500 after  | 2.3

Query:

    select sum(number) from numbers_mt(2000000) settings max_threads=X, max_block_size = 1

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-02-22 18:00:55 +01:00
Robert Schulze
ae597d86dd
Merge remote-tracking branch 'rschu1ze/master' into vector-dot-product 2024-02-22 10:16:28 +00:00
Azat Khuzhin
a4f765cae7
Improve performance of SELECTs with active mutations (#59531) 
* Configure keeper for perf tests

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Improve performance of SELECTs with active mutations

getAlterMutationCommandsForPart() can be a hot path for query execution
when there are pending mutations.

- LOG_TEST - it is not only check one bool, but actually a bunch of
  atomics as well.

- Return std::vector over std::map (map is not required there) - no
  changes in performance.

- Copy only RENAME_COLUMN (since only this mutation is required by
  AlterConversions).

And here are results:

run|result
-|-
SELECT w/o ALTER|queries: 1565, QPS: 355.259, RPS: 355.259
SELECT w/ ALTER unpatched|queries: 2099, QPS: 220.623, RPS: 220.623
SELECT w/ ALTER and w/o LOG_TEST|queries: 2730, QPS: 235.859, RPS: 235.859
SELECT w/ ALTER and w/o LOG_TEST and w/ RENAME_COLUMN only|queries: 2995, QPS: 290.982, RPS: 290.982

But there are still room for improvements, at least MergeTree engines
could implement getStorageSnapshotForQuery().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Add AlterConversions::supportsMutationCommandType(), flatten vector<vector<MutationCommand>>

* Work around what appears to be a clang static analysis bug

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Michael Kolupaev <michael.kolupaev@clickhouse.com>
 2024-02-22 08:51:10 +00:00