thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-01 20:12:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
149,500 Commits 9,193 Branches 1,782 Tags 2.2 GiB
a6ccf19869
Commit Graph

1310 Commits

Author SHA1 Message Date
taiyang-li
47bcc854fc Merge remote-tracking branch 'origin/master' into opt_string_ascii 2024-04-15 15:16:54 +08:00
Kruglov Pavel
ce7432424e
Merge branch 'master' into trivial-insert-select-from-files 2024-04-12 14:26:48 +02:00
kevinyhzou
a9711c4bec modify test 2024-04-12 16:57:34 +08:00
kevinyhzou
878e0ed26b add performance test 2024-04-12 16:37:35 +08:00
lgbo-ustc
feddc40c45 update test 2024-04-12 10:00:52 +08:00
lgbo-ustc
e9635189d2 Merge remote-tracking branch 'origin/master' into json_format_early_skip 2024-04-12 08:53:38 +08:00
Dmitry Novik
5d7882bb60
Always use new analyzer in perf tests 2024-04-11 16:50:01 +02:00
Maksim Kita
1218cf1568 Updated implementation 2024-04-11 11:09:08 +03:00
Maksim Kita
0dcd70b5a5 Added performance tests 2024-04-11 11:09:08 +03:00
lgbo-ustc
d2e840f60a update 2024-04-11 15:59:04 +08:00
lgbo-ustc
a87fb7dc84 Merge remote-tracking branch 'origin/master' into json_format_early_skip 2024-04-11 14:37:12 +08:00
taiyang-li
4f4fa79ce7 last commit 2024-04-10 16:19:34 +08:00
Maksim Kita
35e1e5aff7 Perf script update path in documentation 2024-04-09 14:42:29 +03:00
Kruglov Pavel
19f00e7c42
Restart CI 2024-04-09 12:33:18 +02:00
avogar
ed6e4fbe16 Improve trivial insert select from files, add separate max_parsing_threads setting 2024-04-08 13:56:15 +00:00
taiyang-li
07815d93de add perf tests 2024-04-08 18:31:11 +08:00
lgbo-ustc
28de6d8b25 Merge remote-tracking branch 'origin/master' into json_format_early_skip 2024-04-08 10:15:33 +08:00
lgbo-ustc
cbedf8c5d2 improve by skip to line end when all fields are read 2024-04-03 10:30:32 +08:00
Robert Schulze
d729b63293
Merge pull request #61144 from bigo-sg/inline_poaarray_details 
Inline small but frequently invoked functions in PODArrayDetails
 2024-04-02 13:25:25 +02:00
Raúl Marín
b2e764f633 Revert "Merge pull request #61564 from liuneng1994/optimize_in_single_value" 
This reverts commit 20a45b4073, reversing
changes made to a642f4d3ec.
 2024-04-01 13:51:56 +02:00
János Benjamin Antal
e288528ede
Merge pull request #61931 from ClickHouse/fix-aggregating-merge-tree-simple-aggregate-function-string 
Fix performance test `aggregating_merge_tree_simple_aggregate_function_string`
 2024-03-28 08:39:45 +01:00
Dmitry Novik
65121b712e Merge remote-tracking branch 'origin/master' into enable-analyzer 2024-03-26 19:02:30 +01:00
János Benjamin Antal
1ec7258d1e Enable allow_suspicious_primary_key for perf tests that need it 2024-03-26 11:09:03 +00:00
Dmitry Novik
8bd49aa303 Try to fix perf tests 2024-03-25 17:06:42 +01:00
Yarik Briukhovetskyi
20a45b4073
Merge pull request #61564 from liuneng1994/optimize_in_single_value 
New analyzer pass to optimize in single value
 2024-03-25 12:53:01 +01:00
李扬
be68b21de4
Merge branch 'ClickHouse:master' into opt_memcpy_small 2024-03-21 11:27:10 +08:00
taiyang-li
c350991005 add more tests 2024-03-21 10:29:35 +08:00
李扬
7c44505b49
Merge branch 'ClickHouse:master' into inline_poaarray_details 2024-03-21 10:20:45 +08:00
Raúl Marín
6c7be89c61
Merge pull request #61100 from Algunenano/experiment_int128 
30x faster printing for 256-bit integers
 2024-03-20 12:32:26 +01:00
taiyang-li
49422debaf add perf tests 2024-03-20 18:59:22 +08:00
liuneng
a29d4f5db2 fix bug 2024-03-20 10:24:57 +08:00
liuneng
532d72a7d8 add test case 2024-03-19 17:30:16 +08:00
Raúl Marín
4626a0b3a1 Merge remote-tracking branch 'blessed/master' into experiment_int128 2024-03-18 16:10:32 +01:00
Robert Schulze
f02db275c5
Merge remote-tracking branch 'rschu1ze/master' into dotProduct-memcpy 2024-03-18 09:26:29 +00:00
李扬
4da1dbd2c2
Merge branch 'ClickHouse:master' into inline_poaarray_details 2024-03-18 10:57:48 +08:00
李扬
9296be3747
Merge branch 'ClickHouse:master' into opt_memcpy_small 2024-03-18 10:35:09 +08:00
Sema Checherinda
025ad9791c
Merge pull request #58879 from azat/small-block-lock-contention 
Improve query performance in case of very small blocks
 2024-03-13 17:08:37 +01:00
Raúl Marín
cc92b8ffc5 Merge remote-tracking branch 'blessed/master' into experiment_int128 2024-03-13 13:57:41 +01:00
taiyang-li
dc9548d31b add perf test 2024-03-13 18:57:33 +08:00
taiyang-li
d17212616c add perf tests 2024-03-13 17:26:45 +08:00
Raúl Marín
45efa69189 Add perf tests 2024-03-08 15:17:59 +01:00
Robert Schulze
245ea0c186
Implement const/non-const shortcut 2024-03-06 16:56:01 +00:00
taiyang-li
8b2e163d76 Merge remote-tracking branch 'origin/master' into opt_multiif_decimal 2024-03-05 17:17:10 +08:00
Nikita Taranov
bcd3ad01de
Merge pull request #55809 from amosbird/alt-51399 
Improve the performance of serialized aggregation method when involving multiple [nullable] columns
 2024-03-01 22:44:56 +01:00
Amos Bird
3b04f5c605
Improve the performance of serialized aggregation method when involving multiple [nullable] columns. 2024-02-29 21:48:15 +08:00
taiyang-li
f7d1739808 fix failed uts 2024-02-29 15:00:09 +08:00
Raúl Marín
c1c4a8c02a Merge remote-tracking branch 'blessed/master' into argmin_optimization 2024-02-27 12:49:18 +01:00
Azat Khuzhin
a34f42ca22 Remove lock from the ReadProgressCallback 
It looks redundant (added in 5ef51ed), though it has "fix tests" in the
log message, but CI reports is not available for the commits from that
PR [1], so let's try.

  [1]: https://github.com/ClickHouse/ClickHouse/pull/37543

Also this can be a big problem, since the code under that lock
(throttling or quotas with previous implementation that uses
boost::atomic_shared_ptr) may sleep.

Some numbers:

    run                     | time
    ------------------------|------
    max_threads=100 before  | 23.1
    max_threads=100 after   | 15.1
    max_threads=4500 before | 4.5
    max_threads=4500 after  | 2.3

Query:

    select sum(number) from numbers_mt(2000000) settings max_threads=X, max_block_size = 1

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-02-22 18:00:55 +01:00
Robert Schulze
ae597d86dd
Merge remote-tracking branch 'rschu1ze/master' into vector-dot-product 2024-02-22 10:16:28 +00:00
Azat Khuzhin
a4f765cae7
Improve performance of SELECTs with active mutations (#59531) 
* Configure keeper for perf tests

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Improve performance of SELECTs with active mutations

getAlterMutationCommandsForPart() can be a hot path for query execution
when there are pending mutations.

- LOG_TEST - it is not only check one bool, but actually a bunch of
  atomics as well.

- Return std::vector over std::map (map is not required there) - no
  changes in performance.

- Copy only RENAME_COLUMN (since only this mutation is required by
  AlterConversions).

And here are results:

run|result
-|-
SELECT w/o ALTER|queries: 1565, QPS: 355.259, RPS: 355.259
SELECT w/ ALTER unpatched|queries: 2099, QPS: 220.623, RPS: 220.623
SELECT w/ ALTER and w/o LOG_TEST|queries: 2730, QPS: 235.859, RPS: 235.859
SELECT w/ ALTER and w/o LOG_TEST and w/ RENAME_COLUMN only|queries: 2995, QPS: 290.982, RPS: 290.982

But there are still room for improvements, at least MergeTree engines
could implement getStorageSnapshotForQuery().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Add AlterConversions::supportsMutationCommandType(), flatten vector<vector<MutationCommand>>

* Work around what appears to be a clang static analysis bug

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Michael Kolupaev <michael.kolupaev@clickhouse.com>
 2024-02-22 08:51:10 +00:00
Kruglov Pavel
10535132c3
Merge pull request #59385 from Avogar/fix-bad-types-check 
Fix validating suspicious/experimental types in nested types
 2024-02-21 14:38:01 +01:00
Robert Schulze
7d354164a5
Add performance test for dotProduct() 2024-02-20 21:41:10 +00:00
taiyang-li
2d6b4b400c Merge remote-tracking branch 'origin/master' into opt_sum_decimal 2024-02-19 12:25:22 +08:00
avogar
109720d162 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-bad-types-check 2024-02-15 12:10:49 +00:00
Raúl Marín
11519f949b
Merge pull request #59731 from kitaisreal/asof-join-try-sort-with-radix-sort 
ASOF JOIN use trySort with RadixSort
 2024-02-14 15:54:22 +01:00
avogar
64779835fa Update tests 2024-02-14 12:48:05 +00:00
李扬
90d07ba82c
Trivial optimize of function coalesce. (#59627) 
* reuse result of functionfactory::get

* add perf test

* Update src/Functions/coalesce.cpp

Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>

* change as requested

---------

Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
 2024-02-14 11:29:45 +01:00
Maksim Kita
2caf3f0fbb Fixed tests 2024-02-13 15:41:17 +03:00
Maksim Kita
a359ceecb5 ASOF JOIN use trySort with RadixSort 2024-02-13 15:41:17 +03:00
Raúl Marín
d17b0867e6 Merge remote-tracking branch 'blessed/master' into argmin_optimization 2024-02-09 13:08:23 +01:00
taiyang-li
549b77021d add some perf tests 2024-02-04 15:55:22 +08:00
taiyang-li
ddc6aad8ff merge master 2024-02-01 10:58:33 +08:00
Raúl Marín
f67bff12b7
Merge pull request #59148 from bigo-sg/improve_if_with_floating 
Continue optimizing branch miss of if function when result type is float*/decimal*/int*
 2024-01-31 12:42:06 +01:00
taiyang-li
2ad7607bad opt if when input type is map 2024-01-31 17:33:47 +08:00
Maksim Kita
20c1f0c18f Revert "Revert "Add new aggregation function groupArraySorted()"" 2024-01-30 17:15:29 +03:00
Raúl Marín
cda39e64e4 Perf: Only consider XML files 2024-01-29 17:47:50 +01:00
Raúl Marín
6a2fcb778f Restore comment 2024-01-29 13:07:30 +01:00
Raúl Marín
c79a151cca Simplify query_run_metric_arrays in perf tests 2024-01-29 13:00:49 +01:00
taiyang-li
49fc8a7099 Merge branch 'master' into improve_if_with_floating 2024-01-29 11:02:05 +08:00
Azat Khuzhin
44e42052b1 Fix perf tests after sumMap starts to filter out -0. 
Before perf tests was relying on the following:

    SELECT sumMap(['foo', 'bar'], [-0., -0.])

    ┌─sumMap(['foo', 'bar'], [-0., -0.])─┐
    │ (['bar','foo'],[-0,-0])            │
    └────────────────────────────────────┘

While it got changed, and now:

    ┌─sumMap(['foo', 'bar'], [-0., -0.])─┐
    │ ([],[])                            │
    └────────────────────────────────────┘

But it works for nan:

    SELECT sumMap(['foo', 'bar'], [nan, nan])

    ┌─sumMap(['foo', 'bar'], [nan, nan])─┐
    │ (['bar','foo'],[nan,nan])          │
    └────────────────────────────────────┘

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-27 13:14:07 +01:00
taiyang-li
56db02c953 Merge branch 'master' into improve_if_with_floating 2024-01-26 19:21:49 +08:00
taiyang-li
e8629cf4f5 add another perf and tests 2024-01-25 18:02:55 +08:00
taiyang-li
a657a2631f add perf tests 2024-01-24 19:58:07 +08:00
Robert Schulze
a4c6f87fb9
Further reduce runtime of norm_distance.xml 2024-01-23 10:52:52 +00:00
Raúl Marín
1e00dec997 Merge remote-tracking branch 'blessed/master' into argmin_optimization 2024-01-22 14:56:55 +01:00
Dmitry Novik
a18a8d8ea3
Merge pull request #59009 from nickitat/uniq_optimisation_for_distributed 
`uniqExact` state parallel merging for distributed queries
 2024-01-22 14:05:02 +01:00
Nikita Taranov
2b5482be8c add perf test 2024-01-19 17:57:11 +01:00
Raúl Marín
3739d46817 Merge remote-tracking branch 'blessed/master' into argmin_optimization 2024-01-19 14:43:44 +01:00
Robert Schulze
aa2d36e598
Reduce memory consumption of norm_distance.xml 
The test data was stored in in-memory Array and Tuple tables, each
consuming about 80 GB. The population of the Tuple tables exceeded the
maximum memory thresholds (*).

As a fix,
- Reduce the Array cardinality from 200 to 150 and the number of rows
  from 10 to 8 million, reducing memory consumption to ca. 50 GB. This
  should pass the memory threshold and does not affect the test purpose.
- Don't test tuples. Due to their data layout, vector search is not
  efficient anyways. Saves another 80 GB.
 2024-01-19 13:09:15 +00:00
Robert Schulze
180a07ee4b
Delete redundant test norm_distance_float.xml 
- Duplicates the workload in norm_distance.xml
 2024-01-19 13:09:14 +00:00
Robert Schulze
25fb44f16d
Extend performance test norm_dist.xml 
Dimension = 200 (instead of 10) is more realistic for vector search use cases.
 2024-01-19 13:09:08 +00:00
Raúl Marín
b92073ed7b
Revert "Extend performance test norm_dist.xml" 2024-01-19 13:21:03 +01:00
Raúl Marín
6aa5ac4cc6 Merge remote-tracking branch 'blessed/master' into argmin_optimization 2024-01-18 12:04:19 +01:00
Robert Schulze
abcb1b5c9f
Extend performance test norm_dist.xml 
Dimension = 200 (instead of 10) is more realistic for vector search use cases.
 2024-01-17 18:16:29 +00:00
Raúl Marín
849ac1fe99 Implement findExtremeMinIndex / findExtremeMaxIndex 2024-01-17 16:22:40 +01:00
Raúl Marín
cb9cc7a4cc Fix table names 2024-01-17 11:36:07 +01:00
Alexey Milovidov
1afcab35c1 Fix supply chain attack in performance tests 2024-01-14 08:25:12 +01:00
Duc Canh Le
458c8d758d simplify perf tests and minor code change 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2024-01-11 08:25:35 +00:00
Duc Canh Le
6331d8a6f2 Merge branch 'master' into final_no_copy 
Resolve conflicts

Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2024-01-11 02:55:14 +00:00
Azat Khuzhin
3d88dba0a7 Fix perf tests duration (checks.test_duration_ms) 
The column in the source was seconds in Float32, we need to convert it to
milliseconds in UInt64.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-10 17:03:53 +03:00
taiyang-li
231de4ac49 Merge branch 'master' into ch_opt_array_element 2024-01-10 15:49:43 +08:00
李扬
2c76b1789c
Update tests/performance/array_element.xml 
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
 2024-01-10 10:46:29 +08:00
Nikolay Degterinsky
24733700fb
Merge pull request #57745 from KevinyhZou/imporve_multi_if_nullable 
Improve `MultiIf` function performance while type is nullable
 2024-01-09 23:17:58 +01:00
taiyang-li
e5b4bc8f45 Merge branch 'master' into ch_opt_array_element 2024-01-09 17:17:38 +08:00
李扬
f6691ec334
Update tests/performance/array_element.xml 
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
 2024-01-09 10:15:37 +08:00
taiyang-li
5c0ea7db94 Merge branch 'master' into ch_opt_array_element 2024-01-08 10:41:20 +08:00
Raúl Marín
0522d859c2
Merge pull request #58334 from Algunenano/minmax_non_numeric 
Speedup MIN/MAX for non numeric types
 2024-01-04 19:48:18 +01:00
Raúl Marín
641caba5b0 Adapt more tests 2024-01-04 11:36:33 +00:00
Raúl Marín
1d1edd5b57 Reduce sum_map.xml 2024-01-04 11:31:20 +00:00
Raúl Marín
2aa6690f2c Reduce hashed_dictionary.xml 2024-01-04 11:29:17 +00:00