thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-25 09:02:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
0d71bf1411
ClickHouse/tests/performance
History
Azat Khuzhin 0d71bf1411 Upload time of the perf tests into artifacts as test_duration_ms 
Now perf test changes/failures will have two rows, row for new and row
for old server.

I thought about uploading only the time of the test on the new server,
but because not all perf tests uploaded, you cannot always get the time
of the test without the changes (i.e. from run on the upstream/master
repo/branch).

<details>

Before:

```sql
SELECT
    concat(test, ' #', toString(query_index)),
    'slower' AS test_status,
    0 AS test_duration_ms,
    concat('https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.', test, '.', toString(query_index)) AS report_url
FROM queries
WHERE (changed_fail != 0) AND (diff > 0)
UNION ALL
SELECT
    concat(test, ' #', toString(query_index)),
    'unstable' AS test_status,
    0 AS test_duration_ms,
    concat('https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#unstable-queries.', test, '.', toString(query_index)) AS report_url
FROM queries
WHERE unstable_fail != 0

Query id: 49dfdc9a-f549-4499-9a1a-410e5053f6c1

┌─concat(test, ' #', toString(query_index))─┬─test_status─┬─test_duration_ms─┬─report_url─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ hashed_array_dictionary #16               │ slower      │                0 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.hashed_array_dictionary.16 │
│ ngram_distance #2                         │ slower      │                0 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.2           │
│ ngram_distance #3                         │ slower      │                0 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.3           │
│ ngram_distance #4                         │ slower      │                0 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.4           │
└───────────────────────────────────────────┴─────────────┴──────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```

After:

```sql
SELECT
    concat(test, ' #', toString(query_index), '::', test_desc_.1) AS test_name,
    'slower' AS test_status,
    test_desc_.2 AS test_duration_ms,
    concat('https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.', test, '.', toString(query_index)) AS report_url
FROM queries
ARRAY JOIN map('old', left, 'new', right) AS test_desc_
WHERE (changed_fail != 0) AND (diff > 0)
UNION ALL
SELECT
    concat(test, ' #', toString(query_index), '::', test_desc_.1) AS test_name,
    'unstable' AS test_status,
    test_desc_.2 AS test_duration_ms,
    concat('https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#unstable-queries.', test, '.', toString(query_index)) AS report_url
FROM queries
ARRAY JOIN map('old', left, 'new', right) AS test_desc_
WHERE unstable_fail != 0

Query id: 20475bfd-754b-4159-aa16-7798f4720bf8

┌─test_name────────────────────────┬─test_status─┬─test_duration_ms─┬─report_url─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ hashed_array_dictionary #16::old │ slower      │           0.2149 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.hashed_array_dictionary.16 │
│ hashed_array_dictionary #16::new │ slower      │           0.2519 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.hashed_array_dictionary.16 │
│ ngram_distance #2::old           │ slower      │           0.3598 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.2           │
│ ngram_distance #2::new           │ slower      │           0.4425 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.2           │
│ ngram_distance #3::old           │ slower      │           0.3644 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.3           │
│ ngram_distance #3::new           │ slower      │           0.4716 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.3           │
│ ngram_distance #4::old           │ slower      │           0.3577 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.4           │
│ ngram_distance #4::new           │ slower      │           0.4577 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.4           │
└──────────────────────────────────┴─────────────┴──────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 13:48:41 +01:00
..
scripts Upload time of the perf tests into artifacts as test_duration_ms 2023-12-29 13:48:41 +01:00
agg_functions_min_max_any.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
aggregate_functions_of_group_by_keys.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
aggregate_with_serialized_method.xml optimize 2023-08-08 13:38:25 +08:00
aggregating_merge_tree_simple_aggregate_function_string.xml Update tests for new numbers(limit) syntax 2021-02-10 22:32:32 +03:00
aggregating_merge_tree.xml Update aggregating_merge_tree.xml 2020-06-09 01:41:57 +03:00
aggregation_by_partitions.xml Enable perf tests added in #45364 (#46623) 2023-02-28 00:26:11 +01:00
aggregation_external.xml Added performance test 2023-10-11 19:01:00 +03:00
aggregation_in_order_2.xml better perf test 2021-10-19 17:13:52 +03:00
aggregation_in_order.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
aggregation_overflow.xml Add perftest. 2021-03-18 12:38:37 +03:00
analyze_array_tuples.xml Explicitly mark short perftest queries 2020-06-23 15:09:54 +03:00
and_function.xml Remove garbage from performance tests 2020-04-18 15:54:16 +03:00
any_anyLast.xml speed up some perf tests (for other machines) 2021-01-16 00:15:06 +03:00
arithmetic_operations_in_aggr_func.xml fixup 2020-07-03 11:39:43 +03:00
arithmetic.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
array_auc.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
array_element.xml More stable perf tests 2020-06-08 16:57:33 +03:00
array_fill.xml performance comparison 2020-04-16 23:55:21 +03:00
array_fold.xml Revert "Remove arrayFold" 2023-12-14 09:52:29 +00:00
array_index_low_cardinality_numbers.xml Fix slow performance test 2022-04-29 04:29:06 +02:00
array_index_low_cardinality_strings.xml update perf tests 2021-02-02 17:21:43 +03:00
array_join.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
array_reduce.xml Remove garbage from performance tests 2020-04-18 15:54:16 +03:00
arrow_format.xml Fix performance test 2022-08-15 11:41:03 +00:00
asof.xml Fix ASOF LEFT JOIN performance degradation (#47544) 2023-03-18 23:53:00 +01:00
async_remote_read.xml Update async_remote_read.xml 2022-11-28 14:00:49 +01:00
avg_weighted.xml Update avg_weighted.xml (#56797) 2023-11-16 20:46:17 +01:00
avg.xml Needs more data in avg tests 2021-11-29 19:50:03 +01:00
base64_hits.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
base64.xml Remove garbage from performance tests 2020-04-18 15:54:16 +03:00
basename.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
bigint_arithm.xml Faster 256-bit multiplication (#15418) 2020-09-29 20:52:34 +03:00
bit_operations_fixed_string_numbers.xml More stable perf tests 2020-06-08 16:57:33 +03:00
bit_operations_fixed_string.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
bitmap_array_element.xml Update bitmap_array_element.xml 2022-10-03 14:56:34 +02:00
bloom_filter_insert.xml Fix perf tests 2020-11-08 16:55:26 +03:00
bloom_filter_select.xml Update bloom_filter_select.xml 2020-11-09 17:58:27 +03:00
bounding_ratio.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
casts.xml update perf tests 2021-02-02 17:21:43 +03:00
cidr.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
classification.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
codec_none.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
codecs_float_insert.xml added fpc codec to float perftest 2022-05-26 22:32:56 +03:00
codecs_float_select.xml Small follow-up for FPC codec 2022-06-15 14:21:28 +02:00
codecs_int_insert.xml Deprecate Gorilla compression of non-float columns 2023-01-20 17:31:16 +00:00
codecs_int_select.xml Deprecate Gorilla compression of non-float columns 2023-01-20 17:31:16 +00:00
collations.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
column_array_filter.xml Update tests/performance/column_array_filter.xml 2023-02-04 18:49:13 +08:00
column_array_replicate.xml add performance tests 2023-02-02 20:16:16 +08:00
ColumnMap.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
columns_hashing.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
complex_array_creation.xml fix more queries 2020-06-26 22:11:43 +03:00
concat_hits.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
conditional.xml optimize function coalesce with two arguments 2022-05-30 22:29:35 +00:00
consistent_hashes.xml Rename yandexConsistentHash to kostikConsistentHash 2022-03-24 02:18:25 +01:00
constant_column_comparison.xml Remove tags as well 2022-05-02 02:35:44 +02:00
constant_column_search.xml Merge pull request #36841 from ClickHouse/fix-performance-test-5 2022-05-02 12:22:56 +03:00
count_from_formats.xml Disable cache in perf test 2023-08-23 21:01:18 +00:00
count.xml Make count performance tests longer 2021-11-26 09:06:37 +01:00
countDigits.xml Add performance test 2023-02-09 04:52:33 +01:00
countIf.xml Implement AggregateFunctionCountNotNullUnary::addBatchSinglePlace 2021-11-25 16:41:48 +01:00
countMatches.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
cpu_synthetic.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
cryptographic_hashes.xml Print nicer exception if BLAKE3 is unavailable 2022-10-05 00:11:41 +03:00
date_parsing.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
date_time_64.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
date_time_long.xml Various test, code and docs fixups 2023-01-15 13:47:34 +00:00
date_time_short.xml Various test, code and docs fixups 2023-01-15 13:47:34 +00:00
datetime64_conversion.xml Add a test 2023-02-08 02:17:23 +01:00
datetime_comparison.xml Specialize date time comparision. 2021-08-03 16:49:39 +08:00
decimal_aggregates.xml Optimize quantilesExact{Low,High} to use nth_element instead of sort 2022-02-03 12:24:33 +00:00
decimal_casts.xml Whitespace 2023-02-08 03:38:23 +01:00
decimal_format.xml Add performance test 2021-08-17 09:11:35 +03:00
decimal_parse.xml Added performance test 2020-05-17 06:16:34 +03:00
destroy_aggregate_states.xml impl 2022-06-04 19:14:38 +00:00
dict_join.xml Smaller table in performance/dict_join.xml 2021-06-24 11:27:55 +03:00
direct_dictionary.xml tests/perf: fix dependency check during DROP 2022-12-16 15:15:15 +01:00
distinct_combinator.xml more optimal aggregate functions with both 'if' and 'distinct' combinators 2020-06-22 17:57:30 +03:00
distinct_in_order.xml Polishing 2023-02-05 13:38:20 +00:00
distributed_aggregation_memory_efficient.xml Try fix perftest. 2020-05-06 21:53:40 +03:00
distributed_aggregation.xml Merge remote-tracking branch 'origin/master' into HEAD 2020-04-16 23:03:17 +03:00
empty_string_deserialization.xml fixup 2020-07-02 10:44:16 +03:00
empty_string_serialization.xml Adjust ignored perf test changes after NUMA binding 2020-10-30 18:12:15 +03:00
encodeXMLComponent.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
encrypt_decrypt_empty_string.xml Remove trash 2022-06-22 06:23:35 +02:00
entropy.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
enum_in_set.xml Cache cast function in set during execution (#55712) 2023-10-23 13:31:44 +02:00
explain_ast.xml Make explain_ast even larger 2022-10-14 14:06:44 +02:00
extract.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
file_table_function.xml Disable broken performance test 2022-04-25 11:26:05 +02:00
final_with_lonely_parts.xml more stable stateless test + add perf. test 2023-08-31 06:27:06 +00:00
fixed_string16.xml Remove tags as well 2022-05-02 02:35:44 +02:00
flat_dictionary.xml tests/perf: fix dependency check during DROP 2022-12-16 15:15:15 +01:00
float_formatting.xml adjust perf test thresholds 2021-03-26 00:00:18 +03:00
float_mod.xml Add performance test 2020-07-21 13:38:14 +03:00
float_parsing.xml Trim down some tests 2022-04-29 04:33:12 +02:00
format_date_time.xml fix more queries 2020-06-26 22:11:43 +03:00
format_readable.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
formats_columns_sampling.xml Fix performance test 2022-05-23 17:42:13 +00:00
function_calculation_after_sorting_and_limit.xml fix SortingStep::updateOutputStream() 2022-04-25 17:29:14 +00:00
functions_coding.xml Trim down some tests 2022-04-29 04:33:12 +02:00
functions_geo.xml Remove garbage from performance tests 2020-04-18 15:54:16 +03:00
functions_with_hash_tables.xml Adjust perf test thresholds 2020-11-27 15:08:42 +03:00
fuse_sumcount.xml Some extra perf tests around sumCount 2021-11-24 11:14:31 +01:00
fuzz_bits.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
general_purpose_hashes_on_UUID.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
general_purpose_hashes.xml Add xxh3 hash function (#43411) 2022-11-26 00:14:08 +01:00
generate_table_function.xml Adjust timezone in performance tests 2022-03-11 23:49:13 +01:00
get_map_value.xml Optimization for getting value from map, arrayElement(1/2) (#55929) 2023-10-27 09:54:25 +02:00
grace_hash_join.xml Small fixes 2022-11-25 12:04:12 +00:00
great_circle_dist.xml Improve performance of geo distance functions 2022-05-25 14:22:22 +02:00
group_array_moving_sum.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
group_by_consecutive_keys.xml slightly faster and perf test 2023-12-16 03:35:59 +00:00
group_by_fixed_keys.xml Prefetching in aggregation (#39304) 2022-09-21 18:59:07 +02:00
group_by_sundy_li.xml Make projections production-ready 2023-05-10 03:35:13 +02:00
groupby_onekey_nullable.xml Make projections production-ready 2023-05-10 03:35:13 +02:00
h3.xml Performance tests fix H3 2022-02-18 15:57:54 +00:00
has_all.xml Simplified hasAll performance tests 2022-04-04 13:34:40 +02:00
hash_table_sizes_stats_small.xml Apply preallocation optimisation more carefully (#44455) 2023-01-09 13:30:48 +01:00
hash_table_sizes_stats.xml fix (#41648) 2022-09-21 19:04:03 +02:00
hashed_array_dictionary.xml tests/perf: fix dependency check during DROP 2022-12-16 15:15:15 +01:00
hashed_dictionary_load_factor.xml Add ability to configure maximum load factor for the HASHED/SPARSE_HASHED layout 2023-05-19 06:07:21 +02:00
hashed_dictionary_sharded.xml Optimize SPARSE_HASHED layout (by using PackedHashMap) 2023-05-19 06:07:21 +02:00
hashed_dictionary.xml tests/perf: fix dependency check during DROP 2022-12-16 15:15:15 +01:00
hashjoin_with_large_output.xml Add tests/performance/hashjoin_with_large_output.xml 2023-12-22 15:52:45 +00:00
hierarchical_dictionaries.xml Add ability to load hashed dictionaries using multiple threads 2023-01-13 13:39:25 +01:00
if_array_num.xml update perf tests 2021-02-02 17:21:43 +03:00
if_array_string.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
if_string_const.xml fix more queries 2020-06-26 22:11:43 +03:00
if_string_hits.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
if_to_multiif.xml Adjust perf test 2021-01-15 13:22:51 +03:00
if_transform_strings_to_enum.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
if.xml Fix if.xml. 2023-12-20 10:58:46 +08:00
information_value.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
injective_functions_inside_uniq.xml Delete injective functions inside uniq (#12337) 2020-07-10 13:42:41 +03:00
insert_parallel.xml Adjust ignored perf test changes after NUMA binding 2020-10-30 18:12:15 +03:00
insert_select_default_small_block.xml Add performance test 2020-05-29 05:30:49 +03:00
insert_sequential_and_background_merges.xml Rename test 2020-10-23 13:42:15 +03:00
insert_values_with_expressions.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
inserts_arrays_lowcardinality.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
int_parsing.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
intDiv.xml Add perf test 2021-04-15 01:35:21 +03:00
ip_trie.xml fix tests 2021-11-03 18:06:19 +03:00
IPv4.xml fix more queries 2020-06-26 22:11:43 +03:00
IPv6.xml fix more queries 2020-06-26 22:11:43 +03:00
jit_aggregate_functions.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
jit_large_requests.xml Update jit_large_requests.xml 2021-01-18 14:00:24 +03:00
jit_small_requests.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
jit_sort.xml Fixed tests 2022-05-11 21:59:51 +02:00
join_append_block.xml Fix significant join performance regression 2022-06-14 21:14:18 +08:00
join_filter_pushdown.xml add perf test 2023-07-24 20:34:01 +02:00
join_max_streams.xml Add tests/performance/join_max_streams.xml 2021-07-07 20:51:07 +03:00
join_used_flags.xml enable used flags's reinit only when the hash talbe rehash 2023-05-11 11:06:13 +08:00
joins_in_memory.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
json_extract_rapidjson.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
json_extract_simdjson.xml update benckmark test 2023-09-15 18:09:58 +08:00
json_type.xml minor fixes 2022-06-28 14:21:21 +00:00
least_greatest_hits.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
leftpad.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
lightweight_delete.xml Enable lightweight delete support by default 2023-03-01 19:35:55 +01:00
line_as_string_parsing.xml Support parallel parsing for LineAsString input format 2022-10-28 21:56:09 +00:00
linear_regression.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
local_replica.xml Adjust ignored perf test changes after NUMA binding 2020-10-30 18:12:15 +03:00
logical_functions_large.xml Adjust ignored perf test changes after NUMA binding 2020-10-30 18:12:15 +03:00
logical_functions_medium.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
logical_functions_small.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
lot_of_subcolumns.xml Fix perf test 2021-07-17 13:11:50 +03:00
low_cardinality_argument.xml Update tests/performance/low_cardinality_argument.xml 2022-07-15 18:24:44 +02:00
low_cardinality_from_json.xml Added reverted files and fixes for failing fuzzer tests 2022-10-28 12:37:59 +02:00
low_cardinality_query.xml Add perftest: low_cardinality_query 2022-11-08 17:19:18 +08:00
lower_upper_utf8.xml tests: add perf test for lowerUTF8()/upperUTF8() 2022-09-17 11:16:45 +02:00
lz4_hits_columns.xml Skip empty literals in lz4 decompression (#40142) 2022-09-06 13:58:26 +02:00
lz4.xml Skip empty literals in lz4 decompression (#40142) 2022-09-06 13:58:26 +02:00
map_populate_series.xml Added additional performance test 2022-02-05 16:11:36 +00:00
map_update.xml add tests 2023-03-28 17:20:05 +00:00
materialized_view_parallel_insert.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
materialized_view_parallelize_output_from_storages.xml Adjust min_insert_block_size_rows for materialized_view_parallelize_output_from_storages 2023-06-14 19:11:23 +03:00
math.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
memory_bound_merging.xml Parallel replicas. Part [2] (#43772) 2023-02-03 14:34:18 +01:00
memory_cache_friendliness.xml Add perf test 2021-02-07 05:42:43 +03:00
merge_table_streams.xml fix tests 2022-06-23 21:19:07 +02:00
merge_tree_huge_pk.xml Adjust ignored perf test changes after NUMA binding 2020-10-30 18:12:15 +03:00
merge_tree_insert.xml Revert "Merge pull request #55682 from ClickHouse/revert-35961-decimal-column-improve-get-permutation" 2023-10-25 21:48:13 +03:00
merge_tree_many_partitions_2.xml Fixed performance tests 2022-03-15 15:43:39 +00:00
merge_tree_many_partitions.xml Fixed performance tests 2022-03-15 15:43:39 +00:00
merge_tree_simple_select.xml Adjust ignored perf test changes after NUMA binding 2020-10-30 18:12:15 +03:00
min_max_index.xml Reuse granule during skip index reading 2023-10-18 14:40:34 +08:00
mingroupby-orderbylimit1.xml Adjust ignored perf test changes after NUMA binding 2020-10-30 18:12:15 +03:00
mmap_io.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
modulo.xml Fix a rare false negative in perf tests 2021-03-02 19:21:30 +03:00
monotonous_order_by.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
ngram_distance.xml Remove tags as well 2022-05-02 02:35:44 +02:00
nlp.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
norm_distance_float.xml Perf test for norm/distance with long arrays of floats 2022-07-03 08:01:49 +02:00
norm_distance.xml Use same norm and distance function names for tuples and arrays 2022-05-25 22:39:59 +02:00
normalize_utf8.xml Normalize UTF8 performance test fix 2022-06-09 15:35:53 +02:00
number_formatting_formats.xml Fix some perf tests 2022-01-31 21:07:20 +03:00
optimize_sorting_for_input_stream.xml rename and fix perf test 2022-07-19 21:21:39 +00:00
optimize_window_funnel.xml Update optimize_window_funnel.xml 2021-01-09 05:15:40 +03:00
optimized_select_final_one_part.xml adjust perf test thresholds 2021-03-26 00:00:18 +03:00
optimized_select_final.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
or_null_default.xml adjust perf test thresholds 2021-03-26 00:00:18 +03:00
orc_filter_push_down.xml fix code style 2023-11-02 10:49:18 +08:00
orc_tuple_field_prune.xml fix failed perf test 2023-11-02 15:27:48 +08:00
order_by_decimals.xml Remove tags as well 2022-05-02 02:35:44 +02:00
order_by_read_in_order.xml Tests 2022-07-06 22:27:53 +00:00
order_by_single_column.xml Remove tags as well 2022-05-02 02:35:44 +02:00
order_by_tuple.xml Fix: remove heeavy performance tests, introduced within this PR 2022-07-07 07:57:05 +00:00
order_with_limit.xml Remove tags as well 2022-05-02 02:35:44 +02:00
parallel_final.xml deprecate old MergeTree syntax 2022-06-23 11:24:54 +02:00
parallel_index.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
parallel_insert.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
parallel_mv.xml tests/performance: improve parallel_mv test 2022-07-18 16:23:17 +03:00
parquet_filter.xml Performance test 2023-08-21 14:15:52 -07:00
parquet_read.xml improve parquet struct field reading 2023-11-01 15:18:39 +08:00
parse_engine_file.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
point_in_polygon_const.xml fix more queries 2020-06-26 22:11:43 +03:00
point_in_polygon.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
polymorphic_parts_l.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
polymorphic_parts_m.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
polymorphic_parts_s.xml Deprecate in-memory parts 2023-05-03 00:31:09 +02:00
position_empty_needle.xml change as request 2023-01-30 16:13:12 +08:00
pre_limit_no_sorting.xml Added performance test to resemble questionable benchmark 2020-06-25 23:40:07 +03:00
prefetch_in_aggregation.xml Prefetching in aggregation (#39304) 2022-09-21 18:59:07 +02:00
prepare_hash_before_merge.xml Optimize the merge if all hashSets are singleLevel in UniqExactSet (#52973) 2023-08-30 11:26:16 +02:00
prewhere_with_row_level_filter.xml Update prewhere_with_row_level_filter.xml 2021-03-09 11:15:03 +03:00
prewhere.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
push_down_limit.xml Make some perf tests faster on slower machines 2020-12-23 05:40:55 +03:00
quantile_merge.xml Marked some perf test queries as short 2020-11-11 19:58:31 +03:00
quantile.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
queries_over_aggregation.xml more parallelism 2022-08-16 18:56:22 +02:00
query_interpretation_join.xml Add benchmark for query interpretation with JOINs 2022-11-23 13:15:35 +01:00
rand.xml Add function canonicalRand (#43124) 2022-11-15 00:27:19 +01:00
random_fixed_string.xml fixup 2020-07-02 10:44:16 +03:00
random_printable_ascii.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
random_string_utf8.xml more fixes 2020-10-30 18:54:38 +03:00
random_string.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
range_hashed_dictionary.xml tests/perf: fix dependency check during DROP 2022-12-16 15:15:15 +01:00
range.xml Adjust ignored perf test changes after NUMA binding 2020-10-30 18:12:15 +03:00
re2_regex_caching.xml Fix performance test for regexp cache 2023-07-09 02:21:48 +02:00
read_from_comp_parts.xml Mark query short 2020-07-24 16:59:50 +03:00
read_hits_with_aio.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
read_in_order_many_parts.xml Performance tests fix 2022-03-15 15:43:39 +00:00
reading_from_file.xml Perf test 2023-04-07 20:06:11 +00:00
README.md Fix perf test README 2023-12-27 12:16:17 +00:00
redundant_functions_in_order_by.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
reinterpret_as.xml Fix perf test 2021-12-12 04:31:56 +03:00
removing_group_by_keys.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
rewrite_array_exists.xml add perf test 2023-02-09 12:30:50 +08:00
rewrite_sumIf.xml fix the test 2021-05-15 09:35:00 +03:00
right.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
round_down.xml make the test longer 2020-10-30 18:16:30 +03:00
round_methods.xml fixup 2020-07-02 10:44:16 +03:00
scalar2.xml deprecate old MergeTree syntax 2022-06-23 11:24:54 +02:00
scalar.xml Remove garbage from performance tests 2020-04-18 15:54:16 +03:00
schema_inference_text_formats.xml Refactor and improve schema inference for text formats 2022-12-07 21:19:27 +00:00
select_format.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
sequence_match.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
set_disable_skip_index.xml add perf test 2023-04-04 21:29:52 +00:00
set_hits.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
set_index.xml Change perf set_index test to check build index 2021-12-23 13:53:36 +01:00
set.xml Adjust ignored perf test changes after NUMA binding 2020-10-30 18:12:15 +03:00
short_circuit_functions.xml Update tests 2021-08-13 14:55:33 +03:00
simple_join_query.xml Remove garbage from performance tests 2020-04-18 15:54:16 +03:00
single_fixed_string_groupby.xml fix (#41648) 2022-09-21 19:04:03 +02:00
slices_hits.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
sort_patterns.xml Revert "Merge pull request #55682 from ClickHouse/revert-35961-decimal-column-improve-get-permutation" 2023-10-25 21:48:13 +03:00
sort_radix_trivial.xml fix some broken performance tests 2020-05-28 10:45:03 +03:00
sort.xml Enable query_plan_remove_redundant_sorting optimization by default 2023-01-24 13:38:21 +00:00
sparse_column.xml add perf test 2022-05-28 12:56:38 +00:00
split_filter.xml Review fixes. 2021-01-22 16:46:56 +03:00
storage_join_direct_join.xml fix performance test 2023-08-07 06:25:46 +00:00
string_join.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
string_set.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
string_sort.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
string_to_int.xml Safer readInt 2021-05-27 22:56:51 +08:00
subqueries.xml Update subqueries.xml 2021-02-17 12:39:40 +03:00
sum_map.xml perf: add missing DROP TABLE queries 2021-11-03 08:26:52 +03:00
sum.xml Adjust performance test 2021-09-21 22:19:13 +03:00
sumIf.xml Devirtualize -If and vectorize count 2020-12-21 11:35:38 +08:00
synthetic_hardware_benchmark.xml fix (#41648) 2022-09-21 19:04:03 +02:00
trim_numbers.xml Remove garbage from performance tests 2020-04-18 15:54:16 +03:00
trim_urls.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
trim_whitespace.xml Remove garbage from performance tests 2020-04-18 15:54:16 +03:00
tsv_csv_nullable_parsing.xml Add performance test 2021-10-27 17:29:36 +03:00
unary_arithmetic_functions.xml Dynamic dispatch infrastructure style fixes 2022-05-24 14:25:29 +02:00
unary_logical_functions.xml UnaryLogicalFunctions improve performance using dynamic dispatch 2022-06-14 17:30:11 +02:00
uniq_stored.xml impl 2022-09-16 15:41:15 +02:00
uniq_to_count.xml Add optimization uniq to count 2023-09-13 16:16:11 +08:00
uniq_without_key.xml Optimize merge of uniqExact without_key (#43072) 2022-11-17 13:19:02 +01:00
uniq.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
uniqExactIf.xml Add parallel state merge for some other combinator except If (#50413) 2023-06-08 00:41:32 +02:00
url_hits.xml Add tests and doc for some url-related functions 2022-10-26 10:52:57 +00:00
vectorize_aggregation_combinators.xml Added performance test to resemble questionable benchmark 2020-06-25 23:40:07 +03:00
views_max_insert_threads.xml Fix max_insert_threads while pushing to views 2022-07-03 15:14:05 +03:00
visit_param_extract_raw.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
website.xml fix (#41648) 2022-09-21 19:04:03 +02:00
window_functions_downstream_multithreading.xml Multithreading after window functions (#50771) 2023-10-27 12:36:28 +02:00
window_functions.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00
writing_valid_utf8.xml Remove "preconditions" from performance tests (overengineering, unneeded feature) 2022-05-02 02:33:22 +02:00

README.md

ClickHouse performance tests

This directory contains .xml-files with performance tests for @akuzm tool.

How to write performance test

First of all you should check existing tests don't cover your case. If there are no such tests then you should write your own.

You can use substitions, create, fill and drop queries to prepare test. You can find examples in this folder.

If your test takes more than 10 minutes, please, add tag long to have an opportunity to run all tests and skip long ones.

How to run performance test

TODO @akuzm

How to validate single test

pip3 install clickhouse_driver scipy
../../tests/performance/scripts/perf.py --runs 1 insert_parallel.xml