ClickHouse/tests/performance
Alexander Gololobov 3c000b098a
Merge pull request #36638 from nickitat/fix_sorting_step
Fix SortingStep::updateOutputStream()
2022-04-26 15:49:49 +02:00
..
agg_functions_min_max_any.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
aggregate_functions_of_group_by_keys.xml
aggregating_merge_tree_simple_aggregate_function_string.xml
aggregating_merge_tree.xml
aggregation_in_order_2.xml better perf test 2021-10-19 17:13:52 +03:00
aggregation_in_order.xml
aggregation_overflow.xml
analyze_array_tuples.xml
and_function.xml
any_anyLast.xml
arithmetic_operations_in_aggr_func.xml
arithmetic.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
array_auc.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
array_element.xml
array_fill.xml
array_index_low_cardinality_numbers.xml
array_index_low_cardinality_strings.xml
array_join.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
array_reduce.xml
asof.xml Add ASOF performance test with random data 2022-03-17 19:48:25 +01:00
async_remote_read.xml
avg_weighted.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
avg.xml Needs more data in avg tests 2021-11-29 19:50:03 +01:00
base64_hits.xml
base64.xml
basename.xml
bigint_arithm.xml
bit_operations_fixed_string_numbers.xml
bit_operations_fixed_string.xml
bitCount.xml
bloom_filter_insert.xml
bloom_filter_select.xml
bounding_ratio.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
casts.xml
cidr.xml
classification.xml Fixed performance tests 2022-03-15 15:43:39 +00:00
codec_none.xml Update perf test (non-significant) 2021-08-28 00:32:21 +03:00
codecs_float_insert.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
codecs_float_select.xml
codecs_int_insert.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
codecs_int_select.xml
collations.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
ColumnMap.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
columns_hashing.xml
complex_array_creation.xml
concat_hits.xml
conditional.xml
consistent_hashes.xml Rename yandexConsistentHash to kostikConsistentHash 2022-03-24 02:18:25 +01:00
constant_column_comparison.xml
constant_column_search.xml Fix constant_column_search performance tests 2022-04-12 15:22:14 +02:00
count.xml Make count performance tests longer 2021-11-26 09:06:37 +01:00
countIf.xml Implement AggregateFunctionCountNotNullUnary::addBatchSinglePlace 2021-11-25 16:41:48 +01:00
countMatches.xml
cpu_synthetic.xml
cryptographic_hashes.xml
date_parsing.xml
date_time_64.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
date_time_long.xml replaced toStartOfFiveMinute to toStartOfFiveMinutes 2022-04-22 10:49:59 +08:00
date_time_short.xml Adjust timezone in performance tests 2022-03-11 23:49:13 +01:00
datetime_comparison.xml Specialize date time comparision. 2021-08-03 16:49:39 +08:00
decimal_aggregates.xml Optimize quantilesExact{Low,High} to use nth_element instead of sort 2022-02-03 12:24:33 +00:00
decimal_casts.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
decimal_format.xml Add performance test 2021-08-17 09:11:35 +03:00
decimal_parse.xml
dict_join.xml Smaller table in performance/dict_join.xml 2021-06-24 11:27:55 +03:00
direct_dictionary.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
distinct_combinator.xml
distributed_aggregation_memory_efficient.xml
distributed_aggregation.xml
duplicate_order_by_and_distinct.xml
empty_string_deserialization.xml
empty_string_serialization.xml
encodeXMLComponent.xml fixes 2021-05-13 01:32:53 +03:00
encrypt_decrypt_empty_string_slow.xml
encrypt_decrypt_empty_string.xml
encrypt_decrypt.xml
entropy.xml
explain_ast.xml Add perf test 2021-11-29 13:09:53 +01:00
extract.xml
file_table_function.xml Disable broken performance test 2022-04-25 11:26:05 +02:00
fixed_string16.xml
flat_dictionary.xml Added performance tests 2021-10-23 15:01:30 +03:00
float_formatting.xml
float_mod.xml
float_parsing.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
format_date_time.xml
format_readable.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
function_calculation_after_sorting_and_limit.xml fix SortingStep::updateOutputStream() 2022-04-25 17:29:14 +00:00
functions_coding.xml
functions_geo.xml
functions_with_hash_tables.xml
fuse_sumcount.xml Some extra perf tests around sumCount 2021-11-24 11:14:31 +01:00
fuzz_bits.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
general_purpose_hashes_on_UUID.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
general_purpose_hashes.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
generate_table_function.xml Adjust timezone in performance tests 2022-03-11 23:49:13 +01:00
great_circle_dist.xml
group_array_moving_sum.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
group_array_sorted.xml Fix performance tests 2022-04-12 15:22:14 +02:00
group_by_fixed_keys.xml
group_by_sundy_li.xml Fix group_by_sundy_li performance test 2022-04-12 17:06:38 +02:00
h3.xml Performance tests fix H3 2022-02-18 15:57:54 +00:00
has_all.xml Simplified hasAll performance tests 2022-04-04 13:34:40 +02:00
hash_table_sizes_stats.xml Fix hash_table_sizes_stats performance test 2022-04-12 17:46:16 +02:00
hashed_array_dictionary.xml HashedDictionaries added read performance tests 2021-10-25 20:12:29 +03:00
hashed_dictionary.xml HashedDictionaries added read performance tests 2021-10-25 20:12:29 +03:00
if_array_num.xml
if_array_string.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
if_string_const.xml
if_string_hits.xml
if_to_multiif.xml
if_transform_strings_to_enum.xml
information_value.xml
injective_functions_inside_uniq.xml
insert_parallel.xml
insert_select_default_small_block.xml
insert_sequential_and_background_merges.xml
insert_values_with_expressions.xml
inserts_arrays_lowcardinality.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
int_parsing.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
intDiv.xml
ip_trie.xml fix tests 2021-11-03 18:06:19 +03:00
IPv4.xml
IPv6.xml
jit_aggregate_functions.xml Updated performance tests 2021-08-03 16:19:56 +03:00
jit_large_requests.xml
jit_small_requests.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
join_max_streams.xml Add tests/performance/join_max_streams.xml 2021-07-07 20:51:07 +03:00
joins_in_memory_pmj.xml Fix performance tests 2022-04-12 15:22:14 +02:00
joins_in_memory.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
json_extract_rapidjson.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
json_extract_simdjson.xml Adjust perf test 2021-10-24 00:11:28 +03:00
least_greatest_hits.xml
leftpad.xml
linear_regression.xml
local_replica.xml
logical_functions_large.xml
logical_functions_medium.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
logical_functions_small.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
lot_of_subcolumns.xml Fix perf test 2021-07-17 13:11:50 +03:00
map_populate_series.xml Added additional performance test 2022-02-05 16:11:36 +00:00
materialized_view_parallel_insert.xml
math.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
memory_cache_friendliness.xml
merge_table_streams.xml
merge_tree_huge_pk.xml
merge_tree_insert.xml Updated tests 2022-03-11 21:16:25 +00:00
merge_tree_many_partitions_2.xml Fixed performance tests 2022-03-15 15:43:39 +00:00
merge_tree_many_partitions.xml Fixed performance tests 2022-03-15 15:43:39 +00:00
merge_tree_simple_select.xml
mingroupby-orderbylimit1.xml
mmap_io.xml Minor fix in perf test 2021-08-30 00:18:38 +03:00
modulo.xml
monotonous_order_by.xml
ngram_distance.xml
nlp.xml Improve NLP performance test 2021-08-05 11:07:31 +00:00
normalize_utf8.xml Add nfd and perf test 2021-09-28 13:42:37 +03:00
number_formatting_formats.xml Fix some perf tests 2022-01-31 21:07:20 +03:00
optimize_window_funnel.xml
optimized_select_final_one_part.xml
optimized_select_final.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
or_null_default.xml
order_by_decimals.xml
order_by_read_in_order.xml
order_by_single_column.xml
order_by_tuple.xml Make ORDER BY tuple almost as fast as ORDER BY columns 2022-01-27 21:42:08 +08:00
order_with_limit.xml
parallel_final.xml Fixed performance tests 2022-03-15 15:43:39 +00:00
parallel_index.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
parallel_insert.xml
parallel_mv.xml Disable merges for parallel_mv 2021-10-02 11:54:47 +03:00
parse_engine_file.xml Add Arrow/Parquet/ORC in perf tests 2021-06-15 16:15:27 +03:00
point_in_polygon_const.xml
point_in_polygon.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
polymorphic_parts_l.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
polymorphic_parts_m.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
polymorphic_parts_s.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
pre_limit_no_sorting.xml
prewhere_with_row_level_filter.xml
prewhere.xml
push_down_limit.xml
quantile_merge.xml
quantile.xml Fix slow performance test 2021-06-27 18:54:21 +03:00
questdb_sum_float32.xml
questdb_sum_float64.xml
questdb_sum_int32.xml Trying to have smaller test 2021-11-24 14:13:59 +03:00
rand.xml
random_fixed_string.xml
random_printable_ascii.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
random_string_utf8.xml
random_string.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
range_hashed_dictionary.xml Fix performance tests 2022-01-19 10:41:58 +00:00
range.xml
read_from_comp_parts.xml
read_hits_with_aio.xml
read_in_order_many_parts.xml Performance tests fix 2022-03-15 15:43:39 +00:00
README.md Mention scipy dep in performance bench README 2022-02-20 02:43:28 +01:00
redundant_functions_in_order_by.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
reinterpret_as.xml Fix perf test 2021-12-12 04:31:56 +03:00
removing_group_by_keys.xml
rewrite_sumIf.xml fix the test 2021-05-15 09:35:00 +03:00
right.xml
round_down.xml
round_methods.xml
scalar2.xml Fix performance regression of scalar query 2022-04-06 17:50:22 +08:00
scalar.xml
select_format.xml Fix possible error 'file_size: Operation not supported' 2022-02-10 09:23:27 +03:00
sequence_match.xml Changing the tests to use mainly test.hits dataset, as hits_10[0]m_single datasets have a slightly different schema 2021-08-27 10:14:17 +02:00
set_hits.xml
set_index.xml Change perf set_index test to check build index 2021-12-23 13:53:36 +01:00
set.xml
short_circuit_functions.xml Update tests 2021-08-13 14:55:33 +03:00
simple_join_query.xml
single_fixed_string_groupby.xml
slices_hits.xml
sort_radix_trivial.xml
sort.xml
sparse_column.xml support filtering by sparse columns without convertion to full 2022-02-15 14:30:54 +03:00
split_filter.xml
string_join.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
string_set.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
string_sort.xml
string_to_int.xml Safer readInt 2021-05-27 22:56:51 +08:00
subqueries.xml
sum_map.xml perf: add missing DROP TABLE queries 2021-11-03 08:26:52 +03:00
sum.xml Adjust performance test 2021-09-21 22:19:13 +03:00
sumIf.xml
synthetic_hardware_benchmark.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
trim_numbers.xml
trim_urls.xml
trim_whitespace.xml
tsv_csv_nullable_parsing.xml Add performance test 2021-10-27 17:29:36 +03:00
uniq.xml Remove unrelated change 2021-08-09 17:54:14 +03:00
url_hits.xml update thresholds - 1 2021-05-20 17:14:55 +03:00
vectorize_aggregation_combinators.xml
visit_param_extract_raw.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
website.xml calculate perf test precision thresholds from historical data 2021-05-26 16:30:43 +03:00
window_functions.xml Mark window functions ready for general use. 2021-08-04 17:45:32 +03:00
writing_valid_utf8.xml Fix possible error 'file_size: Operation not supported' 2022-02-10 09:23:27 +03:00

ClickHouse performance tests

This directory contains .xml-files with performance tests for @akuzm tool.

How to write performance test

First of all you should check existing tests don't cover your case. If there are no such tests than you should write your own.

You have to specify preconditions. It contains table names. Only hits_100m_single, hits_10m_single, test.hits are available in CI.

You can use substitions, create, fill and drop queries to prepare test. You can find examples in this folder.

Take into account, that these tests will run in CI which consists of 56-cores and 512 RAM machines. Queries will be executed much faster than on local laptop.

If your test continued more than 10 minutes, please, add tag long to have an opportunity to run all tests and skip long ones.

How to run performance test

TODO @akuzm

How to validate single test

pip3 install clickhouse_driver scipy
../../docker/test/performance-comparison/perf.py --runs 1 insert_parallel.xml