ClickHouse/dbms/tests/performance
alexey-milovidov 1d910c5071
Merge pull request #7608 from amosbird/batchreduce
Optimize arrayReduce, -Array and -State combinators
2019-11-10 22:10:03 +03:00
..
agg_functions_min_max_any.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
array_element.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
array_fill.xml Create array_fill.xml 2019-10-31 14:30:11 +08:00
array_join.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
base64_hits.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
base64.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
basename.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
bounding_ratio.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
cidr.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
codec_double_delta.xml Removed <name> from all performance tests #6179 2019-07-28 00:18:54 +03:00
codec_gorilla.xml Removed <name> from all performance tests #6179 2019-07-28 00:18:54 +03:00
column_column_comparison.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
columns_hashing.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
complex_array_creation.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
concat_hits.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
conditional.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
consistent_hashes.xml Updated performance test 2019-08-03 22:51:47 +03:00
constant_column_comparison.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
constant_column_search.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
count.xml Removed <name> from all performance tests #6179 2019-07-28 00:18:54 +03:00
cpu_synthetic.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
create_benchmark_page.py add string comparisons performance test [#CLICKHOUSE-3070] 2017-06-22 23:10:43 +03:00
cryptographic_hashes.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
date_parsing.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
date_time.xml More performance test for Date and DateTime 2019-10-15 15:20:54 +03:00
early_constant_folding.xml Add perf test for early constant folding 2019-11-01 22:33:03 +08:00
empty_string_deserialization.xml Added performance test to show degradation of performance in gcc-9 in more isolated way 2019-08-02 23:16:02 +03:00
empty_string_serialization.xml Added performance test to show degradation of performance in gcc-9 in more isolated way 2019-08-02 23:18:03 +03:00
entropy.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
first_significant_subdomain.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
fixed_string16.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
float_parsing.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
format_date_time.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
functions_coding.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
functions_geo.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
general_purpose_hashes.xml Update tests. 2019-08-20 16:33:25 +03:00
grear_circle_dist.xml Add test for greatCircleDist 2019-10-29 19:49:37 +03:00
group_array_moving_sum.xml Removed <name> from all performance tests #6179 2019-07-28 00:18:54 +03:00
h3.xml Updated performance test 2019-06-30 23:14:45 +03:00
if_array_num.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
if_array_string.xml Changed repository URL 2019-09-23 19:18:19 +03:00
if_string_const.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
if_string_hits.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
insert_values_with_expressions.xml add performance test 2019-09-20 22:45:26 +03:00
inserts_arrays_lowcardinality.xml performance test for lowcardinality array, showing significant performance degradation comparing to plain array 2019-09-12 22:46:31 +02:00
int_parsing.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
IPv4.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
IPv6.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
jit_large_requests.xml Update jit_large_requests.xml 2019-07-05 15:59:52 +03:00
jit_small_requests.xml Add performance test concerning JIT with bigger dataset 2019-07-05 14:42:20 +02:00
joins_in_memory_pmj.xml add simple perf test for partial merge join 2019-09-18 20:04:10 +03:00
joins_in_memory.xml infinite loop detection in MemoryTracker + shrink joins perf test into 1Gb memory usage 2019-07-17 15:40:05 +03:00
json_extract_rapidjson.xml Updated json perftest to loop. 2019-08-19 00:50:51 +03:00
json_extract_simdjson.xml Updated json perftest to loop. 2019-08-19 00:50:51 +03:00
leftpad.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
linear_regression.xml Whitespaces 2019-08-09 02:55:08 +03:00
math.xml Added performance test 2019-08-01 03:37:47 +03:00
merge_table_streams.xml Added performance test in appreciation of #5797 2019-07-07 03:19:29 +03:00
merge_tree_huge_pk.xml Update merge_tree_huge_pk.xml 2019-09-18 23:07:23 +03:00
merge_tree_many_partitions_2.xml Removed <name> from all performance tests #6179 2019-07-28 00:18:54 +03:00
merge_tree_many_partitions.xml Removed <name> from all performance tests #6179 2019-07-28 00:18:54 +03:00
merge_tree_simple_select.xml Removed <name> from all performance tests #6179 2019-07-28 00:18:54 +03:00
ngram_distance.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
number_formatting_formats.xml fixed heap buffer overflow in PacketPayloadWriteBuffer 2019-07-29 18:41:47 +03:00
nyc_taxi.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
order_by_decimals.xml add perf test 2019-08-07 17:41:03 +03:00
order_by_read_in_order.xml better performance test for order by optimization 2019-08-02 19:17:18 +03:00
order_by_single_column.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
parse_engine_file.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
range.xml add range performance tests 2019-11-05 15:07:25 +08:00
read_hits_with_aio.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
README.md Small performance test documentation. (#5408) 2019-05-25 13:46:08 +03:00
right.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
roundDown.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
scalar.xml add perf test for subqueries with large scalars 2019-10-22 23:55:11 +08:00
select_format.xml fixed heap buffer overflow in PacketPayloadWriteBuffer 2019-07-29 18:41:47 +03:00
set_hits.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
set.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
simple_join_query.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
slices_hits.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
sort.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
string_join.xml String join perf test 2019-11-08 15:07:27 +08:00
string_sort.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
system_numbers.xml Remove <name> and <metrics> from performance tests fixed-5634 2019-06-18 19:55:13 +03:00
trim_numbers.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
trim_urls.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
trim_whitespace.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
uniq.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
url_hits.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
vectorize_aggregation_combinators.xml Vectorize arrayReduce and -Array combinator 2019-11-07 11:53:47 +08:00
visit_param_extract_raw.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00
website.xml Revert "Removed <name> from all performance tests #6179" 2019-07-28 00:17:44 +03:00

ClickHouse performance tests

This directory contains .xml-files with performance tests for clickhouse-performance-test tool.

How to write performance test

First of all you should check existing tests don't cover your case. If there are no such tests than you should write your own.

There two types of performance tests:

  • First is executed in loop, and have tag <type>loop</type> in config.
  • Second one is executed only once and have tag <type>once</type> in config.

Type once should be used only for endless queries. Even if your query really long (10 seconds+), it's better to choose loop test.

After you have choosen type, you have to specify preconditions. It contains table names. Only hits_100m_single, hits_10m_single, test.hits are available in CI.

The most important part of test is stop_conditions. For loop test you should always use min_time_not_changing_for_ms stop condition. For once test you can choose between average_speed_not_changing_for_ms and max_speed_not_changing_for_ms, but first is preferable. Also you should always specify total_time_ms metric. Endless tests will be ignored by CI.

metrics and main_metric settings are not important and can be ommited, because loop tests are always compared by min_time metric and once tests compared by max_rows_per_second.

You can use substitions, create, fill and drop queries to prepare test. You can find examples in this folder.

Take into account, that these tests will run in CI which consists of 56-cores and 512 RAM machines. Queries will be executed much faster than on local laptop.

How to run performance test

You have to run clickhouse-server and after you can start testing:

$ clickhouse-performance-test --input-file my_lovely_test1.xml --input-file my_lovely_test2.xml
$ clickhouse-performance-test --input-file /my_lovely_test_dir/