Alexey Milovidov
88826e2da5
Remove "preconditions" from performance tests (overengineering, unneeded feature)
2022-05-02 02:33:22 +02:00
Alexander Kuzmenkov
5da54c2745
calculate perf test precision thresholds from historical data
2021-05-26 16:30:43 +03:00
Alexander Kuzmenkov
979d23208e
speed up some perf tests (for other machines)
2021-01-16 00:15:06 +03:00
Alexander Kuzmenkov
d9180f1e3e
Make some perf tests faster on slower machines
2020-12-23 05:40:55 +03:00
Alexander Kuzmenkov
3529100da9
Adjust ignored perf test changes after NUMA binding
2020-10-30 18:12:15 +03:00
Alexander Kuzmenkov
242ec7e56c
make some tests faster
2020-10-20 15:12:06 +03:00
Alexander Kuzmenkov
36538ce08f
Don't account for short queries, we'll deal with them separately.
...
New query:
```
WITH ceil(max(q[3]), 1) AS h
SELECT concat('sed -i s\'/^<test.*$/<test max_ignored_relative_change="', toString(h), '">/g\' tests/performance/', test, '.xml') AS s
FROM
(
SELECT
test,
query_index,
count(*),
min(event_time),
max(event_time) AS t,
arrayMap(x -> floor(x, 3), quantiles(0, 0.5, 0.95, 1)(stat_threshold)) AS q,
median(stat_threshold) AS m
FROM perftest.query_metrics
WHERE (metric = 'client_time') AND (abs(diff) < 0.05) AND (old > 0.2)
GROUP BY
test,
query_index,
query_display_name
HAVING (t > '2020-09-01 00:00:00') AND (m > 0.1)
ORDER BY test DESC
)
GROUP BY test
ORDER BY h DESC
FORMAT PrettySpace
```
2020-09-17 13:00:51 +03:00
Alexander Kuzmenkov
0f8aec59a3
Adjust ignore thresholds for unstable perf tests
...
Based on historical data.
```
SELECT
test,
ceil(max(q[3]), 1) AS h
FROM
(
SELECT
test,
query_index,
count(*),
min(event_time),
max(event_time) AS t,
arrayMap(x -> floor(x, 3), quantiles(0, 0.5, 0.95, 1)(stat_threshold)) AS q,
median(stat_threshold) AS m
FROM perftest.query_metrics
WHERE (metric = 'client_time') AND (abs(diff) < 0.05)
GROUP BY
test,
query_index,
query_display_name
HAVING (t > '2020-09-01 00:00:00') AND (m > 0.1)
ORDER BY m DESC
)
GROUP BY test
ORDER BY h DESC
FORMAT TSV
cryptographic_hashes 1.3
collations 0.8
joins_in_memory_pmj 0.8
joins_in_memory 0.7
merge_tree_simple_select 0.7
set_index 0.7
decimal_casts 0.7
website 0.6
logical_functions_medium 0.5
count 0.5
merge_tree_many_partitions 0.5
decimal_aggregates 0.5
codecs_int_insert 0.5
column_column_comparison 0.5
insert_parallel 0.4
parse_engine_file 0.4
read_in_order_many_parts 0.4
logical_functions_small 0.4
parallel_insert 0.3
parallel_index 0.3
push_down_limit 0.3
jit_large_requests 0.3
select_format 0.3
arithmetic 0.3
merge_tree_huge_pk 0.3
materialized_view_parallel_insert 0.3
columns_hashing 0.3
if_array_string 0.3
random_string 0.2
random_printable_ascii 0.2
set 0.2
empty_string_serialization 0.2
```
To apply:
```
sed 's/^\(.*\) \(.*\)$/sed -i "s\/^<test.*$\/<test max_ignored_relative_change="'"'"\2">\/g" tests\/performance\/\1.xml/g' ../bad.tsv | bash
```
2020-09-16 18:27:51 +03:00
Alexander Kuzmenkov
10b6f98ca6
fixup
2020-07-03 11:39:43 +03:00
Alexander Kuzmenkov
a2b8412b41
fixup
2020-07-02 10:44:16 +03:00
Alexander Kuzmenkov
5d324751a0
fix more queries
2020-06-26 22:11:43 +03:00
Alexander Kuzmenkov
6a3df071a2
fixup
2020-06-09 01:13:08 +03:00
Alexander Kuzmenkov
d952e5aba2
performance comparison
2020-04-29 20:26:28 +03:00
Alexey Milovidov
785d2c0e01
Remove garbage from performance tests
2020-04-18 15:54:16 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up ( #9974 )
...
* Move some code outside dbms/src folder
* Fix paths
2020-04-02 02:51:21 +03:00
