Commit Graph

677 Commits

Author SHA1 Message Date
Alexander Kuzmenkov
b0e660651a cleanup 2020-11-16 19:09:58 +03:00
vdimir
5e0e22301b
Slightly improve ip_dict loading performance, handle v4 to v6 masks in prepossessing, add more tests 2020-11-15 18:55:34 +03:00
vdimir
ca8e4b1607
Minor fixes of ip dict 2020-11-14 20:03:13 +03:00
myrrc
b1c0089b78 Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted 2020-11-14 16:50:04 +03:00
Alexander Kuzmenkov
e4dd68039e fix 2020-11-13 08:00:43 +03:00
myrrc
a8d8dd7d3e Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted 2020-11-12 14:50:59 +03:00
Alexander Kuzmenkov
0d10a4990e fix 2020-11-12 11:27:33 +03:00
Alexander Kuzmenkov
91f88c79f9 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-12 11:20:09 +03:00
Alexey Milovidov
f1d721fa74 Marked some perf test queries as short 2020-11-11 19:58:31 +03:00
Alexander Kuzmenkov
f31d7869ac Merge remote-tracking branch 'origin/master' into HEAD 2020-11-11 14:51:35 +03:00
Alexander Kuzmenkov
906b338a1d fixes 2020-11-11 14:51:21 +03:00
Danila Kutenin
c36e6fe378 Fix tests finally 2020-11-10 18:22:59 +03:00
Danila Kutenin
c0308a5d85 Fix tests, I cannot count number of zeros sometimes 2020-11-10 18:21:39 +03:00
Danila Kutenin
847fa9ca3c Merge 2020-11-10 18:15:26 +03:00
Danila Kutenin
8504efde77 Merge 2020-11-10 18:12:49 +03:00
Danila Kutenin
2057198779 Fix performance tests that became very fast 2020-11-10 17:58:05 +03:00
myrrc
e460248624 Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted 2020-11-10 14:54:24 +03:00
Alexander Kuzmenkov
84fb76bad4 fixes 2020-11-10 07:19:43 +03:00
Alexander Kuzmenkov
030419db90 Try 12 threads max in perf test.
This will allow us to use more machines, not only Gold 6320.
2020-11-09 19:14:05 +03:00
alexey-milovidov
e47ed55d32
Merge pull request #14977 from achimbab/indexOf_bloomfilter
Fix indexOf() to use BloomFilter
2020-11-09 17:58:42 +03:00
alexey-milovidov
2994499680
Update bloom_filter_select.xml 2020-11-09 17:58:27 +03:00
Alexey Milovidov
ff906369ea Adjust time for perf test 2020-11-08 23:22:50 +03:00
vdimir
81671ceb39
Add tests/performance/ip_trie.xml [wip] 2020-11-08 21:32:22 +03:00
Alexey Milovidov
f07988eec7 Remove perf test of sumbur hash, because we do not care 2020-11-08 21:17:30 +03:00
Alexey Milovidov
7257a5c3d2 Fix perf tests 2020-11-08 16:55:36 +03:00
Alexey Milovidov
f4178dd91f Fix perf tests 2020-11-08 16:55:26 +03:00
alexey-milovidov
c7aa0c851a
Update bloom_filter.xml 2020-11-08 16:51:45 +03:00
alexey-milovidov
d96914b044
Update bloom_filter.xml 2020-11-08 16:51:03 +03:00
Alexey Milovidov
cc012e2cc7 Merge branch 'master' into achimbab-indexOf_bloomfilter 2020-11-08 14:42:03 +03:00
Alexey Milovidov
990f8a23ab Translate comments to english 2020-11-05 21:51:36 +03:00
alexey-milovidov
ee3e289ec2
Merge pull request #15983 from ClickHouse/no_background_pool_no_more
Replace BackgroundProcessingPool with SchedulePool task and ThreadPool.
2020-11-05 21:38:22 +03:00
myrrc
ddd40fc4ce fixed the perf test 2020-11-05 12:20:05 +03:00
myrrc
9788065786 Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted 2020-11-05 08:44:37 +03:00
alexey-milovidov
dbdff0d779
Merge pull request #16661 from ClickHouse/optimize-or-fill
Optimize -OrNull and -OrDefault aggregate functions by providing specializations
2020-11-04 23:37:11 +03:00
alexey-milovidov
ca0037d18b
Update group_by_sundy_li.xml 2020-11-04 18:07:18 +03:00
myrrc
43b2d20314 updated the decimal template magic 2020-11-04 16:14:07 +03:00
myrrc
3d70ab7f3b Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted 2020-11-04 14:29:59 +03:00
Alexey Milovidov
e0d962a3cf Add a test for #8772 2020-11-04 14:22:02 +03:00
Alexey Milovidov
938ef62532 Add a test 2020-11-04 14:00:43 +03:00
Alexey Milovidov
fdd7f31cf7 Add a test 2020-11-03 23:28:23 +03:00
myrrc
5c6bd218ad adjusted the perftest 2020-11-03 18:01:29 +03:00
myrrc
fcbc0fb91e added specialized performance test 2020-11-03 16:12:27 +03:00
Anton Popov
a3a8e18637
Merge branch 'master' into select_final 2020-11-03 00:00:43 +03:00
alesapin
101aa9e931 Merge branch 'master' into no_background_pool_no_more 2020-11-02 10:49:19 +03:00
Alexander Kuzmenkov
e15a8d5ac0 fixup 2020-10-30 21:03:07 +03:00
Alexander Kuzmenkov
7e76b7b510 more fixes 2020-10-30 18:54:38 +03:00
Alexander Kuzmenkov
38726847fc make the test longer 2020-10-30 18:16:30 +03:00
Alexander Kuzmenkov
3529100da9 Adjust ignored perf test changes after NUMA binding 2020-10-30 18:12:15 +03:00
Anton Popov
48185d437a
Update optimized_select_final.xml 2020-10-30 17:48:59 +03:00
Anton Popov
9f0f981642
Update optimized_select_final.xml 2020-10-29 14:03:50 +03:00
Pavel Kruglov
4592c5e59b Add OPTIMIZE in perf test 2020-10-27 14:42:26 +03:00
alesapin
d5d0819698 Merge branch 'master' into no_background_pool_no_more 2020-10-26 12:58:51 +03:00
alesapin
84dbbdb312 Rename test 2020-10-23 13:42:15 +03:00
alesapin
166b3de466 Make insert test faster 2020-10-23 13:31:33 +03:00
Pavel Kruglov
0274c9e9eb Add perf test 2020-10-23 12:05:57 +03:00
alesapin
7a63b373f9 Merge branch 'blind_fix_of_read_in_order' into no_background_pool_no_more 2020-10-22 15:47:11 +03:00
Alexander Kuzmenkov
6293d1dbbe Merge remote-tracking branch 'origin/master' into HEAD 2020-10-21 18:39:59 +03:00
Alexander Kuzmenkov
242ec7e56c make some tests faster 2020-10-20 15:12:06 +03:00
Vasily Nemkov
1cf9781d4e AES: Attempt to make performance tests faster and more stable. 2020-10-20 08:05:39 +03:00
Vasily Nemkov
4124538b56 Merge remote-tracking branch 'upstream/master' into AES_encrypt_decrypt 2020-10-18 00:00:13 +03:00
alesapin
60d14352f8 Merge branch 'master' into no_background_pool_no_more 2020-10-15 16:34:14 +03:00
alexey-milovidov
1d309c34c8
Merge pull request #15497 from filipecaixeta/master
Add an SQL function formatReadableTimeDelta to format time delta
2020-10-14 22:36:57 +03:00
Alexander Kuzmenkov
cb15e72229 make some perf tests slower 2020-10-14 18:39:27 +03:00
Vasily Nemkov
a6f36fc53b Performance tests for encrypt and decrypt functions 2020-10-14 16:33:07 +03:00
alesapin
163d33fd21 Add perftest for merges
(cherry picked from commit e655f009d2)
2020-10-13 15:12:28 +03:00
Alexey Milovidov
905bb859a5 Add performance test 2020-10-11 18:25:29 +03:00
Artem Zuikov
4fd1db73a1
Faster 256-bit multiplication (#15418) 2020-09-29 20:52:34 +03:00
Alexander Kuzmenkov
8cbfd69adf
Update decimal_casts.xml 2020-09-24 11:35:35 +03:00
Alexander Kuzmenkov
9b29e31d64 Merge remote-tracking branch 'origin/master' into HEAD 2020-09-23 23:58:43 +03:00
alexey-milovidov
3bc09bb867
Merge pull request #15178 from amosbird/dtcf
Fix performance issue when doing datetime comparison
2020-09-23 20:33:23 +03:00
Alexander Kuzmenkov
9f137e77e7
Update decimal_casts.xml 2020-09-23 18:41:52 +03:00
Alexander Kuzmenkov
d2427a121f
Update decimal_casts.xml 2020-09-23 10:15:51 +03:00
Amos Bird
ee45395942
Fix datetime comparison 2020-09-23 10:29:55 +08:00
Artem Zuikov
f80ffc4dce restore decimal_casts perf test 2020-09-22 18:56:24 +03:00
Alexander Kuzmenkov
4937159f85
Merge pull request #14990 from ClickHouse/aku/longer-perf
If perf test definition changed, run everything + longer (as in master)
2020-09-22 13:25:49 +03:00
Alexander Kuzmenkov
e646aa306f
Update columns_hashing.xml 2020-09-22 13:25:16 +03:00
Alexander Kuzmenkov
ba12e68d70
Update insert_values_with_expressions.xml 2020-09-22 13:23:10 +03:00
Alexander Kuzmenkov
4dae587e2d fix test 2020-09-21 15:02:34 +03:00
Alexander Kuzmenkov
85be1f1685 Merge remote-tracking branch 'origin/master' into HEAD 2020-09-21 14:57:25 +03:00
alexey-milovidov
a4337635cc
Update single_fixed_string_groupby.xml 2020-09-20 19:42:11 +03:00
philip.han
2045bc9f8c Fix performance_test for bloom_filter 2020-09-20 19:06:46 +09:00
Amos Bird
8ca4d6634e
better fixed string group by support 2020-09-20 15:20:09 +08:00
Alexander Kuzmenkov
e7b9319e12 If perf test definition changed, run everything + longer (as in master)
Also some other perf test fixes
2020-09-18 16:28:46 +03:00
philip.han
cbedd44a60 Fix indexOf() to use BloomFilter 2020-09-18 17:43:07 +09:00
Alexander Kuzmenkov
36538ce08f Don't account for short queries, we'll deal with them separately.
New query:

```
WITH ceil(max(q[3]), 1) AS h
SELECT concat('sed -i s\'/^<test.*$/<test max_ignored_relative_change="', toString(h), '">/g\' tests/performance/', test, '.xml') AS s
FROM
(
    SELECT
        test,
        query_index,
        count(*),
        min(event_time),
        max(event_time) AS t,
        arrayMap(x -> floor(x, 3), quantiles(0, 0.5, 0.95, 1)(stat_threshold)) AS q,
        median(stat_threshold) AS m
    FROM perftest.query_metrics
    WHERE (metric = 'client_time') AND (abs(diff) < 0.05) AND (old > 0.2)
    GROUP BY
        test,
        query_index,
        query_display_name
    HAVING (t > '2020-09-01 00:00:00') AND (m > 0.1)
    ORDER BY test DESC
)
GROUP BY test
ORDER BY h DESC
FORMAT PrettySpace
```
2020-09-17 13:00:51 +03:00
Alexander Kuzmenkov
0f8aec59a3 Adjust ignore thresholds for unstable perf tests
Based on historical data.

```
SELECT
    test,
    ceil(max(q[3]), 1) AS h
FROM
(
    SELECT
        test,
        query_index,
        count(*),
        min(event_time),
        max(event_time) AS t,
        arrayMap(x -> floor(x, 3), quantiles(0, 0.5, 0.95, 1)(stat_threshold)) AS q,
        median(stat_threshold) AS m
    FROM perftest.query_metrics
    WHERE (metric = 'client_time') AND (abs(diff) < 0.05)
    GROUP BY
        test,
        query_index,
        query_display_name
    HAVING (t > '2020-09-01 00:00:00') AND (m > 0.1)
    ORDER BY m DESC
)
GROUP BY test
ORDER BY h DESC
FORMAT TSV

cryptographic_hashes	1.3
collations	0.8
joins_in_memory_pmj	0.8
joins_in_memory	0.7
merge_tree_simple_select	0.7
set_index	0.7
decimal_casts	0.7
website	0.6
logical_functions_medium	0.5
count	0.5
merge_tree_many_partitions	0.5
decimal_aggregates	0.5
codecs_int_insert	0.5
column_column_comparison	0.5
insert_parallel	0.4
parse_engine_file	0.4
read_in_order_many_parts	0.4
logical_functions_small	0.4
parallel_insert	0.3
parallel_index	0.3
push_down_limit	0.3
jit_large_requests	0.3
select_format	0.3
arithmetic	0.3
merge_tree_huge_pk	0.3
materialized_view_parallel_insert	0.3
columns_hashing	0.3
if_array_string	0.3
random_string	0.2
random_printable_ascii	0.2
set	0.2
empty_string_serialization	0.2
```

To apply:
```
sed 's/^\(.*\)        \(.*\)$/sed -i "s\/^<test.*$\/<test max_ignored_relative_change="'"'"\2">\/g" tests\/performance\/\1.xml/g' ../bad.tsv | bash
```
2020-09-16 18:27:51 +03:00
Alexander Kuzmenkov
9e3797b252
Merge pull request #14710 from ClickHouse/aku/faster-perf
Do less runs in perf test
2020-09-14 19:17:34 +03:00
Alexander Kuzmenkov
3113aa6cfe Avoid extra error in perf report on broken queries 2020-09-14 10:59:45 +03:00
Alexander Kuzmenkov
c701a15271 fixup 2020-09-14 10:27:46 +03:00
Nikolai Kochetov
c3eeb8f71e
Merge pull request #14412 from ClickHouse/pipes-and-creating-sets
Pipes and creating sets
2020-09-08 10:47:56 +03:00
Alexander Kuzmenkov
0904433c87 Temporarily remove UInt256 perf test broken in https://github.com/ClickHouse/ClickHouse/pull/14229
Also add one more timeout to perf test runner.
2020-09-07 09:33:57 +03:00
Nikolai Kochetov
7a03dbaa2f Added test. 2020-09-04 12:11:37 +03:00
Alexander Kuzmenkov
134aae89f3
Merge pull request #14110 from 4ertus2/decimal
Fix Decimal to Float conversion and toDecimal256() function
2020-08-31 17:08:49 +03:00
Artem Zuikov
13113dafdc
Update decimal_casts.xml 2020-08-27 16:53:16 +03:00
Alexey Milovidov
dc889b5c42 Fixup #14120 2020-08-27 15:17:51 +03:00
alexey-milovidov
ea0122de7e
Merge pull request #14120 from ClickHouse/parallel-insert-more-storages
Support parallel INSERT for more table engines
2020-08-27 15:08:56 +03:00
alexey-milovidov
0f706c01ca
Merge pull request #13888 from vladimir-golovchenko/add-date-trunc-function
Added date_trunc function
2020-08-27 02:12:27 +03:00
Alexey Milovidov
83fa57a346 Update instruction for perf tests 2020-08-26 20:34:05 +03:00
Alexey Milovidov
9a4e7cc492 Add perf test 2020-08-26 20:33:14 +03:00
Artem Zuikov
7e984f877b better test 2020-08-26 19:21:26 +03:00
Artem Zuikov
fe5db661ce fix decimal to float conversion and toDecimal256() 2020-08-26 15:26:12 +03:00
vladimir golovchenko
fb06f59ceb Added tests for date_trunc function. 2020-08-25 16:41:23 -07:00
alexey-milovidov
00c697df06
Merge pull request #14005 from ClickHouse/ucasFL-new-branch
Merging #12195
2020-08-26 01:33:54 +03:00
Alexey Milovidov
86d99f7c7f Disable "optimize_trivial_insert_select" for one perf test 2020-08-26 01:33:31 +03:00
alexey-milovidov
6a164634d7
Merge pull request #12550 from myrrc/bug/low-cardinality-arrays-optimisations
Optimising has(), indexOf(), and countEqual() for Array(LowCardinality(T)) and constant right arguments
2020-08-24 22:31:29 +03:00
Alexander Kazakov
d99ddb9ca9 In perf test array_index_lc: adjusted iterations count 2020-08-24 12:02:48 +03:00
Alexey Milovidov
d44982072c More robust 2020-08-24 03:14:24 +03:00
Alexander Kazakov
48333b29f6 Minior rename 2020-08-24 00:42:00 +03:00
Alexander Kazakov
eac2c59055 Fix up 2020-08-24 00:16:16 +03:00
Alexander Kazakov
ce0ab0e28b Split array_index_low_cardinality performance test: numbers/strings 2020-08-24 00:13:57 +03:00
myrrc
a519559644 updated ColumnVector to compare with other U, cleaned ImplString 2020-08-14 21:08:30 +03:00
myrrc
0b53b0379b fixing perftest to get more relevant result 2020-08-14 03:09:44 +03:00
myrrc
82206e89dd fixed perftest long test and StringRef assert 2020-08-12 17:07:56 +03:00
Alexander Kazakov
6e0d536327
Merge branch 'master' into bug/low-cardinality-arrays-optimisations 2020-08-11 16:25:20 +03:00
Alexander Kuzmenkov
e06fd84ca7
Update synthetic_hardware_benchmark.xml
https://github.com/ClickHouse/ClickHouse/pull/13099#issuecomment-671479904
2020-08-10 20:16:23 +03:00
Alexey Milovidov
d1935bc967 Adjust query time in perf test 2020-08-09 02:14:23 +03:00
myrrc
5a742ec0fc test update 2020-08-08 21:12:38 +03:00
myrrc
c4ddabc327 updated perf test with integral example (cloning the column) 2020-08-06 21:54:21 +03:00
myrrc
c8f737c37c fix: re-resizing, Arcadia compatibility, updated perftest 2020-08-06 15:33:16 +03:00
myrrc
2f589c6856 added another query to perftest 2020-08-04 23:10:54 +03:00
myrrc
f2fecb4d1a re-adding the non-const test queries 2020-08-04 23:00:36 +03:00
myrrc
807f1887f4 some data flow fixes 2020-08-04 22:51:15 +03:00
myrrc
aa1582999a re-adding the test with removed slow requests. 2020-08-04 16:06:16 +03:00
myrrc
d4e747ffd7 deleted extra-slow perftest to figure out outer coverage 2020-08-04 16:04:28 +03:00
myrrc
e270706fdb Merge branch 'bug/low-cardinality-arrays-optimisations' of github:myrrc/clickhouse into bug/low-cardinality-arrays-optimisations 2020-08-04 15:37:22 +03:00
myrrc
98119af9de Merge remote-tracking branch 'upstream/master' into bug/low-cardinality-arrays-optimisations 2020-08-04 15:01:27 +03:00
Alexander Kazakov
059a3dcbd9 Fix up typo in test 2020-08-03 23:35:10 +03:00
Alexey Milovidov
006f9a831f More performance tests 2020-08-03 04:37:57 +03:00
alexey-milovidov
935533e841
Merge pull request #13193 from ClickHouse/aku/spurious-perf
Fix some spurious perf test failures
2020-08-03 01:02:53 +03:00
alexey-milovidov
1f19d7e9bd
Merge pull request #13016 from ClickHouse/limit-push-down
Limit push down
2020-08-02 01:23:06 +03:00
alexey-milovidov
626b709085
Merge pull request #13055 from ClickHouse/fixed-hash-map-implicit-zero
Slightly improve the inner loop of aggregation by UInt8/UInt16 keys
2020-08-02 01:09:00 +03:00
Alexey Milovidov
72bb5af15c More perf tests 2020-07-31 21:19:54 +03:00
Alexander Kuzmenkov
1d954581eb Fix some spurious perf test failures 2020-07-31 19:28:04 +03:00
alexey-milovidov
543d2da3da
Merge pull request #12700 from CurtizJ/fix-performance-with-in
Fix performance with large tuples
2020-07-31 14:40:38 +03:00
Alexey Milovidov
b6ac349126 Added another test case for perf test 2020-07-30 05:43:22 +03:00
Nikolai Kochetov
1c5ccd089f Added perftest. 2020-07-29 15:32:59 +03:00
myrrc
da9502e496 Merge remote-tracking branch 'upstream/master' into bug/low-cardinality-arrays-optimisations 2020-07-28 15:06:39 +03:00
Nikolai Kochetov
1f2390b65e
Merge pull request #12840 from ClickHouse/remove-wrong-threads-limiting
Remove thread limiting.
2020-07-28 12:03:07 +03:00
Nikolai Kochetov
abdd160db1
Merge pull request #12589 from bobrik/ivan/parallel-ranges
Parallelize PK range and skipping index stages
2020-07-27 19:23:00 +03:00
Nikolai Kochetov
be2132d437 Added perftest. 2020-07-27 17:27:26 +03:00
Nikolai Kochetov
05ec37d1c9 Added perftest. 2020-07-27 17:25:26 +03:00
Anton Popov
70022c7a6c
Mark query short 2020-07-24 16:59:50 +03:00
Anton Popov
61018c275d Merge remote-tracking branch 'upstream/master' into polymorphic-parts-4 2020-07-23 18:38:29 +03:00
Anton Popov
1d0493cfcc better code in ReaderCompact and perf test added 2020-07-23 18:37:44 +03:00
Nikolai Kochetov
dad9d369a1 Merge branch 'master' into bobrik-parallel-randes 2020-07-23 16:21:32 +03:00
Anton Popov
fc515728f4 fix performance with large tuples, which are interpreted as functions in IN section 2020-07-23 02:33:14 +03:00
myrrc
6af63213ce Merge remote-tracking branch 'upstream/master' into bug/low-cardinality-arrays-optimisations 2020-07-22 22:49:46 +03:00
alexey-milovidov
49adaf95aa
Merge pull request #12585 from ClickHouse/mod-compatible
Make behaviour of remainder of division for floating point numbers compatible with most of DBMS
2020-07-22 08:43:45 +03:00
Nikolai Kochetov
da4a66b443 Added perftest. 2020-07-21 19:28:30 +03:00
Alexey Milovidov
16ba0f91d7 Add performance test 2020-07-21 13:38:14 +03:00
Artem Zuikov
df1213f351
Try to make if_transform_strings perf test faster (#12598) 2020-07-21 13:34:55 +03:00
Alexander Kuzmenkov
da5853f031
Update decimal_aggregates.xml 2020-07-17 18:27:05 +03:00
Alexander Kuzmenkov
e095d04553
Update read_in_order_many_parts.xml 2020-07-17 15:12:49 +03:00
Alexander Kuzmenkov
c306476f1b
Update redundant_functions_in_order_by.xml 2020-07-17 15:11:17 +03:00
Artem Zuikov
c6e3e9b1ff
String convertion to enum in if() and transform() (#12515) 2020-07-17 12:28:51 +03:00
Alexander Kuzmenkov
b558aae851 disable profiler 2020-07-15 19:10:47 +03:00
Alexander Kuzmenkov
fda2595958 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-15 19:09:43 +03:00
Artem Zuikov
5f01a587f6
Optimize monotonous ORDER BY (#12467) 2020-07-15 13:10:21 +03:00
Alexander Kuzmenkov
fa6fc719ab fixup 2020-07-14 15:55:25 +03:00
Alexander Kuzmenkov
d05b984959 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-14 15:50:31 +03:00
alexey-milovidov
969d9c0d47
Merge branch 'master' into remove-useless-setting-from-perf-test 2020-07-14 09:31:34 +03:00
Alexander Kuzmenkov
4da37fb3a0 Fix decimal aggregate perf test 2020-07-13 14:40:46 +03:00
Alexander Kuzmenkov
5a15e03400 Improve data layout in perf test
Also fix a bad test query found with new data.
2020-07-13 14:31:15 +03:00
Alexander Kuzmenkov
c13425f0e9 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-13 14:29:10 +03:00
Alexander Kuzmenkov
d911aa7496 temporarily disable decimal aggregates test 2020-07-13 14:28:35 +03:00
Artem Zuikov
b5c181c24e
Optimize redundant functions in ORDER BY (#12404) 2020-07-13 13:34:25 +03:00
Alexey Milovidov
4ace6f99fa Remove useless setting from performance tests #12424 2020-07-13 05:34:45 +03:00
Alexander Kuzmenkov
48862a1edf fixup 2020-07-11 13:35:44 +03:00
Alexander Kuzmenkov
920c62b629 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-11 13:32:31 +03:00
Artem Zuikov
6b26842ce5
RIGHT and FULL JOIN for MergeJoin (#12118) 2020-07-10 21:10:06 +03:00
Alexander Kuzmenkov
70ba4e16f8 fixup 2020-07-10 17:18:39 +03:00
Alexander Kuzmenkov
cd09cf8c0b Merge remote-tracking branch 'origin/master' into HEAD 2020-07-10 17:18:24 +03:00
Artem Zuikov
01b5c2663c
Delete injective functions inside uniq (#12337) 2020-07-10 13:42:41 +03:00
Alexander Kuzmenkov
394cf20673 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-09 13:46:54 +03:00
Alexander Kuzmenkov
3b428d6585 fixes 2020-07-09 13:46:16 +03:00
Alexander Kuzmenkov
5b3a106f7a fixes 2020-07-08 13:35:49 +03:00
alesapin
84f8bf1e14
Merge pull request #10697 from CurtizJ/polymorphic-parts
Polymorphic parts (in-memory format)
2020-07-08 13:25:45 +03:00
Alexander Kuzmenkov
41addf9173 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-08 13:22:06 +03:00
Alexander Kuzmenkov
68189db872
Update int_parsing.xml 2020-07-08 12:40:33 +03:00
Alexander Kuzmenkov
d85391f3ac too slow 2020-07-07 20:07:04 +03:00
Alexander Kuzmenkov
c49da9053d Merge remote-tracking branch 'origin/master' into HEAD 2020-07-07 20:06:27 +03:00
Alexander Kuzmenkov
62b83b412f split 2020-07-07 11:01:50 +03:00
alesapin
3f90f65678 Merge remote-tracking branch 'origin' into CurtizJ-polymorphic-parts 2020-07-06 21:34:17 +03:00
Nikita Mikhaylov
53522c728b
Merge pull request #11662 from Avogar/orc_output_format
Add ORCBlockOutputFormat
2020-07-06 18:51:35 +04:00
Alexander Kuzmenkov
c1fa01cabf
Update questdb_sum_double.xml 2020-07-06 13:58:47 +03:00
Alexander Kuzmenkov
2fbd3cb6a4 fixup 2020-07-06 13:51:26 +03:00
Alexander Kuzmenkov
432cadae15 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-06 11:44:09 +03:00
Alexander Kuzmenkov
9f32615d32
Merge pull request #11606 from CurtizJ/polymorphic-parts-2
Make polymorphic_parts perf test not blazing fast
2020-07-06 11:43:30 +03:00
Alexey Milovidov
91ca33de86 Merge branch 'perf-test-questdb' of github.com:yandex/ClickHouse into perf-test-questdb 2020-07-04 23:35:15 +03:00
Alexey Milovidov
43218567fa Merge branch 'master' into perf-test-questdb 2020-07-04 23:34:51 +03:00
Anton Popov
73676f5022
Improve performace of reading in order of sorting key. (#11696)
* simplify reading in order of sorting key

* add perf test for reading many parts

* Revert "simplify reading in order of sorting key"

This reverts commit 7267d7c46e.

* add threshold for preliminary merge for reading in order

* better threshold

* limit threads in test
2020-07-04 15:48:51 +03:00
Nikita Mikhaylov
5c97106ea3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into Avogar_orc_output_format 2020-07-03 18:00:28 +03:00
Alexander Kuzmenkov
e27e509667
Update polymorphic_parts_l.xml 2020-07-03 11:46:48 +03:00
Alexander Kuzmenkov
02ee988d80
Update polymorphic_parts_m.xml 2020-07-03 11:46:29 +03:00
Alexander Kuzmenkov
1ffc4ad88a
Update polymorphic_parts_s.xml 2020-07-03 11:46:10 +03:00
Alexander Kuzmenkov
10b6f98ca6 fixup 2020-07-03 11:39:43 +03:00
Alexander Kuzmenkov
7d8a31aff3 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-03 10:09:20 +03:00
Alexander Kuzmenkov
03becdbae4
Update polymorphic_parts_s.xml 2020-07-03 09:12:15 +03:00
Alexander Kuzmenkov
d1a93523b6
Update polymorphic_parts_m.xml 2020-07-03 09:11:46 +03:00
Alexander Kuzmenkov
68ca587a71
Update polymorphic_parts_l.xml 2020-07-03 09:11:17 +03:00
Anton Popov
4422df2e37 Merge remote-tracking branch 'upstream/master' into HEAD 2020-07-02 20:18:21 +03:00
Anton Popov
17d49af183 fix test with buffer table 2020-07-02 20:12:36 +03:00
Alexander Kuzmenkov
a2b8412b41 fixup 2020-07-02 10:44:16 +03:00
Alexander Kuzmenkov
f50f8d936e Merge remote-tracking branch 'origin/master' into HEAD 2020-07-02 10:19:26 +03:00
Alexander Kuzmenkov
41088c3416 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-02 10:13:11 +03:00
Alexander Kuzmenkov
99514a7e24 fixup 2020-07-02 10:13:02 +03:00
Alexander Kuzmenkov
f4869eca41
Update cpu_synthetic.xml 2020-06-30 15:14:18 +03:00
Alexander Kuzmenkov
7e2f6937e0 fixup 2020-06-30 12:45:19 +03:00
Alexander Kuzmenkov
b5f9e9de78 Merge remote-tracking branch 'origin/master' into HEAD 2020-06-30 12:28:14 +03:00
Alexander Kuzmenkov
a4152cfb00 Merge remote-tracking branch 'origin/master' into HEAD 2020-06-30 08:55:56 +03:00
Alexander Kuzmenkov
bfa9a8efb2 fixup 2020-06-30 08:55:37 +03:00
Nikita Mikhaylov
33e627777b Merge branch 'master' of github.com:ClickHouse/ClickHouse into Avogar_orc_output_format 2020-06-29 17:25:33 +03:00
Alexander Kuzmenkov
6d67c77e68 fixup 2020-06-29 17:09:51 +03:00
Alexander Kuzmenkov
de95d42de6 make a short test for DateTime 2020-06-29 10:58:50 +03:00
alexey-milovidov
c6f5a4e074
Update questdb_sum_double.xml 2020-06-27 19:17:03 +03:00
Alexander Kuzmenkov
87c81e9ad2 longer 2020-06-27 03:59:00 +03:00
Alexander Kuzmenkov
e58e1c5471 Try to enable long perf tests 2020-06-27 00:02:13 +03:00
Avogar
81d459a5a8 Small update ORCBlockOutputFormat and add ORC output format to performance test. 2020-06-26 23:17:28 +03:00
Alexander Kuzmenkov
5d324751a0 fix more queries 2020-06-26 22:11:43 +03:00
Alexander Kuzmenkov
c8b633fe47 Merge remote-tracking branch 'origin/master' into HEAD 2020-06-26 21:01:40 +03:00
alexey-milovidov
2efca8e397
Update questdb_sum_double.xml 2020-06-26 20:54:50 +03:00
alesapin
e9c47dc89c Merge branch 'master' into CurtizJ-polymorphic-parts 2020-06-26 14:27:19 +03:00
alexey-milovidov
3843661981
Update questdb_sum_double.xml 2020-06-26 12:43:19 +03:00
alexey-milovidov
23a3a86426
Update questdb_sum_double.xml 2020-06-25 23:43:03 +03:00
Alexey Milovidov
bfc8ddbd5c Added performance test to resemble questionable benchmark 2020-06-25 23:40:07 +03:00
Anton Popov
14e09e5650 Merge remote-tracking branch 'upstream/master' into HEAD 2020-06-25 14:59:15 +03:00
Alexander Kuzmenkov
bfba15d512 Explicitly mark short perftest queries 2020-06-23 15:09:54 +03:00
Alexander Kuzmenkov
385d7d6674
Update polymorphic_parts_s.xml 2020-06-23 10:00:05 +03:00
Alexander Kuzmenkov
61cf7d02d5
Update polymorphic_parts_m.xml 2020-06-23 09:59:29 +03:00
Alexander Kuzmenkov
e9eb2514d6
Update polymorphic_parts_l.xml 2020-06-23 09:58:31 +03:00
Artem Zuikov
1daabb2925 minor test fix 2020-06-22 19:56:35 +03:00
Anton Popov
2f95daf4ed more optimal aggregate functions with both 'if' and 'distinct' combinators 2020-06-22 17:57:30 +03:00
Anton Popov
3f0f0ff5ad add perf test for distinct combinator 2020-06-22 16:31:37 +03:00
alexey-milovidov
746676d762
Update general_purpose_hashes.xml 2020-06-21 21:32:12 +03:00
alexey-milovidov
d4fce9d0f3
Update great_circle_dist.xml 2020-06-21 21:31:58 +03:00
alexey-milovidov
045553d00f
Update synthetic_hardware_benchmark.xml 2020-06-21 21:31:31 +03:00
alexey-milovidov
dcda561815
Update synthetic_hardware_benchmark.xml 2020-06-21 21:31:19 +03:00
alexey-milovidov
c3a5c32a48
Merge branch 'master' into arch-chooser 2020-06-21 21:30:06 +03:00
Nikolai Kochetov
12026ab9c2 Rename test file. 2020-06-19 13:24:17 +03:00
Nikolai Kochetov
690b88d331
Merge pull request #11291 from Provet/master
Partial sorting optimization fix
2020-06-19 12:58:44 +03:00
Ruslan
23cd919681
Any input optimize (#11529)
* workig commit №1

* working commit №2

* first version of ArithmeticOperationsInAgrFuncOptimize (min, max, sum)

* fix conflicts

* fix №2

* attempt №3

* Description + little mistake

* fix style + expanded optimization + add performance test

* style + merge with new version

* one more branch

* bug fix + merge

* just merge

* some code style and logic fixes

* little changes

* tests

* test(2)

* style

* seems to have fixed everything that was required

* bug fix

* style

* build fix

* fix typo

* fix clang-10 warning

* fix clang-10 warning

* nolint

* add new files to ya.make

* extract all functions from Any

* add last new lines.

* extract all functions from Any

* add last new lines.

* build + requested changes

* code style

* build + little perf test + anyLast

* minor changes

* bug fix

* minor changes

* another minor changes

* fix + experement

* experement 2.0

* experement 3.0

* experement 3.1

* experement 4.0

* last experement(think so)

* just another attempt to fix UB

* minor changes

* I think I won

* ya.make

* fix requested changes

Co-authored-by: Artem Zuikov <chertus@gmail.com>
2020-06-18 19:34:29 +03:00
Nikolai Kochetov
38bebfb20d Merge remote-tracking branch 'origin' into provert-master 2020-06-17 22:05:45 +03:00
Nikolai Kochetov
091995f1db Updated perftests. 2020-06-17 13:00:28 +03:00
xPoSx
2d8110ce45
Eliminate min/max/any aggregators of GROUP BY keys in SELECT section (#11667) 2020-06-17 12:32:43 +03:00
Alexander Kuzmenkov
9cab822718
Merge pull request #11683 from ClickHouse/aku/merge-minmap-11603
Merge minMap, maxMap (#11603)
2020-06-17 00:02:33 +03:00
Artem Zuikov
857d84bf7c
Buffer left blocks optimisation for MergeJoin (#10601) 2020-06-16 23:13:18 +03:00
xPoSx
9fc37e1f75
Remove functions of other keys in GROUP BY section (#10051) 2020-06-16 12:14:25 +03:00
Anton Popov
b19d48a11c Merge remote-tracking branch 'upstream/master' into HEAD 2020-06-16 06:37:55 +03:00
Alexander Kuzmenkov
7cc54fd4f1 renames + perf test 2020-06-15 21:53:54 +03:00
Mikhail Malafeev
10566e2b43
Remove duplicate ORDER BY and DISTINCT from subqueries (#10067) 2020-06-15 15:03:01 +03:00
alexey-milovidov
13ceaa4779
Merge pull request #11554 from Enmk/extractAllGroups-Horizontal-and-Vertical
extractAllGroupsHorizontal and extractAllGroupsVertical
2020-06-12 16:56:14 +03:00
Anton Popov
7fb8a98528 increase number of rows 2020-06-12 00:33:30 +03:00
Anton Popov
ebc781a0c8 make polymorphic_parts perf test not blazing fast 2020-06-11 19:14:22 +03:00
Vasily Nemkov
50a184acac extractAllGroupsHorizontal and extractAllGroupsVertical
Split tests, fixed some error messages

Fixed test and error reporting of extractGroups
2020-06-11 11:03:17 +03:00
alexey-milovidov
4f2ed7a5e3
Merge branch 'master' into arch-chooser 2020-06-10 22:31:19 +03:00
Alexander Kuzmenkov
af1c8f8a5f fixup 2020-06-10 02:57:31 +03:00
Alexander Kuzmenkov
898d83491d fixup 2020-06-10 02:37:48 +03:00
Alexey Milovidov
c95d5786fd Fixup 2020-06-10 01:52:25 +03:00
Alexander Kuzmenkov
5719549ee1 fixup 2020-06-09 16:29:07 +03:00
Alexander Kuzmenkov
6071b9b9d9 Merge remote-tracking branch 'origin/master' into HEAD 2020-06-09 15:06:32 +03:00
alexey-milovidov
85363ebe56
Update aggregating_merge_tree.xml 2020-06-09 01:41:57 +03:00
Alexander Kuzmenkov
6a3df071a2 fixup 2020-06-09 01:13:08 +03:00
Alexey Milovidov
f9ea964c87 Added performance test 2020-06-09 00:59:58 +03:00
Alexander Kuzmenkov
e2e5e1d5f8 More stable perf tests 2020-06-08 16:57:33 +03:00
Anton Popov
5c42408add
Merge pull request #9113 from dimarub2000/group_by_in_order_optimization
[WIP] Optimization of GROUP BY with respect to table sorting key.
2020-06-06 14:25:59 +03:00
Albert Kidrachev
61cf6fe711 add perftest 2020-06-05 06:08:37 +03:00
Albert Kidrachev
60b86f524f add perf-test and fix 2020-06-04 20:10:52 +03:00
Anton Popov
1ce09e1faa Merge remote-tracking branch 'upstream/master' into polymorphic-parts 2020-06-03 16:27:54 +03:00
Ruslan
7757c9ec57
Remove arithmetic operations from aggregation functions (#10047) 2020-06-03 13:28:30 +03:00
Alexander Kuzmenkov
bd84c3be18
Merge branch 'master' into fuzz_data 2020-06-02 12:49:43 +03:00
alexey-milovidov
92ac608447
Merge pull request #11330 from ClickHouse/fix-10241
More parallel MV processing.
2020-06-02 04:06:39 +03:00
Albert Kidrachev
3c6a785937 Merge branch 'master' of https://github.com/Provet/ClickHouse 2020-06-01 15:12:37 +03:00
Albert Kidrachev
02611de051 merge 2020-06-01 15:11:36 +03:00
Nikolai Kochetov
c6c1cd1009 Update test. 2020-06-01 12:12:03 +03:00
Nikolai Kochetov
f1796b73ff Add perftest. 2020-06-01 11:52:35 +03:00
Dmitry
8ba17ee272 better test 2020-05-31 03:00:16 +03:00
Dmitry
4b0d32f026 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-31 00:21:02 +03:00
Nikolai Kochetov
f9009809f2
Delete order_by_with_limit.xml
Delete new test to run old ones.
2020-05-30 12:22:05 +03:00
Dmitrii Kovalkov
0f730b2ace multitarget great circle 2020-05-29 07:35:39 +02:00
Dmitrii Kovalkov
4c16f7a70f cosmetics 2020-05-29 07:35:39 +02:00
Dmitrii Kovalkov
cdb353856d remove vectorization from binary arithmetic 2020-05-29 07:35:04 +02:00
Dmitrii Kovalkov
71fabcedc4 Fix test 2020-05-29 07:35:03 +02:00
Dmitrii Kovalkov
ef030349ff Add hashes to multitarget code, a lot of cosmetics 2020-05-29 07:35:03 +02:00
Dmitrii Kovalkov
4a9891c601 use vectorized rand in generateUUIDv4, add multitarget build in intHash32/64 2020-05-29 07:35:03 +02:00
Dmitrii Kovalkov
8483dfa272 Delete needless rand implementations 2020-05-29 07:35:03 +02:00
Dmitrii Kovalkov
2609b1c370 Save test 2020-05-29 07:34:10 +02:00
Dmitrii Kovalkov
808bb14c5c Add xorshift-rand from lemire for comparing performance 2020-05-29 07:33:37 +02:00
Dmitrii Kovalkov
acbd3b3a70 Compile rand function with AVX 2020-05-29 07:31:59 +02:00
Alexey Milovidov
b7b6380141 Add performance test 2020-05-29 05:30:49 +03:00
Andrei Nekrashevich
af469c0da5 fix issues 2020-05-29 05:06:21 +03:00
Albert Kidrachev
c91ca6fd4b fix optimization, add comments and performance test 2020-05-29 04:50:02 +03:00
Dmitry
871539e4cc test and ya.make 2020-05-28 20:30:21 +03:00
Alexander Kuzmenkov
ca5d7cf5bb fix some broken performance tests 2020-05-28 10:45:03 +03:00
Andrei Nekrashevich
3c68ecacfe Added function fuzzBits 2020-05-28 05:51:37 +03:00
alexey-milovidov
35b1fd43fa
Merge pull request #11212 from ClickHouse/fix-perf-test
Fix error in performance test
2020-05-27 13:33:19 +03:00
Alexander Kuzmenkov
cd008f9b7f
Update functions_with_hash_tables.xml 2020-05-27 10:58:29 +03:00
Alexey Milovidov
90f49c0c17 Fix error in performance test 2020-05-26 21:37:20 +03:00
Alexander Kuzmenkov
27df8d5aca fixup 2020-05-26 09:06:46 +03:00
Alexander Kuzmenkov
7142e68d7a fixup 2020-05-26 08:54:04 +03:00
Alexander Kuzmenkov
8d5bbed2de Merge remote-tracking branch 'origin/master' into HEAD 2020-05-25 06:44:11 +03:00
alexey-milovidov
a55a8ba69f
Merge pull request #11158 from ClickHouse/normalize-performance-tests-2
Normalize performance tests 2
2020-05-25 03:49:58 +03:00
Alexey Milovidov
8fa637cf73 Fix long tests, part 2 2020-05-24 23:35:49 +03:00
Alexey Milovidov
02500e4b11 Fix long tests 2020-05-24 23:33:13 +03:00
Alexey Milovidov
a70b819ca2 Adapted test time 2020-05-24 02:33:47 +03:00
alexey-milovidov
17df54785e
Update merge_table_streams.xml 2020-05-23 23:45:28 +03:00
alexey-milovidov
08c7277d2b
Merge pull request #11144 from ClickHouse/dont-validate-polygons-in-non-constant-case
Fix performance issue with pointInPolygon
2020-05-23 18:54:50 +03:00
Alexey Milovidov
3b29ea09c1 Fix another strange test 2020-05-23 17:56:35 +03:00
Alexey Milovidov
2c76899f53 Edited strange test 2020-05-23 17:53:54 +03:00
Alexey Milovidov
9d1d7c61d6 Replace hits_1000m to hits_100m 2020-05-23 17:52:32 +03:00
Alexey Milovidov
d019474641 Replace hits_1000m to hits_100m 2020-05-23 17:51:18 +03:00
Alexey Milovidov
1dc284d9f1 Add performance test #10981 2020-05-23 17:41:13 +03:00
Alexey Milovidov
c79ded1653 Added performance test for point in const polygon 2020-05-22 22:03:30 +03:00
Alexey Milovidov
490a94105e Two more zeros in performance test 2020-05-22 16:51:04 +03:00
Alexey Milovidov
e60fee3f3d Add performance test for non-constant polygons 2020-05-22 14:46:54 +03:00
Alexey Milovidov
1a5fa0e6fa Remove totally wrong performance test #10623 2020-05-22 14:39:21 +03:00
alexey-milovidov
b480f137f3
Merge pull request #10623 from livace/master
Add point in polygon for non-const polygons
2020-05-21 15:18:24 +03:00
Alexey Milovidov
6669607e17 Merge branch 'master' into vectorize-sum 2020-05-21 15:02:51 +03:00
alexey-milovidov
8c2b7c31a5
Fix too long perf test. 2020-05-21 09:49:15 +03:00
Alexander Kuzmenkov
09ea7253cd performance test for some functions that use hash tables 2020-05-20 22:37:14 +03:00
Alexander Kuzmenkov
c3d9bb829b performance comparison 2020-05-19 18:28:03 +03:00
Alexey Milovidov
db422434ff Add performance test 2020-05-18 07:55:06 +03:00
alexey-milovidov
7efd2a825f
Merge pull request #10463 from ClickHouse/parallel-final
Parallel final
2020-05-18 05:20:18 +03:00
Andrei Nekrashevich
eba043aaf8 Merge branch 'master' into random_string_utf8_function 2020-05-17 22:53:42 +03:00
Alexey Milovidov
157d0b612c Merge branch 'random_fixed_string_function' of https://github.com/xolm/ClickHouse into xolm-random_fixed_string_function 2020-05-17 10:54:48 +03:00
Andrei Nekrashevich
675509ed23 Added function randomStringUTF8 2020-05-17 07:43:53 +03:00
Alexey Milovidov
7e24492fb9 Added performance test 2020-05-17 06:16:34 +03:00
Anton Popov
e8262ccaf6 in-memory parts: add perf test 2020-05-15 03:56:44 +03:00
Andrei Nekrashevich
1d52c1b9d0 Added function randomFixedString 2020-05-13 01:48:55 +03:00
alexey-milovidov
40ef113503
Merge branch 'master' into parallel-final 2020-05-11 06:47:44 +03:00
Alexey Milovidov
2438a510b0 Merge branch 'fuzz_functions' of https://github.com/xolm/ClickHouse into xolm-fuzz_functions 2020-05-11 02:42:55 +03:00
Alexey Milovidov
9654532940 Adjust thresholds in perf test 2020-05-11 00:31:01 +03:00
Andrei Nekrashevich
c3873495c9 fix and performance test template 2020-05-10 22:07:02 +03:00
Alexey Milovidov
978d38f34a Raised threshold for one of tests 2020-05-09 04:42:52 +03:00
Alexey Milovidov
666fd29d82 Fix performance test errors 2020-05-09 04:28:31 +03:00
alexey-milovidov
3e583ef45e
Merge pull request #10712 from ClickHouse/merging-extract-groups
Merging #10534
2020-05-07 15:34:41 +03:00
Alexey Milovidov
15c1c96a4b Added performance test 2020-05-07 03:59:04 +03:00
Alexey Milovidov
665c9270ca Fix error in performance test 2020-05-07 00:57:26 +03:00
Alexey Milovidov
6e2505210f Mark some performance tests with very high variance 2020-05-07 00:53:25 +03:00
Alexey Milovidov
786e08c401 Mark some performance tests with very high variance 2020-05-07 00:52:03 +03:00
Anton Popov
2be5c7c420
Merge pull request #10704 from CurtizJ/polymorphic-parts-2
Add perf test for different part types
2020-05-06 22:33:16 +03:00
Nikolai Kochetov
ff8014fa39 Try fix perftest. 2020-05-06 21:53:40 +03:00
Anton Popov
80cf8881da add perf test for different part types 2020-05-06 18:52:56 +03:00
DoomzD
26424ec388 Add perftest for pointInPolygon 2020-05-05 17:46:23 +03:00
Nikolai Kochetov
22d3bba59a Added perftest. 2020-05-04 21:14:31 +03:00
Alexander Kuzmenkov
f4aac4322c Add performance test for sumMap(Tuple). 2020-05-01 10:48:45 +03:00
Nikolai Kochetov
047a78046e Try make tests faster. 2020-04-30 20:58:09 +03:00
Nikolai Kochetov
19dadb8c2d Add parallel final. 2020-04-30 12:59:08 +03:00
Alexander Kuzmenkov
a3843e1f42 performance comparison 2020-04-30 08:10:34 +03:00
Alexander Kuzmenkov
d952e5aba2 performance comparison 2020-04-29 20:26:28 +03:00
Alexander Kuzmenkov
15e94d3de9 performance comparison 2020-04-29 09:30:37 +03:00