thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-11 08:52:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
154,353 Commits 9,280 Branches 1,782 Tags 2.1 GiB
bed8a9ae07
Commit Graph

27 Commits

Author SHA1 Message Date
Robert Schulze
20b25566f5
Remove superfluous --multiquery/-n 2024-09-16 11:30:56 +00:00
Alexander Tokmakov
d2ee805d8f
Revert "Remove obsolete --multiquery parameter (follow-up to #63898), pt. V" 2024-09-09 17:32:22 +02:00
Robert Schulze
5362b117bf
Remove superfluous --multiquery/-n 2024-09-07 16:20:46 +00:00
vdimir
c884a2fb9d
perf tests set cgroups_memory_usage_observer_wait_time to zero 2024-08-27 13:26:20 +00:00
Azat Khuzhin
9ce55b69b4 Fix possible CANNOT_READ_ALL_DATA during server startup in performance tests 
CI [1]:

    2024.08.04 22:09:11.646800 [ 1052 ] {} <Error> Application: Code: 33. DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4.: While checking access for disk backups. (CANNOT_READ_ALL_DATA), Stack trace (when copying this message, always include the lines below):

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/64955/6702acf6f2e4a0ee9697066e38006631fc7f69df/performance_comparison__aarch64__[2_4].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-05 11:23:53 +02:00
Nikita Taranov
3dc73ad677
Merge pull request #65320 from ClickHouse/nickitat-patch-20 
Fix some perf tests
 2024-08-02 16:28:45 +00:00
Nikita Taranov
21aa514c80 don't run removed tests 2024-07-30 15:03:09 +01:00
robot-clickhouse
321766d0b8 Automatic style fix 2024-07-25 22:51:22 +00:00
Nikita Taranov
92f378a121
Merge branch 'master' into nickitat-patch-20 2024-07-24 23:22:55 +02:00
Robert Schulze
67f5ffc592
Fixups 2024-07-16 20:21:21 +00:00
Antonio Andelic
f49c194ad0 Fix server restarts in performance tests 2024-06-26 13:48:35 +02:00
Nikita Taranov
b125f8166f impl 2024-06-19 16:41:50 +01:00
Nikita Taranov
57d036e589 impl 2024-06-15 22:04:24 +01:00
Dmitry Novik
5d7882bb60
Always use new analyzer in perf tests 2024-04-11 16:50:01 +02:00
Maksim Kita
35e1e5aff7 Perf script update path in documentation 2024-04-09 14:42:29 +03:00
Dmitry Novik
8bd49aa303 Try to fix perf tests 2024-03-25 17:06:42 +01:00
Azat Khuzhin
a4f765cae7
Improve performance of SELECTs with active mutations (#59531) 
* Configure keeper for perf tests

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Improve performance of SELECTs with active mutations

getAlterMutationCommandsForPart() can be a hot path for query execution
when there are pending mutations.

- LOG_TEST - it is not only check one bool, but actually a bunch of
  atomics as well.

- Return std::vector over std::map (map is not required there) - no
  changes in performance.

- Copy only RENAME_COLUMN (since only this mutation is required by
  AlterConversions).

And here are results:

run|result
-|-
SELECT w/o ALTER|queries: 1565, QPS: 355.259, RPS: 355.259
SELECT w/ ALTER unpatched|queries: 2099, QPS: 220.623, RPS: 220.623
SELECT w/ ALTER and w/o LOG_TEST|queries: 2730, QPS: 235.859, RPS: 235.859
SELECT w/ ALTER and w/o LOG_TEST and w/ RENAME_COLUMN only|queries: 2995, QPS: 290.982, RPS: 290.982

But there are still room for improvements, at least MergeTree engines
could implement getStorageSnapshotForQuery().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Add AlterConversions::supportsMutationCommandType(), flatten vector<vector<MutationCommand>>

* Work around what appears to be a clang static analysis bug

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Michael Kolupaev <michael.kolupaev@clickhouse.com>
 2024-02-22 08:51:10 +00:00
Raúl Marín
cda39e64e4 Perf: Only consider XML files 2024-01-29 17:47:50 +01:00
Raúl Marín
6a2fcb778f Restore comment 2024-01-29 13:07:30 +01:00
Raúl Marín
c79a151cca Simplify query_run_metric_arrays in perf tests 2024-01-29 13:00:49 +01:00
Azat Khuzhin
44e42052b1 Fix perf tests after sumMap starts to filter out -0. 
Before perf tests was relying on the following:

    SELECT sumMap(['foo', 'bar'], [-0., -0.])

    ┌─sumMap(['foo', 'bar'], [-0., -0.])─┐
    │ (['bar','foo'],[-0,-0])            │
    └────────────────────────────────────┘

While it got changed, and now:

    ┌─sumMap(['foo', 'bar'], [-0., -0.])─┐
    │ ([],[])                            │
    └────────────────────────────────────┘

But it works for nan:

    SELECT sumMap(['foo', 'bar'], [nan, nan])

    ┌─sumMap(['foo', 'bar'], [nan, nan])─┐
    │ (['bar','foo'],[nan,nan])          │
    └────────────────────────────────────┘

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-27 13:14:07 +01:00
Alexey Milovidov
1afcab35c1 Fix supply chain attack in performance tests 2024-01-14 08:25:12 +01:00
Azat Khuzhin
3d88dba0a7 Fix perf tests duration (checks.test_duration_ms) 
The column in the source was seconds in Float32, we need to convert it to
milliseconds in UInt64.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-10 17:03:53 +03:00
Azat Khuzhin
0d71bf1411 Upload time of the perf tests into artifacts as test_duration_ms 
Now perf test changes/failures will have two rows, row for new and row
for old server.

I thought about uploading only the time of the test on the new server,
but because not all perf tests uploaded, you cannot always get the time
of the test without the changes (i.e. from run on the upstream/master
repo/branch).

<details>

Before:

```sql
SELECT
    concat(test, ' #', toString(query_index)),
    'slower' AS test_status,
    0 AS test_duration_ms,
    concat('https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.', test, '.', toString(query_index)) AS report_url
FROM queries
WHERE (changed_fail != 0) AND (diff > 0)
UNION ALL
SELECT
    concat(test, ' #', toString(query_index)),
    'unstable' AS test_status,
    0 AS test_duration_ms,
    concat('https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#unstable-queries.', test, '.', toString(query_index)) AS report_url
FROM queries
WHERE unstable_fail != 0

Query id: 49dfdc9a-f549-4499-9a1a-410e5053f6c1

┌─concat(test, ' #', toString(query_index))─┬─test_status─┬─test_duration_ms─┬─report_url─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ hashed_array_dictionary #16               │ slower      │                0 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.hashed_array_dictionary.16 │
│ ngram_distance #2                         │ slower      │                0 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.2           │
│ ngram_distance #3                         │ slower      │                0 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.3           │
│ ngram_distance #4                         │ slower      │                0 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.4           │
└───────────────────────────────────────────┴─────────────┴──────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```

After:

```sql
SELECT
    concat(test, ' #', toString(query_index), '::', test_desc_.1) AS test_name,
    'slower' AS test_status,
    test_desc_.2 AS test_duration_ms,
    concat('https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.', test, '.', toString(query_index)) AS report_url
FROM queries
ARRAY JOIN map('old', left, 'new', right) AS test_desc_
WHERE (changed_fail != 0) AND (diff > 0)
UNION ALL
SELECT
    concat(test, ' #', toString(query_index), '::', test_desc_.1) AS test_name,
    'unstable' AS test_status,
    test_desc_.2 AS test_duration_ms,
    concat('https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#unstable-queries.', test, '.', toString(query_index)) AS report_url
FROM queries
ARRAY JOIN map('old', left, 'new', right) AS test_desc_
WHERE unstable_fail != 0

Query id: 20475bfd-754b-4159-aa16-7798f4720bf8

┌─test_name────────────────────────┬─test_status─┬─test_duration_ms─┬─report_url─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ hashed_array_dictionary #16::old │ slower      │           0.2149 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.hashed_array_dictionary.16 │
│ hashed_array_dictionary #16::new │ slower      │           0.2519 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.hashed_array_dictionary.16 │
│ ngram_distance #2::old           │ slower      │           0.3598 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.2           │
│ ngram_distance #2::new           │ slower      │           0.4425 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.2           │
│ ngram_distance #3::old           │ slower      │           0.3644 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.3           │
│ ngram_distance #3::new           │ slower      │           0.4716 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.3           │
│ ngram_distance #4::old           │ slower      │           0.3577 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.4           │
│ ngram_distance #4::new           │ slower      │           0.4577 │ https://s3.amazonaws.com/clickhouse-test-reports/$PR_TO_TEST/$SHA_TO_TEST/${CLICKHOUSE_PERFORMANCE_COMPARISON_CHECK_NAME_PREFIX}/report.html#changes-in-performance.ngram_distance.4           │
└──────────────────────────────────┴─────────────┴──────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-29 13:48:41 +01:00
Max K
84e5870b71
Reapply "improve CI with digest for docker, build and test jobs" (#57904) 
* Revert "Revert "improve CI with digest for docker, build and test jobs""

* fix: docker manifest merge for missing images only
 2023-12-18 09:07:22 +01:00
Max K
8c7add0334 Revert "improve CI with digest for docker, build and test jobs (#56317)" 
This reverts commit 7844fcc196.
 2023-12-15 15:48:01 +01:00
Max K
7844fcc196
improve CI with digest for docker, build and test jobs (#56317) 
* CI with digests to tag docker images, reuse builds, reuse tests and others
 2023-12-15 15:21:20 +01:00