thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-29 21:20:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
78,491 Commits 8,956 Branches 1,767 Tags 1.8 GiB
bdb2dce6db
Commit Graph

473 Commits

Author SHA1 Message Date
Mikhail f. Shiryaev
e97e172dd8
Merge pull request #31822 from ClickHouse/rollback-28016 
Rollback changes from #28016: ru.archive.ubuntu.com
 2021-11-29 10:27:59 +01:00
Mikhail f. Shiryaev
5408d6232f
Make archive.ubuntu.com mirror configurable for build 2021-11-26 10:55:41 +01:00
alexey-milovidov
e8b29cab6e
Update report.py 2021-11-25 15:37:00 +03:00
Raúl Marín
a6c34fdbc9 Improve how queries are output in the performance dashboard 2021-11-25 11:22:47 +01:00
Azat Khuzhin
9e622b5add perf: switch *_log tables to Memory engine (attempt to reduce cache misses) 
trace_log/query_log from performance tests shows (for
cases when prewarm query fails with timeout, 15sec) excessive
writeTraceInfo() in trace_log and QueryProfilerRuns in query_log, but
this is not the root cause of the timeout, but consequence.

Also query_log shows that on failures the following profile events has
significantly higher values:
- PerfLocalMemoryMisses (6.3x more)
- PerfLocalMemoryReferences (7x more)
- PerfDataTLBMisses (6.9x more)
- PerfInstructionTLBMisses (6.4x more)

During looking at performance tests logs I noticed that once the prewarm
query fails other server (left/right) was merging (MergeTree) something
in *_log tables.

But, using MergeTree for *_log in performance tests is useless, since
anyway environment for performance tests uses ramdrive.

And so MergeTree merges just increase overhead.

Eventually I expect that this should decrease extra memory referencing
and so this should decrease cache/TLB misses.

CI: https://clickhouse-test-reports.s3.yandex.net/30886/c504e0c08df7a926bb479a1d297f326f5c48a32f/performance_comparison/report.html#fail1

v2: <partition_by remove="remove"/>
 2021-11-04 09:26:46 +03:00
Azat Khuzhin
ff19d22305 perf: remove over *_log tables that does not exported into artifacts 2021-11-03 08:29:18 +03:00
Azat Khuzhin
837fe8554f perf: remove zookeeper to avoid configuring DDLWorker 2021-11-03 08:29:18 +03:00
Azat Khuzhin
acef85b17d perf: cleanup server overrides 2021-11-03 08:29:18 +03:00
Azat Khuzhin
afd960db4e perf: remove redundant query profiler set queries 2021-11-03 08:09:43 +03:00
avogar
4e63311bb6 Try fix perf test 2021-11-01 12:13:29 +03:00
Azat Khuzhin
9eef2ad0c2 perf: reduce number of times the query will run 
Before this patch:

- upstream/master and PR's *with* perf tests or pef scripts changes:
  --runs=13 --max-queries=0

- PRs *without* perf changes:
  --runs=7 --max-queries=20

- PRs w/ only perf tests changes:
  --runs-13 --max-queries=0 <list of perf tests>

After:

- upstream/master and PR's *with* perf tests changes:
  --runs=13 --max-queries=0

- PRs *without* perf changes:
  --runs=7 --max-queries=10

- PRs w/ only perf tests changes:
  --runs-13 --max-queries=0 <list of perf tests>

So to underline, now we will not look at perf scripts changes anymore,
and we will also decrease number of random queries to run to 10.
 2021-10-27 23:20:51 +03:00
Azat Khuzhin
7957079f95 perf: disable query/memory profiling for non prewarm queries 2021-10-27 23:20:51 +03:00
Azat Khuzhin
190459db33 perf: add queries with errors during process too to the perf artifacts 
Sometimes queries fails with timeout, and you will not get any perf
metrics for it before this patch.

Example: https://clickhouse-test-reports.s3.yandex.net/30611/5ff6c5536558821824d5fdf25a75729e5b82060d/performance_comparison/report.html#partial-queries.flat_dictionary.5
 2021-10-27 23:20:51 +03:00
Azat Khuzhin
722d528218 perf: pin clickhouse-driver to 0.2.1 for tzlocal <0.3 to avoid warning 
CI report [1]:

    /usr/local/lib/python3.6/dist-packages/clickhouse_driver/columns/datetimecolumn.py:199: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html

  [1]: https://clickhouse-test-reports.s3.yandex.net/30626/dfc85841134aa96b4e04c401462898eb305e8657/performance_comparison/report.html#changes-in-performance.format_date_time.1
 2021-10-27 23:20:51 +03:00
Alexey Milovidov
2ec75b824b Revert "repo.clickhouse.tech is not 100% ready" 
This reverts commit 8616ba62e4.
 2021-09-29 02:43:20 +03:00
Alexey Milovidov
fb7ce28198 Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-26 02:10:11 +03:00
Alexey Milovidov
90e74aff70 Fix performance test 2021-09-26 02:08:46 +03:00
Alexey Milovidov
8616ba62e4 repo.clickhouse.tech is not 100% ready 2021-09-24 04:41:59 +03:00
Alexey Milovidov
e9e77b4403 .tech -> .com 2021-09-22 03:22:57 +03:00
Alexey Milovidov
e513a5db32 Change <yandex> to <clickhouse> in configs 2021-09-20 01:38:53 +03:00
alesapin
0eac4529e0 Preparation 2021-09-06 14:33:39 +03:00
Alexey Milovidov
143744f5a7 Fix perf test 2021-08-31 02:53:55 +03:00
Yatsishin Ilya
91289983c6 Use ru.archive.ubuntu.com as default one is not responding from CI 2021-08-23 11:22:39 +03:00
alexey-milovidov
7127905909
Merge pull request #27884 from ClickHouse/aku/decimal-formatting 
fix decimal formatting settings in perf test
 2021-08-20 09:04:18 +03:00
Alexander Kuzmenkov
a7d405759c fix decimal formatting settings in perf test 2021-08-19 21:44:04 +03:00
Alexander Kuzmenkov
96a5c4b033
Merge pull request #27752 from ClickHouse/akuzm-patch-1 
try to collect some core dumps in perf tests
 2021-08-19 14:47:49 +03:00
Alexander Kuzmenkov
db53638a95
Update download.sh 2021-08-19 01:21:51 +03:00
Alexander Kuzmenkov
7f15c5c55e
Update entrypoint.sh 2021-08-19 01:21:23 +03:00
Alexander Kuzmenkov
59e3cb18f4
Update entrypoint.sh 2021-08-18 10:58:21 +03:00
Alexander Kuzmenkov
8199399159
Update entrypoint.sh 2021-08-18 07:00:51 +03:00
Alexander Kuzmenkov
c9404c1fb8
Update entrypoint.sh 2021-08-17 13:30:51 +03:00
Alexander Kuzmenkov
7bbbb19b48
try to collect some core dumps in perf tests 2021-08-16 20:05:50 +03:00
Alexander Kuzmenkov
43602a838a
Update compare.sh 2021-08-16 18:14:47 +03:00
Maksim Kita
aabb1d04bd
Merge pull request #27190 from ClickHouse/aku/perf-settings 
properly check the settings in perf test
 2021-08-05 10:56:39 +03:00
Alexander Kuzmenkov
cfcb0e4b22 properly check the settings in perf test 2021-08-04 19:41:21 +03:00
Alexander Kuzmenkov
cfb317a9a9
Update compare.sh 2021-07-22 18:37:20 +03:00
alexey-milovidov
80eaf85301
Merge pull request #24416 from elevankoff/system-metrics 
Common system metrics collection
 2021-07-07 03:21:49 +03:00
Alexey Milovidov
7ae15fee31 Change performance test after adjusted the name of column 2021-07-05 22:08:29 +03:00
Maksim Kita
577e69898a Compile aggregate functions perf tests fix 2021-07-05 12:17:01 +03:00
Azat Khuzhin
a471ebdabe Fix performance tests after converting ProfileEvents to Map type 2021-06-30 21:00:36 +03:00
Maksim Kita
352e1f27ff Update using Map datatyle in system log tables before merge 2021-06-28 14:42:21 +03:00
Maksim Kita
294b937a39
Merge branch 'system-query-log-map-type-merge' into system-querylog-map-continue 2021-06-28 11:27:59 +03:00
Alexey Milovidov
ebc2fbfd63 Performance test: be more generous 2021-06-27 03:14:42 +03:00
Alexey Milovidov
779f5eca94 Relax max test time for performance test; fix comment 2021-06-16 11:26:00 +03:00
Alexander Kuzmenkov
f9f5b7945e
fix shellcheck 2021-06-09 14:40:38 +03:00
Alexander Kuzmenkov
f4d7baaaff Merge remote-tracking branch 'origin/aku/perf-test' into HEAD 2021-06-08 18:10:19 +03:00
Alexander Kuzmenkov
07fceb667e fix 2021-06-08 17:12:47 +03:00
alexey-milovidov
cd2ca76b24
Update compare.sh 2021-06-06 12:57:18 +03:00
Alexander Kuzmenkov
1c5e069d0d fixes 2021-06-04 18:27:21 +03:00
Alexander Kuzmenkov
8554cb621e some perf test script improvements 2021-06-03 20:40:05 +03:00