Nikolay Degterinsky
fc9ef14a73
Merge pull request #29219 from evillique/log_lz4_streaming
...
Add LZ4 stream compression of logs
2021-11-17 21:22:57 +03:00
Nikolay Degterinsky
eafd3ddfac
Add lz4 stream compression
2021-11-16 22:30:29 +00:00
tavplubix
b1079ae309
Update run.sh
2021-11-16 20:03:50 +03:00
tavplubix
224a4c1b6b
Update run.sh
2021-11-16 17:45:37 +03:00
alexey-milovidov
f4fda976ef
Merge pull request #31063 from azat/perf-spikes
...
perf: switch *_log tables to Memory engine (attempt to reduce cache misses)
2021-11-14 05:13:12 +03:00
Azat Khuzhin
48b1b7b7ec
Fix test_kafka_insert_avro by pinning avro version
...
New avro version has a code that does not allow to use BytesIO [1].
[1]: https://github.com/apache/avro/commit/e8c61ba4f0#diff-ca721cc66db80ff81dcd7b8ad1bb0e83b2e5c2de80c207ebec79f5495d38db98R318-R319
2021-11-13 12:01:13 +03:00
Azat Khuzhin
40be97c1dd
Improve fuzzer report in case of fuzzer killed
...
This will avoid Exception from fuzzer in description like in [1], that
someone may think that is related to the failure, which is obviously
don't in case it was KILLed.
[1]: https://clickhouse-test-reports.s3.yandex.net/31259/a49e10c78860a451ff9cc39df6049c88234d507a/fuzzer_ubsan/report.html#fail1
2021-11-12 01:01:10 +03:00
mergify[bot]
045bd40070
Merge branch 'master' into perf-spikes
2021-11-11 07:07:28 +00:00
alesapin
0b28474f09
Merge pull request #31244 from nikitamikhaylov/disable-fuzz-builds
...
Disable fuzzer builds in CI
2021-11-10 16:42:08 +03:00
Nikita Mikhaylov
99f78472bc
Disabled
2021-11-10 11:38:26 +00:00
Kseniia Sumarokova
5f9b35b982
Update docker_compose_postgres.yml
2021-11-10 10:36:59 +03:00
Kseniia Sumarokova
5eef7e0e45
Update docker_compose_postgres.yml
2021-11-09 15:13:01 +03:00
Azat Khuzhin
9e622b5add
perf: switch *_log tables to Memory engine (attempt to reduce cache misses)
...
trace_log/query_log from performance tests shows (for
cases when prewarm query fails with timeout, 15sec) excessive
writeTraceInfo() in trace_log and QueryProfilerRuns in query_log, but
this is not the root cause of the timeout, but consequence.
Also query_log shows that on failures the following profile events has
significantly higher values:
- PerfLocalMemoryMisses (6.3x more)
- PerfLocalMemoryReferences (7x more)
- PerfDataTLBMisses (6.9x more)
- PerfInstructionTLBMisses (6.4x more)
During looking at performance tests logs I noticed that once the prewarm
query fails other server (left/right) was merging (MergeTree) something
in *_log tables.
But, using MergeTree for *_log in performance tests is useless, since
anyway environment for performance tests uses ramdrive.
And so MergeTree merges just increase overhead.
Eventually I expect that this should decrease extra memory referencing
and so this should decrease cache/TLB misses.
CI: https://clickhouse-test-reports.s3.yandex.net/30886/c504e0c08df7a926bb479a1d297f326f5c48a32f/performance_comparison/report.html#fail1
v2: <partition_by remove="remove"/>
2021-11-04 09:26:46 +03:00
Azat Khuzhin
ff19d22305
perf: remove over *_log tables that does not exported into artifacts
2021-11-03 08:29:18 +03:00
Azat Khuzhin
837fe8554f
perf: remove zookeeper to avoid configuring DDLWorker
2021-11-03 08:29:18 +03:00
Azat Khuzhin
acef85b17d
perf: cleanup server overrides
2021-11-03 08:29:18 +03:00
Azat Khuzhin
afd960db4e
perf: remove redundant query profiler set queries
2021-11-03 08:09:43 +03:00
robot-clickhouse
4289336c27
Auto version update to [21.12.1.1] [54457]
2021-11-02 00:56:45 +03:00
Kruglov Pavel
9a1275cb10
Merge pull request #30178 from Avogar/tsv-csv
...
Refactor and improve TSV, CSV, JSONCompactEachRow, RowBinary formats. Fix bugs in formats
2021-11-02 00:38:30 +03:00
alesapin
dbfa17aabf
Add correct S3
2021-11-01 13:32:56 +03:00
avogar
4e63311bb6
Try fix perf test
2021-11-01 12:13:29 +03:00
alesapin
54e404de32
Add github to known hosts in docs release
2021-11-01 10:33:52 +03:00
alesapin
ed3d707513
Merge pull request #30839 from ClickHouse/qoega-docker-docs
...
Docs check on github actions
2021-10-31 20:29:04 +03:00
alexey-milovidov
15b9d65221
Merge pull request #30662 from azat/conf-clickhouse
...
Switch everything left from `<yandex>` to `<clickhouse>`
2021-10-31 14:56:51 +03:00
alesapin
d872eae809
Better
2021-10-29 21:43:03 +03:00
alesapin
4c645f3ef6
Add docs release
2021-10-29 21:43:03 +03:00
alesapin
1a13a741f9
Fixes in docs
2021-10-29 21:43:03 +03:00
alesapin
6c8ba21c52
Add docs check image
2021-10-29 21:43:03 +03:00
alesapin
11f71fffe1
Tabs to spaces
2021-10-29 21:43:03 +03:00
alesapin
afcadc95a7
Add docs check to documentation
2021-10-29 21:43:03 +03:00
Yatsishin Ilya
faeb4ef869
Add docker container to build docs
2021-10-29 21:43:03 +03:00
tavplubix
059c1ebf36
Merge pull request #30847 from ClickHouse/fix_30802
...
Disable check for ZooKeeper session uptime by default
2021-10-29 20:18:25 +03:00
Alexander Tokmakov
d095cfe4c1
disable check for zk session uptime by default
2021-10-29 14:45:09 +03:00
alesapin
4eca493ab8
Fix fuzzer on master
2021-10-29 13:47:29 +03:00
alesapin
11b70a285c
Merge pull request #30795 from ClickHouse/make_fuzzer_check_less_hardcoded
...
Trying to make fuzzer check less hardcoded
2021-10-29 10:21:36 +03:00
alesapin
ddac7a51e2
Merge pull request #30729 from ClickHouse/trying_actions
...
Add functional tests to github actions
2021-10-29 09:49:31 +03:00
mergify[bot]
65df198e9b
Merge branch 'master' into conf-clickhouse
2021-10-28 22:19:42 +00:00
alesapin
ffc8dbd085
Update run-fuzzer.sh
2021-10-28 23:06:04 +03:00
alesapin
313584e553
Update run-fuzzer.sh
2021-10-28 19:12:47 +03:00
mergify[bot]
a94b8902dd
Merge branch 'master' into trying_actions
2021-10-28 15:57:28 +00:00
alesapin
0651a1f0e9
Update run-fuzzer.sh
2021-10-28 17:03:40 +03:00
alesapin
542196128b
Trying to make fuzzer check less hardcoded
2021-10-28 12:29:50 +03:00
alesapin
c0b2f92123
Add stateful test
2021-10-28 12:26:59 +03:00
Azat Khuzhin
9eef2ad0c2
perf: reduce number of times the query will run
...
Before this patch:
- upstream/master and PR's *with* perf tests or pef scripts changes:
--runs=13 --max-queries=0
- PRs *without* perf changes:
--runs=7 --max-queries=20
- PRs w/ only perf tests changes:
--runs-13 --max-queries=0 <list of perf tests>
After:
- upstream/master and PR's *with* perf tests changes:
--runs=13 --max-queries=0
- PRs *without* perf changes:
--runs=7 --max-queries=10
- PRs w/ only perf tests changes:
--runs-13 --max-queries=0 <list of perf tests>
So to underline, now we will not look at perf scripts changes anymore,
and we will also decrease number of random queries to run to 10.
2021-10-27 23:20:51 +03:00
Azat Khuzhin
7957079f95
perf: disable query/memory profiling for non prewarm queries
2021-10-27 23:20:51 +03:00
Azat Khuzhin
190459db33
perf: add queries with errors during process too to the perf artifacts
...
Sometimes queries fails with timeout, and you will not get any perf
metrics for it before this patch.
Example: https://clickhouse-test-reports.s3.yandex.net/30611/5ff6c5536558821824d5fdf25a75729e5b82060d/performance_comparison/report.html#partial-queries.flat_dictionary.5
2021-10-27 23:20:51 +03:00
Azat Khuzhin
722d528218
perf: pin clickhouse-driver to 0.2.1 for tzlocal <0.3 to avoid warning
...
CI report [1]:
/usr/local/lib/python3.6/dist-packages/clickhouse_driver/columns/datetimecolumn.py:199: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
[1]: https://clickhouse-test-reports.s3.yandex.net/30626/dfc85841134aa96b4e04c401462898eb305e8657/performance_comparison/report.html#changes-in-performance.format_date_time.1
2021-10-27 23:20:51 +03:00
Azat Khuzhin
90ea2de9bb
Add unidiff into style image
2021-10-27 10:03:24 +03:00
Azat Khuzhin
b41c77f4f8
Add boto3 to style image
2021-10-27 10:03:24 +03:00
Azat Khuzhin
944437caf6
Add PyGithub into style image
2021-10-27 10:03:23 +03:00
