taiyang-li
ab64a17104
fix fasttest
2021-11-19 16:20:14 +08:00
taiyang-li
e8644807fe
merge master and solve conflict
2021-11-19 15:01:58 +08:00
lgbo-ustc
9902ccefc5
Add integration tests for hive query
2021-11-18 16:17:49 +08:00
Nikolay Degterinsky
fc9ef14a73
Merge pull request #29219 from evillique/log_lz4_streaming
...
Add LZ4 stream compression of logs
2021-11-17 21:22:57 +03:00
Nikolay Degterinsky
eafd3ddfac
Add lz4 stream compression
2021-11-16 22:30:29 +00:00
tavplubix
b1079ae309
Update run.sh
2021-11-16 20:03:50 +03:00
tavplubix
224a4c1b6b
Update run.sh
2021-11-16 17:45:37 +03:00
alexey-milovidov
f4fda976ef
Merge pull request #31063 from azat/perf-spikes
...
perf: switch *_log tables to Memory engine (attempt to reduce cache misses)
2021-11-14 05:13:12 +03:00
Azat Khuzhin
48b1b7b7ec
Fix test_kafka_insert_avro by pinning avro version
...
New avro version has a code that does not allow to use BytesIO [1].
[1]: https://github.com/apache/avro/commit/e8c61ba4f0#diff-ca721cc66db80ff81dcd7b8ad1bb0e83b2e5c2de80c207ebec79f5495d38db98R318-R319
2021-11-13 12:01:13 +03:00
Azat Khuzhin
40be97c1dd
Improve fuzzer report in case of fuzzer killed
...
This will avoid Exception from fuzzer in description like in [1], that
someone may think that is related to the failure, which is obviously
don't in case it was KILLed.
[1]: https://clickhouse-test-reports.s3.yandex.net/31259/a49e10c78860a451ff9cc39df6049c88234d507a/fuzzer_ubsan/report.html#fail1
2021-11-12 01:01:10 +03:00
mergify[bot]
045bd40070
Merge branch 'master' into perf-spikes
2021-11-11 07:07:28 +00:00
taiyang-li
af78116c3f
Merge branch 'master' into hive_table
2021-11-11 11:52:28 +08:00
Kseniia Sumarokova
5f9b35b982
Update docker_compose_postgres.yml
2021-11-10 10:36:59 +03:00
taiyang-li
26988ab757
Merge branch 'master' into hive_table
2021-11-10 11:01:03 +08:00
Kseniia Sumarokova
5eef7e0e45
Update docker_compose_postgres.yml
2021-11-09 15:13:01 +03:00
taiyang-li
5dbbe4b9c5
fix build error
2021-11-09 13:14:53 +08:00
Azat Khuzhin
9e622b5add
perf: switch *_log tables to Memory engine (attempt to reduce cache misses)
...
trace_log/query_log from performance tests shows (for
cases when prewarm query fails with timeout, 15sec) excessive
writeTraceInfo() in trace_log and QueryProfilerRuns in query_log, but
this is not the root cause of the timeout, but consequence.
Also query_log shows that on failures the following profile events has
significantly higher values:
- PerfLocalMemoryMisses (6.3x more)
- PerfLocalMemoryReferences (7x more)
- PerfDataTLBMisses (6.9x more)
- PerfInstructionTLBMisses (6.4x more)
During looking at performance tests logs I noticed that once the prewarm
query fails other server (left/right) was merging (MergeTree) something
in *_log tables.
But, using MergeTree for *_log in performance tests is useless, since
anyway environment for performance tests uses ramdrive.
And so MergeTree merges just increase overhead.
Eventually I expect that this should decrease extra memory referencing
and so this should decrease cache/TLB misses.
CI: https://clickhouse-test-reports.s3.yandex.net/30886/c504e0c08df7a926bb479a1d297f326f5c48a32f/performance_comparison/report.html#fail1
v2: <partition_by remove="remove"/>
2021-11-04 09:26:46 +03:00
Azat Khuzhin
ff19d22305
perf: remove over *_log tables that does not exported into artifacts
2021-11-03 08:29:18 +03:00
Azat Khuzhin
837fe8554f
perf: remove zookeeper to avoid configuring DDLWorker
2021-11-03 08:29:18 +03:00
Azat Khuzhin
acef85b17d
perf: cleanup server overrides
2021-11-03 08:29:18 +03:00
Azat Khuzhin
afd960db4e
perf: remove redundant query profiler set queries
2021-11-03 08:09:43 +03:00
robot-clickhouse
4289336c27
Auto version update to [21.12.1.1] [54457]
2021-11-02 00:56:45 +03:00
Kruglov Pavel
9a1275cb10
Merge pull request #30178 from Avogar/tsv-csv
...
Refactor and improve TSV, CSV, JSONCompactEachRow, RowBinary formats. Fix bugs in formats
2021-11-02 00:38:30 +03:00
alesapin
dbfa17aabf
Add correct S3
2021-11-01 13:32:56 +03:00
avogar
4e63311bb6
Try fix perf test
2021-11-01 12:13:29 +03:00
alexey-milovidov
15b9d65221
Merge pull request #30662 from azat/conf-clickhouse
...
Switch everything left from `<yandex>` to `<clickhouse>`
2021-10-31 14:56:51 +03:00
tavplubix
059c1ebf36
Merge pull request #30847 from ClickHouse/fix_30802
...
Disable check for ZooKeeper session uptime by default
2021-10-29 20:18:25 +03:00
Alexander Tokmakov
d095cfe4c1
disable check for zk session uptime by default
2021-10-29 14:45:09 +03:00
alesapin
4eca493ab8
Fix fuzzer on master
2021-10-29 13:47:29 +03:00
alesapin
11b70a285c
Merge pull request #30795 from ClickHouse/make_fuzzer_check_less_hardcoded
...
Trying to make fuzzer check less hardcoded
2021-10-29 10:21:36 +03:00
alesapin
ddac7a51e2
Merge pull request #30729 from ClickHouse/trying_actions
...
Add functional tests to github actions
2021-10-29 09:49:31 +03:00
mergify[bot]
65df198e9b
Merge branch 'master' into conf-clickhouse
2021-10-28 22:19:42 +00:00
alesapin
ffc8dbd085
Update run-fuzzer.sh
2021-10-28 23:06:04 +03:00
alesapin
313584e553
Update run-fuzzer.sh
2021-10-28 19:12:47 +03:00
mergify[bot]
a94b8902dd
Merge branch 'master' into trying_actions
2021-10-28 15:57:28 +00:00
alesapin
0651a1f0e9
Update run-fuzzer.sh
2021-10-28 17:03:40 +03:00
alesapin
542196128b
Trying to make fuzzer check less hardcoded
2021-10-28 12:29:50 +03:00
alesapin
c0b2f92123
Add stateful test
2021-10-28 12:26:59 +03:00
Azat Khuzhin
9eef2ad0c2
perf: reduce number of times the query will run
...
Before this patch:
- upstream/master and PR's *with* perf tests or pef scripts changes:
--runs=13 --max-queries=0
- PRs *without* perf changes:
--runs=7 --max-queries=20
- PRs w/ only perf tests changes:
--runs-13 --max-queries=0 <list of perf tests>
After:
- upstream/master and PR's *with* perf tests changes:
--runs=13 --max-queries=0
- PRs *without* perf changes:
--runs=7 --max-queries=10
- PRs w/ only perf tests changes:
--runs-13 --max-queries=0 <list of perf tests>
So to underline, now we will not look at perf scripts changes anymore,
and we will also decrease number of random queries to run to 10.
2021-10-27 23:20:51 +03:00
Azat Khuzhin
7957079f95
perf: disable query/memory profiling for non prewarm queries
2021-10-27 23:20:51 +03:00
Azat Khuzhin
190459db33
perf: add queries with errors during process too to the perf artifacts
...
Sometimes queries fails with timeout, and you will not get any perf
metrics for it before this patch.
Example: https://clickhouse-test-reports.s3.yandex.net/30611/5ff6c5536558821824d5fdf25a75729e5b82060d/performance_comparison/report.html#partial-queries.flat_dictionary.5
2021-10-27 23:20:51 +03:00
Azat Khuzhin
722d528218
perf: pin clickhouse-driver to 0.2.1 for tzlocal <0.3 to avoid warning
...
CI report [1]:
/usr/local/lib/python3.6/dist-packages/clickhouse_driver/columns/datetimecolumn.py:199: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
[1]: https://clickhouse-test-reports.s3.yandex.net/30626/dfc85841134aa96b4e04c401462898eb305e8657/performance_comparison/report.html#changes-in-performance.format_date_time.1
2021-10-27 23:20:51 +03:00
Azat Khuzhin
90ea2de9bb
Add unidiff into style image
2021-10-27 10:03:24 +03:00
Azat Khuzhin
b41c77f4f8
Add boto3 to style image
2021-10-27 10:03:24 +03:00
Azat Khuzhin
944437caf6
Add PyGithub into style image
2021-10-27 10:03:23 +03:00
Azat Khuzhin
da90df5658
docker: switch <yandex> to <clickhouse>
2021-10-26 08:53:14 +03:00
Azat Khuzhin
8f015764a7
Add metadata for *_log into tests artifacts
...
This will allow to use artifacts as-is, i.e.:
clickhouse-local --path /path/to/log/artifact
v2: convert database to Ordinary to avoid extra symlinks dependencies.
2021-10-23 20:17:01 +03:00
alesapin
9e8dfdb089
Merge pull request #28851 from ClickHouse/trying_actions
...
Trying github actions
2021-10-20 12:55:53 +03:00
alesapin
6d8853698b
Merge branch 'master' into trying_actions
2021-10-19 17:45:37 +03:00
vdimir
acf416900d
Minor fix in clickhouse/kerberized-hadoop Dockerfile
2021-10-19 16:45:03 +03:00
