Commit Graph

2566 Commits

Author SHA1 Message Date
alesapin
3562264404
Merge pull request #31803 from ClickHouse/faster_tests
Shorter stress and fuzzer tests
2021-11-26 10:17:26 +03:00
alexey-milovidov
6b8f7ad03f
Merge pull request #31798 from ClickHouse/fix-31748
Fix stylecheck for tests/ci/push_to_artifactory.py
2021-11-26 00:11:32 +03:00
alexey-milovidov
6017b982d9
Merge pull request #31780 from Algunenano/fix_performance_report_queries
Improve how queries are output in the performance dashboard
2021-11-25 22:57:29 +03:00
alesapin
71ae890c59
Merge pull request #31544 from azat/ci-actions-fixes
Remove duplicated not compressed logs from CI (actions)
2021-11-25 18:43:44 +03:00
alesapin
5815429059 Shorter stress and fuzzer tests 2021-11-25 17:54:50 +03:00
Mikhail f. Shiryaev
16703c2824
Fix stylecheck for tests/ci/push_to_artifactory.py 2021-11-25 14:41:08 +01:00
alexey-milovidov
e8b29cab6e
Update report.py 2021-11-25 15:37:00 +03:00
Raúl Marín
a6c34fdbc9 Improve how queries are output in the performance dashboard 2021-11-25 11:22:47 +01:00
alesapin
c86b43c7ef
Merge pull request #31685 from ClickHouse/tavplubix-patch-5
Try fix OOMs with TSAN
2021-11-25 11:34:05 +03:00
Alexey Milovidov
21f112b30e Merge branch 'better-crosscompiling-freebsd' into hermetic-builds 2021-11-24 22:43:30 +03:00
Alexey Milovidov
92031e6b31 Fix PVS-Studio 2021-11-24 20:25:11 +03:00
tavplubix
82e4f941a4
Update Dockerfile 2021-11-24 16:00:15 +03:00
tavplubix
556ccd0bc9
Update Dockerfile 2021-11-24 12:47:44 +03:00
Alexey Milovidov
51ee8fdbb0 More easy cross-compiling for FreeBSD 2021-11-24 00:32:04 +03:00
alexey-milovidov
61454ffbcb
Merge branch 'master' into hermetic-builds 2021-11-24 00:25:16 +03:00
alexey-milovidov
341f705419
Merge pull request #30599 from ClickHouse/zlib-ng-own-cmake
Use our own CMakeLists for `zlib-ng`, `cassandra`, `mariadb-connector-c` and `xz`, `re2`, `sentry`, `gsasl`, `arrow`, `protobuf`.
2021-11-23 13:33:29 +03:00
Constantine Peresypkin
fd551ad94c fix kerberized_hadoop image
current image uses non-existing centos6 repos
added direct fetch of actual rpm packages

fixed a typo in docker/images.json: duplicate entry
2021-11-22 17:32:19 +02:00
Alexey Milovidov
d604cf5573 Remove the infamous "unbundled" build 2021-11-21 13:58:26 +03:00
Azat Khuzhin
d0ecdb617e Remove duplicated not compressed logs from CI (actions)
Right now we cannot remove that compression since non-github actions
check will upload non compressed files then, but once old CI will be
removed it seems to be possible to remove that manual compression.
2021-11-19 21:33:48 +03:00
Nikolay Degterinsky
fc9ef14a73
Merge pull request #29219 from evillique/log_lz4_streaming
Add LZ4 stream compression of logs
2021-11-17 21:22:57 +03:00
Nikolay Degterinsky
eafd3ddfac Add lz4 stream compression 2021-11-16 22:30:29 +00:00
tavplubix
b1079ae309
Update run.sh 2021-11-16 20:03:50 +03:00
tavplubix
224a4c1b6b
Update run.sh 2021-11-16 17:45:37 +03:00
alexey-milovidov
f4fda976ef
Merge pull request #31063 from azat/perf-spikes
perf: switch *_log tables to Memory engine (attempt to reduce cache misses)
2021-11-14 05:13:12 +03:00
Azat Khuzhin
48b1b7b7ec Fix test_kafka_insert_avro by pinning avro version
New avro version has a code that does not allow to use BytesIO [1].

  [1]: https://github.com/apache/avro/commit/e8c61ba4f0#diff-ca721cc66db80ff81dcd7b8ad1bb0e83b2e5c2de80c207ebec79f5495d38db98R318-R319
2021-11-13 12:01:13 +03:00
Azat Khuzhin
40be97c1dd Improve fuzzer report in case of fuzzer killed
This will avoid Exception from fuzzer in description like in [1], that
someone may think that is related to the failure, which is obviously
don't in case it was KILLed.

  [1]: https://clickhouse-test-reports.s3.yandex.net/31259/a49e10c78860a451ff9cc39df6049c88234d507a/fuzzer_ubsan/report.html#fail1
2021-11-12 01:01:10 +03:00
mergify[bot]
045bd40070
Merge branch 'master' into perf-spikes 2021-11-11 07:07:28 +00:00
alesapin
0b28474f09
Merge pull request #31244 from nikitamikhaylov/disable-fuzz-builds
Disable fuzzer builds in CI
2021-11-10 16:42:08 +03:00
Nikita Mikhaylov
99f78472bc Disabled 2021-11-10 11:38:26 +00:00
Kseniia Sumarokova
5f9b35b982
Update docker_compose_postgres.yml 2021-11-10 10:36:59 +03:00
Kseniia Sumarokova
5eef7e0e45
Update docker_compose_postgres.yml 2021-11-09 15:13:01 +03:00
Azat Khuzhin
9e622b5add perf: switch *_log tables to Memory engine (attempt to reduce cache misses)
trace_log/query_log from performance tests shows (for
cases when prewarm query fails with timeout, 15sec) excessive
writeTraceInfo() in trace_log and QueryProfilerRuns in query_log, but
this is not the root cause of the timeout, but consequence.

Also query_log shows that on failures the following profile events has
significantly higher values:
- PerfLocalMemoryMisses (6.3x more)
- PerfLocalMemoryReferences (7x more)
- PerfDataTLBMisses (6.9x more)
- PerfInstructionTLBMisses (6.4x more)

During looking at performance tests logs I noticed that once the prewarm
query fails other server (left/right) was merging (MergeTree) something
in *_log tables.

But, using MergeTree for *_log in performance tests is useless, since
anyway environment for performance tests uses ramdrive.

And so MergeTree merges just increase overhead.

Eventually I expect that this should decrease extra memory referencing
and so this should decrease cache/TLB misses.

CI: https://clickhouse-test-reports.s3.yandex.net/30886/c504e0c08df7a926bb479a1d297f326f5c48a32f/performance_comparison/report.html#fail1

v2: <partition_by remove="remove"/>
2021-11-04 09:26:46 +03:00
Azat Khuzhin
ff19d22305 perf: remove over *_log tables that does not exported into artifacts 2021-11-03 08:29:18 +03:00
Azat Khuzhin
837fe8554f perf: remove zookeeper to avoid configuring DDLWorker 2021-11-03 08:29:18 +03:00
Azat Khuzhin
acef85b17d perf: cleanup server overrides 2021-11-03 08:29:18 +03:00
Azat Khuzhin
afd960db4e perf: remove redundant query profiler set queries 2021-11-03 08:09:43 +03:00
robot-clickhouse
4289336c27 Auto version update to [21.12.1.1] [54457] 2021-11-02 00:56:45 +03:00
Kruglov Pavel
9a1275cb10
Merge pull request #30178 from Avogar/tsv-csv
Refactor and improve TSV, CSV, JSONCompactEachRow, RowBinary formats. Fix bugs in formats
2021-11-02 00:38:30 +03:00
alesapin
dbfa17aabf Add correct S3 2021-11-01 13:32:56 +03:00
avogar
4e63311bb6 Try fix perf test 2021-11-01 12:13:29 +03:00
alesapin
54e404de32 Add github to known hosts in docs release 2021-11-01 10:33:52 +03:00
alesapin
ed3d707513
Merge pull request #30839 from ClickHouse/qoega-docker-docs
Docs check on github actions
2021-10-31 20:29:04 +03:00
alexey-milovidov
15b9d65221
Merge pull request #30662 from azat/conf-clickhouse
Switch everything left from `<yandex>` to `<clickhouse>`
2021-10-31 14:56:51 +03:00
alesapin
d872eae809 Better 2021-10-29 21:43:03 +03:00
alesapin
4c645f3ef6 Add docs release 2021-10-29 21:43:03 +03:00
alesapin
1a13a741f9 Fixes in docs 2021-10-29 21:43:03 +03:00
alesapin
6c8ba21c52 Add docs check image 2021-10-29 21:43:03 +03:00
alesapin
11f71fffe1 Tabs to spaces 2021-10-29 21:43:03 +03:00
alesapin
afcadc95a7 Add docs check to documentation 2021-10-29 21:43:03 +03:00
Yatsishin Ilya
faeb4ef869 Add docker container to build docs 2021-10-29 21:43:03 +03:00