thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-15 20:24:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
156,349 Commits 9,137 Branches 1,774 Tags 2.1 GiB
fae5b11709
Commit Graph

5270 Commits

Author SHA1 Message Date
Azat Khuzhin
5390d1b108 docker: use self-compiled gdb 15 from cctools 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-06 17:15:49 +02:00
Azat Khuzhin
72bd43a309 tests: do not capture client stacktraces in stress tests 
They are too uncontrollable, and likely will leave some clients [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/67737/9658be5eea8351655dd3ea77b8c1d4717bac7999/stress_test__ubsan_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-06 16:42:19 +02:00
Azat Khuzhin
7c366a040f ci: use bash arrays to pass opts to clickhouse-test for stateless/stateful 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-06 16:39:16 +02:00
Nikolay Degterinsky
1126ae3e5a
Merge pull request #67560 from evillique/fix-timeout-stateless-tests 
[CI] Check for timeout when we stop logs replication
 2024-08-06 14:35:23 +00:00
Azat Khuzhin
424f19d6c0 Move gdb into cctools 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-06 16:25:56 +02:00
Azat Khuzhin
459dd1ff2d Add a comment of the image name for cctools 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-06 16:25:56 +02:00
Azat Khuzhin
99b18d31db Update gdb to 15.1 (by compiling from sources) 
Right now there are couple of gdb bugs that makes CI unstable:
- https://sourceware.org/bugzilla/show_bug.cgi?id=29185
- https://bugzilla.redhat.com/show_bug.cgi?id=1492496

But ubuntu 22.04 does not have 14+ anywhere, the
~ubuntu-toolchain-r/test contains only gdb 13, so there is no other
options except for compiling it from sources.

But there also other reasons to update it - optimizations, looks like
older gdb versions does not use index fully - 5.6sec vs 56sec:

    # 15.1
    $ time command gdb -batch -ex 'disas main' clickhouse
    ...
    real    0m5.692s
    user    0m29.948s
    sys     0m1.190s

    # 12.1 (from ubuntu 22.04)
    real    0m56.709s
    user    0m59.307s
    sys     0m0.585s

Also note, that we cannot compile gdb in the fasttest (that contains
compiler) since some images does not includes full toolchain, for
instance gdb is added in the following images:
- test-util -> test-base -> lots of other images (no toolchain)
- performance-comparison (no toolchain)
- integration-test (no toolchain)
- integration-tests-runner (no toolchain)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-06 16:25:56 +02:00
Nikolay Degterinsky
103ff7d2f9 Merge remote-tracking branch 'upstream/master' into fix-timeout-stateless-tests 2024-08-06 10:23:57 +00:00
Nikita Mikhaylov
b4a6f249ab
Merge pull request #66438 from ClickHouse/analyzer-beta 
Analyzer is finally Beta
 2024-08-06 09:39:29 +00:00
Raúl Marín
25fa63f7e6 Merge remote-tracking branch 'blessed/master' into unit_test_asan 2024-08-06 11:19:57 +02:00
Michael Kolupaev
f344d833fd
Merge pull request #65253 from ClickHouse/reg 
attach_gdb.lib: print more information before all stacks
 2024-08-06 08:18:46 +00:00
Alexey Katsman
913bc8ab3a
Merge pull request #67869 from alexkats/misc-fixes 
Misc fixes
 2024-08-06 04:10:41 +00:00
Michael Kolupaev
93afcd16b0 Merge remote-tracking branch 'origin/master' into reg 2024-08-05 22:10:56 +00:00
Nikolay Degterinsky
5d0c179ad1 Merge remote-tracking branch 'upstream/master' into fix-timeout-stateless-tests 2024-08-05 19:38:09 +00:00
Azat Khuzhin
618789d196 Do not pass logs for keeper explicitly in the image to allow overriding 
Right now, if you will start keeper like this:

    $ cat /tmp/local.yaml
    ---
    logger:
        log:
            "@remove": remove
        errorlog:
            "@remove": remove
        console: 1
    $ docker run --name keeper -v /tmp/local.yaml:/etc/clickhouse-keeper/conf.d/local.yaml --rm -it clickhouse/clickhouse-keeper

This will still not work, because the --log-file/--errorlog-file passed
explicitly.

So this patch removes this in case config is valid, but, if keeper is
launched with embedded config, this had been left as-is, to keep
previous behavior.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-05 20:28:13 +02:00
Alexey Milovidov
27a06f9eec Merge branch 'master' into tighten-limits-functional-tests 2024-08-05 19:57:21 +02:00
Alex Katsman
c8805fbced Misc fixes 2024-08-05 17:41:47 +00:00
Nikita Mikhaylov
4e9c3baa25 Move analyzer to Beta stage 2024-08-05 15:29:14 +00:00
Yarik Briukhovetskyi
9b06753bb7
Merge branch 'master' into hive_style_partitioning 2024-08-05 17:02:27 +02:00
Alexey Milovidov
a416d2bc4a
Merge pull request #67791 from azat/ci/stateless-v2 
ci/stateless: fix artifacts post-processing and detect if something failed there
 2024-08-05 14:46:58 +00:00
Alexey Milovidov
f1a096e8f6
Merge pull request #67723 from ClickHouse/fix-log-and-exception-messages-formatting 
Fix test `00002_log_and_exception_messages_formatting`
 2024-08-05 12:34:33 +00:00
Alexey Milovidov
5875900189
Merge pull request #67780 from ClickHouse/fix-small-diagnostic-in-test 
Fix diagnostics in the test script
 2024-08-05 12:27:37 +00:00
Alexey Milovidov
6b4353b910
Merge pull request #66495 from azat/ci-images-llvm-cleanup 
Ensure that llvm-symbolizer is used for symbolizing sanitizer reports
 2024-08-05 12:25:32 +00:00
Azat Khuzhin
b47f8a733f ci: fix basic errors collecting after stateless tests 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-05 11:16:08 +02:00
Alexey Milovidov
a573b2926e Fixes for the script 2024-08-04 23:59:44 +02:00
Alexey Milovidov
0bdaa57023 Fix diagnostics in the test script 2024-08-04 23:53:54 +02:00
Azat Khuzhin
43cf85ef28 ci: collect basic issues in stateless tests (dmesg, fatal and similar) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Azat Khuzhin
c200f43774 ci: fix stateless runner to correctly gather artifacts when server failed 
Right now it simply does not work due to "set -e", with it you cannot
use "foo=$(false)" since bash will break execution after, rewrite it to
a plain shell, with "if".

Also use ZSTD everywhere (ugh)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 23:19:33 +02:00
Alexey Milovidov
f08b32d528 Merge branch 'master' into tighten-limits-functional-tests 2024-08-04 21:44:38 +02:00
Alexey Milovidov
a433115434 Fix typo 2024-08-04 19:18:00 +02:00
Azat Khuzhin
1f2bb6676f Remove outdated comments about workaround for #64086 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:14:05 +02:00
Azat Khuzhin
dea95e6c33 Ensure that llvm-symbolizer is used for symbolizing sanitizer reports 
Since you don't want to fallback to addr2line:

    # addr2line
    $ time /bin/test
    set_flag_impl: Success
    set_flag_if: Success
    /usr/bin/addr2line: DWARF error: invalid or unhandled FORM value: 0x23
    ==================
    WARNING: ThreadSanitizer: data race (pid=18)
    ...
    real    3m8.580s
    user    0m21.967s
    sys     0m40.628s

    # llvm-symbolizer
    $ time ./test
    set_flag_impl: Success
    set_flag_if: Success
    ==================
    WARNING: ThreadSanitizer: data race (pid=24884)
    real    0m0.028s
    user    0m0.003s
    sys     0m0.006s

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-08-04 16:14:05 +02:00
Kseniia Sumarokova
881ac7208e
Merge pull request #67361 from ClickHouse/add-no-distributed-cache-tag 
Add no-distributed-cache tag in tests
 2024-08-04 12:36:31 +00:00
Alexey Milovidov
900cc8afc6 Merge branch 'master' into tighten-limits-functional-tests 2024-08-03 19:29:58 +02:00
Alexey Milovidov
fcb0ce7361 Fix docs build 2024-08-02 18:17:32 +02:00
Raúl Marín
86bab5a78a Unit test: Don't return ok if leaks are detected 2024-08-02 16:09:43 +02:00
Kseniia Sumarokova
d8d5d45412
Merge branch 'master' into add-no-distributed-cache-tag 2024-08-02 14:50:43 +02:00
Raúl Marín
e5cf376c4f Don't attach gdb in ASAN runs of fuzzer, stateless or stress checks 2024-08-02 12:56:49 +02:00
Raúl Marín
97f1f6e221 Don't hide errors on clickhouse local runs 2024-08-02 12:46:37 +02:00
Nikolay Degterinsky
2e7a15df89
Update setup_export_logs.sh 2024-08-01 23:23:06 +02:00
Raúl Marín
35b6112b7b Try to stop on leaks 2024-08-01 19:50:13 +02:00
Raúl Marín
c5a8653daf Playing with Docker and the CI 2024-08-01 19:27:12 +02:00
Raúl Marín
4e9761acf9 Don't run ASAN unit tests under gdb 2024-08-01 18:59:17 +02:00
Alexey Milovidov
b8d68add23
Merge branch 'master' into tighten-limits-functional-tests 2024-08-01 17:40:20 +02:00
Yarik Briukhovetskyi
f6f85f90c3
Merge branch 'ClickHouse:master' into hive_style_partitioning 2024-08-01 13:15:20 +02:00
maxvostrikov
928d5df3f0 added tests for https://github.com/ClickHouse/ClickHouse/pull/65475 
Support writting page index into parquet file
 2024-08-01 12:32:43 +02:00
robot-clickhouse
f162d6bd5e Update version_date.tsv and changelogs after v24.7.2.13-stable 2024-08-01 06:35:22 +00:00
Nikolay Degterinsky
89ca6aee4e Check for timeout when we stop logs replication 2024-07-31 20:00:37 +00:00
Yarik Briukhovetskyi
fac73a0944
Merge branch 'master' into hive_style_partitioning 2024-07-31 18:25:08 +02:00
Alexey Milovidov
59b85fb788 Merge branch 'master' into tighten-limits-functional-tests 2024-07-30 23:19:38 +02:00
robot-clickhouse
dacf044c3d Update version_date.tsv and changelogs after v24.7.1.2915-stable 2024-07-30 19:20:52 +00:00
kssenii
bf25f1815e Merge remote-tracking branch 'origin' into add-no-distributed-cache-tag 2024-07-30 13:45:23 +02:00
Yarik Briukhovetskyi
f0da5c528f
Merge branch 'master' into hive_style_partitioning 2024-07-30 11:49:14 +02:00
kssenii
161e794618 Add no-distributed-cache tag 2024-07-29 14:03:22 +02:00
Antonio Andelic
6f2d25cc39 Merge branch 'master' into add-numactl 2024-07-28 11:29:26 +02:00
Alexey Milovidov
8dd0542fc1 Merge branch 'master' into tighten-limits-functional-tests 2024-07-27 13:19:59 +02:00
Alexey Milovidov
b8b6188cbf Merge branch 'master' into tighten-limits-functional-tests 2024-07-27 01:28:19 +02:00
Alexey Milovidov
c4d772557f
Merge pull request #66790 from ClickHouse/better-diagnostics-in-functional-tests 
Better diagnostics in functional tests
 2024-07-26 23:15:42 +00:00
Nikita Fomichev
c95c1a7232
Merge pull request #66901 from ClickHouse/stable-test-runs 
Stateless tests: Change status of failed tests in case of server crash and add no-parallel to high-load tests
 2024-07-26 18:42:42 +00:00
Alexey Milovidov
ffe15e2e37 Merge branch 'master' into tighten-limits-functional-tests 2024-07-26 14:11:59 +02:00
Nikita Fomichev
7f80dab692 CI push 2024-07-26 12:34:36 +02:00
Antonio Andelic
287cce7d21 Fixes 2024-07-26 09:20:15 +02:00
Alexey Milovidov
8f91fa8b78 Merge branch 'fix-unit-tests' into better-diagnostics-in-functional-tests 2024-07-26 04:51:46 +02:00
Ilya Yatsishin
638937674f Merge remote-tracking branch 'origin/master' into qoega-patch-6 2024-07-25 23:19:00 +02:00
Alexey Milovidov
6b56f427d3
Merge pull request #67145 from ClickHouse/lower-max-allocation-size-in-query-fuzzer 
Lower max allocation size in query fuzzer
 2024-07-25 21:00:58 +00:00
Alexey Milovidov
02461263f2 Merge branch 'master' into tighten-limits-functional-tests 2024-07-25 21:27:14 +02:00
Antonio Andelic
cba75018ee
Merge pull request #66847 from ClickHouse/dont-ignore-sigsegv 
Stop ignoring SIGSEGV in GDB
 2024-07-25 17:53:42 +00:00
Alexey Milovidov
1059c7b76e Merge branch 'master' into tighten-limits-functional-tests 2024-07-25 19:34:47 +02:00
Alexey Milovidov
ee193ffa01 Lower max allocation size in query fuzzer 2024-07-25 19:25:58 +02:00
Alexey Milovidov
782d829d87 Merge branch 'master' into better-diagnostics-in-functional-tests 2024-07-25 18:47:09 +02:00
Antonio Andelic
c5e01c91d2 Merge branch 'master' into dont-ignore-sigsegv 2024-07-25 11:23:39 +02:00
János Benjamin Antal
2def300631
Merge pull request #67049 from ClickHouse/fix-sanitizer-allocation-size-is-too-big 
Increase max allocation size for sanitizers
 2024-07-25 08:32:38 +00:00
János Benjamin Antal
e498b86f50 Add option for LSAN in shell also 2024-07-24 14:00:28 +00:00
János Benjamin Antal
c5d262c23b Increase max allocation size for sanitizers 2024-07-24 13:55:59 +00:00
Alexey Milovidov
e2c78844a0 Fix tests 2024-07-24 14:53:09 +02:00
Alexey Milovidov
036485a657 Fix error 2024-07-24 12:15:48 +02:00
Alexey Milovidov
878a340317 Fix tests 2024-07-24 11:47:07 +02:00
Alexey Milovidov
e50e28ac62 Merge branch 'master' into tighten-limits-functional-tests 2024-07-24 03:50:08 +02:00
Alexey Milovidov
a18ac1aa19 Merge branch 'master' into better-diagnostics-in-functional-tests 2024-07-24 03:44:14 +02:00
Ilya Yatsishin
8925060366
Touch dockerfile to forcefully rebuild image 2024-07-24 00:18:12 +02:00
Nikita Fomichev
b6ad57aa37 Stateless tests: change status for failed tests in case of server crash 2024-07-23 10:30:00 +02:00
Michael Kolupaev
997f94169e Merge remote-tracking branch 'origin/master' into reg 2024-07-22 20:52:29 +00:00
Raúl Marín
9f86b22d30 Speed up stateful tests table setup 2024-07-22 14:42:30 +02:00
Ilya Yatsishin
d296e62bf3
Update docker/test/integration/runner/Dockerfile 2024-07-22 14:31:50 +02:00
Ilya Yatsishin
340214a246
Unpin docker-ce in integration-tests-runner 2024-07-22 12:55:50 +02:00
Antonio Andelic
8217dcccc1 Stop ignoring SIGSEGV in GDB 2024-07-22 09:53:56 +02:00
Alexey Milovidov
8c264230e3 Loosen the limit 2024-07-22 07:20:33 +02:00
Alexey Milovidov
ebc05f53c3 Merge branch 'master' into better-diagnostics-in-functional-tests 2024-07-21 04:28:22 +02:00
Alexey Milovidov
444303cb71 Better diagnostics 2024-07-20 00:25:57 +02:00
Alexey Milovidov
5ae3a421e0 Copy-paste 2024-07-20 00:12:02 +02:00
Alexey Milovidov
ba6b7b86ba Log messages 2024-07-20 00:09:53 +02:00
Alexey Milovidov
134c006540 Whitespace 2024-07-20 00:08:25 +02:00
Alexey Milovidov
a373b62bbf Better diagnostics in functional tests 2024-07-20 00:04:52 +02:00
Nikita Fomichev
1e160189f0 Increase parallel jobs for stateless tests 2024-07-19 20:10:41 +02:00
Nikita Fomichev
fa2659a87c Increase CI timeout for stateless tests 2024-07-19 20:10:08 +02:00
Nikita Fomichev
7c1ed1b445 Stateless tests: check parallelization 2024-07-18 21:50:21 +02:00
Nikita Fomichev
bd10951db8 Stateless tests: tune MAX_RUN_TIME 2024-07-18 21:50:21 +02:00
Nikita Fomichev
626e1f70cb Stateful tests: decrease tests timeout 2024-07-18 21:49:47 +02:00
Yakov Olkhovskiy
f07a395bf1 Merge branch 'master' into ci-fuzzer-enable 2024-07-17 03:43:57 +00:00
Alexey Milovidov
60f45cb993 A fix 2024-07-17 03:52:35 +02:00
Alexey Milovidov
408fda417e Merge branch 'master' into fix-fasttest-expect 2024-07-17 03:51:53 +02:00
Alexey Milovidov
c913a36ed6
Minor modifications 2024-07-17 02:13:57 +02:00
Alexey Milovidov
0d1789292b
Add sudo for convenience 2024-07-17 02:11:09 +02:00
Nikita Fomichev
77c4e6ae8c Stateless tests: disable sequential tests in parallel tests 2024-07-16 12:21:08 +02:00
Alexey Milovidov
e292358ade
Update run.sh 2024-07-16 06:12:42 +02:00
Alexey Milovidov
68aedc2191 Fix something in Fast Test 2024-07-16 03:58:55 +02:00
Yakov Olkhovskiy
7de367d692 Merge branch 'master' into ci-fuzzer-enable 2024-07-15 20:58:42 +00:00
Yakov Olkhovskiy
0176e0b2ae fix 2024-07-15 19:27:54 +00:00
Yakov Olkhovskiy
5361ade8e7 optimize 2024-07-15 19:10:07 +00:00
robot-clickhouse
97c9739abc Automatic style fix 2024-07-15 17:21:59 +00:00
Yarik Briukhovetskyi
4a7c52900a
Merge branch 'master' into hive_style_partitioning 2024-07-15 19:05:21 +02:00
Yakov Olkhovskiy
50f068efe9 add dedicated runner to libfuzzer, update docker 2024-07-15 17:00:53 +00:00
Nikita Fomichev
2a0253d1e2 Tests: rename bad log names 2024-07-15 13:14:29 +02:00
Yakov Olkhovskiy
698b5ce7fa Merge branch 'master' into ci-fuzzer-enable 2024-07-14 05:31:56 +00:00
Yakov Olkhovskiy
3ccc2aed4c add fuzzer_arguments to fuzzer runner 2024-07-13 23:44:13 +00:00
Alexey Milovidov
e3a2a2e487
Merge pull request #66353 from ClickHouse/collect-minio-logs 
Collect logs from `minio` in stateless and statefull tests
 2024-07-13 21:34:50 +00:00
Max K
1050217445 add support for new release branch 
Automatic style fix
 2024-07-13 16:21:22 +02:00
Max K
f8e71707f2 update black 2024-07-13 16:21:22 +02:00
Yarik Briukhovetskyi
c09c7a4568
Merge branch 'ClickHouse:master' into hive_style_partitioning 2024-07-13 01:13:46 +02:00
Vitaly Baranov
90142b73ad
Merge pull request #66284 from vitlibar/add-stateless-test-for-grpc 
Add a stateless test for gRPC protocol
 2024-07-12 17:32:06 +00:00
Yarik Briukhovetskyi
17d2f8ed20
Merge branch 'ClickHouse:master' into hive_style_partitioning 2024-07-12 16:33:48 +02:00
Nikita Fomichev
258ae45aa8
Merge pull request #66411 from ClickHouse/fix-hanging-on-gdb 
Stateless tests: deal with hang-ups more roughly
 2024-07-12 13:25:46 +00:00
Nikita Fomichev
dd6dac6c5a Stateless tests: better sort checks in test report 2024-07-12 12:15:59 +02:00
Nikita Fomichev
f30cd12434 Stateless tests: add "Server died" check 2024-07-12 08:38:22 +02:00
Nikita Fomichev
4a56c601b2 Stateless tests: decrease CI timeout 2024-07-12 01:11:13 +02:00
Vitaly Baranov
355a56d1b0 Add a stateless test for gRPC protocol. 2024-07-11 15:36:41 +02:00
Kseniia Sumarokova
60c0957abd
Merge pull request #66290 from ClickHouse/kssenii-patch-10 
Update run.sh
 2024-07-11 10:45:37 +00:00
Antonio Andelic
fbb7a3ed52
Merge pull request #66281 from ClickHouse/collect-core-dumps-stateless 
Collect core dumps in more tests
 2024-07-11 09:45:17 +00:00
János Benjamin Antal
321a58c519 Fix relative path to mc 2024-07-11 08:33:10 +00:00
János Benjamin Antal
4729161f41 Collect logs from minio 2024-07-10 18:41:48 +00:00
Kseniia Sumarokova
e5020690be
Merge branch 'master' into kssenii-patch-10 2024-07-10 20:02:15 +02:00
Yarik Briukhovetskyi
052520fa03
Merge branch 'master' into hive_style_partitioning 2024-07-10 19:46:07 +02:00
Nikita Fomichev
481a1570aa Run sequential tests in parallel to non-sequential 2024-07-10 19:21:32 +02:00
Nikita Fomichev
2662760572 debug tests hung on gdb error 8 2024-07-10 16:38:54 +02:00
Nikita Fomichev
83cfe164bd debug tests hung on gdb error 7 2024-07-10 15:55:30 +02:00
Antonio Andelic
ccffc05681 Fix asan 2024-07-10 15:45:17 +02:00
Nikita Fomichev
ba176a9c4a debug tests hung on gdb error 6 2024-07-10 14:51:27 +02:00
Nikita Fomichev
ede43721b4 debug tests hung on gdb error 4 2024-07-10 13:30:06 +02:00
Kruglov Pavel
7b58722c07
Update setup_minio.sh 2024-07-10 13:15:44 +02:00
Antonio Andelic
1eec1fd77f Merge branch 'master' into collect-core-dumps-stateless 2024-07-10 13:01:41 +02:00
Kseniia Sumarokova
36b9a5641f
Update run.sh 2024-07-10 11:56:43 +02:00
Kseniia Sumarokova
ebc87d0c70
Update run.sh 2024-07-09 22:58:06 +02:00
Kruglov Pavel
d7f08ffdb7
Update setup_minio.sh 2024-07-09 21:01:37 +02:00
Kseniia Sumarokova
5a12659f43
Update run.sh 2024-07-09 20:18:03 +02:00
Nikita Fomichev
eeb3561c37 Stateless tests: run sequential tests in parallel to other tests 2024-07-09 19:41:06 +02:00
Yarik Briukhovetskyi
362bf4befc
Update setup_minio.sh 2024-07-09 17:30:08 +02:00
Antonio Andelic
0821d72f1c Collect core dumps in more tests 2024-07-09 17:06:37 +02:00
Yarik Briukhovetskyi
3cc6a133c6
Update setup_minio.sh 2024-07-09 12:38:16 +02:00
yariks5s
a3c4cbfce2 clang-tidy, fix cp with minio 2024-07-08 13:57:54 +00:00
Raúl Marín
18e2f0da9f Merge remote-tracking branch 'blessed/master' into fix_bugfix_checker 2024-07-08 11:23:47 +02:00