Commit Graph

5146 Commits

Author SHA1 Message Date
divanik
52de06514a Merge branch 'master' of github.com:ClickHouse/ClickHouse into divanik/add_local_and_azure_iceberg_support 2024-08-12 09:14:39 +00:00
Alexey Milovidov
2a12604cf5
Merge pull request #66494 from azat/gdb-image
Update gdb to 15.1 (by compiling from sources)
2024-08-12 05:04:57 +00:00
Antonio Andelic
34fb169aa7 Merge branch 'master' into update-minio-statless 2024-08-09 11:50:17 +02:00
divanik
b67b7a62a4 Merge branches 'divanik/add_local_and_azure_iceberg_support' and 'master' of github.com:ClickHouse/ClickHouse into divanik/add_local_and_azure_iceberg_support 2024-08-09 09:40:32 +00:00
Alexey Milovidov
917920c59e
Merge pull request #67737 from azat/tests-processes-leftovers
Smart handling of processes leftovers in tests
2024-08-09 05:21:12 +00:00
robot-clickhouse
9cf60214e1 Update version_date.tsv and changelogs after v24.7.3.42-stable 2024-08-08 13:43:12 +00:00
Antonio Andelic
4059b45237 Merge branch 'master' into update-minio-statless 2024-08-08 09:00:27 +02:00
Antonio Andelic
ddc058aa6f Update minio in stateless tests 2024-08-07 14:01:46 +02:00
divanik
299af052f8 Merge branch 'master' of github.com:ClickHouse/ClickHouse into divanik/add_local_and_azure_iceberg_support 2024-08-07 11:13:53 +00:00
Raúl Marín
c9340cba32
Merge pull request #67622 from Algunenano/unit_test_asan
Don't run ASAN unit tests under gdb
2024-08-07 10:48:00 +00:00
max-vostrikov
0aed98f3f0
Merge pull request #67596 from ClickHouse/test_parquet_page_index
added tests for page index in parquet files
2024-08-07 00:15:07 +00:00
Alexey Milovidov
e3ead8a3ba
Merge pull request #65564 from azat/override-logs-in-keeper-image
[RFC] Do not pass logs for keeper explicitly in the image to allow overriding
2024-08-06 22:58:20 +00:00
Azat Khuzhin
5390d1b108 docker: use self-compiled gdb 15 from cctools
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-06 17:15:49 +02:00
Azat Khuzhin
72bd43a309 tests: do not capture client stacktraces in stress tests
They are too uncontrollable, and likely will leave some clients [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/67737/9658be5eea8351655dd3ea77b8c1d4717bac7999/stress_test__ubsan_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-06 16:42:19 +02:00
Azat Khuzhin
7c366a040f ci: use bash arrays to pass opts to clickhouse-test for stateless/stateful
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-06 16:39:16 +02:00
Nikolay Degterinsky
1126ae3e5a
Merge pull request #67560 from evillique/fix-timeout-stateless-tests
[CI] Check for timeout when we stop logs replication
2024-08-06 14:35:23 +00:00
Azat Khuzhin
424f19d6c0 Move gdb into cctools
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-06 16:25:56 +02:00
Azat Khuzhin
459dd1ff2d Add a comment of the image name for cctools
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-06 16:25:56 +02:00
Azat Khuzhin
99b18d31db Update gdb to 15.1 (by compiling from sources)
Right now there are couple of gdb bugs that makes CI unstable:
- https://sourceware.org/bugzilla/show_bug.cgi?id=29185
- https://bugzilla.redhat.com/show_bug.cgi?id=1492496

But ubuntu 22.04 does not have 14+ anywhere, the
~ubuntu-toolchain-r/test contains only gdb 13, so there is no other
options except for compiling it from sources.

But there also other reasons to update it - optimizations, looks like
older gdb versions does not use index fully - 5.6sec vs 56sec:

    # 15.1
    $ time command gdb -batch -ex 'disas main' clickhouse
    ...
    real    0m5.692s
    user    0m29.948s
    sys     0m1.190s

    # 12.1 (from ubuntu 22.04)
    real    0m56.709s
    user    0m59.307s
    sys     0m0.585s

Also note, that we cannot compile gdb in the fasttest (that contains
compiler) since some images does not includes full toolchain, for
instance gdb is added in the following images:
- test-util -> test-base -> lots of other images (no toolchain)
- performance-comparison (no toolchain)
- integration-test (no toolchain)
- integration-tests-runner (no toolchain)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-06 16:25:56 +02:00
divanik
c4e29466de qMerge branch 'master' of github.com:ClickHouse/ClickHouse into divanik/add_local_and_azure_iceberg_support 2024-08-06 11:59:47 +00:00
Nikolay Degterinsky
103ff7d2f9 Merge remote-tracking branch 'upstream/master' into fix-timeout-stateless-tests 2024-08-06 10:23:57 +00:00
Nikita Mikhaylov
b4a6f249ab
Merge pull request #66438 from ClickHouse/analyzer-beta
Analyzer is finally Beta
2024-08-06 09:39:29 +00:00
Raúl Marín
25fa63f7e6 Merge remote-tracking branch 'blessed/master' into unit_test_asan 2024-08-06 11:19:57 +02:00
Michael Kolupaev
f344d833fd
Merge pull request #65253 from ClickHouse/reg
attach_gdb.lib: print more information before all stacks
2024-08-06 08:18:46 +00:00
Alexey Katsman
913bc8ab3a
Merge pull request #67869 from alexkats/misc-fixes
Misc fixes
2024-08-06 04:10:41 +00:00
Michael Kolupaev
93afcd16b0 Merge remote-tracking branch 'origin/master' into reg 2024-08-05 22:10:56 +00:00
Nikolay Degterinsky
5d0c179ad1 Merge remote-tracking branch 'upstream/master' into fix-timeout-stateless-tests 2024-08-05 19:38:09 +00:00
Azat Khuzhin
618789d196 Do not pass logs for keeper explicitly in the image to allow overriding
Right now, if you will start keeper like this:

    $ cat /tmp/local.yaml
    ---
    logger:
        log:
            "@remove": remove
        errorlog:
            "@remove": remove
        console: 1
    $ docker run --name keeper -v /tmp/local.yaml:/etc/clickhouse-keeper/conf.d/local.yaml --rm -it clickhouse/clickhouse-keeper

This will still not work, because the --log-file/--errorlog-file passed
explicitly.

So this patch removes this in case config is valid, but, if keeper is
launched with embedded config, this had been left as-is, to keep
previous behavior.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-05 20:28:13 +02:00
Alex Katsman
c8805fbced Misc fixes 2024-08-05 17:41:47 +00:00
divanik
9e85f89e95 Merge branch 'master' of github.com:ClickHouse/ClickHouse into divanik/add_local_and_azure_iceberg_support 2024-08-05 16:15:28 +00:00
Nikita Mikhaylov
4e9c3baa25 Move analyzer to Beta stage 2024-08-05 15:29:14 +00:00
Alexey Milovidov
a416d2bc4a
Merge pull request #67791 from azat/ci/stateless-v2
ci/stateless: fix artifacts post-processing and detect if something failed there
2024-08-05 14:46:58 +00:00
Alexey Milovidov
f1a096e8f6
Merge pull request #67723 from ClickHouse/fix-log-and-exception-messages-formatting
Fix test `00002_log_and_exception_messages_formatting`
2024-08-05 12:34:33 +00:00
Alexey Milovidov
5875900189
Merge pull request #67780 from ClickHouse/fix-small-diagnostic-in-test
Fix diagnostics in the test script
2024-08-05 12:27:37 +00:00
Alexey Milovidov
6b4353b910
Merge pull request #66495 from azat/ci-images-llvm-cleanup
Ensure that llvm-symbolizer is used for symbolizing sanitizer reports
2024-08-05 12:25:32 +00:00
Azat Khuzhin
b47f8a733f ci: fix basic errors collecting after stateless tests
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-05 11:16:08 +02:00
Alexey Milovidov
a573b2926e Fixes for the script 2024-08-04 23:59:44 +02:00
Alexey Milovidov
0bdaa57023 Fix diagnostics in the test script 2024-08-04 23:53:54 +02:00
Azat Khuzhin
43cf85ef28 ci: collect basic issues in stateless tests (dmesg, fatal and similar)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-04 23:19:33 +02:00
Azat Khuzhin
c200f43774 ci: fix stateless runner to correctly gather artifacts when server failed
Right now it simply does not work due to "set -e", with it you cannot
use "foo=$(false)" since bash will break execution after, rewrite it to
a plain shell, with "if".

Also use ZSTD everywhere (ugh)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-04 23:19:33 +02:00
Alexey Milovidov
a433115434 Fix typo 2024-08-04 19:18:00 +02:00
Azat Khuzhin
1f2bb6676f Remove outdated comments about workaround for #64086
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-04 16:14:05 +02:00
Azat Khuzhin
dea95e6c33 Ensure that llvm-symbolizer is used for symbolizing sanitizer reports
Since you don't want to fallback to addr2line:

    # addr2line
    $ time /bin/test
    set_flag_impl: Success
    set_flag_if: Success
    /usr/bin/addr2line: DWARF error: invalid or unhandled FORM value: 0x23
    ==================
    WARNING: ThreadSanitizer: data race (pid=18)
    ...
    real    3m8.580s
    user    0m21.967s
    sys     0m40.628s

    # llvm-symbolizer
    $ time ./test
    set_flag_impl: Success
    set_flag_if: Success
    ==================
    WARNING: ThreadSanitizer: data race (pid=24884)
    real    0m0.028s
    user    0m0.003s
    sys     0m0.006s

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-08-04 16:14:05 +02:00
Kseniia Sumarokova
881ac7208e
Merge pull request #67361 from ClickHouse/add-no-distributed-cache-tag
Add no-distributed-cache tag in tests
2024-08-04 12:36:31 +00:00
Alexey Milovidov
fcb0ce7361 Fix docs build 2024-08-02 18:17:32 +02:00
Raúl Marín
86bab5a78a Unit test: Don't return ok if leaks are detected 2024-08-02 16:09:43 +02:00
Kseniia Sumarokova
d8d5d45412
Merge branch 'master' into add-no-distributed-cache-tag 2024-08-02 14:50:43 +02:00
Raúl Marín
e5cf376c4f Don't attach gdb in ASAN runs of fuzzer, stateless or stress checks 2024-08-02 12:56:49 +02:00
Raúl Marín
97f1f6e221 Don't hide errors on clickhouse local runs 2024-08-02 12:46:37 +02:00
Nikolay Degterinsky
2e7a15df89
Update setup_export_logs.sh 2024-08-01 23:23:06 +02:00