Commit Graph

4203 Commits

Author SHA1 Message Date
Nikita Mikhaylov
2b091983e8 Bump Azure to ea3e19a7be 2024-07-09 19:59:49 +00:00
Nikita Mikhaylov
0d54151cb8 Make the pocketfft to point to the upstream/master branch 2024-07-09 19:55:37 +00:00
Robert Schulze
55468caeae
Fix ARM build 2024-07-09 16:38:22 +00:00
Vitaly Baranov
32b9f7940f
Merge pull request #66191 from vitlibar/grpc-avoid-using-harmful-function-rand
Avoid using harmful function rand() in grpc
2024-07-09 13:42:20 +00:00
Robert Schulze
3f1c0b9966
Merge remote-tracking branch 'ClickHouse/master' into add_statistics_cmsketch 2024-07-09 10:48:19 +00:00
Kruglov Pavel
52b3d1a244
Merge pull request #66130 from slvrtrn/update-avro-submodule-with-array-block-size-fix
Update AVRO submodule with the array block size fix
2024-07-09 07:58:04 +00:00
Nikita Taranov
a9546fe551
Merge pull request #66147 from ClickHouse/fix_mac_build
Playing minesweeper with build system
2024-07-08 22:05:17 +00:00
Robert Schulze
bd42a096b7
Bump rocksdb to v6.29.5 2024-07-08 16:52:36 +00:00
Robert Schulze
1bfafa42be
Bump rocksdb to v6.28.2 2024-07-08 14:47:35 +00:00
Robert Schulze
9509802866
Bump rocksdb to v6.27.3 2024-07-08 13:57:03 +00:00
Robert Schulze
31f9bed442
Bump rocksdb to 6.26.1 2024-07-08 13:49:05 +00:00
Robert Schulze
adcee80b2d
Bump rocksdb to v6.25.3 2024-07-08 13:41:34 +00:00
Robert Schulze
312dd82425
Bump rocksdb to v6.24.2 2024-07-08 12:58:59 +00:00
Robert Schulze
068da21837
Merge pull request #66068 from rschu1ze/bump-rocksdb
Bump rocksdb to v6.23.3
2024-07-08 12:17:40 +00:00
Robert Schulze
5943d60f26
Merge pull request #66097 from azat/build/src-dir-fix
Avoid using source directory for generated files
2024-07-08 09:20:39 +00:00
Vitaly Baranov
7b19076ebe Avoid using harmful function rand() in grpc. 2024-07-07 22:31:45 +02:00
Vitaly Baranov
42945a6d4a
Merge pull request #66137 from vitlibar/switch-contrib-orc-to-better-commit
Switch submodule contrib/orc to a proper commit in the main branch.
2024-07-07 17:38:08 +00:00
Nikita Taranov
0c13289ea3
Merge branch 'master' into fix_mac_build 2024-07-07 16:31:49 +02:00
Yakov Olkhovskiy
6e5e680797 bump libprotobuf-mutator, fix build 2024-07-06 05:42:21 +00:00
Nikita Taranov
6e8edf008f try fix 2024-07-05 18:36:50 +02:00
Robert Schulze
72b3ebd180
Merge pull request #66136 from rschu1ze/bump-s2
Bump s2geometry again
2024-07-05 14:05:31 +00:00
Vitaly Baranov
40fd150279 Switch submodule contrib/orc to proper commit in the main branch. 2024-07-05 15:33:50 +02:00
Robert Schulze
d3f23c2753
Bump s2geometry again 2024-07-05 13:29:34 +00:00
slvrtrn
3ce470c57f Update AVRO submodule with the array block size fix 2024-07-05 14:42:43 +02:00
Vitaly Baranov
f315b36323
Merge pull request #66069 from vitlibar/add-prometheus-protocols-protobufs
Add protobufs for Prometheus remote-write/remote-read protocols
2024-07-05 10:41:04 +00:00
Robert Schulze
3621b35591
Merge pull request #66094 from rschu1ze/bump-s2geometry
Bump s2geometry to latest master
2024-07-05 10:30:20 +00:00
Robert Schulze
4dca031e9d
Merge pull request #66100 from rschu1ze/bump-azure
Bump Azure to 1.12
2024-07-05 09:11:00 +00:00
Vitaly Baranov
1ef5bca591 Fix cmake function PROTOBUF_GENERATE_CPP(): now it returns correct paths in SRCS and HDRS
even if input ".proto" files are located in sibling directories.
2024-07-04 23:07:19 +02:00
Vitaly Baranov
e428542b2e Add prometheus protobufs. 2024-07-04 23:06:14 +02:00
Antonio Andelic
ec6739120c
Merge pull request #66041 from ClickHouse/try-disabling-jemalloc-background-threads
Try disabling jemalloc background threads
2024-07-04 20:30:39 +00:00
Michael Kolupaev
1eb7ca7479 Merge remote-tracking branch 'origin/master' into zup 2024-07-04 19:25:23 +00:00
Robert Schulze
ffe1f8fea0
Bump Azure to 1.12 2024-07-04 19:24:30 +00:00
Robert Schulze
1d8389ddee
Bump rocksdb to v6.23.3 2024-07-04 19:19:34 +00:00
Michael Kolupaev
b04e6c806b Merge remote-tracking branch 'origin/master' into uwu 2024-07-04 19:02:47 +00:00
Robert Schulze
597810b69d
Fix s390x build 2024-07-04 18:46:09 +00:00
Robert Schulze
632f292fd1
Merge pull request #66082 from rschu1ze/bmp-vectorscan11
Bump vectorscan to 5.4.11
2024-07-04 18:11:10 +00:00
Robert Schulze
1bd515cc1f
Merge remote-tracking branch 'ClickHouse/master' into bump-s2geometry 2024-07-04 15:43:47 +00:00
Azat Khuzhin
c49d26bc23 Avoid using source directory for generated files
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-07-04 17:20:16 +02:00
Robert Schulze
11a30d6d60
Bump s2geometry to latest master 2024-07-04 14:20:06 +00:00
Antonio Andelic
613ed1ebbf Merge branch 'master' into try-disabling-jemalloc-background-threads 2024-07-04 13:58:26 +02:00
Antonio Andelic
c93d8cbb66 Fixes 2024-07-04 13:57:47 +02:00
Nikita Mikhaylov
e2a494fa60
Merge pull request #66064 from rschu1ze/mysql-openssl-tsan
Maybe fix tsan assert in `test_mysql_killed_while_insert_8_0`
2024-07-04 11:17:42 +00:00
Robert Schulze
33b7afc1b4
Bump vectorscan to 5.4.11 2024-07-04 10:30:24 +00:00
Robert Schulze
dbac2212a0
Merge pull request #66072 from rschu1ze/llvm-15-burning-some-trash
Clean-up custom LLVM 15 patches
2024-07-04 09:06:38 +00:00
Robert Schulze
43b088dbd0
Merge pull request #66056 from rschu1ze/bmp-vectorscan
Bump vectorscan to 5.4.10.1
2024-07-04 08:59:50 +00:00
JackyWoo
0c5821e5b8 Merge branch 'master' into add_statistics_cmsketch
# Conflicts:
#	docs/en/engines/table-engines/mergetree-family/mergetree.md
#	src/Storages/Statistics/Statistics.cpp
#	src/Storages/Statistics/Statistics.h
#	src/Storages/Statistics/StatisticsTDigest.h
#	src/Storages/Statistics/StatisticsUniq.h
#	src/Storages/Statistics/TDigestStatistics.cpp
#	tests/queries/0_stateless/02864_statistics_uniq.sql
2024-07-04 10:25:53 +08:00
Robert Schulze
eb7ab5128d
Clean-up custom LLVM 15 patches 2024-07-03 21:07:59 +00:00
Robert Schulze
9737c5bab4
Probably fix tsan assert in test_mysql_killed_while_insert_8_0 2024-07-03 17:38:53 +00:00
Robert Schulze
8b14754005
Fix ARM build (upgrade sysroot) 2024-07-03 16:29:47 +00:00
Robert Schulze
39a371b27d
Bump vectorscan 2024-07-03 15:07:31 +00:00
Antonio Andelic
5fd36059e4 Try disabling background threads 2024-07-03 15:28:01 +02:00
Michael Kolupaev
ab4d445656 Merge remote-tracking branch 'origin/master' into uwu 2024-07-02 04:37:46 +00:00
Azat Khuzhin
9c7db6711b Fix possible issues with MySQL client protocol TLS connections
Occasionally, 02479_mysql_connect_to_self fails on CI [1].

  [1]: https://github.com/ClickHouse/ClickHouse/issues/50911

The problem was indeed query profiler and EINTR, but not in a way you
may think.

For such failures you may see the following trace in trace_log:

    contrib/openssl/crypto/bio/bss_sock.c:127::sock_read
    contrib/openssl/crypto/bio/bio_meth.c:121::bread_conv
    contrib/openssl/crypto/bio/bio_lib.c:285::bio_read_intern
    contrib/openssl/crypto/bio/bio_lib.c:311::BIO_read
    contrib/openssl/ssl/record/methods/tls_common.c:398::tls_default_read_n
    contrib/openssl/ssl/record/methods/tls_common.c:575::tls_get_more_records
    contrib/openssl/ssl/record/methods/tls_common.c:1122::tls_read_record
    contrib/openssl/ssl/record/rec_layer_s3.c:645::ssl3_read_bytes
    contrib/openssl/ssl/s3_lib.c:4527::ssl3_read_internal
    contrib/openssl/ssl/s3_lib.c:4551::ssl3_read
    contrib/openssl/ssl/ssl_lib.c:2343::ssl_read_internal
    contrib/openssl/ssl/ssl_lib.c:2357::SSL_read
    contrib/mariadb-connector-c/libmariadb/secure/openssl.c:729::ma_tls_read
    contrib/mariadb-connector-c/libmariadb/ma_tls.c:90::ma_pvio_tls_read
    contrib/mariadb-connector-c/libmariadb/ma_pvio.c:250::ma_pvio_read
    contrib/mariadb-connector-c/libmariadb/ma_pvio.c:297::ma_pvio_cache_read
    contrib/mariadb-connector-c/libmariadb/ma_net.c:373::ma_real_read
    contrib/mariadb-connector-c/libmariadb/ma_net.c:427::ma_net_read
    contrib/mariadb-connector-c/libmariadb/mariadb_lib.c:192::ma_net_safe_read
    contrib/mariadb-connector-c/libmariadb/mariadb_lib.c:2138::mthd_my_read_query_result
    contrib/mariadb-connector-c/libmariadb/mariadb_lib.c:2212::mysql_real_query
    src/Common/mysqlxx/Query.cpp:56::mysqlxx::Query::executeImpl()
    src/Common/mysqlxx/Query.cpp:73::mysqlxx::Query::use()
    src/Processors/Sources/MySQLSource.cpp:50::DB::MySQLSource::Connection::Connection()

After which the connection will fail with:

    Code: 1000. DB::Exception: Received from localhost:9000. DB::Exception: mysqlxx::ConnectionLost: Lost connection to MySQL server during query (127.0.0.1:9004). (POCO_EXCEPTION)

But, if you will take a look at ma_tls_read() you will see that it has
proper retries for SSL_ERROR_WANT_READ (and EINTR is just a special case
of it), but still, for some reason it fails.

And the reason is the units of the read/write timeout, ma_tls_read()
calls poll(read_timeout) in case of SSL_ERROR_WANT_READ, but, it
incorrectly assume that the timeout is in milliseconds, but that timeout
was in seconds, this bug had been fixed in [2], and now it works like a
charm!

  [2]: https://github.com/ClickHouse/mariadb-connector-c/pull/17

I've verified it with patching openssl library:

    diff --git a/crypto/bio/bss_sock.c b/crypto/bio/bss_sock.c
    index 82f7be85ae..3d2f3926a0 100644
    --- a/crypto/bio/bss_sock.c
    +++ b/crypto/bio/bss_sock.c
    @@ -124,7 +124,24 @@ static int sock_read(BIO *b, char *out, int outl)
                 ret = ktls_read_record(b->num, out, outl);
             else
     # endif
    -            ret = readsocket(b->num, out, outl);
    +        {
    +            static int i = 0;
    +            static int j = 0;
    +            if (!(++j % 5))
    +            {
    +                fprintf(stderr, "sock_read: inject EAGAIN with ret=0\n");
    +                ret = 0;
    +                errno = EAGAIN;
    +            }
    +            else if (!(++i % 3))
    +            {
    +                fprintf(stderr, "sock_read: inject EAGAIN with ret=-1\n");
    +                ret = -1;
    +                errno = EAGAIN;
    +            }
    +            else
    +                ret = readsocket(b->num, out, outl);
    +        }
             BIO_clear_retry_flags(b);
             if (ret <= 0) {
                 if (BIO_sock_should_retry(ret))

And before this patch (well, not the patch itself, but the referenced
patch in mariadb-connector-c) if you will pass read_write_timeout=1 it
will fail:

    SELECT * FROM mysql('127.0.0.1:9004', system, one, 'default', '', SETTINGS connect_timeout = 100, connection_wait_timeout = 100, read_write_timeout=1)
    Code: 1000. DB::Exception: Received from localhost:9000. DB::Exception: mysqlxx::ConnectionLost: Lost connection to MySQL server during query (127.0.0.1:9004). (POCO_EXCEPTION)

But after, it always works:

    $ ch benchmark -c10 -q "SELECT * FROM mysql('127.0.0.1:9004', system, one, 'default', '', SETTINGS connection_pool_size=1, connect_timeout = 100, connection_wait_timeout = 100, read_write_timeout=1)"
    ^CStopping launch of queries. SIGINT received.

    Queries executed: 478.

    localhost:9000, queries: 478, QPS: 120.171, RPS: 120.171, MiB/s: 0.001, result RPS: 120.171, result MiB/s: 0.001.

    0.000%          0.014 sec.
    10.000%         0.058 sec.
    20.000%         0.065 sec.
    30.000%         0.073 sec.
    40.000%         0.079 sec.
    50.000%         0.087 sec.
    60.000%         0.089 sec.
    70.000%         0.091 sec.
    80.000%         0.095 sec.
    90.000%         0.100 sec.
    95.000%         0.102 sec.
    99.000%         0.105 sec.
    99.900%         0.140 sec.
    99.990%         0.140 sec.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-07-01 18:39:43 +02:00
Antonio Andelic
8f9beffc65 No jemalloc profiler for non-Linux 2024-06-28 16:04:45 +02:00
Michael Kolupaev
840edd99fa Apply https://github.com/ClickHouse/libunwind/pull/27 too 2024-06-27 21:53:23 +00:00
Michael Kolupaev
36f9b41e7d Merge remote-tracking branch 'origin/master' into uwu 2024-06-27 21:48:37 +00:00
Antonio Andelic
c9fa974472 Use background thread 2024-06-27 08:46:10 +02:00
Antonio Andelic
35820eaa7b Build jemalloc with profiler 2024-06-26 11:13:18 +02:00
JackyWoo
1601420dfd Merge branch 'master' into add_statistics_cmsketch 2024-06-25 10:05:15 +08:00
Alexey Milovidov
49634db3ba Update submodule 2024-06-25 02:51:57 +02:00
Alexey Milovidov
3b377fabe7 Update submodule 2024-06-25 02:49:40 +02:00
JackyWoo
15e20f56fa Add statistics cmsketch 2024-06-24 16:38:56 +08:00
Robert Schulze
605d51700a
Merge pull request #65529 from rschu1ze/openssl-temp-official-fix
OpenSSL: Replace temporary fix for unsynchronized access by official fix
2024-06-23 15:24:28 +00:00
Michael Kolupaev
ad246312d7 Switch to final commit hash 2024-06-21 23:05:41 +00:00
Michael Kolupaev
92ff09f303 There's no Unwind_AppleExtras.cpp anymore 2024-06-21 22:53:38 +00:00
Robert Schulze
0a60dc1672
OpenSSL: Replace temporary fix for unsynchronized access by official fix 2024-06-21 12:57:44 +00:00
Robert Schulze
df857ff17a
Bump re2 to latest HEAD 2024-06-21 10:34:30 +00:00
Michael Kolupaev
06781efcb7 Switch to a cleaner update, hopefully fix builds 2024-06-20 22:59:36 +00:00
Michael Kolupaev
8115926eb6 Fix ARM some more 2024-06-20 22:18:14 +00:00
Michael Kolupaev
62d6e3d339 Fix ARM 2024-06-20 22:18:14 +00:00
Michael Kolupaev
3ab8ba0d4a Update zlib-ng from 2.0.2 to 2.1.6 2024-06-20 22:18:14 +00:00
Michael Kolupaev
424224b202 Set _LIBUNWIND_REMEMBER_STACK_ALLOC explicitly just in case 2024-06-20 22:05:16 +00:00
Michael Kolupaev
8bb60101c4 Update libunwind to 18.1.7 2024-06-20 22:01:16 +00:00
Robert Schulze
fd52e19b3e
Suppress leaksan false positive in OpenSSL 2024-06-19 08:30:47 +00:00
Robert Schulze
abe424b24b
Merge pull request #65111 from rschu1ze/reenable-session-caching
Re-enable OpenSSL session caching
2024-06-18 16:39:38 +00:00
Alexander Tokmakov
b150b0f5fa
Revert "Fix AWS ECS" 2024-06-17 23:11:59 +02:00
Robert Schulze
cdcfcb99fc
Merge remote-tracking branch 'rschu1ze/master' into reenable-session-caching 2024-06-17 20:51:11 +00:00
Robert Schulze
186bd0cc3d
Temporary fix for tsan issue openssl#24629 2024-06-17 12:20:54 +00:00
Alexey Milovidov
2c1918908a Revert "Debug AWS"
This reverts commit 20ca8f8714.
2024-06-16 17:12:55 +02:00
Alexey Milovidov
b41bb96605 Revert "Debug AWS"
This reverts commit 642dc35a76.
2024-06-16 17:12:19 +02:00
Alexey Milovidov
37eb815b17 Fix error 2024-06-16 17:12:05 +02:00
Alexey Milovidov
642dc35a76 Debug AWS 2024-06-16 15:21:17 +02:00
Alexey Milovidov
20ca8f8714 Debug AWS 2024-06-16 15:20:26 +02:00
Alexey Milovidov
cb390cc3d9 Fix alloc/dealloc mismatch or prevent false-positive ASan error in the AWS SDK 2024-06-16 13:14:59 +02:00
Raúl Marín
0d8bd133e6
Merge pull request #64959 from canhld94/bump_fmt_9
Update fmtlib version to 9.1.0
2024-06-14 14:05:04 +00:00
Robert Schulze
d92a2f3ef6
Remove obsolete fix from aws submodule 2024-06-12 13:20:50 +00:00
Robert Schulze
97f96a7b84
Merge pull request #65125 from rschu1ze/bug-63792
Fix false positives leaky memory warnings in OpenSSL
2024-06-12 12:40:50 +00:00
Robert Schulze
92ca8b30b3
Merge pull request #65048 from rschu1ze/bump-absl
Bump abseil to latest HEAD
2024-06-12 08:26:58 +00:00
Robert Schulze
848054e85f
Fix another false positive leak 2024-06-11 21:06:45 +00:00
Robert Schulze
e0279e856f
Fix #63792 (hopefully) 2024-06-11 19:42:20 +00:00
Robert Schulze
af83bc92ce
Switch to same HEAD as before but with s390x-breaking commit reverted 2024-06-11 14:16:38 +00:00
Robert Schulze
e5dcf75968
Add forked abseil submodule back 2024-06-11 14:09:56 +00:00
Robert Schulze
e47bbfb7f2
Remove upstream abseil repository 2024-06-11 14:08:16 +00:00
Duc Canh Le
79a660c018 Merge branch 'master' into bump_fmt_9
Fix CI
2024-06-11 03:39:03 +00:00
Robert Schulze
2ff1e81af6
Merge remote-tracking branch 'rschu1ze/master' into bump-googletest 2024-06-10 16:50:53 +00:00
Robert Schulze
598219c57d
Minor update 2024-06-10 12:56:21 +00:00
Robert Schulze
8af077f3d3
Update build descriptions 2024-06-10 12:53:06 +00:00
Robert Schulze
da91dd6428
Bump absl to 2024-06-07 2024-06-10 11:35:07 +00:00
Robert Schulze
c95ed40d3e
Bump absl to 2024-05-06 2024-06-10 11:32:29 +00:00
Robert Schulze
643444eb11
Bump absl to 2024-05-03 2024-06-10 10:41:47 +00:00
Robert Schulze
61b4643217
Bump absl to 2024-04-24 2024-06-10 10:39:35 +00:00
Robert Schulze
ae7d8821a7
Bump absl to 2024-04-04 2024-06-10 08:46:40 +00:00
Robert Schulze
d4a453aad5
Bump absl to 2024-03-06 2024-06-10 08:32:22 +00:00
Robert Schulze
a0d8d5a37c
Bump absl to 2024-01-02 2024-06-10 08:31:33 +00:00
Robert Schulze
8fe272f210
Bump absl to 2024-01-02 2024-06-10 08:22:27 +00:00
Robert Schulze
1bca6b900b
Bump absl to 2023-12-20 2024-06-10 08:20:25 +00:00
Robert Schulze
70c0589675
Bump absl to 2023-12-12 2024-06-10 08:19:20 +00:00
Robert Schulze
4d3d18cee7
Bump absl to 2023-12-06 2024-06-10 08:18:36 +00:00
Robert Schulze
c6e43f7a7b
Bump absl to 2023-11-28 2024-06-10 08:17:01 +00:00
Robert Schulze
163cacf701
Bump googletest 2024-06-10 07:53:29 +00:00
Raúl Marín
f50a951e8e Fix innocuous data race in detectLanguage 2024-06-07 16:49:07 +02:00
Duc Canh Le
de5258128e update fmtlib version to 9.1.0
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-06-07 06:44:36 +00:00
Michael Kolupaev
fd93097130 Fix writing ORC statistics for unsigned types 2024-06-06 06:39:07 +00:00
Sema Checherinda
6b8ca302d2 test for TotalQpsLimitExceeded 2024-05-28 17:58:32 +02:00
Sema Checherinda
ae4d3f97ae
Merge pull request #64225 from ClickHouse/chesema-patch-aws-error-handler
tests for qps_limit_exceeded
2024-05-24 13:28:33 +00:00
Sema Checherinda
6be79a35b6 update contrib/aws to the last head 2024-05-22 20:30:19 +02:00
Sema Checherinda
a73d60bae5 tests for qps_limit_exceeded 2024-05-22 18:35:28 +02:00
ZhiHong Zhang
4b1c9adb3a
Merge branch 'ClickHouse:master' into gcmaster-parquet 2024-05-22 09:33:01 +08:00
copperybean
b253ca3608 fix clang-tidy warnings
Change-Id: Iff9f5f894e815b184ac35f61b4cac87908c612b5
2024-05-21 16:48:49 +08:00
Alexey Milovidov
29f7a766a5 Merge branch 'master' into libunwind-fix-crash 2024-05-20 19:49:37 +02:00
Alexey Milovidov
dae050d2d4
Merge pull request #64023 from azat/build/freebsd-bcrypt-fix
Fix libbcrypt for FreeBSD build
2024-05-20 09:51:39 +02:00
Alexey Milovidov
27e7086be2 Merge branch 'master' into libunwind-fix-crash 2024-05-20 05:15:52 +02:00
alesapin
3878155b19
Merge pull request #64072 from ClickHouse/try_to_fix_cross_compilation_for_grpc
Try to fix grpc for aarch64 crosscompilation
2024-05-18 18:28:21 +00:00
alesapin
086e5d73a4 Followup 2024-05-18 18:03:55 +02:00
alesapin
2983941ab8 Fix build for openssl
(cherry picked from commit 7efd5d3ab62024619e26dc6c3e28d50bffc98d70)
2024-05-18 16:35:15 +02:00
Alexey Milovidov
a2d9a32652
Merge pull request #63733 from qiangxuhui/qiangxuhui-loongarch64
Initial support for loongarch64
2024-05-18 05:26:26 +02:00
alesapin
24892b151a Try to fix grpc for aarch64 crosscompilation
(cherry picked from commit f3fbf532e0d4d7616f51a9c3d5087cf7b2e6d7d5)
2024-05-17 22:55:12 +02:00
Alexey Milovidov
b9d539d0f7 Merge branch 'master' into libunwind-fix-crash 2024-05-17 20:33:22 +02:00
Alexey Milovidov
e39d8c001f
Merge pull request #63923 from ClickHouse/update-datasketches
Update Datasketches
2024-05-17 20:30:41 +02:00
Azat Khuzhin
bfb3fe0c23 Fix libbcrypt for FreeBSD build
Right now it fails due to [1] with the following error:

    /usr/work/ClickHouse/contrib/libbcrypt/crypt_blowfish/ow-crypt.h:27:14: error: conflicting types for 'crypt_r'
       27 | extern char *crypt_r(__const char *key, __const char *setting, void *data);
          |              ^
    /usr/include/unistd.h:500:7: note: previous declaration is here
      500 | char    *crypt_r(const char *, const char *, struct crypt_data *);
          |          ^

  [1]: 5f521d7ba7

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-17 11:37:01 +02:00
Alexey Milovidov
cb9d35e490 Merge branch 'master' into update-datasketches 2024-05-17 10:23:50 +02:00
Robert Schulze
5fb8ea4c62
Merge remote-tracking branch 'ClickHouse/master' into qiangxuhui-loongarch64 2024-05-17 07:56:01 +00:00
qiangxuhui
7d6d5165fe Update contrib/libunwind to master 2024-05-17 07:45:51 +00:00
Alexey Milovidov
bd6a4a3068 Merge branch 'master' into libunwind-fix-crash 2024-05-17 07:24:39 +02:00
Alexey Milovidov
50dea75fff Update libunwind 2024-05-17 07:15:51 +02:00
Alexey Milovidov
95a452f87c Update to the latest version where the patch is included 2024-05-17 05:41:32 +02:00
Alexey Milovidov
4b73ae8340
Merge pull request #63946 from azat/build/libunwind-fixes
Insignificant libunwind build fixes
2024-05-17 03:34:53 +02:00
Azat Khuzhin
86cf1e13d8 libunwind: fix usage of libunwind.h (by defining -D_LIBUNWIND_IS_NATIVE_ONLY)
From this macros sizeof(unw_context_t)/sizeof(unw_cursor_t) is depends
(_LIBUNWIND_CONTEXT_SIZE/_LIBUNWIND_CURSOR_SIZE).

So it should be not only PRIVATE but for INTERFACE as well.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-16 13:11:47 +02:00
Azat Khuzhin
793a11fd19 libunwind: remove useless _DEBUG flag
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-16 13:11:47 +02:00
Alexey Milovidov
91c240c85c Update Datasketches 2024-05-16 00:43:38 +02:00
Azat Khuzhin
94041d193c Fix SIGSEGV due to CPU/Real profiler
The problem was due to incorrect unwinding due from signal handlers,
which leads to incorrect DWARF (FDE/CIE) interpretation.

After this patch I was not able to reproduce the crash for couple of
hours, while before it was very stable (I've reduced the minimal
threshold for query_profiler_real_time_period_ns), using simply:

    $ clickhouse-benchmark --port 19000 -q "SELECT * FROM remote('127.{1..10}', system, one)" --query_profiler_real_time_period_ns=1

Note, I'm using here remote() for fibers, that has stack with guard
pages that helps with reproducing the crash more faster.

P.S. I also have another implementation of this fix, without patching
unwind and using info from signal context directly, and even though it
is better, because you don't need to trip extra frames and you can use
all the 45 frames for something useful, it is too complex, so let's go
with a simpler patch first, and I think it could be even backported.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-15 18:22:59 +02:00
qiangxuhui
47b7d3748d Fix code for loongarch64
Fix `base/poco` and `contrib/openssl-cmake` based on code review comments(https://github.com/ClickHouse/ClickHouse/pull/61509).
2024-05-14 07:45:57 +00:00
qiangxuhui
eb96f19e48 Add support for building with openssl for loongarch64 2024-05-14 07:45:57 +00:00
qiangxuhui
97a5de9653 Initial support for loongarch64
Make ClickHouse compilable and runnable on loongarch64

So far only basic functionality was tested (on real hw),
clickhouse server runs, exceptions works, client works,
simple tests works.
2024-05-14 07:45:26 +00:00
Vitaly Baranov
6c0450f8ff
Merge pull request #62685 from azat/yaml-cpp-merge-key
Bump yaml-cpp for YAML Merge Key support
2024-05-13 09:49:25 +00:00
Azat Khuzhin
59adb61016 Bump yaml-cpp after PR merge
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-10 20:05:51 +02:00
Azat Khuzhin
a0843f754a Bump yaml-cpp with proper keys merging PR
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-10 20:04:23 +02:00
Azat Khuzhin
439229ef2b Bump yaml-cpp for YAML Merge Key support
Refs: https://github.com/ClickHouse/yaml-cpp/pull/2
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-10 20:04:23 +02:00
Alexey Milovidov
dd58af7d4f Merge branch 'master' of github.com:ClickHouse/ClickHouse into clang-18-ci 2024-05-10 07:17:39 +02:00
Raúl Marín
2f00ad489a Go back to upstream lz4 2024-05-09 17:06:26 +02:00
Alexey Milovidov
224e1940ae Useless changes 2024-05-09 03:58:34 +02:00
Alexey Milovidov
9f6eb467af Merge branch 'master' into libunwind-fix-crash 2024-05-08 05:08:15 +02:00
Azat Khuzhin
95b76bf6a4 Remove leftovers of GCC support in cmake rules
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-07 21:07:02 +02:00
Alexey Milovidov
875507c921 Merge branch 'master' into libunwind-fix-crash 2024-05-03 03:41:36 +02:00
Robert Schulze
15e67fee78
Merge remote-tracking branch 'ClickHouse/master' into hlee-z-build-fix 2024-05-02 07:02:24 +00:00
HarryLeeIBM
be72040922 Use mold linker for s390x 2024-04-30 10:40:00 -07:00
Anton Popov
1c8a2ae2eb fix possible endless loop while reading from azure 2024-04-30 16:14:37 +00:00
HarryLeeIBM
bf3c8b6966 Fix broken s390x build 2024-04-29 15:37:10 -07:00
Robert Schulze
422dc8a2e5
Fix race in X509 store in OpenSSL (#63049) 2024-04-29 11:32:57 +00:00
Vitaly Baranov
860c642535 fix-curl-cmake 2024-04-26 18:58:20 +02:00
Vitaly Baranov
d49fcda09e Unfork and update curl to 8.7.1 2024-04-26 18:58:13 +02:00
Robert Schulze
c91f8ffc39
Unflake 02813_func_now_and_alias.sql 2024-04-24 18:34:59 +00:00
HarryLeeIBM
2f0fb73d7d Add back two fips c files under condition 2024-04-24 07:15:16 -07:00
HarryLeeIBM
f323b68675 Remove unused c files in build 2024-04-24 06:28:40 -07:00
HarryLeeIBM
69b259b409 Fix build for openssl dynamic linking 2024-04-23 16:34:00 -07:00
Robert Schulze
c3f41894f2
Follow-up to #62700: Fix build when $CC isn't set 2024-04-18 10:57:43 +00:00
Robert Schulze
8f1e4d4df8
Merge remote-tracking branch 'rschu1ze/master' into azure-1.11-this-time-really 2024-04-17 14:54:05 +00:00
Raúl Marín
5849aeb8c3
Merge pull request #62297 from Algunenano/rust_vendor
Vendor rust dependencies
2024-04-17 12:36:27 +00:00
Robert Schulze
d3c7615d19
Bump Azure to v1.11 2024-04-17 12:10:12 +00:00
Robert Schulze
bc538f7c6a
Bump Azure to v1.11 2024-04-17 09:09:05 +00:00
Robert Schulze
0fbfe24f73
Merge remote-tracking branch 'rschu1ze/master' into bump-azure-1.8 2024-04-17 08:14:20 +00:00
Robert Schulze
128ecf3d1c
Merge pull request #62700 from rschu1ze/cc-not-set
Fix build when `$CC` isn't set
2024-04-16 14:38:31 +00:00
Robert Schulze
815d7e64d2
Bump Azure to 1.8 2024-04-16 14:36:33 +00:00
Robert Schulze
da225647a4
Add comment 2024-04-16 14:16:02 +00:00
Robert Schulze
cc54479d9f
Fix build when $CC isn't set 2024-04-16 14:10:38 +00:00
Robert Schulze
4e881be128
Merge pull request #62634 from liuneng1994/std-except-parseDateTIme
Speed up `parseDateTime[InJodaSyntax]Or(Null|Zero)` on invalid inputs
2024-04-16 13:42:10 +00:00
Alexey Milovidov
e65dc63609
Merge branch 'master' into rust_vendor 2024-04-16 10:10:34 +02:00
Robert Schulze
01c55e69e6
Merge pull request #59870 from rschu1ze/be-less-boring-32
boringssl --> OpenSSL 3.2
2024-04-16 06:59:08 +00:00
Robert Schulze
7dc05d092b
Merge remote-tracking branch 'rschu1ze/master' into be-less-boring-32 2024-04-15 16:35:10 +00:00
Robert Schulze
3c35f14804
Merge remote-tracking branch 'ClickHouse/master' into mkmkme/protobuf-25.1 2024-04-15 12:38:59 +00:00
Raúl Marín
69245545fa
Merge branch 'master' into rust_vendor 2024-04-15 10:57:01 +01:00
Robert Schulze
a98cba351d
Cosmetics 2024-04-15 08:37:26 +00:00
LiuNeng
3d87a887ec
Merge branch 'master' into std-except-parseDateTIme 2024-04-15 15:51:05 +08:00
Robert Schulze
bff31998f2
Add expected submodule 2024-04-12 14:29:39 +00:00
Robert Schulze
dbc04c84dc
Merge pull request #62177 from rschu1ze/double-conversion
Bump `double-conversion` submodule
2024-04-12 13:28:33 +00:00
Alexey Milovidov
9b75f93470 Merge branch 'master' into libunwind-fix-crash 2024-04-12 02:08:24 +02:00
Raúl Marín
4e055c258f Update to final sysroot 2024-04-11 18:52:12 +02:00
Raúl Marín
0273b39804 Merge remote-tracking branch 'blessed/master' into rust_vendor 2024-04-10 22:05:20 +02:00
Raúl Marín
d16123bdf4 Temp: Update musl sysroot 2024-04-10 16:45:11 +02:00
Kruglov Pavel
d70b6220e0
Merge pull request #61720 from Avogar/try-fix-arrow-abort
Try to fix abort in arrow
2024-04-10 12:43:09 +00:00
Robert Schulze
8167b774b5
Merge pull request #62307 from rschu1ze/cleanup-ssh
Cleanup SSH-based authentication code
2024-04-09 21:10:09 +00:00
Raúl Marín
c6d3f1caa9
Merge pull request #62397 from Algunenano/phony
Fix one phony case
2024-04-09 18:15:00 +00:00
Raúl Marín
9afe5c4b24 Remove windows dependencies from vendor crates 2024-04-09 19:25:24 +02:00
Raúl Marín
26469c9c10 Update rustc to nightly-2024-04-01 2024-04-09 19:16:46 +02:00
Raúl Marín
88b11f7352 Add rust sanitizer support with vendored deps 2024-04-09 17:15:15 +02:00
Raúl Marín
e274bd7b64 Use a rust workspace
This fixes recompilation and avoids duplication of dependencies
2024-04-09 00:50:40 +02:00
Raúl Marín
75aff7fc1a Remove the code 2024-04-08 19:27:46 +02:00
Raúl Marín
b6eef61378 Fix one phony case 2024-04-08 14:37:13 +02:00
Robert Schulze
70f7f54064
Fix 01747_system_session_log_long 2024-04-08 11:04:49 +00:00
Robert Schulze
9400fee0e4
Fix leak-san false positive
This commit adresses below leaksan find.

I tried to reproduce this locally but 02802_clickhouse_disks_s3_copy.sh
but couldn't. clickhouse-disks did not do anything useful (it would not
even print logging), neither with a standard build nor with a leaksan
build. Could not find further documentation of it or even what this tool
is supposed to do, perhaps it is just for internal use. Also,
- line numbers in the leaksan report were partially missing,
- I am not really sure how Sha256HMACOpenSSLImpl::Calculate is calling
  into hmac_init (there must be some sort of static initialization
  somewhere but I did not find it), and
- my fix is in a weird place due to other restrictions (see the commit
  in the aws-sdk-cpp contrib repo).

The chance that this fix fixes the leak are low.
If it doesn't work, add "# Tag no-asan" + a comment to
02802_clickhouse_disks_s3_copy.sh and don't worry further.

EDIT: The commit fixes the issue, everything is good.

https://s3.amazonaws.com/clickhouse-test-reports/59870/b452e3d1ab87b8cc5810693aeea28f69ad28d671/stateless_tests__asan__[3_4].html

2024-03-19 08:34:03 =================================================================
2024-03-19 08:34:03 ==13149==ERROR: LeakSanitizer: detected memory leaks
2024-03-19 08:34:03
2024-03-19 08:34:03 Direct leak of 904 byte(s) in 1 object(s) allocated from:
2024-03-19 08:34:03     #0 0x55f9cb5a18ee in malloc (/usr/bin/clickhouse+0xa9a48ee) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     #1 0x55fa01e34070 in CRYPTO_malloc build_docker/./contrib/openssl/crypto/mem.c:202:11
2024-03-19 08:34:03     #2 0x55fa01e34070 in CRYPTO_zalloc build_docker/./contrib/openssl/crypto/mem.c:222:11
2024-03-19 08:34:03     #3 0x55fa01d6dcca in ossl_err_get_state_int build_docker/./contrib/openssl/crypto/err/err.c:691:17
2024-03-19 08:34:03     #4 0x55fa01d71748 in ERR_set_mark build_docker/./contrib/openssl/crypto/err/err_mark.c:19:10
2024-03-19 08:34:03     #5 0x55fa01f4735b in ossl_prov_digest_load_from_params build_docker/./contrib/openssl/providers/common/provider_util.c:194:5
2024-03-19 08:34:03     #6 0x55fa01ff467a in hmac_set_ctx_params build_docker/./contrib/openssl/providers/implementations/macs/hmac_prov.c:307:10
2024-03-19 08:34:03     #7 0x55fa01ff3ef2 in hmac_init build_docker/./contrib/openssl/providers/implementations/macs/hmac_prov.c:169:37
2024-03-19 08:34:03     #8 0x55f9fb83c75b in Aws::Utils::Crypto::Sha256HMACOpenSSLImpl::Calculate(Aws::Utils::Array<unsigned char> const&, Aws::Utils::Array<unsigned char> const&) (/usr/bin/clickhouse+0x3ac3f75b) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     #9 0x55f9fb82ebeb in Aws::Utils::Crypto::Sha256HMAC::Calculate(Aws::Utils::Array<unsigned char> const&, Aws::Utils::Array<unsigned char> const&) (/usr/bin/clickhouse+0x3ac31beb) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     #10 0x55f9fb6d5afd in Aws::Client::AWSAuthV4Signer::ComputeHash(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const (/usr/bin/clickhouse+0x3aad8afd) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     #11 0x55f9fb6e0bad in Aws::Client::AWSAuthV4Signer::SignRequest(Aws::Http::HttpRequest&, char const*, char const*, bool) const (/usr/bin/clickhouse+0x3aae3bad) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     #12 0x55f9fb72651d in bool smithy::components::tracing::TracingUtils::MakeCallWithTiming<bool>(std::__1::function<bool ()>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, smithy::components::tracing::Meter const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>>&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) (/usr/bin/clickhouse+0x3ab2951d) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     #13 0x55f9fb70adcb in Aws::Client::AWSClient::AttemptOneRequest(std::__1::shared_ptr<Aws::Http::HttpRequest> const&, Aws::AmazonWebServiceRequest const&, char const*, char const*, char const*) const (/usr/bin/clickhouse+0x3ab0ddcb) (BuildId: b3766b865d6580f6dcba75acf37673d4aeedc2b6)
2024-03-19 08:34:03     #14 0x55f9fb6fdb17 in Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char cons
2024-04-08 11:04:43 +00:00