Commit Graph

4161 Commits

Author SHA1 Message Date
Azat Khuzhin
669ce7cd40 Enable jemalloc heap profiler by default for debug builds as well
Since the issue with jemalloc assertion is clear, let's revert that
workaround

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-07-14 08:21:13 +02:00
Azat Khuzhin
22611781b0 Fix jemalloc assertion due to non-monotonic CLOCK_MONOTONIC_COARSE
Recently one tricky assertion of jemalloc had been discovered [1]:

    Failed assertion: "nstime_compare(&decay->epoch, new_time) <= 0"

  [1]: https://github.com/ClickHouse/ClickHouse/issues/66193

And as it turns out it is really possible for CLOCK_MONOTONIC_COARSE to
go backwards, in a nutshell it can be done with ADJ_FREQUENCY, you can
find example here [2]. And I can't trigger this issue for non-coarse
clocks.

  [2]: https://gist.github.com/azat/7ea7f50ed75591b1af2d675a240ea94c?permalink_comment_id=5119222#gistcomment-5119222

But, jemalloc do not call clock_gettime() that frequently (I've verified
it), so it can use non-coarse version - CLOCK_MONOTONIC

I've also measured the latency of CLOCK_MONOTONIC and
CLOCK_MONOTONIC_COARSE, and it is 20ns vs 4ns per call [3], so make this
change affect performance you need really frequently calls of
clock_gettime.

  [3]: https://gist.github.com/azat/622fa1f9a5d8e7d546ee9d294501961d?permalink_comment_id=5119245#gistcomment-5119245

Interesting, that this bug started to appears only after jemalloc heap
profiler had been enabled by default [4], no clue why (I would believe
more in a more frequent calls to clock_adjtime(ADJ_FREQUENCY), but I
can't verify this)

  [4]: https://github.com/ClickHouse/ClickHouse/pull/65702

To be continued...

Fixes: https://github.com/ClickHouse/ClickHouse/issues/66193
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-07-14 08:21:13 +02:00
Yakov Olkhovskiy
698b5ce7fa Merge branch 'master' into ci-fuzzer-enable 2024-07-14 05:31:56 +00:00
Robert Schulze
264d7d760f
Bump rocksdb to 7.0.4 2024-07-13 16:43:39 +00:00
Nikita Mikhaylov
8182ec8762
Merge pull request #66292 from ClickHouse/bump-azure
Bump `azure-sdk-for-cpp`
2024-07-11 12:58:50 +00:00
Nikita Mikhaylov
9c10d4f402
Merge pull request #66291 from ClickHouse/bump-pocketfft
Bump `pocketfft`
2024-07-11 11:50:21 +00:00
Robert Schulze
1ebe1da7b9
Merge pull request #66216 from rschu1ze/bump-rocksdb2
Bump RocksDB
2024-07-11 09:52:59 +00:00
Azat Khuzhin
e6e8b065bc Fix possible deadlock for jemalloc with enabled profiler
ClickHouse uses JEMALLOC_PROF_LIBGCC - _Unwind_Backtrace() for jemalloc,
which calls _Unwind_Backtrace() during bootstrap of jemalloc, and if,
dlsym() has allocations it will lead to deadlock:

    0  __lll_lock_wait (futex=futex@entry=0x16516410 <init_lock+64>, private=0) at lowlevellock.c:52
    1  0x00007ffff7f9d0a3 in __GI___pthread_mutex_lock (mutex=0x16516410 <init_lock+64>) at ../nptl/pthread_mutex_lock.c:80
    2  0x000000000ba6ec23 in pthread_mutex_lock (arg=arg@entry=0x16516410 <init_lock+64>) at ./build/RelWithDebInfo/./src/Common/ThreadFuzzer.cpp:445
    3  0x0000000014215fd4 in malloc_mutex_lock_final (mutex=0x165163d0 <init_lock>) at ./contrib/jemalloc/include/jemalloc/internal/mutex.h:151
    4  malloc_mutex_lock_slow (mutex=0x165163d0 <init_lock>) at ./build/RelWithDebInfo/./contrib/jemalloc/src/mutex.c:90
    5  0x00000000141defc6 in malloc_mutex_lock (tsdn=0x0, mutex=<optimized out>) at ./contrib/jemalloc/include/jemalloc/internal/mutex.h:217
    6  malloc_init_hard () at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:2118
    7  0x00000000141d05db in malloc_init () at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:298
    8  imalloc_init_check (sopts=<optimized out>, dopts=<optimized out>) at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:2658
    9  imalloc (sopts=<optimized out>, dopts=<optimized out>) at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:2689
    10 calloc (num=num@entry=1, size=size@entry=32) at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:2852
    11 0x00007ffff7c42c05 in _dlerror_run (operate=operate@entry=0x7ffff7c42490 <dlsym_doit>, args=args@entry=0x7fffffffda40) at dlerror.c:148
    12 0x00007ffff7c42525 in __dlsym (handle=<optimized out>, name=0xe7f568 "dl_iterate_phdr") at dlsym.c:70
    13 0x000000001408e1a0 in (anonymous namespace)::getOriginalDLIteratePHDR () at ./build/RelWithDebInfo/./base/base/phdr_cache.cpp:44
    14 dl_iterate_phdr (callback=0x16287a60 <libunwind::findUnwindSectionsByPhdr(dl_phdr_info*, unsigned long, void*)>, data=0x7fffffffdb58) at ./build/RelWithDebInfo/./base/base/phdr_cache.cpp:64
    15 0x0000000016288468 in libunwind::LocalAddressSpace::findUnwindSections (this=<optimized out>, targetAddr=371765377, info=...) at ./contrib/libunwind/src/AddressSpace.hpp:605
    16 libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_x86_64>::setInfoBasedOnIPRegister (this=this@entry=0x7fffffffde70, isReturnAddress=false) at ./contrib/libunwind/src/UnwindCursor.hpp:2553
    17 0x0000000016287617 in __unw_init_local (cursor=cursor@entry=0x7fffffffde70, context=context@entry=0x7fffffffddc8) at ./build/RelWithDebInfo/./contrib/libunwind/src/libunwind.cpp:91
    18 0x000000001628b094 in _Unwind_Backtrace (callback=0x14226f40 <prof_unwind_init_callback>, ref=0x0) at ./build/RelWithDebInfo/./contrib/libunwind/src/UnwindLevel1-gcc-ext.c:106
    19 0x000000001421a7e0 in prof_boot2 (tsd=tsd@entry=0x7ffff7c357c0, base=<optimized out>) at ./build/RelWithDebInfo/./contrib/jemalloc/src/prof.c:693
    20 0x00000000141df3e7 in malloc_init_hard () at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:2151
    21 0x00000000141d96f1 in malloc_init () at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:298
    22 nallocx (size=32, flags=0) at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:4011
    23 0x000000000b9fdd9b in _ZN6Memory23getActualAllocationSizeITpTkNSt3__17same_asISt11align_val_tEEJEQsr2DBE16OptionalArgumentIDpT_EEEmmS5_ (size=32) at ./src/Common/memory.h:197
    24 _ZN6Memory11trackMemoryITpTkNSt3__17same_asISt11align_val_tEEJEQsr2DBE16OptionalArgumentIDpT_EEEmmR15AllocationTraceS5_ (size=32, trace=...) at ./src/Common/memory.h:208
    25 operator new (size=size@entry=32) at ./build/RelWithDebInfo/./src/Common/new_delete.cpp:53
    26 0x000000001540e0a6 in google::protobuf::internal::ShutdownData::get () at ./build/RelWithDebInfo/./contrib/google-protobuf/src/google/protobuf/message_lite.cc:679
    27 google::protobuf::internal::OnShutdownRun (f=0x15407420 <google::protobuf::internal::DestroyString(void const*)>, arg=0x167a6368 <google::protobuf::internal::fixed_address_empty_string>) at ./build/RelWithDebInfo/./contrib/google-protobuf/src/google/protobuf/message_lite.cc:697
    28 0x0000000015407560 in global constructors keyed to 000101 () at ./contrib/google-protobuf/src/google/protobuf/generated_message_util.h:180
    29 0x000000001629101d in __libc_csu_init ()
    30 0x00007ffff7dba010 in __libc_start_main (main=0x66e3b00 <main(int, char**)>, argc=4, argv=0x7fffffffe258, init=0x16290fd0 <__libc_csu_init>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe248) at ../csu/libc-start.c:264
    31 0x00000000066e302e in _start ()

And this is indeed what happens for glibc prio 2.34, since only in [2]
the allocation had been removed

  [2]: https://sourceware.org/git/?p=glibc.git;a=commit;h=fada9018199c21c469ff0e731ef75c6020074ac9

But, jemalloc provides also JEMALLOC_PROF_LIBUNWIND, yes it is not llvm
libunwind, but gnu one, but ClickHouse version of llvm libunwind
provides unw_backtrace(), so we can use it, and it will not be called
during jemalloc bootstrap, so deadlock should not be possible.

Funny thing that for this deadlock the workaround is to enable
background_thread, because a thread created for it will initialize TLS for
__dlerror().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-07-10 16:05:36 +02:00
Nikita Mikhaylov
2b091983e8 Bump Azure to ea3e19a7be 2024-07-09 19:59:49 +00:00
Nikita Mikhaylov
0d54151cb8 Make the pocketfft to point to the upstream/master branch 2024-07-09 19:55:37 +00:00
Robert Schulze
55468caeae
Fix ARM build 2024-07-09 16:38:22 +00:00
Vitaly Baranov
32b9f7940f
Merge pull request #66191 from vitlibar/grpc-avoid-using-harmful-function-rand
Avoid using harmful function rand() in grpc
2024-07-09 13:42:20 +00:00
Robert Schulze
3f1c0b9966
Merge remote-tracking branch 'ClickHouse/master' into add_statistics_cmsketch 2024-07-09 10:48:19 +00:00
Kruglov Pavel
52b3d1a244
Merge pull request #66130 from slvrtrn/update-avro-submodule-with-array-block-size-fix
Update AVRO submodule with the array block size fix
2024-07-09 07:58:04 +00:00
Nikita Taranov
a9546fe551
Merge pull request #66147 from ClickHouse/fix_mac_build
Playing minesweeper with build system
2024-07-08 22:05:17 +00:00
Robert Schulze
bd42a096b7
Bump rocksdb to v6.29.5 2024-07-08 16:52:36 +00:00
Robert Schulze
1bfafa42be
Bump rocksdb to v6.28.2 2024-07-08 14:47:35 +00:00
Robert Schulze
9509802866
Bump rocksdb to v6.27.3 2024-07-08 13:57:03 +00:00
Robert Schulze
31f9bed442
Bump rocksdb to 6.26.1 2024-07-08 13:49:05 +00:00
Robert Schulze
adcee80b2d
Bump rocksdb to v6.25.3 2024-07-08 13:41:34 +00:00
Robert Schulze
312dd82425
Bump rocksdb to v6.24.2 2024-07-08 12:58:59 +00:00
Robert Schulze
068da21837
Merge pull request #66068 from rschu1ze/bump-rocksdb
Bump rocksdb to v6.23.3
2024-07-08 12:17:40 +00:00
Robert Schulze
5943d60f26
Merge pull request #66097 from azat/build/src-dir-fix
Avoid using source directory for generated files
2024-07-08 09:20:39 +00:00
Vitaly Baranov
7b19076ebe Avoid using harmful function rand() in grpc. 2024-07-07 22:31:45 +02:00
Vitaly Baranov
42945a6d4a
Merge pull request #66137 from vitlibar/switch-contrib-orc-to-better-commit
Switch submodule contrib/orc to a proper commit in the main branch.
2024-07-07 17:38:08 +00:00
Nikita Taranov
0c13289ea3
Merge branch 'master' into fix_mac_build 2024-07-07 16:31:49 +02:00
Yakov Olkhovskiy
6e5e680797 bump libprotobuf-mutator, fix build 2024-07-06 05:42:21 +00:00
Nikita Taranov
6e8edf008f try fix 2024-07-05 18:36:50 +02:00
Robert Schulze
72b3ebd180
Merge pull request #66136 from rschu1ze/bump-s2
Bump s2geometry again
2024-07-05 14:05:31 +00:00
Vitaly Baranov
40fd150279 Switch submodule contrib/orc to proper commit in the main branch. 2024-07-05 15:33:50 +02:00
Robert Schulze
d3f23c2753
Bump s2geometry again 2024-07-05 13:29:34 +00:00
slvrtrn
3ce470c57f Update AVRO submodule with the array block size fix 2024-07-05 14:42:43 +02:00
Vitaly Baranov
f315b36323
Merge pull request #66069 from vitlibar/add-prometheus-protocols-protobufs
Add protobufs for Prometheus remote-write/remote-read protocols
2024-07-05 10:41:04 +00:00
Robert Schulze
3621b35591
Merge pull request #66094 from rschu1ze/bump-s2geometry
Bump s2geometry to latest master
2024-07-05 10:30:20 +00:00
Robert Schulze
4dca031e9d
Merge pull request #66100 from rschu1ze/bump-azure
Bump Azure to 1.12
2024-07-05 09:11:00 +00:00
Vitaly Baranov
1ef5bca591 Fix cmake function PROTOBUF_GENERATE_CPP(): now it returns correct paths in SRCS and HDRS
even if input ".proto" files are located in sibling directories.
2024-07-04 23:07:19 +02:00
Vitaly Baranov
e428542b2e Add prometheus protobufs. 2024-07-04 23:06:14 +02:00
Antonio Andelic
ec6739120c
Merge pull request #66041 from ClickHouse/try-disabling-jemalloc-background-threads
Try disabling jemalloc background threads
2024-07-04 20:30:39 +00:00
Michael Kolupaev
1eb7ca7479 Merge remote-tracking branch 'origin/master' into zup 2024-07-04 19:25:23 +00:00
Robert Schulze
ffe1f8fea0
Bump Azure to 1.12 2024-07-04 19:24:30 +00:00
Robert Schulze
1d8389ddee
Bump rocksdb to v6.23.3 2024-07-04 19:19:34 +00:00
Michael Kolupaev
b04e6c806b Merge remote-tracking branch 'origin/master' into uwu 2024-07-04 19:02:47 +00:00
Robert Schulze
597810b69d
Fix s390x build 2024-07-04 18:46:09 +00:00
Robert Schulze
632f292fd1
Merge pull request #66082 from rschu1ze/bmp-vectorscan11
Bump vectorscan to 5.4.11
2024-07-04 18:11:10 +00:00
Robert Schulze
1bd515cc1f
Merge remote-tracking branch 'ClickHouse/master' into bump-s2geometry 2024-07-04 15:43:47 +00:00
Azat Khuzhin
c49d26bc23 Avoid using source directory for generated files
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-07-04 17:20:16 +02:00
Robert Schulze
11a30d6d60
Bump s2geometry to latest master 2024-07-04 14:20:06 +00:00
Antonio Andelic
613ed1ebbf Merge branch 'master' into try-disabling-jemalloc-background-threads 2024-07-04 13:58:26 +02:00
Antonio Andelic
c93d8cbb66 Fixes 2024-07-04 13:57:47 +02:00
Nikita Mikhaylov
e2a494fa60
Merge pull request #66064 from rschu1ze/mysql-openssl-tsan
Maybe fix tsan assert in `test_mysql_killed_while_insert_8_0`
2024-07-04 11:17:42 +00:00
Robert Schulze
33b7afc1b4
Bump vectorscan to 5.4.11 2024-07-04 10:30:24 +00:00
Robert Schulze
dbac2212a0
Merge pull request #66072 from rschu1ze/llvm-15-burning-some-trash
Clean-up custom LLVM 15 patches
2024-07-04 09:06:38 +00:00
Robert Schulze
43b088dbd0
Merge pull request #66056 from rschu1ze/bmp-vectorscan
Bump vectorscan to 5.4.10.1
2024-07-04 08:59:50 +00:00
JackyWoo
0c5821e5b8 Merge branch 'master' into add_statistics_cmsketch
# Conflicts:
#	docs/en/engines/table-engines/mergetree-family/mergetree.md
#	src/Storages/Statistics/Statistics.cpp
#	src/Storages/Statistics/Statistics.h
#	src/Storages/Statistics/StatisticsTDigest.h
#	src/Storages/Statistics/StatisticsUniq.h
#	src/Storages/Statistics/TDigestStatistics.cpp
#	tests/queries/0_stateless/02864_statistics_uniq.sql
2024-07-04 10:25:53 +08:00
Robert Schulze
eb7ab5128d
Clean-up custom LLVM 15 patches 2024-07-03 21:07:59 +00:00
Robert Schulze
9737c5bab4
Probably fix tsan assert in test_mysql_killed_while_insert_8_0 2024-07-03 17:38:53 +00:00
Robert Schulze
8b14754005
Fix ARM build (upgrade sysroot) 2024-07-03 16:29:47 +00:00
Robert Schulze
39a371b27d
Bump vectorscan 2024-07-03 15:07:31 +00:00
Antonio Andelic
5fd36059e4 Try disabling background threads 2024-07-03 15:28:01 +02:00
Michael Kolupaev
ab4d445656 Merge remote-tracking branch 'origin/master' into uwu 2024-07-02 04:37:46 +00:00
Azat Khuzhin
9c7db6711b Fix possible issues with MySQL client protocol TLS connections
Occasionally, 02479_mysql_connect_to_self fails on CI [1].

  [1]: https://github.com/ClickHouse/ClickHouse/issues/50911

The problem was indeed query profiler and EINTR, but not in a way you
may think.

For such failures you may see the following trace in trace_log:

    contrib/openssl/crypto/bio/bss_sock.c:127::sock_read
    contrib/openssl/crypto/bio/bio_meth.c:121::bread_conv
    contrib/openssl/crypto/bio/bio_lib.c:285::bio_read_intern
    contrib/openssl/crypto/bio/bio_lib.c:311::BIO_read
    contrib/openssl/ssl/record/methods/tls_common.c:398::tls_default_read_n
    contrib/openssl/ssl/record/methods/tls_common.c:575::tls_get_more_records
    contrib/openssl/ssl/record/methods/tls_common.c:1122::tls_read_record
    contrib/openssl/ssl/record/rec_layer_s3.c:645::ssl3_read_bytes
    contrib/openssl/ssl/s3_lib.c:4527::ssl3_read_internal
    contrib/openssl/ssl/s3_lib.c:4551::ssl3_read
    contrib/openssl/ssl/ssl_lib.c:2343::ssl_read_internal
    contrib/openssl/ssl/ssl_lib.c:2357::SSL_read
    contrib/mariadb-connector-c/libmariadb/secure/openssl.c:729::ma_tls_read
    contrib/mariadb-connector-c/libmariadb/ma_tls.c:90::ma_pvio_tls_read
    contrib/mariadb-connector-c/libmariadb/ma_pvio.c:250::ma_pvio_read
    contrib/mariadb-connector-c/libmariadb/ma_pvio.c:297::ma_pvio_cache_read
    contrib/mariadb-connector-c/libmariadb/ma_net.c:373::ma_real_read
    contrib/mariadb-connector-c/libmariadb/ma_net.c:427::ma_net_read
    contrib/mariadb-connector-c/libmariadb/mariadb_lib.c:192::ma_net_safe_read
    contrib/mariadb-connector-c/libmariadb/mariadb_lib.c:2138::mthd_my_read_query_result
    contrib/mariadb-connector-c/libmariadb/mariadb_lib.c:2212::mysql_real_query
    src/Common/mysqlxx/Query.cpp:56::mysqlxx::Query::executeImpl()
    src/Common/mysqlxx/Query.cpp:73::mysqlxx::Query::use()
    src/Processors/Sources/MySQLSource.cpp:50::DB::MySQLSource::Connection::Connection()

After which the connection will fail with:

    Code: 1000. DB::Exception: Received from localhost:9000. DB::Exception: mysqlxx::ConnectionLost: Lost connection to MySQL server during query (127.0.0.1:9004). (POCO_EXCEPTION)

But, if you will take a look at ma_tls_read() you will see that it has
proper retries for SSL_ERROR_WANT_READ (and EINTR is just a special case
of it), but still, for some reason it fails.

And the reason is the units of the read/write timeout, ma_tls_read()
calls poll(read_timeout) in case of SSL_ERROR_WANT_READ, but, it
incorrectly assume that the timeout is in milliseconds, but that timeout
was in seconds, this bug had been fixed in [2], and now it works like a
charm!

  [2]: https://github.com/ClickHouse/mariadb-connector-c/pull/17

I've verified it with patching openssl library:

    diff --git a/crypto/bio/bss_sock.c b/crypto/bio/bss_sock.c
    index 82f7be85ae..3d2f3926a0 100644
    --- a/crypto/bio/bss_sock.c
    +++ b/crypto/bio/bss_sock.c
    @@ -124,7 +124,24 @@ static int sock_read(BIO *b, char *out, int outl)
                 ret = ktls_read_record(b->num, out, outl);
             else
     # endif
    -            ret = readsocket(b->num, out, outl);
    +        {
    +            static int i = 0;
    +            static int j = 0;
    +            if (!(++j % 5))
    +            {
    +                fprintf(stderr, "sock_read: inject EAGAIN with ret=0\n");
    +                ret = 0;
    +                errno = EAGAIN;
    +            }
    +            else if (!(++i % 3))
    +            {
    +                fprintf(stderr, "sock_read: inject EAGAIN with ret=-1\n");
    +                ret = -1;
    +                errno = EAGAIN;
    +            }
    +            else
    +                ret = readsocket(b->num, out, outl);
    +        }
             BIO_clear_retry_flags(b);
             if (ret <= 0) {
                 if (BIO_sock_should_retry(ret))

And before this patch (well, not the patch itself, but the referenced
patch in mariadb-connector-c) if you will pass read_write_timeout=1 it
will fail:

    SELECT * FROM mysql('127.0.0.1:9004', system, one, 'default', '', SETTINGS connect_timeout = 100, connection_wait_timeout = 100, read_write_timeout=1)
    Code: 1000. DB::Exception: Received from localhost:9000. DB::Exception: mysqlxx::ConnectionLost: Lost connection to MySQL server during query (127.0.0.1:9004). (POCO_EXCEPTION)

But after, it always works:

    $ ch benchmark -c10 -q "SELECT * FROM mysql('127.0.0.1:9004', system, one, 'default', '', SETTINGS connection_pool_size=1, connect_timeout = 100, connection_wait_timeout = 100, read_write_timeout=1)"
    ^CStopping launch of queries. SIGINT received.

    Queries executed: 478.

    localhost:9000, queries: 478, QPS: 120.171, RPS: 120.171, MiB/s: 0.001, result RPS: 120.171, result MiB/s: 0.001.

    0.000%          0.014 sec.
    10.000%         0.058 sec.
    20.000%         0.065 sec.
    30.000%         0.073 sec.
    40.000%         0.079 sec.
    50.000%         0.087 sec.
    60.000%         0.089 sec.
    70.000%         0.091 sec.
    80.000%         0.095 sec.
    90.000%         0.100 sec.
    95.000%         0.102 sec.
    99.000%         0.105 sec.
    99.900%         0.140 sec.
    99.990%         0.140 sec.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-07-01 18:39:43 +02:00
Antonio Andelic
8f9beffc65 No jemalloc profiler for non-Linux 2024-06-28 16:04:45 +02:00
Michael Kolupaev
840edd99fa Apply https://github.com/ClickHouse/libunwind/pull/27 too 2024-06-27 21:53:23 +00:00
Michael Kolupaev
36f9b41e7d Merge remote-tracking branch 'origin/master' into uwu 2024-06-27 21:48:37 +00:00
Antonio Andelic
c9fa974472 Use background thread 2024-06-27 08:46:10 +02:00
Antonio Andelic
35820eaa7b Build jemalloc with profiler 2024-06-26 11:13:18 +02:00
JackyWoo
1601420dfd Merge branch 'master' into add_statistics_cmsketch 2024-06-25 10:05:15 +08:00
Alexey Milovidov
49634db3ba Update submodule 2024-06-25 02:51:57 +02:00
Alexey Milovidov
3b377fabe7 Update submodule 2024-06-25 02:49:40 +02:00
JackyWoo
15e20f56fa Add statistics cmsketch 2024-06-24 16:38:56 +08:00
Robert Schulze
605d51700a
Merge pull request #65529 from rschu1ze/openssl-temp-official-fix
OpenSSL: Replace temporary fix for unsynchronized access by official fix
2024-06-23 15:24:28 +00:00
Michael Kolupaev
ad246312d7 Switch to final commit hash 2024-06-21 23:05:41 +00:00
Michael Kolupaev
92ff09f303 There's no Unwind_AppleExtras.cpp anymore 2024-06-21 22:53:38 +00:00
Robert Schulze
0a60dc1672
OpenSSL: Replace temporary fix for unsynchronized access by official fix 2024-06-21 12:57:44 +00:00
Robert Schulze
df857ff17a
Bump re2 to latest HEAD 2024-06-21 10:34:30 +00:00
Michael Kolupaev
06781efcb7 Switch to a cleaner update, hopefully fix builds 2024-06-20 22:59:36 +00:00
Michael Kolupaev
8115926eb6 Fix ARM some more 2024-06-20 22:18:14 +00:00
Michael Kolupaev
62d6e3d339 Fix ARM 2024-06-20 22:18:14 +00:00
Michael Kolupaev
3ab8ba0d4a Update zlib-ng from 2.0.2 to 2.1.6 2024-06-20 22:18:14 +00:00
Michael Kolupaev
424224b202 Set _LIBUNWIND_REMEMBER_STACK_ALLOC explicitly just in case 2024-06-20 22:05:16 +00:00
Michael Kolupaev
8bb60101c4 Update libunwind to 18.1.7 2024-06-20 22:01:16 +00:00
Robert Schulze
fd52e19b3e
Suppress leaksan false positive in OpenSSL 2024-06-19 08:30:47 +00:00
Robert Schulze
abe424b24b
Merge pull request #65111 from rschu1ze/reenable-session-caching
Re-enable OpenSSL session caching
2024-06-18 16:39:38 +00:00
Alexander Tokmakov
b150b0f5fa
Revert "Fix AWS ECS" 2024-06-17 23:11:59 +02:00
Robert Schulze
cdcfcb99fc
Merge remote-tracking branch 'rschu1ze/master' into reenable-session-caching 2024-06-17 20:51:11 +00:00
Robert Schulze
186bd0cc3d
Temporary fix for tsan issue openssl#24629 2024-06-17 12:20:54 +00:00
Alexey Milovidov
2c1918908a Revert "Debug AWS"
This reverts commit 20ca8f8714.
2024-06-16 17:12:55 +02:00
Alexey Milovidov
b41bb96605 Revert "Debug AWS"
This reverts commit 642dc35a76.
2024-06-16 17:12:19 +02:00
Alexey Milovidov
37eb815b17 Fix error 2024-06-16 17:12:05 +02:00
Alexey Milovidov
642dc35a76 Debug AWS 2024-06-16 15:21:17 +02:00
Alexey Milovidov
20ca8f8714 Debug AWS 2024-06-16 15:20:26 +02:00
Alexey Milovidov
cb390cc3d9 Fix alloc/dealloc mismatch or prevent false-positive ASan error in the AWS SDK 2024-06-16 13:14:59 +02:00
Raúl Marín
0d8bd133e6
Merge pull request #64959 from canhld94/bump_fmt_9
Update fmtlib version to 9.1.0
2024-06-14 14:05:04 +00:00
Robert Schulze
d92a2f3ef6
Remove obsolete fix from aws submodule 2024-06-12 13:20:50 +00:00
Robert Schulze
97f96a7b84
Merge pull request #65125 from rschu1ze/bug-63792
Fix false positives leaky memory warnings in OpenSSL
2024-06-12 12:40:50 +00:00
Robert Schulze
92ca8b30b3
Merge pull request #65048 from rschu1ze/bump-absl
Bump abseil to latest HEAD
2024-06-12 08:26:58 +00:00
Robert Schulze
848054e85f
Fix another false positive leak 2024-06-11 21:06:45 +00:00
Robert Schulze
e0279e856f
Fix #63792 (hopefully) 2024-06-11 19:42:20 +00:00
Robert Schulze
af83bc92ce
Switch to same HEAD as before but with s390x-breaking commit reverted 2024-06-11 14:16:38 +00:00
Robert Schulze
e5dcf75968
Add forked abseil submodule back 2024-06-11 14:09:56 +00:00
Robert Schulze
e47bbfb7f2
Remove upstream abseil repository 2024-06-11 14:08:16 +00:00
Duc Canh Le
79a660c018 Merge branch 'master' into bump_fmt_9
Fix CI
2024-06-11 03:39:03 +00:00
Robert Schulze
2ff1e81af6
Merge remote-tracking branch 'rschu1ze/master' into bump-googletest 2024-06-10 16:50:53 +00:00
Robert Schulze
598219c57d
Minor update 2024-06-10 12:56:21 +00:00
Robert Schulze
8af077f3d3
Update build descriptions 2024-06-10 12:53:06 +00:00
Robert Schulze
da91dd6428
Bump absl to 2024-06-07 2024-06-10 11:35:07 +00:00
Robert Schulze
c95ed40d3e
Bump absl to 2024-05-06 2024-06-10 11:32:29 +00:00
Robert Schulze
643444eb11
Bump absl to 2024-05-03 2024-06-10 10:41:47 +00:00
Robert Schulze
61b4643217
Bump absl to 2024-04-24 2024-06-10 10:39:35 +00:00
Robert Schulze
ae7d8821a7
Bump absl to 2024-04-04 2024-06-10 08:46:40 +00:00
Robert Schulze
d4a453aad5
Bump absl to 2024-03-06 2024-06-10 08:32:22 +00:00
Robert Schulze
a0d8d5a37c
Bump absl to 2024-01-02 2024-06-10 08:31:33 +00:00
Robert Schulze
8fe272f210
Bump absl to 2024-01-02 2024-06-10 08:22:27 +00:00
Robert Schulze
1bca6b900b
Bump absl to 2023-12-20 2024-06-10 08:20:25 +00:00
Robert Schulze
70c0589675
Bump absl to 2023-12-12 2024-06-10 08:19:20 +00:00
Robert Schulze
4d3d18cee7
Bump absl to 2023-12-06 2024-06-10 08:18:36 +00:00
Robert Schulze
c6e43f7a7b
Bump absl to 2023-11-28 2024-06-10 08:17:01 +00:00
Robert Schulze
163cacf701
Bump googletest 2024-06-10 07:53:29 +00:00
Raúl Marín
f50a951e8e Fix innocuous data race in detectLanguage 2024-06-07 16:49:07 +02:00
Duc Canh Le
de5258128e update fmtlib version to 9.1.0
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-06-07 06:44:36 +00:00
Michael Kolupaev
fd93097130 Fix writing ORC statistics for unsigned types 2024-06-06 06:39:07 +00:00
Sema Checherinda
6b8ca302d2 test for TotalQpsLimitExceeded 2024-05-28 17:58:32 +02:00
Sema Checherinda
ae4d3f97ae
Merge pull request #64225 from ClickHouse/chesema-patch-aws-error-handler
tests for qps_limit_exceeded
2024-05-24 13:28:33 +00:00
Sema Checherinda
6be79a35b6 update contrib/aws to the last head 2024-05-22 20:30:19 +02:00
Sema Checherinda
a73d60bae5 tests for qps_limit_exceeded 2024-05-22 18:35:28 +02:00
ZhiHong Zhang
4b1c9adb3a
Merge branch 'ClickHouse:master' into gcmaster-parquet 2024-05-22 09:33:01 +08:00
copperybean
b253ca3608 fix clang-tidy warnings
Change-Id: Iff9f5f894e815b184ac35f61b4cac87908c612b5
2024-05-21 16:48:49 +08:00
Alexey Milovidov
29f7a766a5 Merge branch 'master' into libunwind-fix-crash 2024-05-20 19:49:37 +02:00
Alexey Milovidov
dae050d2d4
Merge pull request #64023 from azat/build/freebsd-bcrypt-fix
Fix libbcrypt for FreeBSD build
2024-05-20 09:51:39 +02:00
Alexey Milovidov
27e7086be2 Merge branch 'master' into libunwind-fix-crash 2024-05-20 05:15:52 +02:00
alesapin
3878155b19
Merge pull request #64072 from ClickHouse/try_to_fix_cross_compilation_for_grpc
Try to fix grpc for aarch64 crosscompilation
2024-05-18 18:28:21 +00:00
alesapin
086e5d73a4 Followup 2024-05-18 18:03:55 +02:00
alesapin
2983941ab8 Fix build for openssl
(cherry picked from commit 7efd5d3ab62024619e26dc6c3e28d50bffc98d70)
2024-05-18 16:35:15 +02:00
Alexey Milovidov
a2d9a32652
Merge pull request #63733 from qiangxuhui/qiangxuhui-loongarch64
Initial support for loongarch64
2024-05-18 05:26:26 +02:00
alesapin
24892b151a Try to fix grpc for aarch64 crosscompilation
(cherry picked from commit f3fbf532e0d4d7616f51a9c3d5087cf7b2e6d7d5)
2024-05-17 22:55:12 +02:00
Alexey Milovidov
b9d539d0f7 Merge branch 'master' into libunwind-fix-crash 2024-05-17 20:33:22 +02:00
Alexey Milovidov
e39d8c001f
Merge pull request #63923 from ClickHouse/update-datasketches
Update Datasketches
2024-05-17 20:30:41 +02:00
Azat Khuzhin
bfb3fe0c23 Fix libbcrypt for FreeBSD build
Right now it fails due to [1] with the following error:

    /usr/work/ClickHouse/contrib/libbcrypt/crypt_blowfish/ow-crypt.h:27:14: error: conflicting types for 'crypt_r'
       27 | extern char *crypt_r(__const char *key, __const char *setting, void *data);
          |              ^
    /usr/include/unistd.h:500:7: note: previous declaration is here
      500 | char    *crypt_r(const char *, const char *, struct crypt_data *);
          |          ^

  [1]: 5f521d7ba7

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-17 11:37:01 +02:00
Alexey Milovidov
cb9d35e490 Merge branch 'master' into update-datasketches 2024-05-17 10:23:50 +02:00
Robert Schulze
5fb8ea4c62
Merge remote-tracking branch 'ClickHouse/master' into qiangxuhui-loongarch64 2024-05-17 07:56:01 +00:00
qiangxuhui
7d6d5165fe Update contrib/libunwind to master 2024-05-17 07:45:51 +00:00
Alexey Milovidov
bd6a4a3068 Merge branch 'master' into libunwind-fix-crash 2024-05-17 07:24:39 +02:00
Alexey Milovidov
50dea75fff Update libunwind 2024-05-17 07:15:51 +02:00
Alexey Milovidov
95a452f87c Update to the latest version where the patch is included 2024-05-17 05:41:32 +02:00
Alexey Milovidov
4b73ae8340
Merge pull request #63946 from azat/build/libunwind-fixes
Insignificant libunwind build fixes
2024-05-17 03:34:53 +02:00
Azat Khuzhin
86cf1e13d8 libunwind: fix usage of libunwind.h (by defining -D_LIBUNWIND_IS_NATIVE_ONLY)
From this macros sizeof(unw_context_t)/sizeof(unw_cursor_t) is depends
(_LIBUNWIND_CONTEXT_SIZE/_LIBUNWIND_CURSOR_SIZE).

So it should be not only PRIVATE but for INTERFACE as well.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-16 13:11:47 +02:00
Azat Khuzhin
793a11fd19 libunwind: remove useless _DEBUG flag
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-16 13:11:47 +02:00
Alexey Milovidov
91c240c85c Update Datasketches 2024-05-16 00:43:38 +02:00
Azat Khuzhin
94041d193c Fix SIGSEGV due to CPU/Real profiler
The problem was due to incorrect unwinding due from signal handlers,
which leads to incorrect DWARF (FDE/CIE) interpretation.

After this patch I was not able to reproduce the crash for couple of
hours, while before it was very stable (I've reduced the minimal
threshold for query_profiler_real_time_period_ns), using simply:

    $ clickhouse-benchmark --port 19000 -q "SELECT * FROM remote('127.{1..10}', system, one)" --query_profiler_real_time_period_ns=1

Note, I'm using here remote() for fibers, that has stack with guard
pages that helps with reproducing the crash more faster.

P.S. I also have another implementation of this fix, without patching
unwind and using info from signal context directly, and even though it
is better, because you don't need to trip extra frames and you can use
all the 45 frames for something useful, it is too complex, so let's go
with a simpler patch first, and I think it could be even backported.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-15 18:22:59 +02:00
qiangxuhui
47b7d3748d Fix code for loongarch64
Fix `base/poco` and `contrib/openssl-cmake` based on code review comments(https://github.com/ClickHouse/ClickHouse/pull/61509).
2024-05-14 07:45:57 +00:00