Commit Graph

146813 Commits

Author SHA1 Message Date
Igor Nikonov
04c3661b0b Randomize parallel_replicas_local_plan 2024-07-12 13:42:25 +00:00
Igor Nikonov
1a8e1eb0fe Merge remote-tracking branch 'origin/master' into pr-local-plan 2024-07-11 11:14:12 +00:00
Igor Nikonov
69652477f4 Merge remote-tracking branch 'origin/master' into pr-local-plan 2024-07-11 10:17:28 +00:00
Antonio Andelic
fbb7a3ed52
Merge pull request #66281 from ClickHouse/collect-core-dumps-stateless
Collect core dumps in more tests
2024-07-11 09:45:17 +00:00
Antonio Andelic
3d0238827c
Merge pull request #66346 from azat/jemalloc-deadlock-fix
Fix possible deadlock for jemalloc with enabled profiler
2024-07-11 09:23:02 +00:00
vdimir
e1ad5c7652
Merge pull request #66310 from ClickHouse/vdimir/fix_join_on_nullsafe_optimization
Disable broken cases from 02911_join_on_nullsafe_optimization
2024-07-11 08:52:17 +00:00
Antonio Andelic
f808894de3
Merge pull request #66349 from ClickHouse/fix-test_parallel_replicas_custom_key
Fix test_parallel_replicas_custom_key
2024-07-11 06:54:16 +00:00
Nikita Fomichev
da37ed7fa2
Merge pull request #66363 from ClickHouse/optimize-tests-3
Stateless tests: Improve tests speed 3
2024-07-11 06:41:36 +00:00
Alexey Milovidov
75dd68ae2b
Merge pull request #66342 from ClickHouse/fix-replaceRegexp_fallback
Fix invalid XML
2024-07-11 02:09:37 +00:00
Justin de Guzman
3308d14fb3
Merge pull request #66051 from gingerwizard/patch-1
Update json.md
2024-07-11 01:39:27 +00:00
Alexey Milovidov
3c93091bd8
Merge pull request #65687 from ClickHouse/checks_for_column_type
Introduce type contract checks in `IColumn`
2024-07-10 23:40:34 +00:00
Nikita Fomichev
6f3ae4a6de Fix path in test 2024-07-10 23:57:23 +02:00
Alexey Katsman
46630ca958
Merge pull request #65335 from korowa/in-part-elimination
support set transformation in partition pruning
2024-07-10 21:53:17 +00:00
Nikita Mikhaylov
77b6c18a50
Merge pull request #66348 from ClickHouse/thomoco-patch-3
Update kill.md
2024-07-10 20:07:06 +00:00
Sema Checherinda
48a47e5f29
Merge pull request #66162 from azat/tests/peak_threads_usage
Use peak_threads_usage instead of arrayUniq(thread_ids) in tests
2024-07-10 19:32:55 +00:00
Nikita Mikhaylov
2d11eb7e49
Merge pull request #66289 from ClickHouse/pr-failed-to-connect
Log message: Failed to connect to replica ...
2024-07-10 17:57:07 +00:00
Nikita Fomichev
481a1570aa Run sequential tests in parallel to non-sequential 2024-07-10 19:21:32 +02:00
Sema Checherinda
be80f4e5d0
Merge pull request #66065 from ClickHouse/chesema-faster-drop
DatabaseCatalog drops tables faster
2024-07-10 17:04:41 +00:00
Nikita Fomichev
c421e0d789
Merge pull request #66305 from ClickHouse/optimize-tests-2
Stateless tests: Improve tests speed 2
2024-07-10 16:52:04 +00:00
Nikita Taranov
e7d5992966
Merge pull request #64553 from nickitat/concurrent_hash_join_cache_ht_sizes
Collect hash table sizes statistics in ConcurrentHashJoin
2024-07-10 16:11:27 +00:00
Nikolai Kochetov
04e83de877
Merge pull request #66261 from ClickHouse/not-ready-set-constraints
Fixing Not-ready set in constraint.
2024-07-10 15:47:39 +00:00
János Benjamin Antal
e5804ae5cd
Use numbers_mt to reduce duration of tests 2024-07-10 17:28:10 +02:00
Nikita Fomichev
e6b3368dc1 debug tests hung on gdb error 9 2024-07-10 17:09:50 +02:00
Antonio Andelic
7cff918a83 Fix test_parallel_replicas_custom_key 2024-07-10 17:04:07 +02:00
Nikita Fomichev
2662760572 debug tests hung on gdb error 8 2024-07-10 16:38:54 +02:00
Thom O'Connor
b7964bc13e
Update kill.md
Including extra verbiage for ON CLUSTER required in ClickHouse Cloud or self-managed clusters
2024-07-10 14:34:54 +00:00
Azat Khuzhin
e6e8b065bc Fix possible deadlock for jemalloc with enabled profiler
ClickHouse uses JEMALLOC_PROF_LIBGCC - _Unwind_Backtrace() for jemalloc,
which calls _Unwind_Backtrace() during bootstrap of jemalloc, and if,
dlsym() has allocations it will lead to deadlock:

    0  __lll_lock_wait (futex=futex@entry=0x16516410 <init_lock+64>, private=0) at lowlevellock.c:52
    1  0x00007ffff7f9d0a3 in __GI___pthread_mutex_lock (mutex=0x16516410 <init_lock+64>) at ../nptl/pthread_mutex_lock.c:80
    2  0x000000000ba6ec23 in pthread_mutex_lock (arg=arg@entry=0x16516410 <init_lock+64>) at ./build/RelWithDebInfo/./src/Common/ThreadFuzzer.cpp:445
    3  0x0000000014215fd4 in malloc_mutex_lock_final (mutex=0x165163d0 <init_lock>) at ./contrib/jemalloc/include/jemalloc/internal/mutex.h:151
    4  malloc_mutex_lock_slow (mutex=0x165163d0 <init_lock>) at ./build/RelWithDebInfo/./contrib/jemalloc/src/mutex.c:90
    5  0x00000000141defc6 in malloc_mutex_lock (tsdn=0x0, mutex=<optimized out>) at ./contrib/jemalloc/include/jemalloc/internal/mutex.h:217
    6  malloc_init_hard () at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:2118
    7  0x00000000141d05db in malloc_init () at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:298
    8  imalloc_init_check (sopts=<optimized out>, dopts=<optimized out>) at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:2658
    9  imalloc (sopts=<optimized out>, dopts=<optimized out>) at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:2689
    10 calloc (num=num@entry=1, size=size@entry=32) at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:2852
    11 0x00007ffff7c42c05 in _dlerror_run (operate=operate@entry=0x7ffff7c42490 <dlsym_doit>, args=args@entry=0x7fffffffda40) at dlerror.c:148
    12 0x00007ffff7c42525 in __dlsym (handle=<optimized out>, name=0xe7f568 "dl_iterate_phdr") at dlsym.c:70
    13 0x000000001408e1a0 in (anonymous namespace)::getOriginalDLIteratePHDR () at ./build/RelWithDebInfo/./base/base/phdr_cache.cpp:44
    14 dl_iterate_phdr (callback=0x16287a60 <libunwind::findUnwindSectionsByPhdr(dl_phdr_info*, unsigned long, void*)>, data=0x7fffffffdb58) at ./build/RelWithDebInfo/./base/base/phdr_cache.cpp:64
    15 0x0000000016288468 in libunwind::LocalAddressSpace::findUnwindSections (this=<optimized out>, targetAddr=371765377, info=...) at ./contrib/libunwind/src/AddressSpace.hpp:605
    16 libunwind::UnwindCursor<libunwind::LocalAddressSpace, libunwind::Registers_x86_64>::setInfoBasedOnIPRegister (this=this@entry=0x7fffffffde70, isReturnAddress=false) at ./contrib/libunwind/src/UnwindCursor.hpp:2553
    17 0x0000000016287617 in __unw_init_local (cursor=cursor@entry=0x7fffffffde70, context=context@entry=0x7fffffffddc8) at ./build/RelWithDebInfo/./contrib/libunwind/src/libunwind.cpp:91
    18 0x000000001628b094 in _Unwind_Backtrace (callback=0x14226f40 <prof_unwind_init_callback>, ref=0x0) at ./build/RelWithDebInfo/./contrib/libunwind/src/UnwindLevel1-gcc-ext.c:106
    19 0x000000001421a7e0 in prof_boot2 (tsd=tsd@entry=0x7ffff7c357c0, base=<optimized out>) at ./build/RelWithDebInfo/./contrib/jemalloc/src/prof.c:693
    20 0x00000000141df3e7 in malloc_init_hard () at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:2151
    21 0x00000000141d96f1 in malloc_init () at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:298
    22 nallocx (size=32, flags=0) at ./build/RelWithDebInfo/./contrib/jemalloc/src/jemalloc.c:4011
    23 0x000000000b9fdd9b in _ZN6Memory23getActualAllocationSizeITpTkNSt3__17same_asISt11align_val_tEEJEQsr2DBE16OptionalArgumentIDpT_EEEmmS5_ (size=32) at ./src/Common/memory.h:197
    24 _ZN6Memory11trackMemoryITpTkNSt3__17same_asISt11align_val_tEEJEQsr2DBE16OptionalArgumentIDpT_EEEmmR15AllocationTraceS5_ (size=32, trace=...) at ./src/Common/memory.h:208
    25 operator new (size=size@entry=32) at ./build/RelWithDebInfo/./src/Common/new_delete.cpp:53
    26 0x000000001540e0a6 in google::protobuf::internal::ShutdownData::get () at ./build/RelWithDebInfo/./contrib/google-protobuf/src/google/protobuf/message_lite.cc:679
    27 google::protobuf::internal::OnShutdownRun (f=0x15407420 <google::protobuf::internal::DestroyString(void const*)>, arg=0x167a6368 <google::protobuf::internal::fixed_address_empty_string>) at ./build/RelWithDebInfo/./contrib/google-protobuf/src/google/protobuf/message_lite.cc:697
    28 0x0000000015407560 in global constructors keyed to 000101 () at ./contrib/google-protobuf/src/google/protobuf/generated_message_util.h:180
    29 0x000000001629101d in __libc_csu_init ()
    30 0x00007ffff7dba010 in __libc_start_main (main=0x66e3b00 <main(int, char**)>, argc=4, argv=0x7fffffffe258, init=0x16290fd0 <__libc_csu_init>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe248) at ../csu/libc-start.c:264
    31 0x00000000066e302e in _start ()

And this is indeed what happens for glibc prio 2.34, since only in [2]
the allocation had been removed

  [2]: https://sourceware.org/git/?p=glibc.git;a=commit;h=fada9018199c21c469ff0e731ef75c6020074ac9

But, jemalloc provides also JEMALLOC_PROF_LIBUNWIND, yes it is not llvm
libunwind, but gnu one, but ClickHouse version of llvm libunwind
provides unw_backtrace(), so we can use it, and it will not be called
during jemalloc bootstrap, so deadlock should not be possible.

Funny thing that for this deadlock the workaround is to enable
background_thread, because a thread created for it will initialize TLS for
__dlerror().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-07-10 16:05:36 +02:00
Nikita Fomichev
83cfe164bd debug tests hung on gdb error 7 2024-07-10 15:55:30 +02:00
Antonio Andelic
ccffc05681 Fix asan 2024-07-10 15:45:17 +02:00
jsc0218
b0580eb575
Merge pull request #66239 from azat/processors_profile_log-fixes
Reduce inaccuracy of input_wait_elapsed_us/input_wait_elapsed_us/elapsed_us
2024-07-10 13:37:53 +00:00
János Benjamin Antal
211994333f
Fix invalid XML 2024-07-10 14:56:28 +02:00
Yarik Briukhovetskyi
d2ad1cef33
Merge pull request #66192 from Blargian/docs_fix
[Docs] update `singleValueOrNull`
2024-07-10 12:53:48 +00:00
Nikita Fomichev
ba176a9c4a debug tests hung on gdb error 6 2024-07-10 14:51:27 +02:00
Konstantin Bogdanov
4a7e81a9fe
Merge pull request #66309 from GoshaLetov/patch-1
Remove typo from distinctive-features.md
2024-07-10 12:08:06 +00:00
Sema Checherinda
bde1d2fe92 less noisy logs 2024-07-10 13:48:52 +02:00
Sema Checherinda
bd62fecd31 avoid lock-order-inversion 2024-07-10 13:47:58 +02:00
Sema Checherinda
eb59f1c66d Merge branch 'master' into chesema-faster-drop 2024-07-10 13:47:06 +02:00
Nikita Fomichev
f96d40d828 debug tests hung on gdb error 5 2024-07-10 13:35:19 +02:00
Antonio Andelic
65dc1c8515
Merge pull request #66124 from ClickHouse/keeper-correct-log-long-total
Correctly print long processing requests in Keeper
2024-07-10 11:34:28 +00:00
Nikita Fomichev
ede43721b4 debug tests hung on gdb error 4 2024-07-10 13:30:06 +02:00
Sema Checherinda
63b9f28df9 Merge branch 'master' into chesema-faster-drop 2024-07-10 13:29:31 +02:00
Antonio Andelic
1eec1fd77f Merge branch 'master' into collect-core-dumps-stateless 2024-07-10 13:01:41 +02:00
Kseniia Sumarokova
57ae55b42c
Merge pull request #66317 from ClickHouse/davenger-patch-1
Clarify ordered mode description for s3Queue
2024-07-10 10:54:34 +00:00
Max K
a3c4419e42
Merge pull request #66276 from ClickHouse/ci_running_status_fix
CI: Do not finalize CI running status unless all success
2024-07-10 10:53:34 +00:00
Nikita Fomichev
ef08055db9 debug tests hung on gdb error 2 2024-07-10 12:47:32 +02:00
Alexander Gololobov
74cc20b286
Make spellcheck happy 2024-07-10 12:18:50 +02:00
Alexander Gololobov
56c751a10a
Update docs/en/engines/table-engines/integrations/s3queue.md
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2024-07-10 12:17:39 +02:00
Kruglov Pavel
6a9e2d9e82
Merge pull request #65980 from Avogar/add-missing-settings-in-schema-cache
Add missing settings in schema inference cache
2024-07-10 10:15:22 +00:00
Alexander Gololobov
73c4eaa0f2
Clarify ordered mode description for s3Queue 2024-07-10 12:09:32 +02:00
Nikita Fomichev
17d5ec28d4 debug tests hung on gdb error 2024-07-10 12:08:33 +02:00