Commit Graph

51685 Commits

Author SHA1 Message Date
Antonio Andelic
2bfed6b8eb Add system commands and 4lw for jemalloc 2024-01-11 08:24:56 +00:00
Kruglov Pavel
3fb1df2275
Merge pull request #56403 from bigo-sg/ch_opt_array_element
Optimize array element function when input is array(map)/array(array(num)/array(array(string))/array(bigint)/array(decimal)
2024-01-10 13:00:27 +01:00
vdimir
9368c074fa
Merge pull request #58596 from ClickHouse/vdimir/join_issue_57894
Fix join using nullable in old analyzer
2024-01-10 12:20:00 +01:00
Dmitry Novik
b2bf1372aa
Merge pull request #58457 from ClickHouse/analyzer-fuzz-low-condinality
Analyzer: Fix LOGICAL_ERROR with LowCardinality
2024-01-10 11:06:42 +01:00
Robert Schulze
4cacfa6b43
Merge pull request #57509 from jinjunzh/qatzstd_main
Add Intel-QAT-based `ZSTD_QAT` compression codec
2024-01-10 10:31:39 +01:00
Kseniia Sumarokova
1b517105db
Merge pull request #57323 from valbok/mysql-binlog-client
MaterializedMySQL: Introduce MySQL Binlog Client
2024-01-10 09:08:13 +01:00
taiyang-li
231de4ac49 Merge branch 'master' into ch_opt_array_element 2024-01-10 15:49:43 +08:00
Alexey Milovidov
b8d9880b17
Merge pull request #55575 from jh0x/no-octal-leading-zeroes
Consume leading zeroes when parsing a number in ConstantExpressionTemplate
2024-01-10 05:16:53 +01:00
Nikolay Degterinsky
24733700fb
Merge pull request #57745 from KevinyhZou/imporve_multi_if_nullable
Improve `MultiIf` function performance while type is nullable
2024-01-09 23:17:58 +01:00
Robert Schulze
4db26ad56b
Merge pull request #58531 from rschu1ze/string-similarity
Implement Damerau-Levenshtein distance and Jaro[-Winkler] string distance functions
2024-01-09 22:46:57 +01:00
Robert Schulze
ced9e93ac6
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main 2024-01-09 20:36:02 +00:00
pufit
6cf55b82f4
Merge pull request #58539 from canhld94/file_custom_compress_level
Allow explicitly set compression level in output format
2024-01-09 13:43:38 -05:00
Robert Schulze
69ba1a706a
Merge remote-tracking branch 'rschu1ze/master' into string-similarity 2024-01-09 15:22:52 +00:00
Anton Popov
2fd0f7be7b
Merge pull request #55579 from CurtizJ/remove-old-projections-code
Remove more old code of projection analysis
2024-01-09 15:51:12 +01:00
Alexander Tokmakov
92daa17b31
Merge pull request #57697 from ClickHouse/dependency-failure
Customizable dependency failure handling for AsyncLoader
2024-01-09 14:03:35 +01:00
Robert Schulze
db3b430d41
Stylistic changes 2024-01-09 12:53:17 +00:00
Sema Checherinda
7c7e72c4b7
Merge pull request #58573 from ClickHouse/chesema-s3-client-creation
fix and test that S3Clients are reused
2024-01-09 13:28:20 +01:00
Sema Checherinda
6dbf7c967c
Merge pull request #57663 from ClickHouse/CheSema-remote-blobs-naming-2
generate object storage key by template
2024-01-09 13:26:56 +01:00
Robert Schulze
195f10010d
Cosmetics + make things more atomic 2024-01-09 12:26:38 +00:00
jinjunzh
e2fb33e612 refine zstd_qat initialization 2024-01-09 13:42:01 -05:00
jinjunzh
8f8230df07 Merge branch 'qatzstd_main' of https://github.com/jinjunzh/ClickHouse into qatzstd_main 2024-01-09 11:46:53 -05:00
jinjunzh
1f3ae5ec6b qat device init only once 2024-01-09 11:38:55 -05:00
vdimir
4a57fd599a
Merge pull request #58595 from ClickHouse/vdimir/concurrent_hash_max_joined_rows
Disable max_joined_block_rows in ConcurrentHashJoin
2024-01-09 13:02:49 +01:00
vdimir
46c43e2cce
Another fix join using nullable in old analyzer 2024-01-09 11:58:50 +00:00
vdimir
06652f9714
Fix join using nullable in old analyzer 2024-01-09 11:45:34 +00:00
Kruglov Pavel
22a0e085b7
Merge pull request #58597 from rschu1ze/makedatetime64-nonconst-fraction
`makeDateTime64()`: Allow non-const fraction argument
2024-01-09 12:17:12 +01:00
vdimir
21e4e0537d
Merge pull request #58607 from azat/stacktrace-fix-inline-frames
Fix possible NULL dereference during symbolizing inline frames
2024-01-09 12:10:35 +01:00
Robert Schulze
64ab30f446
Merge pull request #57078 from ClibMouse/seriesDecompose
Seasonal decompose of time series into seasonal, trend and residue components
2024-01-09 11:18:13 +01:00
Raúl Marín
803d8d6320
Merge pull request #58594 from Algunenano/missing_includes
Add some missing LLVM includes
2024-01-09 11:01:44 +01:00
taiyang-li
e5b4bc8f45 Merge branch 'master' into ch_opt_array_element 2024-01-09 17:17:38 +08:00
Duc Canh Le
37a7ce2950
Apply suggestions from code review
Co-authored-by: pufit <pufit@yandex.ru>
2024-01-09 17:17:30 +08:00
Bharat Nallan
ee37f551a2
independent registration of interpreters (#58443)
* make interpreter factory an instance

* add registerInterpreter

* register interpreters individually

* try wire everything up

* fix style

* fix test
2024-01-08 22:33:48 -08:00
Yakov Olkhovskiy
cfa41e7dc6
Merge pull request #58038 from ClickHouse/revert-58022-revert-57555-ft-dropped_tables_parts
Add system.dropped_tables_parts table - second attempt
2024-01-09 00:47:07 -05:00
Alexey Milovidov
5210bbc587
Update ConstantExpressionTemplate.cpp 2024-01-09 05:13:52 +03:00
Bhavna Jindal
b2434d0c10 minor refactoring and cosmetic changes 2024-01-08 10:49:09 -08:00
Joanna Hulboj
f67edfb397 Consume leading zeroes - use find_first_not_symbols 2024-01-08 18:42:33 +00:00
Azat Khuzhin
9b6c8cd44f Fix possible NULL dereference during symbolizing inline frames
It is possible sometimes:

<details>

```
2024.01.02 15:18:49.542342 [ 102142 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.
2024.01.02 15:18:49.542356 [ 102142 ] {} <Fatal> BaseDaemon: Stack trace: 0x0000000012817007 0x00000000120b3e88 0x000000001209c3e8 0x000000001209b7f3 0x000000001213d936 0x0000000011f71042 0x0000000011f6dd22 0x0000
000012496116 0x000000001249181c 0x00000000133197ec 0x000000001332bc79 0x0000000015d0eb14 0x0000000015d0fd11 0x0000000015e1b367 0x0000000015e195fc 0x00007f011cb31fa3 0x00007f011ca624cf
2024.01.02 15:18:50.052773 [ 102142 ] {} <Fatal> BaseDaemon: 2.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:603: shared_ptr<DB::Context, void>
2024.01.02 15:18:50.052829 [ 102142 ] {} <Fatal> BaseDaemon: 2.2. inlined from ./build_docker/./src/Planner/PlannerContext.h:56: DB::PlannerContext::getQueryContext() const
2024.01.02 15:18:50.052853 [ 102142 ] {} <Fatal> BaseDaemon: 2.3. inlined from ./build_docker/./src/Storages/StorageDistributed.cpp:666: DB::(anonymous namespace)::buildQueryTreeDistributed(DB::SelectQueryInfo&, s
td::shared_ptr<DB::StorageSnapshot> const&, DB::StorageID const&, std::shared_ptr<DB::IAST> const&)
2024.01.02 15:18:50.052872 [ 102142 ] {} <Fatal> BaseDaemon: 2. ./build_docker/./src/Storages/StorageDistributed.cpp:743: DB::StorageDistributed::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> co
nst&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000012817007 in /usr/lib/debug/usr
/bin/clickhouse.debug
2024.01.02 15:18:50.211769 [ 102142 ] {} <Fatal> BaseDaemon: 3.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: std::shared_ptr<DB::Context>::operator->[abi:v15000]()
 const
2024.01.02 15:18:50.211815 [ 102142 ] {} <Fatal> BaseDaemon: 3. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:2488: DB::InterpreterSelectQuery::executeFetchColumns(DB::QueryProcessingStage::Enum, DB
::QueryPlan&) @ 0x00000000120b3e88 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.295382 [ 102142 ] {} <Fatal> BaseDaemon: 4. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:1444: DB::InterpreterSelectQuery::executeImpl(DB::QueryPlan&, std::optional<DB::Pipe>) @
 0x000000001209c3e8 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.375779 [ 102142 ] {} <Fatal> BaseDaemon: 5.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/optional:260: ~__optional_destruct_base
2024.01.02 15:18:50.375820 [ 102142 ] {} <Fatal> BaseDaemon: 5. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:868: DB::InterpreterSelectQuery::buildQueryPlan(DB::QueryPlan&) @ 0x000000001209b7f3 in
/usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.403975 [ 102142 ] {} <Fatal> BaseDaemon: 6. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::buildQueryPlan(DB::QueryPlan&) @ 0x0000
00001213d936 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.432051 [ 102142 ] {} <Fatal> BaseDaemon: 7.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:603: shared_ptr<DB::Context, void>
2024.01.02 15:18:50.432472 [ 102143 ] {} <Fatal> BaseDaemon: ########## Short fault info ############
2024.01.02 15:18:50.432500 [ 102143 ] {} <Fatal> BaseDaemon: (version 23.9.2.56 (official build), build id: 76109A79FA62B9BC630A6C39438DEA7D28147B68, git hash: a1bf3f1de5) (from threa
d 102142) Received signal 11
2024.01.02 15:18:50.432516 [ 102143 ] {} <Fatal> BaseDaemon: Signal description: Segmentation fault
2024.01.02 15:18:50.432526 [ 102143 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.
2024.01.02 15:18:50.432539 [ 102143 ] {} <Fatal> BaseDaemon: Stack trace: 0x00007f011cac5181 0x0000000015ccd934 0x000000000c76771e 0x000000000ca0fe32 0x000000000ca0ccf5 0x00007f011cb31fa3 0x00007f011ca624cf
2024.01.02 15:18:50.432547 [ 102143 ] {} <Fatal> BaseDaemon: ########################################
2024.01.02 15:18:50.432565 [ 102143 ] {} <Fatal> BaseDaemon: (version 23.9.2.56 (official build), build id: 76109A79FA62B9BC630A6C39438DEA7D28147B68, git hash: a1bf3f1de5) (from threa
d 102142) (no query) Received signal Segmentation fault (11)
```

```
2024.01.02 15:18:50.432588 [ 102143 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.
2024.01.02 15:18:50.432601 [ 102143 ] {} <Fatal> BaseDaemon: Stack trace: 0x00007f011cac5181 0x0000000015ccd934 0x000000000c76771e 0x000000000ca0fe32 0x000000000ca0ccf5 0x00007f011cb31fa3 0x00007f011ca624cf
2024.01.02 15:18:50.432638 [ 102143 ] {} <Fatal> BaseDaemon: 2. ? @ 0x000000000015c181 in /usr/lib/x86_64-linux-gnu/libc-2.28.so
2024.01.02 15:18:50.446953 [ 102143 ] {} <Fatal> BaseDaemon: 3.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/string:1955: String::__init(char const*, unsigned long)
2024.01.02 15:18:50.446981 [ 102143 ] {} <Fatal> BaseDaemon: 3.2. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/string:843: basic_string<std::nullptr_t>
2024.01.02 15:18:50.446998 [ 102143 ] {} <Fatal> BaseDaemon: 3. ./build_docker/./base/base/demangle.cpp:25: demangle(char const*, int&) @ 0x0000000015ccd934 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.458431 [ 102143 ] {} <Fatal> BaseDaemon: 4. ./build_docker/./src/Common/StackTrace.cpp:389: void toStringEveryLineImpl<std::function<void (std::basic_string_view<char, std::char_traits<char>>)>>(bool, StackTraceRefTriple const&, std::function<void (std::basic_string_view<char, std::char_traits<char>>)>&&) (.llvm.10713299015003964940) @ 0x000000000c76771e in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.474955 [ 102143 ] {} <Fatal> BaseDaemon: 5.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:818: ~__policy_func
2024.01.02 15:18:50.474997 [ 102143 ] {} <Fatal> BaseDaemon: 5.2. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1174: ~function
2024.01.02 15:18:50.475010 [ 102143 ] {} <Fatal> BaseDaemon: 5. ./build_docker/./src/Daemon/BaseDaemon.cpp:415: SignalListener::onFault(int, siginfo_t const&, ucontext_t*, StackTrace const&, std::vector<std::array<void*, 45ul>, std::allocator<std::array<void*, 45ul>>> const&, unsigned int, DB::ThreadStatus*) const @ 0x000000000ca0fe32 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.512270 [ 102143 ] {} <Fatal> BaseDaemon: 6.1. inlined from ./build_docker/./src/Daemon/BaseDaemon.cpp:284: operator()
2024.01.02 15:18:50.512591 [ 102143 ] {} <Fatal> BaseDaemon: 6.2. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<SignalListener::run()::'lambda'()>()()) std::__invoke[abi:v15000]<SignalListener::run()::'lambda'()>(SignalListener::run()::'lambda'()&&)
2024.01.02 15:18:50.512648 [ 102143 ] {} <Fatal> BaseDaemon: 6.3. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/thread:284: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, SignalListener::run()::'lambda'()>(std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, SignalListener::run()::'lambda'()>&, std::__tuple_indices<>)
2024.01.02 15:18:50.512664 [ 102143 ] {} <Fatal> BaseDaemon: 6. ./build_docker/./contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, SignalListener::run()::'lambda'()>>(void*) @ 0x000000000ca0ccf5 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.512698 [ 102143 ] {} <Fatal> BaseDaemon: 7. start_thread @ 0x0000000000007fa3 in /usr/lib/x86_64-linux-gnu/libpthread-2.28.so
2024.01.02 15:18:50.512721 [ 102143 ] {} <Fatal> BaseDaemon: 8. ? @ 0x00000000000f94cf in /usr/lib/x86_64-linux-gnu/libc-2.28.so
2024.01.02 15:18:50.666148 [ 102143 ] {} <Fatal> BaseDaemon: Integrity check of the executable successfully passed (checksum: 3A99FBFAA2DA54D46975E9ABC04E53E0)
2024.01.02 15:18:50.746677 [ 102143 ] {} <Fatal> BaseDaemon: Report this error to https://github.com/ClickHouse/ClickHouse/issues
```

**The problem is actually here** - [1].
Though to be precise here - [2].
See also - [3].

  [1]: 5e467b69c8/src/Common/StackTrace.cpp (L405)
  [2]: 4d734cf1e5/src/Common/StackTrace.cpp (L389)
  [3]: 5e467b69c8/src/Common/Dwarf.cpp (L1161)

_(It is not obvious from the patch, so will post a snippet)_

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-08 21:16:49 +03:00
Antonio Andelic
4640049b4a
Merge pull request #58598 from ClickHouse/keeper-small-fix
Small fix in Keeper
2024-01-08 17:07:16 +01:00
jinjunzh
cf53d9a17c add USE_QAT into build_options 2024-01-08 11:05:36 -05:00
Anton Popov
3e933c9099 Merge remote-tracking branch 'upstream/master' into HEAD 2024-01-08 15:33:38 +00:00
Sema Checherinda
55ec1a17e7 remove debug logging 2024-01-08 15:58:47 +01:00
Robert Schulze
fb4fd29110
Merge pull request #58454 from rschu1ze/punycode-revert-revert
Implement IDNA and punycode encoding/decoding (2nd attempt)
2024-01-08 14:26:03 +01:00
Antonio Andelic
a891750f53 Fix race 2024-01-08 12:08:17 +00:00
Robert Schulze
f3c77c046a
makeDateTime64(): Allow non-const fraction argument 2024-01-08 11:58:32 +00:00
Antonio Andelic
6aee18ced8
Merge pull request #58487 from ClickHouse/keeper-lower-log-levels-test
Lower log levels for some Raft logs to new test level
2024-01-08 12:19:02 +01:00
vdimir
631e16d8c3
Fix off-by-one error for max_joined_block_rows 2024-01-08 11:02:17 +00:00
Raúl Marín
c1b95560db Add some missing LLVM includes 2024-01-08 11:25:57 +01:00
vdimir
0f2588334f
Disable max_joined_block_rows in ConcurrentHashJoin 2024-01-08 10:20:39 +00:00
Antonio Andelic
c7b9f4aaed
Merge pull request #58455 from ClickHouse/keeper-even-better-startup
Avoid huge memory consumption during Keeper startup for more cases
2024-01-08 10:46:17 +01:00
Raúl Marín
8f998eaa82
Merge pull request #58447 from canhld94/fix_symlink_in_user_files
Allow users to work with symlinks in user_files_path (again)
2024-01-08 10:40:36 +01:00