Azat Khuzhin
9b6c8cd44f
Fix possible NULL dereference during symbolizing inline frames
...
It is possible sometimes:
<details>
```
2024.01.02 15:18:49.542342 [ 102142 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.
2024.01.02 15:18:49.542356 [ 102142 ] {} <Fatal> BaseDaemon: Stack trace: 0x0000000012817007 0x00000000120b3e88 0x000000001209c3e8 0x000000001209b7f3 0x000000001213d936 0x0000000011f71042 0x0000000011f6dd22 0x0000
000012496116 0x000000001249181c 0x00000000133197ec 0x000000001332bc79 0x0000000015d0eb14 0x0000000015d0fd11 0x0000000015e1b367 0x0000000015e195fc 0x00007f011cb31fa3 0x00007f011ca624cf
2024.01.02 15:18:50.052773 [ 102142 ] {} <Fatal> BaseDaemon: 2.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:603: shared_ptr<DB::Context, void>
2024.01.02 15:18:50.052829 [ 102142 ] {} <Fatal> BaseDaemon: 2.2. inlined from ./build_docker/./src/Planner/PlannerContext.h:56: DB::PlannerContext::getQueryContext() const
2024.01.02 15:18:50.052853 [ 102142 ] {} <Fatal> BaseDaemon: 2.3. inlined from ./build_docker/./src/Storages/StorageDistributed.cpp:666: DB::(anonymous namespace)::buildQueryTreeDistributed(DB::SelectQueryInfo&, s
td::shared_ptr<DB::StorageSnapshot> const&, DB::StorageID const&, std::shared_ptr<DB::IAST> const&)
2024.01.02 15:18:50.052872 [ 102142 ] {} <Fatal> BaseDaemon: 2. ./build_docker/./src/Storages/StorageDistributed.cpp:743: DB::StorageDistributed::read(DB::QueryPlan&, std::vector<String, std::allocator<String>> co
nst&, std::shared_ptr<DB::StorageSnapshot> const&, DB::SelectQueryInfo&, std::shared_ptr<DB::Context const>, DB::QueryProcessingStage::Enum, unsigned long, unsigned long) @ 0x0000000012817007 in /usr/lib/debug/usr
/bin/clickhouse.debug
2024.01.02 15:18:50.211769 [ 102142 ] {} <Fatal> BaseDaemon: 3.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:815: std::shared_ptr<DB::Context>::operator->[abi:v15000]()
const
2024.01.02 15:18:50.211815 [ 102142 ] {} <Fatal> BaseDaemon: 3. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:2488: DB::InterpreterSelectQuery::executeFetchColumns(DB::QueryProcessingStage::Enum, DB
::QueryPlan&) @ 0x00000000120b3e88 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.295382 [ 102142 ] {} <Fatal> BaseDaemon: 4. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:1444: DB::InterpreterSelectQuery::executeImpl(DB::QueryPlan&, std::optional<DB::Pipe>) @
0x000000001209c3e8 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.375779 [ 102142 ] {} <Fatal> BaseDaemon: 5.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/optional:260: ~__optional_destruct_base
2024.01.02 15:18:50.375820 [ 102142 ] {} <Fatal> BaseDaemon: 5. ./build_docker/./src/Interpreters/InterpreterSelectQuery.cpp:868: DB::InterpreterSelectQuery::buildQueryPlan(DB::QueryPlan&) @ 0x000000001209b7f3 in
/usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.403975 [ 102142 ] {} <Fatal> BaseDaemon: 6. ./build_docker/./src/Interpreters/InterpreterSelectWithUnionQuery.cpp:0: DB::InterpreterSelectWithUnionQuery::buildQueryPlan(DB::QueryPlan&) @ 0x0000
00001213d936 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.432051 [ 102142 ] {} <Fatal> BaseDaemon: 7.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:603: shared_ptr<DB::Context, void>
2024.01.02 15:18:50.432472 [ 102143 ] {} <Fatal> BaseDaemon: ########## Short fault info ############
2024.01.02 15:18:50.432500 [ 102143 ] {} <Fatal> BaseDaemon: (version 23.9.2.56 (official build), build id: 76109A79FA62B9BC630A6C39438DEA7D28147B68, git hash: a1bf3f1de5
) (from threa
d 102142) Received signal 11
2024.01.02 15:18:50.432516 [ 102143 ] {} <Fatal> BaseDaemon: Signal description: Segmentation fault
2024.01.02 15:18:50.432526 [ 102143 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.
2024.01.02 15:18:50.432539 [ 102143 ] {} <Fatal> BaseDaemon: Stack trace: 0x00007f011cac5181 0x0000000015ccd934 0x000000000c76771e 0x000000000ca0fe32 0x000000000ca0ccf5 0x00007f011cb31fa3 0x00007f011ca624cf
2024.01.02 15:18:50.432547 [ 102143 ] {} <Fatal> BaseDaemon: ########################################
2024.01.02 15:18:50.432565 [ 102143 ] {} <Fatal> BaseDaemon: (version 23.9.2.56 (official build), build id: 76109A79FA62B9BC630A6C39438DEA7D28147B68, git hash: a1bf3f1de5
) (from threa
d 102142) (no query) Received signal Segmentation fault (11)
```
```
2024.01.02 15:18:50.432588 [ 102143 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.
2024.01.02 15:18:50.432601 [ 102143 ] {} <Fatal> BaseDaemon: Stack trace: 0x00007f011cac5181 0x0000000015ccd934 0x000000000c76771e 0x000000000ca0fe32 0x000000000ca0ccf5 0x00007f011cb31fa3 0x00007f011ca624cf
2024.01.02 15:18:50.432638 [ 102143 ] {} <Fatal> BaseDaemon: 2. ? @ 0x000000000015c181 in /usr/lib/x86_64-linux-gnu/libc-2.28.so
2024.01.02 15:18:50.446953 [ 102143 ] {} <Fatal> BaseDaemon: 3.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/string:1955: String::__init(char const*, unsigned long)
2024.01.02 15:18:50.446981 [ 102143 ] {} <Fatal> BaseDaemon: 3.2. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/string:843: basic_string<std::nullptr_t>
2024.01.02 15:18:50.446998 [ 102143 ] {} <Fatal> BaseDaemon: 3. ./build_docker/./base/base/demangle.cpp:25: demangle(char const*, int&) @ 0x0000000015ccd934 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.458431 [ 102143 ] {} <Fatal> BaseDaemon: 4. ./build_docker/./src/Common/StackTrace.cpp:389: void toStringEveryLineImpl<std::function<void (std::basic_string_view<char, std::char_traits<char>>)>>(bool, StackTraceRefTriple const&, std::function<void (std::basic_string_view<char, std::char_traits<char>>)>&&) (.llvm.10713299015003964940) @ 0x000000000c76771e in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.474955 [ 102143 ] {} <Fatal> BaseDaemon: 5.1. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:818: ~__policy_func
2024.01.02 15:18:50.474997 [ 102143 ] {} <Fatal> BaseDaemon: 5.2. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1174: ~function
2024.01.02 15:18:50.475010 [ 102143 ] {} <Fatal> BaseDaemon: 5. ./build_docker/./src/Daemon/BaseDaemon.cpp:415: SignalListener::onFault(int, siginfo_t const&, ucontext_t*, StackTrace const&, std::vector<std::array<void*, 45ul>, std::allocator<std::array<void*, 45ul>>> const&, unsigned int, DB::ThreadStatus*) const @ 0x000000000ca0fe32 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.512270 [ 102143 ] {} <Fatal> BaseDaemon: 6.1. inlined from ./build_docker/./src/Daemon/BaseDaemon.cpp:284: operator()
2024.01.02 15:18:50.512591 [ 102143 ] {} <Fatal> BaseDaemon: 6.2. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<SignalListener::run()::'lambda'()>()()) std::__invoke[abi:v15000]<SignalListener::run()::'lambda'()>(SignalListener::run()::'lambda'()&&)
2024.01.02 15:18:50.512648 [ 102143 ] {} <Fatal> BaseDaemon: 6.3. inlined from ./build_docker/./contrib/llvm-project/libcxx/include/thread:284: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, SignalListener::run()::'lambda'()>(std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, SignalListener::run()::'lambda'()>&, std::__tuple_indices<>)
2024.01.02 15:18:50.512664 [ 102143 ] {} <Fatal> BaseDaemon: 6. ./build_docker/./contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, SignalListener::run()::'lambda'()>>(void*) @ 0x000000000ca0ccf5 in /usr/lib/debug/usr/bin/clickhouse.debug
2024.01.02 15:18:50.512698 [ 102143 ] {} <Fatal> BaseDaemon: 7. start_thread @ 0x0000000000007fa3 in /usr/lib/x86_64-linux-gnu/libpthread-2.28.so
2024.01.02 15:18:50.512721 [ 102143 ] {} <Fatal> BaseDaemon: 8. ? @ 0x00000000000f94cf in /usr/lib/x86_64-linux-gnu/libc-2.28.so
2024.01.02 15:18:50.666148 [ 102143 ] {} <Fatal> BaseDaemon: Integrity check of the executable successfully passed (checksum: 3A99FBFAA2DA54D46975E9ABC04E53E0)
2024.01.02 15:18:50.746677 [ 102143 ] {} <Fatal> BaseDaemon: Report this error to https://github.com/ClickHouse/ClickHouse/issues
```
**The problem is actually here** - [1].
Though to be precise here - [2].
See also - [3].
[1]: 5e467b69c8/src/Common/StackTrace.cpp (L405)
[2]: 4d734cf1e5/src/Common/StackTrace.cpp (L389)
[3]: 5e467b69c8/src/Common/Dwarf.cpp (L1161)
_(It is not obvious from the patch, so will post a snippet)_
</details>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-08 21:16:49 +03:00
Bharat Nallan Chakravarthy
00b5e28d6a
Merge upstream/master into ncb/system-database-engines
2023-12-30 23:10:48 -08:00
Bharat Nallan Chakravarthy
3e8439a78b
add to context access and suggest
2023-12-30 20:38:35 -08:00
Bharat Nallan Chakravarthy
3c4ac0ba08
add system.database_engines table
2023-12-30 20:31:25 -08:00
Alexey Milovidov
7bded0a5e7
Merge pull request #58316 from ClickHouse/reintroduce_is_deleted
...
Re-introduce `is_deleted` column for ReplacingMergeTree
2023-12-31 00:57:19 +01:00
Alexey Milovidov
b752a1c62e
Merge pull request #58365 from bharatnc/ncb/refactor-db-factory
...
independent registration of database engines
2023-12-31 00:56:24 +01:00
Bharat Nallan Chakravarthy
6c4a4991e2
few more fixes
2023-12-30 09:26:53 -08:00
Alexey Milovidov
c1c7218d53
Merge pull request #58369 from ClickHouse/remove-function
...
Remove function `reverseDNSQuery`
2023-12-30 17:51:56 +01:00
Alexey Milovidov
063463b0a8
Merge pull request #58359 from ClickHouse/all-system-tables-local
...
Attach all system tables in `clickhouse-local`
2023-12-30 17:51:27 +01:00
Alexey Milovidov
78523639e0
Remove function
2023-12-30 14:52:54 +01:00
Antonio Andelic
d86e59eb08
Merge pull request #58308 from ClickHouse/keeper-preprocess-fix
...
Handle another case for preprocessing in Keeper
2023-12-30 14:13:04 +01:00
Alexey Milovidov
8fc05e25fe
Merge pull request #58310 from azat/kafka-fix-stat-leak-resubmit
...
Create consumers for Kafka tables on fly with TTL (resubmit)
2023-12-30 13:03:16 +01:00
Alexey Milovidov
e1812f3b58
Merge pull request #58266 from ClickHouse/vdimir/simple_fix_tuple_elimination
...
Analyzer: fix tuple comparison when result is always null
2023-12-30 13:02:38 +01:00
Alexey Milovidov
8365722c95
Merge pull request #58336 from amosbird/macro2constexpr
...
Organize symbols from src/* into DB namespace
2023-12-30 12:40:36 +01:00
Alexey Milovidov
aa6ecd2d59
Merge pull request #58343 from azat/s3/optional-gcs-compose
...
Avoid sending ComposeObject requests after upload to GCS
2023-12-30 12:40:04 +01:00
Alexey Milovidov
40ca9c202d
Merge pull request #58346 from ClickHouse/check-what-would-be-ifremove-array-joined-columns-from-key-condition
...
Check what happen if remove array joined columns from KeyCondition
2023-12-30 12:38:57 +01:00
Alexey Milovidov
f058394d92
Merge pull request #58351 from ClickHouse/fix_00002
...
Keep exception format string in retries ctl
2023-12-30 12:37:36 +01:00
Alexey Milovidov
ebd95586d2
Merge pull request #58358 from ClickHouse/format-constexpr
...
Make function `format` return a constant on constant arguments.
2023-12-30 12:34:56 +01:00
Alexey Milovidov
1d344026be
Merge pull request #58360 from ClickHouse/enum-transform
...
Support for Enum in function `transform`
2023-12-30 12:32:49 +01:00
Bharat Nallan Chakravarthy
906745feaa
validation and left over fixes
2023-12-29 20:06:07 -08:00
robot-ch-test-poll3
c0290d1cfd
Merge pull request #58354 from azat/xml-conf-dot
...
[RFC] Correctly handle keys with dot in the name in configurations XMLs
2023-12-30 03:59:16 +01:00
Amos Bird
907e844635
Fix examples
2023-12-30 10:48:19 +08:00
Bharat Nallan Chakravarthy
af534db4d9
style fixes and cleanup
2023-12-29 16:38:17 -08:00
Bharat Nallan Chakravarthy
ab28da85d3
try wire everything up
2023-12-29 16:38:17 -08:00
Bharat Nallan Chakravarthy
8e83d3604d
register all remaining databases
2023-12-29 16:38:02 -08:00
Alexey Milovidov
e1eca1ee39
Support for Enum in function transform
2023-12-29 21:48:38 +01:00
Alexey Milovidov
7c2d868ef1
Revert "Support for Enum in function transform
"
...
This reverts commit 06628f76d7
.
2023-12-29 21:48:14 +01:00
Alexey Milovidov
06628f76d7
Support for Enum in function transform
2023-12-29 21:47:53 +01:00
Alexey Milovidov
39b239683c
Attach all system tables in clickhouse-local
2023-12-29 21:25:22 +01:00
Alexey Milovidov
9d8c26331e
Make format
constexpr
2023-12-29 21:09:58 +01:00
Amos Bird
18395a387c
Fix another unitest
2023-12-30 01:14:21 +08:00
Alexander Tokmakov
790ededf80
Update BackupCoordinationStageSync.cpp
2023-12-29 17:58:24 +01:00
Azat Khuzhin
9a8b308a3d
Correctly handle keys with dot in the name in configurations XMLs
...
For this I've added escape of the keys returned by keys() method, and
handle this escaping in get*() methods.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 17:30:23 +01:00
Alexander Tokmakov
302ddeb6d0
Update BackupCoordinationStageSync.cpp
2023-12-29 17:08:13 +01:00
robot-ch-test-poll3
07ba672e37
Merge pull request #58142 from canhld94/final_less_compare
...
MergeTree FINAL to not compare rows from same non-L0 part
2023-12-29 16:47:14 +01:00
Alexander Tokmakov
1013f6b23f
Merge branch 'master' into reintroduce_is_deleted
2023-12-29 15:46:24 +01:00
Alexander Tokmakov
72a0797b88
keep exception format string in retries ctl
2023-12-29 15:21:46 +01:00
Alexey Milovidov
ea03cc82aa
Merge pull request #58320 from ClickHouse/mv3
...
Refreshable materialized views again
2023-12-29 14:44:50 +01:00
Azat Khuzhin
853fdfe775
Clean cached messages on destroy kafka consumer
...
The callchain of the kafka consumer is very tricky, so for the sake of
common sense let's just clean the messages on moving out consumer (and
in dtor, but this is just to keep that two code path in sync).
(Also reported by @filimonov)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 14:30:21 +01:00
Azat Khuzhin
b3d6caf37f
Unsubscribe kafka consumer before cleaning it by TTL
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 14:03:53 +01:00
Amos Bird
0d801c4f6b
Fix unitest
2023-12-29 20:40:52 +08:00
Nikolai Kochetov
0e8232a8c3
Check what happen if remove array joined columns from KeyCondition
2023-12-29 12:24:19 +00:00
Azat Khuzhin
f578541ded
Fix destructing kafka consumer via member orders
...
We've discussed this with @filimonov and he pointed out that everything
else (except for rdkafka_stat/rdkafka_stat_mutex) is done via members
orders, so let's do it in the same style.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 13:19:11 +01:00
Alexey Milovidov
09181b6c37
Merge pull request #58335 from lingtaolf/feature/sparkBar_alias_to_sparkbar
...
sparkBar as an alias to sparkbar
2023-12-29 13:02:55 +01:00
Azat Khuzhin
8c54380d80
Avoid sending ComposeObject requests after upload to GCS
...
This should not be required anymore, but leave it as an option, since
likely this is required for old files.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
Azat Khuzhin
f4a7789cd4
Convert various S3::Client settings into separate ClientSettings struct
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
Azat Khuzhin
2b2d10689b
Make S3::Client::ComposeObject private
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
Duc Canh Le
91a87d6b6c
better implementation
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-12-29 07:27:10 +00:00
Amos Bird
6b6e40831c
Move symbols from src/* into namespace DB
2023-12-29 14:37:08 +08:00
Bharat Nallan Chakravarthy
1dce048b58
initial refactor of db registration
2023-12-28 18:59:45 -08:00