thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 20:42:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
131,051 Commits 9,183 Branches 1,782 Tags 2.2 GiB
e09c68db32
Commit Graph

50828 Commits

Author SHA1 Message Date
Raúl Marín
e6be38be4d Adapt from HEAD 2023-12-11 16:28:00 +01:00
Vitaly Baranov
ca262d6e10 Fix profile events. 2023-12-11 16:03:07 +01:00
avogar
a87a8e91cf Slightly better inference of unnamed tupes in JSON formats 2023-12-11 14:46:12 +00:00
Vitaly Baranov
2c03348732 Use checkTimeLimit() to check if query is cancelled. 2023-12-11 15:33:16 +01:00
Raúl Marín
a55a0c068e Merge remote-tracking branch 'blessed/master' into backup_1 2023-12-11 15:27:21 +01:00
Nikolai Kochetov
fcaa556bf7 Fixing test. 2023-12-11 13:39:10 +00:00
Raúl Marín
1b844e2116 Apply JOIN check to custom key parallel replicas too 2023-12-11 14:22:58 +01:00
Raúl Marín
6370a1b824 Merge remote-tracking branch 'blessed/master' into i48496 2023-12-11 14:22:54 +01:00
kssenii
e10d06af6f Better 2023-12-11 14:07:28 +01:00
Igor Nikonov
4ccf4e11a8 Parallel replicas (perf): announcement response handling improvement 2023-12-11 13:00:08 +00:00
Maksim Kita
821c7d6527 Fixed tests 2023-12-11 15:48:52 +03:00
kssenii
1e4d61d55a Change error code 2023-12-11 13:44:12 +01:00
Vitaly Baranov
ef8068ed03
Merge pull request #57733 from vitlibar/fix-rwlock 
Fix RWLock inconsistency after write lock timeout (again)
 2023-12-11 13:35:17 +01:00
vdimir
af45e138ad
fix 2023-12-11 12:33:11 +00:00
kssenii
b8caa00d24 Commit forgotten file 2023-12-11 13:28:36 +01:00
Nikolay Degterinsky
64cd83fcd9
Merge pull request #57713 from evillique/fix-unary-operators-parsing 
Fix unary operators parsing
 2023-12-11 13:18:00 +01:00
alesapin
a62a0b92de Increase async block cache deduplication timeout 2023-12-11 12:46:22 +01:00
kssenii
966a09fb70 Review fixes 2023-12-11 12:42:23 +01:00
vdimir
cdfe99c380
Remove unused template parameter from fillColumnsFromMap 2023-12-11 11:34:54 +00:00
Robert Schulze
c141dd1330
Merge pull request #57549 from rschu1ze/format-arbitrary-types 
Support arguments of arbitary type in function `format()`
 2023-12-11 12:33:51 +01:00
Kruglov Pavel
6213593206
Merge pull request #57735 from bharatnc/ncb/functions-hashing-ssl-inline-doc 
add inline documentation for ssl hashing functions
 2023-12-11 12:27:10 +01:00
kssenii
d34a2782ff Allow to decrease the number of download threads 2023-12-11 12:17:26 +01:00
flynn
c78649bdcf Support negative positional arguments 2023-12-11 10:36:26 +00:00
Robert Schulze
9e434d43d7
Merge remote-tracking branch 'rschu1ze/master' into substr-with-enums 2023-12-11 10:21:08 +00:00
kssenii
a06179da00 Merge remote-tracking branch 'origin/master' into fix-materialized-pg-issue-with-generated-columns 2023-12-11 11:15:00 +01:00
kssenii
3652419873 Update run.sh 2023-12-11 11:01:13 +01:00
Maksim Kita
81250d1a90 SerializationString improve performance 2023-12-11 12:55:31 +03:00
vdimir
efb82a93c5
Merge pull request #57662 from ClickHouse/vdimir/analyzer_join_on_true_nullable 
Analyzer: Fix JOIN ON true with join_use_nulls
 2023-12-11 10:52:21 +01:00
Konstantin Bogdanov
7172a8ec9a
Remove redundant include 2023-12-11 10:20:43 +01:00
Konstantin Bogdanov
76966818ba
Fix FreeBSD build 2023-12-11 10:20:34 +01:00
Konstantin Bogdanov
771d5c58ea
Fix Keeper standalone build 2023-12-11 10:19:49 +01:00
Konstantin Bogdanov
89ba4a8452
Add HTTP readiness endpoint 2023-12-11 10:19:42 +01:00
MikhailBurdukov
119e451967 Merge branch 'master' into native_copy_for_s3_disks 2023-12-11 07:25:20 +00:00
Bharat Nallan Chakravarthy
e006454eb2 add inline docs for hash functions 2023-12-10 22:05:24 -08:00
Alexey Milovidov
e443567c0e
Merge pull request #52717 from azat/wait-threadpool 
[RFC] Correctly wait background threads
 2023-12-11 01:06:23 +01:00
Vitaly Baranov
952e58287a Fix flaky test RWLockWriteLockTimeoutDuringTwoReads, add more comments. 2023-12-11 00:56:59 +01:00
Vitaly Baranov
73547e6d8a Fixed condition and add more chasserts for consistency checks. 2023-12-11 00:56:51 +01:00
Vitaly Baranov
4aba3ec685 Add some comments. 2023-12-11 00:56:40 +01:00
Vitaly Baranov
f1ccf4e29f Show owner query ids in the message for the DEADLOCK_AVOIDED error. 2023-12-11 00:56:17 +01:00
Vitaly Baranov
dc83e27164 Fix RWLock inconsistency after write lock timeout. 2023-12-11 00:56:06 +01:00
Vitaly Baranov
fc00569db6 Add more tests for RWLock. 2023-12-11 00:55:55 +01:00
Nikolay Degterinsky
ed23f47e72 Yet another fix 2023-12-10 23:04:29 +00:00
Alexey Milovidov
644ef7b63f
Revert "Fix RWLock inconsistency after write lock timeout" 2023-12-10 23:51:59 +03:00
Alexey Milovidov
0a20ce5d32
Merge pull request #55271 from azat/local-lazy-database 
Replace --no-system-tables with loading virtual tables of system database lazily
 2023-12-10 19:51:17 +01:00
Azat Khuzhin
450e8da7ce Fix data race on vptr (ctor/dtor vs virtual call) in AsynchronousMetrics 
TSan report [1]:

     Exception: Sanitizer assert found for instance ==================
     WARNING: ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) (pid=1)
       Write of size 8 at 0x7ffce68f5680 by main thread:
         0 DB::AsynchronousMetrics::~AsynchronousMetrics() build_docker/./src/Common/AsynchronousMetrics.cpp:299:1 (clickhouse+0xf38fa6e) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         1 DB::ServerAsynchronousMetrics::~ServerAsynchronousMetrics() build_docker/./src/Interpreters/ServerAsynchronousMetrics.h:10:7 (clickhouse+0xf2b2f0f) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         2 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) build_docker/./programs/server/Server.cpp:1996:1 (clickhouse+0xf2b2f0f)
         3 Poco::Util::Application::run() build_docker/./base/poco/Util/src/Application.cpp:315:8 (clickhouse+0x1d6c95fe) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         4 DB::Server::run() build_docker/./programs/server/Server.cpp:397:25 (clickhouse+0xf29e041) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         5 Poco::Util::ServerApplication::run(int, char**) build_docker/./base/poco/Util/src/ServerApplication.cpp:131:9 (clickhouse+0x1d6e9354) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         6 mainEntryClickHouseServer(int, char**) build_docker/./programs/server/Server.cpp:203:20 (clickhouse+0xf29b1a3) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         7 main build_docker/./programs/main.cpp:505:12 (clickhouse+0x72d7fa0) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)

       Previous read of size 8 at 0x7ffce68f5680 by thread T649:
         0 DB::AsynchronousMetrics::update(std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000l>>>) build_docker/./src/Common/AsynchronousMetrics.cpp:1559:5 (clickhouse+0xf38ebc5) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         1 DB::AsynchronousMetrics::run() build_docker/./src/Common/AsynchronousMetrics.cpp:354:13 (clickhouse+0xf390696) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         2 DB::AsynchronousMetrics::start()::$_0::operator()() const build_docker/./src/Common/AsynchronousMetrics.cpp:273:62 (clickhouse+0xf392eca) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         3 decltype(std::declval<DB::AsynchronousMetrics::start()::$_0&>()()) std::__1::__invoke[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&>(DB::AsynchronousMetrics::start()::$_0&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf392eca)
         4 decltype(auto) std::__1::__apply_tuple_impl[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&>(DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/tuple:1789:1 (clickhouse+0xf392eca)
         5 decltype(auto) std::__1::apply[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&>(DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&) build_docker/./contrib/llvm-project/libcxx/include/tuple:1798:1 (clickhouse+0xf392eca)
         6 ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()::operator()() build_docker/./src/Common/ThreadPool.h:253:13 (clickhouse+0xf392eca)
         7 decltype(std::declval<DB::AsynchronousMetrics::start()::$_0>()()) std::__1::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&>(DB::AsynchronousMetrics::start()::$_0&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf392eca)
         8 void std::__1::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9 (clickhouse+0xf392eca)
         9 std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 (clickhouse+0xf392eca)
         10 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'(), void ()>>(std::__1::__function::__policy_storage const*) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 (clickhouse+0xf392eca)
         11 std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 (clickhouse+0xf30682e) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         12 std::__1::function<void ()>::operator()() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 (clickhouse+0xf30682e)
         13 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) build_docker/./src/Common/ThreadPool.cpp:421:13 (clickhouse+0xf30682e)
         14 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()::operator()() const build_docker/./src/Common/ThreadPool.cpp:183:73 (clickhouse+0xf30d3d1) (BuildId: d49dc55f80b62802f8721fdbcb07bdc88ac204a4)
         15 decltype(std::declval<void>()()) std::__1::__invoke[abi:v15000]<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf30d3d1)
         16 void std::__1::__thread_execute[abi:v15000]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/thread:284:5 (clickhouse+0xf30d3d1)
         17 void* std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>>(void*) build_docker/./contrib/llvm-project/libcxx/include/thread:295:5 (clickhouse+0xf30d3d1)

     SUMMARY: ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) build_docker/./src/Common/AsynchronousMetrics.cpp:299:1 in DB::AsynchronousMetrics::~AsynchronousMetrics()

And the problem is real, see [2].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/52717/a6cf2297ef07ca0b065d944c9e30a3986285331d/integration_tests__tsan__[2_6]/integration_run_parallel0_0.log
  [2]: https://github.com/google/sanitizers/wiki/ThreadSanitizerPopularDataRaces#data-race-on-vptr

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-10 18:32:40 +01:00
Alexey Milovidov
5a33fdcd4d
Merge pull request #57645 from bharatnc/ncb/sha512-256 
add SHA512_256 function
 2023-12-10 18:02:09 +01:00
Alexey Milovidov
e21c60191a
Merge pull request #57710 from azat/sparse-column-fix-aggregation 
Fix SIGSEGV for aggregation of sparse columns with any() RESPECT NULL
 2023-12-10 17:54:57 +01:00
Alexey Milovidov
d119052895
Merge pull request #51858 from ClickHouse/CurtizJ-fix-ttl-non-deterministic 
Merging #37286
 2023-12-10 17:50:36 +01:00
Nikolay Degterinsky
8b5c694684 Fix 2023-12-10 16:45:17 +00:00
Azat Khuzhin
2707e38c69 Fix dead-lock in DatabaseMaterializedPostgreSQL after interface changes 
Deadlock for ATTACH:

    /src/ch/clickhouse/src/Common/ThreadFuzzer.cpp:0::pthread_mutex_lock
    /src/ch/clickhouse/contrib/llvm-project/libcxx/src/mutex.cpp:39::std::__1::mutex::lock()
    /src/ch/clickhouse/src/Databases/DatabaseOrdinary.cpp:240::DB::DatabaseOrdinary::waitTableStarted(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const
    /src/ch/clickhouse/src/Databases/DatabasesCommon.cpp:203::DB::DatabaseWithOwnTablesBase::tryGetTable(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>) const
    /src/ch/clickhouse/src/Databases/PostgreSQL/DatabaseMaterializedPostgreSQL.cpp:335::DB::DatabaseMaterializedPostgreSQL::attachTableUnlocked(std::__1::shared_ptr<DB::Context const>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::IStorage> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)
    /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__mutex_base💯:DB::IDatabase::attachTable(std::__1::shared_ptr<DB::Context const>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::IStorage> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)
    /src/ch/clickhouse/contrib/llvm-project/libcxx/include/string:1499::DB::DatabaseMaterializedPostgreSQL::createTable(std::__1::shared_ptr<DB::Context const>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::IAST> const&)
    /src/ch/clickhouse/contrib/llvm-project/libcxx/include/string:1499::DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::__1::unique_ptr<DB::DDLGuard, std::__1::default_delete<DB::DDLGuard>>&)
    /src/ch/clickhouse/src/Interpreters/InterpreterCreateQuery.cpp:0::DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&)
    /src/ch/clickhouse/src/Interpreters/InterpreterCreateQuery.cpp:1751::DB::InterpreterCreateQuery::execute()
    /src/ch/clickhouse/src/Interpreters/executeQuery.cpp:0::DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*)
    /src/ch/clickhouse/src/Interpreters/executeQuery.cpp:1287::DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum)
    /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612::DB::TCPHandler::runImpl()
    /src/ch/clickhouse/src/Server/TCPHandler.cpp:2294::DB::TCPHandler::run()
    /src/ch/clickhouse/base/poco/Net/src/TCPServerConnection.cpp:57::Poco::Net::TCPServerConnection::start()
    /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48::Poco::Net::TCPServerDispatcher::run()
    /src/ch/clickhouse/base/poco/Foundation/src/ThreadPool.cpp:202::Poco::PooledThread::run()
    /src/ch/clickhouse/base/poco/Foundation/include/Poco/SharedPtr.h:139::Poco::ThreadImpl::runnableEntry(void*)

CI: https://s3.amazonaws.com/clickhouse-test-reports/55271/ece8c86ebc02908e36786ca32d68f9ce0ddcb35d/integration_tests__tsan__[1_6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-10 16:06:18 +01:00
Azat Khuzhin
a6cf2297ef Fix Context use-after-free in ServerAsynchronousMetrics 
TSan found [1]:

    WARNING: ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) (pid=598)
      Write of size 8 at 0x7ffcf9ad9cb0 by main thread:
        0 DB::AsynchronousMetrics::~AsynchronousMetrics() build_docker/./src/Common/AsynchronousMetrics.cpp:299:1 (clickhouse+0xf38ed6e) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        1 DB::ServerAsynchronousMetrics::~ServerAsynchronousMetrics() build_docker/./src/Interpreters/ServerAsynchronousMetrics.h:10:7 (clickhouse+0xf2b2220) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        2 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) build_docker/./programs/server/Server.cpp:1995:1 (clickhouse+0xf2b2220)
        3 Poco::Util::Application::run() build_docker/./base/poco/Util/src/Application.cpp:315:8 (clickhouse+0x1d6c535e) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        4 DB::Server::run() build_docker/./programs/server/Server.cpp:397:25 (clickhouse+0xf29d341) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        5 Poco::Util::ServerApplication::run(int, char**) build_docker/./base/poco/Util/src/ServerApplication.cpp:131:9 (clickhouse+0x1d6e50b4) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        6 mainEntryClickHouseServer(int, char**) build_docker/./programs/server/Server.cpp:203:20 (clickhouse+0xf29a4a3) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        7 main build_docker/./programs/main.cpp:505:12 (clickhouse+0x72d72a0) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)

      Previous read of size 8 at 0x7ffcf9ad9cb0 by thread T727:
        0 DB::AsynchronousMetrics::update(std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000l>>>) build_docker/./src/Common/AsynchronousMetrics.cpp:1559:5 (clickhouse+0xf38dec5) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        1 DB::AsynchronousMetrics::run() build_docker/./src/Common/AsynchronousMetrics.cpp:354:13 (clickhouse+0xf38f996) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        2 DB::AsynchronousMetrics::start()::$_0::operator()() const build_docker/./src/Common/AsynchronousMetrics.cpp:273:62 (clickhouse+0xf3921ca) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        3 decltype(std::declval<DB::AsynchronousMetrics::start()::$_0&>()()) std::__1::__invoke[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&>(DB::AsynchronousMetrics::start()::$_0&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf3921ca)
        4 decltype(auto) std::__1::__apply_tuple_impl[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&>(DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/tuple:1789:1 (clickhouse+0xf3921ca)
        5 decltype(auto) std::__1::apply[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&>(DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&) build_docker/./contrib/llvm-project/libcxx/include/tuple:1798:1 (clickhouse+0xf3921ca)
        6 ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()::operator()() build_docker/./src/Common/ThreadPool.h:253:13 (clickhouse+0xf3921ca)
        7 decltype(std::declval<DB::AsynchronousMetrics::start()::$_0>()()) std::__1::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&>(DB::AsynchronousMetrics::start()::$_0&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf3921ca)
        8 void std::__1::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9 (clickhouse+0xf3921ca)
        9 std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 (clickhouse+0xf3921ca)
        10 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'(), void ()>>(std::__1::__function::__policy_storage const*) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 (clickhouse+0xf3921ca)
        11 std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 (clickhouse+0xf305b2e) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        12 std::__1::function<void ()>::operator()() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 (clickhouse+0xf305b2e)
        13 ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) build_docker/./src/Common/ThreadPool.cpp:421:13 (clickhouse+0xf305b2e)
        14 void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()::operator()() const build_docker/./src/Common/ThreadPool.cpp:183:73 (clickhouse+0xf30c6d1) (BuildId: 31dcc7d77a0b2aaf9de7aca070b5f6ed6ac3dcbf)
        15 decltype(std::declval<void>()()) std::__1::__invoke[abi:v15000]<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf30c6d1)
        16 void std::__1::__thread_execute[abi:v15000]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/thread:284:5 (clickhouse+0xf30c6d1)
        17 void* std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>>(void*) build_docker/./contrib/llvm-project/libcxx/include/thread:295:5 (clickhouse+0xf30c6d1)

    SUMMARY: ThreadSanitizer: data race on vptr (ctor/dtor vs virtual call) build_docker/./src/Common/AsynchronousMetrics.cpp:299:1 in DB::AsynchronousMetrics::~AsynchronousMetrics()

Or MSan [2]:

    ==573==WARNING: MemorySanitizer: use-of-uninitialized-value
        0 0x55b57079112d in std::__1::weak_ptr<DB::Context const>::lock() const build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1645:20
        1 0x55b57079112d in DB::WithContextImpl<std::__1::shared_ptr<DB::Context const>>::getContext() const build_docker/./src/Interpreters/Context_fwd.h:41:28
        2 0x55b57079112d in DB::ServerAsynchronousMetrics::updateImpl(std::__1::unordered_map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, DB::AsynchronousMetricValue, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, DB::AsynchronousMetricValue>>>&, std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000l>>>, std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000l>>>) build_docker/./src/Interpreters/ServerAsynchronousMetrics.cpp:260:63
        3 0x55b559540309 in DB::AsynchronousMetrics::update(std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000l>>>) build_docker/./src/Common/AsynchronousMetrics.cpp:1559:5
        4 0x55b55954258c in DB::AsynchronousMetrics::run() build_docker/./src/Common/AsynchronousMetrics.cpp:354:13
        5 0x55b559549111 in DB::AsynchronousMetrics::start()::$_0::operator()() const build_docker/./src/Common/AsynchronousMetrics.cpp:273:62
        6 0x55b559549111 in decltype(std::declval<DB::AsynchronousMetrics::start()::$_0&>()()) std::__1::__invoke[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&>(DB::AsynchronousMetrics::start()::$_0&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23
        7 0x55b559549111 in decltype(auto) std::__1::__apply_tuple_impl[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&>(DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/tuple:1789:1
        8 0x55b559549111 in decltype(auto) std::__1::apply[abi:v15000]<DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&>(DB::AsynchronousMetrics::start()::$_0&, std::__1::tuple<>&) build_docker/./contrib/llvm-project/libcxx/include/tuple:1798:1
        9 0x55b559549111 in ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()::operator()() build_docker/./src/Common/ThreadPool.h:253:13
        10 0x55b559549111 in decltype(std::declval<DB::AsynchronousMetrics::start()::$_0>()()) std::__1::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&>(DB::AsynchronousMetrics::start()::$_0&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23
        11 0x55b559549111 in void std::__1::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'()&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9
        12 0x55b559549111 in std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12
        13 0x55b559549111 in void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<DB::AsynchronousMetrics::start()::$_0>(DB::AsynchronousMetrics::start()::$_0&&)::'lambda'(), void ()>>(std::__1::__function::__policy_storage const*) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16
        14 0x55b5593eb38a in std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16
        15 0x55b5593eb38a in std::__1::function<void ()>::operator()() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12
        16 0x55b5593eb38a in ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) build_docker/./src/Common/ThreadPool.cpp:421:13
        17 0x55b5593f9a0a in void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()::operator()() const build_docker/./src/Common/ThreadPool.cpp:183:73
        18 0x55b5593f9a0a in decltype(std::declval<void>()()) std::__1::__invoke[abi:v15000]<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(void&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23
        19 0x55b5593f9a0a in void std::__1::__thread_execute[abi:v15000]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>(std::__1::tuple<void, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>&, std::__1::__tuple_indices<>) build_docker/./contrib/llvm-project/libcxx/include/thread:284:5
        20 0x55b5593f9a0a in void* std::__1::__thread_proxy[abi:v15000]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, Priority, std::__1::optional<unsigned long>, bool)::'lambda0'()>>(void*) build_docker/./contrib/llvm-project/libcxx/include/thread:295:5
        21 0x7f7ff3899ac2  (/lib/x86_64-linux-gnu/libc.so.6+0x94ac2) (BuildId: a43bfc8428df6623cd498c9c0caeb91aec9be4f9)
        22 0x7f7ff392ba3f  (/lib/x86_64-linux-gnu/libc.so.6+0x126a3f) (BuildId: a43bfc8428df6623cd498c9c0caeb91aec9be4f9)

      Member fields were destroyed
        0 0x55b541a72c9d in __sanitizer_dtor_callback_fields (/usr/bin/clickhouse+0x7c6dc9d) (BuildId: 57941f7730deefd0a2028f1d9f3e173472a4aa76)
        1 0x55b559304b83 in std::__1::weak_ptr<DB::Context const>::~weak_ptr() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1397:26
        2 0x55b559304b83 in std::__1::weak_ptr<DB::Context const>::~weak_ptr() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1553:1
        3 0x55b559304b83 in DB::WithContextImpl<std::__1::shared_ptr<DB::Context const>>::~WithContextImpl() build_docker/./src/Interpreters/Context_fwd.h:30:8
        4 0x55b559304b83 in DB::ServerAsynchronousMetrics::~ServerAsynchronousMetrics() build_docker/./src/Interpreters/ServerAsynchronousMetrics.h:10:7
        5 0x55b559304b83 in DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) build_docker/./programs/server/Server.cpp:1995:1
        6 0x55b5789f579e in Poco::Util::Application::run() build_docker/./base/poco/Util/src/Application.cpp:315:8
        7 0x55b5592d762a in DB::Server::run() build_docker/./programs/server/Server.cpp:397:25
        8 0x55b578a3efdf in Poco::Util::ServerApplication::run(int, char**) build_docker/./base/poco/Util/src/ServerApplication.cpp:131:9
        9 0x55b5592d0489 in mainEntryClickHouseServer(int, char**) build_docker/./programs/server/Server.cpp:203:20
        10 0x55b541acd4ab in main build_docker/./programs/main.cpp:505:12
        11 0x7f7ff382ed8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: a43bfc8428df6623cd498c9c0caeb91aec9be4f9)

SUMMARY: MemorySanitizer: use-of-uninitialized-value build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:1645:20 in std::__1::weak_ptr<DB::Context const>::lock() const

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/52717/fcdead023c4350233ef1e0f7f82a71653ed62229/stress_test__tsan_.html
  [2]: https://s3.amazonaws.com/clickhouse-test-reports/52717/fcdead023c4350233ef1e0f7f82a71653ed62229/stress_test__msan_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-10 16:03:52 +01:00
Maksim Kita
cdf6da88fe Fixed code review issues 2023-12-10 16:10:11 +03:00
Maksim Kita
0eb7a41bab CHJIT add assembly printer 2023-12-10 16:10:11 +03:00
Vitaly Baranov
a058a26b7e
Merge pull request #57454 from vitlibar/fix-rwlock 
Fix RWLock inconsistency after write lock timeout
 2023-12-10 14:09:29 +01:00
Azat Khuzhin
ece8c86ebc Fix attaching table in Atomic engine after attachTableUnlocked() interface 2023-12-10 08:28:10 +01:00
Bharat Nallan Chakravarthy
bfe2ffbce9 use the EVP interface to satisfy s390x builds 2023-12-09 21:31:21 -08:00
Vitaly Baranov
ab2f60ccef sleep() function now can be cancelled with KILL QUERY. 2023-12-10 03:19:25 +01:00
Alexey Milovidov
100b36e8a9 Do not check in replication 2023-12-10 01:32:42 +01:00
Alexey Milovidov
33b0ec7d55 Merge branch 'master' into CurtizJ-fix-ttl-non-deterministic 2023-12-10 01:21:43 +01:00
Nikolay Degterinsky
fc8656ee9e Fix unary operators parsing 2023-12-09 22:44:06 +00:00
Azat Khuzhin
4f9d90761b Attach tables lazily only for clickhouse-local 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 23:08:29 +01:00
Azat Khuzhin
c14721bb03 Move lazy table initialization interface into IDatabase 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 23:08:29 +01:00
Azat Khuzhin
f75484a0b1 Make IDatabase::attachTableUnlocked() virtual and use it for overrides (mostly) 2023-12-09 23:08:29 +01:00
Azat Khuzhin
cc9afe6f6e Replace --no-system-tables with loading virtual tables of system database lazily 
First of all the problem is that that simple 'SELECT 1' cannot be runned
without system.one, which makes --no-system-tables almost useless:

    $ ./clickhouse-debug local --no-system-tables -q "select 1"
    Code: 81. DB::Exception: Database system does not exist. (UNKNOWN_DATABASE)

Secondly there are just too much flags, and this one
(--no-system-tables) is too damn specific.

This patch should improve startup time of the clickhouse-local almost
3x in debug builds.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 23:08:29 +01:00
Azat Khuzhin
6ccbc2ea75 Move io_uring reader into the Context from static to make it's thread joinable 
v2: fix for standalone keeper build
CI: https://s3.amazonaws.com/clickhouse-test-reports/52717/72b1052f7c2d453308262924e767ab8dc2206933/stateless_tests__debug__[4_5].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 22:50:48 +01:00
Azat Khuzhin
7af1c0885d Do not call add*() from addBatchSparseSinglePlace() if there are no rows 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 22:47:25 +01:00
Azat Khuzhin
472a31e581 Fix SIGSEGV for aggregation of sparse columns with any() RESPECT NULL 
Add proper check into any() RESPECT NULL

Fixes: https://github.com/ClickHouse/ClickHouse/issues/57709
Fixes: https://github.com/ClickHouse/ClickHouse/pull/57189 (cc @Algunenano)
CI: https://s3.amazonaws.com/clickhouse-test-reports/52717/7ec9c6a7f7490b5075407410dc193a2534216e15/fuzzer_astfuzzerdebug/report.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 22:47:15 +01:00
Igor Nikonov
87e47bdbee
Merge pull request #57467 from ClickHouse/rm-select 
Remove select() usage
 2023-12-09 22:27:28 +01:00
Vitaly Baranov
179a0a2e8f Fixed condition and add more chasserts for consistency checks. 2023-12-09 20:22:48 +01:00
Yakov Olkhovskiy
63619b5935 allow IPv6 to UInt128 conversion and binary arithmetic 2023-12-09 17:57:32 +00:00
Azat Khuzhin
55d5a3affa Correctly wait background threads 
There are some places that make it possible to trigger use-after-free
from threads because some global variable had been destroyed before, for
example some logger.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 17:54:16 +01:00
Alexey Milovidov
c77183a597
Merge pull request #57480 from azat/dist/async-INSERT-fixes 
Fix possible distributed sends stuck due to "No such file or directory" (during recovering batch from disk)
 2023-12-09 17:11:35 +01:00
Alexey Milovidov
56cd9196b0
Merge pull request #57641 from azat/system.stack_trace-fix 
Avoid hangs of system.stack_trace by correctly prohibit parallel read from it
 2023-12-09 17:07:48 +01:00
Bharat Nallan Chakravarthy
64ae040954 fix test 2023-12-08 20:47:31 -08:00
Nikolai Kochetov
5e1dccf1f4 Fixing tests 2023-12-08 22:36:01 +00:00
Nikolay Degterinsky
c5e52d260f
Merge pull request #57553 from evillique/fix-local-crash 
Fix crash in clickhouse-local
 2023-12-08 22:57:52 +01:00
Nikolai Kochetov
a76d9c259f Fixing style 2023-12-08 21:06:18 +00:00
Alexey Milovidov
2a77c3a2ff
Merge pull request #57633 from ClickHouse/fix-logs-and-exceptions-messages-formatting-3 
Fix "logs and exception messages formatting", part 3
 2023-12-08 22:00:29 +01:00
Alexey Milovidov
407761d67c
Merge pull request #57630 from ClickHouse/fix-logs-and-exceptions-messages-formatting-1 
Fix "logs and exception messages formatting", part 1
 2023-12-08 21:58:38 +01:00
Han Fei
881e58dfcb support soft limit ratio 2023-12-08 21:48:27 +01:00
Gagan Goel
e547db0a8c Issue 43666: Add skip_unavailable_shards as a setting for Distributed table. 
This setting, when enabled (disabled by default), allows ClickHouse to
silently skip unavailable shards of a Distributed table during a query
execution, instead of throwing an exception to the client.
 2023-12-08 15:43:59 -05:00
avogar
ee7af95bc0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-union 2023-12-08 20:29:28 +00:00
Alexey Milovidov
56dc2b06ca Merge branch 'master' into CurtizJ-fix-ttl-non-deterministic 2023-12-08 20:54:24 +01:00
Nikolai Kochetov
ff0340b0fc Release memory for aggregation earlier. 2023-12-08 18:18:45 +00:00
kssenii
02d19fa7ef Make querying system.filesystem_cache not memory intensive 2023-12-08 17:09:09 +01:00
vdimir
8ebd5a2d6c
Fix type correction in HashJoin for nested low cardinality 2023-12-08 15:50:33 +00:00
vdimir
f3fada0873
Analyzer: Fix JOIN ON true with join_use_nulls 2023-12-08 15:49:16 +00:00
Alexey Milovidov
ba5ff075bb
Merge pull request #57624 from yariks5s/date_trunc_case_insensitive_arguments 
Case-insensitive arguments for `date_trunc()`
 2023-12-08 16:37:49 +01:00
Bhavna Jindal
1f58b0f150
Merge branch 'master' into dev-seriesPeriodDetect 2023-12-08 10:26:39 -05:00
Azat Khuzhin
b1043b7ad3 Increase log verbosity of some messages in system.stack_trace 
This can help to debug 01051_system_stack_trace test failures.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-08 16:18:49 +01:00
alesapin
dc7347e0ba
Merge pull request #57271 from hanfei1991/hanfei/keeper_soft_limit 
support memory soft limit for keeper
 2023-12-08 16:12:47 +01:00
Bhavna Jindal
94943ec83d fix for non-constant arguments 2023-12-08 07:10:47 -08:00
Kseniia Sumarokova
928993f2bd
Merge branch 'master' into fix-materialized-pg-issue-with-generated-columns 2023-12-08 15:37:29 +01:00
Kseniia Sumarokova
bd3665863c
Merge branch 'master' into allow-to-change-some-cache-settings-without-restart 2023-12-08 15:36:40 +01:00
kssenii
ea63819c76 Better 2023-12-08 15:21:42 +01:00
zhongyuankai
7b0f8d44e8
Make DirectoryMonitor handle cluster node list change (#42826) 2023-12-08 14:41:51 +01:00
MikhailBurdukov
d10217af16 style 2023-12-08 13:14:47 +00:00
Vitaly Baranov
252de64af3
Merge pull request #57362 from aiven-sal/aiven-sal/dict_fix 
Prevent dictionary's ACL bypass via dictionary() table function
 2023-12-08 14:09:40 +01:00
kssenii
d77938c3ff Add validation of columns 2023-12-08 12:42:38 +01:00
kssenii
ffd0d16f79 Merge remote-tracking branch 'origin/master' into fix-materialized-pg-issue-with-generated-columns 2023-12-08 12:42:21 +01:00
alesapin
11b90f8dd4
Merge pull request #57626 from ClickHouse/better_exception 
Throw more clear exception
 2023-12-08 12:03:14 +01:00
Azat Khuzhin
16cbcf700e Prohibit parallel read from system.stack_trace 
Before rewriting system.stack_trace to handle max_block_size (in #54946)
parallel reading from system.stack_trace was prohibited, because this
could lead to hang of system.stack_trace table.

But that rewrite broke this guarantee, so let's fix it to avoid possible
hung.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-08 11:05:25 +01:00
Kseniia Sumarokova
1ec2d29302
Merge pull request #57567 from ClickHouse/kssenii-patch-8 
Fix possible segfault in PostgreSQLSource
 2023-12-08 10:51:08 +01:00
vdimir
6df715f5d7
Merge pull request #57627 from ClickHouse/prometheus-all-errors 
Expose the total number of errors to Prometheus endpoint
 2023-12-08 10:42:01 +01:00
vdimir
86bdd883f4
Merge pull request #57622 from ClickHouse/vdimir/analyzer_multiif_logical_err 
Analyzer: Fix logical error in MultiIfToIfPass
 2023-12-08 10:28:55 +01:00
Bharat Nallan Chakravarthy
e01e6671b3 Merge remote-tracking branch 'upstream/master' into ncb/sha512-256 2023-12-07 23:53:18 -08:00
Alexey Milovidov
64e64797ae Update version 2023-12-08 03:15:01 +01:00
Alexey Milovidov
ce9e134043 Merge branch 'master' into CurtizJ-fix-ttl-non-deterministic 2023-12-08 03:10:09 +01:00
Alexey Milovidov
f5f28208cb
Merge pull request #57618 from myrrc/fix/psql-sigabrt 
Fix sigabrt while getting PostgreSQL table structure
 2023-12-07 23:42:02 +01:00
Vitaly Baranov
b4cac92e28 Add some comments. 2023-12-07 22:18:49 +01:00
Alexey Milovidov
0ee13fde16 Fix "logs and exception messages formatting", part 3 2023-12-07 22:01:53 +01:00
Alexey Milovidov
5de99eceec Fix "logs and exception messages formatting", part 1 2023-12-07 21:38:59 +01:00
Alexey Milovidov
4d41c9fa0a
Merge pull request #57587 from azat/client-spawn-editor-fix 
Correctly handle errors during opening query in editor in client
 2023-12-07 21:22:46 +01:00
Alexey Milovidov
429ed34607
Merge pull request #57479 from mkmkme/mkmkme/zookeeper-read-only 
Add support for read-only mode in ZooKeeper
 2023-12-07 20:22:00 +01:00
kssenii
ff65d0e727 Adjustments after merge with master 2023-12-07 19:49:30 +01:00
kssenii
725571461d Merge remote-tracking branch 'origin/master' into slru-for-filesystem-cache 2023-12-07 19:49:03 +01:00
kssenii
7205e4edbe Review fix 2023-12-07 18:47:14 +01:00
Azat Khuzhin
2c6604ec24 Allow nodes in config with from_env/from_zk and non empty element with replace=1 
Such nodes in config are useful as a nodes with default values, that can
be overwritten from ZooKeeper/env.

So after this patch the following is valid, and is interpreted as
default value 86400, and can be overwritten via env:

```xml
<asynchronous_metrics_update_period_s replace="1" from_env="CH_ASYNCHRONOUS_METRICS_UPDATE_PERIOD_S">86400</asynchronous_metrics_update_period_s>
```

While the following is not:

```xml
<asynchronous_metrics_update_period_s             from_env="CH_ASYNCHRONOUS_METRICS_UPDATE_PERIOD_S">86400</asynchronous_metrics_update_period_s>
```

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-07 18:01:40 +01:00
Nikita Mikhaylov
08cc9408f2 Better 2023-12-07 17:53:40 +01:00
Bharat Nallan Chakravarthy
27bcd14181 add SHA512_256 function 2023-12-07 08:43:00 -08:00
alesapin
46f04d6d0e Throw more clear exception 2023-12-07 17:15:39 +01:00
vdimir
04933ddcc6
Analyer: Fix logical error in MultiIfToIfPass 2023-12-07 16:11:15 +00:00
Yarik Briukhovetskyi
71277892cf init 2023-12-07 15:55:23 +00:00
Alexey Milovidov
11920e751a
Merge pull request #57592 from bharatnc/ncb/alias-format-readable-size 
Add FORMAT_BYTES as an alias for formatReadableSize
 2023-12-07 15:15:56 +01:00
vdimir
f058fe1d20
Merge pull request #57564 from ClickHouse/vdimir/join_on_false_const_fix 
Materialize block in HashJoin for Type::EMPTY
 2023-12-07 15:09:05 +01:00
Mike Kot
4ef9065567 fix conversion error while getting psql table structure for empty array 2023-12-07 13:38:32 +00:00
kssenii
c785ebe921 Fix style check 2023-12-07 12:47:51 +01:00
kssenii
188d4d66c9 Merge with master 2023-12-07 12:45:36 +01:00
FFish
add243593d
fix: split lttb bucket strategy, first bucket and last bucket should only contain single point (#57003) 
* fix: split lttb bucket policy, first bucket and last bucket should only
contain single point

* add comments and modify the corresponding query test

* style: update code format

* style: remove useless comments

* feat: add lttb bucket size test

* fix: typo, duplicate sql
 2023-12-07 12:29:37 +01:00
kssenii
cc483c19a8 Fix merge with master 2023-12-07 12:14:49 +01:00
kssenii
8be3c9d218 Merge remote-tracking branch 'origin/master' into allow-to-change-some-cache-settings-without-restart 2023-12-07 12:14:24 +01:00
kssenii
614da21144 Better 2023-12-07 12:12:10 +01:00
Kseniia Sumarokova
b3f22449f6
Merge pull request #57515 from ClickHouse/fix-materialized-pg-issue-with-incorrect-connection-options 
MaterializedPostgreSQL: fix issue #41922, add test for #41923
 2023-12-07 10:42:14 +01:00
Robert Schulze
554c5d690c
Merge remote-tracking branch 'rschu1ze/master' into format-arbitrary-types 2023-12-07 09:36:39 +00:00
Robert Schulze
0159a4d4a2
Merge pull request #57512 from rschu1ze/revert-revert-sqid 
Revert "Revert "Add `sqid()` function""
 2023-12-07 10:26:02 +01:00
凌涛
d0675488ac rebase 2023-12-07 11:14:55 +08:00
Ryan Jacobs
6a698d9480
clang-tidy 2023-12-06 18:56:54 -08:00
Alexey Milovidov
b85214ca1a
Merge pull request #57546 from rschu1ze/revert-revert-support_alias_column_in_indices 
Revert "Revert "Support use alias column in indices""
 2023-12-07 01:22:03 +01:00
Alexey Milovidov
5fe1b8926c
Merge pull request #57595 from ClickHouse/23.12-prepare 
Update version after release
 2023-12-06 23:53:53 +01:00
Alexey Milovidov
c8fb574fc8 Update autogenerated version to 23.12.1.1 and contributors 2023-12-06 23:53:06 +01:00
Bharat Nallan Chakravarthy
194d8cdf35 add FORMAT_BYTES as alias for formatReadableSize 2023-12-06 13:59:48 -08:00
Yakov Olkhovskiy
ac7210b9b3
Merge pull request #57461 from ClickHouse/fix-ephemeral-matview 
Fix: don't exclude ephemeral column when building pushing to view chain
 2023-12-06 16:46:54 -05:00
Bhavna Jindal
4974ae019d Merge branch 'revert-the-revert-seriesPeriodDetectFFt' into dev-seriesPeriodDetect 2023-12-06 13:32:50 -08:00
Bhavna Jindal
dad268c33b Revert "Revert "Implemented series period detect method using pocketfft lib"" 
This reverts commit d7d83c99e5.
 2023-12-06 13:31:49 -08:00
Alexey Milovidov
05bc8ef1e0
Merge pull request #57548 from ClickHouse/s3queue_production_ready 
`S3Queue` is production ready
 2023-12-06 22:21:08 +01:00
Ryan Jacobs
b57340bde2
postgresql integration: Throw errors instead of assuming array_ndim == 1 2023-12-06 11:54:45 -08:00
Azat Khuzhin
477b782f86 Handle editor exit code correctly 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-06 20:21:15 +01:00
Azat Khuzhin
028c01c458 Do not abort the client if temporary file for query editor cannot be created 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-06 20:21:15 +01:00
kssenii
f44f7c8c28 Allow to change some cache settings without server restart 2023-12-06 19:29:18 +01:00
Alexey Milovidov
3723573f81 Revert "Merge pull request #56724 from canhld94/ch_replicated_column_mismatch" 
This reverts commit a9bb76378f, reversing
changes made to 588fd16518.
 2023-12-06 19:09:07 +01:00
Bhavna Jindal
82d492d85a override executeImplDryRun 2023-12-06 08:38:20 -08:00
Kseniia Sumarokova
9d8e3f8bd9
Update fetchPostgreSQLTableStructure.cpp 2023-12-06 16:47:58 +01:00
skyoct
2762529776 fix: fix format error 2023-12-06 14:04:44 +00:00
kssenii
806061642a Support GENERATED and DEFAULT columns 2023-12-06 15:01:43 +01:00
vdimir
8f32293104
Merge pull request #57416 from ClickHouse/vdimir/fix_test_encrypted_disk 
Change type of s3_cache in test_encrypted_disk
 2023-12-06 14:37:40 +01:00
Kseniia Sumarokova
29c770502a
Fix possible segfault in PostgreSQLSource 2023-12-06 14:34:06 +01:00
Yakov Olkhovskiy
8f00678abe fix style 2023-12-06 13:24:25 +00:00
vdimir
b0239dbc87
Merge pull request #57464 from azat/client-log-comment-fix 
Fix adjusting log_comment in case of multiple files passed
 2023-12-06 12:39:58 +01:00
Alexey Milovidov
f8de7f6bc2 Merge branch 'master' into s3queue_production_ready 2023-12-06 12:25:29 +01:00
vdimir
c42fade8a7
Analyzer allow join on const for semi and anti 2023-12-06 11:19:14 +00:00
vdimir
f29386de0b
Materialize block in HashJoin for Type::EMPTY 2023-12-06 11:17:29 +00:00
Kseniia Sumarokova
e02883cd4a
Merge branch 'master' into fix-materialized-pg-issue-with-incorrect-connection-options 2023-12-06 12:16:04 +01:00
Igor Nikonov
b4966b980e Merge remote-tracking branch 'origin/master' into rm-select 2023-12-06 10:47:46 +00:00
alesapin
e8b017d323
Merge pull request #57513 from ClickHouse/enable_async_block_ids_cache_by_default 
Enable async block ids cache by default
 2023-12-06 11:35:25 +01:00
Nikolai Kochetov
718b542f40
Merge pull request #57518 from ClickHouse/cleanup-around-distributed 
Simple cleanup in distributed (while dealing with parallel replicas)
 2023-12-06 10:22:17 +01:00
Robert Schulze
cfdfde6879
Fix style 2023-12-06 08:34:32 +00:00
Yakov Olkhovskiy
f5fb169c65 add system.dropped_tables_parts table 2023-12-06 06:00:00 +00:00
Nikolay Degterinsky
ed618d3be1 Fix crash in clickhouse-local 2023-12-06 04:24:43 +00:00
Robert Schulze
c0d56c7bee
Implement format() with arbitrary arguments 2023-12-05 22:51:30 +00:00
Robert Schulze
3305f46676
Less silly #includes 2023-12-05 20:14:11 +00:00
Robert Schulze
4acfbe948d
Iterate less awkwardly 2023-12-05 20:10:49 +00:00
Alexey Milovidov
76bbeaa676 S3Queue is production ready 2023-12-05 21:10:19 +01:00
Robert Schulze
7fed06158b
Remove unnecessary template + whitespace 2023-12-05 20:09:17 +00:00
Robert Schulze
a7f0488059
Rename Functions/formatString.cpp --> Functions/format.cpp 2023-12-05 20:07:26 +00:00
jsc0218
cdd5280272
Merge pull request #57387 from evillique/better-disks 
Initialize only required disks in clickhouse-disks
 2023-12-05 13:59:04 -05:00
slvrtrn
e275080f52 Merge remote-tracking branch 'origin' into substr-with-enums 2023-12-05 19:48:10 +01:00
Robert Schulze
f56dd6561f
Merge remote-tracking branch 'rschu1ze/master' into revert-revert-sqid 2023-12-05 18:41:10 +00:00
Robert Schulze
8e4e37d8ed
Revert "Revert "Support use alias column in indices"" 
This reverts commit d0b591be2a.
 2023-12-05 18:22:32 +00:00
Igor Nikonov
f123dd0070 Fix addShard() 2023-12-05 18:07:51 +00:00
Nikolai Kochetov
681cfaa38a
Merge pull request #56764 from ClickHouse/avoid-stuck-for-indefinite-sourcers 
Try to fix hang in 01104_distributed_numbers_test
 2023-12-05 16:34:02 +01:00
Alexey Milovidov
d0b591be2a
Revert "Support use alias column in indices" 2023-12-05 18:21:45 +03:00
Alexey Milovidov
d7d83c99e5
Revert "Implemented series period detect method using pocketfft lib" 2023-12-05 18:17:07 +03:00
MikhailBurdukov
c41511e51c ignore_on_cluster_for_grants 2023-12-05 15:11:32 +00:00
Nikolay Degterinsky
a776a5cac5
Merge pull request #57483 from joelynch/stop-listen-tcp-secure 
bugfix: correctly parse SYSTEM STOP LISTEN TCP SECURE
 2023-12-05 15:25:34 +01:00
Nikolay Degterinsky
7f59eb438d
Merge pull request #57379 from evillique/fix-jsonmergepatch 
Fix function jsonMergePatch for partially const columns
 2023-12-05 15:14:30 +01:00
Igor Nikonov
d196973ab3 Merge remote-tracking branch 'origin/master' into rm-select 2023-12-05 12:59:18 +00:00
Igor Nikonov
60ba1abc17 Fix build 2023-12-05 12:58:29 +00:00
Igor Nikonov
b2a269e8ce Formatting 2023-12-05 12:31:24 +00:00
Alexey Milovidov
2988f6f92a
Revert "Add new aggregation function groupArraySorted()" 2023-12-05 15:31:17 +03:00
Robert Schulze
f1a2b46d60
Incorporate feedback 2023-12-05 12:28:28 +00:00
Igor Nikonov
87a4c0bde4 Simple cleanup in distributed (while dealing with parallel replicas) 2023-12-05 12:25:02 +00:00
kssenii
0d313acd03 Fix 2023-12-05 13:22:35 +01:00
Alexander Sapin
93a16b7cd6 Fix comments 2023-12-05 12:39:00 +01:00
Alexander Sapin
2587f368a6 Enable async block ids cache by default 2023-12-05 12:37:22 +01:00
Anton Popov
5770def9af
Merge pull request #57491 from jkartseva/limit-table-projections 
limit merge tree projections
 2023-12-05 12:13:41 +01:00
Robert Schulze
a2cd240317
Revert "Revert "Add sqid() function"" 
This reverts commit 8d5d0ef38f.
 2023-12-05 10:50:30 +00:00
Raúl Marín
cf22ff1673 Fix report by tidy 2023-12-05 10:47:21 +00:00
Alexey Milovidov
8d5d0ef38f
Revert "Add sqid() function" 2023-12-05 13:45:33 +03:00
Alexey Milovidov
cbea3545aa
Revert "add function getClientHTTPHeader" 2023-12-05 13:34:34 +03:00
Julia Kartseva
2ac104240d limit merge tree projections 
Fixes: https://github.com/ClickHouse/ClickHouse/issues/56427
 2023-12-05 06:44:39 +00:00
Alexey Milovidov
1f752b79db
Merge pull request #55894 from bharatnc/ncb/hostname-system-log-tables 
add hostname column to system log tables
 2023-12-05 06:42:43 +01:00
Alexey Milovidov
c729d06e27
Merge pull request #56078 from hanfei1991/hanfei/sqllogic 
try to work sqllogic-test out
 2023-12-05 06:37:29 +01:00
Alexey Milovidov
10d65a1ade
Merge pull request #55559 from azat/s3-fix-excessive-reads 
Add ability to disable checksums for S3 to avoid excessive input file read
 2023-12-05 06:34:21 +01:00
Alexey Milovidov
862171e3a2
Update Settings.h 2023-12-05 08:33:43 +03:00
Alexey Milovidov
e1628731fd
Merge pull request #57489 from ClickHouse/nickitat-patch-15 
Decrease default value for `filesystem_prefetch_min_bytes_for_single_read_task`
 2023-12-05 06:14:24 +01:00
凌涛
9e05e79d66
Merge branch 'master' into feature/more_warnings 2023-12-05 11:50:05 +08:00
凌涛
6794bbe196 rebase master 2023-12-05 11:42:25 +08:00
凌涛
5a51fdac95 modify 2023-12-05 11:33:40 +08:00
Han Fei
70f436483c address part of comments 2023-12-05 00:40:15 +01:00
Nikita Taranov
c4df0d0428
Decrease default value for filesystem_prefetch_min_bytes_for_single_read_task 
Follow up #55841
 2023-12-04 23:09:32 +01:00
joelynch
f346667956
Simpler fix 2023-12-04 20:13:53 +01:00
joelynch
b312a9f4ee
bugfix: parse system stop listen tcp secure 2023-12-04 19:56:35 +01:00
kssenii
4a28f10c3d Minor cache changes 2023-12-04 19:02:37 +01:00
Nikolay Degterinsky
265e2b30a8
Even better 2023-12-04 18:42:25 +01:00
Nikolay Degterinsky
947106972d
Better type check 2023-12-04 18:27:10 +01:00
Alexey Milovidov
02439eee58
Merge pull request #49351 from ClickHouse/async-loader-integration 
Asynchronous loading of tables
 2023-12-04 17:16:12 +01:00
Azat Khuzhin
7986fe619a Introduce DistributedAsyncInsertionFailures - event for async INSERT failures 
Useful for alerts

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-04 16:52:53 +01:00
Azat Khuzhin
604cec475a Fix possible distributed sends stuck due to "No such file or directory" 
In case of restoring from current_batch.txt it is possible that the some
file from the batch will not be exist, and the fix submitted in #49884
was not complete, since it will fail later in markAsSend() (due to it
tries to obtain file size there):

    2023.12.04 05:43:12.676658 [ 5006 ] {} <Error> dist.DirectoryMonitor.work4: std::exception. Code: 1001, type: std::__1::__fs::filesystem::filesystem_error, e.what() = filesystem error: in file_size: No such file or directory ["/work4/clickhouse/data/dist/shard8_all_replicas//150426396.bin"], Stack trace (when copying this message, always include the lines below):

    0. ./.build/./contrib/llvm-project/libcxx/include/exception:134: std::runtime_error::runtime_error(String const&) @ 0x00000000177e83f4 in /usr/lib/debug/usr/bin/clickhouse.debug
    1. ./.build/./contrib/llvm-project/libcxx/include/string:1499: std::system_error::system_error(std::error_code, String const&) @ 0x00000000177f0fd5 in /usr/lib/debug/usr/bin/clickhouse.debug
    2. ./.build/./contrib/llvm-project/libcxx/include/__filesystem/filesystem_error.h:42: std::__fs::filesystem::filesystem_error::filesystem_error[abi:v15000](String const&, std::__fs::filesystem::path const&, std::error_code) @ 0x000000000b844ca1 in /usr/lib/debug/usr/bin/clickhouse.debug
    3. ./.build/./contrib/llvm-project/libcxx/include/__filesystem/filesystem_error.h:90: void std::__fs::filesystem::__throw_filesystem_error[abi:v15000]<String&, std::__fs::filesystem::path const&, std::error_code const&>(String&, std::__fs::filesystem::path const&, std::error_code const&) @ 0x000000001778f953 in /usr/lib/debug/usr/bin/clickhouse.debug
    4. ./.build/./contrib/llvm-project/libcxx/src/filesystem/filesystem_common.h:0: std::__fs::filesystem::detail::(anonymous namespace)::ErrorHandler<unsigned long>::report(std::error_code const&) const @ 0x0000000017793ef7 in /usr/lib/debug/usr/bin/clickhouse.debug
    5. ./.build/./contrib/llvm-project/libcxx/src/filesystem/operations.cpp:0: std::__fs::filesystem::__file_size(std::__fs::filesystem::path const&, std::error_code*) @ 0x0000000017793e26 in /usr/lib/debug/usr/bin/clickhouse.debug
    6. ./.build/./src/Storages/Distributed/DistributedAsyncInsertDirectoryQueue.cpp:707: DB::DistributedAsyncInsertDirectoryQueue::markAsSend(String const&) @ 0x0000000011cd92c5 in /usr/lib/debug/usr/bin/clickhouse.debug
    7. ./.build/./contrib/llvm-project/libcxx/include/__iterator/wrap_iter.h💯 DB::DistributedAsyncInsertBatch::send() @ 0x0000000011cdd81c in /usr/lib/debug/usr/bin/clickhouse.debug
    8. ./.build/./src/Storages/Distributed/DistributedAsyncInsertDirectoryQueue.cpp:0: DB::DistributedAsyncInsertDirectoryQueue::processFilesWithBatching() @ 0x0000000011cd5054 in /usr/lib/debug/usr/bin/clickhouse.debug
    9. ./.build/./src/Storages/Distributed/DistributedAsyncInsertDirectoryQueue.cpp:417: DB::DistributedAsyncInsertDirectoryQueue::processFiles() @ 0x0000000011cd3440 in /usr/lib/debug/usr/bin/clickhouse.debug
    10. ./.build/./src/Storages/Distributed/DistributedAsyncInsertDirectoryQueue.cpp:0: DB::DistributedAsyncInsertDirectoryQueue::run() @ 0x0000000011cd3878 in /usr/lib/debug/usr/bin/clickhouse.debug
    11. ./.build/./contrib/llvm-project/libcxx/include/__functional/function.h:0: DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x00000000103dbc34 in /usr/lib/debug/usr/bin/clickhouse.debug
    12. ./.build/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::BackgroundSchedulePool::threadFunction() @ 0x00000000103de1b6 in /usr/lib/debug/usr/bin/clickhouse.debug
    13. ./.build/./src/Core/BackgroundSchedulePool.cpp:0: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000103de7d1 in /usr/lib/debug/usr/bin/clickhouse.debug
    14. ./.build/./base/base/../base/wide_integer_impl.h:809: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000b8c5502 in /usr/lib/debug/usr/bin/clickhouse.debug
    15. ./.build/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000b8c936e in /usr/lib/debug/usr/bin/clickhouse.debug
    16. ? @ 0x00007f1be8b30fd4 in ?
    17. ? @ 0x00007f1be8bb15bc in ?

And instead of ignoring errors, DistributedAsyncInsertBatch::valid() had
been added, that should be called when the files had been read from the
current_batch.txt, if it is not valid (some files from the batch did not
exist), then there is no sense in trying to send the same batch, so just
this file will be ignored, and files will be processed in a regular
order.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-04 16:52:53 +01:00
Azat Khuzhin
638d0102f8 Fix error_count in case of distributed_directory_monitor_max_sleep_time_ms>5min 
In this case the error counter will be decremented everytime.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-04 16:52:53 +01:00
Robert Schulze
8db3bac7bd
Merge pull request #57442 from awakeljw/dev 
Add `sqid()` function
 2023-12-04 15:50:28 +01:00
Mikhail Koviazin
bef3fff731
Add support for read-only mode in ZooKeeper 
This commit enables the read-only flag when connecting to the ZooKeeper server.

This flag is enabled by sending one extra byte when connecting,
and then receiving one extra byte during the first response.

In addition to that, we modify createIfNotExists to not complain
about attempting to alter a read-only ZooKeeper cluster if the node
already exists.

This makes ClickHouse more useful in the event of a loss of quorum,
user credentials are still accessible, which makes it possible to
connect to the cluster and run read queries.

Any DDL or DML query on a Distributed database or ReplicatedMergeTree
table will correctly fail, since it needs to write to ZooKeeper to
execute the query.

Any non-distributed query will be possible, which is ok since the
query was never replicated in the first place, there is no loss of
consistency.

Fixes #53749 as it seems to be the only thing 3.9 enforced.
 2023-12-04 16:47:12 +02:00
vdimir
7e624ae911
Review fixes 2023-12-04 13:56:51 +00:00
vdimir
a4ae90de0d
Merge pull request #57275 from ClickHouse/vdimir/merge_task_tmp_data 
Background merges correctly use temporary data storage in the cache
 2023-12-04 14:52:20 +01:00
Kseniia Sumarokova
715ec0d334
Merge pull request #56383 from ClickHouse/fix-backup-restore-with-mv 
Fix restore from backup with mat view and dropped source table
 2023-12-04 13:25:13 +01:00
Raúl Marín
be367ec793 Merge remote-tracking branch 'blessed/master' into i48496 2023-12-04 12:41:53 +01:00
Igor Nikonov
434c2113d2 Fix style 2023-12-04 10:51:17 +00:00
vdimir
b2e29d1416
Change type of s3_cache in test_encrypted_disk 2023-12-04 10:34:13 +00:00
Kseniia Sumarokova
4d3fde3624
Merge pull request #57424 from ClickHouse/fix-cache-background-download-queue-limit 
fs cache: add limit for background download
 2023-12-04 11:03:20 +01:00
Igor Nikonov
73ff984755 Remove select() usage 2023-12-04 09:33:34 +00:00
robot-ch-test-poll1
d63b652dae
Merge pull request #57385 from vitlibar/fix-inconsistent-metadata-for-backup-2 
Stop using INCONSISTENT_METADATA_FOR_BACKUP that much
 2023-12-04 10:29:57 +01:00
Azat Khuzhin
699d11628a Fix adjusting log_comment in case of multiple files passed 
Previously, due to adjusting global_context and checking it as well, in
case of multiple files the log_comment will be adjusted only the first
time. Fix this by caching does user passed the log_comment explicitly or
not.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-04 10:02:24 +01:00
Yakov Olkhovskiy
b601b66146 don't exclude ephemeral column when pushing to view 2023-12-04 06:56:33 +00:00
awakeljw
6c70e32da1 rename squids to sqids 2023-12-04 11:26:55 +08:00
Bharat Nallan Chakravarthy
440dc66a5c Merge remote-tracking branch 'upstream/master' into ncb/hostname-system-log-tables 2023-12-03 15:19:47 -08:00
Vitaly Baranov
0457aba4a9 Show owner query ids in the message for the DEADLOCK_AVOIDED error. 2023-12-04 00:15:31 +01:00
Vitaly Baranov
44d6605b8a Fix RWLock inconsistency after write lock timeout. 2023-12-04 00:15:31 +01:00
Vitaly Baranov
fefbf397c0 Add more tests for RWLock. 2023-12-04 00:15:27 +01:00
Robert Schulze
f6036fbb29
Some fixups 2023-12-03 16:20:59 +00:00
KevinyhZou
adbc819a09
Fix JSON_QUERY function with multiple numeric paths (#57096) 
* Fix json_query parse error while path has numuber

* Fix json_query with numberic path

* remove unuse code

* remove code

* review

* resume some code
 2023-12-03 15:53:50 +01:00
awakeljw
06241de257 fix codes according to the reviews 2023-12-03 22:00:42 +08:00
alesapin
acc53431c7
Update src/Coordination/KeeperDispatcher.cpp 2023-12-03 12:43:20 +01:00
alesapin
9a68461cdb
Update src/Coordination/KeeperDispatcher.cpp 2023-12-03 12:43:16 +01:00
Nikolay Degterinsky
2150308c23
Merge pull request #56948 from amosbird/fix-56932 
Prevent incompatible ALTER of projection columns
 2023-12-03 04:24:51 +01:00
Nikolay Degterinsky
16f6a6de17 Better fix 2023-12-02 23:48:12 +00:00
awakeljw
1a0dc1375c sqids only support unsigned integers 2023-12-03 05:12:12 +08:00
Sema Checherinda
c61cd6aab2
Merge pull request #57335 from sichenzhao/increase_default_replicated_deduplication_window 
increase default replicated_deduplication_window from 100 to 1k
 2023-12-02 21:23:39 +01:00
Nikolay Degterinsky
a53e604429
Merge branch 'master' into fix-56932 2023-12-02 21:19:45 +01:00
Sema Checherinda
32e3b2657b
Merge pull request #57205 from CheSema/zero-locks-leak 
fix zero-copy locks leaking
 2023-12-02 15:54:12 +01:00
Ryan Jacobs
17aacda9e5
fix: format code for stylecheck 2023-12-02 06:46:16 -08:00
awakeljw
e5a72f15dd support sqids 2023-12-02 21:12:48 +08:00