thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
62,510 Commits 9,319 Branches 1,783 Tags 2.1 GiB
01b1440bd4
Commit Graph

12685 Commits

Author SHA1 Message Date
Azat Khuzhin
3056d2c5d1 Fix data-race during server shutdown the context 
Found with 01737_clickhouse_server_wait_server_pool:

==================
WARNING: ThreadSanitizer: data race (pid=13248)
  Write of size 1 at 0x7b9000003a38 by main thread:
    0 std::__1::__optional_destruct_base<DB::SystemLogs, false>::reset() obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:246:24 (clickhouse-tsan+0x11e3043e)
    1 DB::ContextShared::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:441:21 (clickhouse-tsan+0x11e3043e)
    2 DB::Context::shutdown() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:2249:13 (clickhouse-tsan+0x11e28c17)
    3 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5::operator()() const obj-x86_64-linux-gnu/../programs/server/Server.cpp:892:5 (clickhouse-tsan+0x8ab1a32)
    4 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::invoke() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:97:9 (clickhouse-tsan+0x8ab1a32)
    5 ext::basic_scope_guard<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&)::$_5>::~basic_scope_guard() obj-x86_64-linux-gnu/../base/common/../ext/scope_guard.h:47:28 (clickhouse-tsan+0x8ab1a32)
    6 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:1395:1 (clickhouse-tsan+0x8aacb39)
    7 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
    8 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
    9 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
    10 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
    11 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)

  Previous read of size 1 at 0x7b9000003a38 by thread T2 (mutexes: write M2504):
    0 std::__1::__optional_storage_base<DB::SystemLogs, false>::has_value() const obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:295:22 (clickhouse-tsan+0x11e25348)
    1 std::__1::optional<DB::SystemLogs>::operator bool() const obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:938:64 (clickhouse-tsan+0x11e25348)
    2 DB::Context::getQueryLog() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1875:10 (clickhouse-tsan+0x11e25348)
    3 DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:657:50 (clickhouse-tsan+0x12653751)
    4 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:904:30 (clickhouse-tsan+0x12651308)
    5 DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:289:24 (clickhouse-tsan+0x12f04b45)
    6 DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1500:9 (clickhouse-tsan+0x12f13907)
    7 Poco::Net::TCPServerConnection::start() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse-tsan+0x15b1f722)
    8 Poco::Net::TCPServerDispatcher::run() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:113:19 (clickhouse-tsan+0x15b1fe4e)
    9 Poco::PooledThread::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14 (clickhouse-tsan+0x15c86fe1)
    10 Poco::(anonymous namespace)::RunnableHolder::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:55:11 (clickhouse-tsan+0x15c8557f)
    11 Poco::ThreadImpl::runnableEntry(void*) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27 (clickhouse-tsan+0x15c83d87)

  Location is heap block of size 7296 at 0x7b9000002000 allocated by main thread:
    0 operator new(unsigned long) <null> (clickhouse-tsan+0x8a9aae7)
    1 std::__1::__unique_if<DB::ContextShared>::__unique_single std::__1::make_unique<DB::ContextShared>() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2068:28 (clickhouse-tsan+0x11e15c2c)
    2 DB::Context::createShared() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:503:32 (clickhouse-tsan+0x11e15c2c)
    3 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:426:27 (clickhouse-tsan+0x8aa19ee)
    4 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
    5 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
    6 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
    7 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
    8 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)

  Mutex M2504 (0x7b9000002008) created at:
    0 pthread_mutex_init <null> (clickhouse-tsan+0x8a0d37d)
    1 std::__1::__libcpp_recursive_mutex_init(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:370:10 (clickhouse-tsan+0x17cc4d93)
    2 std::__1::recursive_mutex::recursive_mutex() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:56:14 (clickhouse-tsan+0x17cc4d93)
    3 DB::ContextShared::ContextShared() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:394:5 (clickhouse-tsan+0x11e40bc3)
    4 std::__1::__unique_if<DB::ContextShared>::__unique_single std::__1::make_unique<DB::ContextShared>() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2068:32 (clickhouse-tsan+0x11e15c37)
    5 DB::Context::createShared() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:503:32 (clickhouse-tsan+0x11e15c37)
    6 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:426:27 (clickhouse-tsan+0x8aa19ee)
    7 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
    8 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
    9 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
    10 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
    11 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)

  Thread T2 'TCPHandler' (tid=13643, running) created by main thread at:
    0 pthread_create <null> (clickhouse-tsan+0x8a0bf0b)
    1 Poco::ThreadImpl::startImpl(Poco::SharedPtr<Poco::Runnable, Poco::ReferenceCounter, Poco::ReleasePolicy<Poco::Runnable> >) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:202:6 (clickhouse-tsan+0x15c83827)
    2 Poco::Thread::start(Poco::Runnable&) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:128:2 (clickhouse-tsan+0x15c84f6c)
    3 Poco::PooledThread::start() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:85:10 (clickhouse-tsan+0x15c873e2)
    4 Poco::ThreadPool::ThreadPool(int, int, int, int) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:252:12 (clickhouse-tsan+0x15c873e2)
    5 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/server/Server.cpp:843:22 (clickhouse-tsan+0x8aa8e5f)
    6 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse-tsan+0x15b3446b)
    7 DB::Server::run() obj-x86_64-linux-gnu/../programs/server/Server.cpp:342:25 (clickhouse-tsan+0x8aa04be)
    8 Poco::Util::ServerApplication::run(int, char**) obj-x86_64-linux-gnu/../contrib/poco/Util/src/ServerApplication.cpp:611:9 (clickhouse-tsan+0x15b50883)
    9 mainEntryClickHouseServer(int, char**) obj-x86_64-linux-gnu/../programs/server/Server.cpp:134:20 (clickhouse-tsan+0x8a9f08e)
    10 main obj-x86_64-linux-gnu/../programs/main.cpp:368:12 (clickhouse-tsan+0x8a9d5f9)

SUMMARY: ThreadSanitizer: data race obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:246:24 in std::__1::__optional_destruct_base<DB::SystemLogs, false>::reset()

v2: fix deadlock by calling SystemLogs::shutdown w/o Context lock
 2021-04-13 21:37:38 +03:00
Alexey Milovidov
cdbc9d9fa0 Merge branch 'master' into speedup-intdiv 2021-04-13 20:43:54 +03:00
Alexey Milovidov
c01756014e More generic implementation of has 2021-04-13 20:08:49 +03:00
Kruglov Pavel
6350f734dc
Merge branch 'master' into datasketches-uniq 2021-04-13 19:34:15 +03:00
alesapin
6d8efe5d5b
Merge pull request #22992 from ClickHouse/ssl_keeper 
Add SSL to keeper (both client and internal)
 2021-04-13 19:15:07 +03:00
tavplubix
1525e38a3c
Merge pull request #22990 from ClickHouse/tavplubix-patch-1 
Fix excessive warning in StorageDistributed with cross-replication
 2021-04-13 18:58:12 +03:00
tavplubix
da681d9907
Merge pull request #22998 from ClickHouse/fix_22989 
Fix assertion when filtering tables in StorageMerge
 2021-04-13 18:57:24 +03:00
alexey-milovidov
46e1da03fb
Update KeyCondition.cpp 2021-04-13 18:47:11 +03:00
pingyu
8244e9fc94 address comments 2021-04-13 23:09:05 +08:00
Alexey Milovidov
83a78a5aa7 Fix unfinished code in GatherUtils #20272 2021-04-13 16:56:39 +03:00
Alexander Tokmakov
fdf3cf378f fix testkeeper multi response 2021-04-13 15:49:40 +03:00
alesapin
60792843ea Less garbage 2021-04-13 15:04:18 +03:00
alesapin
b115b8af3d Something working 2021-04-13 14:55:08 +03:00
songenjie
7c32cc1e18 fix case style for local variable 2021-04-13 17:34:04 +08:00
hexiaoting
4f36ac7f64 Fix error 2021-04-13 17:31:49 +08:00
alexey-milovidov
1945d01719
Merge pull request #21972 from kssenii/nanodbc 
Switch to nanodbc
 2021-04-13 12:23:57 +03:00
Maksim Kita
305d3d6b16
Merge pull request #22978 from ClickHouse/filter-by-virtual-columns-fix 
Fix trivial mistake in filtering by virtual columns
 2021-04-13 11:42:16 +03:00
Maksim Kita
e687fc320f
Merge pull request #22997 from ClickHouse/increase-max-uri-size 
Increase `max_uri_size` to 1 MiB by default
 2021-04-13 11:26:23 +03:00
alesapin
cbd8a57fda Merge branch 'master' into ssl_keeper 2021-04-13 10:34:23 +03:00
Nikolai Kochetov
e731dfe650
Merge pull request #22991 from ClickHouse/untuple-and-subquery 
Fix subquery with untuple.
 2021-04-13 09:53:13 +03:00
songenjie
564136ec46 [clickhouse][server][dll][alter]support fetch part 2021-04-13 12:40:33 +08:00
hexiaoting
15f3a441e3 Fix error 2021-04-13 11:41:43 +08:00
hexiaoting
77c460e8d1 Merge remote-tracking branch 'origin/master' into dev-sumcount 2021-04-13 11:14:14 +08:00
hexiaoting
52bdec396b Merge remote-tracking branch 'origin/master' into dev_nested_upadte 2021-04-13 09:56:47 +08:00
alexey-milovidov
ee9aaab4bc
Merge pull request #22937 from Enmk/formatDateTime_toDateTime64_fixes 
Fixes: formatDateTime and toDateTime64
 2021-04-13 01:40:25 +03:00
alexey-milovidov
9e88f5fbd3
Update ReadHelpers.h 2021-04-13 01:39:15 +03:00
alexey-milovidov
a780e8e7e1
Update ReadHelpers.h 2021-04-13 01:38:59 +03:00
alexey-milovidov
e2edecc8e2
Merge pull request #22434 from Avogar/catboost-reload 
Update paths to the catboost model configs in config reloading
 2021-04-13 01:21:04 +03:00
alexey-milovidov
d239c66fd3
Merge pull request #21511 from azat/optimize_skip_unused_shards-improvement 
Rewrite IN in query for remote shards to exclude values that do not belong to the shard
 2021-04-13 01:18:11 +03:00
Alexey Milovidov
1b8a054880 Fix error 2021-04-13 00:11:30 +03:00
Alexey Milovidov
d5580a8e71 Dynamic dispatch for intDiv 2021-04-12 23:02:42 +03:00
Alexey Milovidov
ff0d3860d4 Fix typos + minor improvement #22314 2021-04-12 23:02:16 +03:00
alesapin
56f0e290de
Merge pull request #22945 from ClickHouse/increase-fetches-pool-size 
More conservative value of `background_fetches_pool_size` setting
 2021-04-12 22:41:04 +03:00
Azat Khuzhin
a497d4d462 Do not perform optimize_skip_unused_shards for cluster with one node 2021-04-12 22:18:31 +03:00
Alexander Tokmakov
ad414bab05 fix assertion when filtering tables in StorageMerge 2021-04-12 22:04:26 +03:00
Alexey Milovidov
e260b2a60a Increase max_uri_size to 1 MiB by default 2021-04-12 21:06:47 +03:00
Alexey Milovidov
fafe8c00fe Fix warning #22987 2021-04-12 20:57:03 +03:00
Denny Crane
eaa092cae2
Update MergeTreeData.cpp 
Better error message.
 2021-04-12 10:58:06 -03:00
alesapin
bc8ebb1028 Simplify build 2021-04-12 16:11:43 +03:00
alesapin
338ff1615c Simplier config 2021-04-12 15:40:01 +03:00
alesapin
ef34c95c7e Add SSL to keeper (both client and internal) 2021-04-12 15:25:52 +03:00
Nikolai Kochetov
f6d86d6032 Merge branch 'master' into add-read-from-mt-step 2021-04-12 15:23:32 +03:00
Nikolai Kochetov
0448e1415f Fix subquery with untuple. 2021-04-12 15:15:55 +03:00
tavplubix
a995962e6a
Update StorageDistributed.cpp 2021-04-12 14:58:24 +03:00
Vladimir
4f34bc4e73
Merge pull request #22973 from kreuzerkrieg/Fix_Poco_HTTP_Client 
Fix an error handling in Poco HTTP Client for AWS.
 2021-04-12 13:40:44 +03:00
Dmitry Krylov
df60724847 Fix constness 2021-04-12 20:29:58 +10:00
Dmitry Krylov
a3d57bd5af Check number of arguments 2021-04-12 20:29:35 +10:00
Nikolai Kochetov
7019a9a659
Merge pull request #22673 from nvartolomei/nv/exp-arrow-stream 
Experiment with ArrowStream streaming
 2021-04-12 13:29:25 +03:00
Pavel Kruglov
baccd09efe Fix synchronization 2021-04-12 13:05:46 +03:00
Pavel Kruglov
e4f3678ae9 Fix build 2021-04-12 12:28:46 +03:00
Nikolai Kochetov
28d6e3f56c
Merge pull request #22977 from azat/combinator-fix 
Fix combinators with common prefix name (State and SimpleState) with libstdc++
 2021-04-12 12:27:01 +03:00
Maksim Kita
b9fe85d537
Merge pull request #22964 from ClickHouse/uniq-up-to-ubsan 
Check out of range values in FieldVisitorConverToNumber
 2021-04-12 11:50:19 +03:00
alesapin
fea0ec5046 Some code 2021-04-12 11:10:23 +03:00
Maksim Kita
3078fd7d79
Merge pull request #22983 from borodark/master 
Error message reads better
 2021-04-12 10:56:10 +03:00
Azat Khuzhin
260a77118f Apply idle_connnection_timeout/poll_interval after each query 
Since right now it is impossible to change those settings, only via
user profile.
 2021-04-12 10:39:28 +03:00
Azat Khuzhin
79bd8d4d3f Respect optimize_skip_unused_shards_rewrite_in with optimize_skip_unused_shards_limit 2021-04-12 10:37:28 +03:00
Azat Khuzhin
e439914d38 Fix optimized cluster logic for optimize_skip_unused_shards 2021-04-12 10:37:28 +03:00
Azat Khuzhin
fbb386dca5 Rewrite IN in query for remote shards to exclude values that does not belongs to shard 
v2: fix optimize_skip_unused_shards_rewrite_in for sharding_key wrapped into function
v3: fix column name for optimize_skip_unused_shards_rewrite_in
v4: fix optimize_skip_unused_shards_rewrite_in with Null
v5:
- squash with Remove query argument for IStreamFactory::createForShard()
- use proper column after function execution (using sharding_key_column_name)
- update the test reference since (X) now is tuple(X)
 2021-04-12 10:37:28 +03:00
Azat Khuzhin
773212529b Make Cluster::SlotToShards public 2021-04-12 10:32:04 +03:00
Dmitry Krylov
68bd27f3c5 Fix code style 2021-04-12 12:12:40 +10:00
Dmitry Krylov
35472bcc2f Update to new IFunction::create() 2021-04-12 11:37:44 +10:00
Egor O'Sten
c8b572ed59 Readability of error message for ORDER BY 2021-04-11 20:42:05 -04:00
Egor O'Sten
c77dd36ca9 Readability of error message for ORDER BY 2021-04-11 20:40:00 -04:00
Dmitry Krylov
582f6d3a72 Remove unneeded branches of code 2021-04-12 10:23:31 +10:00
Dmitry Krylov
c2f326e2ae Remove unneeded branches of code 2021-04-12 10:23:31 +10:00
Dmitry Krylov
ce1b9b20d5 Remove unneeded branches of code 2021-04-12 10:23:31 +10:00
Dmitry Krylov
15ea9b3f22 Move arrayFold functionality to arrayFold.cpp 2021-04-12 10:23:31 +10:00
Dmitry Krylov
517ef8ebf4 Bug with string functions in arrayFold fixed 2021-04-12 10:23:31 +10:00
Dmitry Krylov
e3502dabc7 Remove debug messages 2021-04-12 10:23:31 +10:00
Dmitry Krylov
1861817bfc Remove debug messages 2021-04-12 10:23:31 +10:00
Dmitry Krylov
9446df026b Cleaning code 2021-04-12 10:23:31 +10:00
Dmitry Krylov
b19a06ba48 Fix to work with empty arrays 2021-04-12 10:23:31 +10:00
Dmitry Krylov
fbd43aeea8 Draft: taking into accout input arrays variable length 2021-04-12 10:23:31 +10:00
Dmitry Krylov
2e8a296cc9 Draft: very simple variant 2021-04-12 10:23:31 +10:00
Dmitry Krylov
7c47832405 Experiment: attempt to shrink arrays 2021-04-12 10:23:31 +10:00
kssenii
df84e4e93a Fix 2021-04-11 23:39:18 +00:00
kssenii
7a287e6fe9 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into nanodbc 2021-04-11 21:36:08 +00:00
Kseniia Sumarokova
71484fcfbd
Merge branch 'master' into postgresql-protocol-with-row-policy 2021-04-12 00:12:49 +03:00
kssenii
792d57d630 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into postgresql-protocol-with-row-policy 2021-04-11 21:10:33 +00:00
Alexey Milovidov
22720dd7a4 Fix trivial mistake in filtering by virtual columns 2021-04-11 22:39:22 +03:00
Azat Khuzhin
0bf6b61b59 Fix combinators with common prefix name (State and SimpleState) with libstdc++ 
Previously sort order of the std::unordered_map in libstdc++ was
different and any *SimpleState() reports an error that function does not
exists.

Fix this by using proper order in container, and use std::vector over
std::unordered_map, since there linear traversing anyway in the single
method -- tryFindSuffix()

Note that test is not required, since it either fail with unknown
function or not.
 2021-04-11 22:10:02 +03:00
Maksim Kita
b571656fd7
Merge pull request #22931 from Enmk/dateDiff_DateTime64_fix 
Fixed dateDiff for DateTime64
 2021-04-11 21:36:24 +03:00
Maksim Kita
aa6183af86
Merge pull request #22971 from azat/max_part_removal_threads 
Respect max_part_removal_threads for ReplicatedMergeTree
 2021-04-11 21:14:47 +03:00
Vasily Nemkov
ffd808f877
Removed redundant access specifier 2021-04-11 17:38:11 +03:00
Vasily Nemkov
0f7c83b5ac Fixed parsing DateTime64 as unix timestamp with subsecond precision scaled to integer. 
e.g. toDateTime64('1111111111222', 3)
 2021-04-11 17:28:58 +03:00
kreuzerkrieg
228a0a0647 Fix an error handling in Poco HTTP Client for AWS. 2021-04-11 17:22:19 +03:00
Azat Khuzhin
b24d1251de Introduce helper for removing one part from fs for ReplicatedMergeTree 2021-04-11 14:38:46 +03:00
alexey-milovidov
0f6fc5bd72
Merge pull request #22962 from kitaisreal/storage-external-distributed-arcadia-fix 
StorageExternalDistributed arcadia fix
 2021-04-11 13:34:03 +03:00
alexey-milovidov
d21ffd83a7
Merge branch 'master' into catboost-reload 2021-04-11 13:29:36 +03:00
Maksim Kita
20d5d550c4
Merge pull request #22960 from kitaisreal/dictionary-added-decimal256-attribute-type-support 
Dictionary added Decimal256 attribute type support
 2021-04-11 13:07:00 +03:00
Azat Khuzhin
940ff6cb76 Do not use c-style cast for std::min(max_part_removal_threads) 2021-04-11 12:22:39 +03:00
Maksim Kita
8dbf721e52
Merge pull request #22961 from kitaisreal/dictionaries-standardize-exceptions 
Dictionaries standardize exceptions
 2021-04-11 12:05:59 +03:00
Azat Khuzhin
6e71757814 Fix thread accounting for max_part_removal_threads 2021-04-11 11:43:30 +03:00
Azat Khuzhin
dfe3699e77 Respect max_part_removal_threads for ReplicatedMergeTree 2021-04-11 11:33:45 +03:00
pingyu
251804e9aa fix style check error 2021-04-11 14:23:39 +08:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
Alexey Milovidov
d369da6643 Update tests 2021-04-11 00:52:42 +03:00
alexey-milovidov
5469cc15c1
Trigger CI 2021-04-10 23:43:56 +03:00
Alexey Milovidov
1f2226b9db Check out of range values in FieldVisitorConverToNumber 2021-04-10 23:38:14 +03:00
alexey-milovidov
e9b39c40f8
Merge pull request #22947 from azat/inactive_parts_to_throw_insert-fix 
Fix inactive_parts_to_throw_insert=0 with inactive_parts_to_delay_insert>0
 2021-04-10 23:21:16 +03:00
Denis Glazachev
5e8fe847a2 Remove unneeded changes 
Fix GCC build
 2021-04-10 23:23:18 +04:00
Maksim Kita
685da4e234 StorageExternalDistributed arcadia fix 2021-04-10 21:52:17 +03:00
Maksim Kita
b43a709fec Dictionaries standardize exceptions 2021-04-10 21:48:36 +03:00
Maksim Kita
7df43891c1 Dictionary added Decimal256 attribute type support 2021-04-10 19:53:21 +03:00
pingyu
df78a1b3fd address review comments 2021-04-10 23:00:34 +08:00
changvvb
03a2ba315b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into support-apple-m1 2021-04-10 21:31:38 +08:00
alexey-milovidov
2d4a8c4787
Merge pull request #22885 from traceon/gcc-macos-compilation-fixes 
Fix vanilla GCC compilation in macOS
 2021-04-10 14:17:11 +03:00
alexey-milovidov
969d9f495b
Merge pull request #22939 from CurtizJ/fix-const-map-conversion 
Fix conversion from const map with nulls
 2021-04-10 14:13:58 +03:00
Maksim Kita
8b34a54c2d
Merge pull request #22938 from kitaisreal/aggregate-function-allocates-memory-in-arena-default-impl-fix 
IAggreagteFunction allocatesMemoryInArena removed default implementation
 2021-04-10 14:08:59 +03:00
Azat Khuzhin
a6703c5ac2 Fix inactive_parts_to_throw_insert=0 with inactive_parts_to_delay_insert>0 2021-04-10 09:57:08 +03:00
Alexey Milovidov
173c0e5c43 More conservative value of background_fetches_pool_size setting 2021-04-09 23:45:14 +03:00
Alexander Kuzmenkov
e44b3822e3
Merge pull request #21850 from fastio/handle_errors_for_kafka_engine 
Handle errors for Kafka engine
 2021-04-09 22:59:40 +03:00
Anton Popov
329c9033a8 fix conversion from const map with nulls 2021-04-09 20:50:10 +03:00
Maksim Kita
d9e9224188 IAggreagteFunction allocatesMemoryInArena removed default implementation 2021-04-09 20:12:06 +03:00
Vasily Nemkov
c882d290bd Fixed parsing string to DateTime64 for large values. 
Before fix:
SELECT toDateTime64('2201-01-12 12:12:12', 3, 'UTC')
1970-03-26 09:10:48.237

After fix:
SELECT toDateTime64('2201-01-12 12:12:12', 3, 'UTC')
2201-01-12 12:12:12.000
 2021-04-09 19:39:54 +03:00
Vasily Nemkov
4dcc1b2d2e Fixed formatDateTime for DateTime64 arguments 2021-04-09 19:39:54 +03:00
changvvb
facbb0368b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into support-apple-m1 2021-04-09 23:19:19 +08:00
Alexander Kuzmenkov
0264124146
Merge pull request #21942 from ucasFL/distributed_depth 
Add settings max_distributed_depth
 2021-04-09 15:52:58 +03:00
Alexander Kuzmenkov
5db8e4c111
Update SystemLog.h 2021-04-09 15:48:09 +03:00
Vasily Nemkov
4a784bd2e3 Fixed dateDiff for DateTime64 
Added tests that verify fix
- against DateTime64 vals of equal scale
- DateTime64 vs DateTime
- DateTime vs DateTime64
- DateTime64 vs Date
- Date vs DateTime64
 2021-04-09 15:29:16 +03:00
kssenii
72d89bc933 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into nanodbc 2021-04-09 11:13:01 +00:00
Nikolai Kochetov
d0b5615641 Add comments. 2021-04-09 12:35:21 +03:00
Maksim Kita
0cacca5b6e
Merge pull request #22833 from ClickHouse/trivial-fix 
Fix what looks like a trivial mistake
 2021-04-09 12:02:16 +03:00
Nikolai Kochetov
28ca191102
Merge pull request #22087 from ClickHouse/better-filter-push-down 
Better filter push down
 2021-04-09 10:22:17 +03:00
alesapin
4b0f973f93
Merge pull request #18215 from filimonov/quorum-dedup-issue 
Fix issue with quorum retries behaviour
 2021-04-09 10:19:50 +03:00
alexey-milovidov
42b865c9c1
Merge pull request #22829 from azat/dist-send-log 
Slightly improve logging messages for Distributed async sends
 2021-04-09 09:51:53 +03:00
alexey-milovidov
ee9b53a342
Merge pull request #22863 from kitaisreal/data-type-low-cardinality-format-tsv-parsing-issue 
DataTypeLowCardinality format tsv parsing issue
 2021-04-09 09:50:50 +03:00
alexey-milovidov
f7edcdf7c8
Merge pull request #22867 from azat/sparse_hashed-bytes_allocated-fix 
Fix bytes_allocated for sparse_hashed dictionaries
 2021-04-09 09:50:04 +03:00
alexey-milovidov
bcb8da031d
Merge pull request #22860 from ClickHouse/fix_coordination_build_macos 
Fix Coordination MacOS build
 2021-04-09 09:48:44 +03:00
alexey-milovidov
b02a82250b
Merge pull request #22858 from vitlibar/fix-hanging-in-temporary-live-view-cleaner 
Fix hanging in TemporaryLiveViewCleaner
 2021-04-09 09:25:38 +03:00
alexey-milovidov
8af2352e27
Merge pull request #22876 from nikitamikhaylov/mann-whitney-over-window 
`mannWhitneyUTest` and `rankCorr` with window functions
 2021-04-09 09:17:27 +03:00
Denis Glazachev
e8464d61f1 Fix vanilla GCC compilation in macOS 2021-04-09 04:28:24 +04:00
Nikita Mikhaylov
86933749bd done 2021-04-09 00:22:18 +03:00
Anton Popov
1c28878f5d
Merge pull request #22726 from azat/mt-reverse-total-rows-approx-fix 
Fix approx total rows accounting for reverse reading from MergeTree
 2021-04-09 00:16:49 +03:00
Azat Khuzhin
93c9a02480 Fix bytes_allocated for sparse_hashed dictionaries 2021-04-08 23:29:29 +03:00
Nikolai Kochetov
3426bc3906
Merge pull request #22763 from CurtizJ/fix-having-push-down 
Fix pushdown of having
 2021-04-08 21:53:50 +03:00
Maksim Kita
942399983c DataTypeLowCardinality format tsv parsing issue 2021-04-08 20:12:41 +03:00
Nikolai Kochetov
3ea186358e Fix fuzz crash. 2021-04-08 18:00:32 +03:00
kssenii
80fc675258 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into postgresql-protocol-with-row-policy 2021-04-08 14:46:56 +00:00
alesapin
07442b1170 Fix stupid sed 2021-04-08 17:24:05 +03:00
alesapin
c28a3b860c Fix Coordination darwin build 2021-04-08 17:17:57 +03:00
Nikolai Kochetov
794d7c89b6 Remove ReverseRowsStep 2021-04-08 16:18:17 +03:00
Vitaly Baranov
5bcb7a0fed Fix hanging in TemporaryLiveViewCleaner 2021-04-08 16:14:49 +03:00
Nikolai Kochetov
febb187da0 Added test, 2021-04-08 15:54:27 +03:00
Alexander Kuzmenkov
4a66548c5d
write distributed depth as VarUInt to save space, because it's small 2021-04-08 15:17:47 +03:00
Nikolai Kochetov
7ffbeac9df Add info about indexes to ReadFromMergeTree step. 2021-04-08 14:48:54 +03:00
Alexander Kuzmenkov
dc7a2b7f88
Merge pull request #22761 from ClickHouse/aku/flush-log 
add more messages when flushing the logs
 2021-04-08 13:50:17 +03:00
alesapin
e762d02e37 More const 2021-04-08 13:38:40 +03:00
alesapin
e17444757b Fix rare case when quorum insert is not really quorum because of deduplication 2021-04-08 13:35:38 +03:00
changvvb
b070090ca7 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into support-apple-m1 2021-04-08 18:11:16 +08:00
Nikolai Kochetov
118090396f Filter column after filter push down now always the first one.. 2021-04-08 12:54:39 +03:00
Alexey Milovidov
8fb6e6fbf2 Fix mistake 2021-04-08 11:51:22 +03:00
Nikolai Kochetov
7c5a9133df Add index info to ReadFromStorageStep. 2021-04-08 11:19:04 +03:00
alesapin
1533f9b9aa
Merge pull request #14113 from johnskopis/dynamic-interserver-creds-v20 
Support interserver credential rotation
 2021-04-08 11:01:40 +03:00
alexey-milovidov
b8e239c211
Merge pull request #22821 from kitaisreal/functions-external-dictionaries-standardize-exception-throw 
Functions ExternalDictionaries standardize exception throw
 2021-04-08 10:41:41 +03:00
alexey-milovidov
ed367855d5
Merge pull request #22822 from kitaisreal/file-dictionary-source-fix-absolute-path 
FileDictionarySource fix absolute file path
 2021-04-08 10:34:07 +03:00
Azat Khuzhin
c27b931f6a Slightly improve logging messages for Distributed async sends 
- add took time (in ms)
- add rows/bytes
 2021-04-08 08:10:39 +03:00
Nicolae Vartolomei
7259834fad More Arrow like code style 2021-04-07 23:33:25 +03:00
Nicolae Vartolomei
4ea363006b Experiment with ArrowStream streaming 2021-04-07 23:33:25 +03:00
Maksim Kita
ef7571c226
Merge pull request #22521 from kitaisreal/dictionaries-update-field-fix 
FlatDictionary, HashedDictionary update_field optimizations
 2021-04-07 23:29:53 +03:00
alesapin
c85a60d795
Merge pull request #22743 from ClickHouse/fix_potential_segfault_on_startup 
Fix potential segfault on Keeper startup
 2021-04-07 23:10:29 +03:00
Maksim Kita
ff04292ac8
Update FileDictionarySource.cpp 2021-04-07 21:42:09 +03:00
Maksim Kita
f7cc15fdcd FileDictionarySource fix absolute file path 2021-04-07 21:38:24 +03:00
Maksim Kita
caff65f1c2 Functions ExternalDictionaries standardize exception throw 2021-04-07 20:52:12 +03:00
alesapin
b2a370ad49 Fix PVS 2021-04-07 19:06:20 +03:00
alesapin
98871f8c5a More clear messages 2021-04-07 17:05:40 +03:00
Anton Popov
fa9a19cbdd fix pushdown of having 2021-04-07 17:02:59 +03:00
alesapin
2a6560b1c6 Better interface 2021-04-07 16:59:18 +03:00
alesapin
2987bbc948 Small improvements 2021-04-07 16:52:11 +03:00
Ivan
aa47eb7d5e
Fix checkpoint position after putting it on buffer end (#22518) 2021-04-07 16:22:38 +03:00
Alexander Kuzmenkov
fecd5f3435 more debug messages 2021-04-07 16:16:19 +03:00
Nikita Mikhaylov
48af7a898c
Merge pull request #22299 from nikitamikhaylov/keen-wolf-storagefile-column-oriented 
Merging #21302
 2021-04-07 16:01:12 +03:00
kssenii
15ae912b56 Fix postgresql protocol with row policy 2021-04-07 12:56:09 +00:00
Alexander Kuzmenkov
06bb58cb69
Merge pull request #22704 from ClickHouse/aku/window-multiple 
fix window functions with multiple input streams and no sorting
 2021-04-07 15:15:47 +03:00
Vladimir
9fe20c1628
Revert "Move conditions from JOIN ON to WHERE" 2021-04-07 14:57:20 +03:00
Kruglov Pavel
5b52657428
Merge pull request #22725 from azat/buffer-lock-metrics 
Add metric to track how much time is spend during waiting for Buffer layer lock
 2021-04-07 14:35:57 +03:00
alesapin
feff1175f4 Merge branch 'master' into johnskopis/dynamic-interserver-creds-v20 2021-04-07 13:55:38 +03:00
alesapin
a1164a7e4c More consistent 2021-04-07 13:21:53 +03:00
alesapin
36c0e601a9 Better non-dirty fix 2021-04-07 13:18:07 +03:00
Ilya Yatsishin
ed2d82603e
Merge pull request #22588 from foolchi/fix_dist_map_select 
fix dist map type select
 2021-04-07 11:53:23 +03:00
Vladimir
864ab4a7ab
Merge pull request #21370 from vdimir/fix-merge-engine-join-group-by 2021-04-07 11:52:27 +03:00
Vladimir
db0550013c
Merge pull request #22325 from azat/dist-async-insert-header-check 
Check only column name and type for async distributed blocks
 2021-04-07 11:51:12 +03:00
alesapin
05eeec16c1 Fix potential segfault on Keeper startup 2021-04-07 11:49:10 +03:00
Maksim Kita
7baafcb567
Merge pull request #22630 from kitaisreal/direct-dictionary-dict-get-multiple-columns-optimization 
DirectDictionary dictGet multiple columns optimization
 2021-04-07 10:29:42 +03:00
alesapin
9fd251eaaf
Merge pull request #22514 from ClickHouse/merge_tree_deduplication 
Non replicated merge tree deduplication
 2021-04-07 10:19:43 +03:00
alexey-milovidov
80cb871e20
Merge pull request #22684 from ClickHouse/exception-safety-in-zk-client-2 
Always call event.set for zk client callbacks.
 2021-04-07 07:20:41 +03:00
alexey-milovidov
a7e216969a
Merge pull request #22592 from azat/uncaught-exceptions 
Fix some uncaught exceptions (in SCOPE_EXIT) under memory pressure
 2021-04-07 07:17:26 +03:00
hexiaoting
ee24d9a5cc Merge remote-tracking branch 'origin/master' into dev_nested_upadte 2021-04-07 10:42:51 +08:00
hexiaoting
8c60f95551 Fix error 2021-04-07 10:39:01 +08:00
feng lv
78afb5b23c fix 2021-04-07 01:58:57 +00:00
alesapin
c77a006325
Merge pull request #22707 from ClickHouse/don_t_fsync_on_log_rotate 
Remove redundant fsync on coordination logs rotation
 2021-04-06 23:51:28 +03:00
kssenii
0b39480d51 Better 2021-04-06 20:01:23 +00:00
alexey-milovidov
5d672d4529 Update FormatFactory.cpp 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
71a2c85d9a better 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
af85ce2fd1 better 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
37f48d13b4 add test 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
282775b10c save 2021-04-06 22:23:16 +03:00
keenwolf
1a4961499b the Native format is not supported after test 2021-04-06 22:23:16 +03:00
keenwolf
94c8e7af1a remove const for value-copy-return 2021-04-06 22:23:16 +03:00
keenwolf
3d3923b518 update comments 2021-04-06 22:23:16 +03:00
keenwolf
b95f54edce Small fix 2021-04-06 22:23:16 +03:00
keenwolf
139bb678a9 Branch with Format isColumnOriented() or not 2021-04-06 22:23:15 +03:00
keenwolf
4eaf718fd2 fix the getColumsForNames() to bring the whole column info from metadata 2021-04-06 22:23:15 +03:00
keenwolf
9d6e2b4bad update comments 2021-04-06 22:23:15 +03:00
keenwolf
de365a5852 fix 2021-04-06 22:23:15 +03:00
keenwolf
6f9bad682f Only read needed columns for formats as parquet etc 2021-04-06 22:23:15 +03:00
Maksim Kita
e38206ee02
Merge pull request #22721 from kitaisreal/memory-storage-sync-comments-and-code 
MemoryStorage sync comments and code
 2021-04-06 21:56:41 +03:00
Azat Khuzhin
60b000ed51 Fix approx total rows accounting for reverse reading from MergeTree 2021-04-06 21:13:51 +03:00
Azat Khuzhin
c4a7e81287 Add metric to track how much time is spend during waiting for Buffer layer lock 
It uses very fast CLOCK_MONOTONIC_COARSE, so this should not be a
problem.
Also note that there is no sense in using microseconds/nanoseconds since
accuracy of CLOCK_MONOTONIC_COARSE usually milliseconds.
 2021-04-06 21:13:24 +03:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT 
It executes the code in the dtor, that should never throw.
 2021-04-06 21:10:37 +03:00
Azat Khuzhin
b25a4e066c Lock MEMORY_LIMIT_EXCEEDED in ThreadStatus::detachQuery() 
Found with fuzzer [1]:

    <Fatal> BaseDaemon: (version 21.5.1.6440, build id: 3B097C902DDAA35688D90750552ED499DC5D10A0) (from thread 8012) Terminate called for uncaught exception:
    Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 153.51 MiB (attempt to allocate chunk of 4194368 bytes), maximum: 150.00 MiB, Stack trace (when copying this message, always include the lines below):

    0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x26fdcdd9 in /usr/bin/clickhouse
    1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:57: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xad0df02 in /usr/bin/clickhouse
    2. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:1444: DB::Exception::Exception<char const*, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*&&, char const*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0xacc7bef in /usr/bin/clickhouse
    3. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:219: MemoryTracker::alloc(long) @ 0xacc65eb in /usr/bin/clickhouse
    4. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
    5. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/Common/AllocatorWithMemoryTracking.h:35: AllocatorWithMemoryTracking<DB::Field>::allocate(unsigned long) @ 0xad0a2fe in /usr/bin/clickhouse
    7. void std::__1::vector<DB::Field, AllocatorWithMemoryTracking<DB::Field> >::__push_back_slow_path<DB::Field>(DB::Field&&) @ 0x11712a51 in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Interpreters/ThreadStatusExt.cpp:356: DB::ThreadStatus::detachQuery(bool, bool) @ 0x1f5d5237 in /usr/bin/clickhouse
    9. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:0: void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0x20c488e6 in /usr/bin/clickhouse
    10. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:0: ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xad9f6cc in /usr/bin/clickhouse
    11. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1655: void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) @ 0xada8264 in /usr/bin/clickhouse
    12. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    13. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
     (version 21.5.1.6440)

  [1]: https://clickhouse-test-reports.s3.yandex.net/22583/69296876005c0fa171c755f8b224e4d58192c402/stress_test_(address).html#fail1
 2021-04-06 21:10:36 +03:00
alesapin
95881bf2dd Fix duplicated headers 2021-04-06 19:31:24 +03:00
Alexander Kuzmenkov
d9ad06bd6d fix ubsan 2021-04-06 19:24:56 +03:00
alesapin
b4fc13de8e fix style 2021-04-06 17:02:56 +03:00
alesapin
4633afe962 Some fixes 2021-04-06 16:56:14 +03:00
alesapin
d913f327ed Buildable code 2021-04-06 16:42:38 +03:00
alesapin
1c8e33414f Merge branch 'master' into johnskopis/dynamic-interserver-creds-v20 2021-04-06 16:28:46 +03:00
changvvb
55a2ea4a79 GTEST_ALLOW_UNINSTANTIATED_PARAMETERIZED_TEST(DateLUTWithTimeZoneAndTimeRange); 2021-04-06 21:26:51 +08:00
Amos Bird
a89e3df40a
Disable table function view in expression (#21465) 2021-04-06 16:08:29 +03:00
Nikolai Kochetov
9f39f5d52d Add more counters to MergeTreeDataSelectExecutor 2021-04-06 15:39:55 +03:00
alesapin
a40209e84d Remove strange fsync on coordination logs rotation 2021-04-06 15:25:15 +03:00
Alexander Kuzmenkov
585da2c8d0 Merge remote-tracking branch 'origin/master' into HEAD 2021-04-06 14:58:21 +03:00
Alexey Milovidov
f70c9e8752 Merge branch 'master' into protocol-compression-auto 2021-04-06 14:53:31 +03:00
alexey-milovidov
3708821689
Merge pull request #22655 from azat/system.errors-invalid-read 
Fix impossible invalid-read for system.errors accounting
 2021-04-06 14:48:33 +03:00
Alexander Kuzmenkov
06e7a9f8f2 fix window functions with multiple input streams and no sorting 2021-04-06 14:43:27 +03:00
Maksim Kita
af19f89273 MemoryStorage sync comments and code 2021-04-06 13:41:48 +03:00
Nikolai Kochetov
6102652c99 Merge branch 'master' into better-filter-push-down 2021-04-06 13:38:03 +03:00
Kseniia Sumarokova
ad85467128
Merge pull request #22668 from kssenii/postgres-add-bool 
Add bool type in postgres engine
 2021-04-06 13:29:33 +03:00
alesapin
53a7836a9b Fix typos 2021-04-06 13:16:34 +03:00
Alexander Kuzmenkov
4402ec3101
Merge pull request #22324 from ClickHouse/aku/perf-stable 
make some perf test queries more stable
 2021-04-06 13:15:38 +03:00
alesapin
6456a1507c Support alter setting 2021-04-06 13:14:44 +03:00
vdimir
735154c81a
Handle not plain where tree in StorageMerge modifySelect 2021-04-06 12:29:29 +03:00
alesapin
86a843bb51
Merge pull request #21509 from kssenii/library-bridge 
clickhouse-library-bridge for library dictionary source
 2021-04-06 12:26:08 +03:00
Nikolai Kochetov
44700d97e9 Add setting to disable query plan optimization. Fix perftest. 2021-04-06 11:25:54 +03:00
tavplubix
8b1d320633
Merge pull request #22217 from kssenii/replicas-shards-for-mysql-and-postgres 
Replicas and shards for mysql and postgres storages
 2021-04-06 11:05:24 +03:00
alesapin
3cd571f92b
Merge pull request #22669 from ClickHouse/fix_two_bugs_with_mutation_wait 
Fix two stupid bugs in mutation wait
 2021-04-06 11:04:16 +03:00
changvvb
1f8bd034a3 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into support-apple-m1 2021-04-06 15:55:03 +08:00
changvvb
3a9d063f12 Fix unit test 2021-04-06 15:54:35 +08:00
Azat Khuzhin
27d4fbd13b Compare Block itself for distributed async INSERT batches 
INSERT into Distributed with insert_distributed_sync=1 stores the
distributed batches on the disk for sending in background.

But types may be a little bit different for the Distributed and it's
underlying table, so the initiator need to know whether conversion is
required or not.

Before this patch those on disk distributed batches contains header,
which includes dumpStructure() for the block in that batch, however it
checks not only names and types and plus dumpStructure() is a debug
method.

So instead of storing string representation for the block header we
should store empty block in the file header (note, that we cannot store
the empty block not in header, since this will require reading all
blocks from file, due to some trickery of the readers interface).

Note, that this patch also contains tiny refactoring:
- s/header/distributed_header/

v1: dumpNamesAndTypes()
v2: dump empty block into the batch itself
v3: move empty block into the header
 2021-04-06 10:05:21 +03:00
Amos Bird
46c9affa83
Fix CTE in view. 2021-04-06 11:32:16 +08:00
Anton Popov
c5354885a6
Merge pull request #22654 from azat/log-nested-fix 
[RFC] Fix LOGICAL_ERROR for Log with nested types w/o columns in the SELECT clause
 2021-04-06 00:34:58 +03:00
kssenii
68154fe10a Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into postgres-add-bool 2021-04-05 20:27:04 +00:00
Maksim Kita
de71553c9d
Merge pull request #22593 from zlobober/introduce_distributed_write 
Introduce IStorage::distributedWrite method for distributed INSERT SELECTS
 2021-04-05 22:59:57 +03:00
kssenii
dc42d5189d Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-05 15:36:35 +00:00
alesapin
c830397317 Fix two stupid bugs in mutation wait 2021-04-05 18:08:43 +03:00
kssenii
cc2fdc6c62 Add bool type 2021-04-05 14:49:52 +00:00
kssenii
89a2e94364 Fixes 2021-04-05 14:08:49 +00:00
feng lv
f6e4e13f89 fix conflict 2021-04-05 14:08:30 +00:00
feng lv
56073db22d max distributed depth 
Add settings max_distributed_depth

fix style

fix

fix

fix

fix test

fix

fix
 2021-04-05 14:00:54 +00:00
kssenii
9dba529494 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-04-05 13:14:51 +00:00
alesapin
156ba8a7fc Disable setting by default and ban for old-syntax MergeTree 2021-04-05 12:16:38 +03:00
pingyu
e05ee09621 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into datasketches-uniq 2021-04-05 16:47:53 +08:00
Maksim Kita
d9cc03bc34 Fixed tests 2021-04-05 11:29:05 +03:00
alesapin
8d5a787f6b Merge branch 'master' into merge_tree_deduplication 2021-04-05 10:40:03 +03:00
Azat Khuzhin
181cde0970 Fix impossible invalid-read for system.errors accounting 
Simple reproducer is throw Exception(-1, "foo"), but there is no such
code, hence it is impossible.
 2021-04-05 09:01:29 +03:00
Azat Khuzhin
42d858b7e5 Fix LOGICAL_ERROR for Log with nested types w/o columns in the SELECT clause 
Found with fuzzer [1]:

<details>

    2021.04.04 10:25:46.762596 [ 135 ] {4b6b5de3-d242-4267-910a-76b235467356} <Fatal> : Logical error: 'Logical error: no information about file column%2Ename.size0 in StorageLog'.
    ...
    2021.04.04 10:25:46.763563 [ 44 ] {} <Trace> BaseDaemon: Received signal 6
    2021.04.04 10:25:46.765884 [ 165 ] {} <Fatal> BaseDaemon: 5. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.04.04 10:25:46.901540 [ 165 ] {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:51: DB::handle_error_code() @ 0x11cc4f16 in /workspace/clickhouse
    2021.04.04 10:25:47.030755 [ 165 ] {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:58: DB::Exception::Exception() @ 0x11cc5025 in /workspace/clickhouse
    2021.04.04 10:25:47.617201 [ 165 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:183: DB::LogSource::readData()

</details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/22583/69296876005c0fa171c755f8b224e4d58192c402/fuzzer_debug/report.html#fail1

The problem here is that there is no column%2Ename.size0 there is
column.size0 instead, and it seems that reading sizeX file is not enough
in other places, so just filter them out in
ExpressionActions::getSmallestColumn() (fallback method for storages w/o
getColumnSizes() implemented)
 2021-04-05 08:48:04 +03:00
alexey-milovidov
6cefb8e620
Merge pull request #22583 from azat/log-rwlock-ub-fix 
Avoid UB in *Log engines for rwlock unlock due to unlock from another thread
 2021-04-05 05:20:16 +03:00
alexey-milovidov
f78c846bdf
Merge pull request #22591 from nvartolomei/nv/out-init-check 
Try flush write buffer only if it is initialized
 2021-04-05 04:32:54 +03:00
Maxim Akhmedov
725fa17961 Introduce IStorage::distributedWrite method for distributed INSERT SELECT. 2021-04-05 02:14:27 +03:00
Alexey Milovidov
e50d82ab45 Merge branch 'master' into change-accurate-comparison 2021-04-05 01:38:55 +03:00
Alexey Milovidov
6e93d936d3 Highlight potential bugs in code 2021-04-05 01:38:47 +03:00
Maksim Kita
deb35c5f4e DirectDictionary dictGet multiple columns optimization 2021-04-05 01:26:26 +03:00
Maksim Kita
e486fde35e
Update HashMap.h 2021-04-05 01:20:17 +03:00
Maksim Kita
fa49817421
Update HashMap.h 2021-04-05 01:01:20 +03:00
alexey-milovidov
b1e7624b3f
Merge pull request #22589 from ClickHouse/fix-comments 
Fix comments
 2021-04-05 00:22:27 +03:00
alexey-milovidov
c0a13bb5d2
Merge pull request #22586 from ClickHouse/minor-improvement-deserialize-index 
Minor improvement in index deserialization
 2021-04-05 00:20:29 +03:00
alesapin
caaa19edb8
Merge pull request #22373 from ClickHouse/jepsen_for_ci 
Add image to run jepsen tests
 2021-04-04 18:35:21 +03:00
alesapin
0ceb95aa55 Merge branch 'master' into merge_tree_deduplication 2021-04-04 18:32:27 +03:00
Anton Popov
0d03b8c0c4
Merge pull request #21889 from CurtizJ/fix-aggregation-in-order-1 
Fix bugs in aggregation by primary key
 2021-04-04 18:17:04 +03:00
pingyu
1ea74a1947 bug fix. add unit tests.# 2021-04-04 22:22:47 +08:00
Maksim Kita
027ca2484e FlatDictionary added layout options initial_array_size, max_array_size 2021-04-04 16:30:48 +03:00
Maksim Kita
41ef9291f1 Fixed tests 2021-04-04 16:30:48 +03:00
Maksim Kita
ff86c21e65 Dictionary update field fix 2021-04-04 16:30:48 +03:00
alesapin
76d0e60e26 Merge branch 'master' into jepsen_for_ci 2021-04-04 16:07:53 +03:00
alexey-milovidov
0e1035735c
Merge pull request #22565 from ClickHouse/fix-assert-arena 
Fix assert in Arena when doing GROUP BY Array of Nothing of non-zero size.
 2021-04-04 15:38:17 +03:00
alexey-milovidov
72cf7b01e8
Merge pull request #22571 from mastertheknife/allow_unbundled_xz_lzma 
Allow building with unbundled xz
 2021-04-04 15:36:45 +03:00
Alexey Milovidov
32f9a407ff More precise and safe accurate comparison of int vs. float 2021-04-04 14:49:48 +03:00
Vladimir Chebotarev
0eb051961c Consistent AWS timeouts. 2021-04-04 14:06:10 +03:00
Maksim Kita
0bfb429c42
Merge pull request #22479 from kitaisreal/clickhouse-dictionary-source-loop-fix 
ClickHouseDictionarySource loop fix
 2021-04-04 13:29:17 +03:00
Azat Khuzhin
44abb7588a StorageStripeLog: avoid UB for rwlock unlock due to unlock from another thread 2021-04-04 13:25:44 +03:00
Azat Khuzhin
89a96cf14d StorageLog: avoid UB for rwlock unlock due to unlock from another thread 2021-04-04 13:25:44 +03:00
Azat Khuzhin
5d795661ed StorageTinyLog: add regression test for UB due to unlock from different thread 
Refs: #22560
 2021-04-04 13:25:44 +03:00
Azat Khuzhin
5ced6e2e9a StorageStripeLog: drop mutable qualifier for rwlock 2021-04-04 13:25:43 +03:00
Azat Khuzhin
5ef845af40 StorageLog: drop mutable qualifier for rwlock 2021-04-04 13:25:43 +03:00
Alexey Milovidov
d67ced585a Fix engrish 2021-04-04 13:09:39 +03:00
fuqi
043bd11d2e fix dist map type select 2021-04-04 18:00:30 +08:00
Nicolae Vartolomei
49abdeb967 Try flush write buffer only if it is initialized 
Fixes #22579
 2021-04-04 12:59:51 +03:00
Alexey Milovidov
54caa61af5 Fix comments 2021-04-04 12:33:06 +03:00
fuqi
b590926c1c fix dist map type select 2021-04-04 17:23:47 +08:00
Alexey Milovidov
8b5d0a598f Minor improvement in index deserialization 2021-04-04 12:17:54 +03:00
alesapin
548ec6d2cb Trying enabled setting 2021-04-04 11:02:09 +03:00
Alexey Milovidov
a0f2b85cd3 Make round function to behave consistently on non-x86_64 2021-04-04 05:21:56 +03:00
alexey-milovidov
b35ba1152f
Update IColumnDummy.h 2021-04-04 05:04:11 +03:00
Maksim Kita
b9e8ff137a Fixed build 2021-04-04 01:19:57 +03:00
mastertheknife
65ca073f42 Allow unbundled xz (lzma) using USE_INTERNAL_XZ_LIBRARY=OFF 2021-04-04 01:05:07 +03:00
alexey-milovidov
cb4e119c3d
Merge pull request #22561 from traceon/apple-clang-compilation-fix 
AppleClang compilation fix
 2021-04-04 00:25:58 +03:00
Alexey Milovidov
3050486308 Fix assert in Arena 2021-04-03 22:50:49 +03:00
alexey-milovidov
67fdfd4b89
Merge pull request #22560 from azat/tiny-log-rwlock-ub-fix 
Fix UB by unlocking the rwlock of the TinyLog from the same thread
 2021-04-03 21:59:01 +03:00
Denis Glazachev
36b03b2740 Compilation fix 2021-04-03 22:17:51 +04:00
alesapin
1906c77884 Fix tidy one more time 2021-04-03 17:57:03 +03:00
Azat Khuzhin
6cebd3ee1f Fix UB by unlocking the rwlock of the TinyLog from the same thread 
Before this patch the build with libstdc++ hangs for
00967_insert_into_distributed_different_types test (and I guess some
others), this is because the rwlock is acquired from different thread as
it was unlocked which causes UB [1], fix this by moving unlock into
writeSuffix().

  [1]: The pthread_rwlock_unlock() function shall release a lock held on
       the read-write lock object referenced by rwlock. Results are undefined
       if the read-write lock rwlock is not held by the **calling thread**.
 2021-04-03 16:04:39 +03:00
Azat Khuzhin
10d35867e3 Drop mutable qualifier for StorageTinyLog::rwlock 2021-04-03 16:03:17 +03:00
alesapin
0204f5dd35 Merge branch 'master' into merge_tree_deduplication 2021-04-03 15:24:26 +03:00
alesapin
b127e31451 CI fixes 2021-04-03 14:56:56 +03:00
alexey-milovidov
30ece2c28f
Merge pull request #22510 from vdimir/merge-join-lc-bug-fix 
Convert right block to full in MergeJoin
 2021-04-03 12:54:34 +03:00
alexey-milovidov
143f4b59a0
Merge pull request #22527 from ClickHouse/fix-missing-nl-parse-tsv 
Fix deserialization of empty string without newline at end of TSV format
 2021-04-03 12:51:10 +03:00
alexey-milovidov
2de808ec5a
Update SerializationNullable.cpp 2021-04-03 12:50:57 +03:00
alexey-milovidov
687d1e9b54
Merge pull request #22528 from kitaisreal/format-settings-null-as-default-default-value-fix 
FormatSettings null_as_default default value fix
 2021-04-03 12:49:08 +03:00
kssenii
13d1f17d3e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-03 08:00:09 +00:00
alexey-milovidov
9f5cd35a69
Merge pull request #22323 from abyss7/poco-http2 
Follow up to #19516
 2021-04-03 02:09:10 +03:00
alexey-milovidov
3d860fc6ac
Merge pull request #22322 from abyss7/issue-21907 
Do not limit HTTP chunk size
 2021-04-03 01:58:22 +03:00
Maksim Kita
07d5a1ec21 ClickHouseDictionarySource loop fix 2021-04-03 00:40:07 +03:00
Maksim Kita
5ba6c7b731 FormatSettings null_as_default default value fix 2021-04-03 00:05:40 +03:00
Alexey Milovidov
7a4637c645 Fix deserialization of empty string without newline at end of TSV format 2021-04-02 23:54:03 +03:00
alexey-milovidov
c3a9cbe01c
Merge pull request #22421 from ClickHouse/fix-buffer-overflow-in-token-extractor 
Fix buffer overflow in token extractor
 2021-04-02 22:08:28 +03:00
alexey-milovidov
840ecb5e99
Update MergeJoin.cpp 2021-04-02 21:07:11 +03:00
alesapin
b884593186 Followup fix 2021-04-02 20:55:05 +03:00
alesapin
ca30c91868 More comments 2021-04-02 20:54:24 +03:00
alesapin
fc3afe1087 Fix typos 2021-04-02 20:51:57 +03:00
alesapin
b105153d2c Useful comment 2021-04-02 20:48:58 +03:00
alesapin
ab34873344 Add some comments 2021-04-02 20:46:01 +03:00
alesapin
a555d078a2 Add exception handling 2021-04-02 19:56:02 +03:00
Alexander Kuzmenkov
d0037329cf
Merge pull request #22459 from ClickHouse/aku/window-frame-offset 
fix window frame offset check and add more tests
 2021-04-02 19:49:59 +03:00
alesapin
759dd79c76 Add tests 2021-04-02 19:45:18 +03:00
kssenii
2de6b99f84 Better names 2021-04-02 16:12:14 +00:00
kssenii
02c6332e86 Pass null values properly 2021-04-02 18:45:42 +03:00
robot-clickhouse
39b5674bef Auto version update to [21.5.1.1] [54450] 2021-04-02 18:34:29 +03:00
Alexander Kuzmenkov
e7df8893df
Merge pull request #22300 from ClickHouse/trying_parallel_func_tests 
Trying parallel func tests
 2021-04-02 18:09:58 +03:00
vdimir
c8f51998ba
Remove LowCardinality from right block sample in MergeJoin 2021-04-02 17:51:02 +03:00
tavplubix
93bd2c3b67
Merge pull request #18978 from myrrc/feature/attach-partition-local 
Lookup parts/partitions in replica's own detached/ folder before downloading them from other replicas
 2021-04-02 17:46:03 +03:00
Nikolai Kochetov
0421460943 Always call even.set for zk client callbacks. 2021-04-02 17:06:53 +03:00
vdimir
8cd841c4ac
Handle lowcardinality block in MergeJoin extractMinMax 2021-04-02 16:55:42 +03:00
alesapin
6f36661575 Buildable code 2021-04-02 16:34:36 +03:00
tavplubix
c1ab9de720
Update MergeTreeDataPartChecksum.cpp 2021-04-02 15:57:30 +03:00
alesapin
2ab87d61eb Some build fixes 2021-04-02 15:37:42 +03:00
Alexander Kuzmenkov
16c5658df6 Merge remote-tracking branch 'origin/master' into HEAD 2021-04-02 15:16:24 +03:00
Anton Popov
a000d6154c fix clang-tidy 2021-04-02 15:10:49 +03:00
alesapin
14634689e5 Idea code 2021-04-02 14:46:42 +03:00
hexiaoting
6e8583a028 Add array size check for Nested Type subcolumns when Update 2021-04-02 19:31:33 +08:00
Ivan Lezhankin
8a455d347a Fix build 2021-04-02 14:08:32 +03:00
Pavel Kruglov
dff7834af0 Fix config comparison 2021-04-02 13:17:28 +03:00
Kruglov Pavel
6fb70cfd3f
Merge pull request #22443 from Avogar/hedged 
Remove socket from epoll before cancelling packet receiver in HedgedConnections to prevent possible race
 2021-04-02 11:49:21 +03:00
Vladimir
9b40f384e9
Merge pull request #20622 from vdimir/issue-20497 2021-04-02 11:48:29 +03:00
Maksim Kita
3b799c5d7d
Merge pull request #22413 from kitaisreal/added-function-dict-get-or-null 
Added function dictGetOrNull
 2021-04-02 11:39:01 +03:00
Maksim Kita
d06fb1197a
Merge pull request #22096 from kitaisreal/hierarchy-dictionaries-updated 
Refactored hierarchy dictionaries interface
 2021-04-02 11:38:01 +03:00
Kseniia Sumarokova
b80951b06a
Merge pull request #21519 from fastio/aliases_to_visitParam 
add aliases to visitParam/visitParamExtract* functions
 2021-04-02 10:31:32 +03:00
Anton Popov
56320bcbfd minor refactoring 2021-04-02 01:16:54 +03:00
Anton Popov
88cd775f6a Merge remote-tracking branch 'upstream/master' into HEAD 2021-04-02 00:14:03 +03:00
kssenii
b66e0401f6 Lowercase 2021-04-01 23:07:01 +03:00
alesapin
4eaf585fe5 Almost nothing added 2021-04-01 22:49:51 +03:00