thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-19 06:01:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
58,890 Commits 9,136 Branches 1,778 Tags 2 GiB
705c4dd60c
Commit Graph

11147 Commits

Author SHA1 Message Date
alesapin
4e175bd988 Useful comments 2021-03-03 19:27:08 +03:00
alesapin
e6083c4dc4 Fix style 2021-03-03 19:25:47 +03:00
alesapin
eb5e416fd5 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-03 19:19:02 +03:00
alesapin
7e28bfeb9e Fix log store start 2021-03-03 18:37:31 +03:00
Pavel Kruglov
dc9dd8ccdc Fix typo 2021-03-03 16:16:17 +03:00
alesapin
186b39f0d5 Followup fix 2021-03-03 15:31:21 +03:00
alesapin
0136d7d51a Get rid of unused field 2021-03-03 15:29:00 +03:00
alesapin
37fc5faa6f Fix nasty serialization bug 2021-03-03 15:21:21 +03:00
Vladimir Chebotarev
4022418ff3 Case-insensitive compression methods. 2021-03-03 14:51:09 +03:00
alesapin
1707e7f1c3 Rename file 2021-03-03 14:22:38 +03:00
alesapin
e615299ae6 Fix some bugs and add snapshots validation 2021-03-03 14:10:24 +03:00
Nikolai Kochetov
cb122160ae
Merge pull request #19291 from Avogar/hedged-requests 
Implementation of HedgedRequests
 2021-03-03 12:58:37 +03:00
feng lv
742e5a05ff fix conflict 2021-03-03 09:38:30 +00:00
alesapin
07e39ef47a Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-03 11:12:01 +03:00
alesapin
6b1005aea5
Merge pull request #21334 from ClickHouse/fix_alter_partition_key 
Fix alter modify query for partition key and other metadata fields
 2021-03-03 10:15:11 +03:00
alexey-milovidov
bd7b540b82
Merge pull request #21307 from amosbird/moduleopt 
Specialize intDiv/module
 2021-03-03 04:49:29 +03:00
Anton Popov
a4c00ab5dc
Merge pull request #21303 from ucasFL/forbid 
Forbid to drop a column if it's referenced by materialized view
 2021-03-03 02:55:06 +03:00
alexey-milovidov
33bb77f777
Merge pull request #21367 from amosbird/mysqlfix 
mysql client compatibility
 2021-03-03 02:13:58 +03:00
alexey-milovidov
6e5638cee8
Merge pull request #21366 from songenjie/shutdown_timeout_log 
[ClickHouse][LOG]correct shutdown timeout log
 2021-03-03 01:55:15 +03:00
alexey-milovidov
c78256ad65
Merge pull request #21327 from amosbird/hasequalvalue 
IColunm::hasEqualValues()
 2021-03-03 01:19:26 +03:00
Pavel Kruglov
82a6be44cf Disable changing replica when we have made progress 2021-03-03 01:03:03 +03:00
Kruglov Pavel
fbe0a14a2f
Merge pull request #21035 from anmolarora/system-parts-requested-columns-fill 
Fill only requested columns when querying system.parts & system.parts…
 2021-03-03 00:39:45 +03:00
Nikolai Kochetov
8e6fa404c4
Merge pull request #21246 from ucasFL/union-distinct-improve 
Improve Normalization of ASTSelectWithUnionQuery
 2021-03-02 21:37:07 +03:00
alesapin
c29d7c7f49 Shutup clang tidy 2021-03-02 19:13:36 +03:00
alesapin
ee185bc536 Start from committed plus 1 2021-03-02 18:58:02 +03:00
alesapin
346d2b1d27 Deepbugging 2021-03-02 18:19:05 +03:00
Amos Bird
9411d496d9
Fix build 2021-03-02 23:17:02 +08:00
alesapin
8e6252b25f Use snapshot object in serialization 2021-03-02 17:30:56 +03:00
Amos Bird
134216dc4a Specialize intDiv/module 2021-03-02 16:27:28 +03:00
alesapin
10e16e39ea Written synchronous snapshots 2021-03-02 16:18:04 +03:00
Pavel Kruglov
4166ae1f45 Restart timeout when we make progress 2021-03-02 15:40:24 +03:00
alesapin
48bf55aef9 Even better tests 2021-03-02 15:37:00 +03:00
alesapin
08ccae32f3 Fix nasty bug 2021-03-02 15:34:18 +03:00
Amos Bird
494c2dba6c
Fix error 2021-03-02 19:12:51 +08:00
alesapin
e7399bf66f Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-02 14:03:42 +03:00
alesapin
4e33587043 Comments 2021-03-02 13:57:09 +03:00
Amos Bird
ad4fd75fb4
mycli compatibility 2021-03-02 18:53:06 +08:00
Nikolai Kochetov
ffbea93b10
Merge pull request #19576 from traceon/row-policy-with-prewhere 
Allow row policies with PREWHERE
 2021-03-02 13:51:37 +03:00
alesapin
06678d650d Merge branch 'master' into fix_alter_partition_key 2021-03-02 13:43:41 +03:00
alesapin
9ebf1b4fad Get rid of separate minmax index fields 2021-03-02 13:33:54 +03:00
tavplubix
ee22eeea6c
Merge pull request #21357 from ClickHouse/fix-21140 
Fix inconsistent column type for Values input format.
 2021-03-02 13:12:12 +03:00
Nikolai Kochetov
59a2c45555
Update executeQuery.cpp 2021-03-02 13:09:29 +03:00
Amos Bird
3df57da2d8
mysql client compatibility 2021-03-02 17:06:40 +08:00
Pavel Kruglov
153bfbfc28 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-03-02 11:59:32 +03:00
tavplubix
d305b23338
Merge pull request #21079 from azat/distributed_ddl_pool_size-zk-fix 
Fix various issues in DDLWorker (SIGSEGV and others)
 2021-03-02 11:59:30 +03:00
Pavel Kruglov
780a65e3a0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into system-parts-requested-columns-fill 2021-03-02 10:29:26 +03:00
turbo jason
3a6307a990 [ClickHouse][LOG]correct shutdown timeout log 2021-03-02 15:13:54 +08:00
Amos Bird
c41dedc922
optimize for ColumnLowCardinality 2021-03-02 11:55:58 +08:00
alexey-milovidov
32c011052f
Update IColumnImpl.h 2021-03-02 11:55:58 +08:00
alexey-milovidov
220d3cae7f
Update IColumn.h 2021-03-02 11:55:58 +08:00
Amos Bird
62e8684ad1
IColunm::hasEqualValues() 2021-03-02 11:55:58 +08:00
feng lv
a26c9e64a9 fix 
fix
 2021-03-02 03:20:03 +00:00
Nikolai Kochetov
f82e00ff3c Fix inconsistent column type for Values input format. 2021-03-01 23:10:50 +03:00
Maksim Kita
ca98bb68d4
Merge pull request #21353 from kitaisreal/aggregate-function-sum-map-better-comment 
AggregateFunctionSumMap better comment message
 2021-03-01 19:19:07 +03:00
Maksim Kita
7ed5900251
Updated style 2021-03-01 19:18:56 +03:00
Maksim Kita
019a2090c7
Fix unused variable 2021-03-01 19:18:14 +03:00
Maksim Kita
3764a2a2be AggregateFunctionSumMap better comment message 2021-03-01 19:15:59 +03:00
alesapin
40c7455d33 Test for snapshot removal 2021-03-01 19:02:15 +03:00
alesapin
321dd3e76b Fix bug 2021-03-01 18:32:27 +03:00
alesapin
b7b6fd7cb3 Max snapshots on disk 2021-03-01 17:54:08 +03:00
alesapin
73d3c20554 Test simple serde 2021-03-01 17:40:32 +03:00
tavplubix
10aa26576c
Update DatabaseReplicatedWorker.cpp 2021-03-01 17:11:45 +03:00
alesapin
03960b1eed Some compileable code 2021-03-01 16:33:34 +03:00
Anton Popov
c98e6bb1ad
Merge pull request #20550 from ClickHouse/fix-number-of-threads-in-subqueries 
Fix number of threads in subqueries
 2021-03-01 15:49:38 +03:00
Kruglov Pavel
caaa0c5e86
Restart tests. 2021-03-01 13:25:20 +03:00
alesapin
91bc4478d7 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-01 13:18:35 +03:00
alesapin
9c8afbeb53 Fix alter modify query for partition key and other metadata fields 2021-03-01 12:59:19 +03:00
Nikolai Kochetov
976dbe8077
Merge pull request #20341 from ClickHouse/filter-push-down 
Filter push down
 2021-03-01 12:35:06 +03:00
vdimir
b7c7c97d10
Merge branch 'master' into join-cast-types-v2 2021-03-01 11:42:21 +03:00
Nikolai Kochetov
eb42896e26 Merge branch 'master' into ucasFL-union-distinct-improve 2021-03-01 10:43:39 +03:00
alesapin
11f2a271a2 Remove useless unit test 2021-03-01 10:40:00 +03:00
alexey-milovidov
174f49a9f1
Merge pull request #21308 from fuwhu/refine-wal-restore-code 
remove unused code in MergeTreeWriteAheadLog::restore
 2021-03-01 01:44:23 +03:00
alexey-milovidov
81be2d1ed9
Merge pull request #21305 from azat/terminate-nested-writers 
Fix abnormal server termination for nested writers
 2021-02-28 21:54:13 +03:00
alexey-milovidov
b924b28c57
Merge pull request #21306 from azat/unused-writers 
[RFC] Remove unused writers
 2021-02-28 21:47:01 +03:00
Denis Glazachev
eb80c189e4 Merge branch 'master' into row-policy-with-prewhere 2021-02-28 20:14:31 +04:00
fuwhu
2df33be7c2 Remove unused code in MergeTreeWriteAheadLog::restore 2021-02-28 23:17:31 +08:00
Azat Khuzhin
eb0387c5a9 Fix abnormal server termination for nested writers 
Writers with nested writer can call next() from the dtor for nested
writer and this will cause exception again, so the buffer position
should be updated on exceptions.

Found by stress test (thread) here [1] and here [2]:

    2021.02.27 19:27:53.498977 [ 302 ] {} <Fatal> BaseDaemon: (version 21.3.1.6130, build id: 2DAEC5DEBF03C5A1C3BF66B7779C886F16239345) (from thread 1614) Terminate called for uncaught exception:
    Code: 24, e.displayText() = DB::Exception: Cannot write to ostream at offset 2097498, Stack trace (when copying this message, always include the lines below):

    0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:0: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x15bef2ab in /usr/bin/clickhouse
    1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:56: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8aea92e in /usr/bin/clickhouse
    2. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromOStream.cpp:0: DB::WriteBufferFromOStream::nextImpl() @ 0x8bbbc45 in /usr/bin/clickhouse
    3. ./obj-x86_64-linux-gnu/../src/IO/BufferBase.h:39: DB::WriteBufferFromOStream::~WriteBufferFromOStream() @ 0x8bbc077 in /usr/bin/clickhouse
    4. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromOStream.cpp:44: DB::WriteBufferFromOStream::~WriteBufferFromOStream() @ 0x8bbc18a in /usr/bin/clickhouse
    5. ./obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:137: DB::ZstdDeflatingWriteBuffer::~ZstdDeflatingWriteBuffer() @ 0x118bdc29 in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/IO/ZstdDeflatingWriteBuffer.cpp:32: DB::ZstdDeflatingWriteBuffer::~ZstdDeflatingWriteBuffer() @ 0x118be3ea in /usr/bin/clickhouse
    7. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:0: DB::WriteBufferFromHTTPServerResponse::finalize() @ 0x12f1dceb in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Server/HTTPHandler.cpp:703: DB::HTTPHandler::trySendExceptionToClient(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, DB::HTTPServerRequest&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&) @ 0x12e9fecc in /usr/bin/clickhouse
    9. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:1444: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x12ea0d60 in /usr/bin/clickhouse
    10. ./obj-x86_64-linux-gnu/../src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x12f16db1 in /usr/bin/clickhouse
    11. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x15b184f3 in /usr/bin/clickhouse
    12. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:0: Poco::Net::TCPServerDispatcher::run() @ 0x15b18c1f in /usr/bin/clickhouse
    13. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/ScopedLock.h:36: Poco::PooledThread::run() @ 0x15c7fdb2 in /usr/bin/clickhouse
    14. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:56: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x15c7e350 in /usr/bin/clickhouse
    15. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/SharedPtr.h:277: Poco::ThreadImpl::runnableEntry(void*) @ 0x15c7cb58 in /usr/bin/clickhouse
    16. __tsan_thread_start_func @ 0x8a04ced in /usr/bin/clickhouse
    17. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    18. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
     (version 21.3.1.6130)

  [1]: https://clickhouse-test-reports.s3.yandex.net/21279/4f61ef3099f42f17b496a0b0424773978d9a32dc/stress_test_(thread).html#fail1
  [2]: https://clickhouse-test-reports.s3.yandex.net/21292/ae9fea1d0af118a8f87b224d194d61da1567188b/stress_test_(thread).html#fail1

v2: https://clickhouse-test-reports.s3.yandex.net/21305/e969daa6e86c5e09cfef08cfde19712982b64e59/stress_test_(thread).html#fail1
 2021-02-28 15:13:42 +03:00
feng lv
cbeb2853fb improve clickhouse-format 
fix

fix error word
 2021-02-28 11:34:25 +00:00
Azat Khuzhin
568a49dad5 Remove unused AsynchronousWriteBuffer 2021-02-28 11:19:56 +03:00
Azat Khuzhin
8b58dba094 Drop unused HexWriteBuffer 2021-02-28 10:52:09 +03:00
feng lv
51021c1164 forbid to drop a column if it's referenced by materialized view 2021-02-28 05:24:39 +00:00
Azat Khuzhin
7d51ae3212 DDLWorker: Fix processing of tasks in parallel (correct queue begin) 
Otherwise it will:
- start from incorrect task and bail because finished node already
  exists
- and also process alreayd processed items
 2021-02-28 05:45:12 +03:00
Azat Khuzhin
955974a8b5 DDLWorker: avoid NULL dereference on termination and failed zookeeper initialization 
Log snipped shows the problem:

    2021.02.24 04:40:29.349181 [ 39 ] {} <Warning> DDLWorker: DDLWorker is configured to use multiple threads. It's not recommended because queries can be reordered. Also it may cause some unknown issues to appear.
    2021.02.24 04:40:29.349516 [ 39 ] {} <Information> Application: Ready for connections.
    2021.02.24 04:40:29.349602 [ 74 ] {} <Debug> DDLWorker: Started DDLWorker cleanup thread
    2021.02.24 04:40:29.349639 [ 73 ] {} <Debug> DDLWorker: Starting DDLWorker thread
    2021.02.24 04:40:29.349698 [ 73 ] {} <Debug> DDLWorker: Started DDLWorker thread
    2021.02.24 04:40:29.352548 [ 73 ] {} <Error> virtual void DB::DDLWorker::initializeMainThread(): Code: 999, e.displayText() = Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
     (Connection loss), Stack trace (when copying this message, always include the lines below):

    0. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0xfe93923 in /usr/bin/clickhouse
    1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0xfe93ba2 in /usr/bin/clickhouse
    2. Coordination::ZooKeeper::connect(std::__1::vector<Coordination::ZooKeeper::Node, std::__1::allocator<Coordination::ZooKeeper::Node> > const&, Poco::Timespan) @ 0xfed3a01 in /usr/bin/clickhouse
    3. Coordination::ZooKeeper::ZooKeeper(std::__1::vector<Coordination::ZooKeeper::Node, std::__1::allocator<Coordination::ZooKeeper::Node> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Poco::Timespan, Poco::Timespan, Poco::Timespan) @ 0xfed2222 in /usr/bin/clickhouse
    4. zkutil::ZooKeeper::init(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfe961cd in /usr/bin/clickhouse
    5. zkutil::ZooKeeper::ZooKeeper(Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfe97a96 in /usr/bin/clickhouse
    6. void std::__1::allocator_traits<std::__1::allocator<zkutil::ZooKeeper> >::__construct<zkutil::ZooKeeper, Poco::Util::AbstractConfiguration const&, char const (&) [10]>(std::__1::integral_constant<bool, true>, std::__1::allocator<zkutil::ZooKeeper>&, zkutil::ZooKeeper*, Poco::Util::AbstractConfiguration const&, char const (&) [10]) @ 0xed98387 in /usr/bin/clickhouse
    7. DB::Context::getZooKeeper() const @ 0xed75190 in /usr/bin/clickhouse
    8. DB::DDLWorker::getAndSetZooKeeper() @ 0xedb81c9 in /usr/bin/clickhouse
    9. DB::DDLWorker::initializeMainThread() @ 0xedc9eb0 in /usr/bin/clickhouse
    10. DB::DDLWorker::runMainThread() @ 0xedb5d01 in /usr/bin/clickhouse
    11. ThreadFromGlobalPool::ThreadFromGlobalPool<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()::operator()() @ 0xedcafa1 in /usr/bin/clickhouse
    12. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x892651f in /usr/bin/clickhouse
    13. ? @ 0x8929fb3 in /usr/bin/clickhouse
    14. start_thread @ 0x8ea7 in /lib/x86_64-linux-gnu/libpthread-2.31.so
    15. __clone @ 0xfddef in /lib/x86_64-linux-gnu/libc-2.31.so
     (version 21.3.1.1)
    ...
    2021.02.24 04:40:30.025278 [ 41 ] {} <Trace> BaseDaemon: Received signal 15
    2021.02.24 04:40:30.025336 [ 41 ] {} <Information> Application: Received termination signal (Terminated)
    ...
    2021.02.24 04:40:30.582078 [ 39 ] {} <Information> Application: Closed all listening sockets.
    2021.02.24 04:40:30.582124 [ 39 ] {} <Information> Application: Closed connections.
    2021.02.24 04:40:30.583770 [ 39 ] {} <Information> Application: Shutting down storages.
    2021.02.24 04:40:30.583932 [ 39 ] {} <Information> Context: Shutdown disk data
    2021.02.24 04:40:30.583951 [ 39 ] {} <Information> Context: Shutdown disk default
    2021.02.24 04:40:30.584163 [ 46 ] {} <Trace> SystemLog (system.query_log): Terminating
    2021.02.24 04:40:30.586025 [ 39 ] {} <Trace> BackgroundSchedulePool/BgSchPool: Waiting for threads to finish.
    2021.02.24 04:40:34.352701 [ 73 ] {} <Debug> DDLWorker: Initialized DDLWorker thread
    2021.02.24 04:40:34.352758 [ 73 ] {} <Debug> DDLWorker: Scheduling tasks
 2021-02-28 05:45:12 +03:00
Azat Khuzhin
3c7e765b27 DDLWorker: process unfinished tasks only after reinitialization 2021-02-28 05:45:12 +03:00
Azat Khuzhin
54b1496408 DDLWorker: optimize processing of unfinished tasks 
Do not look at zookeeper if was_executed is false
 2021-02-28 05:45:12 +03:00
Azat Khuzhin
eeda1fe756 DDLWorker: wait for pending async tasks 2021-02-28 05:45:12 +03:00
Azat Khuzhin
0cd67ed051 DDLWorker: remove only completed tasks (significant for distributed_ddl.pool_size > 1) 
Otherwise it will SIGSEGV due to invalid-read (UAF).
 2021-02-28 05:45:12 +03:00
Azat Khuzhin
7481b05d27 DDLWorker: avoid comparing pointers for getting min element 
The current_tasks always updated with emplace_back, so front() is ok.
 2021-02-28 05:45:11 +03:00
Azat Khuzhin
78c6e0527d DDLWorker: Fix reference to zookeeper with distributed_ddl.pool_size > 1 (thread pool) 2021-02-28 05:45:11 +03:00
Azat Khuzhin
f7dbcfc12a DDLTask: Use std::uncaught_exceptions() over deprecated std::uncaught_exception() 2021-02-28 05:45:11 +03:00
Anmol Arora
2d9b524bdd Fill only requested columns when querying system.parts & system.parts_columns 2021-02-28 01:58:38 +00:00
Alexey Milovidov
ae9fea1d0a Fix gcc and clang-tidy 2021-02-28 04:05:04 +03:00
Alexey Milovidov
107d8ec811 Merge branch 'master' into extract-text-from-html 2021-02-28 04:03:32 +03:00
Alexey Milovidov
e5ae9cbb63 Fix Arcadia 2021-02-28 04:03:22 +03:00
alexey-milovidov
b8fba768e5
Merge pull request #21264 from ClickHouse/fix_zookeeper_update 
Fix several bugs with ZooKeeper client
 2021-02-28 01:57:04 +03:00
Alexey Milovidov
4ab18cdcd8 Tests and documentation 2021-02-28 00:59:27 +03:00
Alexey Milovidov
2ac673b12a Update logic and tests 2021-02-27 22:33:55 +03:00
Alexey Milovidov
fd1cf49e92 Rewrite extractTextFromHTML function 2021-02-27 22:22:38 +03:00
Maksim Kita
315824978d CheckConstraintsBlockOutputStream optimize nullable column case 2021-02-27 19:19:21 +03:00
alexey-milovidov
56af09891b
Merge pull request #20991 from kssenii/brotli 
Fix Brotli http compression issue
 2021-02-27 16:39:49 +03:00
alesapin
9dc3952ff3 Fix typo 2021-02-27 16:30:43 +03:00
Maksim Kita
9996f68160
Merge pull request #20990 from kitaisreal/constraints-complex-type-support 
Constraints complex types support
 2021-02-27 13:16:29 +03:00
Pavel Kruglov
1b1f50e324 Update comment 2021-02-27 11:45:32 +03:00
Pavel Kruglov
4d7ba0443f Add checking pending data 2021-02-27 11:40:03 +03:00
Vladimir
df728dc4ba
Merge pull request #21248 from vdimir/fix-join-totals-nullable 
Convert columns to nullable for totals in join
 2021-02-27 11:31:02 +03:00
Alexander Kazakov
63b95c7451
MySQL dictionary source: A mechanism for retrying unexpected connection loss during communication with mysql server (#21237) 
* Added a new type of Exception

for better recognition of connection failures

* Added more logging for mysql communication

* Retries on connection loss during query.

Make MySQL dictionary source resilient to unexpected loss of connection
during querying MySQL. This behavior is controlled with
".fail_on_connection_loss" config parameter, which defaults to false.

* Stripped some debugging leftover garbage

* Minor followup corrections

* Corrections after PR comments

* Yet more fixes
 2021-02-27 11:18:28 +03:00
alesapin
9e93d7f507 Fix tidy and add comments 2021-02-27 11:07:14 +03:00
feng lv
4ac5b62f19 Merge branch 'master' of github.com:ClickHouse/ClickHouse into union-distinct-improve 2021-02-27 08:02:41 +00:00
Pavel Kruglov
197058dc12 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-27 00:36:33 +03:00
alesapin
184e9b9424
Merge pull request #21207 from ClickHouse/better_nukeeper_for_tests 
Trying to make nukeeper better in single server mode
 2021-02-26 23:59:41 +03:00
alesapin
a48bb7e3a0 Fix redundant zookeeper reconnect and possibility of multiple zookeeper session for a single ClickHouse server 2021-02-26 22:05:25 +03:00
Kruglov Pavel
cb638e0f87
Fix style 2021-02-26 21:06:17 +03:00
Nikolai Kochetov
d91b8a3acb
Merge branch 'master' into filter-push-down 2021-02-26 19:33:27 +03:00
Nikolai Kochetov
b082b661f7 Fix ya.make 2021-02-26 19:32:12 +03:00
Nikolai Kochetov
d328bfa41f Review fixes. Add setting max_optimizations_to_apply. 2021-02-26 19:29:56 +03:00
Pavel Kruglov
91f6f9227b Small fixes and tests updates 2021-02-26 18:55:58 +03:00
alesapin
83e2f1ca64 More fixes 2021-02-26 17:54:59 +03:00
kssenii
991718d459 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into brotli 2021-02-26 14:31:24 +00:00
tavplubix
09f62447af
Merge pull request #21060 from mxzlxy/fix-bug-for-replicated-engine-parameter 
fix default_replica_path and default_replica_name values are useless …
 2021-02-26 17:18:56 +03:00
alesapin
12f74d4e4a Fix build 2021-02-26 16:56:57 +03:00
alesapin
8130726794 Move storage to new map 2021-02-26 16:53:34 +03:00
vdimir
932286df1f
Convert columns to nullable for totals in join 2021-02-26 16:32:34 +03:00
feng lv
4c30c10092 add test 
fix
 2021-02-26 12:14:57 +00:00
Vladimir
78024ae2d6
Merge pull request #21009 from vdimir/fix-race-storage-join 2021-02-26 14:26:10 +03:00
Nikolai Kochetov
f63d7b45ce Merge branch 'master' into fix-number-of-threads-in-subqueries 2021-02-26 11:15:03 +03:00
Maksim Kita
749dd8d555
Merge pull request #21193 from kitaisreal/storage-rabbit-mq-added-uvloop 
StorageRabbitMQ added UVLoop
 2021-02-26 11:04:02 +03:00
alesapin
56d9d99d1c Fix style and bayan 2021-02-26 09:59:40 +03:00
alesapin
5b72cd231f
Merge pull request #21183 from ClickHouse/fix_mutation_for_storages 
Restrict mutations for engines which doesn't support them
 2021-02-26 09:54:09 +03:00
alesapin
84102a22e7 Trying to make testkeeper better in single server mode 2021-02-25 23:30:55 +03:00
alesapin
15face72c5 Stl containers 2021-02-25 22:52:22 +03:00
kssenii
1247c0f36c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into rabbit-fix 2021-02-25 16:49:29 +00:00
alexey-milovidov
6a5ab40988
Merge pull request #21119 from fastio/bugfix_drop_metadata_from_auxiliary 
fix the metadata leak when the Replicated*MergeTree with custom ZooKeeper cluster is dropped
 2021-02-25 19:24:58 +03:00
vdimir
9c1ba71416
Merge remote-tracking branch 'upstream/master' into fix-race-storage-join 2021-02-25 19:13:49 +03:00
alexey-milovidov
0067a3beac
Merge pull request #19801 from ucasFL/ms 
continue of #19487
 2021-02-25 19:12:47 +03:00
Maksim Kita
63173eebdb
Merge pull request #21088 from azat/__has_feature-gcc-fix 
Fix __has_feature macros under gcc
 2021-02-25 18:34:14 +03:00
alesapin
653d962cdd Fix style and add low-level check 2021-02-25 17:43:58 +03:00
Maksim Kita
29e783999c Fixed typo 2021-02-25 17:43:32 +03:00
alesapin
356e548267 Experimenting 2021-02-25 17:23:12 +03:00
alesapin
cf42824756 Experimenting 2021-02-25 17:22:51 +03:00
Maksim Kita
f077a25a7a StorageRabbitMQ added UVLoop 2021-02-25 17:02:34 +03:00
vdimir
4c6cde4d53
Merge remote-tracking branch 'upstream/master' into join-cast-types-v2 2021-02-25 16:24:28 +03:00
Maksim Kita
da321c2bfe Fixed check for null value in null map 2021-02-25 16:08:04 +03:00
Maksim Kita
23af53067d Updated support for Nullable column 2021-02-25 14:27:46 +03:00
Maksim Kita
2eec1d021b Fixed unused code 2021-02-25 14:27:46 +03:00
Maksim Kita
8fec34af12 Constraints complex types support 2021-02-25 14:27:46 +03:00
vdimir
6f864ddb42
Minor changes in Hash/StorageJoin, add TableLockHolder to ExecutableFunctionJoinGet 2021-02-25 14:21:06 +03:00
Nikolai Kochetov
33364f6bdf Remove debug output 2021-02-25 13:50:35 +03:00
alesapin
5c6c318737 Restrict mutations for engines which doesn't support them 2021-02-25 13:07:48 +03:00
vdimir
f1c60109f6
Add static_assert to joinGet to keep information about current locking logic 2021-02-25 12:42:20 +03:00
vdimir
2f70e895fc
Update StorageJoin locking 
Move joinGet into StorageJoin

Protect JoinSource with lock, add test

Add comments about locking logic
 2021-02-25 12:31:22 +03:00
alesapin
c4a6289a9b Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-02-25 11:39:52 +03:00
alesapin
f8b81eeeb0 Get rid of own zxid 2021-02-25 11:34:05 +03:00
feng lv
731faeadbf union distinct improve 2021-02-25 07:47:08 +00:00
Vladimir
36c178c023
Merge pull request #21117 from amosbird/joingetlc 2021-02-25 10:06:57 +03:00
alesapin
8f81dce32f
Merge pull request #20585 from ClickHouse/persistent_nukeeper_log_storage 
Persistent coordination log storage
 2021-02-25 10:06:03 +03:00
fastio
fea2836673 get zookeeper from global context 2021-02-25 14:19:58 +08:00
fastio
3ddb729e4a fix metadata leak when drop Replicated*MergeTree 2021-02-25 14:19:58 +08:00
Vitaly Baranov
8601b2f3e7
Merge pull request #21148 from vitlibar/protobuf-format-better-tests-2 
Better tests for protobuf format #2.
 2021-02-25 06:01:47 +03:00
Nikolai Kochetov
6f5d4ba8cd Fix tests. 2021-02-25 00:31:15 +03:00
Nikolai Kochetov
aa8632a1bc Fix tests. 2021-02-25 00:27:47 +03:00
alexey-milovidov
b20efdcc01
Merge pull request #21020 from azat/trace-collector-non-recursive 
Fix buffer size for trace collection
 2021-02-24 21:57:30 +03:00
alexey-milovidov
89884797ab
Merge pull request #20879 from vitlibar/reservoir-sample-deterministic-ubsan-fix 
Fix undefined-behavior in ReservoirSamplerDeterministic.h
 2021-02-24 21:46:42 +03:00
Vitaly Baranov
2480e4ee3d Better tests for protobuf format #2. 2021-02-24 21:06:29 +03:00
kssenii
6eaf9281b5 Review fix 2021-02-24 16:52:14 +00:00
Maksim Kita
ae79fb22be
Merge pull request #21121 from amosbird/enumdefaultfix 
disable null_as_default when type is Nullable
 2021-02-24 19:31:30 +03:00
vdimir
6aa4791be2
Remove unused *Locked methods from HashJoin, add comments 2021-02-24 19:19:04 +03:00
alexey-milovidov
29a29f5060
Merge pull request #19765 from kssenii/fffix 
Fix #19371
 2021-02-24 17:40:15 +03:00
Maksim Kita
395526cb80
Merge pull request #21120 from hexiaoting/dev-map-bug 
Bug fix: Tuple cast to Map
 2021-02-24 17:21:00 +03:00
kssenii
3728480f04 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into rabbit-fix 2021-02-24 14:18:25 +00:00
Nikolai Kochetov
d966725f33 Try fix hung in void PullingAsyncPipelineExecutor::cancel() 2021-02-24 17:10:35 +03:00
Vitaly Baranov
92bf2cfb50
Merge pull request #20968 from vitlibar/protobuf-format-better-tests 
Better tests for protobuf format.
 2021-02-24 15:47:39 +03:00
Pavel Kruglov
24e9abddb8 Fix 2021-02-24 15:11:07 +03:00
alesapin
a1901383ae Improve performance and fix condition variable wait 2021-02-24 14:54:14 +03:00
kssenii
96f22b4290 Fix rabbit bug 2021-02-24 10:41:08 +00:00
alexey-milovidov
c1a077a111
Merge pull request #21098 from amosbird/re2fix 
Fix replace[All] functions so that they don't generate garbage to stderr
 2021-02-24 13:38:42 +03:00
Kseniia Sumarokova
de2ae9c73d
Merge pull request #20874 from kssenii/fix-postgres-numeric 
Better postgres db engine numeric conversion
 2021-02-24 12:28:00 +03:00
Maksim Kita
7bbc4e013a
Merge pull request #20541 from kitaisreal/replace-null-fields-in-tuple-during-parsing-with-default-values 
Replace null fields in tuple during parsing with default values
 2021-02-24 12:00:48 +03:00
Amos Bird
2cb273d436
Fix 2021-02-24 16:44:51 +08:00
alesapin
427e5f295c
Merge pull request #21007 from CurtizJ/alter-bad-conversions 
Do not start mutation for alters with wrong type conversion
 2021-02-24 10:08:15 +03:00
vdimir
c006b41afc
Lock before reuseJoinedData 2021-02-24 09:48:12 +03:00
Amos Bird
abf26d4935
disable null_as_default when type is Nullable 2021-02-24 14:45:14 +08:00
Amos Bird
0f7f2a4962
Support LC type for joinGet 2021-02-24 11:35:45 +08:00
Pavel Kruglov
5628fd6da5 Small update 2021-02-24 02:22:07 +03:00
alesapin
b0065334e2 Summing raft test more stable 2021-02-24 00:27:10 +03:00
Azat Khuzhin
810023a65c Do not undefine __has_feature (may break c++ headers) 
If the order of <common/defines.h> and c++ header will be wrong the
compilation will be broken.

v2: rename __ch_has_feature to ch_has_feature to fix -Wreserved-id-macro
v3: do not fallback to 0
 2021-02-23 19:46:47 +03:00
Amos Bird
acfbae7107
Fix replace,All functions so that they don't generate garbage to stderr 2021-02-23 22:39:05 +08:00
kssenii
6554965711 Fix build 2021-02-23 10:46:41 +00:00
hexiaoting
e123c6ef9d Bug fix: Tuple cast to Map 2021-02-23 18:42:33 +08:00
Kseniia Sumarokova
bf88d693d5
Update fetchPostgreSQLTableStructure.cpp 2021-02-23 11:59:38 +03:00
Kseniia Sumarokova
6c5781cf34
Update fetchPostgreSQLTableStructure.cpp 2021-02-23 11:48:51 +03:00
Kseniia Sumarokova
b2850ec94f
Merge branch 'master' into fix-postgres-numeric 2021-02-23 11:40:28 +03:00
Azat Khuzhin
2ced21c837 Decrease number of stored stack frame pointers to fit into 512 PIPE_BUF 2021-02-23 07:52:01 +03:00
Azat Khuzhin
6b39e61d09 Decrease QUERY_ID_MAX_LEN for freebsd/osx (PIPE_BUF=512) 2021-02-23 07:47:58 +03:00
tavplubix
4661073750
Update src/Storages/MergeTree/registerStorageMergeTree.cpp 2021-02-23 00:09:20 +03:00
alesapin
5b888e1193 Merge branch 'master' into persistent_nukeeper_log_storage 2021-02-22 23:52:26 +03:00
alesapin
d7c3dfe5fa Don't allow to start with broken changelogs 2021-02-22 23:51:58 +03:00
Maksim Kita
f274343cf0 Fixed Map type getExtremes 2021-02-22 23:32:43 +03:00
Maksim Kita
f001a6755e Updated for map 2021-02-22 23:32:43 +03:00
Maksim Kita
3b80395b1b Fixed style check 2021-02-22 23:32:43 +03:00
Maksim Kita
5eec754fd6 Use standard checkStackSize 2021-02-22 23:32:43 +03:00
Maksim Kita
992747e5ec Updated for Array 2021-02-22 23:32:43 +03:00
Maksim Kita
3f6050cee9 Fixed error code include 2021-02-22 23:32:43 +03:00
Maksim Kita
8520fc3e4c Replace null fields in tuple during parsing with default values 2021-02-22 23:32:43 +03:00
alexey-milovidov
cb8cfdba77
Merge branch 'master' into fffix 2021-02-22 23:08:44 +03:00
Azat Khuzhin
89c311981d Fix buffer size for trace collection 
[1]: https://clickhouse-test-reports.s3.yandex.net/0/a860e3e7582f489a640c42db3bd4bd320f97a5bc/stress_test_(debug).html#fail1
 2021-02-22 22:45:04 +03:00
Pavel Kruglov
edee592cb0 Fix 2021-02-22 20:57:44 +03:00
jennyma
6433ab3da6 fix default_replica_path and default_replica_name values are useless on Replicated(*)MergeTree engine when the engine needs specify other parameters 2021-02-22 18:34:01 +08:00
alexey-milovidov
073ee6c2c3
Merge pull request #21053 from ClickHouse/more-consistent-datetime64-parsing 
More consistent DateTime64 parsing
 2021-02-22 13:13:21 +03:00
alexey-milovidov
8b82c7ea0c
Merge pull request #21050 from azat/datetime64-float-fix 
Fix DateTime64 from Float
 2021-02-22 13:12:09 +03:00
alexey-milovidov
50639b7dba
Merge pull request #21052 from ClickHouse/file-engine-settings 
Merging #20620
 2021-02-22 13:10:31 +03:00
Pavel Kruglov
1dbab60235 Fix 2021-02-22 10:52:19 +03:00
Pavel Kruglov
6d8eb44c01 Fix PSV and remove debug logging 2021-02-22 00:59:07 +03:00
alesapin
1aa0c0bff0 More stable test 2021-02-21 23:36:05 +03:00
Alexey Milovidov
952cee9ec7 Merge remote-tracking branch 'azat/datetime64-float-fix' into more-consistent-datetime64-parsing 2021-02-21 23:24:38 +03:00
Alexey Milovidov
f7b7561552 Merge branch 'master' into more-consistent-datetime64-parsing 2021-02-21 23:24:07 +03:00
Alexey Milovidov
46ac365629 Merge branch 'master' into topvisor/file-engine-settings 2021-02-21 23:10:26 +03:00
Alexey Milovidov
0a62f95bbe Merge branch 'master' into distributed_aggregation_memory_efficient_by_default 2021-02-21 23:06:49 +03:00
Alexey Milovidov
d7f017c4dd Improve performance of trivial count query in presense of "distributed_aggregation_memory_efficient" 2021-02-21 23:06:44 +03:00
Alexey Milovidov
0ab14120ef Improve performance of trivial count query in presense of "distributed_aggregation_memory_efficient" 2021-02-21 23:06:31 +03:00
Azat Khuzhin
521537eed8 Fix DateTime64 from Float 2021-02-21 22:36:40 +03:00
alexey-milovidov
2ac5c4b71b
Merge pull request #20996 from azat/system-stack-trace-diagnostics 
Add log message when stacktrace cannot be obtained for thread
 2021-02-21 22:18:13 +03:00
Pavel Kruglov
4320782032 Style 2021-02-21 20:43:36 +03:00
alesapin
f8a581daba Remove possibly redundant allocations 2021-02-21 20:42:19 +03:00
Pavel Kruglov
db591309a3 fix style 2021-02-21 17:51:36 +03:00
Pavel Kruglov
93162716f4 remove logging 2021-02-21 17:50:47 +03:00
alexey-milovidov
125bf1d0d9
Merge pull request #20479 from ucasFL/tr 
fix transform with floating point key
 2021-02-21 17:37:20 +03:00
Pavel Kruglov
10f54f1714 remove include 2021-02-21 17:18:50 +03:00
Pavel Kruglov
7376e34788 remove test from cmake 2021-02-21 17:15:17 +03:00
Pavel Kruglov
abb5993bd5 return const 2021-02-21 17:13:33 +03:00
Pavel Kruglov
7c76b4a98a Remove debug logging 2021-02-21 17:12:26 +03:00
Kruglov Pavel
815b4ff0d9
Merge branch 'master' into hedged-requests 2021-02-21 17:10:24 +03:00
Pavel Kruglov
dd5185c779 Refactor 4.0 2021-02-21 17:03:24 +03:00
alexey-milovidov
20a7ccf5b1
Merge pull request #20174 from amosbird/alias 
Normalized function names (lower-case case-insensitive names)
 2021-02-21 16:23:51 +03:00
Alexey Milovidov
4085782b05 More gentle exception messages 2021-02-21 15:34:48 +03:00
Alexey Milovidov
af2f4ceb0e Add disambiguation while parsing already scaled DateTime64 #13194 2021-02-21 15:34:04 +03:00
Alexey Milovidov
00e6b6232f Revert "optimize aggfunc column data copy (#19407)" 
This reverts commit 7e3186c826.
 2021-02-21 12:49:50 +03:00
Denis Glazachev
c1158b50f4 Merge branch 'master' into row-policy-with-prewhere 
* master: (155 commits)
  Update version_date.tsv after release 20.8.13.15
  Update version_date.tsv after release 20.12.7.3
  Update version_date.tsv after release 21.1.5.4
  Update version_date.tsv after release 21.2.4.6
  fix
  Add test to skip list
  Fix WriteBufferFromHTTPServerResponse usage in other places (add missing finalize())
  Fix WriteBufferFromHTTPServerResponse usage in odbc-bridge
  Update config.xml
  Suppress signed overflow in AggregateFunctionGroupArrayMoving 2
  Update BaseDaemon.cpp
  review suggestions
  Fix bash syntax in 01731_async_task_queue_wait
  Do not use view() in 01731_async_task_queue_wait to fix ANTLR parser
  Increase buffer for uncaught exception / std::terminate
  Even more better
  Fix uncaught exception when HTTP client goes away
  test for decimal ( p , s) in dictionaries
  Just little better
  Fixed style check
  ...
 2021-02-21 13:25:47 +04:00
feng lv
5d3b8d99e7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into tr 2021-02-21 07:34:09 +00:00
Alexey Milovidov
b736f4fdee Suppression for PVS-Studio (looks like a false positive) 2021-02-21 10:21:59 +03:00
feng lv
a33183ee1e fix transform with floating point key 
better

update test
 2021-02-21 05:20:42 +00:00
tavplubix
aa48d967b7
Update DatabaseAtomic.cpp 2021-02-21 01:13:14 +03:00
alesapin
331de3d653 Trying to add tests more stable 2021-02-21 00:08:19 +03:00
alesapin
48d5d75282 Make unit test more stable 2021-02-20 23:47:18 +03:00
Vitaly Baranov
2eecaee08d Better tests for protobuf format. 2021-02-20 23:13:32 +03:00
Denis Glazachev
dfde9de0e9 Remove commented-out code 2021-02-20 23:30:21 +04:00
Alexey Milovidov
64e76a4a8d Minor changes in Decimal 2021-02-20 21:13:36 +03:00
Nikolai Kochetov
00e0dbc3e5 Fix test. 2021-02-20 20:42:06 +03:00
tavplubix
103d569885
Merge branch 'master' into fix_force_drop_materialized_view 2021-02-20 20:41:03 +03:00
tavplubix
a697b578bc
Merge pull request #16193 from ClickHouse/database_replicated 
DatabaseReplicated: continuation of #10485
 2021-02-20 19:39:34 +03:00
Nikolai Kochetov
2ae0b47edb Refactor tryPushDownFilter optimization. 2021-02-20 19:25:47 +03:00