Nikolai Kochetov
|
976dbe8077
|
Merge pull request #20341 from ClickHouse/filter-push-down
Filter push down
|
2021-03-01 12:35:06 +03:00 |
|
Nikolai Kochetov
|
eb42896e26
|
Merge branch 'master' into ucasFL-union-distinct-improve
|
2021-03-01 10:43:39 +03:00 |
|
alesapin
|
11f2a271a2
|
Remove useless unit test
|
2021-03-01 10:40:00 +03:00 |
|
alexey-milovidov
|
174f49a9f1
|
Merge pull request #21308 from fuwhu/refine-wal-restore-code
remove unused code in MergeTreeWriteAheadLog::restore
|
2021-03-01 01:44:23 +03:00 |
|
alexey-milovidov
|
9b633ac564
|
Update ontime.md
|
2021-02-28 23:27:01 +03:00 |
|
alexey-milovidov
|
81be2d1ed9
|
Merge pull request #21305 from azat/terminate-nested-writers
Fix abnormal server termination for nested writers
|
2021-02-28 21:54:13 +03:00 |
|
alexey-milovidov
|
3bd180c416
|
Merge pull request #20149 from keen-wolf/timezone_about
fix toMinute function to handle special timezone
|
2021-02-28 21:53:51 +03:00 |
|
alexey-milovidov
|
b924b28c57
|
Merge pull request #21306 from azat/unused-writers
[RFC] Remove unused writers
|
2021-02-28 21:47:01 +03:00 |
|
Vladimir
|
6aa72bcd43
|
Merge pull request #19591 from ka1bi4/romanzhukov-DOCSUP-5266-PR-translate
|
2021-02-28 20:40:04 +03:00 |
|
Dmitriy
|
187eb837db
|
Add the links
Добавил ссылки.
|
2021-02-28 20:10:44 +03:00 |
|
Denis Glazachev
|
eb80c189e4
|
Merge branch 'master' into row-policy-with-prewhere
|
2021-02-28 20:14:31 +04:00 |
|
fuwhu
|
2df33be7c2
|
Remove unused code in MergeTreeWriteAheadLog::restore
|
2021-02-28 23:17:31 +08:00 |
|
alexey-milovidov
|
467fcbec06
|
Merge pull request #21292 from ClickHouse/extract-text-from-html
Rewrite extractTextFromHTML function
|
2021-02-28 16:36:18 +03:00 |
|
Azat Khuzhin
|
eb0387c5a9
|
Fix abnormal server termination for nested writers
Writers with nested writer can call next() from the dtor for nested
writer and this will cause exception again, so the buffer position
should be updated on exceptions.
Found by stress test (thread) here [1] and here [2]:
2021.02.27 19:27:53.498977 [ 302 ] {} <Fatal> BaseDaemon: (version 21.3.1.6130, build id: 2DAEC5DEBF03C5A1C3BF66B7779C886F16239345) (from thread 1614) Terminate called for uncaught exception:
Code: 24, e.displayText() = DB::Exception: Cannot write to ostream at offset 2097498, Stack trace (when copying this message, always include the lines below):
0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:0: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x15bef2ab in /usr/bin/clickhouse
1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:56: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8aea92e in /usr/bin/clickhouse
2. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromOStream.cpp:0: DB::WriteBufferFromOStream::nextImpl() @ 0x8bbbc45 in /usr/bin/clickhouse
3. ./obj-x86_64-linux-gnu/../src/IO/BufferBase.h:39: DB::WriteBufferFromOStream::~WriteBufferFromOStream() @ 0x8bbc077 in /usr/bin/clickhouse
4. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromOStream.cpp:44: DB::WriteBufferFromOStream::~WriteBufferFromOStream() @ 0x8bbc18a in /usr/bin/clickhouse
5. ./obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:137: DB::ZstdDeflatingWriteBuffer::~ZstdDeflatingWriteBuffer() @ 0x118bdc29 in /usr/bin/clickhouse
6. ./obj-x86_64-linux-gnu/../src/IO/ZstdDeflatingWriteBuffer.cpp:32: DB::ZstdDeflatingWriteBuffer::~ZstdDeflatingWriteBuffer() @ 0x118be3ea in /usr/bin/clickhouse
7. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:0: DB::WriteBufferFromHTTPServerResponse::finalize() @ 0x12f1dceb in /usr/bin/clickhouse
8. ./obj-x86_64-linux-gnu/../src/Server/HTTPHandler.cpp:703: DB::HTTPHandler::trySendExceptionToClient(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, DB::HTTPServerRequest&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&) @ 0x12e9fecc in /usr/bin/clickhouse
9. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:1444: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x12ea0d60 in /usr/bin/clickhouse
10. ./obj-x86_64-linux-gnu/../src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x12f16db1 in /usr/bin/clickhouse
11. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x15b184f3 in /usr/bin/clickhouse
12. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:0: Poco::Net::TCPServerDispatcher::run() @ 0x15b18c1f in /usr/bin/clickhouse
13. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/ScopedLock.h:36: Poco::PooledThread::run() @ 0x15c7fdb2 in /usr/bin/clickhouse
14. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:56: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x15c7e350 in /usr/bin/clickhouse
15. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/SharedPtr.h:277: Poco::ThreadImpl::runnableEntry(void*) @ 0x15c7cb58 in /usr/bin/clickhouse
16. __tsan_thread_start_func @ 0x8a04ced in /usr/bin/clickhouse
17. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
18. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
(version 21.3.1.6130)
[1]: https://clickhouse-test-reports.s3.yandex.net/21279/4f61ef3099f42f17b496a0b0424773978d9a32dc/stress_test_(thread).html#fail1
[2]: https://clickhouse-test-reports.s3.yandex.net/21292/ae9fea1d0af118a8f87b224d194d61da1567188b/stress_test_(thread).html#fail1
v2: https://clickhouse-test-reports.s3.yandex.net/21305/e969daa6e86c5e09cfef08cfde19712982b64e59/stress_test_(thread).html#fail1
|
2021-02-28 15:13:42 +03:00 |
|
keenwolf
|
303c389b36
|
change the time_t to unsigned to handle properly the start date of 1970-01-01
|
2021-02-28 18:11:54 +08:00 |
|
Azat Khuzhin
|
568a49dad5
|
Remove unused AsynchronousWriteBuffer
|
2021-02-28 11:19:56 +03:00 |
|
Azat Khuzhin
|
8b58dba094
|
Drop unused HexWriteBuffer
|
2021-02-28 10:52:09 +03:00 |
|
feng lv
|
51021c1164
|
forbid to drop a column if it's referenced by materialized view
|
2021-02-28 05:24:39 +00:00 |
|
Azat Khuzhin
|
d42d4cfd6b
|
Improve test_distributed_ddl_parallel to cover more cases
Refs: #21264
|
2021-02-28 05:45:40 +03:00 |
|
Azat Khuzhin
|
7d51ae3212
|
DDLWorker: Fix processing of tasks in parallel (correct queue begin)
Otherwise it will:
- start from incorrect task and bail because finished node already
exists
- and also process alreayd processed items
|
2021-02-28 05:45:12 +03:00 |
|
Azat Khuzhin
|
0e68fc67aa
|
Add other distributed_ddl settings into config.xml as an example
|
2021-02-28 05:45:12 +03:00 |
|
Azat Khuzhin
|
955974a8b5
|
DDLWorker: avoid NULL dereference on termination and failed zookeeper initialization
Log snipped shows the problem:
2021.02.24 04:40:29.349181 [ 39 ] {} <Warning> DDLWorker: DDLWorker is configured to use multiple threads. It's not recommended because queries can be reordered. Also it may cause some unknown issues to appear.
2021.02.24 04:40:29.349516 [ 39 ] {} <Information> Application: Ready for connections.
2021.02.24 04:40:29.349602 [ 74 ] {} <Debug> DDLWorker: Started DDLWorker cleanup thread
2021.02.24 04:40:29.349639 [ 73 ] {} <Debug> DDLWorker: Starting DDLWorker thread
2021.02.24 04:40:29.349698 [ 73 ] {} <Debug> DDLWorker: Started DDLWorker thread
2021.02.24 04:40:29.352548 [ 73 ] {} <Error> virtual void DB::DDLWorker::initializeMainThread(): Code: 999, e.displayText() = Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: 192.168.112.3:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
(Connection loss), Stack trace (when copying this message, always include the lines below):
0. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0xfe93923 in /usr/bin/clickhouse
1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0xfe93ba2 in /usr/bin/clickhouse
2. Coordination::ZooKeeper::connect(std::__1::vector<Coordination::ZooKeeper::Node, std::__1::allocator<Coordination::ZooKeeper::Node> > const&, Poco::Timespan) @ 0xfed3a01 in /usr/bin/clickhouse
3. Coordination::ZooKeeper::ZooKeeper(std::__1::vector<Coordination::ZooKeeper::Node, std::__1::allocator<Coordination::ZooKeeper::Node> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Poco::Timespan, Poco::Timespan, Poco::Timespan) @ 0xfed2222 in /usr/bin/clickhouse
4. zkutil::ZooKeeper::init(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfe961cd in /usr/bin/clickhouse
5. zkutil::ZooKeeper::ZooKeeper(Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfe97a96 in /usr/bin/clickhouse
6. void std::__1::allocator_traits<std::__1::allocator<zkutil::ZooKeeper> >::__construct<zkutil::ZooKeeper, Poco::Util::AbstractConfiguration const&, char const (&) [10]>(std::__1::integral_constant<bool, true>, std::__1::allocator<zkutil::ZooKeeper>&, zkutil::ZooKeeper*, Poco::Util::AbstractConfiguration const&, char const (&) [10]) @ 0xed98387 in /usr/bin/clickhouse
7. DB::Context::getZooKeeper() const @ 0xed75190 in /usr/bin/clickhouse
8. DB::DDLWorker::getAndSetZooKeeper() @ 0xedb81c9 in /usr/bin/clickhouse
9. DB::DDLWorker::initializeMainThread() @ 0xedc9eb0 in /usr/bin/clickhouse
10. DB::DDLWorker::runMainThread() @ 0xedb5d01 in /usr/bin/clickhouse
11. ThreadFromGlobalPool::ThreadFromGlobalPool<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()::operator()() @ 0xedcafa1 in /usr/bin/clickhouse
12. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x892651f in /usr/bin/clickhouse
13. ? @ 0x8929fb3 in /usr/bin/clickhouse
14. start_thread @ 0x8ea7 in /lib/x86_64-linux-gnu/libpthread-2.31.so
15. __clone @ 0xfddef in /lib/x86_64-linux-gnu/libc-2.31.so
(version 21.3.1.1)
...
2021.02.24 04:40:30.025278 [ 41 ] {} <Trace> BaseDaemon: Received signal 15
2021.02.24 04:40:30.025336 [ 41 ] {} <Information> Application: Received termination signal (Terminated)
...
2021.02.24 04:40:30.582078 [ 39 ] {} <Information> Application: Closed all listening sockets.
2021.02.24 04:40:30.582124 [ 39 ] {} <Information> Application: Closed connections.
2021.02.24 04:40:30.583770 [ 39 ] {} <Information> Application: Shutting down storages.
2021.02.24 04:40:30.583932 [ 39 ] {} <Information> Context: Shutdown disk data
2021.02.24 04:40:30.583951 [ 39 ] {} <Information> Context: Shutdown disk default
2021.02.24 04:40:30.584163 [ 46 ] {} <Trace> SystemLog (system.query_log): Terminating
2021.02.24 04:40:30.586025 [ 39 ] {} <Trace> BackgroundSchedulePool/BgSchPool: Waiting for threads to finish.
2021.02.24 04:40:34.352701 [ 73 ] {} <Debug> DDLWorker: Initialized DDLWorker thread
2021.02.24 04:40:34.352758 [ 73 ] {} <Debug> DDLWorker: Scheduling tasks
|
2021-02-28 05:45:12 +03:00 |
|
Azat Khuzhin
|
dfd1c73b31
|
Load dictionaries before starting accepting connections and DDLWorker
|
2021-02-28 05:45:12 +03:00 |
|
Azat Khuzhin
|
3c7e765b27
|
DDLWorker: process unfinished tasks only after reinitialization
|
2021-02-28 05:45:12 +03:00 |
|
Azat Khuzhin
|
54b1496408
|
DDLWorker: optimize processing of unfinished tasks
Do not look at zookeeper if was_executed is false
|
2021-02-28 05:45:12 +03:00 |
|
Azat Khuzhin
|
eeda1fe756
|
DDLWorker: wait for pending async tasks
|
2021-02-28 05:45:12 +03:00 |
|
Azat Khuzhin
|
0cd67ed051
|
DDLWorker: remove only completed tasks (significant for distributed_ddl.pool_size > 1)
Otherwise it will SIGSEGV due to invalid-read (UAF).
|
2021-02-28 05:45:12 +03:00 |
|
Azat Khuzhin
|
7481b05d27
|
DDLWorker: avoid comparing pointers for getting min element
The current_tasks always updated with emplace_back, so front() is ok.
|
2021-02-28 05:45:11 +03:00 |
|
Azat Khuzhin
|
a795db57f1
|
tests: decrease distributed_ddl_task_timeout to 60 in test_distributed_ddl_parallel
|
2021-02-28 05:45:11 +03:00 |
|
Azat Khuzhin
|
06e8bb955d
|
tests: catch exceptions in threads in test_distributed_ddl_parallel
Otherwise the test will not report an error.
|
2021-02-28 05:45:11 +03:00 |
|
Azat Khuzhin
|
b899d8d919
|
tests: regression for test for invalid reference to zookeeper for distributed_ddl.pool_size > 1
|
2021-02-28 05:45:11 +03:00 |
|
Azat Khuzhin
|
78c6e0527d
|
DDLWorker: Fix reference to zookeeper with distributed_ddl.pool_size > 1 (thread pool)
|
2021-02-28 05:45:11 +03:00 |
|
Azat Khuzhin
|
f7dbcfc12a
|
DDLTask: Use std::uncaught_exceptions() over deprecated std::uncaught_exception()
|
2021-02-28 05:45:11 +03:00 |
|
Anmol Arora
|
2d9b524bdd
|
Fill only requested columns when querying system.parts & system.parts_columns
|
2021-02-28 01:58:38 +00:00 |
|
Alexey Milovidov
|
220a494fa4
|
Fix idiotic syntax highlight in docs #18432
|
2021-02-28 04:42:45 +03:00 |
|
Alexey Milovidov
|
ae9fea1d0a
|
Fix gcc and clang-tidy
|
2021-02-28 04:05:04 +03:00 |
|
Alexey Milovidov
|
107d8ec811
|
Merge branch 'master' into extract-text-from-html
|
2021-02-28 04:03:32 +03:00 |
|
Alexey Milovidov
|
e5ae9cbb63
|
Fix Arcadia
|
2021-02-28 04:03:22 +03:00 |
|
alexey-milovidov
|
9f0400f015
|
Merge pull request #21171 from azat/tests-bump-zookeeper
Bump zookeeper version to 3.6.2 in tests
|
2021-02-28 02:35:58 +03:00 |
|
Alexey Milovidov
|
01ef06a42c
|
Fix broken links
|
2021-02-28 02:31:47 +03:00 |
|
alexey-milovidov
|
528f1d9a39
|
Merge pull request #21266 from Sin4wd/patch-1
Fix broken links in "See Also"
|
2021-02-28 02:29:18 +03:00 |
|
alexey-milovidov
|
b8fba768e5
|
Merge pull request #21264 from ClickHouse/fix_zookeeper_update
Fix several bugs with ZooKeeper client
|
2021-02-28 01:57:04 +03:00 |
|
alexey-milovidov
|
470bc2b01b
|
Merge pull request #21279 from kssenii/add-tests
Add tests for zstd and zlib http compression
|
2021-02-28 01:33:21 +03:00 |
|
Alexey Milovidov
|
4ab18cdcd8
|
Tests and documentation
|
2021-02-28 00:59:27 +03:00 |
|
Alexey Milovidov
|
0ab4afeeed
|
Tests and documentation
|
2021-02-28 00:55:56 +03:00 |
|
alexey-milovidov
|
7d8ddbb813
|
Merge pull request #21278 from ClickHouse/cell-towers
Add new dataset: Cell Towers
|
2021-02-27 23:42:58 +03:00 |
|
Nikita Mikhaylov
|
ad72a83711
|
Merge pull request #20325 from gyuton/gyuton-DOCSUP-6073-Document_mapContains_mapKeys_mapValues_functions
DOCSUP-6073: Documented mapContains, mapKeys, mapValues functions
|
2021-02-27 23:16:45 +03:00 |
|
Alexey Milovidov
|
2ac673b12a
|
Update logic and tests
|
2021-02-27 22:33:55 +03:00 |
|
Alexey Milovidov
|
fd1cf49e92
|
Rewrite extractTextFromHTML function
|
2021-02-27 22:22:38 +03:00 |
|
alexey-milovidov
|
4868266e74
|
Merge pull request #21291 from kolya7k/patch-1
Add information about my ClickHouse PHP extension
|
2021-02-27 21:41:12 +03:00 |
|