thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-14 03:25:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
58,598 Commits 9,131 Branches 1,774 Tags 2.1 GiB
1c8ad914e6
Commit Graph

58598 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Azat Khuzhin
568a49dad5 Remove unused AsynchronousWriteBuffer 2021-02-28 11:19:56 +03:00
Azat Khuzhin
8b58dba094 Drop unused HexWriteBuffer 2021-02-28 10:52:09 +03:00
feng lv
51021c1164 forbid to drop a column if it's referenced by materialized view 2021-02-28 05:24:39 +00:00
Azat Khuzhin
d42d4cfd6b Improve test_distributed_ddl_parallel to cover more cases 
Refs: #21264
 2021-02-28 05:45:40 +03:00
Azat Khuzhin
7d51ae3212 DDLWorker: Fix processing of tasks in parallel (correct queue begin) 
Otherwise it will:
- start from incorrect task and bail because finished node already
  exists
- and also process alreayd processed items
 2021-02-28 05:45:12 +03:00
Azat Khuzhin
0e68fc67aa Add other distributed_ddl settings into config.xml as an example 2021-02-28 05:45:12 +03:00
Azat Khuzhin
955974a8b5 DDLWorker: avoid NULL dereference on termination and failed zookeeper initialization 
Log snipped shows the problem:

    2021.02.24 04:40:29.349181 [ 39 ] {} <Warning> DDLWorker: DDLWorker is configured to use multiple threads. It's not recommended because queries can be reordered. Also it may cause some unknown issues to appear.
    2021.02.24 04:40:29.349516 [ 39 ] {} <Information> Application: Ready for connections.
    2021.02.24 04:40:29.349602 [ 74 ] {} <Debug> DDLWorker: Started DDLWorker cleanup thread
    2021.02.24 04:40:29.349639 [ 73 ] {} <Debug> DDLWorker: Starting DDLWorker thread
    2021.02.24 04:40:29.349698 [ 73 ] {} <Debug> DDLWorker: Started DDLWorker thread
    2021.02.24 04:40:29.352548 [ 73 ] {} <Error> virtual void DB::DDLWorker::initializeMainThread(): Code: 999, e.displayText() = Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
    Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused (version 21.3.1.1), 192.168.112.3:2181
     (Connection loss), Stack trace (when copying this message, always include the lines below):

    0. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0xfe93923 in /usr/bin/clickhouse
    1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0xfe93ba2 in /usr/bin/clickhouse
    2. Coordination::ZooKeeper::connect(std::__1::vector<Coordination::ZooKeeper::Node, std::__1::allocator<Coordination::ZooKeeper::Node> > const&, Poco::Timespan) @ 0xfed3a01 in /usr/bin/clickhouse
    3. Coordination::ZooKeeper::ZooKeeper(std::__1::vector<Coordination::ZooKeeper::Node, std::__1::allocator<Coordination::ZooKeeper::Node> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Poco::Timespan, Poco::Timespan, Poco::Timespan) @ 0xfed2222 in /usr/bin/clickhouse
    4. zkutil::ZooKeeper::init(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfe961cd in /usr/bin/clickhouse
    5. zkutil::ZooKeeper::ZooKeeper(Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfe97a96 in /usr/bin/clickhouse
    6. void std::__1::allocator_traits<std::__1::allocator<zkutil::ZooKeeper> >::__construct<zkutil::ZooKeeper, Poco::Util::AbstractConfiguration const&, char const (&) [10]>(std::__1::integral_constant<bool, true>, std::__1::allocator<zkutil::ZooKeeper>&, zkutil::ZooKeeper*, Poco::Util::AbstractConfiguration const&, char const (&) [10]) @ 0xed98387 in /usr/bin/clickhouse
    7. DB::Context::getZooKeeper() const @ 0xed75190 in /usr/bin/clickhouse
    8. DB::DDLWorker::getAndSetZooKeeper() @ 0xedb81c9 in /usr/bin/clickhouse
    9. DB::DDLWorker::initializeMainThread() @ 0xedc9eb0 in /usr/bin/clickhouse
    10. DB::DDLWorker::runMainThread() @ 0xedb5d01 in /usr/bin/clickhouse
    11. ThreadFromGlobalPool::ThreadFromGlobalPool<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()::operator()() @ 0xedcafa1 in /usr/bin/clickhouse
    12. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x892651f in /usr/bin/clickhouse
    13. ? @ 0x8929fb3 in /usr/bin/clickhouse
    14. start_thread @ 0x8ea7 in /lib/x86_64-linux-gnu/libpthread-2.31.so
    15. __clone @ 0xfddef in /lib/x86_64-linux-gnu/libc-2.31.so
     (version 21.3.1.1)
    ...
    2021.02.24 04:40:30.025278 [ 41 ] {} <Trace> BaseDaemon: Received signal 15
    2021.02.24 04:40:30.025336 [ 41 ] {} <Information> Application: Received termination signal (Terminated)
    ...
    2021.02.24 04:40:30.582078 [ 39 ] {} <Information> Application: Closed all listening sockets.
    2021.02.24 04:40:30.582124 [ 39 ] {} <Information> Application: Closed connections.
    2021.02.24 04:40:30.583770 [ 39 ] {} <Information> Application: Shutting down storages.
    2021.02.24 04:40:30.583932 [ 39 ] {} <Information> Context: Shutdown disk data
    2021.02.24 04:40:30.583951 [ 39 ] {} <Information> Context: Shutdown disk default
    2021.02.24 04:40:30.584163 [ 46 ] {} <Trace> SystemLog (system.query_log): Terminating
    2021.02.24 04:40:30.586025 [ 39 ] {} <Trace> BackgroundSchedulePool/BgSchPool: Waiting for threads to finish.
    2021.02.24 04:40:34.352701 [ 73 ] {} <Debug> DDLWorker: Initialized DDLWorker thread
    2021.02.24 04:40:34.352758 [ 73 ] {} <Debug> DDLWorker: Scheduling tasks
 2021-02-28 05:45:12 +03:00
Azat Khuzhin
dfd1c73b31 Load dictionaries before starting accepting connections and DDLWorker 2021-02-28 05:45:12 +03:00
Azat Khuzhin
3c7e765b27 DDLWorker: process unfinished tasks only after reinitialization 2021-02-28 05:45:12 +03:00
Azat Khuzhin
54b1496408 DDLWorker: optimize processing of unfinished tasks 
Do not look at zookeeper if was_executed is false
 2021-02-28 05:45:12 +03:00
Azat Khuzhin
eeda1fe756 DDLWorker: wait for pending async tasks 2021-02-28 05:45:12 +03:00
Azat Khuzhin
0cd67ed051 DDLWorker: remove only completed tasks (significant for distributed_ddl.pool_size > 1) 
Otherwise it will SIGSEGV due to invalid-read (UAF).
 2021-02-28 05:45:12 +03:00
Azat Khuzhin
7481b05d27 DDLWorker: avoid comparing pointers for getting min element 
The current_tasks always updated with emplace_back, so front() is ok.
 2021-02-28 05:45:11 +03:00
Azat Khuzhin
a795db57f1 tests: decrease distributed_ddl_task_timeout to 60 in test_distributed_ddl_parallel 2021-02-28 05:45:11 +03:00
Azat Khuzhin
06e8bb955d tests: catch exceptions in threads in test_distributed_ddl_parallel 
Otherwise the test will not report an error.
 2021-02-28 05:45:11 +03:00
Azat Khuzhin
b899d8d919 tests: regression for test for invalid reference to zookeeper for distributed_ddl.pool_size > 1 2021-02-28 05:45:11 +03:00
Azat Khuzhin
78c6e0527d DDLWorker: Fix reference to zookeeper with distributed_ddl.pool_size > 1 (thread pool) 2021-02-28 05:45:11 +03:00
Azat Khuzhin
f7dbcfc12a DDLTask: Use std::uncaught_exceptions() over deprecated std::uncaught_exception() 2021-02-28 05:45:11 +03:00
Anmol Arora
2d9b524bdd Fill only requested columns when querying system.parts & system.parts_columns 2021-02-28 01:58:38 +00:00
Alexey Milovidov
220a494fa4 Fix idiotic syntax highlight in docs #18432 2021-02-28 04:42:45 +03:00
Alexey Milovidov
ae9fea1d0a Fix gcc and clang-tidy 2021-02-28 04:05:04 +03:00
Alexey Milovidov
107d8ec811 Merge branch 'master' into extract-text-from-html 2021-02-28 04:03:32 +03:00
Alexey Milovidov
e5ae9cbb63 Fix Arcadia 2021-02-28 04:03:22 +03:00
alexey-milovidov
9f0400f015
Merge pull request #21171 from azat/tests-bump-zookeeper 
Bump zookeeper version to 3.6.2 in tests
 2021-02-28 02:35:58 +03:00
Alexey Milovidov
01ef06a42c Fix broken links 2021-02-28 02:31:47 +03:00
alexey-milovidov
528f1d9a39
Merge pull request #21266 from Sin4wd/patch-1 
Fix broken links in "See Also"
 2021-02-28 02:29:18 +03:00
alexey-milovidov
b8fba768e5
Merge pull request #21264 from ClickHouse/fix_zookeeper_update 
Fix several bugs with ZooKeeper client
 2021-02-28 01:57:04 +03:00
alexey-milovidov
470bc2b01b
Merge pull request #21279 from kssenii/add-tests 
Add tests for zstd and zlib http compression
 2021-02-28 01:33:21 +03:00
Alexey Milovidov
4ab18cdcd8 Tests and documentation 2021-02-28 00:59:27 +03:00
Alexey Milovidov
0ab4afeeed Tests and documentation 2021-02-28 00:55:56 +03:00
alexey-milovidov
7d8ddbb813
Merge pull request #21278 from ClickHouse/cell-towers 
Add new dataset: Cell Towers
 2021-02-27 23:42:58 +03:00
Nikita Mikhaylov
ad72a83711
Merge pull request #20325 from gyuton/gyuton-DOCSUP-6073-Document_mapContains_mapKeys_mapValues_functions 
DOCSUP-6073: Documented mapContains, mapKeys, mapValues functions
 2021-02-27 23:16:45 +03:00
Alexey Milovidov
2ac673b12a Update logic and tests 2021-02-27 22:33:55 +03:00
Alexey Milovidov
fd1cf49e92 Rewrite extractTextFromHTML function 2021-02-27 22:22:38 +03:00
alexey-milovidov
4868266e74
Merge pull request #21291 from kolya7k/patch-1 
Add information about my ClickHouse PHP extension
 2021-02-27 21:41:12 +03:00
Nikolay
bb5a8241b9
Add information about my ClickHouse PHP extension 
I made a native PHP extension for ClickHouse using the clickhouse-cpp library.
My goal is to make a fast extension for ClickHouse with interface similar to mysqli.
 2021-02-27 21:05:04 +03:00
alexey-milovidov
0f1686c31a
Merge pull request #21285 from kitaisreal/constraints-block-output-stream-optimize-nullable-column-case 
CheckConstraintsBlockOutputStream optimize nullable column case
 2021-02-27 19:24:26 +03:00
Maksim Kita
315824978d CheckConstraintsBlockOutputStream optimize nullable column case 2021-02-27 19:19:21 +03:00
Alexey Milovidov
5e3c9a5deb Rename 2021-02-27 17:44:12 +03:00
Alexey Milovidov
2cafd54c17 Add new dataset: Cell Towers 2021-02-27 17:42:31 +03:00
kssenii
4f61ef3099 Tests for zstd and zlib 2021-02-27 14:12:45 +00:00
alexey-milovidov
56af09891b
Merge pull request #20991 from kssenii/brotli 
Fix Brotli http compression issue
 2021-02-27 16:39:49 +03:00
alesapin
9dc3952ff3 Fix typo 2021-02-27 16:30:43 +03:00
alexey-milovidov
22d703d560
Merge pull request #21258 from ClickHouse/allow-binary-modifications-under-gdb 
Allow to start up with modified binary under gdb
 2021-02-27 16:25:33 +03:00
Maksim Kita
9996f68160
Merge pull request #20990 from kitaisreal/constraints-complex-type-support 
Constraints complex types support
 2021-02-27 13:16:29 +03:00
Pavel Kruglov
1b1f50e324 Update comment 2021-02-27 11:45:32 +03:00
Pavel Kruglov
4d7ba0443f Add checking pending data 2021-02-27 11:40:03 +03:00
Vladimir
df728dc4ba
Merge pull request #21248 from vdimir/fix-join-totals-nullable 
Convert columns to nullable for totals in join
 2021-02-27 11:31:02 +03:00
Alexander Kazakov
63b95c7451
MySQL dictionary source: A mechanism for retrying unexpected connection loss during communication with mysql server (#21237) 
* Added a new type of Exception

for better recognition of connection failures

* Added more logging for mysql communication

* Retries on connection loss during query.

Make MySQL dictionary source resilient to unexpected loss of connection
during querying MySQL. This behavior is controlled with
".fail_on_connection_loss" config parameter, which defaults to false.

* Stripped some debugging leftover garbage

* Minor followup corrections

* Corrections after PR comments

* Yet more fixes
 2021-02-27 11:18:28 +03:00
alesapin
9e93d7f507 Fix tidy and add comments 2021-02-27 11:07:14 +03:00