thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 02:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
74,818 Commits 9,328 Branches 1,783 Tags 2.1 GiB
677eb5bbda
Commit Graph

604 Commits

Author SHA1 Message Date
alesapin
3d954c4314 Better request/response logic 2021-02-19 14:06:17 +03:00
alesapin
6c9322bb2e Sane constant while reading requests 2021-02-19 12:57:39 +03:00
Amos Bird
fc185e5fb7
Another try 2021-02-19 11:56:24 +08:00
Alexander Tokmakov
97f4c457ec fix MySQL COMM_FIELD_LIST response 2021-02-18 16:27:51 +03:00
Amos Bird
2c4bc43014
Backward compatible 2021-02-18 20:20:29 +08:00
Ivan Lezhankin
dee051dc8e Merge remote-tracking branch 'upstream/master' into async-insert 2021-02-17 21:48:48 +03:00
Pavel Kruglov
1b78de2142 Use fibers in HedgedRequests 2021-02-17 20:34:52 +03:00
Ivan Lezhankin
f2b1708164 Refactor inserts and its tails 2021-02-15 21:57:35 +03:00
Kruglov Pavel
598576ce70
Merge branch 'master' into hedged-requests 2021-02-15 16:51:45 +03:00
Azat Khuzhin
33f54cdb3c Fix abnormal server termination when http client goes away 
In [1] stress tests found:

    2021.02.12 14:20:58.800988 [ 17728 ] {de3e7894-b401-4f7d-8530-90cd5ab06682} <Debug> executeQuery: (from [::1]:45792, using production parser) (comment: /usr/share/clickhouse-test/queries/0_stateless/01520_client_print_query_id.expect) SELECT * FROM numbers(34599)
    2021.02.12 14:20:58.916484 [ 17728 ] {de3e7894-b401-4f7d-8530-90cd5ab06682} <Trace> ContextAccess (default): Access granted: CREATE TEMPORARY TABLE ON *.*
    2021.02.12 14:20:59.071980 [ 17728 ] {de3e7894-b401-4f7d-8530-90cd5ab06682} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
    2021.02.12 14:21:10.708202 [ 17728 ] {de3e7894-b401-4f7d-8530-90cd5ab06682} <Information> executeQuery: Read 34599 rows, 270.30 KiB in 11.876294055 sec., 2913 rows/sec., 22.76 KiB/sec.
    2021.02.12 14:22:10.506261 [ 17728 ] {de3e7894-b401-4f7d-8530-90cd5ab06682} <Debug> DynamicQueryHandler: Done processing query
    2021.02.12 14:22:18.238037 [ 375 ] {} <Fatal> BaseDaemon: (version 21.3.1.5996, build id: 8DBCED54529C989F7AD4D991F51410774D55DE6C) (from thread 17728) Terminate called for uncaught exception:
    Code: 24, e.displayText() = DB::Exception: Cannot write to ostream at offset 262994, Stack trace (when copying this message, always include the lines below):

    0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:0: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x15c976cb in /usr/bin/clickhouse
    1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:56: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8c9320e in /usr/bin/clickhouse
    2. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromOStream.cpp:0: DB::WriteBufferFromOStream::nextImpl() @ 0x8d54da5 in /usr/bin/clickhouse
    3. ./obj-x86_64-linux-gnu/../src/IO/BufferBase.h:39: DB::WriteBufferFromOStream::~WriteBufferFromOStream() @ 0x8d551d7 in /usr/bin/clickhouse
    4. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromOStream.cpp:44: DB::Write

    2021.02.12 14:22:18.811071 [ 18134 ] {} <Fatal> BaseDaemon: ########################################
    2021.02.12 14:22:18.878935 [ 18134 ] {} <Fatal> BaseDaemon: (version 21.3.1.5996, build id: 8DBCED54529C989F7AD4D991F51410774D55DE6C) (from thread 17728) (query_id: de3e7894-b401-4f7d-8530-90cd5ab06682) Received signal Aborted (6)
    2021.02.12 14:22:18.943148 [ 18134 ] {} <Fatal> BaseDaemon:
    2021.02.12 14:22:19.007073 [ 18134 ] {} <Fatal> BaseDaemon: Stack trace: 0x7f109932018b 0x7f10992ff859 0x8bb33ae 0x8e301dd 0x17dac8c4 0x17dac7c7 0x8c3fe0b 0x8d552c5 0x8d552ea 0x11a29914 0x11a2a2ca 0x12f96092 0x12f8c65e 0x12f84300 0x15b84110 0x15bc0913 0x15bc103f 0x15d29a12 0x15d27fb0 0x15d267b8 0x8badbad 0x7f10994d5609 0x7f10993fc293
    2021.02.12 14:22:19.255998 [ 18134 ] {} <Fatal> BaseDaemon: 5. raise @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.02.12 14:22:19.270203 [ 18134 ] {} <Fatal> BaseDaemon: 6. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.02.12 14:22:50.108918 [ 370 ] {} <Fatal> Application: Child process was terminated by signal 6.

  [1]: https://clickhouse-test-reports.s3.yandex.net/19580/6aecb62416ece880cbb8ee3a803e14d841388dde/stress_test_(thread).html#fail1

Verified locally by commenting out->next() call in
WriteBufferFromHTTPServerResponse::nextImpl(), adding a sleep(1) and
canceling HTTP request before it finished, the stacktrace as follow:

    [ 6351 ] {} <Fatal> BaseDaemon: (version 21.3.1.1, build id: 9B40466BF3D2F5AED78A52A995A4A2FD3116787C) (from thread 6677) Terminate called for uncaught exception:
    Code: 24, e.displayText() = DB::Exception: Cannot write to ostream at offset 4, Stack trace (when copying this message, always include the lines below):

    0. /src/ch/clickhouse/.cmake/../src/Common/StackTrace.cpp:298: StackTrace::tryCapture() @ 0x30a52a in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    1. /src/ch/clickhouse/.cmake/../src/Common/StackTrace.cpp:260: StackTrace::StackTrace() @ 0x30a4e5 in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    2. /src/ch/clickhouse/.cmake/../src/Common/Exception.cpp:53: DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, bool) @ 0x2a61ae in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    3. /src/ch/clickhouse/.cmake/../src/IO/WriteBufferFromOStream.cpp:22: DB::WriteBufferFromOStream::nextImpl() @ 0x3b468a in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    4. /src/ch/clickhouse/.cmake/../src/IO/WriteBuffer.h:47

    <snip>
    [ 8966 ] {} <Fatal> BaseDaemon: 7. __cxxabiv1::__terminate(void (*)()) @ 0x1784ca in /src/ch/clickhouse/.cmake/contrib/replxx-cmake/libreplxxd.so
    <snip>
    [ 8966 ] {} <Fatal> BaseDaemon: 10. /src/ch/clickhouse/.cmake/../src/IO/WriteBufferFromOStream.cpp:0: DB::WriteBufferFromOStream::~WriteBufferFromOStream() @ 0x3b48c1 in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    [ 8966 ] {} <Fatal> BaseDaemon: 11. /src/ch/clickhouse/.cmake/../src/IO/WriteBufferFromOStream.cpp:44: DB::WriteBufferFromOStream::~WriteBufferFromOStream() @ 0x3b48ec in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    <snip>
    [ 8966 ] {} <Fatal> BaseDaemon: 14. /src/ch/clickhouse/.cmake/../src/IO/WriteBufferFromHTTPServerResponse.cpp:218: DB::WriteBufferFromHTTPServerResponse::~WriteBufferFromHTTPServerResponse() @ 0x3b33cd in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    <snip>
    [ 8966 ] {} <Fatal> BaseDaemon: 22. /src/ch/clickhouse/.cmake/../src/Server/HTTPHandler.h:43: DB::HTTPHandler::Output::~Output() @ 0x260421 in /src/ch/clickhouse/.cmake/src/libclickhouse_serverd.so
    [ 8966 ] {} <Fatal> BaseDaemon: 23. /src/ch/clickhouse/.cmake/../src/Server/HTTPHandler.cpp:778: DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&) @ 0x253fd4 in /src/ch/clickhouse/.cmake/src/libclickhouse_serverd.so
 2021-02-13 13:23:03 +03:00
Azat Khuzhin
592f62d0af Remove superfluous out->next() call in HTTPHandler 2021-02-13 13:22:37 +03:00
alesapin
5f32f65211 Merge branch 'master' into in_memory_raft 2021-02-11 23:09:29 +03:00
alexey-milovidov
dc3ffd3fe2
Merge pull request #19451 from azat/safe-writes 
Do not silently ignore write errors
 2021-02-11 21:19:11 +03:00
Alexey Milovidov
dbb41ce4e2 Remove debug output 2021-02-11 02:55:52 +03:00
Alexey Milovidov
f442b30f30 Fix test 2021-02-11 01:23:27 +03:00
Alexey Milovidov
d3dba0e52a Merge branch 'master' into no-excessive-squash-memory 2021-02-10 23:45:12 +03:00
Alexey Milovidov
30b2554fa3 Fix error 2021-02-10 20:48:39 +03:00
alesapin
9667bdcbd2 Merge branch 'master' into in_memory_raft 2021-02-09 16:06:31 +03:00
alesapin
b8baf3a443 Fix some warnings 2021-02-08 22:40:44 +03:00
tavplubix
ac477d9850
Merge pull request #19771 from ClickHouse/thread_state_improvements 
Minor code improvements around ThreadStatus
 2021-02-08 22:34:55 +03:00
alesapin
add89c17f2 Less serde in responses, watches on followers 2021-02-08 16:06:55 +03:00
tavplubix
060be5b2db
rerun CI checks 2021-02-08 15:04:40 +03:00
alesapin
d5ee7d33e9 Merge branch 'master' into in_memory_raft 2021-02-08 10:30:31 +03:00
Alexey Milovidov
869bca74a7 Fix some tests 2021-02-07 23:37:55 +03:00
alexey-milovidov
f2ab4f2ce9
Merge pull request #20037 from abyss7/better-read-buffers-2 
ReadBuffer: check for unread data on next()
 2021-02-07 01:33:06 +03:00
Pavel Kruglov
f946aab759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-06 17:38:56 +03:00
alesapin
449e8e3fd9 More checks for setting 2021-02-05 13:15:02 +03:00
Azat Khuzhin
64c0bf9829 TCPHandler: catch exceptions from the WriteBuffer in destructor 
For TCPHandler it is safe thing todo.

Otherwise *San will report [1]:

    2021.01.24 15:33:40.103996 [ 270 ] {} <Trace> BaseDaemon: Received signal -1
    2021.01.24 15:33:40.110693 [ 270 ] {} <Fatal> BaseDaemon: (version 21.2.1.5789, build id: FF421B087D1E2EAA19FA17B5AB3AE413832744E0) (from thread 48318) Terminate called for uncaught exception:
    2021.01.24 15:33:40.114845 [ 270 ] {} <Trace> BaseDaemon: Received signal 6
    2021.01.24 15:33:40.138738 [ 218027 ] {} <Fatal> BaseDaemon: ########################################
    2021.01.24 15:33:40.138838 [ 218027 ] {} <Fatal> BaseDaemon: (version 21.2.1.5789, build id: FF421B087D1E2EAA19FA17B5AB3AE413832744E0) (from thread 48318) (no query) Received signal Aborted (6)
    2021.01.24 15:33:40.138912 [ 218027 ] {} <Fatal> BaseDaemon:
    2021.01.24 15:33:40.139277 [ 218027 ] {} <Fatal> BaseDaemon: Stack trace: 0x7f185474118b 0x7f1854720859 0xaddc0cc 0x2af9fab8 0x2af9fa04 0xa91758b 0x1e418bb5 0x20725b4f 0x20725d9e 0x266b47a3 0x269772f5 0x26971847 0x7f18548f6609 0x7f185481d293
    2021.01.24 15:33:40.139637 [ 218027 ] {} <Fatal> BaseDaemon: 3. raise @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.01.24 15:33:40.140113 [ 218027 ] {} <Fatal> BaseDaemon: 4. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.01.24 15:33:40.144121 [ 218027 ] {} <Fatal> BaseDaemon: 5. ./obj-x86_64-linux-gnu/../base/daemon/BaseDaemon.cpp:0: terminate_handler() @ 0xaddc0cc in /usr/bin/clickhouse
    2021.01.24 15:33:40.151208 [ 218027 ] {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:61: std::__terminate(void (*)()) @ 0x2af9fab8 in /usr/bin/clickhouse
    2021.01.24 15:33:40.153085 [ 218027 ] {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:0: std::terminate() @ 0x2af9fa04 in /usr/bin/clickhouse
    2021.01.24 15:33:40.155209 [ 218027 ] {} <Fatal> BaseDaemon: 8. ? @ 0xa91758b in /usr/bin/clickhouse
    2021.01.24 15:33:40.156621 [ 218027 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromPocoSocket.cpp:0: DB::WriteBufferFromPocoSocket::~WriteBufferFromPocoSocket() @ 0x1e418bb5 in /usr/bin/clickhouse
    2021.01.24 15:33:40.161041 [ 218027 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2518: DB::TCPHandler::~TCPHandler() @ 0x20725b4f in /usr/bin/clickhouse
    2021.01.24 15:33:40.164557 [ 218027 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.h:101: DB::TCPHandler::~TCPHandler() @ 0x20725d9e in /usr/bin/clickhouse
    2021.01.24 15:33:40.165921 [ 218027 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/AtomicCounter.h:314: Poco::Net::TCPServerDispatcher::run() @ 0x266b47a3 in /usr/bin/clickhouse
    2021.01.24 15:33:40.167347 [ 218027 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x269772f5 in /usr/bin/clickhouse
    2021.01.24 15:33:40.169401 [ 218027 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x26971847 in /usr/bin/clickhouse
    2021.01.24 15:33:40.169498 [ 218027 ] {} <Fatal> BaseDaemon: 15. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    2021.01.24 15:33:40.169566 [ 218027 ] {} <Fatal> BaseDaemon: 16. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.01.24 15:33:41.027601 [ 218027 ] {} <Fatal> BaseDaemon: Calculated checksum of the binary: 63D7491B39260494BA0D785E1860B427. There is no information about the reference checksum.

  [1]: https://clickhouse-test-reports.s3.yandex.net/19451/1e16bd6f337985a82fbdf4eded695dc6e663af58/stress_test_(address).html#fail1

v2: Fix catching errors in WriteBufferFromPocoSocket destructor
 2021-02-05 01:32:36 +03:00
Azat Khuzhin
4beb5c1b8a TCPHandler: Move constructor into the module and add missing headers 2021-02-05 01:31:42 +03:00
alesapin
c5312bf362 Trying to disable suspicious parameter 2021-02-04 22:29:46 +03:00
Ivan Lezhankin
ac4c3a6b27 Don't pass empty vector to ConcatReadBuffer 2021-02-04 18:59:54 +03:00
alesapin
da51ea1794 Simplify shutdown and requests processing 2021-02-04 15:07:41 +03:00
alesapin
1ff87ac6f9 Add background session lifetime control 2021-02-03 23:32:15 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
alesapin
9979181c02 Merge branch 'master' into in_memory_raft 2021-02-02 17:05:20 +03:00
alexey-milovidov
f5dc38a33c
Merge pull request #19584 from azat/client-complete-error-codes 
client: more suggestions
 2021-02-02 11:05:17 +03:00
Maksim Kita
d0151de4bb
Merge pull request #19608 from kreuzerkrieg/Add_IStoragePolicy_interface 
Add IStoragePolicy interface
 2021-02-02 11:03:20 +03:00
Azat Khuzhin
37797fdf5b Merge remote-tracking branch 'upstream/master' into client-complete-error-codes 2021-02-02 00:14:53 +03:00
alesapin
365bf65f5a Fix install script 2021-02-01 17:14:59 +03:00
alesapin
57c9b6c864 Fix build without nuraft 2021-02-01 16:18:17 +03:00
alesapin
0aca40d8cb Merge branch 'master' into in_memory_raft 2021-02-01 14:30:11 +03:00
alesapin
eb5c77f558 Fix some races and better reaction to leader change 2021-02-01 14:27:26 +03:00
alesapin
67412bd529 Don't accept connections if we don't see leader + race fix from upstream 2021-02-01 10:51:10 +03:00
Azat Khuzhin
978f23e343 Fix SIGSEGV due to accessing GRPCServer::currentConnections() before initialization 
This PR makes asynchronous metrics available just after start, and this
breaks GRPCServer, since it requires the server be started before
accessing currentConnections().

However it is possible to trigger the same SIGSEGV even without that
patch, with some timeouts during previous server initilizations and
small asynchronous_metrics_update_period_s.

Fix this by creating GRPCServer::Runner in the ctor.

Stacktrace:
    26.842505 [ 7 ] {} <Information> Application: Listening for MySQL compatibility protocol: 0.0.0.0:9004
    26.842562 [ 7 ] {} <Information> Application: Listening for gRPC protocol: 0.0.0.0:9100
    26.842600 [ 7 ] {} <Debug> AsynchronousMetrics: MemoryTracking: was 350.72 KiB, peak 5.25 MiB, will set to 232.63 MiB (RSS), difference: 232.29 MiB
    26.842834 [ 8 ] {} <Trace> BaseDaemon: Received signal 11
    26.843014 [ 39 ] {} <Fatal> BaseDaemon: ########################################
    26.843055 [ 39 ] {} <Fatal> BaseDaemon: (version 21.2.1.5858, build id: B27D5550AC34F9091BC4437D8021B752EDB34FBB) (from thread 7) (no query) Received signal Segmentation fault (11)
    26.843085 [ 39 ] {} <Fatal> BaseDaemon: Address: 0x78 Access: read. Address not mapped to object.
    26.843104 [ 39 ] {} <Fatal> BaseDaemon: Stack trace: 0x7fe8c4e6afc4 0x13683cf1 0xf52efe5 0xe7c9137 0x85596e1 0x8553635 0x11b7a313 0x8545ebc 0x8544b25 0x84e1cbe 0x7fe8c4c940b3 0x84ac22e
    26.843136 [ 39 ] {} <Fatal> BaseDaemon: 2. pthread_mutex_lock @ 0xbfc4 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    26.843162 [ 39 ] {} <Fatal> BaseDaemon: 3. std::__1::mutex::lock() @ 0x13683cf1 in ?
    26.843188 [ 39 ] {} <Fatal> BaseDaemon: 4. DB::GRPCServer::currentConnections() const @ 0xf52efe5 in /usr/bin/clickhouse
    26.843204 [ 39 ] {} <Fatal> BaseDaemon: 5. DB::AsynchronousMetrics::update() @ 0xe7c9137 in /usr/bin/clickhouse
    26.843228 [ 39 ] {} <Fatal> BaseDaemon: 6. DB::AsynchronousMetrics::start() @ 0x85596e1 in /usr/bin/clickhouse
    26.843248 [ 39 ] {} <Fatal> BaseDaemon: 7. DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0x8553635 in /usr/bin/clickhouse
    26.843270 [ 39 ] {} <Fatal> BaseDaemon: 8. Poco::Util::Application::run() @ 0x11b7a313 in /usr/bin/clickhouse
    26.843284 [ 39 ] {} <Fatal> BaseDaemon: 9. DB::Server::run() @ 0x8545ebc in /usr/bin/clickhouse
    26.843299 [ 39 ] {} <Fatal> BaseDaemon: 10. mainEntryClickHouseServer(int, char**) @ 0x8544b25 in /usr/bin/clickhouse
    26.843313 [ 39 ] {} <Fatal> BaseDaemon: 11. main @ 0x84e1cbe in /usr/bin/clickhouse
    26.843331 [ 39 ] {} <Fatal> BaseDaemon: 12. __libc_start_main @ 0x270b3 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    26.843346 [ 39 ] {} <Fatal> BaseDaemon: 13. _start @ 0x84ac22e in /usr/bin/clickhouse
 2021-01-31 11:29:46 +03:00
Alexey Milovidov
d604cbc986 Maybe improve server latency 2021-01-31 10:41:24 +03:00
Pavel Kruglov
25e85d71ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-01-29 21:08:47 +03:00
Alexander Tokmakov
004517009e fix 2021-01-29 18:13:27 +03:00
alesapin
bac8cc55d2 Now we answer from follower nodes 2021-01-29 15:39:04 +03:00
Alexander Tokmakov
ffaa8e34a6 minor code improvements around ThreadStatus 2021-01-28 16:57:36 +03:00
alesapin
a33963e211 Better raft server startup 2021-01-27 20:54:25 +03:00
kreuzerkrieg
29a2ef3089 Add IStoragePolicy interface 2021-01-26 10:55:28 +02:00
alesapin
61d006cbab Fix typo 2021-01-26 11:18:00 +03:00
alesapin
dea4b5009b Some server initialization 2021-01-25 15:29:12 +03:00
alesapin
173b6fefdf Merge branch 'master' into in_memory_raft 2021-01-25 12:03:28 +03:00
alesapin
7d4945a073
Merge pull request #19412 from ClickHouse/fix_one_more_tiny_race 
Fix one more race in TestKeeper
 2021-01-22 23:04:01 +03:00
alexey-milovidov
b6822bfd04
Merge pull request #19365 from ClickHouse/tavplubix-patch-3 
Better logging in MySQLHandler
 2021-01-22 17:29:59 +03:00
alesapin
4a06bf1d79 Merge branch 'fix_one_more_tiny_race' into in_memory_raft 2021-01-22 15:56:43 +03:00
alesapin
cc2788af88 Fix typo 2021-01-22 13:48:40 +03:00
alesapin
b2a4921558 Fix style 2021-01-22 13:14:53 +03:00
alesapin
83862799cd Fix one more race in TestKeeper 2021-01-22 13:02:48 +03:00
Alexey Milovidov
afc6f2fd8e Add http_referer to client info 2021-01-22 01:55:45 +03:00
alesapin
4aa11b3494 Remove zkutil namespace from TestKeeperStorage 2021-01-21 18:09:48 +03:00
tavplubix
600862f4a2
Better logging in MySQLHandler 2021-01-21 18:00:58 +03:00
alesapin
b8cdd8e375 Merge branch 'fix_race_test_keeper' into in_memory_raft 2021-01-21 15:45:41 +03:00
alesapin
7f32926a21 Fix race condition in TestKeeperHandler on session finish 2021-01-21 14:37:20 +03:00
alesapin
a241793a14 Merge branch 'decompose_test_keeper_storage' into in_memory_raft 2021-01-20 15:51:54 +03:00
alesapin
70679e4ee1 Fix test keeper handler 2021-01-20 15:11:26 +03:00
Pavel Kruglov
97b5179e55 Implement HedgedRequests 2021-01-19 22:41:05 +03:00
alesapin
374cee47e0 Fix typo 2021-01-19 19:22:40 +03:00
alesapin
56f19e4790 Remove unused headers 2021-01-19 18:52:28 +03:00
alesapin
6c6bf60937 Rename function 2021-01-19 18:23:32 +03:00
alesapin
0ee5629527 Fix style 2021-01-19 18:10:49 +03:00
alesapin
ace6d906b0 Fix normal close scenario 2021-01-19 17:45:45 +03:00
alesapin
ac64a13392 Split storage and requests processing 2021-01-19 17:22:28 +03:00
alesapin
6896558424 Some code movements 2021-01-18 19:03:26 +03:00
Alexey Milovidov
438f0f971b Fix the issue with async Distributed INSERTs and network_compression_method #18741 2021-01-06 03:24:42 +03:00
John Skopis
9bcc5a6175 Support interserver credential rotation 
Restarting a server instance to change the interserver password results
in many replicas being out of sync until all clusters are using the new
credential.

This commit adds dynamic credential loading for both the client
(Replicated* tables) and server (InterserverIOHTTPHandler).

This commit also adds the ability to rotate credentials, i.e. accept more
than one credential during a credential change.

state0 (no auth):

    <interserver_http_credentials />

state1 (auth+allow_empty migration):

    <interserver_http_credentials>
        <user>admin</user>
        <password>222</password>
        <allow_empty>true</allow_empty>
    </interserver_http_credentials>

state2 (auth+new admin password migration):

    <interserver_http_credentials>
        <user>admin</user>
        <password>333</password>
        <users>
            <admin>222</admin>
        </users>
    </interserver_http_credentials>
 2021-01-04 09:59:09 +00:00
Nikita Mikhailov
c5f92e5096 better formatfactory 2020-12-30 06:07:30 +03:00
Yatsishin Ilya
f36e8f13c0 more fixes after rebase 2020-12-24 13:48:10 +03:00
Yatsishin Ilya
ce1e2b18a3 Fix codespell warnings. Split style checks. Update style checks docker 2020-12-24 13:17:52 +03:00
alexey-milovidov
021aba5769
Merge pull request #18217 from ClickHouse/fix_stranges_in_testkeeper 
Fix watches processing in TestKeeper
 2020-12-19 03:29:51 +03:00
alesapin
24eed37315 Fix style check 2020-12-18 12:23:27 +03:00
alesapin
3937942fae Fix bug in watches processing 2020-12-18 12:13:50 +03:00
Amos Bird
c447628d42
Add connection thread metrics 2020-12-18 15:22:55 +08:00
alesapin
b347fa322c Trying to fix races one more time 2020-12-16 15:58:00 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers 
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
 2020-12-12 17:43:10 +03:00
filimonov
8d9ad60afa
Decrease verbosity of disconnecting clients 
Right now they go to error log.
 2020-12-11 11:51:48 +01:00
alesapin
850f903d40 Merge branch 'master' into complete_zk_api 2020-12-09 18:18:38 +03:00
alesapin
758dcd1972 Separate factory for test keeper handlers 2020-12-09 11:45:36 +03:00
alesapin
7ff50fb352 Close requests without timeout 2020-12-09 11:19:52 +03:00
Vitaly Baranov
f6f336ac78
Merge pull request #17888 from vitlibar/retrieve-logs-from-grpc 
Retrieve logs from grpc
 2020-12-08 20:54:02 +03:00
alesapin
3d0dbbe411 Fix build 2020-12-08 17:49:18 +03:00
alesapin
91e3162e09 More correct session close 2020-12-08 17:28:51 +03:00
alesapin
f4231d552e Banch of fixes 2020-12-08 16:28:39 +03:00
Vitaly Baranov
559afe028f Retrieve logs from grpc library and pass them to our logging system. 2020-12-07 23:39:55 +03:00
Alexey Milovidov
1ed329017b Merge branch 'master' into quota-by-x-forwarded-for 2020-12-07 18:22:19 +03:00
alesapin
278bbf6d51 Merge branch 'master' into complete_zk_api 2020-12-04 16:48:19 +03:00
Alexey Milovidov
50fe5f9211 Added note 2020-12-03 00:09:46 +03:00
Alexey Milovidov
3e2447391b Support for PROXY protocol 2020-12-03 00:05:51 +03:00
alexey-milovidov
85f4045160
Merge pull request #17144 from fibersel/issue-16791 
introduce zstd compression (for data import/export)
 2020-12-02 22:41:07 +03:00
Alexey Milovidov
2c7b03ab6e Fix error 2020-12-02 22:20:47 +03:00
alesapin
93ea2f5b1f Merge branch 'master' into complete_zk_api 2020-12-02 11:30:09 +03:00
Alexey Milovidov
c9aa412151 Allow quotas to be keyed by proxy-forwarded IP address 2020-12-02 00:09:16 +03:00
Alexander Kuzmenkov
c5a81f6172
Merge pull request #17437 from vitlibar/add-new-constants-to-client-info-interface 
Add constants to ClientInfo::Interface for MySQL and PostgreSQL protocols
 2020-11-30 17:02:30 +03:00
a.palagashvili
08b8eb8fc2 resolved conflict, added new compression method in http 2020-11-29 15:59:56 +03:00
a.palagashvili
98ee8e8d5e resolved conflict in src/Server/HTTPHandler.cpp 2020-11-29 15:48:30 +03:00
a.palagashvili
502355922d added errors checking, removed redundant variables 2020-11-29 15:14:42 +03:00
alesapin
00c190aeed Fix unitialized memory 2020-11-27 14:57:48 +03:00
alesapin
d199a2bf76 Merge branch 'master' into complete_zk_api 2020-11-27 14:55:55 +03:00
alesapin
89c292f5b7 Fix response pipe read 2020-11-27 13:12:11 +03:00
alesapin
15a76b4763 Fix clang-tidy 2020-11-26 15:28:54 +03:00
alesapin
fc0bca1ff2 Fix bug with buffered read 2020-11-26 15:26:41 +03:00
alesapin
0b079cdeb8 Better polling 2020-11-26 11:59:23 +03:00
Vitaly Baranov
99343bc127 Add constants to ClientInfo::Interface for MySQL and PostgreSQL protocols. 2020-11-26 11:30:38 +03:00
alesapin
6f6476ad95 Remove complex code on session timeout 2020-11-26 00:00:51 +03:00
alesapin
77651dd239 Don't epoll_create each poll 2020-11-25 23:42:23 +03:00
alesapin
fd67864e15 Remove double space 2020-11-25 17:34:13 +03:00
alesapin
fb86eaf6fc Better errors and better config 2020-11-25 16:19:09 +03:00
alesapin
f3555ad2d4 Fix build on non-linux platforms 2020-11-25 15:36:46 +03:00
alesapin
d277db52fd Maybe unused 2020-11-25 15:14:31 +03:00
Alexander Kuzmenkov
edce1e636e
Merge pull request #16535 from ClickHouse/aku/opentelemetry 
OpenTelemetry improvements
 2020-11-25 14:10:17 +03:00
alesapin
8acbf75dc9 Fix segfault 2020-11-25 11:56:40 +03:00
alesapin
310307fa72 Replace eventfd with pipes 2020-11-25 11:18:15 +03:00
alesapin
4d40d9be3f Merge branch 'master' into complete_zk_api 2020-11-25 10:31:16 +03:00
Alexander Kuzmenkov
1d7dcef4a0 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-24 21:45:10 +03:00
Vitaly Baranov
be056a93ab Add comments desribing the protocol. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
a0e384b0c0 Added support for external tables. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
23842e7ac6 Added support for input() function. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
0e3a8840b5 Support cancellation of executing query via gRPC. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
18ebea5d66 Optimization: Forward reading of next query infos. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
eab3006a4d Support SSL and compression when using gRPC protocol. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
b0cb3eb306 Extend the protocol with streaming and nonstreaming functions. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
b51e14253d Added support for sessions in gRPC protocol. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
9285f7edc1 Send profile info to client. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
98e2cc4117 Fix sending progress. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
4f0405af93 Send logs via gRPC protocol too. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
218d9ea3e8 Added input_data_delimiter to protocol. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
797c84889f Use ReadBuffer to join input data from multiple query infos. 
That fixes inserting of a row splitted between two query infos.
 2020-11-24 17:55:01 +03:00
Vitaly Baranov
de4586739a Fix using output format. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
1bab6a2337 Added logging for grpc calls. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
8146093794 Provide more information about errors. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
fb90e9d091 Simplify the code: pass callbacks as tags to grpc. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
ba723d6d75 Simplify the code: get rid of WriteBufferFromGRPC. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
c2edd9f8ce Rework async server: switch to coroutine-like approach, no using statuses anymore for control flow. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
ff62fd4967 Add an adapter for protocol servers. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
a327f24e3c Refine the protocol. Code cleanup in tests. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
44717797c9 Fix code style and compilation. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
13f2352c57 Move files to right places. 2020-11-24 17:53:34 +03:00
alesapin
0d52cfb1be Less clumsy code with interruptable code 2020-11-24 17:02:55 +03:00
alesapin
c1a7e4f5fa Merge branch 'master' into complete_zk_api 2020-11-24 13:37:51 +03:00
zhang2014
ade04b5dc4 ISSUES-16605 try fix review comment 2020-11-22 20:42:55 +08:00
zhang2014
6b2fa22000 ISSUES-16605 try fix MySQL handler affected rows when insert select query 2020-11-22 20:13:55 +08:00
alesapin
73ed05492b More correct initialization 2020-11-20 11:37:16 +03:00
alesapin
5925760892 Fix handler 2020-11-19 19:31:55 +03:00
alesapin
8e3f7e4dbd Some non working code 2020-11-19 19:06:19 +03:00
Alexander Kuzmenkov
6cb378e072 cleanup 2020-11-19 18:52:11 +03:00
Alexander Kuzmenkov
1570320e20 fixes for context hierarchy 2020-11-18 20:43:18 +03:00
alesapin
2cd66f23a3 Merge branch 'master' into complete_zk_api 2020-11-18 17:50:50 +03:00
Alexander Kuzmenkov
b16c5a1748 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-17 22:54:54 +03:00
a.palagashvili
b737a5ea9f added newlines, fixed formatting 2020-11-17 21:36:18 +03:00
a.palagashvili
6f5390cc70 introduce zstd compression 2020-11-17 19:02:10 +03:00
Alexey Milovidov
24f4fa6edf Follow Arcadia ya.make rules 2020-11-17 00:16:50 +03:00
Alexey Milovidov
3df04ce0c2 Follow Arcadia ya.make rules 2020-11-16 21:24:58 +03:00
alesapin
4aad6a597c Merge branch 'correctly_send_close_request' into complete_zk_api 2020-11-12 17:46:02 +03:00
alexey-milovidov
34b2a460ba
Merge pull request #16578 from fibersel/issue-8828 
add compression method for files: Xz
 2020-11-12 02:58:52 +03:00
alesapin
d83c68fca8 Fix timeouts 2020-11-11 16:55:28 +03:00
alesapin
66236d6ebb Refactored a little 2020-11-11 16:07:06 +03:00
a.palagashvili
55d05c95bf fixed style, xz check fasttest skipped, removed fast-lzma2 2020-11-11 15:34:28 +03:00
alesapin
0388006ed6 Less garbage 2020-11-11 12:05:45 +03:00
alesapin
bb3b420057 Ugly working code 2020-11-11 11:51:54 +03:00
a.palagashvili
124ef2f5fa added and successfully passed tests for content encoding and file() function, formatted code 2020-11-11 04:50:56 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
alesapin
d8f515c4f3 Almost watches 2020-11-10 16:43:10 +03:00
alesapin
932ea0ef8b Merge branch 'master' into complete_zk_api 2020-11-10 13:56:11 +03:00
Alexander Kuzmenkov
26229ed231 tmp spans for threads 
(doesn't compile because of json metadata changes)
 2020-11-09 18:07:38 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
alesapin
a9529e8d65 Bad code but all events 2020-11-04 21:54:55 +03:00
alesapin
93c2ad6034 Something work 2020-11-03 17:49:30 +03:00
alesapin
a02b4322b6 Merge branch 'master' into complete_zk_api 2020-11-03 13:13:07 +03:00
alesapin
1fc7633289 Heartbeats working 2020-10-30 22:57:30 +03:00
alesapin
c2525ef211 Server and client pinging each other 2020-10-30 17:16:47 +03:00
Maxim Akhmedov
3627fabfb9 Remove -g0 form Arcadia build settings. 2020-10-29 17:37:23 +03:00
alexey-milovidov
bf3acecf6f
Merge pull request #16032 from Avogar/mysql_bug 
Don't touch MySQL database if it's unnecessary
 2020-10-29 04:14:26 +03:00
Alexander Kuzmenkov
5a70e26c70 Merge remote-tracking branch 'origin/master' into HEAD 2020-10-26 19:21:13 +03:00
Alexey Milovidov
74558a4e27 Better diagnostics when client has dropped connection 2020-10-24 06:41:47 +03:00
Alexander Kuzmenkov
5cbf645de4 Merge remote-tracking branch 'origin/master' into tmp 2020-10-22 19:48:02 +03:00
alexey-milovidov
2ae32e3d5c
Merge pull request #16101 from azat/http-memory-tracking-fixes 
Fix memory tracker settings (for HTTP queries)
 2020-10-22 00:00:47 +03:00
Alexander Kuzmenkov
84908df6d8 fix setting parent from HTTP traceparent 2020-10-20 14:35:13 +03:00
Alexander Kuzmenkov
f3abb76e57 disable traceparent header in Arcadia 
it interferes with the test_clickhouse.TestTracing.test_tracing_via_http_proxy[traceparent] test
 2020-10-20 11:13:21 +03:00
Alexey Milovidov
a79460b644 Modify comment 2020-10-20 04:03:03 +03:00
Alexey Milovidov
a091892a13 Fix Arcadia 2020-10-20 04:01:47 +03:00
Alexey Milovidov
a554ca5e62 Added /play handler; added a test 2020-10-20 03:28:33 +03:00
Alexander Kuzmenkov
41ac15a71c fix initial query id 2020-10-20 00:26:10 +03:00
Azat Khuzhin
fea788480f Fix QueryScope initialization for HTTPHandler 
This will fix incorrect query_id for the per-thread memory tracker (but
later there will be child tracker, so it is not a big deal).
 2020-10-17 08:55:02 +03:00
Pavel Kruglov
ee8a9d2cf1 Don't touch MySQL database if it's unnecessary 2020-10-15 18:57:17 +03:00
Alexander Kuzmenkov
6200433e12 Merge origin/master into tmp (using imerge) 2020-10-12 21:15:31 +03:00
Nikolai Kochetov
190fd88af9
Merge pull request #15785 from amosbird/e1 
Code refactor.
 2020-10-12 00:36:36 +03:00
Alexey Milovidov
7502fad77c Change level of some log messages 2020-10-10 20:47:34 +03:00
Amos Bird
8e0862a0c3
BlockIO can be empty. 2020-10-10 23:25:50 +08:00
Amos Bird
dbc14655fc
Code refactoring. 2020-10-10 18:22:33 +08:00
Alexander Kuzmenkov
dde4cf70e1 Merge remote-tracking branch 'origin/master' into HEAD 2020-09-22 14:03:59 +03:00
Alexander Kuzmenkov
7b64ca33b1 Merge remote-tracking branch 'origin/master' into tmp 2020-09-22 14:02:20 +03:00
Alexander Kuzmenkov
bfc3be9e43 Merge remote-tracking branch 'origin/aku/tcp-versio' into tmp 2020-09-22 14:02:15 +03:00
alexey-milovidov
3e0509af69
Merge pull request #13573 from amosbird/rms 
specific ReplicatedMergeTree settings
 2020-09-21 01:36:33 +03:00
Vitaly Baranov
46ab2f76cd
Merge pull request #14919 from vitlibar/add-login-function-to-access-storage 
Add login() function to IAccessStorage
 2020-09-20 16:58:22 +03:00
Vitaly Baranov
02acedd6a3 Use initial_address for quota calculation when the interserver-secret mode enabled. 2020-09-19 01:04:25 +03:00
Alexander Kuzmenkov
fd33fa7f39 fixup 2020-09-17 17:55:41 +03:00
Alexander Kuzmenkov
fb64cf210a straighten the protocol version 2020-09-17 17:37:29 +03:00
Vitaly Baranov
3356d75b23
Merge pull request #13156 from azat/cluster-secure 
Secure inter-cluster query execution (with initial_user as current query user) [v3]
 2020-09-17 17:11:00 +03:00
Alexander Kuzmenkov
a374541214 straighten the protocol version 2020-09-17 15:15:05 +03:00
Alexander Kuzmenkov
7b8ad02a12 cleanup 2020-09-17 15:14:49 +03:00
Alexander Kuzmenkov
6217dd77b5 Merge remote-tracking branch 'origin/master' into tmp 2020-09-17 12:35:10 +03:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
Azat Khuzhin
785d1b2a75 OpenSSLHelpers cleanup 
Add few more specializations for encodeSHA256():
- std::string encodeSHA256(const std::string_view &);
- std::string encodeSHA256(const void *, size_t);
- void encodeSHA256(const void *, size_t, unsigned char *);
 2020-09-15 01:36:28 +03:00
Azat Khuzhin
0159c74f21 Secure inter-cluster query execution (with initial_user as current query user) [v3] 
Add inter-server cluster secret, it is used for Distributed queries
inside cluster, you can configure in the configuration file:

  <remote_servers>
      <logs>
          <shard>
              <secret>foobar</secret> <!-- empty -- works as before -->
              ...
          </shard>
      </logs>
  </remote_servers>

And this will allow clickhouse to make sure that the query was not
faked, and was issued from the node that knows the secret. And since
trust appeared it can use initial_user for query execution, this will
apply correct *_for_user (since with inter-server secret enabled, the
query will be executed from the same user on the shards as on initator,
unlike "default" user w/o it).

v2: Change user to the initial_user for Distributed queries if secret match
v3: Add Protocol::Cluster package
v4: Drop Protocol::Cluster and use plain Protocol::Hello + user marker
v5: Do not use user from Hello for cluster-secure (superfluous)
 2020-09-15 01:36:28 +03:00
Azat Khuzhin
ec9d7745f3 Break the query loop on UNEXPECTED_PACKET_FROM_CLIENT 2020-09-15 01:36:28 +03:00
Azat Khuzhin
d53bbd3394 Not only send the exception to the client but also log it 
Otherwise there will be only tricky "Unknown packet" exception after
exception during handling INSERT queries.
 2020-09-15 01:36:28 +03:00
Alexander Kuzmenkov
c62bf555d2 Merge origin/master into tmp (using imerge) 2020-09-08 16:26:38 +03:00
Alexander Kuzmenkov
d0a9926e7d fixes and some docs 2020-09-08 16:19:27 +03:00
Alexey Milovidov
e3924b8057 Fix "Arcadia" 2020-09-08 01:14:13 +03:00
BohuTANG
c094ff7eca ISSUES-9336 support MySQL handler 'set @@xx' settting 2020-09-04 12:51:15 +08:00
alesapin
10c7a6c45e
Add ability to specify Default codec for columns (#14049) 
* Add ability to specify DefaultCompression codec which correspond to settings specified in config.xml

* Fix style

* Rename DefaultCompression to simple Default

* Fix compression codec

* Better codec description representation

* Less strange code and one method

* Fix delta
 2020-08-28 20:40:45 +03:00
Alexander Kuzmenkov
4612794678 opentelemetry context propagation 2020-08-28 04:21:08 +03:00
Amos Bird
05a5a13e08
specific ReplicatedMergeTree settings 2020-08-27 22:37:41 +08:00
alexey-milovidov
75a3bbf5b2
Merge pull request #13722 from javisantana/fix/replicas_status_verbose 
return 200 when replicas status is ok and verbose = 1
 2020-08-26 00:35:47 +03:00
Javi Santana
ce64a73342 fixed tests and style 2020-08-17 09:46:38 +02:00
Javi Santana
c6780148ec return 200 when replicas status is ok and verbose = 1 2020-08-14 14:29:16 +02:00
zhang2014
96bd3ac34b ISSUES-4006 split mysql protocol 2020-08-13 20:41:36 +08:00
zhang2014
34f4c8972e ISSUES-4006 split replication packets 2020-08-13 20:07:02 +08:00
zhang2014
c76d7e3121 ISSUES-4006 split generic packets 2020-08-13 18:17:13 +08:00
zhang2014
c0d42b764e ISSUES-4006 split protocol text packet 2020-08-13 17:20:45 +08:00
zhang2014
4fcc178deb ISSUES-4006 split mysql protocol 2020-08-13 16:17:33 +08:00
zhang2014
688836cdc4 ISSUES-4006 split msql protocol & fix build 2020-08-13 14:30:29 +08:00
tavplubix
4aff48dc3f
Merge pull request #10851 from zhang2014/feature/materialize_mysql_database 
ISSUES-4006 support MaterializeMySQL database engine
 2020-08-12 15:59:28 +03:00
Alexey Milovidov
edd89a8610 Fix half of typos 2020-08-08 03:47:03 +03:00
zhang2014
1df8bfe4fb ISSUES-4006 fix build failure after refactor 2020-08-04 10:31:25 +08:00
Alexey Milovidov
6f690b7c0d Normalize ya.make files, fix "Arcadia" build 2020-08-02 16:57:38 +03:00
Vitaly Baranov
56665a15f7 Rework and rename the template class SettingsCollection => BaseSettings. 2020-07-31 20:54:18 +03:00
Nikolai Kochetov
a767b59f41
Merge pull request #12981 from hczhcz/patch-0728 
Add http headers X-ClickHouse-Database and X-ClickHouse-Format
 2020-07-31 14:19:54 +03:00
hcz
486d63864b Add X-ClickHouse-Database and X-ClickHouse-Format 2020-07-29 14:03:57 +08:00
Gleb Novikov
7f5b6fba78 Generic volume is coming... 
1. SingleDiskVolume for temporary volumes
2. Generic VolumePtr in StoragePolicies
3. Removed max_data_part_size in system.storage_policies, added volume_type
 2020-07-26 21:27:36 +03:00
alexey-milovidov
b5e87f5b25
Revert "Abort on std::out_of_range in debug builds" 2020-07-24 18:12:03 +03:00
Azat Khuzhin
eaaa4543f4 Abort on std::out_of_range in debug builds 
This helps notice them in tests, same as:
- std::logic_error
- LOGICAL_ERROR
 2020-07-23 11:36:12 +03:00
Alexander Kuzmenkov
a4a23673cc fixup 2020-07-17 17:25:13 +03:00
Alexander Kuzmenkov
4089f761e1 fix one out_of_range 2020-07-17 17:20:33 +03:00
Alexander Kuzmenkov
fb7743d8f5 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-17 16:19:05 +03:00
Alexander Kuzmenkov
9d728d60d0 Abort on std::logic_error in debug builds 
This helps notice them in tests, same as assert() and LOGICAL_ERROR.
 2020-07-15 15:08:01 +03:00
Denis Glazachev
f787702922 Merge branch 'master' into ldap-per-user-authentication 
* master: (27 commits)
  Whitespaces
  Fix typo
  Fix UBSan report in base64
  Correct default secure port for clickhouse-benchmark #11044
  Remove test with bug #10697
  Update in-functions.md (#12430)
  Allow nullable key in MergeTree
  Update arithmetic-functions.md
  [docs] add rabbitmq docs (#12326)
  Lower block sizes and look what will happen #9248
  Fix lifetime_bytes/lifetime_rows for Buffer direct block write
  Retrigger CI
  Fix up  test_mysql_protocol failed
  Implement lifetime_rows/lifetime_bytes for Buffer engine
  Add comment regarding proxy tunnel usage in PocoHTTPClient.cpp
  Add lifetime_rows/lifetime_bytes interface (exported via system.tables)
  Tiny IStorage refactoring
  Trigger integration-test-runner image rebuild.
  Delete log.txt
  Fix test_mysql_client/test_python_client error
  ...
 2020-07-13 15:46:27 +04:00
Denis Glazachev
edb6ef8c09 Merge commit 'ceac649c01b0158090cd271776f3219f5e7ff57c' into ldap-per-user-authentication 
* commit 'ceac649c01b0158090cd271776f3219f5e7ff57c': (75 commits)
  [docs] split misc statements (#12403)
  Update 00405_pretty_formats.reference
  Update PrettyCompactBlockOutputFormat.cpp
  Update PrettyBlockOutputFormat.cpp
  Update DataTypeNullable.cpp
  Update 01383_remote_ambiguous_column_shard.sql
  add output_format_pretty_grid_charset setting in docs
  add setting output_format_pretty_grid_charset
  Added a test for #11135
  Update index.md
  RIGHT and FULL JOIN for MergeJoin (#12118)
  Update MergeTreeIndexFullText.cpp
  restart the tests
  [docs] add syntax highlight (#12398)
  query fuzzer
  Fix std::bad_typeid when JSON functions called with argument of wrong type.
  Allow typeid_cast() to cast nullptr to nullptr.
  fix another context-related segfault
  [security docs] actually, only admins can create advisories
  query fuzzer
  ...
 2020-07-11 21:32:36 +04:00
alexey-milovidov
ef41ef3162
Update MySQLHandler.cpp 2020-07-10 21:03:12 +03:00
BohuTANG
86590d4aca Change the exception catch to the generic way 2020-07-10 17:18:00 +08:00
BohuTANG
64fdce9398 Write through Poco::Exception exception to MySQL protocol client 2020-07-10 12:54:08 +08:00
BohuTANG
260bcb9d79 Add integration test for mysql replacement query 2020-07-09 22:20:54 +08:00
BohuTANG
5ca3aef919 Support MySQL 'SELECT DATABASE()' query replacement 2020-07-09 11:07:20 +08:00
Denis Glazachev
9effacfbc1 Merge branch 'master' into ldap-per-user-authentication 
* master: (1102 commits)
  Update README.md
  Update README.md
  Update README.md
  Update index.md
  [docs] add intrdocution for statements page (#12189)
  Revert "Run perf tests with memory sampling (for allocations >1M)"
  Sanitize LINK_LIBRARIES property for the directories (#12160)
  [docs] refactor Domains overview (#12186)
  DOCS-647: toStartOfSecond (#12190)
  [docs] add intrdocution for commercial page (#12187)
  DOCSUP-1348 Russian translation for new functions (#133) (#12194)
  changelog fixes
  Update index.md (#12191)
  Update zh kafka.md title (#12192)
  Added test for #3767
  style fix for #12152
  Tests for fixed issues #10846 and #7347
  changelog fixes
  [docs] introduction for special table engines (#12170)
  [docs] introduction for third-party interfaces (#12175)
  ...

# Conflicts:
#	src/Access/ya.make
#	src/Common/ErrorCodes.cpp
 2020-07-08 00:42:09 +04:00
Alexander Kuzmenkov
688f185f9e style fix for #12152 2020-07-07 13:28:45 +03:00
BohuTANG
cb04c503d7 Add MySQL to ClickHouse query replacement mapping table 2020-07-06 10:07:38 +08:00
BohuTANG
37ac456439 Support KILL QUERY [connection_id] for MySQL 2020-07-06 09:02:02 +08:00
BohuTANG
53997f23e6 Merge remote-tracking branch 'ck/master' into mysql_global_variables 2020-06-25 14:04:10 +08:00
BohuTANG
4d96a25655 Change MySQL global variables query to globalVariable function 2020-06-24 13:40:18 +08:00
alesapin
4185fe9975 Merge branch 'master' into atomic_metadata5 2020-06-23 11:13:05 +03:00
alexey-milovidov
1c1e12f06b
Merge pull request #11826 from ClickHouse/send-logs-on-fatal-errors 
Send logs to client on fatal errors if possible
 2020-06-23 09:49:48 +03:00
alesapin
b9e74f4e82 Merge branch 'master' into atomic_metadata5 2020-06-22 12:03:53 +03:00
Alexey Milovidov
71317dd59c Merge branch 'master' into send-logs-on-fatal-errors 2020-06-21 19:05:20 +03:00
Azat Khuzhin
844960195d Fix memory accounting via HTTP interface 
function perf_test()
{
    time yes '127.1:8123/?wait_end_of_query=1' | head -n10000 | xargs -P10000 curl -s -d 'select 1' | grep -x -c 1
}
function server()
{
    local limit=$1 && shift
    clickhouse-server "$@" -- --max_server_memory_usage=$limit
}

perf_test:

- before this patch with 1G  limit: succeed only ~800  queries
- after  this patch with 1G  limit: succeed      ~8000 queries

- before this patch with 10G limit: succeed only ~3000  queries
- after  this patch with 10G limit: succeed      ~10000 queries

Fixes: #11153
 2020-06-21 17:27:54 +03:00
Alexey Milovidov
3f96bdaa24 Minor changes after merge 2020-06-21 15:16:08 +03:00
alexey-milovidov
4ee623ccac
Merge pull request #10242 from MovElb/movelb-postgresql-wire-protocol-impl 
PostgreSQL wire protocol implementation
 2020-06-21 14:39:22 +03:00
Alexey Milovidov
7ba25bd799 Make it work 2020-06-20 14:17:15 +03:00
Denis Glazachev
5db60202b6 Merge branch 'master' into ldap-per-user-authentication 2020-06-19 00:11:08 +04:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
alesapin
1ddeb3d149 Buildable getSampleBlock in StorageInMemoryMetadata 2020-06-16 18:51:29 +03:00
alesapin
eae901f58d Merge branch 'master' into atomic_metadata5 2020-06-16 14:44:45 +03:00
alesapin
36ba0192df Metadata in read and write methods of IStorage 2020-06-15 22:08:58 +03:00
Alexander Tokmakov
def0158638 configure query handler as default 2020-06-15 14:33:44 +03:00
Winter Zhang
dc6f904f0a
Merge branch 'master' into feature/ISSUES-7572 2020-06-14 18:28:06 +08:00
alexey-milovidov
da3a075821
Merge pull request #11660 from bharatnc/bnc/fix-remove-unused-imports 
remove unused imports from HTTPHandlerFactory
 2020-06-14 09:43:52 +03:00
Bharat Nallan
4953b5fc84 remove unused imports from HTTPHandlerFactory 
This removes unused imports from `src/Server/HTTPHandlerFactory.cpp`:

```bash
- #include <re2/re2.h>
- #include <common/find_symbols.h>
- #include <Poco/StringTokenizer.h>
```
 2020-06-13 21:47:44 -07:00
Alexey Milovidov
970a8e3ecc Skip empty URL parameters #10749 2020-06-14 03:56:13 +03:00
zhang2014
de96296e01 ISSUES-7572 fix build failure 2020-06-13 10:17:02 +08:00
zhang2014
8d9b770da4 ISSUES-7572 fix defaults config level & add replicas_status and prometheus handler 2020-06-13 00:15:02 +08:00
zhang2014
2c439afc01 ISSUES-7572 fix build failure 2020-06-12 19:17:34 +08:00
zhang2014
80b765542f ISSUES-7572 support config default http handler 2020-06-12 18:04:42 +08:00
Denis Glazachev
9e3a28a6b8 Merge branch 'master' into ldap-per-user-authentication 
* master: (414 commits)
  Update file.md
  Update merge.md
  Update dictionary.md
  Update external-data.md
  Update distributed.md
  Update null.md
  Update set.md
  Update join.md
  Update url.md
  Update view.md
  Update materializedview.md
  Update memory.md
  Update buffer.md
  Update generate.md
  removed a sentence about global lock during rename (#11577)
  greatCircleAngle en translation (#11584)
  Update configuration-files.md
  try fix flacky test
  Update why.html
  Update rich.html
  ...

# Conflicts:
#	src/Common/ErrorCodes.cpp
#	utils/ci/jobs/quick-build/run.sh
 2020-06-11 03:06:17 +04:00
MovElb
ef5d03a273 fix build 2020-06-08 16:49:10 +03:00
MovElb
fc981a2869 fix for review 2020-06-08 15:05:49 +03:00
Alexander Kuzmenkov
435f53e05a Improve error messages 
Also support fmt for exception messages, the same way we use it in
logging macros.
 2020-06-04 20:50:29 +03:00
Denis Glazachev
9331387340 Merge branch 'master' into ldap-per-user-authentication 
* master: (29 commits)
  DOCS-620: Background pools settings (#11358)
  Update formats.md
  ya.make
  DOCS-611: background_buffer_flush_schedule_pool_size setting (#11176)
  Remove structure lock on parts send
  Update adopters.md
  This word "collapse" in Chinese means "折叠” (#11353)
  interval type link (#11351)
  Try fix tets.
  DOCS-573: system.data_type_families (#11344)
  fix flacky test with ttl
  Added test result
  Fix error with Lazy database
  Fix nullptr dereference in SYSTEM query and Lazy database
  Better code
  Added a test
  Fix unit test with UBSan
  Fix tests.
  Try fix build.
  Try fix build.
  ...
 2020-06-03 00:48:21 +04:00
Denis Glazachev
979fa13b26 Simplify condition 2020-06-03 00:38:00 +04:00
Denis Glazachev
c61cbe8e98 Merge branch 'master' into ldap-per-user-authentication 
* master: (114 commits)
  Update PushingToViewsBlockOutputStream.cpp
  Update PushingToViewsBlockOutputStream.cpp
  make clang-10 happy
  Fix sync_async test (remove timeout)
  CLICKHOUSEDOCS-631: temporary_files_codec, join_on_disk_max_files_to_merge settings. (#11242)
  Suppress output of cancelled queries in clickhouse-client #9473
  Better log messages in ConfigReloader
  fix select from StorageJoin
  Fix unit tests under MSan
  Added test.
  Fix build.
  Fix arguments for AggregateFunctionQuantile/
  Update style.md
  Add a guide on error messages.
  Report dictionary name on dictionary load errors.
  more types in ASOF JOIN (#11301)
  Fix part_log test
  Update test.
  Add perftest.
  Parallel processing for PushingToViewsBlockOutputStream::writeSuffix
  ...
 2020-06-02 12:42:29 +04:00
Alexey Milovidov
a6bf4b5d70 Fix error with Lazy database 2020-06-02 05:06:16 +03:00
MovElb
fd5b1741a6 build fix 2020-05-31 17:05:41 +03:00
Denis Glazachev
e8144976fe Merge branch 'master' into ldap-per-user-authentication 2020-05-31 15:09:16 +04:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
MovElb
142c63e487 done rebase 2020-05-30 23:02:11 +03:00
Alexey Milovidov
ffdec23d08 Remove wrong code #8441 2020-05-30 01:43:25 +03:00
tavplubix
a4e40fb5f2
Merge pull request #11269 from ClickHouse/add_context_to_idatabase_methods 
Add context to IDatabase methods
 2020-05-29 21:15:35 +03:00
Denis Glazachev
f1cfc7b472 Rename LDAP_PASSWORD to LDAP_SERVER and use "ldap_server" as a string key 
Some refactoring
 2020-05-29 11:47:01 +04:00
Denis Glazachev
f9d4136792 Merge branch 'master' into ldap-per-user-authentication 2020-05-29 10:52:44 +04:00
Alexey Milovidov
23d7947549 Fix constraints for DEFAULT fields; fix constraints for temporary tables; improve performance of INSERT for small blocks 2020-05-29 05:08:48 +03:00
Alexander Tokmakov
c071e0a473 add context to DatabaseCatalog methods 2020-05-29 02:01:18 +03:00
Alexander Tokmakov
3cff070f89 add context to IDatabase methods 2020-05-28 23:10:45 +03:00
Denis Glazachev
2863de750e Merge branch 'master' into ldap-per-user-authentication 2020-05-28 01:30:52 +04:00
Nikolai Kochetov
2dcee3db5c Merge branch 'master' into remove-experimental-use-processors-flag-3 2020-05-27 21:25:34 +03:00
Ivan
5b858de374
Split programs/server into actual program and library (#11186) 
* Split programs/server into actual program and library
 2020-05-27 20:52:52 +03:00