Commit Graph

10 Commits

Author SHA1 Message Date
alesapin
d913f327ed Buildable code 2021-04-06 16:42:38 +03:00
alesapin
1c8e33414f Merge branch 'master' into johnskopis/dynamic-interserver-creds-v20 2021-04-06 16:28:46 +03:00
Azat Khuzhin
e1de960025 Fix uncaught exception in InterserverIOHTTPHandler
There was one more uncaught exception case [1]:

    2021.03.19 18:11:00.845632 [ 17469 ] {} <Debug> InterserverIOHTTPHandler: Done processing query
    ...
    2021.03.19 18:11:31.698961 [ 80145 ] {} <Fatal> BaseDaemon: ########################################
    2021.03.19 18:11:31.699903 [ 80145 ] {} <Fatal> BaseDaemon: (version 21.4.1.6293 (official build), build id: 859E400E1C65C4702FE491420741DD8B58190002) (from thread 17469) (no query) Received signal Aborted (6)
    2021.03.19 18:11:32.614075 [ 80145 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:89: std::terminate() @ 0x21e9b3a2 in /usr/bin/clickhouse
    2021.03.19 18:11:43.831215 [ 80145 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Server/HTTP/WriteBufferFromHTTPServerResponse.cpp:201: ? @ 0x1be77038 in /usr/bin/clickhouse
    2021.03.19 18:11:44.743193 [ 80145 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:892: std::__1::allocator<DB::WriteBufferFromHTTPServerResponse>::destroy(DB::WriteBufferFromHTTPServerResponse*) @ 0x1bddd7c9 in /usr/bin/clickhouse
    2021.03.19 18:11:45.283905 [ 80145 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:541: void std::__1::allocator_traits<std::__1::allocator<DB::WriteBufferFromHTTPServerResponse> >::__destroy<DB::WriteBufferFromHTTPServerResponse>(std::__1::integral_constant<bool, true>, std::__1::allocator<DB::WriteBufferFromHTTPServerResponse>&, DB::WriteBufferFromHTTPServerResponse*) @ 0x1bddd79d in /usr/bin/clickhouse
    2021.03.19 18:11:45.805233 [ 80145 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:487: void std::__1::allocator_traits<std::__1::allocator<DB::WriteBufferFromHTTPServerResponse> >::destroy<DB::WriteBufferFromHTTPServerResponse>(std::__1::allocator<DB::WriteBufferFromHTTPServerResponse>&, DB::WriteBufferFromHTTPServerResponse*) @ 0x1bddd76d in /usr/bin/clickhouse
    2021.03.19 18:11:46.351371 [ 80145 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2611: std::__1::__shared_ptr_emplace<DB::WriteBufferFromHTTPServerResponse, std::__1::allocator<DB::WriteBufferFromHTTPServerResponse> >::__on_zero_shared() @ 0x1bddd525 in /usr/bin/clickhouse
    2021.03.19 18:11:46.579263 [ 80145 ] {} <Fatal> BaseDaemon: 15. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2476: std::__1::__shared_count::__release_shared() @ 0x119490ed in /usr/bin/clickhouse
    2021.03.19 18:11:46.790912 [ 80145 ] {} <Fatal> BaseDaemon: 16. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2517: std::__1::__shared_weak_count::__release_shared() @ 0x1194908f in /usr/bin/clickhouse
    2021.03.19 18:11:47.277990 [ 80145 ] {} <Fatal> BaseDaemon: 17. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3213: std::__1::shared_ptr<DB::WriteBufferFromHTTPServerResponse>::~shared_ptr() @ 0x1bdd75fc in /usr/bin/clickhouse
    2021.03.19 18:11:47.649213 [ 80145 ] {} <Fatal> BaseDaemon: 18. ./obj-x86_64-linux-gnu/../src/Server/InterserverIOHTTPHandler.h:34: DB::InterserverIOHTTPHandler::Output::~Output() @ 0x1bdf6bd5 in /usr/bin/clickhouse
    2021.03.19 18:11:47.921556 [ 80145 ] {} <Fatal> BaseDaemon: 19. ./obj-x86_64-linux-gnu/../src/Server/InterserverIOHTTPHandler.cpp:154: DB::InterserverIOHTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x1bdf653f in /usr/bin/clickhouse

  [1]: https://clickhouse-test-reports.s3.yandex.net/0/78c56b891383288cf3a893139e796fc87476412e/stress_test_(debug).html

Since in case of no errors during processing we should call finalize, to
ensure that it will not be called from dtor.

Fixes: #22046
Fixes: #22067
2021-03-25 23:41:03 +03:00
Azat Khuzhin
48fe30e5a2 Add missing logging for exception in InterserverIOHTTPHandler 2021-03-25 23:41:03 +03:00
Azat Khuzhin
1ccb333ac5 Fix WriteBufferFromHTTPServerResponse usage in other places (add missing finalize())
Since I saw the following:

    0. DB::WriteBufferFromOStream::nextImpl()
    1. DB::WriteBufferFromHTTPServerResponse::nextImpl()
    2. DB::WriteBufferFromHTTPServerResponse::finalize()
    3. DB::WriteBufferFromHTTPServerResponse::~WriteBufferFromHTTPServerResponse()
    4. DB::StaticRequestHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&)
    5. Poco::Net::HTTPServerConnection::run()
    6. Poco::Net::TCPServerConnection::start()
2021-02-20 10:15:38 +03:00
Ivan
414f470c79
Make Poco HTTP Server zero-copy again (#19516)
* Refactoring: part 1

* Refactoring: part 2

* Handle request using ReadBuffer interface

* Struggles with ReadBuffer's

* Fix URI parsing

* Implement parsing of multipart/form-data

* Check HTTP_LENGTH_REQUIRED before eof() or will hang

* Fix HTTPChunkedReadBuffer

* Fix build and style

* Fix test

* Resist double-eof

* Fix arcadian build
2021-02-19 15:51:26 +03:00
John Skopis
9bcc5a6175 Support interserver credential rotation
Restarting a server instance to change the interserver password results
in many replicas being out of sync until all clusters are using the new
credential.

This commit adds dynamic credential loading for both the client
(Replicated* tables) and server (InterserverIOHTTPHandler).

This commit also adds the ability to rotate credentials, i.e. accept more
than one credential during a credential change.

state0 (no auth):

    <interserver_http_credentials />

state1 (auth+allow_empty migration):

    <interserver_http_credentials>
        <user>admin</user>
        <password>222</password>
        <allow_empty>true</allow_empty>
    </interserver_http_credentials>

state2 (auth+new admin password migration):

    <interserver_http_credentials>
        <user>admin</user>
        <password>333</password>
        <users>
            <admin>222</admin>
        </users>
    </interserver_http_credentials>
2021-01-04 09:59:09 +00:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
2020-12-12 17:43:10 +03:00
Alexey Milovidov
7502fad77c Change level of some log messages 2020-10-10 20:47:34 +03:00
Ivan
5b858de374
Split programs/server into actual program and library (#11186)
* Split programs/server into actual program and library
2020-05-27 20:52:52 +03:00