Commit Graph

688 Commits

Author SHA1 Message Date
alesapin
b2271cc2d9
Merge pull request #33288 from JackyWoo/add_lower_bound_session_timeout_to_keeper
Add lower bound session timeout to keeper
2022-01-17 14:06:15 +03:00
alesapin
ec6840f49c Make ZooKeeper client better interpret keeper server connection reject 2022-01-13 17:12:10 +03:00
Nikita Mikhaylov
340ec0fd20
Revert "Ignore parse failure of opentelemetry header (#32116)" (#33594)
This reverts commit 4e5f9c8ff7.
2022-01-13 16:29:18 +04:00
mergify[bot]
085492614b
Merge branch 'master' into add_lower_bound_session_timeout_to_keeper 2022-01-13 12:21:46 +00:00
Frank Chen
4e5f9c8ff7
Ignore parse failure of opentelemetry header (#32116) 2022-01-13 16:16:10 +04:00
Azat Khuzhin
e8c7e5e8f0 Fix typo in sendMergeTreeReadTaskRequestAssumeLocked() 2022-01-07 20:04:57 +03:00
JackyWoo
569ce62e8d use session_timeout as session timeout uper bound 2021-12-30 17:18:51 +08:00
JackyWoo
19b5394be8 fix tests 2021-12-30 15:49:48 +08:00
JackyWoo
d35e5f8319 add lower bound session timeout to keeper 2021-12-29 20:59:01 +08:00
Vitaly Baranov
f80a3e3247
Merge pull request #30549 from aiven/kmichel-server-reload
Start/stop servers when `listen_host`/`*_port` changes
2021-12-27 14:51:00 +03:00
Alexey Milovidov
29d28c531f Move code around to avoid dlsym on Musl 2021-12-24 12:25:27 +03:00
Kevin Michel
ffc1fca296
Start/stop servers when listen_host/*_port changes
This allows starting and stopping separately each protocol server
without restarting ClickHouse.

This also allows adding or removing `listen_host` entries, which
start and stops servers for all enabled ports.

When stopping a server, the listening socket is immediately closed
(and available for another server).

Protocols with persistent connections try to wait for any currently
running query to finish before closing the connection, but idle
connection are closed quickly (depending on how often the protocol
is polled).

An extra ProfileEvent is added, `MainConfigLoads`, it is
incremented every time the configuration is reloaded. This helps
when trying to assess whether the new configuration was applied.
2021-12-24 08:26:02 +01:00
alexey-milovidov
8c9843caf2
Merge pull request #32982 from azat/http-context-leak
Fix Context leak in case of cancel_http_readonly_queries_on_client_close
2021-12-23 07:58:18 +03:00
mergify[bot]
04f727e84c
Merge branch 'master' into read-fix 2021-12-22 13:49:04 +00:00
alesapin
754785fee5 Better container 2021-12-21 11:07:24 +03:00
Azat Khuzhin
08ced87880 Rework MultipartReadBuffer::skipToNextBoundary()
As suggested by @vitlibar
2021-12-21 08:38:41 +03:00
Azat Khuzhin
d7c7a91f66 Check that at least one boundary exist in HTMLForm::readMultipart() 2021-12-21 08:38:41 +03:00
Azat Khuzhin
4dfa9324d1 Check for EOF in MultipartReadBuffer::skipToNextBoundary()
Otherwise you may hit internal assertion when client goes away:

    clickhouse-server: ./src/Server/HTTP/HTMLForm.cpp:245: bool DB::HTMLForm::MultipartReadBuffer::skipToNextBoundary(): Assertion `boundary_hit' failed.

Fixes: 02151_http_s_structure_set_eof
2021-12-21 08:38:41 +03:00
JackyWoo
7a19570853 keeper handler should remove operation when response sent 2021-12-21 11:43:14 +08:00
Azat Khuzhin
a9f80dd705 Fix Context leak in case of cancel_http_readonly_queries_on_client_close
To handle cancel_http_readonly_queries_on_client_close=true context
attaches progress callback with holding a copy of the current context,
and this creates recursive context reference and so shared_ptr will be
never released, [1] contains simplest example.

One example of a memory leak, external tables passed with HTTP query
(s_structure + file upload), since they are stored in the context, and
can occupate enough memory to make it visible to user.

Note, that the lifetime of the Context should be fine, since callback
can be called only when query is executed.

  [1]: Example of recursive reference

       ```c
       #include <memory>
       #include <functional>
       #include <cassert>

       bool released = true;

       class Context : public std::enable_shared_from_this<Context>
       {
           using Func = std::function<void()>;
           Func func;

       public:
           Context()  { released = false; }
           ~Context() { released = true; }
           Context(const Context & context) = delete;

           void addCallback(Func func_)
           {
               func = func_;
           }
       };

       int main()
       {
           {
               auto context = std::make_shared<Context>();
               context->addCallback([context]() {});
           }
           assert(released == true);
       }
       ```
2021-12-20 21:05:38 +03:00
Azat Khuzhin
dd8c64e380 Set QueryScope earlier for HTTP queries 2021-12-20 21:05:38 +03:00
Vitaly Baranov
e61d3eef0c
Merge pull request #32747 from vitlibar/improve-grpc-compression
Improve gRPC compression support
2021-12-20 19:44:58 +03:00
Azat Khuzhin
6aebc3e94c Do not loose ProfileEvents in case of thread destroyed before
v2: drop std::move and add copy ctor for ProfileEvents::Counter::Snapshot
v2: remove std::move
2021-12-19 23:43:05 +03:00
Vitaly Baranov
708439b036 Support customized compression for input/output data in gRPC protocol. 2021-12-19 16:29:44 +03:00
Alexey Milovidov
2c68192bb6 Fix strange code in TCPHandler 2021-12-12 05:35:33 +03:00
Azat Khuzhin
57c027be0e Remove arcadia build support 2021-12-11 21:25:23 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
alexey-milovidov
96ec92c7cd
Merge pull request #31155 from ClickHouse/incremental-profile-events
Send incremental profile events to client
2021-12-01 04:04:41 +03:00
alesapin
c1f38752c4
Merge pull request #31859 from JackyWoo/keeper_timeout
keeper session timeout doesn't work
2021-11-28 16:28:17 +03:00
Raúl Marín
051dddd8df Reduce dependencies on ASTIdentifier.h
Goes from rebuilding 483 objects to 165 when it's modified
2021-11-26 16:49:40 +01:00
JackyWoo
4c6ad94109 fix ch keeper session timeout doesn't work 2021-11-26 18:17:49 +08:00
Kruglov Pavel
690b460c3d
Fix style 2021-11-24 22:36:36 +03:00
Kruglov Pavel
ad7768e42a
Fix 2021-11-24 22:25:41 +03:00
Kruglov Pavel
b20f4439df
Make better 2021-11-24 22:25:03 +03:00
avogar
1bd8c57908 Fix 2021-11-24 20:21:21 +03:00
avogar
7c84d95908 Fix possible Logical error: Cannot write to finalized buffer 2021-11-24 18:32:41 +03:00
alesapin
646cf38213
Merge pull request #28981 from JackyWoo/add_4_letter_words_commands
Add four letter commands to keeper
2021-11-24 13:32:36 +03:00
Kruglov Pavel
b63b47f0f0
Merge pull request #31265 from Avogar/fix-write-buffers
Fix and refactor WriteBiffer-s a little
2021-11-23 16:46:09 +03:00
Kruglov Pavel
6d942401bb
Merge pull request #31510 from azat/fix-progress-for-insert
Fix progress for short INSERT SELECT queries
2021-11-23 16:37:06 +03:00
alesapin
16f034fe4c Flush bytes after 4 letter commands 2021-11-23 13:31:10 +03:00
alesapin
5825242a7d Merge branch 'add_4_letter_words_commands' of github.com:JackyWoo/ClickHouse into JackyWoo_add_4_letter_words_commands 2021-11-22 18:02:22 +03:00
alesapin
35de9e42d9
Merge branch 'master' into add_4_letter_words_commands 2021-11-22 18:02:02 +03:00
alesapin
1cf204c581 Small changes 2021-11-21 15:28:45 +03:00
Alexander Tokmakov
856502fa81 log long operations in Keeper 2021-11-20 20:39:31 +03:00
Kruglov Pavel
d9c1a0c8ec
Merge branch 'master' into fix-write-buffers 2021-11-20 17:48:24 +03:00
Azat Khuzhin
dd7f780978 Fix Progress for short INSERT SELECT queries
Note, that we cannot send Progress for regular INSERT (w/ VALUES) w/o
breaking protocol compatibility, but it can be done by increasing
revision.
2021-11-20 16:41:35 +03:00
Azat Khuzhin
0b4d855c08 Reduce copy-paste in TCPHandler::processInsertQuery() 2021-11-20 16:41:35 +03:00
alesapin
3f8f08e816 Slightly better 2021-11-19 12:30:58 +03:00
alesapin
dcec086573 Small refactoring 2021-11-18 23:17:22 +03:00
JackyWoo
27d50edc59 adjust log output 2021-11-18 15:49:43 +08:00