thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
94,952 Commits 9,315 Branches 1,783 Tags 2.1 GiB
627424889d
Commit Graph

867 Commits

Author SHA1 Message Date
Antonio Andelic
e15fccd31b Add is_secure info to HTTP 2022-03-30 06:39:40 +00:00
Antonio Andelic
d14ad227b1 Polish TCP is_secure flag 2022-03-30 06:39:40 +00:00
alesapin
70d410dc4d Fix keeper client timeout bug 2022-03-22 11:41:50 +01:00
Maksim Kita
2fdcf53a76 Fix clang-tidy warnings in Server, Storages folders 2022-03-14 18:17:35 +00:00
alesapin
96c0e9fddf Better cmake 2022-03-11 15:47:07 +01:00
alesapin
33ff2f76e5 Merge branch 'master' into standalone_keeper_build 2022-03-11 13:58:44 +01:00
Roman Nikonov
3b60116e6d Fix merge mistake 2022-03-10 08:57:31 +03:00
Roman Nikonov
83787b9651 Fix support content_type for dynamic_query_handler 2022-03-09 23:48:53 +03:00
alesapin
0c93c9e21d
Merge pull request #35010 from bigo-sg/keeperstatopt 
Keeper atomic stat
 2022-03-07 12:34:28 +01:00
Kseniia Sumarokova
cdb9a05229
Merge pull request #34605 from bigo-sg/add_metric_for_local 
Add cpu/mem metric for clickhouse-local
 2022-03-07 11:40:11 +01:00
Roman Nikonov
fde94e3cb2 Merge branch 'master' into pr/content-type 2022-03-07 13:19:33 +03:00
taiyang-li
1b9987ceb1 fix mistake in check-style 2022-03-05 14:22:56 +08:00
alesapin
0eb7d28192 Building small keeper binary 2022-03-03 21:27:46 +01:00
zhanglistar
9ee0d2c8a0 keeper atomic stat 2022-03-03 14:30:22 +08:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
taiyang-li
f83132bad2 finish dev 2022-03-01 15:54:23 +08:00
taiyang-li
b31440c77a Merge branch 'master' into add_metric_for_local 2022-03-01 12:29:36 +08:00
Filatenkov Artur
af4362e40a
Improve certificate reloader (#34887) 
* add ec support

* Add test

* fix tests and improve code style
 2022-02-28 21:30:02 +03:00
Roman Nikonov
db63b82e1a Support content_type in HTTP handler config 2022-02-26 03:06:17 +03:00
Vitaly Baranov
aee67a6693
Merge pull request #31484 from eungenue/Implement-SSL-X509-certificate-authentication 
Implement ssl x509 certificate authentication
 2022-02-21 11:30:52 +03:00
Vitaly Baranov
cb66a63aa4 Rename header and config setting for consistency. 2022-02-21 07:41:06 +03:00
Vitaly Baranov
765d136d2a A few improvements in the implementation of SSL certificate authentication. 2022-02-21 07:41:02 +03:00
taiyang-li
e6b29167da fix style 2022-02-15 20:26:53 +08:00
taiyang-li
b4440131d0 add metrics for clickhouse-local 2022-02-15 16:25:07 +08:00
Maksim Kita
91bc9cd4cf
Merge pull request #34499 from vitlibar/grpc-send-output-format-back-to-client 
gRPC: Send output format back to client
 2022-02-13 15:34:32 +01:00
Vitaly Baranov
cf2e205b08 Add field output_columns to first Result sent in gRPC protocol. 2022-02-10 21:05:56 +07:00
Vitaly Baranov
bd746fd82b Add fields query_id and time_zone to first Result sent in gRPC protocol. 2022-02-10 21:05:47 +07:00
Vitaly Baranov
f8c8eeceea Add field output_format to first Result sent in gRPC protocol. 2022-02-10 21:05:08 +07:00
Dmitry Novik
d405523598 Fix ProfileEvents sending in INSERT SELECT 2022-02-10 13:27:27 +00:00
Kruglov Pavel
738355b724
Merge pull request #34408 from vitlibar/split-and-rename-compression-fields-in-grpc 
Split and rename compression fields in gRPC
 2022-02-10 15:40:20 +03:00
Anton Popov
298838f891 avoid unnecessary copying of Settings 2022-02-10 12:13:51 +03:00
Vitaly Baranov
1341b4b4de Rename QueryInfo's field result_compression -> transport_compression_type and change its type for better consistency. 
Make the previous field obsolete.
 2022-02-10 02:46:05 +07:00
Vitaly Baranov
1f6b65d39d gRPC: Split compression_type field into input_compression_type and output_compression_type. 
Make the previous field obsolete.
 2022-02-10 02:46:05 +07:00
Vitaly Baranov
b2ba0c4320 Fix inserting to temporary tables via gRPC. 2022-02-07 12:36:17 +07:00
Vitaly Baranov
5dd49a44e6 Add test for propagating OpenTelemetry context via gRPC protocol. 2022-02-06 02:09:56 +07:00
alexey-milovidov
665fdc9187
Merge pull request #31257 from FArthur-cmd/merge_15765 
Merge #15765
 2022-02-04 00:39:57 +03:00
alexey-milovidov
4742cf3074
Update CertificateReloader.cpp 2022-02-03 03:03:46 +03:00
Azat Khuzhin
5472aef084 Fix current_user/current_address for interserver mode 
Before this patch current_user/current_address will be preserved from
the previous query.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-02 17:44:19 +03:00
Sergei Trifonov
68bc456830
Merge pull request #34223 from azat/bump-fmt 
Bump fmtlib from 7.0.0 to 8.1.1
 2022-02-02 00:03:25 +03:00
Vitaly Baranov
2ef2479ddc
Add const to make clang-tidy happy 2022-02-01 18:38:52 +07:00
Azat Khuzhin
de80014eb5 Use fmt::format over Poco::format in PostgreSQLHandler 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:04 +03:00
Azat Khuzhin
bedf208cbd Use fmt::runtime() for LOG_* for non constexpr 
Here is oneliner:

    $ gg 'LOG_\(DEBUG\|TRACE\|INFO\|TEST\|WARNING\|ERROR\|FATAL\)([^,]*, [a-zA-Z]' -- :*.cpp :*.h | cut -d: -f1 | sort -u | xargs -r sed -E -i 's#(LOG_[A-Z]*)\(([^,]*), ([A-Za-z][^,)]*)#\1(\2, fmt::runtime(\3)#'

Note, that I tried to do this with coccinelle (tool for semantic
patchin), but it cannot parse C++:

    $ cat fmt.cocci
    @@
    expression log;
    expression var;
    @@

    -LOG_DEBUG(log, var)
    +LOG_DEBUG(log, fmt::runtime(var))

I've also tried to use some macros/templates magic to do this implicitly
in logger_useful.h, but I failed to do so, and apparently it is not
possible for now.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

v2: manual fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:03 +03:00
FArthur-cmd
3dc64f031a correct code style 2022-01-31 12:21:22 +00:00
FArthur-cmd
891411a2be add try catch to avoid errors on start 2022-01-31 12:13:28 +00:00
Andre Marianiello
f5cbc6da43 Propagate trace context from GRPC calls 2022-01-28 14:27:04 -05:00
FArthur-cmd
cb4ad97d45 add test 2022-01-28 17:37:52 +00:00
Filatenkov Artur
82b0fc0bd0
Merge branch 'master' into merge_15765 2022-01-24 13:38:53 +03:00
alesapin
5ec76cd827 Merge branch 'master' into better_hashmap 2022-01-21 12:47:41 +03:00
Azat Khuzhin
6acb4d6ac5 Remove unbundled gRPC support 2022-01-20 08:47:16 +03:00
Nikita Mikhaylov
e3de3889e3
Revert "Revert "Ignore parse failure of opentelemetry header (#32116)" (#33594)" (#33595) 2022-01-19 20:53:39 +03:00
alesapin
3d325aacf6 Merge branch 'master' into better_hashmap 2022-01-19 12:31:23 +03:00
Eugene Galkin
0681bb4bed remove unused var and function declaration 2022-01-17 18:48:04 +03:00
tavplubix
34b3894422
Merge pull request #33676 from ClickHouse/fix-8340 
Correctly determine current database if CREATE TEMPORARY TABLE AS SELECT is queried inside a named HTTP session.
 2022-01-17 16:52:09 +03:00
alesapin
4fb8761a22 Merge branch 'master' into better_hashmap 2022-01-17 15:51:50 +03:00
Eugene Galkin
30a741a19f fix header authentication check condition 2022-01-17 15:47:25 +03:00
Eugene Galkin
f2612d42dc Implement SSL X.509 certificate authentication 2022-01-17 15:01:38 +03:00
Eugene Galkin
f46dca4793 support x509 ssl certificate authentication 2022-01-17 15:01:38 +03:00
alesapin
b2271cc2d9
Merge pull request #33288 from JackyWoo/add_lower_bound_session_timeout_to_keeper 
Add lower bound session timeout to keeper
 2022-01-17 14:06:15 +03:00
Alexey Milovidov
4ab7dfc741 Fix #8340 2022-01-16 08:12:55 +03:00
alesapin
ec6840f49c Make ZooKeeper client better interpret keeper server connection reject 2022-01-13 17:12:10 +03:00
Nikita Mikhaylov
340ec0fd20
Revert "Ignore parse failure of opentelemetry header (#32116)" (#33594) 
This reverts commit 4e5f9c8ff7.
 2022-01-13 16:29:18 +04:00
mergify[bot]
085492614b
Merge branch 'master' into add_lower_bound_session_timeout_to_keeper 2022-01-13 12:21:46 +00:00
Frank Chen
4e5f9c8ff7
Ignore parse failure of opentelemetry header (#32116) 2022-01-13 16:16:10 +04:00
Azat Khuzhin
e8c7e5e8f0 Fix typo in sendMergeTreeReadTaskRequestAssumeLocked() 2022-01-07 20:04:57 +03:00
alesapin
f8114126ed Use HashMap and arena with free lists for keeper 2021-12-30 19:21:49 +03:00
JackyWoo
569ce62e8d use session_timeout as session timeout uper bound 2021-12-30 17:18:51 +08:00
JackyWoo
19b5394be8 fix tests 2021-12-30 15:49:48 +08:00
JackyWoo
d35e5f8319 add lower bound session timeout to keeper 2021-12-29 20:59:01 +08:00
Vitaly Baranov
f80a3e3247
Merge pull request #30549 from aiven/kmichel-server-reload 
Start/stop servers when `listen_host`/`*_port` changes
 2021-12-27 14:51:00 +03:00
Alexey Milovidov
29d28c531f Move code around to avoid dlsym on Musl 2021-12-24 12:25:27 +03:00
Kevin Michel
ffc1fca296
Start/stop servers when listen_host/*_port changes 
This allows starting and stopping separately each protocol server
without restarting ClickHouse.

This also allows adding or removing `listen_host` entries, which
start and stops servers for all enabled ports.

When stopping a server, the listening socket is immediately closed
(and available for another server).

Protocols with persistent connections try to wait for any currently
running query to finish before closing the connection, but idle
connection are closed quickly (depending on how often the protocol
is polled).

An extra ProfileEvent is added, `MainConfigLoads`, it is
incremented every time the configuration is reloaded. This helps
when trying to assess whether the new configuration was applied.
 2021-12-24 08:26:02 +01:00
alexey-milovidov
8c9843caf2
Merge pull request #32982 from azat/http-context-leak 
Fix Context leak in case of cancel_http_readonly_queries_on_client_close
 2021-12-23 07:58:18 +03:00
mergify[bot]
04f727e84c
Merge branch 'master' into read-fix 2021-12-22 13:49:04 +00:00
alesapin
754785fee5 Better container 2021-12-21 11:07:24 +03:00
Azat Khuzhin
08ced87880 Rework MultipartReadBuffer::skipToNextBoundary() 
As suggested by @vitlibar
 2021-12-21 08:38:41 +03:00
Azat Khuzhin
d7c7a91f66 Check that at least one boundary exist in HTMLForm::readMultipart() 2021-12-21 08:38:41 +03:00
Azat Khuzhin
4dfa9324d1 Check for EOF in MultipartReadBuffer::skipToNextBoundary() 
Otherwise you may hit internal assertion when client goes away:

    clickhouse-server: ./src/Server/HTTP/HTMLForm.cpp:245: bool DB::HTMLForm::MultipartReadBuffer::skipToNextBoundary(): Assertion `boundary_hit' failed.

Fixes: 02151_http_s_structure_set_eof
 2021-12-21 08:38:41 +03:00
JackyWoo
7a19570853 keeper handler should remove operation when response sent 2021-12-21 11:43:14 +08:00
Azat Khuzhin
a9f80dd705 Fix Context leak in case of cancel_http_readonly_queries_on_client_close 
To handle cancel_http_readonly_queries_on_client_close=true context
attaches progress callback with holding a copy of the current context,
and this creates recursive context reference and so shared_ptr will be
never released, [1] contains simplest example.

One example of a memory leak, external tables passed with HTTP query
(s_structure + file upload), since they are stored in the context, and
can occupate enough memory to make it visible to user.

Note, that the lifetime of the Context should be fine, since callback
can be called only when query is executed.

  [1]: Example of recursive reference

       ```c
       #include <memory>
       #include <functional>
       #include <cassert>

       bool released = true;

       class Context : public std::enable_shared_from_this<Context>
       {
           using Func = std::function<void()>;
           Func func;

       public:
           Context()  { released = false; }
           ~Context() { released = true; }
           Context(const Context & context) = delete;

           void addCallback(Func func_)
           {
               func = func_;
           }
       };

       int main()
       {
           {
               auto context = std::make_shared<Context>();
               context->addCallback([context]() {});
           }
           assert(released == true);
       }
       ```
 2021-12-20 21:05:38 +03:00
Azat Khuzhin
dd8c64e380 Set QueryScope earlier for HTTP queries 2021-12-20 21:05:38 +03:00
Vitaly Baranov
e61d3eef0c
Merge pull request #32747 from vitlibar/improve-grpc-compression 
Improve gRPC compression support
 2021-12-20 19:44:58 +03:00
Azat Khuzhin
6aebc3e94c Do not loose ProfileEvents in case of thread destroyed before 
v2: drop std::move and add copy ctor for ProfileEvents::Counter::Snapshot
v2: remove std::move
 2021-12-19 23:43:05 +03:00
Vitaly Baranov
708439b036 Support customized compression for input/output data in gRPC protocol. 2021-12-19 16:29:44 +03:00
Alexey Milovidov
2c68192bb6 Fix strange code in TCPHandler 2021-12-12 05:35:33 +03:00
Azat Khuzhin
57c027be0e Remove arcadia build support 2021-12-11 21:25:23 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
alexey-milovidov
96ec92c7cd
Merge pull request #31155 from ClickHouse/incremental-profile-events 
Send incremental profile events to client
 2021-12-01 04:04:41 +03:00
alesapin
c1f38752c4
Merge pull request #31859 from JackyWoo/keeper_timeout 
keeper session timeout doesn't work
 2021-11-28 16:28:17 +03:00
Raúl Marín
051dddd8df Reduce dependencies on ASTIdentifier.h 
Goes from rebuilding 483 objects to 165 when it's modified
 2021-11-26 16:49:40 +01:00
JackyWoo
4c6ad94109 fix ch keeper session timeout doesn't work 2021-11-26 18:17:49 +08:00
Kruglov Pavel
690b460c3d
Fix style 2021-11-24 22:36:36 +03:00
Kruglov Pavel
ad7768e42a
Fix 2021-11-24 22:25:41 +03:00
Kruglov Pavel
b20f4439df
Make better 2021-11-24 22:25:03 +03:00
avogar
1bd8c57908 Fix 2021-11-24 20:21:21 +03:00
avogar
7c84d95908 Fix possible Logical error: Cannot write to finalized buffer 2021-11-24 18:32:41 +03:00
alesapin
646cf38213
Merge pull request #28981 from JackyWoo/add_4_letter_words_commands 
Add four letter commands to keeper
 2021-11-24 13:32:36 +03:00
Kruglov Pavel
b63b47f0f0
Merge pull request #31265 from Avogar/fix-write-buffers 
Fix and refactor WriteBiffer-s a little
 2021-11-23 16:46:09 +03:00
Kruglov Pavel
6d942401bb
Merge pull request #31510 from azat/fix-progress-for-insert 
Fix progress for short INSERT SELECT queries
 2021-11-23 16:37:06 +03:00
alesapin
16f034fe4c Flush bytes after 4 letter commands 2021-11-23 13:31:10 +03:00
alesapin
5825242a7d Merge branch 'add_4_letter_words_commands' of github.com:JackyWoo/ClickHouse into JackyWoo_add_4_letter_words_commands 2021-11-22 18:02:22 +03:00
alesapin
35de9e42d9
Merge branch 'master' into add_4_letter_words_commands 2021-11-22 18:02:02 +03:00
alesapin
1cf204c581 Small changes 2021-11-21 15:28:45 +03:00
Alexander Tokmakov
856502fa81 log long operations in Keeper 2021-11-20 20:39:31 +03:00
Kruglov Pavel
d9c1a0c8ec
Merge branch 'master' into fix-write-buffers 2021-11-20 17:48:24 +03:00
Azat Khuzhin
dd7f780978 Fix Progress for short INSERT SELECT queries 
Note, that we cannot send Progress for regular INSERT (w/ VALUES) w/o
breaking protocol compatibility, but it can be done by increasing
revision.
 2021-11-20 16:41:35 +03:00
Azat Khuzhin
0b4d855c08 Reduce copy-paste in TCPHandler::processInsertQuery() 2021-11-20 16:41:35 +03:00
alesapin
3f8f08e816 Slightly better 2021-11-19 12:30:58 +03:00
alesapin
dcec086573 Small refactoring 2021-11-18 23:17:22 +03:00
JackyWoo
27d50edc59 adjust log output 2021-11-18 15:49:43 +08:00
JackyWoo
007366b506 remove sizeOf method from SnapshotableHashTable 2021-11-18 12:35:32 +08:00
Kruglov Pavel
f27fcf8372
Merge pull request #31004 from Avogar/output-formats 
Small refactoring in formats
 2021-11-17 20:44:46 +03:00
alesapin
5f3eca4cd1 Merge branch 'master' into JackyWoo_add_4_letter_words_commands 2021-11-17 12:59:20 +03:00
Nikolay Degterinsky
ba50e8387d Merge branch 'master' into query_parameters 2021-11-15 19:55:54 +00:00
JackyWoo
a2f3337ca1 code style change 2021-11-12 20:48:42 +08:00
avogar
e5661b9b62 Minor change 2021-11-11 21:09:21 +03:00
avogar
51831afff8 Fix tests 2021-11-11 20:27:23 +03:00
mergify[bot]
58c5981cab
Merge branch 'master' into refactor-pipeline-executor 2021-11-11 16:16:52 +00:00
Nikolay Degterinsky
7b5954b736 Merge branch 'master' into query_parameters 2021-11-11 11:43:00 +00:00
Nikolai Kochetov
90cf835277 A little bit more refactoring. 2021-11-11 14:41:15 +03:00
avogar
c521a9131a Small refactoring of WriteBiffer-s 2021-11-11 02:11:18 +03:00
Artur
4aaa034c8e Correct behaviour if certificate or key was not set 2021-11-10 16:56:07 +00:00
Artur Filatenkov
744d50f328 merge with master 2021-11-10 18:35:35 +03:00
Artur Filatenkov
68bc8d63a1 Working version 2021-11-10 15:55:57 +03:00
JackyWoo
1cbe9f6024 fix build error 2021-11-10 10:50:12 +08:00
JackyWoo
c71fb3337f add more 4lwd commands 2021-11-09 17:39:28 +08:00
Dmitry Novik
0cd858263f Update protocol version 2021-11-08 16:43:34 +03:00
Dmitry Novik
a093395b9b Process incremental profile events on client 2021-11-08 16:38:31 +03:00
Dmitry Novik
8708e00b3a Send incremental profile events to client 2021-11-08 12:40:31 +03:00
Artur Filatenkov
d8510583d0 resolve conflict 2021-11-05 17:07:29 +03:00
avogar
2dd4393ca1 Small refactoring in formats 2021-11-03 20:07:05 +03:00
Vitaly Baranov
ab01b9afc8 Split Authentication.h to common and main parts. 2021-11-01 19:13:49 +03:00
Vitaly Baranov
3ed7f8f0b3 Move access-rights' source files needed for parser to a separate target. 2021-11-01 19:13:49 +03:00
Alexey Milovidov
993f3faefd Miscellaneous 2021-10-31 18:11:46 +03:00
alesapin
30ab4b3a7f
Update KeeperTCPHandler.cpp 2021-10-30 16:02:05 +03:00
alesapin
40bffcec3a Fix error message in Keeper 2021-10-30 15:32:29 +03:00
JackyWoo
694306ee62 add 4lw white list to keeper 2021-10-29 17:54:25 +08:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
JackyWoo
e5b0eedd31 adjust code style for keeper 4lw cmd 2021-10-27 22:26:53 +08:00
JackyWoo
a60663e33d add 4lw commands to keeper 2021-10-27 20:26:42 +08:00
Nikolay Degterinsky
677eb5bbda Fix tests 2021-10-21 15:53:02 +03:00
mergify[bot]
69667aab1a
Merge branch 'master' into fix-backward-compatibility-after-21196 2021-10-19 10:11:16 +00:00
Ilya Yatsishin
4396daeb1f
Merge pull request #30064 from azat/client-print-profile-events 2021-10-19 12:42:09 +03:00
Azat Khuzhin
f7b76373ce Move ProfileEvents packet type from TCPHandler into ProfileEventsExt 2021-10-19 00:54:38 +03:00
alesapin
f1fe96e194 Merge branch 'master' into debug_keeper 2021-10-18 10:16:07 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
Nikolai Kochetov
067eaadadd Merge branch 'master' into removing-data-streams-folder 2021-10-16 09:46:05 +03:00
Nikolai Kochetov
c668696047
Merge pull request #30171 from ClickHouse/remove-stream-interfaces 
Remove stream interfaces
 2021-10-16 09:34:01 +03:00
Nikolai Kochetov
fd14faeae2 Remove DataStreams folder. 2021-10-15 23:18:20 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Maksim Kita
8ed4e73705
Merge pull request #30000 from ClickHouse/profile-snapshot 
Make read of Counters snapshot non-atomic
 2021-10-14 13:22:44 +03:00