Commit Graph

1105 Commits

Author SHA1 Message Date
pufit
f4d9b4a888
Merge pull request #47086 from pkit/pkit/insert_http_body
use http request body in predefined http handlers
2023-03-15 17:41:05 -04:00
Sema Checherinda
67ac858a52 remove counters for threads, fix negative counters 2023-03-15 22:22:22 +01:00
alexX512
f32df219ef Merge branch 'master' of github.com:alexX512/ClickHouse 2023-03-15 19:54:24 +00:00
alexX512
1f21ac8fcf Fix fast test 2023-03-15 19:53:58 +00:00
Alexey Perevyshin
1e9ef12f51
Merge branch 'master' into master 2023-03-15 22:21:02 +04:00
Vladimir C
045960f1a1
Merge pull request #47108 from ClickHouse/vdimir/http_wait_end_of_query_settings 2023-03-15 15:31:12 +01:00
alexX512
0088c18192 Fix fasttest error 2023-03-15 13:37:47 +00:00
alexX512
cbbdb5aabc Fix typos 2023-03-15 13:05:38 +00:00
Azat Khuzhin
727ad9289b Use VAR_UINT_MAX for unknown_packet_in_send_data
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-15 13:11:26 +01:00
alexX512
d47e10484f Merge branch 'master' of github.com:ClickHouse/ClickHouse 2023-03-15 11:14:34 +00:00
Azat Khuzhin
bcf381c5ae Reimplement interserver mode to avoid replay attacks
Prevous implementation (DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET)
accepts the salt from the client, which make it useless.

Reimplement the protocol to send the salt by the server and use it in
the client instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-15 08:28:43 +01:00
alexX512
14a7c5a019 Add stop_reading_on_first_cancel setting 2023-03-15 06:06:55 +00:00
Nikolay Degterinsky
398978e0bc Fix build 2023-03-15 00:09:29 +00:00
Nikolay Degterinsky
f3908a9e84 Fix style 2023-03-14 22:49:10 +00:00
Nikolay Degterinsky
eddda2eb73 Add SSL authentication to the native protocol 2023-03-14 22:10:08 +00:00
Alexey Milovidov
a411ae10b5
Merge pull request #47486 from ClickHouse/revert_25674
Remove "asynchronous connection drain"
2023-03-13 21:40:39 +03:00
Vladimir C
9cb018545c
Merge pull request #47499 from FrankChen021/span_kind 2023-03-13 11:35:03 +01:00
Frank Chen
0bdd645049 SET SpanKind for HTTP/TCP/GRPC handler 2023-03-12 21:31:08 +08:00
Alexander Tokmakov
7b1b238d0b Revert "Merge pull request #25674 from amosbird/distributedreturnconnection"
This reverts commit 5ffd99dfd4, reversing
changes made to 2796aa333f.
2023-03-11 19:09:47 +01:00
Vladimir C
1b264ad51e
Merge branch 'master' into vdimir/http_wait_end_of_query_settings 2023-03-09 13:37:56 +01:00
Mike Kot
9920a52c51 use std::lerp, constexpr hex.h 2023-03-07 22:50:17 +00:00
Kruglov Pavel
9e64441353
Merge pull request #47214 from azat/interserver-mode-address
Add real client (initiator server) address into the logs for interserver mode
2023-03-07 12:40:59 +01:00
Alexander Tokmakov
082194a755
Merge pull request #47175 from ClickHouse/correct_fatal_handling
More correct handling of fatal errors
2023-03-04 00:38:00 +03:00
Azat Khuzhin
5403360924 Add real client (initiator server) address into the logs for interserver mode
It is useful to understand what is going on, in some obscure cases, for
instance if someone will copy configuration from the production to some
docker env, and then you will see docker's private network addresses
in the logs.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-03 16:38:10 +01:00
Alexander Tokmakov
cadaf06829
Merge pull request #46681 from ClickHouse/fix_insert_cancellation_in_native_protocol
Fix queries cancellation when a client dies
2023-03-03 16:27:20 +03:00
Alexander Tokmakov
b1a288d271
Merge branch 'master' into correct_fatal_handling 2023-03-03 14:50:07 +03:00
Alexander Tokmakov
619b282060 more correct handling of fatal errors 2023-03-03 01:22:04 +01:00
Alexander Tokmakov
edd238273c fix another bug in client 2023-03-02 19:51:05 +01:00
vdimir
b782cba809 Add settings http_wait_end_of_query and http_response_buffer_size 2023-03-02 16:03:10 +01:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
Constantine Peresypkin
fdc6de2a27 use http request body in predefined http handlers
adds a new "virtual" parameter `__request_body`
which can be used in http rules, see below

tremendously helps in producing arbitrary REST interfaces

for example:

```xml
<http_handlers>
  <rule>
    <methods>POST</methods>
    <url>/ingest</url>
    <handler>
      <type>predefined_query_handler</type>
      <query>INSERT INTO events(id, data) SELECT {id:UInt32}, {__request_body:String}</query>
    </handler>
  </rule>
</http_handlers>
```
2023-03-02 12:51:48 +01:00
Alexander Tokmakov
ad4a44df52 fix 2023-03-02 02:59:27 +01:00
Alexander Tokmakov
a97e15e36f Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-01 14:26:41 +01:00
vdimir
ea30bc7e91
Fix tmp_path_template in HTTPHandler::processQuery 2023-02-28 12:47:35 +00:00
Alexey Milovidov
25853da4ad
Merge pull request #46846 from ClickHouse/better-fatal-handling
Better handling of fatal errors
2023-02-25 22:55:14 +03:00
Anton Popov
37570f10ea
Merge pull request #46364 from CurtizJ/add-query-id-in-response
Add `X-ClickHouse-Query-Id` header in all queries
2023-02-25 14:18:35 +01:00
Alexander Tokmakov
cad1e0b768 fix 2023-02-25 01:18:34 +01:00
Alexey Milovidov
f28ab14759 Better handling of fatal errors 2023-02-24 23:29:06 +01:00
Anton Popov
1fbe96d20c fix mysql protocol 2023-02-24 18:08:51 +00:00
Alexander Tokmakov
fe92fd8a61 Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-02-23 22:35:31 +01:00
Alexander Tokmakov
e660c0838c fix multipart requests 2023-02-22 17:54:35 +01:00
Alexander Tokmakov
592af6d652 fix incomplete interst through http 2023-02-22 02:34:03 +01:00
Alexander Tokmakov
f5c232707d cancel insert queries correctly 2023-02-21 22:04:39 +01:00
Alexander Tokmakov
b5e6d74d48
More interesting settings for Stress Tests (#41534)
* Update stress

* fix

---------

Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2023-02-21 15:36:23 +03:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Azat Khuzhin
6f7b6e9206 Remove extra try/catch for QueryState reset
It was possible back when destuctors may throw, in a pre C++11.

I've found one place where it may throw from 2012:

    ~UnionBlockInputStream()
    {
        ...
        if (exception && !std::uncaught_exception())
            exception->rethrow();
        ...
    }

Refs: 8a053aba54
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-18 10:55:15 +01:00
Alexey Milovidov
993a39e2a1
Merge pull request #46360 from genzgd/http_client_version
Allow to provide the protocol version by HTTP clients to tune the compatibility of the Native output format.
2023-02-16 22:17:04 +03:00
Alexey Milovidov
c2660646fe
Merge branch 'master' into debug-docker 2023-02-15 15:59:49 +03:00
Geoff Genz
b40174e4a8
Merge branch 'master' into http_client_version 2023-02-14 18:35:06 -07:00
Alexey Milovidov
dd79c5f4f6 Easy debugging of clickhouse-server in Docker 2023-02-14 09:49:05 +01:00
Alexey Milovidov
bd64098ee9
Update HTTPHandler.cpp 2023-02-14 11:11:24 +03:00
Anton Popov
e9e6d735ef add query_id header in all queries 2023-02-13 16:44:54 +00:00
Geoff Genz
be8bf3a6a3
Merge branch 'master' into http_client_version 2023-02-13 08:43:59 -07:00
mateng0915
6e9dc2527f fixed review comments 2023-02-13 11:43:01 +08:00
mateng915
c876ad901b
Merge branch 'master' into replicas_status_api_optimize 2023-02-13 11:38:55 +08:00
Alexey Milovidov
2e57fab8e3 Instrumentation of callbacks for distributed queries 2023-02-12 10:04:05 +01:00
Geoff Genz
99c3ff53c5 Merge remote-tracking branch 'origin/master' into http_client_version
# Conflicts:
#	src/Interpreters/Context.cpp
#	src/Interpreters/Context.h
2023-02-10 04:35:53 -07:00
Geoff Genz
7ed8ed0284 Add support for client_protocol_version sent with HTTP 2023-02-10 03:47:06 -07:00
Azat Khuzhin
8cc41b7f41 Check return value of ::close()
Note, that according close(2), EINTR should not be retriable for close()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-07 11:28:22 +01:00
mateng915
3e5d3914ba
Merge branch 'master' into replicas_status_api_optimize 2023-02-06 09:58:31 +08:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] (#43772) 2023-02-03 14:34:18 +01:00
Alexander Tokmakov
3744fa2c63 format more messages 2023-01-25 21:16:42 +01:00
mateng915
8243384a7e
Merge branch 'master' into replicas_status_api_optimize 2023-01-25 09:30:08 +08:00
Alexander Tokmakov
d1baa7300c reformat ParsingException 2023-01-24 23:21:29 +01:00
Alexander Tokmakov
9d13ca4a16 fix 2023-01-24 01:49:16 +01:00
Alexander Tokmakov
6ddd76c5a0 fix 2023-01-24 01:30:26 +01:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
mateng915
1e7c76ea45
Merge branch 'master' into replicas_status_api_optimize 2023-01-23 17:04:40 +08:00
mateng915
e22673b1c9
Merge branch 'master' into replicas_status_api_optimize 2023-01-18 11:06:36 +08:00
Alexander Tokmakov
5cd90c1a3e Merge branch 'master' into exception_message_patterns 2023-01-17 20:04:04 +01:00
Nikita Mikhaylov
0fc755806e
One more attempt to fix race in TCPHandler (#45240) 2023-01-17 16:17:14 +01:00
Alexander Tokmakov
870cfcc36a less fmt::runtime usages 2023-01-17 00:11:59 +01:00
mateng0915
eae0f9957c Feature: Optimize the replica delay api logic
Description:
============
Currently if in the Whole CK instance has one table is read only status
then the API  /replicas_status will throw error, xxx table is read only

For make this monitor can work in prod env, we can catch the read only
status instead of directly throw error

Solution:
=========
Return other normal table's delay value even if the CK instance has readonly Replicatedxxx table

Please enter the commit message for your changes. Lines starting
2023-01-13 14:44:27 +08:00
Kseniia Sumarokova
db3e0219fc
Merge pull request #41687 from ClickHouse/40907_Parameterized_views_as_table_functions
40907 Parameterized views as table functions
2023-01-12 14:24:32 +01:00
Nikolai Kochetov
299fcff8bc
Merge pull request #45123 from ClickHouse/fix-race-in-s3cluster
Fix data race in s3Cluster.
2023-01-10 19:47:42 +01:00
Nikolai Kochetov
9893d35476 Fix data race in s3Cluster. 2023-01-10 15:34:52 +00:00
Anton Popov
1f32ffedf8
Merge pull request #43221 from ClickHouse/refactoring-ip-types
Replace domain IP types (IPv4, IPv6) with native
2023-01-07 12:01:21 +01:00
SmitaRKulkarni
4d7459bd65
Merge branch 'master' into 40907_Parameterized_views_as_table_functions 2022-12-27 08:15:42 +01:00
Yakov Olkhovskiy
ce533af162 remove unnecessary includes 2022-12-23 19:29:51 +00:00
Yakov Olkhovskiy
a166dde9c4 add member function getLastForwardedFor to ClientInfo 2022-12-23 18:32:29 +00:00
Yakov Olkhovskiy
a8cb29da4b
Merge branch 'master' into refactoring-ip-types 2022-12-21 23:56:24 -05:00
Yakov Olkhovskiy
8462b724ca
Merge branch 'master' into feature-protocol-proxy 2022-12-20 13:12:07 -05:00
Yakov Olkhovskiy
9092134544 auth_use_forwarded_address config setting added, using forwarded address for proxied TCP and HTTP connection, likely bug is fixed in AccessControl, relevant test is appended 2022-12-20 17:39:40 +00:00
Smita Kulkarni
16d8fd02fb Merge branch 'master' into 40907_Parameterized_views_as_table_functions 2022-12-16 08:52:48 +01:00
Anton Popov
cce3257f39
Merge branch 'master' into optimize-storage-s3 2022-12-13 21:35:12 +01:00
Anton Popov
0c87031e80 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-13 16:33:21 +00:00
Nikolay Degterinsky
19e95e8cd8 Better implementation of password rules 2022-12-11 23:59:04 +00:00
Yakov Olkhovskiy
a98ce7d2be propagate PROXYv1 forwarded_for address to http handler in composable protocols; some logging added 2022-12-11 23:39:16 +00:00
Yakov Olkhovskiy
0641066183
Merge branch 'master' into refactoring-ip-types 2022-12-08 11:12:05 -05:00
Yakov Olkhovskiy
a20783a25d exception codes to http status 2022-12-07 21:50:13 +00:00
Anton Popov
8c95308a08 fix table function s3Cluster 2022-12-07 17:00:10 +00:00
Antonio Andelic
e78761b66c
Merge branch 'master' into keeper-prometheus 2022-11-21 09:10:10 +01:00
Smita Kulkarni
c84a50e77c Merge branch 'master' into 40907_Parameterized_views_as_table_functions 2022-11-16 12:40:04 +01:00
Alexey Milovidov
04687c0d69
Merge pull request #43206 from ClickHouse/miscellaneous-changes-parallel-replicas
Miscellaneous changes
2022-11-16 05:30:03 +01:00
Antonio Andelic
96838f5be7 Merge branch 'master' into keeper-prometheus 2022-11-15 12:00:02 +00:00
Alexey Milovidov
bfea3deed9 Miscellaneous changes 2022-11-14 06:09:03 +01:00
Alexey Milovidov
ba05b7dd2c Add documentation for AsynchronousMetrics 2022-11-13 03:36:20 +01:00
Alexey Milovidov
5a0b7b2947 Add documentation for AsynchronousMetrics 2022-11-13 02:59:01 +01:00
Antonio Andelic
36d763011b Compile standalone keeper 2022-11-10 09:23:52 +01:00