thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
116,374 Commits 9,315 Branches 1,783 Tags 2.1 GiB
6caf8080b0
Commit Graph

1105 Commits

Author SHA1 Message Date
Yakov Olkhovskiy
5727517713 add variadic constructor 2022-09-02 16:22:57 +00:00
Yakov Olkhovskiy
b6e3680d60 test with TLS 2022-09-02 04:47:04 +00:00
Yakov Olkhovskiy
e7959d74bc add composable factory and protocol 2022-09-01 22:06:54 +00:00
Frank Chen
9d63cbe811 Merge 'origin/master' into tracing_context_propagation to resolve conflicts 2022-09-01 23:18:59 +08:00
Kruglov Pavel
be81d21fdf
Merge pull request #40597 from helifu/master 
Fix a crash while the grpc port is conflict
 2022-08-30 13:17:25 +02:00
Frank Chen
f17d56b528 Merge branch 'master' into tracing_context_propagation 2022-08-30 14:24:36 +08:00
Alexey Milovidov
90cf0bb6e4 Fix style 2022-08-27 23:40:24 +02:00
Alexey Milovidov
6b2e227c8b Fix integration test 2022-08-27 22:28:38 +02:00
Frank Chen
bb00dcc19b Remove using namespace from header 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-25 20:20:13 +08:00
helifu
b52e098a40 Fix a crash while the grpc port is conflict 2022-08-25 10:56:53 +08:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
Alexander Tokmakov
9820c3fce0
Merge pull request #40545 from ClickHouse/fix_restart_replica_race 
Fix race on system restart replica
 2022-08-24 15:18:36 +03:00
Frank Chen
cd19366b44 Move classes into DB::OpenTelemetry namespace 2022-08-24 16:41:40 +08:00
Alexander Tokmakov
77178a18fa fix race on system restart replica 2022-08-23 16:10:44 +02:00
Alexey Milovidov
34b5d267f3
Merge pull request #40461 from ClickHouse/dashboard 
Dashboard
 2022-08-23 15:34:28 +03:00
Alexey Milovidov
7c8e540dea Implement code review suggestions 2022-08-22 21:55:41 +02:00
Alexey Milovidov
6ae9e0f64f Fix error 2022-08-22 01:46:46 +02:00
Alexey Milovidov
9635fa817b Embedded dashboard 2022-08-21 12:11:25 +02:00
Alexey Milovidov
3649aac593 Fix style 2022-08-20 17:59:51 +02:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Frank Chen
024fef1aec
Merge branch 'master' into tracing_context_propagation 2022-08-20 14:23:36 +08:00
Frank Chen
a3b6ad2a65
Merge branch 'master' into tracing_context_propagation 2022-08-18 20:59:07 +08:00
Sema Checherinda
89bada927c fix heap buffer overflow by limiting http chunk size 2022-08-18 14:30:52 +02:00
Alexey Milovidov
8c3d1f2b5b
Merge branch 'master' into server-side-time 2022-08-12 17:40:02 +03:00
Nikita Taranov
17956cb668
Extend protocol with query parameters (#39906) 2022-08-12 14:28:35 +02:00
Alexey Milovidov
33e43cc6f9
Merge branch 'master' into server-side-time 2022-08-10 07:28:15 +03:00
Yakov Olkhovskiy
5304e773cd no session in interserver mode 2022-08-09 12:10:09 -04:00
Yakov Olkhovskiy
d39e9f65de
Merge branch 'master' into fix-quota-key 2022-08-08 11:54:21 -04:00
Alexey Milovidov
855e52a7c9 Send final ProfileEvents just in case 2022-08-08 07:12:08 +02:00
Alexey Milovidov
db85ebfaa7 Add server-side time to Progress 2022-08-08 06:55:41 +02:00
Alexander Tokmakov
c983f14ed1
Merge pull request #39925 from ClickHouse/log_test_on_connection_close 
Add logging to debug flaky tests
 2022-08-05 16:20:17 +03:00
Alexander Tokmakov
e59f217f8d add test logs on connection close 2022-08-05 13:33:27 +02:00
Alexander Tokmakov
20cf4e8d22 slightly more readable if conditions 2022-08-04 18:04:06 +02:00
Yakov Olkhovskiy
1680be19ca some refactoring 2022-08-03 16:36:52 -04:00
Yakov Olkhovskiy
2e34b384c1 update tcp protocol, add quota_key 2022-08-03 15:44:08 -04:00
Frank Chen
40c6e4c0d6 Merge remote-tracking branch 'origin/master' into tracing_context_propagation 2022-08-02 10:02:09 +08:00
Frank Chen
f205b5480d Fix test cases 2022-08-02 09:50:40 +08:00
Anton Kozlov
ae7fd5bf93 Default database resolution in distributed reads 2022-07-29 13:35:57 +00:00
Nikolai Kochetov
91043351aa Fixing build. 2022-07-20 20:30:16 +00:00
Robert Schulze
13482af4ee
First try at reducing the use of StringRef 
- to be replaced by std::string_view
- suggested in #39262
 2022-07-17 17:26:02 +00:00
Robert Schulze
deda29b46b
Pass const StringRef by value, not by reference 
See #39224
 2022-07-15 11:34:56 +00:00
Amos Bird
90fa4863f9
Text log source regexp 2022-07-13 17:52:14 +08:00
Frank Chen
c559370ec9 Fix test cases 2022-07-09 20:20:04 +08:00
Frank Chen
da57a993e4 Fix CI 2022-07-09 13:43:10 +08:00
Frank Chen
90efbc323d Fix conflicts 2022-07-07 17:44:32 +08:00
Frank Chen
93dc109e36 Fix code 2022-07-07 17:44:19 +08:00
Frank Chen
8ce5bae2dd Add support to GRPCServer 2022-07-07 17:43:52 +08:00
Frank Chen
19f3f3e6ca Initialize thread tracing context for TCPServer 2022-07-07 17:42:22 +08:00
Frank Chen
8c04456148 Fix query span at executeQuery layer 2022-07-07 17:41:58 +08:00
Frank Chen
0a44a44feb Start tracing at HTTP entry 2022-07-07 17:41:27 +08:00
Alexey Milovidov
02adad3f27
Merge pull request #37860 from amosbird/remove-duplicated-log 
Remove duplicate peak mem log
 2022-07-07 08:16:06 +03:00
Robert Schulze
f692ead6ad
Don't use std::unique_lock unless we have to 
Replace where possible by std::lock_guard which is more light-weight.
 2022-06-28 19:19:06 +00:00
Azat Khuzhin
9eeb856519 Fix INSERT into Distributed hung due to ProfileEvents 
Right now RemoteInserter does not read ProfileEvents for INSERT, it
handles them only after sending the query or on finish.

But #37391 sends them for each INSERT block, but sometimes they can be
no ProfileEvents packet, since it sends only non-empty blocks.

And this adds too much complexity, and anyway ProfileEvents are useless
for the server, so let's send them only if the query is initial (i.e.
send by user).

Note, that it is okay to change the logic of sending ProfileEvents w/o
changing DBMS_TCP_PROTOCOL_VERSION, because there were no public
releases with the original patch included yet.

Fixes: #37391
Refs: #35075
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-22 15:41:15 +03:00
Kseniia Sumarokova
a756b4be27
Merge pull request #37391 from azat/insert-profile-events-fix 
Send profile events for INSERT queries (previously only SELECT was supported)
 2022-06-20 12:16:29 +02:00
Alexey Milovidov
73709b0488
Revert "Revert "Add a setting to use more memory for zstd decompression"" 2022-06-18 15:55:35 +03:00
alesapin
16e8b85fbf
Revert "Add a setting to use more memory for zstd decompression" 2022-06-18 14:08:14 +02:00
Alexey Milovidov
e20259e9ca
Merge pull request #37015 from wuxiaobai24/zstd_window_log_max 
Add a setting to use more memory for zstd decompression
 2022-06-18 04:19:27 +03:00
Azat Khuzhin
4baa7690ae Send profile events for INSERT queries (previously only SELECT was supported) 
Reproducer:

    echo "1" | clickhouse-client --query="insert into function null('foo String') format TSV" --print-profile-events --profile-events-delay-ms=-1

However, clickhouse-local is differnt, it does sent the periodically,
but only if query was long enough, i.e.:

    # yes | head -n100000 | clickhouse-local --query="insert into function null('foo String') format TSV" --print-profile-events --profile-events-delay-ms=-1
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] ContextLock: 10 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] DiskReadElapsedMicroseconds: 29 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] IOBufferAllocBytes: 200000 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] IOBufferAllocs: 1 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] InsertQuery: 1 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] InsertedBytes: 1000000 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] InsertedRows: 100000 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] MemoryTrackerUsage: 1521975 (gauge)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] OSCPUVirtualTimeMicroseconds: 102148 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] OSReadChars: 135700 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] OSWriteChars: 8 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] Query: 1 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] RWLockAcquiredReadLocks: 2 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] ReadBufferFromFileDescriptorRead: 5 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] ReadBufferFromFileDescriptorReadBytes: 134464 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] RealTimeMicroseconds: 293747 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] SoftPageFaults: 382 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] TableFunctionExecute: 1 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] UserTimeMicroseconds: 102148 (increment)

v2: Proper support ProfileEvents in INSERTs (with protocol change)
v3: Receive profile events on INSERT queries
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-16 11:59:01 +03:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type" 2022-06-07 13:53:10 +02:00
Vitaly Baranov
d199478169
Merge pull request #37303 from ClickHouse/fix_trash 
Try to fix some trash
 2022-06-07 10:17:39 +02:00
Robert Schulze
2d87af2a15
Merge pull request #37647 from DevTeamBK/Fix-all-CheckTriviallyCopyableMove-Errors 
Fix errors of CheckTriviallyCopyableMove type
 2022-06-05 19:58:47 +02:00
Amos Bird
a5afb73c52
Remove duplicated peak mem log 
Also log "Processed in {} sec" a little bit earlier so that it's also
sent to client and have the log entry related to the query_id.
 2022-06-05 17:02:58 +08:00
Nikolai Kochetov
8991f39412 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-06-02 17:00:08 +00:00
Alexander Tokmakov
75f49a48e1 Merge branch 'master' into fix_trash 2022-06-01 14:20:46 +02:00
HeenaBansal2009
b7eb6bbd38 Fixed clang-tidy-CheckTriviallyCopyableMove-errors 2022-05-30 11:09:03 -07:00
Nikolai Kochetov
5b4658aa5e Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-30 09:47:35 +00:00
taiyang-li
0b0d38b18c Merge branch 'master' into fix_settings 2022-05-30 10:07:09 +08:00
Alexander Tokmakov
4e52f45695 Merge branch 'master' into fix_trash 2022-05-28 19:43:19 +02:00
Alexey Milovidov
c50791dd3b Fix clang-tidy-14, part 1 2022-05-27 22:52:14 +02:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
Kruglov Pavel
0c3bcfa122
Merge pull request #36884 from Algunenano/http_proper_summary_and_exception 
HTTP: Always return summary data and exception (when possible)
 2022-05-24 18:33:24 +02:00
Alexander Tokmakov
3f44f34fe1 review suggestions 2022-05-23 21:55:17 +02:00
Alexander Tokmakov
d0f998fb88 Merge branch 'master' into fix_trash 2022-05-23 21:25:56 +02:00
taiyang-li
f13dcad58c Merge branch 'master' into fix_settings 2022-05-23 17:36:45 +08:00
Alexander Tokmakov
c48410b574 fix trash in my code 2022-05-21 02:05:02 +02:00
Nikolai Kochetov
56feef01e7 Move some resources 2022-05-20 19:49:31 +00:00
Anton Popov
cb0e6c2718 mark all operators bool() as explicit 2022-05-20 15:29:54 +00:00
Alexander Tokmakov
e9568d8cbb fixes 2022-05-19 00:00:05 +02:00
Alexander Tokmakov
3bc39f9929 fixes 2022-05-18 21:57:20 +02:00
Alexander Tokmakov
9772924d06 Merge branch 'master' into fix_trash 2022-05-18 17:27:54 +02:00
Alexander Tokmakov
19afeda4b1 fix style 2022-05-18 14:23:52 +02:00
Alexander Tokmakov
538b5cd536 fix more trash 2022-05-18 14:06:52 +02:00
Alexander Tokmakov
5590adeffa fix more trash 2022-05-17 23:04:24 +02:00
Alexander Tokmakov
dea39d8175 fix some trash 2022-05-17 18:22:52 +02:00
taiyang-li
9f3dadf17c Merge branch 'master' into fix_settings 2022-05-17 16:19:29 +08:00
Robert Schulze
e3cfec5b09
Merge remote-tracking branch 'origin/master' into clangtidies 2022-05-16 10:12:50 +02:00
wuxiaobai24
f3e573bfde Merge remote-tracking branch 'upstream/master' into zstd_window_log_max 2022-05-13 20:42:54 +08:00
wuxiaobai24
ab5636a46a fix 2022-05-13 17:52:39 +08:00
mergify[bot]
5856856884
Merge branch 'master' into http_proper_summary_and_exception 2022-05-12 16:43:25 +00:00
kssenii
61f2737e17 Merge master 2022-05-10 19:31:22 +02:00
kssenii
0feda67ec4 Fix progress for insert select 2022-05-10 15:04:08 +02:00
taiyang-li
fd878f7e7b change as requested 2022-05-10 12:06:50 +08:00
Anton Popov
e911900054 remove last mentions of data streams 2022-05-09 19:15:24 +00:00
Robert Schulze
1b81bb49b4
Enable clang-tidy modernize-deprecated-headers & hicpp-deprecated-headers 
Official docs:

  Some headers from C library were deprecated in C++ and are no longer
  welcome in C++ codebases. Some have no effect in C++. For more details
  refer to the C++ 14 Standard [depr.c.headers] section. This check
  replaces C standard library headers with their C++ alternatives and
  removes redundant ones.
 2022-05-09 08:23:33 +02:00
Alexey Milovidov
3755466e8d
Merge pull request #34916 from nic11/pr/content-type 
Support content_type in HTTP handler config
 2022-05-08 16:36:21 +03:00
wuxiaobai24
58755cb156 add zstd_window_log_max setting 2022-05-08 14:02:40 +08:00
Raúl Marín
5190d9e85a Give exception_code an initialization value 2022-05-04 16:06:43 +02:00
Raúl Marín
6919b459ee HTTP: Return a 408 on query timeout 2022-05-04 14:24:34 +02:00
Raúl Marín
d899ef2e48 HTTP: Always return summary data and exception (when possible) 2022-05-04 14:24:04 +02:00
mergify[bot]
64084b5e32
Merge branch 'master' into shared_ptr_helper3 2022-05-03 20:46:16 +00:00
Kruglov Pavel
8d647eff06
Merge pull request #36749 from Avogar/fix-timeouts 
Fix bug in receive timeouts in Hedged requests
 2022-05-02 13:16:03 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying 
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
 2022-05-02 08:46:52 +02:00
Azat Khuzhin
d82f2ff162 Set is_all_data_sent on exceptions too 
This will avoid clickhouse-test complains about left queries after the
test, like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36258/9646487c093a75dc31e3e818417cfad83580b40f/stateful_tests__memory__actions_.html

Follow-up for: #36649
 2022-04-30 13:00:44 +03:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
mergify[bot]
d96c9c5cff
Merge branch 'master' into fix-timeouts 2022-04-28 15:03:19 +00:00
Nikita Mikhaylov
7dc084419e
Check socket is connected in HTTPSession (#36683) 2022-04-28 15:12:25 +02:00
avogar
81f85892eb Fix bug in receive timeouts in Hedged requests 2022-04-28 13:10:27 +00:00
alesapin
8f1a0f6625
Merge pull request #36258 from ClickHouse/keeper-recovery-mode 
clickhouse-keeper support for force recovery
 2022-04-28 11:20:59 +02:00
Kseniia Sumarokova
0d09bb9897
Merge pull request #36699 from amosbird/bettermysqlhandler 
Slightly better mysql handler
 2022-04-27 17:07:35 +02:00
Amos Bird
1cba00478d
Slightly better mysql handler 2022-04-27 18:28:09 +08:00
Azat Khuzhin
6339a48923 Add is_all_data_sent into system.processes 
v2: fix SHOW PROCESSLIST (does not have process list entry)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-26 12:15:00 +03:00
Antonio Andelic
cf022542bd Merge branch 'master' into keeper-recovery-mode 2022-04-26 06:50:02 +00:00
Alexey Milovidov
da1b0b491f
Merge pull request #36487 from FArthur-cmd/fix_certificates 
Add passphrase for certificates
 2022-04-24 04:58:21 +03:00
Alexey Milovidov
3b9c72ba94
Update CertificateReloader.cpp 2022-04-24 04:57:46 +03:00
Kseniia Sumarokova
33bb48106f
Merge pull request #36314 from CurtizJ/print-bad-filenames 
Show names of erroneous files in case of parsing errors while executing table functions
 2022-04-22 13:24:55 +02:00
Antonio Andelic
03779fd463 Merge branch 'master' into keeper-recovery-mode 2022-04-21 12:34:18 +00:00
FArthur-cmd
f0a4009f60 Add passphrase for certificates 2022-04-21 12:26:30 +00:00
Antonio Andelic
bbb0be6a44
Merge pull request #36402 from rschu1ze/clang-tidy-contains 
Activate clang-tidy warning "readability-container-contains"
 2022-04-20 08:15:46 +02:00
Anton Popov
3e361c9759 Merge remote-tracking branch 'upstream/master' into HEAD 2022-04-19 14:18:04 +00:00
Antonio Andelic
3e77340a81
Allow clickhouse-keeper to serve 4-letter commands before quorum (#35992) 
* Use async start for clichkouse-keeper

* Return non active server for 4lw commands if quorum not achieved
 2022-04-19 14:03:00 +02:00
Antonio Andelic
cd548aeb30 Refactor test for multiple cluster size 2022-04-19 08:08:36 +00:00
Antonio Andelic
6c3bf0a5d3 Format files 2022-04-19 08:08:36 +00:00
Antonio Andelic
ff2ebe113c WIP 2022-04-19 08:08:35 +00:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains" 
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
 2022-04-18 23:53:11 +02:00
Alexey Milovidov
1333b4cd89
Merge pull request #36345 from ucasfl/throw-no-data 
Add setting throw_if_no_data_to_insert
 2022-04-18 07:04:16 +03:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia 
Remove remaining parts of Arcadia
 2022-04-18 07:02:54 +03:00
Alexey Milovidov
242919eddd Remove abbreviation 2022-04-18 01:02:49 +02:00
fenglv
0024a08ace fix 
fix typo
 2022-04-17 15:12:07 +00:00
Anton Popov
2de6668b3f show names of erroneous files 2022-04-16 00:10:47 +00:00
Alexey Milovidov
416fa95b87 Remove "Arcadia" build system 2022-04-15 23:57:09 +02:00
Azat Khuzhin
097e2f626d Do not overlap profile events snapshots for queries 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 20:54:56 +03:00
Roman Nikonov
daceac5c0b Merge branch 'master' into pr/content-type 2022-04-10 01:21:28 +03:00
Azat Khuzhin
33d99c8ffb Introduce compatiblity mode for SETTINGS after FORMAT 
Add allow_settings_after_format_in_insert setting, OFF by default.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: s/parser_settings_after_format_compact/allow_settings_after_format_in_insert/ (suggested by vitlibar)
v3: replace ParserSettings with a flag (requested by vitlibar)
 2022-04-07 16:36:34 +03:00
Roman Nikonov
d81a8cd317 Merge branch 'master' into pr/content-type 2022-04-03 20:53:51 +03:00
Antonio Andelic
8cc18c851d
Merge pull request #35705 from ClickHouse/is-secure-client-info 
Added `is_secure` column to `query_log`
 2022-04-01 08:27:20 +02:00
Azat Khuzhin
ff0e46c394 clickhouse-keeper: finish session on client EOF (fixes heap-use-after-free) 
CI founds [1]:

    WARNING: ThreadSanitizer: heap-use-after-free (pid=1)
      Read of size 8 at 0x7b48001e0088 by thread T7 (mutexes: write M2588):
        ...
        1 DB::KeeperTCPHandler::runImpl()::$_0::operator()() const build_docker/../src/Server/KeeperTCPHandler.cpp:384:14 (clickhouse+0x194f974f)
        ...
        8 DB::KeeperDispatcher::setResponse() build_docker/../src/Coordination/KeeperDispatcher.cpp:222:9 (clickhouse+0x1999de82)
        9 DB::KeeperDispatcher::responseThread() build_docker/../src/Coordination/KeeperDispatcher.cpp:161:18 (clickhouse+0x1999d7a7)
        10 DB::KeeperDispatcher::initialize()::$_1::operator()() const build_docker/../src/Coordination/KeeperDispatcher.cpp:273:54 (clickhouse+0x199a55d1)

      Previous write of size 8 at 0x7b48001e0088 by thread T3:
        0 operator delete(void*, unsigned long) <null> (clickhouse+0xa0dad50)
        1 DB::KeeperTCPHandler::~KeeperTCPHandler() build_docker/../src/Server/KeeperTCPHandler.cpp:648:1 (clickhouse+0x194f8ee6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/35772/38873d726d22b15d4c5a284410956927b2e2f95f/integration_tests__thread__actions__[4/4].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-31 09:31:34 +03:00
Azat Khuzhin
38873d726d clickhouse-keeper: fix implementation of server with poll() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-30 21:30:12 +03:00
Azat Khuzhin
9820230e95 clickhouse-keeper: correctly handle closed client connection 
This will avoid noisy message like:

    DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-30 20:29:04 +03:00
Antonio Andelic
e15fccd31b Add is_secure info to HTTP 2022-03-30 06:39:40 +00:00
Antonio Andelic
d14ad227b1 Polish TCP is_secure flag 2022-03-30 06:39:40 +00:00
alesapin
70d410dc4d Fix keeper client timeout bug 2022-03-22 11:41:50 +01:00
Maksim Kita
2fdcf53a76 Fix clang-tidy warnings in Server, Storages folders 2022-03-14 18:17:35 +00:00
alesapin
96c0e9fddf Better cmake 2022-03-11 15:47:07 +01:00
alesapin
33ff2f76e5 Merge branch 'master' into standalone_keeper_build 2022-03-11 13:58:44 +01:00
Roman Nikonov
3b60116e6d Fix merge mistake 2022-03-10 08:57:31 +03:00
Roman Nikonov
83787b9651 Fix support content_type for dynamic_query_handler 2022-03-09 23:48:53 +03:00
alesapin
0c93c9e21d
Merge pull request #35010 from bigo-sg/keeperstatopt 
Keeper atomic stat
 2022-03-07 12:34:28 +01:00
Kseniia Sumarokova
cdb9a05229
Merge pull request #34605 from bigo-sg/add_metric_for_local 
Add cpu/mem metric for clickhouse-local
 2022-03-07 11:40:11 +01:00
Roman Nikonov
fde94e3cb2 Merge branch 'master' into pr/content-type 2022-03-07 13:19:33 +03:00
taiyang-li
1b9987ceb1 fix mistake in check-style 2022-03-05 14:22:56 +08:00
alesapin
0eb7d28192 Building small keeper binary 2022-03-03 21:27:46 +01:00
zhanglistar
9ee0d2c8a0 keeper atomic stat 2022-03-03 14:30:22 +08:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
taiyang-li
f83132bad2 finish dev 2022-03-01 15:54:23 +08:00
taiyang-li
b31440c77a Merge branch 'master' into add_metric_for_local 2022-03-01 12:29:36 +08:00
Filatenkov Artur
af4362e40a
Improve certificate reloader (#34887) 
* add ec support

* Add test

* fix tests and improve code style
 2022-02-28 21:30:02 +03:00
Roman Nikonov
db63b82e1a Support content_type in HTTP handler config 2022-02-26 03:06:17 +03:00
Vitaly Baranov
aee67a6693
Merge pull request #31484 from eungenue/Implement-SSL-X509-certificate-authentication 
Implement ssl x509 certificate authentication
 2022-02-21 11:30:52 +03:00
Vitaly Baranov
cb66a63aa4 Rename header and config setting for consistency. 2022-02-21 07:41:06 +03:00
Vitaly Baranov
765d136d2a A few improvements in the implementation of SSL certificate authentication. 2022-02-21 07:41:02 +03:00
taiyang-li
e6b29167da fix style 2022-02-15 20:26:53 +08:00
taiyang-li
b4440131d0 add metrics for clickhouse-local 2022-02-15 16:25:07 +08:00
Maksim Kita
91bc9cd4cf
Merge pull request #34499 from vitlibar/grpc-send-output-format-back-to-client 
gRPC: Send output format back to client
 2022-02-13 15:34:32 +01:00
Vitaly Baranov
cf2e205b08 Add field output_columns to first Result sent in gRPC protocol. 2022-02-10 21:05:56 +07:00
Vitaly Baranov
bd746fd82b Add fields query_id and time_zone to first Result sent in gRPC protocol. 2022-02-10 21:05:47 +07:00
Vitaly Baranov
f8c8eeceea Add field output_format to first Result sent in gRPC protocol. 2022-02-10 21:05:08 +07:00
Dmitry Novik
d405523598 Fix ProfileEvents sending in INSERT SELECT 2022-02-10 13:27:27 +00:00
Kruglov Pavel
738355b724
Merge pull request #34408 from vitlibar/split-and-rename-compression-fields-in-grpc 
Split and rename compression fields in gRPC
 2022-02-10 15:40:20 +03:00
Anton Popov
298838f891 avoid unnecessary copying of Settings 2022-02-10 12:13:51 +03:00
Vitaly Baranov
1341b4b4de Rename QueryInfo's field result_compression -> transport_compression_type and change its type for better consistency. 
Make the previous field obsolete.
 2022-02-10 02:46:05 +07:00
Vitaly Baranov
1f6b65d39d gRPC: Split compression_type field into input_compression_type and output_compression_type. 
Make the previous field obsolete.
 2022-02-10 02:46:05 +07:00
Vitaly Baranov
b2ba0c4320 Fix inserting to temporary tables via gRPC. 2022-02-07 12:36:17 +07:00
Vitaly Baranov
5dd49a44e6 Add test for propagating OpenTelemetry context via gRPC protocol. 2022-02-06 02:09:56 +07:00
alexey-milovidov
665fdc9187
Merge pull request #31257 from FArthur-cmd/merge_15765 
Merge #15765
 2022-02-04 00:39:57 +03:00
alexey-milovidov
4742cf3074
Update CertificateReloader.cpp 2022-02-03 03:03:46 +03:00
Azat Khuzhin
5472aef084 Fix current_user/current_address for interserver mode 
Before this patch current_user/current_address will be preserved from
the previous query.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-02 17:44:19 +03:00
Sergei Trifonov
68bc456830
Merge pull request #34223 from azat/bump-fmt 
Bump fmtlib from 7.0.0 to 8.1.1
 2022-02-02 00:03:25 +03:00
Vitaly Baranov
2ef2479ddc
Add const to make clang-tidy happy 2022-02-01 18:38:52 +07:00
Azat Khuzhin
de80014eb5 Use fmt::format over Poco::format in PostgreSQLHandler 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:04 +03:00
Azat Khuzhin
bedf208cbd Use fmt::runtime() for LOG_* for non constexpr 
Here is oneliner:

    $ gg 'LOG_\(DEBUG\|TRACE\|INFO\|TEST\|WARNING\|ERROR\|FATAL\)([^,]*, [a-zA-Z]' -- :*.cpp :*.h | cut -d: -f1 | sort -u | xargs -r sed -E -i 's#(LOG_[A-Z]*)\(([^,]*), ([A-Za-z][^,)]*)#\1(\2, fmt::runtime(\3)#'

Note, that I tried to do this with coccinelle (tool for semantic
patchin), but it cannot parse C++:

    $ cat fmt.cocci
    @@
    expression log;
    expression var;
    @@

    -LOG_DEBUG(log, var)
    +LOG_DEBUG(log, fmt::runtime(var))

I've also tried to use some macros/templates magic to do this implicitly
in logger_useful.h, but I failed to do so, and apparently it is not
possible for now.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

v2: manual fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:03 +03:00
FArthur-cmd
3dc64f031a correct code style 2022-01-31 12:21:22 +00:00
FArthur-cmd
891411a2be add try catch to avoid errors on start 2022-01-31 12:13:28 +00:00
Andre Marianiello
f5cbc6da43 Propagate trace context from GRPC calls 2022-01-28 14:27:04 -05:00
FArthur-cmd
cb4ad97d45 add test 2022-01-28 17:37:52 +00:00
Filatenkov Artur
82b0fc0bd0
Merge branch 'master' into merge_15765 2022-01-24 13:38:53 +03:00
alesapin
5ec76cd827 Merge branch 'master' into better_hashmap 2022-01-21 12:47:41 +03:00
Azat Khuzhin
6acb4d6ac5 Remove unbundled gRPC support 2022-01-20 08:47:16 +03:00
Nikita Mikhaylov
e3de3889e3
Revert "Revert "Ignore parse failure of opentelemetry header (#32116)" (#33594)" (#33595) 2022-01-19 20:53:39 +03:00
alesapin
3d325aacf6 Merge branch 'master' into better_hashmap 2022-01-19 12:31:23 +03:00
Eugene Galkin
0681bb4bed remove unused var and function declaration 2022-01-17 18:48:04 +03:00
tavplubix
34b3894422
Merge pull request #33676 from ClickHouse/fix-8340 
Correctly determine current database if CREATE TEMPORARY TABLE AS SELECT is queried inside a named HTTP session.
 2022-01-17 16:52:09 +03:00
alesapin
4fb8761a22 Merge branch 'master' into better_hashmap 2022-01-17 15:51:50 +03:00
Eugene Galkin
30a741a19f fix header authentication check condition 2022-01-17 15:47:25 +03:00
Eugene Galkin
f2612d42dc Implement SSL X.509 certificate authentication 2022-01-17 15:01:38 +03:00
Eugene Galkin
f46dca4793 support x509 ssl certificate authentication 2022-01-17 15:01:38 +03:00
alesapin
b2271cc2d9
Merge pull request #33288 from JackyWoo/add_lower_bound_session_timeout_to_keeper 
Add lower bound session timeout to keeper
 2022-01-17 14:06:15 +03:00
Alexey Milovidov
4ab7dfc741 Fix #8340 2022-01-16 08:12:55 +03:00
alesapin
ec6840f49c Make ZooKeeper client better interpret keeper server connection reject 2022-01-13 17:12:10 +03:00
Nikita Mikhaylov
340ec0fd20
Revert "Ignore parse failure of opentelemetry header (#32116)" (#33594) 
This reverts commit 4e5f9c8ff7.
 2022-01-13 16:29:18 +04:00
mergify[bot]
085492614b
Merge branch 'master' into add_lower_bound_session_timeout_to_keeper 2022-01-13 12:21:46 +00:00
Frank Chen
4e5f9c8ff7
Ignore parse failure of opentelemetry header (#32116) 2022-01-13 16:16:10 +04:00
Azat Khuzhin
e8c7e5e8f0 Fix typo in sendMergeTreeReadTaskRequestAssumeLocked() 2022-01-07 20:04:57 +03:00
alesapin
f8114126ed Use HashMap and arena with free lists for keeper 2021-12-30 19:21:49 +03:00
JackyWoo
569ce62e8d use session_timeout as session timeout uper bound 2021-12-30 17:18:51 +08:00
JackyWoo
19b5394be8 fix tests 2021-12-30 15:49:48 +08:00
JackyWoo
d35e5f8319 add lower bound session timeout to keeper 2021-12-29 20:59:01 +08:00
Vitaly Baranov
f80a3e3247
Merge pull request #30549 from aiven/kmichel-server-reload 
Start/stop servers when `listen_host`/`*_port` changes
 2021-12-27 14:51:00 +03:00
Alexey Milovidov
29d28c531f Move code around to avoid dlsym on Musl 2021-12-24 12:25:27 +03:00
Kevin Michel
ffc1fca296
Start/stop servers when listen_host/*_port changes 
This allows starting and stopping separately each protocol server
without restarting ClickHouse.

This also allows adding or removing `listen_host` entries, which
start and stops servers for all enabled ports.

When stopping a server, the listening socket is immediately closed
(and available for another server).

Protocols with persistent connections try to wait for any currently
running query to finish before closing the connection, but idle
connection are closed quickly (depending on how often the protocol
is polled).

An extra ProfileEvent is added, `MainConfigLoads`, it is
incremented every time the configuration is reloaded. This helps
when trying to assess whether the new configuration was applied.
 2021-12-24 08:26:02 +01:00
alexey-milovidov
8c9843caf2
Merge pull request #32982 from azat/http-context-leak 
Fix Context leak in case of cancel_http_readonly_queries_on_client_close
 2021-12-23 07:58:18 +03:00
mergify[bot]
04f727e84c
Merge branch 'master' into read-fix 2021-12-22 13:49:04 +00:00
alesapin
754785fee5 Better container 2021-12-21 11:07:24 +03:00
Azat Khuzhin
08ced87880 Rework MultipartReadBuffer::skipToNextBoundary() 
As suggested by @vitlibar
 2021-12-21 08:38:41 +03:00
Azat Khuzhin
d7c7a91f66 Check that at least one boundary exist in HTMLForm::readMultipart() 2021-12-21 08:38:41 +03:00
Azat Khuzhin
4dfa9324d1 Check for EOF in MultipartReadBuffer::skipToNextBoundary() 
Otherwise you may hit internal assertion when client goes away:

    clickhouse-server: ./src/Server/HTTP/HTMLForm.cpp:245: bool DB::HTMLForm::MultipartReadBuffer::skipToNextBoundary(): Assertion `boundary_hit' failed.

Fixes: 02151_http_s_structure_set_eof
 2021-12-21 08:38:41 +03:00
JackyWoo
7a19570853 keeper handler should remove operation when response sent 2021-12-21 11:43:14 +08:00
Azat Khuzhin
a9f80dd705 Fix Context leak in case of cancel_http_readonly_queries_on_client_close 
To handle cancel_http_readonly_queries_on_client_close=true context
attaches progress callback with holding a copy of the current context,
and this creates recursive context reference and so shared_ptr will be
never released, [1] contains simplest example.

One example of a memory leak, external tables passed with HTTP query
(s_structure + file upload), since they are stored in the context, and
can occupate enough memory to make it visible to user.

Note, that the lifetime of the Context should be fine, since callback
can be called only when query is executed.

  [1]: Example of recursive reference

       ```c
       #include <memory>
       #include <functional>
       #include <cassert>

       bool released = true;

       class Context : public std::enable_shared_from_this<Context>
       {
           using Func = std::function<void()>;
           Func func;

       public:
           Context()  { released = false; }
           ~Context() { released = true; }
           Context(const Context & context) = delete;

           void addCallback(Func func_)
           {
               func = func_;
           }
       };

       int main()
       {
           {
               auto context = std::make_shared<Context>();
               context->addCallback([context]() {});
           }
           assert(released == true);
       }
       ```
 2021-12-20 21:05:38 +03:00
Azat Khuzhin
dd8c64e380 Set QueryScope earlier for HTTP queries 2021-12-20 21:05:38 +03:00
Vitaly Baranov
e61d3eef0c
Merge pull request #32747 from vitlibar/improve-grpc-compression 
Improve gRPC compression support
 2021-12-20 19:44:58 +03:00
Azat Khuzhin
6aebc3e94c Do not loose ProfileEvents in case of thread destroyed before 
v2: drop std::move and add copy ctor for ProfileEvents::Counter::Snapshot
v2: remove std::move
 2021-12-19 23:43:05 +03:00
Vitaly Baranov
708439b036 Support customized compression for input/output data in gRPC protocol. 2021-12-19 16:29:44 +03:00
Alexey Milovidov
2c68192bb6 Fix strange code in TCPHandler 2021-12-12 05:35:33 +03:00
Azat Khuzhin
57c027be0e Remove arcadia build support 2021-12-11 21:25:23 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
alexey-milovidov
96ec92c7cd
Merge pull request #31155 from ClickHouse/incremental-profile-events 
Send incremental profile events to client
 2021-12-01 04:04:41 +03:00
alesapin
c1f38752c4
Merge pull request #31859 from JackyWoo/keeper_timeout 
keeper session timeout doesn't work
 2021-11-28 16:28:17 +03:00
Raúl Marín
051dddd8df Reduce dependencies on ASTIdentifier.h 
Goes from rebuilding 483 objects to 165 when it's modified
 2021-11-26 16:49:40 +01:00
JackyWoo
4c6ad94109 fix ch keeper session timeout doesn't work 2021-11-26 18:17:49 +08:00
Kruglov Pavel
690b460c3d
Fix style 2021-11-24 22:36:36 +03:00
Kruglov Pavel
ad7768e42a
Fix 2021-11-24 22:25:41 +03:00
Kruglov Pavel
b20f4439df
Make better 2021-11-24 22:25:03 +03:00
avogar
1bd8c57908 Fix 2021-11-24 20:21:21 +03:00
avogar
7c84d95908 Fix possible Logical error: Cannot write to finalized buffer 2021-11-24 18:32:41 +03:00
alesapin
646cf38213
Merge pull request #28981 from JackyWoo/add_4_letter_words_commands 
Add four letter commands to keeper
 2021-11-24 13:32:36 +03:00
Kruglov Pavel
b63b47f0f0
Merge pull request #31265 from Avogar/fix-write-buffers 
Fix and refactor WriteBiffer-s a little
 2021-11-23 16:46:09 +03:00
Kruglov Pavel
6d942401bb
Merge pull request #31510 from azat/fix-progress-for-insert 
Fix progress for short INSERT SELECT queries
 2021-11-23 16:37:06 +03:00
alesapin
16f034fe4c Flush bytes after 4 letter commands 2021-11-23 13:31:10 +03:00
alesapin
5825242a7d Merge branch 'add_4_letter_words_commands' of github.com:JackyWoo/ClickHouse into JackyWoo_add_4_letter_words_commands 2021-11-22 18:02:22 +03:00
alesapin
35de9e42d9
Merge branch 'master' into add_4_letter_words_commands 2021-11-22 18:02:02 +03:00
alesapin
1cf204c581 Small changes 2021-11-21 15:28:45 +03:00
Alexander Tokmakov
856502fa81 log long operations in Keeper 2021-11-20 20:39:31 +03:00
Kruglov Pavel
d9c1a0c8ec
Merge branch 'master' into fix-write-buffers 2021-11-20 17:48:24 +03:00
Azat Khuzhin
dd7f780978 Fix Progress for short INSERT SELECT queries 
Note, that we cannot send Progress for regular INSERT (w/ VALUES) w/o
breaking protocol compatibility, but it can be done by increasing
revision.
 2021-11-20 16:41:35 +03:00
Azat Khuzhin
0b4d855c08 Reduce copy-paste in TCPHandler::processInsertQuery() 2021-11-20 16:41:35 +03:00
alesapin
3f8f08e816 Slightly better 2021-11-19 12:30:58 +03:00
alesapin
dcec086573 Small refactoring 2021-11-18 23:17:22 +03:00
JackyWoo
27d50edc59 adjust log output 2021-11-18 15:49:43 +08:00
JackyWoo
007366b506 remove sizeOf method from SnapshotableHashTable 2021-11-18 12:35:32 +08:00
Kruglov Pavel
f27fcf8372
Merge pull request #31004 from Avogar/output-formats 
Small refactoring in formats
 2021-11-17 20:44:46 +03:00
alesapin
5f3eca4cd1 Merge branch 'master' into JackyWoo_add_4_letter_words_commands 2021-11-17 12:59:20 +03:00