Commit Graph

125238 Commits

Author SHA1 Message Date
Robert Schulze
acb6acc0e6
Merge pull request #54801 from rschu1ze/qc-throw
Query Cache: Reject queries with non-deterministic functions by default
2023-09-20 15:30:09 +02:00
Robert Schulze
c75f7c8434
Correct merge result 2023-09-20 13:27:11 +00:00
Robert Schulze
5729871605
Merge remote-tracking branch 'rschu1ze/master' into qc-throw 2023-09-20 13:25:51 +00:00
alesapin
bfca5f735a
Merge pull request #54438 from canhld94/fix_http_timeout
Pass http retry timeout as milliseconds
2023-09-20 15:21:17 +02:00
Alexander Tokmakov
06513f60eb
Revert "refine error code of duplicated index in create query" 2023-09-20 15:16:39 +02:00
slvrtrn
fb0ba86489 Merge remote-tracking branch 'origin' into tableau-online-information-schema 2023-09-20 15:14:40 +02:00
Antonio Andelic
2805ebf2b2 Set correct size for signal pipe buffer 2023-09-20 12:26:06 +00:00
Azat Khuzhin
9c3fb64106 Check type of the filter expressions while filtering by virtual columns
This should fix filtering by virtual columns with non-uint8 types, i.e.
queries like:

    SELECT * FROM data WHERE 1.0

Fixes: 02346_full_text_search
Fixes: 00990_hasToken_and_tokenbf
v2: move check out from is_const to filterBlockWithQuery(), since in is_const there is no way to validate sets.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-09-20 13:36:25 +02:00
Kruglov Pavel
165b0f0afe
Merge pull request #54807 from ClickHouse/own
Make clickhouse-local logging (server_logs_file) prepend timestamps etc
2023-09-20 13:20:04 +02:00
flynn
dfa95ecc2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into comment 2023-09-20 11:16:33 +00:00
Nikolai Kochetov
0bc41bab74
Merge pull request #54544 from kitaisreal/key-condition-in-function-different-types
KeyCondition IN function support different types
2023-09-20 12:54:01 +02:00
Kruglov Pavel
49ee14f701
Merge pull request #54809 from ClickHouse/pqmeta
Prevent ParquetMetadata reading 40 MB from each file unnecessarily
2023-09-20 12:53:22 +02:00
Duc Canh Le
1200e03f20
correctly implement swap 2023-09-20 18:51:31 +08:00
János Benjamin Antal
eb841d59c4
Merge pull request #54735 from bigo-sg/improve_json_query_2
Improve json sql functions by resuing `GeneratorJSONPath`
2023-09-20 12:50:08 +02:00
robot-clickhouse
428a689196
Merge pull request #54810 from ClickHouse/imp-invalid-identifier
Use appropriate error code instead of LOGICAL_ERROR
2023-09-20 12:32:00 +02:00
Alexey Milovidov
d31a6f253e
Merge pull request #54816 from azat/tests/fix-test_backup_restore_on_cluster
Fix test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup flakiness
2023-09-20 13:30:19 +03:00
vdimir
37123d06b3
Merge pull request #54799 from ClickHouse/vdimir/update_pr_status_label_no_pending
Do not set PR status label
2023-09-20 12:11:57 +02:00
SmitaRKulkarni
135df6870c
Merge pull request #54706 from ClickHouse/replacing_merge_tree_with_clean_deleted_rows_disabled
Fix ReplacingMergeTree with vertical merge and cleanup
2023-09-20 12:06:55 +02:00
alesapin
2ff36b6e0c
Merge pull request #54768 from ClickHouse/better_exception_in_check_data_part
Better exception message in checkDataPart
2023-09-20 12:04:46 +02:00
SmitaRKulkarni
d8adf05de2
Added a new column _block_number (#47532)
Added a new virtual column _block_number which is persisted on merges when allow_experimental_block_number_column is enabled
2023-09-20 11:31:12 +02:00
Duc Canh Le
e6a3f3ed01 get rid of unique pointer
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-09-20 09:31:03 +00:00
Azat Khuzhin
d9a634eb0f Fix filtering parts with indexHint for non analyzer
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit ffa82e9297)
2023-09-20 11:29:35 +02:00
vdimir
65baf129bf
Do not set PR status label 2023-09-20 09:21:58 +00:00
Igor Nikonov
eb8f9f1e76 Remove config files sizes check 2023-09-20 09:21:12 +00:00
Robert Schulze
cbcff6ef08
Merge remote-tracking branch 'rschu1ze/master' into clang-17 2023-09-20 09:15:58 +00:00
Robert Schulze
a175a7e0fc
Use error name instead of error code 2023-09-20 09:08:05 +00:00
Duc Canh Le
5ace2a15db
Update src/Common/isLocalAddress.cpp
Co-authored-by: Igor Nikonov <954088+devcrafter@users.noreply.github.com>
2023-09-20 16:46:32 +08:00
Vitaly Baranov
b1bddae5ce Rewrite the test to check the query plan too. 2023-09-20 10:26:29 +02:00
Igor Nikonov
f676de6625
Merge pull request #54817 from ClickHouse/serxa-patch-3
Doc. Update query-complexity.md
2023-09-20 10:10:50 +02:00
Mikhail f. Shiryaev
4882b99715
Merge pull request #54310 from ClickHouse/ci-libfuzzer-integration
CI: libFuzzer integration
2023-09-20 10:01:39 +02:00
robot-ch-test-poll2
37f732f622
Merge pull request #54808 from ClickHouse/eof
Prevent parquet schema inference reading the first 1 MB of the file unnecessarily
2023-09-20 09:49:42 +02:00
Duc Canh Le
2a5b4ee29a Merge branch 'master' into fix_http_timeout
To trigger CI pipeline
2023-09-20 07:22:02 +00:00
Igor Nikonov
ba37dacba6
Merge pull request #54334 from ClickHouse/pr-distributed-prefer-localhost-replica
Fix: parallel replicas over distributed with prefer_localhost_replica=1
2023-09-20 09:00:32 +02:00
Duc Canh Le
074ee40e3b avoid excessive calls to getifaddrs
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-09-20 06:55:39 +00:00
Sergei Trifonov
696ab745aa
Doc. Update query-complexity.md 2023-09-20 07:51:27 +02:00
Azat Khuzhin
56dca144cc Fix test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup flakiness
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-09-20 07:09:51 +02:00
pufit
34aecc0bf3 Adjusting num_streams by expected work in StorageS3 2023-09-19 23:05:48 -04:00
taiyang-li
d59db55b14 add uts about array join 2023-09-20 10:23:47 +08:00
lgbo-ustc
e9e5c7a09b update 2023-09-20 08:44:30 +08:00
robot-clickhouse
c7c4166a73
Merge pull request #54670 from ClickHouse/fix-parser-unit-tests
Fix parser unit tests
2023-09-20 02:24:27 +02:00
Yakov Olkhovskiy
e8cd429453 add error code INVALID_IDENTIFIER 2023-09-19 22:11:58 +00:00
Michael Kolupaev
c856ec4087 Prevent ParquetMetadata reading 40 MB from each file unnecessarily 2023-09-19 21:58:50 +00:00
Michael Kolupaev
7271cfd187 Prevent parquet schema inference reading the first 1 MB of the file unnecessarily 2023-09-19 21:58:12 +00:00
slvrtrn
636fc506aa Replace information_schema view columns aliases with uppercase 2023-09-19 23:42:40 +02:00
Michael Kolupaev
fa51399ede Make clickhouse-local logging (server_logs_file) prepend timestamps etc 2023-09-19 21:39:59 +00:00
Igor Nikonov
e28f4e819c
Merge branch 'master' into parallel-replicas-not-enough-replicas 2023-09-19 23:05:24 +02:00
Igor Nikonov
3b4ff5a3d0
Merge branch 'master' into pr-distributed-prefer-localhost-replica 2023-09-19 23:01:30 +02:00
Azat Khuzhin
630eddbbbc Disable forwarding signals by watchdog in systemd service
With default KillMode=control-group, systemd will send signals to all
processes in cgroup and this will lead to server will be terminated
forcefully due to double signal.

    2023.09.19 12:47:06.369090 [ 763 ] {} <Information> Application: Received termination signal (Terminated)
    2023.09.19 12:47:06.369141 [ 762 ] {} <Debug> Application: Received termination signal.
    2023.09.19 12:47:06.369215 [ 763 ] {} <Information> Application: Received termination signal (Terminated)
    2023.09.19 12:47:06.369225 [ 763 ] {} <Information> Application: This is the second termination signal. Immediately terminate.
    2023.09.19 12:47:06.400959 [ 761 ] {} <Information> Application: Child process exited normally with code 143.

Someone may naively think that, hey, I can change KillMode to
process/mixed, but this will not work either, because in this case
systemd cannot wait for the $MainPID (and main_pid_alien=true in
systemd's sources), because it is not a child of systemd, and this will
lead to double signal again:

    2023.09.19 16:24:19.694473 [ 3118 ] {} <Information> Application: Received termination signal (Terminated)
    2023.09.19 16:24:19.694894 [ 3118 ] {} <Information> Application: Received termination signal (Terminated)
    2023.09.19 16:24:19.695060 [ 3118 ] {} <Information> Application: This is the second termination signal. Immediately terminate.

And this is because it sends signal firstly on a normal termnation and
then when it cleans up left over processes:

    clickhouse-server.service: Found left-over process 3117 (clickhouse-serv) in control group while starting unit. Ignoring.

And yes, even though it prints "Ignoring" here (I guess it is related to
the fact that it can be ignored if the signal will not be handled)

Here is a proof of double signal by systemd:

    # pgrep clickhouse-serv | xargs strace  -e /kill -fp
    strace: Process 3117 attached with 469 threads
    [pid  3582] --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=1, si_uid=0} ---
    [pid  3580] --- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=1, si_uid=0} ---
    [pid  3582] --- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=1, si_uid=0} ---
    [pid  3580] --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=1, si_uid=0} ---
                                                                      ^^^
    [pid  3118] tgkill(3117, 3118, SIGTERM) = 0 # and this is a force termination

So yes, there is no other way except for disabling signal forwarding.

*Well, there is another way, but I guess it is will be unwelcome (even
though systemd can be configured in multiple ways right now, and there
is even systemd-oomd instead of clickhouse'es watchdog) - disable
watchdog completelly.*

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-09-19 22:09:38 +02:00
Azat Khuzhin
5e90cd413b Add ability to disable forwarding signals to the child by watchdog
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-09-19 22:09:38 +02:00
Azat Khuzhin
4c30156595 Disable killing the server by systemd (that may lead to data loss for Buffer)
Default systemd's timeout for sending SIGKILL after SIGTERM is 1m30s
(TimeoutStopSec), which is can be not enough to wait for queries or
shutdown the storages.

And besides in this case shutdown_wait_unfinished server settings are
ignored.

So let's just disable this systemd logic and rely on
shutdown_wait_unfinished instead.

But note shutting down the storages can take a while, but it is better
to give it time instead of killing the process, since killing may lead
to data loss.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-09-19 22:09:38 +02:00