Commit Graph

53 Commits

Author SHA1 Message Date
Nikita Mikhaylov
aff486dac1
Additional scripts for packager (#55000) 2023-09-26 15:59:57 +02:00
Azat Khuzhin
630eddbbbc Disable forwarding signals by watchdog in systemd service
With default KillMode=control-group, systemd will send signals to all
processes in cgroup and this will lead to server will be terminated
forcefully due to double signal.

    2023.09.19 12:47:06.369090 [ 763 ] {} <Information> Application: Received termination signal (Terminated)
    2023.09.19 12:47:06.369141 [ 762 ] {} <Debug> Application: Received termination signal.
    2023.09.19 12:47:06.369215 [ 763 ] {} <Information> Application: Received termination signal (Terminated)
    2023.09.19 12:47:06.369225 [ 763 ] {} <Information> Application: This is the second termination signal. Immediately terminate.
    2023.09.19 12:47:06.400959 [ 761 ] {} <Information> Application: Child process exited normally with code 143.

Someone may naively think that, hey, I can change KillMode to
process/mixed, but this will not work either, because in this case
systemd cannot wait for the $MainPID (and main_pid_alien=true in
systemd's sources), because it is not a child of systemd, and this will
lead to double signal again:

    2023.09.19 16:24:19.694473 [ 3118 ] {} <Information> Application: Received termination signal (Terminated)
    2023.09.19 16:24:19.694894 [ 3118 ] {} <Information> Application: Received termination signal (Terminated)
    2023.09.19 16:24:19.695060 [ 3118 ] {} <Information> Application: This is the second termination signal. Immediately terminate.

And this is because it sends signal firstly on a normal termnation and
then when it cleans up left over processes:

    clickhouse-server.service: Found left-over process 3117 (clickhouse-serv) in control group while starting unit. Ignoring.

And yes, even though it prints "Ignoring" here (I guess it is related to
the fact that it can be ignored if the signal will not be handled)

Here is a proof of double signal by systemd:

    # pgrep clickhouse-serv | xargs strace  -e /kill -fp
    strace: Process 3117 attached with 469 threads
    [pid  3582] --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=1, si_uid=0} ---
    [pid  3580] --- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=1, si_uid=0} ---
    [pid  3582] --- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=1, si_uid=0} ---
    [pid  3580] --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=1, si_uid=0} ---
                                                                      ^^^
    [pid  3118] tgkill(3117, 3118, SIGTERM) = 0 # and this is a force termination

So yes, there is no other way except for disabling signal forwarding.

*Well, there is another way, but I guess it is will be unwelcome (even
though systemd can be configured in multiple ways right now, and there
is even systemd-oomd instead of clickhouse'es watchdog) - disable
watchdog completelly.*

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-09-19 22:09:38 +02:00
Azat Khuzhin
4c30156595 Disable killing the server by systemd (that may lead to data loss for Buffer)
Default systemd's timeout for sending SIGKILL after SIGTERM is 1m30s
(TimeoutStopSec), which is can be not enough to wait for queries or
shutdown the storages.

And besides in this case shutdown_wait_unfinished server settings are
ignored.

So let's just disable this systemd logic and rely on
shutdown_wait_unfinished instead.

But note shutting down the storages can take a while, but it is better
to give it time instead of killing the process, since killing may lead
to data loss.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-09-19 22:09:38 +02:00
Tomas Barton
7a12488db5
Fix clickhouse-keeper-client symlink 2023-09-18 10:40:15 +02:00
Pradeep Chhetri
0b1f6faec2 Add clickhouse-keeper-converter symlink to clickhouse-keeper package 2023-08-14 15:13:26 +08:00
Pradeep Chhetri
70bb66e39d Add clickhouse-keeper-client symlink to clickhouse-keeper package 2023-08-07 23:21:18 +08:00
Mikhail f. Shiryaev
8095ac2eef
Merge branch 'master' into keeper-client-package 2023-08-02 11:30:41 +02:00
Azat Khuzhin
1fe8076b94 Fix capabilities installed via systemd service (fixes netlink/IO priorities)
CapabilityBoundingSet that contained in systemd unit before is about
allowing to set some capabilities, not about granting them.

To grant them you need to use AmbientCapabilities.

And if you do not use 'clickhouse install' then:
- IO priorities was unavailable (since they requires CAP_SYS_NICE)
- For taskstats the procfs was used instead of netlink

Not a big deal, but still.

Here how it had been tested:

    $ systemd-run -p CapabilityBoundingSet=CAP_NET_ADMIN --shell
    root:/etc (master)# capsh --print
    Current: cap_net_admin=ep
    Bounding set =cap_net_admin
    Ambient set =

    $ systemd-run -p User=azat -p CapabilityBoundingSet=CAP_NET_ADMIN --shell
    azat:/etc$ capsh --print
    Current: =
    Bounding set =cap_net_admin
    Ambient set =

    $ systemd-run -p User=azat -p AmbientCapabilities=CAP_NET_ADMIN -p CapabilityBoundingSet=CAP_NET_ADMIN --shell
    azat:/etc$ capsh --print
    Current: cap_net_admin=eip
    Bounding set =cap_net_admin
    Ambient set =cap_net_admin

Note, if you are running it under root (without changing user) you don't
need to specify AmbientCapabilities additionally, because root has all
capabilities by default and they had been inherited.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-21 13:57:31 +02:00
Mikhail f. Shiryaev
57e9cbfe58
Add clickhouse-keeper-client to the packages 2023-07-06 11:47:54 +02:00
Alexey Milovidov
98ae9be734
Revert "Added tests for ClickHouse apps help and fixed help issues" 2023-04-21 01:54:34 +03:00
Yatsishin Ilya
9bddda6a6e move clickhouse-su to clickhouse-server packager 2023-04-17 09:56:32 +00:00
Yatsishin Ilya
afda642a47 Try to add lost symlinks in packages 2023-04-13 07:10:59 +00:00
Thomas Casteleyn
b8d2432296
Fix startup on older systemd versions
Fixes #47293
2023-03-17 17:56:56 +01:00
cmsxbc
6537029ccc
fix: keeper systemd service file include invalid inline comment 2023-03-02 02:15:09 +08:00
Palash Goel
3615c6f4fb
Fix typo in service definition 2023-03-01 10:37:21 +05:30
Mikhail f. Shiryaev
cf067c1b67
Use /etc/default/clickhouse in systemd too 2023-02-28 12:00:08 +01:00
Azat Khuzhin
19d0c929d6 Disable timeout logic for starting clickhouse-server from systemd service
After ClickHouse became systemd aware (#43400), it waits not more then
TimeoutStartSec (1m30sec by default), while before it simply ensures
that the process is there.

And likely 1m30sec can be not enough for some cluster, and this will
lead to endless restarts.

At first I've increased it to 10min, but there was a comment about that
this is not enough, and I agree with this.

But I'm not sure that using "inifinity" is a good option, but I cannot
think of any downsides of this.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-21 10:08:15 +01:00
cmsxbc
1ad9e217e5 chore: remove starting blank 2023-02-16 19:24:46 +08:00
SuperDJY
7de008a58a Fix systemd service file wrong inline comment
There is no inline comment in systemd unit file.
2023-02-16 19:24:46 +08:00
Mikhail f. Shiryaev
7a7b16dd13
Do not use debconf/confmodule in tgz packages 2023-02-03 12:16:19 +01:00
Mikhail f. Shiryaev
496701bd59
Add necessary postinst steps for clickhouse-keeper 2023-02-03 12:16:18 +01:00
Mikhail f. Shiryaev
360ea59ecf
Remove unused variables from clickhouse-server.postinstall 2023-02-03 12:16:18 +01:00
Mikhail f. Shiryaev
548b89b7e0
Use systemd %-specifiers in clickhouse-server.service 2023-02-03 12:16:16 +01:00
Mikhail f. Shiryaev
26a292dafc
Add systemd service-file for clickhouse-keeper 2023-02-03 12:16:10 +01:00
Alexey Milovidov
869428f228 Remove adduser dependency 2023-01-07 01:45:54 +01:00
Alexey Milovidov
30fd9b8317
Merge pull request #43597 from ClickHouse/sd_notify_followup_fixes
Followup fixes for systemd notification (#43400)
2022-12-17 04:35:05 +03:00
Mikhail f. Shiryaev
e1584c5d3a
Provide sha512 hashsums for tgz artifacts 2022-12-07 21:44:19 +01:00
Alexander Gololobov
688e6fe714 Send MAINPID= notification from the parent (watchdog) process to make systemd handle READY=1 notifiaction from the child 2022-12-05 23:03:40 +01:00
Коренберг ☢️ Марк
6dbeee69d6 Fix ##8685 - added systemd sd_notify implementation 2022-11-20 19:34:12 +02:00
Mikhail f. Shiryaev
5adb312c29
Fix preserving user configs in rpm packages 2022-11-03 16:58:12 +01:00
Azat Khuzhin
043c4163f4 Use nfpm packager for archlinux packages
Since nfpm v2.20.0 it can build packages for archlinux [1].

  [1]: https://github.com/goreleaser/nfpm/pull/543

Something like this:

    $ mkdir -p /src/packages/root
    $ cmake -DCMAKE_INSTALL_PREFIX=/usr /src
    $ DESTDIR=/src/packages/root ninja install
    $ cd /src/packages
    $ ./build --archlinux

    $ root@s3:/src/packages# ./build --archlinux
    Current version is 22.10.1.1
    Building archlinux package for clickhouse-client.yaml
    using archlinux packager...
    created package: /src/clickhouse-client-22.10.1.1-1-x86_64.pkg.tar.zst
    ...

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-18 18:33:48 +03:00
Azat Khuzhin
ff146211a5 Do not build deb packages by default
CI should be fine, since it set MAKE_DEB, see next commit.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-18 18:33:48 +03:00
Azat Khuzhin
af28937cac packages: do not use type=config with directories
archlinux packager does not support this yet.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-18 18:33:48 +03:00
Chun-Sheng, Li
33edab5042
Adding cron config checking before running sed cmd (#42081)
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2022-10-10 13:34:38 +02:00
Mikhail f. Shiryaev
5d8b8a7db6
Add deb Source for packages 2022-09-19 20:42:02 +02:00
Denny Crane
6e5ba2648d
Update clickhouse-server.init 2022-09-13 20:54:56 -03:00
Mikhail f. Shiryaev
1204b9e303
Fix doinst.sh generator for tgz packages 2022-08-26 20:09:56 +02:00
Mikhail f. Shiryaev
517f821e94
Clean out our clickhouse-server.service from /etc 2022-07-18 15:02:01 +02:00
Mikhail f. Shiryaev
7c812df47e
Merge pull request #38580 from ClickHouse/artifactory-arm64
Push arm64 packages to artifactory and release assets
2022-07-04 12:31:52 +02:00
Mikhail f. Shiryaev
7388b6fb55
Test and build clickhouse-diagnostics in release builds 2022-07-04 10:59:37 +02:00
Mikhail f. Shiryaev
e76ebdfb47
Change clickhouse-{client,server} packages architecture 2022-06-30 16:31:31 +02:00
Alexey Milovidov
c9aac5dd02 Fix strange whitespace (or I do not know YAML) 2022-04-24 22:38:59 +02:00
Denny Crane
4debd3b609 fix service start with systemd 2022-04-02 19:26:41 -03:00
alesapin
37e1160c67
Update packages/clickhouse-keeper-dbg.yaml
Co-authored-by: Mikhail f. Shiryaev <felixoid@clickhouse.com>
2022-03-28 11:18:16 +02:00
alesapin
08db628cc0
Update packages/clickhouse-keeper.yaml
Co-authored-by: Mikhail f. Shiryaev <felixoid@clickhouse.com>
2022-03-28 11:18:11 +02:00
alesapin
543c46d2fd Fix dbg package 2022-03-25 17:29:15 +01:00
alesapin
86dcbe100c Add dbg symbols 2022-03-25 11:25:42 +01:00
alesapin
c95a9971bc Merge branch 'master' into clickhouse-keeper 2022-03-25 11:22:57 +01:00
Mikhail f. Shiryaev
572b15a63c
Fix paths and some cmake typos 2022-03-24 23:11:19 +01:00
alesapin
e790a73081 Simplify strip for new packages 2022-03-23 15:14:30 +01:00