2020-04-03 13:23:32 +00:00
---
2022-08-28 14:53:34 +00:00
slug: /en/operations/troubleshooting
2022-04-09 13:29:05 +00:00
sidebar_position: 46
sidebar_label: Troubleshooting
2022-09-12 17:27:52 +00:00
title: Troubleshooting
2020-04-03 13:23:32 +00:00
---
2020-03-21 04:11:51 +00:00
- [Installation ](#troubleshooting-installation-errors )
- [Connecting to the server ](#troubleshooting-accepts-no-connections )
- [Query processing ](#troubleshooting-does-not-process-queries )
- [Efficiency of query processing ](#troubleshooting-too-slow )
2019-02-04 13:30:28 +00:00
2020-03-18 18:43:51 +00:00
## Installation {#troubleshooting-installation-errors}
2019-02-04 13:30:28 +00:00
2020-04-08 14:22:25 +00:00
### You Cannot Get Deb Packages from ClickHouse Repository with Apt-get {#you-cannot-get-deb-packages-from-clickhouse-repository-with-apt-get}
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
- Check firewall settings.
2022-04-11 05:01:34 +00:00
- If you cannot access the repository for any reason, download packages as described in the [install guide ](../getting-started/install.md ) article and install them manually using the `sudo dpkg -i <packages>` command. You will also need the `tzdata` package.
2019-02-04 13:30:28 +00:00
2022-10-25 08:16:59 +00:00
### You Cannot Update Deb Packages from ClickHouse Repository with Apt-get {#you-cannot-update-deb-packages-from-clickhouse-repository-with-apt-get}
- The issue may be happened when the GPG key is changed.
Please use the following scripts to resolve the issue:
```bash
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 8919F6BD2B48D754
sudo apt-get update
```
2022-11-01 12:06:35 +00:00
### You Get Different Warnings with `apt-get update` {#you-get-different-warnings-with-apt-get-update}
2022-10-25 08:16:59 +00:00
2022-11-01 12:06:35 +00:00
- The completed warning messages are as one of following:
2022-10-25 08:16:59 +00:00
```
N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'https://packages.clickhouse.com/deb stable InRelease' doesn't support architecture 'i386'
```
2022-11-01 12:06:35 +00:00
```
E: Failed to fetch https://packages.clickhouse.com/deb/dists/stable/main/binary-amd64/Packages.gz File has unexpected size (30451 != 28154). Mirror sync in progress?
```
```
E: Repository 'https://packages.clickhouse.com/deb stable InRelease' changed its 'Origin' value from 'Artifactory' to 'ClickHouse'
E: Repository 'https://packages.clickhouse.com/deb stable InRelease' changed its 'Label' value from 'Artifactory' to 'ClickHouse'
N: Repository 'https://packages.clickhouse.com/deb stable InRelease' changed its 'Suite' value from 'stable' to ''
N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
```
```
Err:11 https://packages.clickhouse.com/deb stable InRelease
400 Bad Request [IP: 172.66.40.249 443]
```
2022-10-25 08:16:59 +00:00
To resolve the above issue, please use the following script:
```bash
2022-11-01 12:06:35 +00:00
sudo rm /var/lib/apt/lists/packages.clickhouse.com_* /var/lib/dpkg/arch /var/lib/apt/lists/partial/packages.clickhouse.com_*
2022-10-25 08:16:59 +00:00
sudo apt-get clean
sudo apt-get autoclean
```
2020-04-08 14:22:25 +00:00
## Connecting to the Server {#troubleshooting-accepts-no-connections}
2019-02-04 13:30:28 +00:00
2019-02-05 14:35:58 +00:00
Possible issues:
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
- The server is not running.
- Unexpected or wrong configuration parameters.
2019-02-04 13:30:28 +00:00
2020-03-20 10:10:48 +00:00
### Server Is Not Running {#server-is-not-running}
2019-02-04 13:30:28 +00:00
2022-06-02 13:01:59 +00:00
**Check if server is running**
2019-02-04 13:30:28 +00:00
Command:
2020-03-20 10:10:48 +00:00
``` bash
2019-09-23 15:31:46 +00:00
$ sudo service clickhouse-server status
2019-02-04 13:30:28 +00:00
```
If the server is not running, start it with the command:
2020-03-20 10:10:48 +00:00
``` bash
2019-09-23 15:31:46 +00:00
$ sudo service clickhouse-server start
2019-02-04 13:30:28 +00:00
```
**Check logs**
2019-02-04 19:09:53 +00:00
The main log of `clickhouse-server` is in `/var/log/clickhouse-server/clickhouse-server.log` by default.
2019-02-04 13:30:28 +00:00
2019-02-11 14:48:37 +00:00
If the server started successfully, you should see the strings:
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
- `<Information> Application: starting up.` — Server started.
- `<Information> Application: Ready for connections.` — Server is running and ready for connections.
2019-02-04 13:30:28 +00:00
2019-02-11 14:48:37 +00:00
If `clickhouse-server` start failed with a configuration error, you should see the `<Error>` string with an error description. For example:
2019-02-04 13:30:28 +00:00
2020-03-20 10:10:48 +00:00
``` text
2019-02-04 13:30:28 +00:00
2019.01.11 15:23:25.549505 [ 45 ] {} < Error > ExternalDictionaries: Failed reloading 'event2id' external dictionary: Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused, e.what() = Connection refused
```
2021-05-27 19:44:11 +00:00
If you do not see an error at the end of the file, look through the entire file starting from the string:
2019-02-04 13:30:28 +00:00
2020-03-20 10:10:48 +00:00
``` text
2019-02-04 13:30:28 +00:00
< Information > Application: starting up.
```
2019-02-11 14:48:37 +00:00
If you try to start a second instance of `clickhouse-server` on the server, you see the following log:
2019-02-04 13:30:28 +00:00
2020-03-20 10:10:48 +00:00
``` text
2019-02-04 13:30:28 +00:00
2019.01.11 15:25:11.151730 [ 1 ] {} < Information > : Starting ClickHouse 19.1.0 with revision 54413
2019.01.11 15:25:11.154578 [ 1 ] {} < Information > Application: starting up
2019.01.11 15:25:11.156361 [ 1 ] {} < Information > StatusFile: Status file ./status already exists - unclean restart. Contents:
PID: 8510
Started at: 2019-01-11 15:24:23
Revision: 54413
2019.01.11 15:25:11.156673 [ 1 ] {} < Error > Application: DB::Exception: Cannot lock file ./status. Another server instance in same directory is already running.
2019.01.11 15:25:11.156682 [ 1 ] {} < Information > Application: shutting down
2019.01.11 15:25:11.156686 [ 1 ] {} < Debug > Application: Uninitializing subsystem: Logging Subsystem
2019.01.11 15:25:11.156716 [ 2 ] {} < Information > BaseDaemon: Stop SignalListener thread
```
**See system.d logs**
2021-05-27 19:44:11 +00:00
If you do not find any useful information in `clickhouse-server` logs or there aren’ t any logs, you can view `system.d` logs using the command:
2019-02-04 13:30:28 +00:00
2020-03-20 10:10:48 +00:00
``` bash
2019-09-23 15:31:46 +00:00
$ sudo journalctl -u clickhouse-server
2019-02-04 13:30:28 +00:00
```
**Start clickhouse-server in interactive mode**
2020-03-20 10:10:48 +00:00
``` bash
2019-09-23 15:31:46 +00:00
$ sudo -u clickhouse /usr/bin/clickhouse-server --config-file /etc/clickhouse-server/config.xml
2019-02-04 13:30:28 +00:00
```
2019-02-11 14:48:37 +00:00
This command starts the server as an interactive app with standard parameters of the autostart script. In this mode `clickhouse-server` prints all the event messages in the console.
2019-02-04 13:30:28 +00:00
2020-03-20 10:10:48 +00:00
### Configuration Parameters {#configuration-parameters}
2019-02-04 13:30:28 +00:00
Check:
2020-03-21 04:11:51 +00:00
- Docker settings.
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
If you run ClickHouse in Docker in an IPv6 network, make sure that `network=host` is set.
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
- Endpoint settings.
2019-02-04 13:30:28 +00:00
2020-10-13 17:23:29 +00:00
Check [listen_host ](../operations/server-configuration-parameters/settings.md#server_configuration_parameters-listen_host ) and [tcp_port ](../operations/server-configuration-parameters/settings.md#server_configuration_parameters-tcp_port ) settings.
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
ClickHouse server accepts localhost connections only by default.
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
- HTTP protocol settings.
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
Check protocol settings for the HTTP API.
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
- Secure connection settings.
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
Check:
2019-02-04 13:30:28 +00:00
2020-10-13 17:23:29 +00:00
- The [tcp_port_secure ](../operations/server-configuration-parameters/settings.md#server_configuration_parameters-tcp_port_secure ) setting.
2020-06-18 08:24:31 +00:00
- Settings for [SSL certificates ](../operations/server-configuration-parameters/settings.md#server_configuration_parameters-openssl ).
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
Use proper parameters while connecting. For example, use the `port_secure` parameter with `clickhouse_client` .
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
- User settings.
2019-02-04 13:30:28 +00:00
2020-03-21 04:11:51 +00:00
You might be using the wrong user name or password.
2019-02-04 13:30:28 +00:00
2020-03-18 18:43:51 +00:00
## Query Processing {#troubleshooting-does-not-process-queries}
2019-02-04 13:30:28 +00:00
2019-02-11 14:48:37 +00:00
If ClickHouse is not able to process the query, it sends an error description to the client. In the `clickhouse-client` you get a description of the error in the console. If you are using the HTTP interface, ClickHouse sends the error description in the response body. For example:
2019-02-04 13:30:28 +00:00
2020-03-20 10:10:48 +00:00
``` bash
2019-02-04 13:30:28 +00:00
$ curl 'http://localhost:8123/' --data-binary "SELECT a"
2019-02-10 18:14:23 +00:00
Code: 47, e.displayText() = DB::Exception: Unknown identifier: a. Note that there are no tables (FROM clause) in your query, context: required_names: 'a' source_tables: table_aliases: private_aliases: column_aliases: public_columns: 'a' masked_columns: array_join_columns: source_columns: , e.what() = DB::Exception
2019-02-04 13:30:28 +00:00
```
2019-02-11 14:48:37 +00:00
If you start `clickhouse-client` with the `stack-trace` parameter, ClickHouse returns the server stack trace with the description of an error.
2019-02-04 13:30:28 +00:00
2019-02-11 14:48:37 +00:00
You might see a message about a broken connection. In this case, you can repeat the query. If the connection breaks every time you perform the query, check the server logs for errors.
2019-02-04 13:30:28 +00:00
2020-04-08 14:22:25 +00:00
## Efficiency of Query Processing {#troubleshooting-too-slow}
2019-02-04 13:30:28 +00:00
2019-02-11 14:48:37 +00:00
If you see that ClickHouse is working too slowly, you need to profile the load on the server resources and network for your queries.
2019-02-04 13:30:28 +00:00
2019-02-11 14:48:37 +00:00
You can use the clickhouse-benchmark utility to profile queries. It shows the number of queries processed per second, the number of rows processed per second, and percentiles of query processing times.