mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-27 01:51:59 +00:00
start renaming
This commit is contained in:
parent
0c21434ec7
commit
c8f9af1afa
@ -3,7 +3,7 @@ slug: /en/operations/system-tables/dictionaries
|
|||||||
---
|
---
|
||||||
# dictionaries
|
# dictionaries
|
||||||
|
|
||||||
Contains information about [external dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
|
Contains information about [dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
|
||||||
|
|
||||||
Columns:
|
Columns:
|
||||||
|
|
||||||
@ -33,7 +33,7 @@ Columns:
|
|||||||
- `lifetime_min` ([UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges)) — Minimum [lifetime](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds.
|
- `lifetime_min` ([UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges)) — Minimum [lifetime](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds.
|
||||||
- `lifetime_max` ([UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges)) — Maximum [lifetime](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds.
|
- `lifetime_max` ([UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges)) — Maximum [lifetime](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds.
|
||||||
- `loading_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Start time for loading the dictionary.
|
- `loading_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Start time for loading the dictionary.
|
||||||
- `last_successful_update_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — End time for loading or updating the dictionary. Helps to monitor some troubles with external sources and investigate causes.
|
- `last_successful_update_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — End time for loading or updating the dictionary. Helps to monitor some troubles with dictionary sources and investigate the causes.
|
||||||
- `loading_duration` ([Float32](../../sql-reference/data-types/float.md)) — Duration of a dictionary loading.
|
- `loading_duration` ([Float32](../../sql-reference/data-types/float.md)) — Duration of a dictionary loading.
|
||||||
- `last_exception` ([String](../../sql-reference/data-types/string.md)) — Text of the error that occurs when creating or reloading the dictionary if the dictionary couldn’t be created.
|
- `last_exception` ([String](../../sql-reference/data-types/string.md)) — Text of the error that occurs when creating or reloading the dictionary if the dictionary couldn’t be created.
|
||||||
- `comment` ([String](../../sql-reference/data-types/string.md)) — Text of the comment to dictionary.
|
- `comment` ([String](../../sql-reference/data-types/string.md)) — Text of the comment to dictionary.
|
||||||
|
@ -1,8 +1,8 @@
|
|||||||
position: 37
|
position: 37
|
||||||
label: 'External Dictionaries'
|
label: 'Dictionaries'
|
||||||
collapsible: true
|
collapsible: true
|
||||||
collapsed: true
|
collapsed: true
|
||||||
link:
|
link:
|
||||||
type: generated-index
|
type: generated-index
|
||||||
title: External Dictionaries
|
title: Dictionaries
|
||||||
slug: /en/sql-reference/dictionaries/external-dictionaries
|
slug: /en/sql-reference/dictionaries/external-dictionaries
|
||||||
|
@ -1,12 +1,12 @@
|
|||||||
---
|
---
|
||||||
slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources
|
slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources
|
||||||
sidebar_position: 43
|
sidebar_position: 43
|
||||||
sidebar_label: Sources of External Dictionaries
|
sidebar_label: Dictionary Sources
|
||||||
---
|
---
|
||||||
|
|
||||||
# Sources of External Dictionaries
|
# Dictionary Sources
|
||||||
|
|
||||||
An external dictionary can be connected to ClickHouse from many different sources.
|
A dictionary can be connected to ClickHouse from many different sources.
|
||||||
|
|
||||||
If the dictionary is configured using an xml-file, the configuration looks like this:
|
If the dictionary is configured using an xml-file, the configuration looks like this:
|
||||||
|
|
||||||
|
@ -171,5 +171,5 @@ Configuration fields:
|
|||||||
|
|
||||||
**See Also**
|
**See Also**
|
||||||
|
|
||||||
- [Functions for working with external dictionaries](../../../sql-reference/functions/ext-dict-functions.md).
|
- [Functions for working with dictionaries](../../../sql-reference/functions/ext-dict-functions.md).
|
||||||
|
|
||||||
|
@ -1,10 +1,10 @@
|
|||||||
---
|
---
|
||||||
slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict
|
slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict
|
||||||
sidebar_position: 40
|
sidebar_position: 40
|
||||||
sidebar_label: Configuring an External Dictionary
|
sidebar_label: Configuring a Dictionary
|
||||||
---
|
---
|
||||||
|
|
||||||
# Configuring an External Dictionary
|
# Configuring a Dictionary
|
||||||
|
|
||||||
If dictionary is configured using xml file, than dictionary configuration has the following structure:
|
If dictionary is configured using xml file, than dictionary configuration has the following structure:
|
||||||
|
|
||||||
|
@ -4,17 +4,17 @@ sidebar_position: 39
|
|||||||
sidebar_label: General Description
|
sidebar_label: General Description
|
||||||
---
|
---
|
||||||
|
|
||||||
# External Dictionaries
|
# Dictionaries
|
||||||
|
|
||||||
You can add your own dictionaries from various data sources. The data source for a dictionary can be a local text or executable file, an HTTP(s) resource, or another DBMS. For more information, see “[Sources for external dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)”.
|
You can add your own dictionaries from various data sources. The source for a dictionary can be a ClickHouse table, a DDL query, a local text or executable file, an HTTP(s) resource, or another DBMS. For more information, see “[Dictionary Sources](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)”.
|
||||||
|
|
||||||
ClickHouse:
|
ClickHouse:
|
||||||
|
|
||||||
- Fully or partially stores dictionaries in RAM.
|
- Fully or partially stores dictionaries in RAM.
|
||||||
- Periodically updates dictionaries and dynamically loads missing values. In other words, dictionaries can be loaded dynamically.
|
- Periodically updates dictionaries and dynamically loads missing values. In other words, dictionaries can be loaded dynamically.
|
||||||
- Allows to create external dictionaries with xml files or [DDL queries](../../../sql-reference/statements/create/dictionary.md).
|
- Allows creating dictionaries with xml files or [DDL queries](../../../sql-reference/statements/create/dictionary.md).
|
||||||
|
|
||||||
The configuration of external dictionaries can be located in one or more xml-files. The path to the configuration is specified in the [dictionaries_config](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_config) parameter.
|
The configuration of dictionaries can be located in one or more xml-files. The path to the configuration is specified in the [dictionaries_config](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_config) parameter.
|
||||||
|
|
||||||
Dictionaries can be loaded at server startup or at first use, depending on the [dictionaries_lazy_load](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_lazy_load) setting.
|
Dictionaries can be loaded at server startup or at first use, depending on the [dictionaries_lazy_load](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_lazy_load) setting.
|
||||||
|
|
||||||
@ -24,6 +24,16 @@ The [dictionaries](../../../operations/system-tables/dictionaries.md#system_tabl
|
|||||||
- Configuration parameters.
|
- Configuration parameters.
|
||||||
- Metrics like amount of RAM allocated for the dictionary or a number of queries since the dictionary was successfully loaded.
|
- Metrics like amount of RAM allocated for the dictionary or a number of queries since the dictionary was successfully loaded.
|
||||||
|
|
||||||
|
## Creating a dictionary with a DDL query
|
||||||
|
|
||||||
|
Dictionaries can be created with [DDL queries](../../../sql-reference/statements/create/dictionary.md). This does not require any additional records in a server configuration file. This allows dictionaries to be worked with as first-class entities, like tables or views.
|
||||||
|
|
||||||
|
## Creating a dictionary with a configuration file
|
||||||
|
|
||||||
|
:::note
|
||||||
|
Creating a dictionary with a configuration file is not applicable to ClickHouse Cloud. Please use DDL (see above).
|
||||||
|
:::
|
||||||
|
|
||||||
The dictionary configuration file has the following format:
|
The dictionary configuration file has the following format:
|
||||||
|
|
||||||
``` xml
|
``` xml
|
||||||
@ -44,18 +54,17 @@ The dictionary configuration file has the following format:
|
|||||||
|
|
||||||
You can [configure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md) any number of dictionaries in the same file.
|
You can [configure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md) any number of dictionaries in the same file.
|
||||||
|
|
||||||
[DDL queries for dictionaries](../../../sql-reference/statements/create/dictionary.md) does not require any additional records in server configuration. They allow to work with dictionaries as first-class entities, like tables or views.
|
|
||||||
|
|
||||||
:::note
|
:::note
|
||||||
You can convert values for a small dictionary by describing it in a `SELECT` query (see the [transform](../../../sql-reference/functions/other-functions.md) function). This functionality is not related to external dictionaries.
|
You can convert values for a small dictionary by describing it in a `SELECT` query (see the [transform](../../../sql-reference/functions/other-functions.md) function). This functionality is not related to dictionaries.
|
||||||
:::
|
:::
|
||||||
|
|
||||||
## See Also
|
## See Also
|
||||||
|
|
||||||
- [Configuring an External Dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md)
|
- [Configuring a Dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md)
|
||||||
- [Storing Dictionaries in Memory](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md)
|
- [Storing Dictionaries in Memory](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md)
|
||||||
- [Dictionary Updates](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md)
|
- [Dictionary Updates](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md)
|
||||||
- [Sources of External Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)
|
- [Dictionary Sources](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)
|
||||||
- [Dictionary Key and Fields](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md)
|
- [Dictionary Key and Fields](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md)
|
||||||
- [Functions for Working with External Dictionaries](../../../sql-reference/functions/ext-dict-functions.md)
|
- [Functions for Working with Dictionaries](../../../sql-reference/functions/ext-dict-functions.md)
|
||||||
|
|
||||||
|
@ -12,6 +12,6 @@ ClickHouse supports special functions for working with dictionaries that can be
|
|||||||
|
|
||||||
ClickHouse supports:
|
ClickHouse supports:
|
||||||
|
|
||||||
- [Built-in dictionaries](../../sql-reference/dictionaries/internal-dicts.md#internal_dicts) with a specific [set of functions](../../sql-reference/functions/ym-dict-functions.md).
|
- [Dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md#dicts-external-dicts) with a [set of functions](../../sql-reference/functions/ext-dict-functions.md).
|
||||||
- [Plug-in (external) dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md#dicts-external-dicts) with a [set of functions](../../sql-reference/functions/ext-dict-functions.md).
|
- [Embedded dictionaries](../../sql-reference/dictionaries/internal-dicts.md#internal_dicts) with a specific [set of functions](../../sql-reference/functions/ym-dict-functions.md).
|
||||||
|
|
||||||
|
@ -1,10 +1,10 @@
|
|||||||
---
|
---
|
||||||
slug: /en/sql-reference/dictionaries/internal-dicts
|
slug: /en/sql-reference/dictionaries/internal-dicts
|
||||||
sidebar_position: 39
|
sidebar_position: 39
|
||||||
sidebar_label: Internal Dictionaries
|
sidebar_label: Embedded Dictionaries
|
||||||
---
|
---
|
||||||
|
|
||||||
# Internal Dictionaries
|
# Embedded Dictionaries
|
||||||
|
|
||||||
ClickHouse contains a built-in feature for working with a geobase.
|
ClickHouse contains a built-in feature for working with a geobase.
|
||||||
|
|
||||||
|
@ -5,9 +5,9 @@ sidebar_label: DICTIONARY
|
|||||||
title: "CREATE DICTIONARY"
|
title: "CREATE DICTIONARY"
|
||||||
---
|
---
|
||||||
|
|
||||||
Creates a new [external dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) with given [structure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md), [source](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md), [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) and [lifetime](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md).
|
Creates a new [dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) with given [structure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md), [source](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md), [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) and [lifetime](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md).
|
||||||
|
|
||||||
**Syntax**
|
## Syntax
|
||||||
|
|
||||||
``` sql
|
``` sql
|
||||||
CREATE [OR REPLACE] DICTIONARY [IF NOT EXISTS] [db.]dictionary_name [ON CLUSTER cluster]
|
CREATE [OR REPLACE] DICTIONARY [IF NOT EXISTS] [db.]dictionary_name [ON CLUSTER cluster]
|
||||||
@ -25,17 +25,25 @@ SETTINGS(setting_name = setting_value, setting_name = setting_value, ...)
|
|||||||
COMMENT 'Comment'
|
COMMENT 'Comment'
|
||||||
```
|
```
|
||||||
|
|
||||||
External dictionary structure consists of attributes. Dictionary attributes are specified similarly to table columns. The only required attribute property is its type, all other properties may have default values.
|
The dictionary structure consists of attributes. Dictionary attributes are specified similarly to table columns. The only required attribute property is its type, all other properties may have default values.
|
||||||
|
|
||||||
`ON CLUSTER` clause allows creating dictionary on a cluster, see [Distributed DDL](../../../sql-reference/distributed-ddl.md).
|
`ON CLUSTER` clause allows creating dictionary on a cluster, see [Distributed DDL](../../../sql-reference/distributed-ddl.md).
|
||||||
|
|
||||||
Depending on dictionary [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) one or more attributes can be specified as dictionary keys.
|
Depending on dictionary [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) one or more attributes can be specified as dictionary keys.
|
||||||
|
|
||||||
For more information, see [External Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) section.
|
### SOURCE
|
||||||
|
|
||||||
|
The source for a dictionary can be a:
|
||||||
|
- table in the current ClickHouse service
|
||||||
|
- table in a remote ClickHouse service
|
||||||
|
- file available by HTTP(S)
|
||||||
|
- another database
|
||||||
|
|
||||||
|
|
||||||
You can add a comment to the dictionary when you creating it using `COMMENT` clause.
|
You can add a comment to the dictionary when you creating it using `COMMENT` clause.
|
||||||
|
|
||||||
**Example**
|
#### Create a dictionary from a table in the current ClickHouse service
|
||||||
|
|
||||||
|
|
||||||
Input table `source_table`:
|
Input table `source_table`:
|
||||||
|
|
||||||
@ -49,27 +57,26 @@ Input table `source_table`:
|
|||||||
Creating the dictionary:
|
Creating the dictionary:
|
||||||
|
|
||||||
``` sql
|
``` sql
|
||||||
CREATE DICTIONARY dictionary_with_comment
|
CREATE DICTIONARY id_value_dictionary
|
||||||
(
|
(
|
||||||
id UInt64,
|
id UInt64,
|
||||||
value String
|
value String
|
||||||
)
|
)
|
||||||
PRIMARY KEY id
|
PRIMARY KEY id
|
||||||
SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table'))
|
SOURCE(CLICKHOUSE(TABLE 'source_table'))
|
||||||
LAYOUT(FLAT())
|
LAYOUT(FLAT())
|
||||||
LIFETIME(MIN 0 MAX 1000)
|
LIFETIME(MIN 0 MAX 1000)
|
||||||
COMMENT 'The temporary dictionary';
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Output the dictionary:
|
Output the dictionary:
|
||||||
|
|
||||||
``` sql
|
``` sql
|
||||||
SHOW CREATE DICTIONARY dictionary_with_comment;
|
SHOW CREATE DICTIONARY id_value_dictionary;
|
||||||
```
|
```
|
||||||
|
|
||||||
```text
|
```text
|
||||||
┌─statement───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
|
┌─statement───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
|
||||||
│ CREATE DICTIONARY default.dictionary_with_comment
|
│ CREATE DICTIONARY default.id_value_dictionary
|
||||||
(
|
(
|
||||||
`id` UInt64,
|
`id` UInt64,
|
||||||
`value` String
|
`value` String
|
||||||
@ -77,10 +84,41 @@ SHOW CREATE DICTIONARY dictionary_with_comment;
|
|||||||
PRIMARY KEY id
|
PRIMARY KEY id
|
||||||
SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table'))
|
SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table'))
|
||||||
LIFETIME(MIN 0 MAX 1000)
|
LIFETIME(MIN 0 MAX 1000)
|
||||||
LAYOUT(FLAT())
|
LAYOUT(FLAT()) |
|
||||||
COMMENT 'The temporary dictionary' │
|
|
||||||
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
#### Create a dictionary from a table in a remote ClickHouse service
|
||||||
|
Creating the dictionary:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
CREATE DICTIONARY id_value_dictionary
|
||||||
|
(
|
||||||
|
id UInt64,
|
||||||
|
value String
|
||||||
|
)
|
||||||
|
PRIMARY KEY id
|
||||||
|
SOURCE(CLICKHOUSE(HOST 'HOSTNAME' PORT '8443' TABLE 'source_table'))
|
||||||
|
LAYOUT(FLAT())
|
||||||
|
LIFETIME(MIN 0 MAX 1000)
|
||||||
|
```
|
||||||
|
#### Create a dictionary from a file available by HTTP(S)
|
||||||
|
```sql
|
||||||
|
statement: CREATE DICTIONARY default.taxi_zone_dictionary
|
||||||
|
(
|
||||||
|
`LocationID` UInt16 DEFAULT 0,
|
||||||
|
`Borough` String,
|
||||||
|
`Zone` String,
|
||||||
|
`service_zone` String
|
||||||
|
)
|
||||||
|
PRIMARY KEY LocationID
|
||||||
|
SOURCE(HTTP(URL 'https://datasets-documentation.s3.eu-west-3.amazonaws.com/nyc-taxi/taxi_zone_lookup.csv' FORMAT 'CSVWithNames'))
|
||||||
|
LIFETIME(MIN 0 MAX 0)
|
||||||
|
LAYOUT(HASHED())
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Create a dictionary from another database
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Output the comment to dictionary:
|
Output the comment to dictionary:
|
||||||
|
|
||||||
@ -96,4 +134,5 @@ SELECT comment FROM system.dictionaries WHERE name == 'dictionary_with_comment'
|
|||||||
|
|
||||||
**See Also**
|
**See Also**
|
||||||
|
|
||||||
- [system.dictionaries](../../../operations/system-tables/dictionaries.md) — This table contains information about [external dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
|
- For more information, see the [Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) section.
|
||||||
|
- [system.dictionaries](../../../operations/system-tables/dictionaries.md) — This table contains information about [dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
|
||||||
|
Loading…
Reference in New Issue
Block a user