start renaming

This commit is contained in:
DanRoscigno 2022-11-02 15:47:11 -04:00
parent 0c21434ec7
commit c8f9af1afa
9 changed files with 84 additions and 36 deletions

View File

@ -3,7 +3,7 @@ slug: /en/operations/system-tables/dictionaries
--- ---
# dictionaries # dictionaries
Contains information about [external dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md). Contains information about [dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).
Columns: Columns:
@ -33,7 +33,7 @@ Columns:
- `lifetime_min` ([UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges)) — Minimum [lifetime](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds. - `lifetime_min` ([UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges)) — Minimum [lifetime](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds.
- `lifetime_max` ([UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges)) — Maximum [lifetime](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds. - `lifetime_max` ([UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges)) — Maximum [lifetime](../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds.
- `loading_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Start time for loading the dictionary. - `loading_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Start time for loading the dictionary.
- `last_successful_update_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — End time for loading or updating the dictionary. Helps to monitor some troubles with external sources and investigate causes. - `last_successful_update_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — End time for loading or updating the dictionary. Helps to monitor some troubles with dictionary sources and investigate the causes.
- `loading_duration` ([Float32](../../sql-reference/data-types/float.md)) — Duration of a dictionary loading. - `loading_duration` ([Float32](../../sql-reference/data-types/float.md)) — Duration of a dictionary loading.
- `last_exception` ([String](../../sql-reference/data-types/string.md)) — Text of the error that occurs when creating or reloading the dictionary if the dictionary couldnt be created. - `last_exception` ([String](../../sql-reference/data-types/string.md)) — Text of the error that occurs when creating or reloading the dictionary if the dictionary couldnt be created.
- `comment` ([String](../../sql-reference/data-types/string.md)) — Text of the comment to dictionary. - `comment` ([String](../../sql-reference/data-types/string.md)) — Text of the comment to dictionary.

View File

@ -1,8 +1,8 @@
position: 37 position: 37
label: 'External Dictionaries' label: 'Dictionaries'
collapsible: true collapsible: true
collapsed: true collapsed: true
link: link:
type: generated-index type: generated-index
title: External Dictionaries title: Dictionaries
slug: /en/sql-reference/dictionaries/external-dictionaries slug: /en/sql-reference/dictionaries/external-dictionaries

View File

@ -1,12 +1,12 @@
--- ---
slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources
sidebar_position: 43 sidebar_position: 43
sidebar_label: Sources of External Dictionaries sidebar_label: Dictionary Sources
--- ---
# Sources of External Dictionaries # Dictionary Sources
An external dictionary can be connected to ClickHouse from many different sources. A dictionary can be connected to ClickHouse from many different sources.
If the dictionary is configured using an xml-file, the configuration looks like this: If the dictionary is configured using an xml-file, the configuration looks like this:

View File

@ -171,5 +171,5 @@ Configuration fields:
**See Also** **See Also**
- [Functions for working with external dictionaries](../../../sql-reference/functions/ext-dict-functions.md). - [Functions for working with dictionaries](../../../sql-reference/functions/ext-dict-functions.md).

View File

@ -1,10 +1,10 @@
--- ---
slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict slug: /en/sql-reference/dictionaries/external-dictionaries/external-dicts-dict
sidebar_position: 40 sidebar_position: 40
sidebar_label: Configuring an External Dictionary sidebar_label: Configuring a Dictionary
--- ---
# Configuring an External Dictionary # Configuring a Dictionary
If dictionary is configured using xml file, than dictionary configuration has the following structure: If dictionary is configured using xml file, than dictionary configuration has the following structure:

View File

@ -4,17 +4,17 @@ sidebar_position: 39
sidebar_label: General Description sidebar_label: General Description
--- ---
# External Dictionaries # Dictionaries
You can add your own dictionaries from various data sources. The data source for a dictionary can be a local text or executable file, an HTTP(s) resource, or another DBMS. For more information, see “[Sources for external dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)”. You can add your own dictionaries from various data sources. The source for a dictionary can be a ClickHouse table, a DDL query, a local text or executable file, an HTTP(s) resource, or another DBMS. For more information, see “[Dictionary Sources](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)”.
ClickHouse: ClickHouse:
- Fully or partially stores dictionaries in RAM. - Fully or partially stores dictionaries in RAM.
- Periodically updates dictionaries and dynamically loads missing values. In other words, dictionaries can be loaded dynamically. - Periodically updates dictionaries and dynamically loads missing values. In other words, dictionaries can be loaded dynamically.
- Allows to create external dictionaries with xml files or [DDL queries](../../../sql-reference/statements/create/dictionary.md). - Allows creating dictionaries with xml files or [DDL queries](../../../sql-reference/statements/create/dictionary.md).
The configuration of external dictionaries can be located in one or more xml-files. The path to the configuration is specified in the [dictionaries_config](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_config) parameter. The configuration of dictionaries can be located in one or more xml-files. The path to the configuration is specified in the [dictionaries_config](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_config) parameter.
Dictionaries can be loaded at server startup or at first use, depending on the [dictionaries_lazy_load](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_lazy_load) setting. Dictionaries can be loaded at server startup or at first use, depending on the [dictionaries_lazy_load](../../../operations/server-configuration-parameters/settings.md#server_configuration_parameters-dictionaries_lazy_load) setting.
@ -24,6 +24,16 @@ The [dictionaries](../../../operations/system-tables/dictionaries.md#system_tabl
- Configuration parameters. - Configuration parameters.
- Metrics like amount of RAM allocated for the dictionary or a number of queries since the dictionary was successfully loaded. - Metrics like amount of RAM allocated for the dictionary or a number of queries since the dictionary was successfully loaded.
## Creating a dictionary with a DDL query
Dictionaries can be created with [DDL queries](../../../sql-reference/statements/create/dictionary.md). This does not require any additional records in a server configuration file. This allows dictionaries to be worked with as first-class entities, like tables or views.
## Creating a dictionary with a configuration file
:::note
Creating a dictionary with a configuration file is not applicable to ClickHouse Cloud. Please use DDL (see above).
:::
The dictionary configuration file has the following format: The dictionary configuration file has the following format:
``` xml ``` xml
@ -44,18 +54,17 @@ The dictionary configuration file has the following format:
You can [configure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md) any number of dictionaries in the same file. You can [configure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md) any number of dictionaries in the same file.
[DDL queries for dictionaries](../../../sql-reference/statements/create/dictionary.md) does not require any additional records in server configuration. They allow to work with dictionaries as first-class entities, like tables or views.
:::note :::note
You can convert values for a small dictionary by describing it in a `SELECT` query (see the [transform](../../../sql-reference/functions/other-functions.md) function). This functionality is not related to external dictionaries. You can convert values for a small dictionary by describing it in a `SELECT` query (see the [transform](../../../sql-reference/functions/other-functions.md) function). This functionality is not related to dictionaries.
::: :::
## See Also ## See Also
- [Configuring an External Dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md) - [Configuring a Dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict.md)
- [Storing Dictionaries in Memory](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) - [Storing Dictionaries in Memory](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md)
- [Dictionary Updates](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md) - [Dictionary Updates](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md)
- [Sources of External Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md) - [Dictionary Sources](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md)
- [Dictionary Key and Fields](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md) - [Dictionary Key and Fields](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md)
- [Functions for Working with External Dictionaries](../../../sql-reference/functions/ext-dict-functions.md) - [Functions for Working with Dictionaries](../../../sql-reference/functions/ext-dict-functions.md)

View File

@ -12,6 +12,6 @@ ClickHouse supports special functions for working with dictionaries that can be
ClickHouse supports: ClickHouse supports:
- [Built-in dictionaries](../../sql-reference/dictionaries/internal-dicts.md#internal_dicts) with a specific [set of functions](../../sql-reference/functions/ym-dict-functions.md). - [Dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md#dicts-external-dicts) with a [set of functions](../../sql-reference/functions/ext-dict-functions.md).
- [Plug-in (external) dictionaries](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md#dicts-external-dicts) with a [set of functions](../../sql-reference/functions/ext-dict-functions.md). - [Embedded dictionaries](../../sql-reference/dictionaries/internal-dicts.md#internal_dicts) with a specific [set of functions](../../sql-reference/functions/ym-dict-functions.md).

View File

@ -1,10 +1,10 @@
--- ---
slug: /en/sql-reference/dictionaries/internal-dicts slug: /en/sql-reference/dictionaries/internal-dicts
sidebar_position: 39 sidebar_position: 39
sidebar_label: Internal Dictionaries sidebar_label: Embedded Dictionaries
--- ---
# Internal Dictionaries # Embedded Dictionaries
ClickHouse contains a built-in feature for working with a geobase. ClickHouse contains a built-in feature for working with a geobase.

View File

@ -5,9 +5,9 @@ sidebar_label: DICTIONARY
title: "CREATE DICTIONARY" title: "CREATE DICTIONARY"
--- ---
Creates a new [external dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) with given [structure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md), [source](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md), [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) and [lifetime](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md). Creates a new [dictionary](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) with given [structure](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-structure.md), [source](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-sources.md), [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) and [lifetime](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-lifetime.md).
**Syntax** ## Syntax
``` sql ``` sql
CREATE [OR REPLACE] DICTIONARY [IF NOT EXISTS] [db.]dictionary_name [ON CLUSTER cluster] CREATE [OR REPLACE] DICTIONARY [IF NOT EXISTS] [db.]dictionary_name [ON CLUSTER cluster]
@ -25,17 +25,25 @@ SETTINGS(setting_name = setting_value, setting_name = setting_value, ...)
COMMENT 'Comment' COMMENT 'Comment'
``` ```
External dictionary structure consists of attributes. Dictionary attributes are specified similarly to table columns. The only required attribute property is its type, all other properties may have default values. The dictionary structure consists of attributes. Dictionary attributes are specified similarly to table columns. The only required attribute property is its type, all other properties may have default values.
`ON CLUSTER` clause allows creating dictionary on a cluster, see [Distributed DDL](../../../sql-reference/distributed-ddl.md). `ON CLUSTER` clause allows creating dictionary on a cluster, see [Distributed DDL](../../../sql-reference/distributed-ddl.md).
Depending on dictionary [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) one or more attributes can be specified as dictionary keys. Depending on dictionary [layout](../../../sql-reference/dictionaries/external-dictionaries/external-dicts-dict-layout.md) one or more attributes can be specified as dictionary keys.
For more information, see [External Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) section. ### SOURCE
The source for a dictionary can be a:
- table in the current ClickHouse service
- table in a remote ClickHouse service
- file available by HTTP(S)
- another database
You can add a comment to the dictionary when you creating it using `COMMENT` clause. You can add a comment to the dictionary when you creating it using `COMMENT` clause.
**Example** #### Create a dictionary from a table in the current ClickHouse service
Input table `source_table`: Input table `source_table`:
@ -49,27 +57,26 @@ Input table `source_table`:
Creating the dictionary: Creating the dictionary:
``` sql ``` sql
CREATE DICTIONARY dictionary_with_comment CREATE DICTIONARY id_value_dictionary
( (
id UInt64, id UInt64,
value String value String
) )
PRIMARY KEY id PRIMARY KEY id
SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table')) SOURCE(CLICKHOUSE(TABLE 'source_table'))
LAYOUT(FLAT()) LAYOUT(FLAT())
LIFETIME(MIN 0 MAX 1000) LIFETIME(MIN 0 MAX 1000)
COMMENT 'The temporary dictionary';
``` ```
Output the dictionary: Output the dictionary:
``` sql ``` sql
SHOW CREATE DICTIONARY dictionary_with_comment; SHOW CREATE DICTIONARY id_value_dictionary;
``` ```
```text ```text
┌─statement───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ ┌─statement───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ CREATE DICTIONARY default.dictionary_with_comment │ CREATE DICTIONARY default.id_value_dictionary
( (
`id` UInt64, `id` UInt64,
`value` String `value` String
@ -77,10 +84,41 @@ SHOW CREATE DICTIONARY dictionary_with_comment;
PRIMARY KEY id PRIMARY KEY id
SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table')) SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table'))
LIFETIME(MIN 0 MAX 1000) LIFETIME(MIN 0 MAX 1000)
LAYOUT(FLAT()) LAYOUT(FLAT()) |
COMMENT 'The temporary dictionary' │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
``` ```
#### Create a dictionary from a table in a remote ClickHouse service
Creating the dictionary:
``` sql
CREATE DICTIONARY id_value_dictionary
(
id UInt64,
value String
)
PRIMARY KEY id
SOURCE(CLICKHOUSE(HOST 'HOSTNAME' PORT '8443' TABLE 'source_table'))
LAYOUT(FLAT())
LIFETIME(MIN 0 MAX 1000)
```
#### Create a dictionary from a file available by HTTP(S)
```sql
statement: CREATE DICTIONARY default.taxi_zone_dictionary
(
`LocationID` UInt16 DEFAULT 0,
`Borough` String,
`Zone` String,
`service_zone` String
)
PRIMARY KEY LocationID
SOURCE(HTTP(URL 'https://datasets-documentation.s3.eu-west-3.amazonaws.com/nyc-taxi/taxi_zone_lookup.csv' FORMAT 'CSVWithNames'))
LIFETIME(MIN 0 MAX 0)
LAYOUT(HASHED())
```
#### Create a dictionary from another database
Output the comment to dictionary: Output the comment to dictionary:
@ -96,4 +134,5 @@ SELECT comment FROM system.dictionaries WHERE name == 'dictionary_with_comment'
**See Also** **See Also**
- [system.dictionaries](../../../operations/system-tables/dictionaries.md) — This table contains information about [external dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md). - For more information, see the [Dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) section.
- [system.dictionaries](../../../operations/system-tables/dictionaries.md) — This table contains information about [dictionaries](../../../sql-reference/dictionaries/external-dictionaries/external-dicts.md).