mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-24 16:42:05 +00:00
elenbaskakova-DOCSUP-780 (#87)
* "docs(system.dictionaries): Table 'system.dictionaries' has been edited" * "docs(system.dictionaries): Table 'system.dictionaries' has been edited" * "docs(system.dictionaries): Table 'system.dictionaries' has been edited" * "docs(system.dictionaries): Table 'system.dictionaries' has been edited" * "docs(system.dictionaries): Table 'system.dictionaries' has been edited" * "docs(system.dictionaries): Table 'system.dictionaries' has been edited" * Update docs/en/operations/system_tables.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * "docs(system.dictionaries): Table 'system.dictionaries' has been edited" * "docs(system.dictionaries): Table 'system.dictionaries' has been edited" * "docs(system.dictionaries): Table 'system.dictionaries' has been edited" * Update docs/en/operations/system_tables.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/system_tables.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/system_tables.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/system_tables.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * "docs(system.settings): Table 'system.settings' has been edited" * "docs(system.dictionaries): Table 'system.dictionaries' has been edited" * "docs(system.dictionaries): Table 'system.dictionaries has been edited" Co-authored-by: elenbaskakova <elenbaskakova@yandex-team.ru> Co-authored-by: BayoNet <da-daos@yandex.ru>
This commit is contained in:
elenaspb2019
GitHub
parent
dd18ae0991
commit
7376428a6b
@ -147,27 +147,68 @@ This system table is used for implementing the `SHOW DATABASES` query.
|
||||
|
||||
Contains information about detached parts of [MergeTree](../engines/table_engines/mergetree_family/mergetree.md) tables. The `reason` column specifies why the part was detached. For user-detached parts, the reason is empty. Such parts can be attached with [ALTER TABLE ATTACH PARTITION\|PART](../query_language/query_language/alter/#alter_attach-partition) command. For the description of other columns, see [system.parts](#system_tables-parts). If part name is invalid, values of some columns may be `NULL`. Such parts can be deleted with [ALTER TABLE DROP DETACHED PART](../query_language/query_language/alter/#alter_drop-detached).
|
||||
|
||||
## system.dictionaries {#system-dictionaries}
|
||||
## system.dictionaries {#system_tables-dictionaries}
|
||||
|
||||
Contains information about external dictionaries.
|
||||
Contains information about [external dictionaries](../query_language/dicts/external_dicts.md).
|
||||
|
||||
Columns:
|
||||
|
||||
- `name` (String) — Dictionary name.
|
||||
- `type` (String) — Dictionary type: Flat, Hashed, Cache.
|
||||
- `origin` (String) — Path to the configuration file that describes the dictionary.
|
||||
- `attribute.names` (Array(String)) — Array of attribute names provided by the dictionary.
|
||||
- `attribute.types` (Array(String)) — Corresponding array of attribute types that are provided by the dictionary.
|
||||
- `has_hierarchy` (UInt8) — Whether the dictionary is hierarchical.
|
||||
- `bytes_allocated` (UInt64) — The amount of RAM the dictionary uses.
|
||||
- `hit_rate` (Float64) — For cache dictionaries, the percentage of uses for which the value was in the cache.
|
||||
- `element_count` (UInt64) — The number of items stored in the dictionary.
|
||||
- `load_factor` (Float64) — The percentage filled in the dictionary (for a hashed dictionary, the percentage filled in the hash table).
|
||||
- `creation_time` (DateTime) — The time when the dictionary was created or last successfully reloaded.
|
||||
- `last_exception` (String) — Text of the error that occurs when creating or reloading the dictionary if the dictionary couldn’t be created.
|
||||
- `source` (String) — Text describing the data source for the dictionary.
|
||||
- `database` ([String](../data_types/string.md)) — Database name where the dictionary is located. Only for dictionaries created by DDL query, for others is always an empty string.
|
||||
- `name` ([String](../data_types/string.md)) — [Dictionary name](../query_language/dicts/external_dicts_dict.md).
|
||||
- `status` ([Enum8](../data_types/enum.md)) — Dictionary status. Possible values:
|
||||
- `NOT_LOADED` — Dictionary was not loaded because it was not used.
|
||||
- `LOADED` — Dictionary loaded successfully.
|
||||
- `FAILED` — Unable to load the dictionary as a result of an error.
|
||||
- `LOADING` — Dictionary is loading now.
|
||||
- `LOADED_AND_RELOADING` — Dictionary is loaded successfully, and is being reloaded right now (frequent reasons: [SYSTEM RELOAD DICTIONARY](../query_language/system.md#query_language-system-reload-dictionary) query, timeout, dictionary config has changed).
|
||||
- `FAILED_AND_RELOADING` — Could not load the dictionary as a result of an error and is loading now.
|
||||
- `origin` ([String](../data_types/string.md)) — Path to the configuration file that describes the dictionary.
|
||||
- `type` ([String](../data_types/string.md)) — Type of a dictionary allocation. [Storing Dictionaries in Memory](../query_language/dicts/external_dicts_dict_layout.md).
|
||||
- `key` — [Key type](../query_language/dicts/external_dicts_dict_structure.md#ext_dict_structure-key): Numeric Key ([UInt64](../data_types/int_uint.md#uint-ranges)) or Сomposite key ([String](../data_types/string.md)) — form "(type 1, type 2, ..., type n)".
|
||||
- `attribute.names` ([Array](../data_types/array.md)([String](../data_types/string.md))) — Array of [attribute names](../query_language/dicts/external_dicts_dict_structure.md#ext_dict_structure-attributes) provided by the dictionary.
|
||||
- `attribute.types` ([Array](../data_types/array.md)([String](../data_types/string.md))) — Corresponding array of [attribute types](../query_language/dicts/external_dicts_dict_structure.md#ext_dict_structure-attributes) that are provided by the dictionary.
|
||||
- `bytes_allocated` ([UInt64](../data_types/int_uint.md#uint-ranges)) — Amount of RAM allocated for the dictionary.
|
||||
- `query_count` ([UInt64](../data_types/int_uint.md#uint-ranges)) — Number of queries since the dictionary was loaded or since the last successful reboot.
|
||||
- `hit_rate` ([Float64](../data_types/float.md)) — For cache dictionaries, the percentage of uses for which the value was in the cache.
|
||||
- `element_count` ([UInt64](../data_types/int_uint.md#uint-ranges)) — Number of items stored in the dictionary.
|
||||
- `load_factor` ([Float64](../data_types/float.md)) — Percentage filled in the dictionary (for a hashed dictionary, the percentage filled in the hash table).
|
||||
- `source` ([String](../data_types/string.md)) — Text describing the [data source](../query_language/dicts/external_dicts_dict_sources.md) for the dictionary.
|
||||
- `lifetime_min` ([UInt64](../data_types/int_uint.md#uint-ranges)) — Minimum [lifetime](../query_language/dicts/external_dicts_dict_lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds.
|
||||
- `lifetime_max` ([UInt64](../data_types/int_uint.md#uint-ranges)) — Maximum [lifetime](../query_language/dicts/external_dicts_dict_lifetime.md) of the dictionary in memory, after which ClickHouse tries to reload the dictionary (if `invalidate_query` is set, then only if it has changed). Set in seconds.
|
||||
- `loading_start_time` ([DateTime](../data_types/datetime.md)) — Start time for loading the dictionary.
|
||||
- `last_successful_update_time` ([DateTime](../data_types/datetime.md)) — End time for loading or updating the dictionary. Helps to monitor some troubles with external sources and investigate causes.
|
||||
- `loading_duration` ([Float32](../data_types/float.md)) — Duration of a dictionary loading.
|
||||
- `last_exception` ([String](../data_types/string.md)) — Text of the error that occurs when creating or reloading the dictionary if the dictionary couldn't be created.
|
||||
|
||||
Note that the amount of memory used by the dictionary is not proportional to the number of items stored in it. So for flat and cached dictionaries, all the memory cells are pre-assigned, regardless of how full the dictionary actually is.
|
||||
|
||||
**Example**
|
||||
|
||||
Configure the dictionary.
|
||||
|
||||
```sql
|
||||
CREATE DICTIONARY dictdb.dict
|
||||
(
|
||||
`key` Int64 DEFAULT -1,
|
||||
`value_default` String DEFAULT 'world',
|
||||
`value_expression` String DEFAULT 'xxx' EXPRESSION 'toString(127 * 172)'
|
||||
)
|
||||
PRIMARY KEY key
|
||||
SOURCE(CLICKHOUSE(HOST 'localhost' PORT 9000 USER 'default' TABLE 'dicttbl' DB 'dictdb'))
|
||||
LIFETIME(MIN 0 MAX 1)
|
||||
LAYOUT(FLAT())
|
||||
```
|
||||
|
||||
Make sure that the dictionary is loaded.
|
||||
|
||||
```sql
|
||||
SELECT * FROM system.dictionaries
|
||||
```
|
||||
|
||||
```text
|
||||
┌─database─┬─name─┬─status─┬─origin──────┬─type─┬─key────┬─attribute.names──────────────────────┬─attribute.types─────┬─bytes_allocated─┬─query_count─┬─hit_rate─┬─element_count─┬───────────load_factor─┬─source─────────────────────┬─lifetime_min─┬─lifetime_max─┬──loading_start_time─┌──last_successful_update_time─┬──────loading_duration─┬─last_exception─┐
|
||||
│ dictdb │ dict │ LOADED │ dictdb.dict │ Flat │ UInt64 │ ['value_default','value_expression'] │ ['String','String'] │ 74032 │ 0 │ 1 │ 1 │ 0.0004887585532746823 │ ClickHouse: dictdb.dicttbl │ 0 │ 1 │ 2020-03-04 04:17:34 │ 2020-03-04 04:30:34 │ 0.002 │ │
|
||||
└──────────┴──────┴────────┴─────────────┴──────┴────────┴──────────────────────────────────────┴─────────────────────┴─────────────────┴─────────────┴──────────┴───────────────┴───────────────────────┴────────────────────────────┴──────────────┴──────────────┴─────────────────────┴──────────────────────────────┘───────────────────────┴────────────────┘
|
||||
```
|
||||
|
||||
## system.events {#system_tables-events}
|
||||
|
||||
|
||||
83
docs/en/query_language/dicts/external_dicts_dict_lifetime.md
Normal file
83
docs/en/query_language/dicts/external_dicts_dict_lifetime.md
Normal file
@ -0,0 +1,83 @@
|
||||
|
||||
# Dictionary Updates
|
||||
|
||||
ClickHouse periodically updates the dictionaries. The update interval for fully downloaded dictionaries and the invalidation interval for cached dictionaries are defined in the `<lifetime>` tag in seconds.
|
||||
|
||||
Dictionary updates (other than loading for first use) do not block queries. During updates, the old version of a dictionary is used. If an error occurs during an update, the error is written to the server log, and queries continue using the old version of dictionaries.
|
||||
|
||||
Example of settings:
|
||||
|
||||
```xml
|
||||
<dictionary>
|
||||
...
|
||||
<lifetime>300</lifetime>
|
||||
...
|
||||
</dictionary>
|
||||
```
|
||||
|
||||
```sql
|
||||
CREATE DICTIONARY (...)
|
||||
...
|
||||
LIFETIME(300)
|
||||
...
|
||||
```
|
||||
|
||||
Setting `<lifetime>0</lifetime>` (`LIFETIME(0)`) prevents dictionaries from updating.
|
||||
|
||||
You can set a time interval for upgrades, and ClickHouse will choose a uniformly random time within this range. This is necessary in order to distribute the load on the dictionary source when upgrading on a large number of servers.
|
||||
|
||||
Example of settings:
|
||||
|
||||
```xml
|
||||
<dictionary>
|
||||
...
|
||||
<lifetime>
|
||||
<min>300</min>
|
||||
<max>360</max>
|
||||
</lifetime>
|
||||
...
|
||||
</dictionary>
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```sql
|
||||
LIFETIME(MIN 300 MAX 360)
|
||||
```
|
||||
If `<min>0</min>` and `<max>0</max>`, ClickHouse does not reload the dictionary by timeout.
|
||||
In this case, ClickHouse can reload the dictionary earlier if the dictionary configuration file was changed or the `SYSTEM RELOAD DICTIONARY` command was executed.
|
||||
|
||||
When upgrading the dictionaries, the ClickHouse server applies different logic depending on the type of [ source](external_dicts_dict_sources.md):
|
||||
|
||||
- For a text file, it checks the time of modification. If the time differs from the previously recorded time, the dictionary is updated.
|
||||
- For MyISAM tables, the time of modification is checked using a `SHOW TABLE STATUS` query.
|
||||
- Dictionaries from other sources are updated every time by default.
|
||||
|
||||
For MySQL (InnoDB), ODBC and ClickHouse sources, you can set up a query that will update the dictionaries only if they really changed, rather than each time. To do this, follow these steps:
|
||||
|
||||
- The dictionary table must have a field that always changes when the source data is updated.
|
||||
- The settings of the source must specify a query that retrieves the changing field. The ClickHouse server interprets the query result as a row, and if this row has changed relative to its previous state, the dictionary is updated. Specify the query in the `<invalidate_query>` field in the settings for the [source](external_dicts_dict_sources.md).
|
||||
|
||||
Example of settings:
|
||||
|
||||
```xml
|
||||
<dictionary>
|
||||
...
|
||||
<odbc>
|
||||
...
|
||||
<invalidate_query>SELECT update_time FROM dictionary_source where id = 1</invalidate_query>
|
||||
</odbc>
|
||||
...
|
||||
</dictionary>
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```sql
|
||||
...
|
||||
SOURCE(ODBC(... invalidate_query 'SELECT update_time FROM dictionary_source where id = 1'))
|
||||
...
|
||||
```
|
||||
|
||||
|
||||
[Original article](https://clickhouse.yandex/docs/en/query_language/dicts/external_dicts_dict_lifetime/) <!--hide-->
|
||||
@ -135,27 +135,66 @@ SELECT * FROM system.contributors WHERE name='Olga Khvostikova'
|
||||
Такие куски могут быть присоединены с помощью [ALTER TABLE ATTACH PARTITION\|PART](../query_language/query_language/alter/#alter_attach-partition). Остальные столбцы описаны в [system.parts](#system_tables-parts).
|
||||
Если имя куска некорректно, значения некоторых столбцов могут быть `NULL`. Такие куски могут быть удалены с помощью [ALTER TABLE DROP DETACHED PART](../query_language/query_language/alter/#alter_drop-detached).
|
||||
|
||||
## system.dictionaries {#system-dictionaries}
|
||||
## system.dictionaries {#system_tables-dictionaries}
|
||||
|
||||
Содержит информацию о внешних словарях.
|
||||
Содержит информацию о [внешних словарях](../query_language/dicts/external_dicts.md).
|
||||
|
||||
Столбцы:
|
||||
|
||||
- `name String` — Имя словаря.
|
||||
- `type String` — Тип словаря: Flat, Hashed, Cache.
|
||||
- `origin String` — Путь к конфигурационному файлу, в котором описан словарь.
|
||||
- `attribute.names Array(String)` — Массив имён атрибутов, предоставляемых словарём.
|
||||
- `attribute.types Array(String)` — Соответствующий массив типов атрибутов, предоставляемых словарём.
|
||||
- `has_hierarchy UInt8` — Является ли словарь иерархическим.
|
||||
- `bytes_allocated UInt64` — Количество оперативной памяти, которое использует словарь.
|
||||
- `hit_rate Float64` — Для cache-словарей - доля использований, для которых значение было в кэше.
|
||||
- `element_count UInt64` — Количество хранящихся в словаре элементов.
|
||||
- `load_factor Float64` — Доля заполненности словаря (для hashed словаря - доля заполнения хэш-таблицы).
|
||||
- `creation_time DateTime` — Время создания или последней успешной перезагрузки словаря.
|
||||
- `last_exception String` — Текст ошибки, возникшей при создании или перезагрузке словаря, если словарь не удалось создать.
|
||||
- `source String` - Текст, описывающий источник данных для словаря.
|
||||
- `database` ([String](../data_types/string.md)) — Имя базы данных, в которой находится словарь. Только для словарей, созданных с помощью DDL-запроса, для остальных — всегда пустая строка.
|
||||
- `name` ([String](../data_types/string.md)) — [Имя словаря](../query_language/dicts/external_dicts_dict.md).
|
||||
- `status` ([Enum8](../data_types/enum.md)) — Статус словаря. Возможные значения:
|
||||
- `NOT_LOADED` — Словарь не загружен, потому что не использовался.
|
||||
- `LOADED` — Словарь загружен успешно.
|
||||
- `FAILED` — Словарь не загружен в результате ошибки.
|
||||
- `LOADING` — Словарь в процессе загрузки.
|
||||
- `LOADED_AND_RELOADING` — Словарь загружен успешно, сейчас перезагружается (частые причины: запрос [SYSTEM RELOAD DICTIONARY](../query_language/system.md#query_language-system-reload-dictionary), таймаут, изменение настроек словаря).
|
||||
- `FAILED_AND_RELOADING` — Словарь не загружен в результате ошибки, сейчас перезагружается.
|
||||
- `origin` ([String](../data_types/string.md)) — Путь к конфигурационному файлу, описывающему словарь.
|
||||
- `type` ([String](../data_types/string.md)) — Тип размещения словаря. [Хранение словарей в памяти](../query_language/dicts/external_dicts_dict_layout.md).
|
||||
- `key` — [Тип ключа](../query_language/dicts/external_dicts_dict_structure.md#ext_dict_structure-key): Числовой ключ ([UInt64](../data_types/int_uint.md#uint-ranges)) или Составной ключ ([String](../data_types/string.md)) — строка вида "(тип 1, тип 2, ..., тип n)".
|
||||
- `attribute.names` ([Array](../data_types/array.md)([String](../data_types/string.md))) — Массив [имен атрибутов](../query_language/dicts/external_dicts_dict_structure.md#ext_dict_structure-attributes), предоставляемых справочником.
|
||||
- `attribute.types` ([Array](../data_types/array.md)([String](../data_types/string.md))) — Соответствующий массив [типов атрибутов](../query_language/dicts/external_dicts_dict_structure.md#ext_dict_structure-attributes), предоставляемых справочником.
|
||||
- `bytes_allocated` ([UInt64](../data_types/int_uint.md#uint-ranges)) — Объем оперативной памяти, используемый словарем.
|
||||
- `query_count` ([UInt64](../data_types/int_uint.md#uint-ranges)) — Количество запросов с момента загрузки словаря или с момента последней успешной перезагрузки.
|
||||
- `hit_rate` ([Float64](../data_types/float.md)) — Для cache-словарей — процент закэшированных значений.
|
||||
- `element_count` ([UInt64](../data_types/int_uint.md#uint-ranges)) — Количество элементов, хранящихся в словаре.
|
||||
- `load_factor` ([Float64](../data_types/float.md)) — Процент заполнения словаря (для хэшированного словаря — процент заполнения хэш-таблицы).
|
||||
- `source` ([String](../data_types/string.md)) — Текст, описывающий [источник данных](../query_language/dicts/external_dicts_dict_sources.md) для словаря.
|
||||
- `lifetime_min` ([UInt64](../data_types/int_uint.md#uint-ranges)) — Минимальное [время обновления](../query_language/dicts/external_dicts_dict_lifetime.md) словаря в памяти, по истечении которого Clickhouse попытается перезагрузить словарь (если задано `invalidate_query`, то только если он изменился). Задается в секундах.
|
||||
- `lifetime_max` ([UInt64](../data_types/int_uint.md#uint-ranges)) — Максимальное [время обновления](../query_language/dicts/external_dicts_dict_lifetime.md) словаря в памяти, по истечении которого Clickhouse попытается перезагрузить словарь (если задано `invalidate_query`, то только если он изменился). Задается в секундах.
|
||||
- `loading_start_time` ([DateTime](../data_types/datetime.md)) — Время начала загрузки словаря.
|
||||
- `loading_duration` ([Float32](../data_types/float.md)) — Время, затраченное на загрузку словаря.
|
||||
- `last_exception` ([String](../data_types/string.md)) — Текст ошибки, возникающей при создании или перезагрузке словаря, если словарь не удалось создать.
|
||||
|
||||
Заметим, что количество оперативной памяти, которое использует словарь, не является пропорциональным количеству элементов, хранящихся в словаре. Так, для flat и cached словарей, все ячейки памяти выделяются заранее, независимо от реальной заполненности словаря.
|
||||
**Пример**
|
||||
|
||||
Настройте словарь.
|
||||
|
||||
```sql
|
||||
CREATE DICTIONARY dictdb.dict
|
||||
(
|
||||
`key` Int64 DEFAULT -1,
|
||||
`value_default` String DEFAULT 'world',
|
||||
`value_expression` String DEFAULT 'xxx' EXPRESSION 'toString(127 * 172)'
|
||||
)
|
||||
PRIMARY KEY key
|
||||
SOURCE(CLICKHOUSE(HOST 'localhost' PORT 9000 USER 'default' TABLE 'dicttbl' DB 'dictdb'))
|
||||
LIFETIME(MIN 0 MAX 1)
|
||||
LAYOUT(FLAT())
|
||||
```
|
||||
|
||||
Убедитесь, что словарь загружен.
|
||||
|
||||
```sql
|
||||
SELECT * FROM system.dictionaries
|
||||
```
|
||||
|
||||
```text
|
||||
┌─database─┬─name─┬─status─┬─origin──────┬─type─┬─key────┬─attribute.names──────────────────────┬─attribute.types─────┬─bytes_allocated─┬─query_count─┬─hit_rate─┬─element_count─┬───────────load_factor─┬─source─────────────────────┬─lifetime_min─┬─lifetime_max─┬──loading_start_time─┌──last_successful_update_time─┬──────loading_duration─┬─last_exception─┐
|
||||
│ dictdb │ dict │ LOADED │ dictdb.dict │ Flat │ UInt64 │ ['value_default','value_expression'] │ ['String','String'] │ 74032 │ 0 │ 1 │ 1 │ 0.0004887585532746823 │ ClickHouse: dictdb.dicttbl │ 0 │ 1 │ 2020-03-04 04:17:34 │ 2020-03-04 04:30:34 │ 0.002 │ │
|
||||
└──────────┴──────┴────────┴─────────────┴──────┴────────┴──────────────────────────────────────┴─────────────────────┴─────────────────┴─────────────┴──────────┴───────────────┴───────────────────────┴────────────────────────────┴──────────────┴──────────────┴─────────────────────┴──────────────────────────────┘───────────────────────┴────────────────┘
|
||||
```
|
||||
|
||||
## system.events {#system_tables-events}
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user