thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-10 01:25:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

Use custom partition key for the query logs table

Browse Source
This commit is contained in:
Kirill Shvakov 2018-02-14 09:21:15 +02:00
parent f30ab52834
commit 034566fff3
4 changed files with 24 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
dbms/src/Interpreters/Context.cpp
View File

@ -1423,13 +1423,24 @@ QueryLog & Context::getQueryLog()
auto & config = getConfigRef();
String database = config.getString("query_log.database", "system");
String table = config.getString("query_log.table", "query_log");
String database = config.getString("query_log.database", "system");
String table = config.getString("query_log.table", "query_log");
String partition_by = config.getString("query_log.partition_by", "month");
size_t flush_interval_milliseconds = config.getUInt64(
"query_log.flush_interval_milliseconds", DEFAULT_QUERY_LOG_FLUSH_INTERVAL_MILLISECONDS);
String engine;
system_logs->query_log = std::make_unique<QueryLog>(
*global_context, database, table, "ENGINE = MergeTree(event_date, event_time, 1024)", flush_interval_milliseconds);
if (partition_by == "day")
engine = "ENGINE = MergeTree() PARTITION BY (event_date) ORDER BY (event_time) SETTINGS index_granularity = 1024";
else if (partition_by == "week")
engine = "ENGINE = MergeTree() PARTITION BY (toMonday(event_date)) ORDER BY (event_date, event_time) SETTINGS index_granularity = 1024";
else if (partition_by == "month")
engine = "ENGINE = MergeTree(event_date, event_time, 1024)";
else
throw Exception("Logical error: invalid value for query_log.partition_by", ErrorCodes::LOGICAL_ERROR);
system_logs->query_log = std::make_unique<QueryLog>(*global_context, database, table, engine, flush_interval_milliseconds);
}
return *system_logs->query_log;

2
dbms/src/Server/config.xml
View File

@ -226,7 +226,7 @@
-->
<database>system</database>
<table>query_log</table>
<partition_by>month</partition_by> <!-- possible values day, month, week -->
<!-- Interval of flushing data. -->
<flush_interval_milliseconds>7500</flush_interval_milliseconds>
</query_log>

4
docs/en/operations/server_settings/settings.md
View File

@ -518,6 +518,7 @@ Use the following parameters to configure logging:
- database – Name of the database.
- table – Name of the table.
- partition_by - Sets the partition key, possible values: day, week, month.
- flush_interval_milliseconds – Interval for flushing data from memory to the disk.
**Example**
@ -526,6 +527,7 @@ Use the following parameters to configure logging:
<part_log>
<database>system</database>
<table>part_log</table>
<partition_by>day</partition_by>
<flush_interval_milliseconds>7500</flush_interval_milliseconds>
</part_log>
```
@ -560,6 +562,7 @@ Use the following parameters to configure logging:
- database – Name of the database.
- table – Name of the table.
- partition_by - Sets the partition key, possible values: day, week, month.
- flush_interval_milliseconds – Interval for flushing data from memory to the disk.
If the table doesn't exist, ClickHouse will create it. If the structure of the query log changed when the ClickHouse server was updated, the table with the old structure is renamed, and a new table is created automatically.
@ -570,6 +573,7 @@ If the table doesn't exist, ClickHouse will create it. If the structure of the q
<query_log>
<database>system</database>
<table>query_log</table>
<partition_by>day</partition_by>
<flush_interval_milliseconds>7500</flush_interval_milliseconds>
</query_log>
```

4
docs/ru/operations/server_settings/settings.md
View File

@ -520,6 +520,7 @@ ClickHouse проверит условия `min_part_size` и `min_part_size_rat
- database - Имя базы данных.
- table - Имя таблицы.
- partition_by - Устанавливает ключ партиционирования, доступные значения: day, week, month.
- flush_interval_milliseconds - Период сброса данных из оперативной памяти на диск.
@ -529,6 +530,7 @@ ClickHouse проверит условия `min_part_size` и `min_part_size_rat
<part_log>
<database>system</database>
<table>part_log</table>
<partition_by>day</partition_by>
<flush_interval_milliseconds>7500</flush_interval_milliseconds>
</part_log>
```
@ -563,6 +565,7 @@ ClickHouse проверит условия `min_part_size` и `min_part_size_rat
- database - Имя базы данных.
- table - Имя таблицы.
- partition_by - Устанавливает ключ партиционирования, доступные значения: day, week, month.
- flush_interval_milliseconds - Период сброса данных из оперативной памяти на диск.
Если таблица не существует, то ClickHouse создаст её. Если структура журнала запросов изменилась при обновлении сервера ClickHouse, то таблица со старой структурой переименовывается, а новая таблица создается автоматически.
@ -573,6 +576,7 @@ ClickHouse проверит условия `min_part_size` и `min_part_size_rat
<query_log>
<database>system</database>
<table>query_log</table>
<partition_by>day</partition_by>
<flush_interval_milliseconds>7500</flush_interval_milliseconds>
</query_log>
```