ClickHouse/docs/en/engines/table_engines/special/merge.md

69 lines
3.2 KiB
Markdown
Raw Normal View History

2020-04-03 13:23:32 +00:00
---
toc_priority: 36
toc_title: Merge
---
2020-03-20 10:10:48 +00:00
# Merge {#merge}
2017-04-03 19:49:50 +00:00
The `Merge` engine (not to be confused with `MergeTree`) does not store data itself, but allows reading from any number of other tables simultaneously.
2017-04-26 17:26:17 +00:00
Reading is automatically parallelized. Writing to a table is not supported. When reading, the indexes of tables that are actually being read are used, if they exist.
The `Merge` engine accepts parameters: the database name and a regular expression for tables.
Example:
2020-03-20 10:10:48 +00:00
``` sql
Merge(hits, '^WatchLog')
```
CLICKHOUSE-2720: progress on website (#865) * update presentations * CLICKHOUSE-2936: redirect from clickhouse.yandex.ru and clickhouse.yandex.com * update submodule * lost files * CLICKHOUSE-2981: prefer sphinx docs over original reference * CLICKHOUSE-2981: docs styles more similar to main website + add flags to switch language links * update presentations * Less confusing directory structure (docs -> doc/reference/) * Minify sphinx docs too * Website release script: fail fast + pass docker hash on deploy * Do not underline links in docs * shorter * cleanup docker images * tune nginx config * CLICKHOUSE-3043: get rid of habrastorage links * Lost translation * CLICKHOUSE-2936: temporary client-side redirect * behaves weird in test * put redirect back * CLICKHOUSE-3047: copy docs txts to public too * move to proper file * remove old pages to avoid confusion * Remove reference redirect warning for now * Refresh README.md * Yellow buttons in docs * Use svg flags instead of unicode ones in docs * fix test website instance * Put flags to separate files * wrong flag * Copy Yandex.Metrica introduction from main page to docs * Yet another home page structure change, couple new blocks (CLICKHOUSE-3045) * Update Contacts section * CLICKHOUSE-2849: more detailed legal information * CLICKHOUSE-2978 preparation - split by files * More changes in Contacts block * Tune texts on index page * update presentations * One more benchmark * Add usage sections to index page, adapted from slides * Get the roadmap started, based on slides from last ClickHouse Meetup * CLICKHOUSE-2977: some rendering tuning * Get rid of excessive section in the end of getting started * Make headers linkable * CLICKHOUSE-2981: links to editing reference - https://github.com/yandex/ClickHouse/issues/849 * CLICKHOUSE-2981: fix mobile styles in docs * Ban crawling of duplicating docs * Open some external links in new tab * Ban old docs too * Lots of trivial fixes in english docs * Lots of trivial fixes in russian docs * Remove getting started copies in markdown * Add Yandex.Webmaster * Fix some sphinx warnings * More warnings fixed in english docs * More sphinx warnings fixed * Add code-block:: text * More code-block:: text * These headers look not that well * Better switch between documentation languages * merge use_case.rst into ya_metrika_task.rst * Edit the agg_functions.rst texts * Add lost empty lines
2017-06-13 04:15:47 +00:00
2020-03-20 10:10:48 +00:00
Data will be read from the tables in the `hits` database that have names that match the regular expression `^WatchLog`.
2017-04-03 19:49:50 +00:00
Instead of the database name, you can use a constant expression that returns a string. For example, `currentDatabase()`.
2017-04-03 19:49:50 +00:00
2018-04-23 06:20:21 +00:00
Regular expressions — [re2](https://github.com/google/re2) (supports a subset of PCRE), case-sensitive.
2020-03-20 10:10:48 +00:00
See the notes about escaping symbols in regular expressions in the “match” section.
2017-04-03 19:49:50 +00:00
When selecting tables to read, the `Merge` table itself will not be selected, even if it matches the regex. This is to avoid loops.
2020-03-20 10:10:48 +00:00
It is possible to create two `Merge` tables that will endlessly try to read each others data, but this is not a good idea.
2017-04-03 19:49:50 +00:00
The typical way to use the `Merge` engine is for working with a large number of `TinyLog` tables as if with a single table.
2017-04-03 19:49:50 +00:00
2018-10-04 20:58:28 +00:00
Example 2:
2020-03-20 10:10:48 +00:00
Lets say you have a old table (WatchLog\_old) and decided to change partitioning without moving data to a new table (WatchLog\_new) and you need to see data from both tables.
2018-10-04 20:58:28 +00:00
2020-03-20 10:10:48 +00:00
``` sql
CREATE TABLE WatchLog_old(date Date, UserId Int64, EventType String, Cnt UInt64)
2018-10-03 17:43:47 +00:00
ENGINE=MergeTree(date, (UserId, EventType), 8192);
INSERT INTO WatchLog_old VALUES ('2018-01-01', 1, 'hit', 3);
CREATE TABLE WatchLog_new(date Date, UserId Int64, EventType String, Cnt UInt64)
2018-10-03 17:43:47 +00:00
ENGINE=MergeTree PARTITION BY date ORDER BY (UserId, EventType) SETTINGS index_granularity=8192;
INSERT INTO WatchLog_new VALUES ('2018-01-02', 2, 'hit', 3);
CREATE TABLE WatchLog as WatchLog_old ENGINE=Merge(currentDatabase(), '^WatchLog');
SELECT *
FROM WatchLog
```
2020-03-20 10:10:48 +00:00
``` text
2018-10-03 17:43:47 +00:00
┌───────date─┬─UserId─┬─EventType─┬─Cnt─┐
│ 2018-01-01 │ 1 │ hit │ 3 │
└────────────┴────────┴───────────┴─────┘
┌───────date─┬─UserId─┬─EventType─┬─Cnt─┐
│ 2018-01-02 │ 2 │ hit │ 3 │
└────────────┴────────┴───────────┴─────┘
```
2018-10-04 20:58:28 +00:00
2020-03-20 10:10:48 +00:00
## Virtual Columns {#virtual-columns}
2017-04-03 19:49:50 +00:00
2020-04-03 13:23:32 +00:00
- `_table` — Contains the name of the table from which data was read. Type: [String](../../../sql_reference/data_types/string.md).
You can set the constant conditions on `_table` in the `WHERE/PREWHERE` clause (for example, `WHERE _table='xyz'`). In this case the read operation is performed only for that tables where the condition on `_table` is satisfied, so the `_table` column acts as an index.
**See Also**
- [Virtual columns](index.md#table_engines-virtual_columns)
2017-04-03 19:49:50 +00:00
2020-01-30 10:34:55 +00:00
[Original article](https://clickhouse.tech/docs/en/operations/table_engines/merge/) <!--hide-->