ClickHouse/docs/en/engines/database-engines/materialized-postgresql.md

67 lines
3.2 KiB
Markdown
Raw Normal View History

2021-04-09 14:07:18 +00:00
---
toc_priority: 30
2021-06-27 19:09:17 +00:00
toc_title: MaterializedPostgreSQL
2021-04-09 14:07:18 +00:00
---
2021-06-27 19:09:17 +00:00
# MaterializedPostgreSQL {#materialize-postgresql}
2021-04-09 14:07:18 +00:00
## Creating a Database {#creating-a-database}
2021-05-04 10:43:21 +00:00
``` sql
CREATE DATABASE test_database
ENGINE = MaterializedPostgreSQL('postgres1:5432', 'postgres_database', 'postgres_user', 'postgres_password');
2021-05-04 10:43:21 +00:00
SELECT * FROM test_database.postgres_table;
```
## Settings {#settings}
2021-07-25 20:29:37 +00:00
1. `materialized_postgresql_max_block_size` — Number of rows collected in memory before flushing data into table. Default: `65536`.
2021-05-04 10:43:21 +00:00
2021-07-25 20:29:37 +00:00
2. `materialized_postgresql_tables_list` — A comma-separated list of PostgreSQL database tables, which will be replicated via MaterializedPostgreSQL database engine. Default: empty list - means whole PostgreSQL database will be replicated.
2021-05-04 10:43:21 +00:00
2021-07-25 20:38:14 +00:00
3. `materialized_postgresql_allow_automatic_update` — Allow to reload table in the background, when schema changes are detected. Default: `0` (`false`). DDL queries on PostgreSQL side are not replicated via ClickHouse `MaterializedPostgreSQL` engine, because it is not allowed with PostgreSQL logical replication protocol, but the fact of DDL changes is detected transactionally. In this case the default behaviour is to stop replicating those tables once DDL is detected. However, if this setting is enabled, then, instead of stopping replication of those tables, they will be reloaded in the background via database snapshot without data losses and replication will continue for them.
2021-05-04 10:43:21 +00:00
``` sql
CREATE DATABASE test_database
ENGINE = MaterializedPostgreSQL('postgres1:5432', 'postgres_database', 'postgres_user', 'postgres_password')
2021-06-27 19:09:17 +00:00
SETTINGS materialized_postgresql_max_block_size = 65536,
materialized_postgresql_tables_list = 'table1,table2,table3';
2021-05-04 10:43:21 +00:00
SELECT * FROM test_database.table1;
```
2021-04-11 19:58:33 +00:00
## Requirements {#requirements}
2021-04-09 14:07:18 +00:00
1. Setting [wal_level](https://www.postgresql.org/docs/current/runtime-config-wal.html) to `logical` and `max_replication_slots` to at least `2` in the PostgreSQL config file.
2021-05-04 10:43:21 +00:00
2021-07-25 20:38:14 +00:00
2. Each replicated table must have one of the following [replica identity](https://www.postgresql.org/docs/10/sql-altertable.html#SQL-CREATETABLE-REPLICA-IDENTITY):
2021-04-09 14:07:18 +00:00
- primary key (by default)
2021-04-09 14:07:18 +00:00
- index
2021-04-09 14:07:18 +00:00
2021-04-10 14:42:45 +00:00
``` bash
2021-04-09 14:07:18 +00:00
postgres# CREATE TABLE postgres_table (a Integer NOT NULL, b Integer, c Integer NOT NULL, d Integer, e Integer NOT NULL);
postgres# CREATE unique INDEX postgres_table_index on postgres_table(a, c, e);
postgres# ALTER TABLE postgres_table REPLICA IDENTITY USING INDEX postgres_table_index;
```
The primary key is always checked first. If it is absent, then the index, defined as replica identity index, is checked.
If the index is used as a replica identity, there has to be only one such index in a table.
2021-04-09 14:07:18 +00:00
You can check what type is used for a specific table with the following command:
2021-04-10 14:42:45 +00:00
``` bash
2021-04-09 14:07:18 +00:00
postgres# SELECT CASE relreplident
WHEN 'd' THEN 'default'
WHEN 'n' THEN 'nothing'
WHEN 'f' THEN 'full'
WHEN 'i' THEN 'index'
END AS replica_identity
FROM pg_class
WHERE oid = 'postgres_table'::regclass;
```
2021-06-27 19:09:17 +00:00
!!! warning "Warning"
2021-07-25 20:33:17 +00:00
Replication of [**TOAST**](https://www.postgresql.org/docs/9.5/storage-toast.html) values is not supported. Default value for the data type will be used.