ClickHouse/docs/en/engines/database-engines/materialized-postgresql.md

85 lines
3.3 KiB
Markdown
Raw Normal View History

2021-04-09 14:07:18 +00:00
---
toc_priority: 30
2021-06-27 19:09:17 +00:00
toc_title: MaterializedPostgreSQL
2021-04-09 14:07:18 +00:00
---
# [experimental] MaterializedPostgreSQL {#materialize-postgresql}
2021-04-09 14:07:18 +00:00
Creates ClickHouse database with an initial data dump of PostgreSQL database tables and starts replication process, i.e. executes background job to apply new changes as they happen on PostgreSQL database tables in the remote PostgreSQL database.
2021-07-29 06:02:20 +00:00
ClickHouse server works as PostgreSQL replica. It reads WAL and performs DML queries. DDL is not replicated, but can be handled (described below).
2021-04-09 14:07:18 +00:00
## Creating a Database {#creating-a-database}
2021-05-04 10:43:21 +00:00
``` sql
CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster]
ENGINE = MaterializedPostgreSQL('host:port', ['database' | database], 'user', 'password') [SETTINGS ...]
2021-05-04 10:43:21 +00:00
```
**Engine Parameters**
- `host:port` — PostgreSQL server endpoint.
- `database` — PostgreSQL database name.
- `user` — PostgreSQL user.
- `password` — User password.
2021-05-04 10:43:21 +00:00
## Settings {#settings}
- [materialized_postgresql_max_block_size](../../operations/settings/settings.md#materialized-postgresql-max-block-size)
2021-05-04 10:43:21 +00:00
- [materialized_postgresql_tables_list](../../operations/settings/settings.md#materialized-postgresql-tables-list)
2021-05-04 10:43:21 +00:00
- [materialized_postgresql_allow_automatic_update](../../operations/settings/settings.md#materialized-postgresql-allow-automatic-update)
2021-05-04 10:43:21 +00:00
``` sql
CREATE DATABASE database1
ENGINE = MaterializedPostgreSQL('postgres1:5432', 'postgres_database', 'postgres_user', 'postgres_password')
2021-06-27 19:09:17 +00:00
SETTINGS materialized_postgresql_max_block_size = 65536,
materialized_postgresql_tables_list = 'table1,table2,table3';
2021-05-04 10:43:21 +00:00
SELECT * FROM database1.table1;
2021-05-04 10:43:21 +00:00
```
2021-04-11 19:58:33 +00:00
## Requirements {#requirements}
2021-04-09 14:07:18 +00:00
1. The [wal_level](https://www.postgresql.org/docs/current/runtime-config-wal.html) setting must have a value `logical` and `max_replication_slots` parameter must have a value at least `2` in the PostgreSQL config file.
2021-05-04 10:43:21 +00:00
2021-07-25 20:38:14 +00:00
2. Each replicated table must have one of the following [replica identity](https://www.postgresql.org/docs/10/sql-altertable.html#SQL-CREATETABLE-REPLICA-IDENTITY):
2021-04-09 14:07:18 +00:00
- primary key (by default)
2021-04-09 14:07:18 +00:00
- index
2021-04-09 14:07:18 +00:00
2021-04-10 14:42:45 +00:00
``` bash
2021-04-09 14:07:18 +00:00
postgres# CREATE TABLE postgres_table (a Integer NOT NULL, b Integer, c Integer NOT NULL, d Integer, e Integer NOT NULL);
postgres# CREATE unique INDEX postgres_table_index on postgres_table(a, c, e);
postgres# ALTER TABLE postgres_table REPLICA IDENTITY USING INDEX postgres_table_index;
```
The primary key is always checked first. If it is absent, then the index, defined as replica identity index, is checked.
If the index is used as a replica identity, there has to be only one such index in a table.
2021-04-09 14:07:18 +00:00
You can check what type is used for a specific table with the following command:
2021-04-10 14:42:45 +00:00
``` bash
2021-04-09 14:07:18 +00:00
postgres# SELECT CASE relreplident
WHEN 'd' THEN 'default'
WHEN 'n' THEN 'nothing'
WHEN 'f' THEN 'full'
WHEN 'i' THEN 'index'
END AS replica_identity
FROM pg_class
WHERE oid = 'postgres_table'::regclass;
```
2021-06-27 19:09:17 +00:00
!!! warning "Warning"
Replication of [**TOAST**](https://www.postgresql.org/docs/9.5/storage-toast.html) values is not supported. The default value for the data type will be used.
## Example of Use {#example-of-use}
``` sql
CREATE DATABASE postgresql_db
ENGINE = MaterializedPostgreSQL('postgres1:5432', 'postgres_database', 'postgres_user', 'postgres_password');
SELECT * FROM postgresql_db.postgres_table;
```