ClickHouse/docs/en/engines/table-engines/integrations/materialized-postgresql.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

60 lines
3.1 KiB
Markdown
Raw Normal View History

2021-04-11 20:26:59 +00:00
---
2022-08-28 14:53:34 +00:00
slug: /en/engines/table-engines/integrations/materialized-postgresql
sidebar_position: 12
sidebar_label: MaterializedPostgreSQL
2022-08-29 16:19:50 +00:00
title: MaterializedPostgreSQL
2021-04-11 20:26:59 +00:00
---
Creates ClickHouse table with an initial data dump of PostgreSQL table and starts replication process, i.e. executes background job to apply new changes as they happen on PostgreSQL table in the remote PostgreSQL database.
2021-12-25 13:37:44 +00:00
If more than one table is required, it is highly recommended to use the [MaterializedPostgreSQL](../../../engines/database-engines/materialized-postgresql.md) database engine instead of the table engine and use the `materialized_postgresql_tables_list` setting, which specifies the tables to be replicated (will also be possible to add database `schema`). It will be much better in terms of CPU, fewer connections and fewer replication slots inside the remote PostgreSQL database.
2021-04-11 20:26:59 +00:00
## Creating a Table {#creating-a-table}
2021-05-04 10:43:21 +00:00
``` sql
CREATE TABLE postgresql_db.postgresql_replica (key UInt64, value UInt64)
2021-06-27 19:09:17 +00:00
ENGINE = MaterializedPostgreSQL('postgres1:5432', 'postgres_database', 'postgresql_replica', 'postgres_user', 'postgres_password')
2021-05-04 10:43:21 +00:00
PRIMARY KEY key;
```
**Engine Parameters**
- `host:port` — PostgreSQL server address.
- `database` — Remote database name.
- `table` — Remote table name.
- `user` — PostgreSQL user.
- `password` — User password.
2021-04-11 20:26:59 +00:00
## Requirements {#requirements}
1. The [wal_level](https://www.postgresql.org/docs/current/runtime-config-wal.html) setting must have a value `logical` and `max_replication_slots` parameter must have a value at least `2` in the PostgreSQL config file.
2021-04-11 20:26:59 +00:00
2. A table with `MaterializedPostgreSQL` engine must have a primary key — the same as a replica identity index (by default: primary key) of a PostgreSQL table (see [details on replica identity index](../../../engines/database-engines/materialized-postgresql.md#requirements)).
2021-04-11 20:26:59 +00:00
3. Only database [Atomic](https://en.wikipedia.org/wiki/Atomicity_(database_systems)) is allowed.
2021-05-04 10:43:21 +00:00
2023-04-15 14:24:27 +00:00
4. The `MaterializedPostgreSQL` table engine only works for PostgreSQL versions >= 11 as the implementation requires the [pg_replication_slot_advance](https://pgpedia.info/p/pg_replication_slot_advance.html) PostgreSQL function.
## Virtual columns {#virtual-columns}
2021-05-04 10:43:21 +00:00
- `_version` — Transaction counter. Type: [UInt64](../../../sql-reference/data-types/int-uint.md).
2021-05-04 10:43:21 +00:00
- `_sign` — Deletion mark. Type: [Int8](../../../sql-reference/data-types/int-uint.md). Possible values:
2021-12-25 13:37:44 +00:00
- `1` — Row is not deleted,
- `-1` — Row is deleted.
2021-05-13 07:36:40 +00:00
These columns do not need to be added when a table is created. They are always accessible in `SELECT` query.
2021-05-13 07:36:40 +00:00
`_version` column equals `LSN` position in `WAL`, so it might be used to check how up-to-date replication is.
2021-05-04 10:43:21 +00:00
``` sql
CREATE TABLE postgresql_db.postgresql_replica (key UInt64, value UInt64)
2021-06-27 19:09:17 +00:00
ENGINE = MaterializedPostgreSQL('postgres1:5432', 'postgres_database', 'postgresql_replica', 'postgres_user', 'postgres_password')
2021-05-04 10:43:21 +00:00
PRIMARY KEY key;
2021-05-13 07:36:40 +00:00
SELECT key, value, _version FROM postgresql_db.postgresql_replica;
2021-05-04 10:43:21 +00:00
```
2021-06-27 19:09:17 +00:00
2023-03-27 18:54:05 +00:00
:::note
Replication of [**TOAST**](https://www.postgresql.org/docs/9.5/storage-toast.html) values is not supported. The default value for the data type will be used.
:::