2021-02-27 00:01:02 +00:00
---
2022-08-28 14:53:34 +00:00
slug: /en/sql-reference/table-functions/postgresql
2023-06-23 13:16:22 +00:00
sidebar_position: 160
2022-04-09 13:29:05 +00:00
sidebar_label: postgresql
2021-02-27 00:01:02 +00:00
---
2022-06-02 10:55:18 +00:00
# postgresql
2021-02-27 00:01:02 +00:00
Allows `SELECT` and `INSERT` queries to be performed on data that is stored on a remote PostgreSQL server.
**Syntax**
``` sql
2023-12-04 16:49:24 +00:00
postgresql({host:port, database, table, user, password[, schema, [, on_conflict]] | named_collection[, option=value [,..]]})
2021-02-27 00:01:02 +00:00
```
2023-12-04 16:49:24 +00:00
**Parameters**
2021-02-27 00:01:02 +00:00
2023-04-19 15:55:29 +00:00
- `host:port` — PostgreSQL server address.
- `database` — Remote database name.
- `table` — Remote table name.
- `user` — PostgreSQL user.
- `password` — User password.
- `schema` — Non-default table schema. Optional.
2023-12-04 16:49:24 +00:00
- `on_conflict` — Conflict resolution strategy. Example: `ON CONFLICT DO NOTHING` . Optional.
Arguments also can be passed using [named collections ](/docs/en/operations/named-collections.md ). In this case `host` and `port` should be specified separately. This approach is recommended for production environment.
2021-02-27 00:01:02 +00:00
2021-03-30 17:35:38 +00:00
**Returned Value**
A table object with the same columns as the original PostgreSQL table.
2023-03-18 02:45:43 +00:00
:::note
2022-04-09 13:29:05 +00:00
In the `INSERT` query to distinguish table function `postgresql(...)` from table name with column names list you must use keywords `FUNCTION` or `TABLE FUNCTION` . See examples below.
:::
2021-03-30 17:35:38 +00:00
2022-06-02 10:55:18 +00:00
## Implementation Details
2021-02-22 15:41:23 +00:00
2021-03-20 09:48:25 +00:00
`SELECT` queries on PostgreSQL side run as `COPY (SELECT ...) TO STDOUT` inside read-only PostgreSQL transaction with commit after each `SELECT` query.
2021-02-22 15:41:23 +00:00
2021-03-30 17:35:38 +00:00
Simple `WHERE` clauses such as `=` , `!=` , `>` , `>=` , `<` , `<=` , and `IN` are executed on the PostgreSQL server.
2021-02-22 15:41:23 +00:00
All joins, aggregations, sorting, `IN [ array ]` conditions and the `LIMIT` sampling constraint are executed in ClickHouse only after the query to PostgreSQL finishes.
2021-03-20 09:48:25 +00:00
`INSERT` queries on PostgreSQL side run as `COPY "table_name" (field1, field2, ... fieldN) FROM STDIN` inside PostgreSQL transaction with auto-commit after each `INSERT` statement.
2021-02-22 15:41:23 +00:00
PostgreSQL Array types converts into ClickHouse arrays.
2021-02-23 08:08:11 +00:00
2023-03-18 02:45:43 +00:00
:::note
2022-04-09 13:29:05 +00:00
Be careful, in PostgreSQL an array data type column like Integer[] may contain arrays of different dimensions in different rows, but in ClickHouse it is only allowed to have multidimensional arrays of the same dimension in all rows.
:::
2021-07-29 15:27:50 +00:00
2021-06-27 09:26:20 +00:00
Supports multiple replicas that must be listed by `|` . For example:
2021-02-22 15:41:23 +00:00
2021-06-24 19:29:57 +00:00
```sql
2021-06-27 20:25:36 +00:00
SELECT name FROM postgresql(`postgres{1|2|3}:5432`, 'postgres_database', 'postgres_table', 'user', 'password');
2021-06-24 19:29:57 +00:00
```
or
```sql
2021-06-27 20:25:36 +00:00
SELECT name FROM postgresql(`postgres1:5431|postgres2:5432`, 'postgres_database', 'postgres_table', 'user', 'password');
2021-06-24 19:29:57 +00:00
```
Supports replicas priority for PostgreSQL dictionary source. The bigger the number in map, the less the priority. The highest priority is `0` .
2021-02-27 00:01:02 +00:00
**Examples**
2021-02-22 15:41:23 +00:00
Table in PostgreSQL:
2021-02-27 00:01:02 +00:00
2021-02-22 15:41:23 +00:00
``` text
postgres=# CREATE TABLE "public"."test" (
"int_id" SERIAL,
"int_nullable" INT NULL DEFAULT NULL,
"float" FLOAT NOT NULL,
"str" VARCHAR(100) NOT NULL DEFAULT '',
"float_nullable" FLOAT NULL DEFAULT NULL,
PRIMARY KEY (int_id));
CREATE TABLE
2021-03-30 17:35:38 +00:00
postgres=# INSERT INTO test (int_id, str, "float") VALUES (1,'test',2);
2021-02-22 15:41:23 +00:00
INSERT 0 1
2021-03-30 17:35:38 +00:00
postgresql> SELECT * FROM test;
2021-04-08 19:32:27 +00:00
int_id | int_nullable | float | str | float_nullable
--------+--------------+-------+------+----------------
1 | | 2 | test |
2021-02-22 15:41:23 +00:00
(1 row)
2021-02-27 00:01:02 +00:00
```
2023-12-04 16:49:24 +00:00
Selecting data from ClickHouse using plain arguments:
2021-02-27 00:01:02 +00:00
2021-02-22 15:41:23 +00:00
```sql
2021-02-23 08:08:11 +00:00
SELECT * FROM postgresql('localhost:5432', 'test', 'test', 'postgresql_user', 'password') WHERE str IN ('test');
2021-02-27 00:01:02 +00:00
```
2023-12-04 16:49:24 +00:00
Or using [named collections ](/docs/en/operations/named-collections.md ):
```sql
CREATE NAMED COLLECTION mypg AS
host = 'localhost',
port = 5432,
database = 'test',
user = 'postgresql_user',
password = 'password';
SELECT * FROM postgresql(mypg, table='test') WHERE str IN ('test');
```
2021-02-27 00:01:02 +00:00
``` text
2021-02-22 15:41:23 +00:00
┌─int_id─┬─int_nullable─┬─float─┬─str──┬─float_nullable─┐
│ 1 │ ᴺᵁᴸᴸ │ 2 │ test │ ᴺᵁᴸᴸ │
└────────┴──────────────┴───────┴──────┴────────────────┘
2021-02-27 00:01:02 +00:00
```
2021-02-22 15:41:23 +00:00
Inserting:
2021-02-27 00:01:02 +00:00
```sql
2021-02-22 15:41:23 +00:00
INSERT INTO TABLE FUNCTION postgresql('localhost:5432', 'test', 'test', 'postgrsql_user', 'password') (int_id, float) VALUES (2, 3);
SELECT * FROM postgresql('localhost:5432', 'test', 'test', 'postgresql_user', 'password');
2021-02-27 00:01:02 +00:00
```
``` text
2021-02-22 15:41:23 +00:00
┌─int_id─┬─int_nullable─┬─float─┬─str──┬─float_nullable─┐
│ 1 │ ᴺᵁᴸᴸ │ 2 │ test │ ᴺᵁᴸᴸ │
│ 2 │ ᴺᵁᴸᴸ │ 3 │ │ ᴺᵁᴸᴸ │
└────────┴──────────────┴───────┴──────┴────────────────┘
2021-02-27 00:01:02 +00:00
```
2021-03-30 17:35:38 +00:00
Using Non-default Schema:
```text
postgres=# CREATE SCHEMA "nice.schema";
postgres=# CREATE TABLE "nice.schema"."nice.table" (a integer);
postgres=# INSERT INTO "nice.schema"."nice.table" SELECT i FROM generate_series(0, 99) as t(i)
```
```sql
CREATE TABLE pg_table_schema_with_dots (a UInt32)
ENGINE PostgreSQL('localhost:5432', 'clickhouse', 'nice.table', 'postgrsql_user', 'password', 'nice.schema');
```
2021-02-27 00:01:02 +00:00
**See Also**
2023-04-19 15:55:29 +00:00
- [The PostgreSQL table engine ](../../engines/table-engines/integrations/postgresql.md )
- [Using PostgreSQL as a dictionary source ](../../sql-reference/dictionaries/index.md#dictionary-sources#dicts-external_dicts_dict_sources-postgresql )
2023-01-17 15:38:10 +00:00
## Related content
2023-04-10 14:23:00 +00:00
2023-01-17 15:38:10 +00:00
- Blog: [ClickHouse and PostgreSQL - a match made in data heaven - part 1 ](https://clickhouse.com/blog/migrating-data-between-clickhouse-postgres )
2023-04-10 14:23:00 +00:00
- Blog: [ClickHouse and PostgreSQL - a Match Made in Data Heaven - part 2 ](https://clickhouse.com/blog/migrating-data-between-clickhouse-postgres-part-2 )
2024-08-02 06:10:53 +00:00
### Replicating or migrating Postgres data with with PeerDB
> In addition to table functions, you can always use [PeerDB](https://docs.peerdb.io/introduction) by ClickHouse to set up a continuous data pipeline from Postgres to ClickHouse. PeerDB is a tool designed specifically to replicate data from Postgres to ClickHouse using change data capture (CDC).