ClickHouse/docs/en/engines/table-engines/integrations/mongodb.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

165 lines
3.7 KiB
Markdown
Raw Normal View History

2021-02-07 14:29:54 +00:00
---
2022-08-28 14:53:34 +00:00
slug: /en/engines/table-engines/integrations/mongodb
sidebar_position: 135
sidebar_label: MongoDB
2021-02-07 14:29:54 +00:00
---
2022-06-02 10:55:18 +00:00
# MongoDB
2021-02-07 14:29:54 +00:00
2024-05-02 14:58:50 +00:00
MongoDB engine is read-only table engine which allows to read data from remote MongoDB collection.
2021-02-07 14:29:54 +00:00
2024-04-20 18:26:06 +00:00
Only MongoDB v3.6+ servers are supported.
## Types mappings
2024-05-16 20:35:23 +00:00
| MongoDB | ClickHouse |
|-------------|------------------------------------|
| bool | UInt8, String |
| int32 | Int32, String |
| int64 | Int64, String |
| double | Float64, String |
| date | Date, DateTime, DateTime64, String |
| timestamp | Date, DateTime, DateTime64, String |
| string | String, UUID |
| document | String(as JSON) |
| array | Array, String(as JSON) |
| oid | String |
| *any other* | String |
2024-04-20 18:26:06 +00:00
If key not found in MongoDB document, default value or null(if the column is nullable) will be inserted.
2024-05-16 20:35:23 +00:00
## Supported clauses
*Hint: you can use MongoDB table in CTE to perform any clauses, but be aware, that in some cases, performance will be significantly degraded.*
2024-05-02 14:58:50 +00:00
### WHERE
Only constant literals are allowed.
2024-04-20 18:26:06 +00:00
Types that can be used in WHERE section:
* Null
* UInt64
* Int64
* Float64
* String
* Array
* Tuple
* Map
* UUID
* Bool
* Object
2024-05-12 18:00:34 +00:00
PREWHERE and HAVING are not supported.
2024-05-02 14:58:50 +00:00
### LIMIT and OFFSET
Only `LIMIT` is supported.
### ORDER BY
Simple expressions only are supported, without any modification like COLLATE, WITH, TO, etc.
### WINDOW
Not supported.
### GROUP BY
Not supported.
### Aggregation functions
Not supported.
2021-02-07 14:29:54 +00:00
## Creating a Table {#creating-a-table}
``` sql
CREATE TABLE [IF NOT EXISTS] [db.]table_name
(
name1 [type1],
name2 [type2],
...
2021-03-23 15:18:12 +00:00
) ENGINE = MongoDB(host:port, database, collection, user, password [, options]);
2021-02-07 14:29:54 +00:00
```
**Engine Parameters**
- `host:port` — MongoDB server address.
2021-02-07 14:29:54 +00:00
- `database` — Remote database name.
2021-02-07 14:29:54 +00:00
- `collection` — Remote collection name.
2021-02-07 14:29:54 +00:00
- `user` — MongoDB user.
2021-02-07 14:29:54 +00:00
- `password` — User password.
2021-02-07 14:29:54 +00:00
- `options` — MongoDB connection string options (optional parameter).
2021-03-23 15:18:12 +00:00
2023-07-07 14:32:44 +00:00
:::tip
If you are using the MongoDB Atlas cloud offering please add these options:
```
'connectTimeoutMS=10000&ssl=true&authSource=admin'
```
:::
2024-05-12 18:00:34 +00:00
Also, you can simply pass an URI:
``` sql
ENGINE = MongoDB(uri, collection);
```
**Engine Parameters**
- `uri` — MongoDB server's connection URI
- `collection` — Remote collection name.
2021-02-07 14:29:54 +00:00
## Usage Example {#usage-example}
2021-03-23 15:18:12 +00:00
Create a table in ClickHouse which allows to read data from MongoDB collection:
2021-02-07 14:29:54 +00:00
2021-09-07 17:07:27 +00:00
``` sql
2021-02-07 14:29:54 +00:00
CREATE TABLE mongo_table
(
2021-03-23 15:18:12 +00:00
key UInt64,
2021-02-07 14:29:54 +00:00
data String
2021-02-08 14:30:15 +00:00
) ENGINE = MongoDB('mongo1:27017', 'test', 'simple_table', 'testuser', 'clickhouse');
2021-02-07 14:29:54 +00:00
```
2024-05-12 18:00:34 +00:00
or
``` sql
ENGINE = MongoDB('mongodb://testuser:clickhouse@mongo1:27017/test', 'simple_table');
```
2021-02-07 14:29:54 +00:00
2021-03-23 15:18:12 +00:00
To read from an SSL secured MongoDB server:
2021-09-07 17:07:27 +00:00
``` sql
2021-03-23 15:18:12 +00:00
CREATE TABLE mongo_table_ssl
(
key UInt64,
data String
) ENGINE = MongoDB('mongo2:27017', 'test', 'simple_table', 'testuser', 'clickhouse', 'ssl=true');
```
2021-02-07 14:29:54 +00:00
Query:
``` sql
SELECT COUNT() FROM mongo_table;
```
``` text
┌─count()─┐
│ 4 │
└─────────┘
```
2022-01-13 06:37:57 +00:00
You can also adjust connection timeout:
``` sql
CREATE TABLE mongo_table
(
key UInt64,
data String
) ENGINE = MongoDB('mongo2:27017', 'test', 'simple_table', 'testuser', 'clickhouse', 'connectTimeoutMS=100000');
```
2024-05-02 14:58:50 +00:00
## Troubleshooting
You can see the generated MongoDB query in DEBUG level logs.
2024-05-12 18:00:34 +00:00
Implementation details can be found in [mongocxx](https://github.com/mongodb/mongo-cxx-driver) and [mongoc](https://github.com/mongodb/mongo-c-driver) documentations.