ClickHouse/docs/en/engines/table-engines/integrations/mongodb.md
2024-05-12 21:00:34 +03:00

3.4 KiB

slug sidebar_position sidebar_label
/en/engines/table-engines/integrations/mongodb 135 MongoDB

MongoDB

MongoDB engine is read-only table engine which allows to read data from remote MongoDB collection.

Only MongoDB v3.6+ servers are supported.

Types mappings

MongoDB ClickHouse
bool UInt8
int32 Int32
int64 Int64
double Float64
date Date, DateTime, DateTime64
timestamp Date, DateTime, DateTime64
string String, UUID
document String(as JSON)
array String(as JSON)
oid String

If key not found in MongoDB document, default value or null(if the column is nullable) will be inserted.

Supported conditions

WHERE

Only constant literals are allowed.

Types that can be used in WHERE section:

  • Null
  • UInt64
  • Int64
  • Float64
  • String
  • Array
  • Tuple
  • Map
  • UUID
  • Bool
  • Object

PREWHERE and HAVING are not supported.

LIMIT and OFFSET

Only LIMIT is supported.

ORDER BY

Simple expressions only are supported, without any modification like COLLATE, WITH, TO, etc.

WINDOW

Not supported.

GROUP BY

Not supported.

Aggregation functions

Not supported.

Creating a Table

CREATE TABLE [IF NOT EXISTS] [db.]table_name
(
    name1 [type1],
    name2 [type2],
    ...
) ENGINE = MongoDB(host:port, database, collection, user, password [, options]);

Engine Parameters

  • host:port — MongoDB server address.

  • database — Remote database name.

  • collection — Remote collection name.

  • user — MongoDB user.

  • password — User password.

  • options — MongoDB connection string options (optional parameter).

:::tip If you are using the MongoDB Atlas cloud offering please add these options:

'connectTimeoutMS=10000&ssl=true&authSource=admin'

:::

Also, you can simply pass an URI:

ENGINE = MongoDB(uri, collection);

Engine Parameters

  • uri — MongoDB server's connection URI

  • collection — Remote collection name.

Usage Example

Create a table in ClickHouse which allows to read data from MongoDB collection:

CREATE TABLE mongo_table
(
    key UInt64,
    data String
) ENGINE = MongoDB('mongo1:27017', 'test', 'simple_table', 'testuser', 'clickhouse');

or

ENGINE = MongoDB('mongodb://testuser:clickhouse@mongo1:27017/test', 'simple_table');

To read from an SSL secured MongoDB server:

CREATE TABLE mongo_table_ssl
(
    key UInt64,
    data String
) ENGINE = MongoDB('mongo2:27017', 'test', 'simple_table', 'testuser', 'clickhouse', 'ssl=true');

Query:

SELECT COUNT() FROM mongo_table;
┌─count()─┐
│       4 │
└─────────┘

You can also adjust connection timeout:

CREATE TABLE mongo_table
(
    key UInt64,
    data String
) ENGINE = MongoDB('mongo2:27017', 'test', 'simple_table', 'testuser', 'clickhouse', 'connectTimeoutMS=100000');

Troubleshooting

You can see the generated MongoDB query in DEBUG level logs.

Implementation details can be found in mongocxx and mongoc documentations.