mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-20 06:32:08 +00:00
e8a5804e76
* update presentations * CLICKHOUSE-2936: redirect from clickhouse.yandex.ru and clickhouse.yandex.com * update submodule * lost files * CLICKHOUSE-2981: prefer sphinx docs over original reference * CLICKHOUSE-2981: docs styles more similar to main website + add flags to switch language links * update presentations * Less confusing directory structure (docs -> doc/reference/) * Minify sphinx docs too * Website release script: fail fast + pass docker hash on deploy * Do not underline links in docs * shorter * cleanup docker images * tune nginx config * CLICKHOUSE-3043: get rid of habrastorage links * Lost translation * CLICKHOUSE-2936: temporary client-side redirect * behaves weird in test * put redirect back * CLICKHOUSE-3047: copy docs txts to public too * move to proper file * remove old pages to avoid confusion * Remove reference redirect warning for now * Refresh README.md * Yellow buttons in docs * Use svg flags instead of unicode ones in docs * fix test website instance * Put flags to separate files * wrong flag * Copy Yandex.Metrica introduction from main page to docs * Yet another home page structure change, couple new blocks (CLICKHOUSE-3045) * Update Contacts section * CLICKHOUSE-2849: more detailed legal information * CLICKHOUSE-2978 preparation - split by files * More changes in Contacts block * Tune texts on index page * update presentations * One more benchmark * Add usage sections to index page, adapted from slides * Get the roadmap started, based on slides from last ClickHouse Meetup * CLICKHOUSE-2977: some rendering tuning * Get rid of excessive section in the end of getting started * Make headers linkable * CLICKHOUSE-2981: links to editing reference - https://github.com/yandex/ClickHouse/issues/849 * CLICKHOUSE-2981: fix mobile styles in docs * Ban crawling of duplicating docs * Open some external links in new tab * Ban old docs too * Lots of trivial fixes in english docs * Lots of trivial fixes in russian docs * Remove getting started copies in markdown * Add Yandex.Webmaster * Fix some sphinx warnings * More warnings fixed in english docs * More sphinx warnings fixed * Add code-block:: text * More code-block:: text * These headers look not that well * Better switch between documentation languages * merge use_case.rst into ya_metrika_task.rst * Edit the agg_functions.rst texts * Add lost empty lines
60 lines
3.4 KiB
ReStructuredText
60 lines
3.4 KiB
ReStructuredText
External data for query processing
|
|
==================================
|
|
|
|
ClickHouse allows sending a server the data that is needed for processing a query, together with a SELECT query. This data is put in a temporary table (see the section "Temporary tables") and can be used in the query (for example, in IN operators).
|
|
|
|
For example, if you have a text file with important user identifiers, you can upload it to the server along with a query that uses filtration by this list.
|
|
|
|
If you need to run more than one query with a large volume of external data, don't use this feature. It is better to upload the data to the DB ahead of time.
|
|
|
|
External data can be uploaded using the command-line client (in non-interactive mode), or using the HTTP interface.
|
|
|
|
In the command-line client, you can specify a parameters section in the format
|
|
.. code-block:: bash
|
|
|
|
--external --file=... [--name=...] [--format=...] [--types=...|--structure=...]
|
|
|
|
You may have multiple sections like this, for the number of tables being transmitted.
|
|
|
|
**--external** - Marks the beginning of the section.
|
|
**--file** - Path to the file with the table dump, or ``-``, which refers to stdin
|
|
Only a single table can be retrieved from stdin.
|
|
|
|
The following parameters are optional:
|
|
**--name** - Name of the table. If omitted, ``_data`` is used.
|
|
**--format** - Data format in the file. If omitted, ``TabSeparated`` is used.
|
|
|
|
One of the following parameters is required:
|
|
**--types** - A comma-separated list of column types. For example, ``UInt64,String``. Columns will be named ``_1``, ``_2``, ...
|
|
**--structure** - Table structure, in the format ``UserID UInt64, URL String``. Defines the column names and types.
|
|
|
|
The files specified in ``file`` will be parsed by the format specified in ``format``, using the data types specified in ``types`` or ``structure``. The table will be uploaded to the server and accessible there as a temporary table with the name ``name``.
|
|
|
|
Examples:
|
|
.. code-block:: bash
|
|
|
|
echo -ne "1\n2\n3\n" | clickhouse-client --query="SELECT count() FROM test.visits WHERE TraficSourceID IN _data" --external --file=- --types=Int8
|
|
849897
|
|
cat /etc/passwd | sed 's/:/\t/g' | clickhouse-client --query="SELECT shell, count() AS c FROM passwd GROUP BY shell ORDER BY c DESC" --external --file=- --name=passwd --structure='login String, unused String, uid UInt16, gid UInt16, comment String, home String, shell String'
|
|
/bin/sh 20
|
|
/bin/false 5
|
|
/bin/bash 4
|
|
/usr/sbin/nologin 1
|
|
/bin/sync 1
|
|
|
|
When using the HTTP interface, external data is passed in the multipart/form-data format. Each table is transmitted as a separate file. The table name is taken from the file name. The 'query_string' passes the parameters 'name_format', 'name_types', and 'name_structure', where name is the name of the table that these parameters correspond to. The meaning of the parameters is the same as when using the command-line client.
|
|
|
|
Example:
|
|
.. code-block:: bash
|
|
|
|
cat /etc/passwd | sed 's/:/\t/g' > passwd.tsv
|
|
|
|
curl -F 'passwd=@passwd.tsv;' 'http://localhost:8123/?query=SELECT+shell,+count()+AS+c+FROM+passwd+GROUP+BY+shell+ORDER+BY+c+DESC&passwd_structure=login+String,+unused+String,+uid+UInt16,+gid+UInt16,+comment+String,+home+String,+shell+String'
|
|
/bin/sh 20
|
|
/bin/false 5
|
|
/bin/bash 4
|
|
/usr/sbin/nologin 1
|
|
/bin/sync 1
|
|
|
|
For distributed query processing, the temporary tables are sent to all the remote servers.
|