2018-12-21 19:23:55 +00:00
|
|
|
# URL(URL, Format) {#table_engines-url}
|
2018-07-27 10:21:04 +00:00
|
|
|
|
2018-09-04 11:18:59 +00:00
|
|
|
Manages data on a remote HTTP/HTTPS server. This engine is similar
|
2018-12-18 11:32:08 +00:00
|
|
|
to the [File](file.md) engine.
|
2018-07-27 10:21:04 +00:00
|
|
|
|
2018-09-04 11:18:59 +00:00
|
|
|
## Using the engine in the ClickHouse server
|
2018-07-27 10:21:04 +00:00
|
|
|
|
2019-05-15 09:04:24 +00:00
|
|
|
The `format` must be one that ClickHouse can use in
|
2018-09-04 11:18:59 +00:00
|
|
|
`SELECT` queries and, if necessary, in `INSERTs`. For the full list of supported formats, see
|
|
|
|
[Formats](../../interfaces/formats.md#formats).
|
2018-07-27 10:21:04 +00:00
|
|
|
|
2019-05-15 09:04:24 +00:00
|
|
|
The `URL` must conform to the structure of a Uniform Resource Locator. The specified URL must point to a server
|
2018-09-04 11:18:59 +00:00
|
|
|
that uses HTTP or HTTPS. This does not require any
|
|
|
|
additional headers for getting a response from the server.
|
2018-07-27 10:21:04 +00:00
|
|
|
|
2018-09-04 11:18:59 +00:00
|
|
|
`INSERT` and `SELECT` queries are transformed to `POST` and `GET` requests,
|
|
|
|
respectively. For processing `POST` requests, the remote server must support
|
|
|
|
[Chunked transfer encoding](https://en.wikipedia.org/wiki/Chunked_transfer_encoding).
|
2018-07-27 10:21:04 +00:00
|
|
|
|
|
|
|
**Example:**
|
|
|
|
|
2018-09-04 11:18:59 +00:00
|
|
|
**1.** Create a `url_engine_table` table on the server :
|
2018-07-27 10:21:04 +00:00
|
|
|
|
2019-09-23 15:31:46 +00:00
|
|
|
```sql
|
2018-07-27 10:21:04 +00:00
|
|
|
CREATE TABLE url_engine_table (word String, value UInt64)
|
|
|
|
ENGINE=URL('http://127.0.0.1:12345/', CSV)
|
|
|
|
```
|
|
|
|
|
2018-09-04 11:18:59 +00:00
|
|
|
**2.** Create a basic HTTP server using the standard Python 3 tools and
|
|
|
|
start it:
|
2018-07-27 10:21:04 +00:00
|
|
|
|
|
|
|
```python3
|
|
|
|
from http.server import BaseHTTPRequestHandler, HTTPServer
|
|
|
|
|
|
|
|
class CSVHTTPServer(BaseHTTPRequestHandler):
|
|
|
|
def do_GET(self):
|
|
|
|
self.send_response(200)
|
|
|
|
self.send_header('Content-type', 'text/csv')
|
|
|
|
self.end_headers()
|
|
|
|
|
|
|
|
self.wfile.write(bytes('Hello,1\nWorld,2\n', "utf-8"))
|
|
|
|
|
|
|
|
if __name__ == "__main__":
|
|
|
|
server_address = ('127.0.0.1', 12345)
|
|
|
|
HTTPServer(server_address, CSVHTTPServer).serve_forever()
|
|
|
|
```
|
|
|
|
|
|
|
|
```bash
|
2019-09-23 15:31:46 +00:00
|
|
|
$ python3 server.py
|
2018-07-27 10:21:04 +00:00
|
|
|
```
|
|
|
|
|
2018-09-04 11:18:59 +00:00
|
|
|
**3.** Request data:
|
2018-07-27 10:21:04 +00:00
|
|
|
|
2019-09-23 15:31:46 +00:00
|
|
|
```sql
|
2018-07-27 10:21:04 +00:00
|
|
|
SELECT * FROM url_engine_table
|
|
|
|
```
|
|
|
|
|
2019-09-23 15:31:46 +00:00
|
|
|
```text
|
2018-07-27 10:21:04 +00:00
|
|
|
┌─word──┬─value─┐
|
|
|
|
│ Hello │ 1 │
|
|
|
|
│ World │ 2 │
|
|
|
|
└───────┴───────┘
|
|
|
|
```
|
|
|
|
|
2018-08-10 14:44:49 +00:00
|
|
|
## Details of Implementation
|
2018-07-27 10:21:04 +00:00
|
|
|
|
|
|
|
- Reads and writes can be parallel
|
|
|
|
- Not supported:
|
2018-09-04 11:18:59 +00:00
|
|
|
- `ALTER` and `SELECT...SAMPLE` operations.
|
|
|
|
- Indexes.
|
|
|
|
- Replication.
|
2018-10-16 10:47:17 +00:00
|
|
|
|
|
|
|
[Original article](https://clickhouse.yandex/docs/en/operations/table_engines/url/) <!--hide-->
|