<p>Let's get started with sample dataset from open sources. We will use USA civil flights data since 1987 till 2015. It's hard to call this sample a Big Data (contains 166 millions rows, 63 Gb of uncompressed data) but this allows us to quickly get to work. Dataset is available for download <ahref="https://yadi.sk/d/pOZxpa42sDdgm">here</a>. Also you may download it from the original datasource <ahref="https://github.com/yandex/ClickHouse/raw/master/doc/example_datasets/1_ontime.txt">as described here</a>.</p>
<p>Firstly we will deploy ClickHouse to a single server. Below that we will also review the process of deployment to a cluster with support for sharding and replication.</p>
<p>On Ubuntu and Debian Linux ClickHouse can be installed from <ahref="https://clickhouse.yandex/#download">packages</a>. For other Linux distributions you can <ahref="https://github.com/yandex/ClickHouse/blob/master/doc/build.md">compile ClickHouse from sources</a> and then install.</p>
<p>Server config files are located in /etc/clickhouse-server/. Before getting to work please notice the <b>path</b> element in config. <b>Path</b> determines the location for data storage. It's not really handy to directly edit <b>config.xml</b> file considering package updates. Recommended way is to override the config elements in <ahref="https://clickhouse.yandex/reference_en.html#Configuration%20files">files of config.d directory</a>.
<p><b>clickhouse-server</b> won't be launched automatically after package installation. It won't be automatically restarted after updates either. Start the server with:
<pre>sudo service clickhouse-server start</pre>
Default location for server logs is /var/log/clickhouse-server/
<p>Now we have a table of <ahref="https://clickhouse.yandex/reference_en.html#MergeTree">MergeTree type</a>. MergeTree table type is recommended for usage in production. Table of this kind has a primary key used for incremental sort of table data. This allows fast execution of queries in ranges of a primary key.</p>
Let assume that our aim is to provide a set of reports for each advertiser. Common and frequently demanded query would be to count impressions for a specific Advertiser ID. This means that table primary key should start with <source>Advertiser ID</pre>. In this case ClickHouse needs to read smaller amount of data to perform the query for a given <source>Advertiser ID</pre>.</p>
<pre>xz -v -c -d < ontime.csv.xz | clickhouse-client --query="INSERT INTO ontime FORMAT CSV"</pre>
<p>ClickHouse INSERT query allows to load data in any <ahref="https://clickhouse.yandex/reference_en.html#Formats">supported format</a>. Data load requires just O(1) RAM consumption. INSERT query can receive any data volume as input. It's strongly recommended to insert data with <ahref="https://clickhouse.yandex/reference_en.html#Performance%20on%20data%20insertion.">not too small size blocks</a>. Notice that insert of blocks with size up to max_insert_block_size (= 1 048 576 rows by default) is an atomic operation: data block will be inserted completely or not inserted at all. In case of disconnect during insert operation you may not know if the block was inserted successfully. To achieve exactly-once semantics ClickHouse supports idempotency for <ahref="https://clickhouse.yandex/reference_en.html#Data%20replication">replicated tables</a>. This means that you may retry insert of the same data block (possibly on a different replicas) but this block will be inserted just once. Anyway in this guide we will load data from our localhost so we may not take care about data blocks generation and exactly-once semantics.</p>
<p>INSERT query into tables of MergeTree type is non-blocking (so does a SELECT query). You can execute SELECT queries right after of during insert operation.</p>
<p>The first is that String data type is used in cases when <ahref="https://clickhouse.yandex/reference_en.html#Enum">Enum</a> or numeric type would fit best.</p>
<pclass="tip"><b>⚖</b> When set of possible values is determined and known to be small. (E.g. OS name, browser vendors etc.) it's recommended to use Enums or numbers to improve performance.
When set of possible values is not limited (search query, URL, etc.) just go ahead with String.</p>
<p>The second is that dataset contains redundant fields like Year, Quarter, Month, DayOfMonth, DayOfWeek. In fact a single FlightDate would be enough. Most likely they have been added to improve performance for other DBMS'es which DateTime handling functions may be not efficient.</p>
<pclass="tip"><b>✯</b> ClickHouse <ahref="https://clickhouse.yandex/reference_en.html#Functions%20for%20working%20with%20dates%20and%20times">functions for operating with DateTime fields</a> are well-optimized so such redundancy is not required. Anyway much columns is not a reason to worry — ClickHouse is a <ahref="https://en.wikipedia.org/wiki/Column-oriented_DBMS">column-oriented DBMS</a>. This allows you to have as much fields as you need. Hundreds of columns in a table is fine for ClickHouse.</p>
<p><ahref="https://clickhouse.yandex/reference_en.html#Distributed">Distributed-table</a> is actually a kind of "view" to local tables of ClickHouse cluster. SELECT query from a distributed table will be executed using resources of all cluster's shards. You may specify configs for multiple clusters and create multiple Distributed-tables providing views to different clusters.</p>
<p>You can create a Distributed table on all machines in the cluster. This would allow to run distributed queries on any machine of the cluster. Besides distributed table you can also use <ahref="https://clickhouse.yandex/reference_en.html#remote">*remote* table function</a>.</p>
<p>Let's run <ahref="https://clickhouse.yandex/reference_en.html#INSERT">INSERT SELECT</a> into Distributed table to spread the table to multiple servers.</p>
<pclass="tip"><b>⚠</b> Worth to notice that the approach given above wouldn't fit for sharding of large tables.<br/>Please use <ahref="https://clickhouse.yandex/reference_en.html#Resharding">built-in sharding feature</a>.</p>
<p>You may have noticed that quantiles calculation are slightly different. This happens due to <ahref="https://github.com/tdunning/t-digest/raw/master/docs/t-digest-paper/histo.pdf">t-digest</a> algorithm implementation which is non-deterministic — it depends on the order of data processing.</p>
<p>To provide for resilience in production environment we recommend that each shard should contain 2-3 replicas distributed between multiple data-centers. Note that ClickHouse supports unlimited number of replicas.</p>
<p>To enable replication <ahref="http://zookeeper.apache.org/">ZooKeeper</a> is required. ClickHouse will take care of data consistency on all replicas and run restore procedure after failure automatically. It's recommended to deploy ZooKeeper cluster to separate servers.</p>
<p>ZooKeeper is not a requirement — in some simple cases you can duplicate the data by writing it into all the replicas from your application code. This approach is not recommended — in this case ClickHouse is not able to guarantee data consistency on all replicas. This remains the responsibility of your application.</p>
<p>If there are no replicas at the moment on replicated table creation — a new first replica will be instantiated. If there are already live replicas — new replica will clone the data from existing ones. You have an option to create all replicated tables first and that insert data to it. Another option is to create some replicas and add the others after or during data insertion.</p>
<p>Here we use <ahref="https://clickhouse.yandex/reference_en.html#ReplicatedMergeTree">ReplicatedMergeTree</a> table type. In parameters we specify ZooKeeper path containing shard and replica identifiers.</p>
<pre>INSERT INTO ontime_replica SELECT * FROM ontime;</pre>
<p>Replication operates in multi-master mode. Data can be loaded into any replica — it will be synced with other instances automatically. Replication is asynchronous so at a given moment of time not all replicas may contain recently inserted data. To allow data insertion at least one replica should be up. Others will sync up data and repair consistency once they will become active again. Please notice that such scheme allows for the possibility of just appended data loss.</p>
<p>Ask any questions on <ahref="https://stackoverflow.com/">Stackoverflow</a>. Use <ahref="https://groups.google.com/group/clickhouse">Google Group</a> for discussion.<br/>Or send private message to developers: <ahref="mailto:clickhouse-feedback@yandex-team.com">clickhouse-feedback@yandex-team.com</a>.</p>
<pclass="warranty">Software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</p>
