mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-12-05 05:52:05 +00:00
97f2a2213e
* Move some code outside dbms/src folder * Fix paths |
||
---|---|---|
.. | ||
benchmark.sh | ||
dump_dataset_from_ch.sh | ||
load_data_set.sql | ||
queries.sql | ||
README | ||
result_parser.py | ||
schema.sql |
Folder structure ______________ dump_dataset_from_ch.sh - bash script that dumps a dataset from Clickhouse schema.sql - schema for a Greenplum cluster to load dumped dataset in load_data_set.sql - the script that loads up a dumped dataset queries.sql - SQL statements used in the benchmark benchmark.sh - this piece of bash conducts a benchmark result_parser.py - script to parse benchmark.sh's output and produce python code to build a graph to compare up to 4 benchmark results. Requirements ____________ Greenplum uses a separate server as a point of entry, so you need 2 servers at least to run a cluster: master and segment hosts. 2 segments host and 56 segments(28 per host) had been used while conducting the test. You has has to put segment hostnames in the benchmark.sh. Greenplum quick installation instructions _________________________________________ Obtain a stable Greenplum version here(4.3.9.1 was used while conducting the benchmark): https://network.pivotal.io/products/pivotal-gpdb and install it using this detailed guide: http://gpdb.docs.pivotal.io/4340/install_guide/install_guide.html You should change gp_interconnect_type to 'tcp' if cluster members are connected via 1GB link or lower. There are some variables that has to be changed prior the first benchmark run: gp_vmem_protect_limit and max_statement_mem to allow each segment to use more virtual memory. Here are commands to change this GUCS that has to be executed as gpadmin at the master host: gpconfig -c gp_interconnect_type -v tcp gpconfig -c gp_vmem_protect_limit -v 3000 gpconfig -c max_statement_mem -v '4000MB' How to prepare data ------------------- One can prepare datasets to run the benchmark on using dump_dataset_from_ch.sh script from this repo. The script has to be run at at Clickhouse host. It takes a long time to get dumps. Upload the datasets into Greenplum master.Then run schema.sql to prepare schema and load_data_set.sql to load data up. This operation also takes a long time. How to conduct the benchmark __________________________ There is a benchmark.sh that take some arguments. Here is the syntax: ./benchmark.sh sql_statements_file tablename dbname orca_switch If you don't know about the last one then just use a default value.