ClickHouse

mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-21 23:21:59 +00:00

History

Alexey Milovidov 328768e778 Convert results to JSON		2022-07-07 09:15:24 +02:00
..
results	Convert results to JSON	2022-07-07 09:15:24 +02:00
create_partitioned.sql	Remove old benchmarks and move new one level up	2022-07-04 13:37:42 +02:00
create_single.sql	Remove old benchmarks and move new one level up	2022-07-04 13:37:42 +02:00
queries.sql	Remove old benchmarks and move new one level up	2022-07-04 13:37:42 +02:00
README.md	Remove old benchmarks and move new one level up	2022-07-04 13:37:42 +02:00
run.sh	Some results	2022-07-04 15:12:12 +02:00

README.md

Data Sources -> AWS Data Catalog -> Create Table -> Use S3 bucket data

Note: Athena does not support files. Only directories.

Go to query editor and run create.sql.

sudo apt-get install -y jq
export OUTPUT='s3://athena-experiments-milovidov/'

./run1.sh | tee ids.txt

Wait a few minutes. Then:

cat ids.txt | xargs -I{} aws --output json athena get-query-execution --query-execution-id {} | tee log.txt

cat log.txt | grep -P 'TotalExecutionTimeInMillis|FAILED' | grep -oP '\d+|FAILED' | 
    awk '{ if ($1 == "ERROR") { skip = 1 } else { if (i % 3 == 0) { printf "[" }; printf skip ? "null" : ($1 / 1000); if (i % 3 != 2) { printf "," } else { print "]," }; ++i; skip = 0; } }'