ClickHouse/dbms/scripts/test_uniq_functions.sh

#!/bin/bash
for ((p = 2; p <= 10; p++))
do
    for ((i = 1; i <= 9; i++))
    do
        n=$(( 10**p * i ))
        echo -n "$n "
        clickhouse-client -q "select uniqHLL12(number), uniq(number), uniqCombined(number) from numbers($n);"
    done
done
Fix uniqHLL12 and uniqCombined for cardinalities 100M+. * Changing size() return type from UInt32 to UInt64 to prevent overflow (this shouldn't break back compatibility). * Removing "big cardinalities fix" for cardinalities >2^32/30 as it was very inaccurate and for estimates >2^32 it was trying to do 'log' of negative number which is NaN and it was casted to 0. * Adding python script to show that intHash32 is not a good choice for HyperLogLog algorithm when it's used for linear counting branch of it. * Adding bash script to test uniq, uniqHLL12, uniqCombined on different set cardinalities. * Altering documentation of uniq* aggregate functions with recommendations to use uniq instead of uniqHLL12 or uniqCombined. 2018-01-31 11:36:01 +00:00			`#!/bin/bash`
			`for ((p = 2; p <= 10; p++))`
			`do`
			`for ((i = 1; i <= 9; i++))`
			`do`
			`n=$(( 10*p i ))`
			`echo -n "$n "`
			`clickhouse-client -q "select uniqHLL12(number), uniq(number), uniqCombined(number) from numbers($n);"`
			`done`
			`done`