thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-21 01:00:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
95a69b9f4b
ClickHouse/benchmark/monetdb/aws.log

1775 lines
93 KiB
Plaintext
Raw Normal View History

Added a log of running benchmark of MonetDB on AWS m5a.8xlarge #13632
2020-08-13 00:34:48 +00:00
milovidov@milovidov-desktop:~$ ssh -i ▉▉▉.pem ubuntu@ec2-▉▉▉.us-east-2.compute.amazonaws.com
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.3.0-1032-aws x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Wed Aug 12 22:11:00 UTC 2020
System load: 0.06 Processes: 318
Usage of /: 2.3% of 193.82GB Users logged in: 1
Memory usage: 0% IP address for ens5: ▉▉▉
Swap usage: 0%
0 packages can be updated.
0 updates are security updates.
Last login: Wed Aug 12 22:00:27 2020 from 109.252.51.97
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu@ip-▉▉▉:~$ wget https://raw.githubusercontent.com/ClickHouse/ClickHouse/master/benchmark/clickhouse/benchmark-new.sh
--2020-08-12 22:11:07-- https://raw.githubusercontent.com/ClickHouse/ClickHouse/master/benchmark/clickhouse/benchmark-new.sh
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.248.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.248.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 468 [text/plain]
Saving to: ‘benchmark-new.sh’
benchmark-new.sh 100%[========================================================================================================================>] 468 --.-KB/s in 0s
2020-08-12 22:11:12 (44.9 MB/s) - ‘benchmark-new.sh’ saved [468/468]
ubuntu@ip-▉▉▉:~$ chmod a+x benchmark-new.sh
ubuntu@ip-▉▉▉:~$ wget https://raw.githubusercontent.com/ClickHouse/ClickHouse/master/benchmark/clickhouse/queries.sql
--2020-08-12 22:11:12-- https://raw.githubusercontent.com/ClickHouse/ClickHouse/master/benchmark/clickhouse/queries.sql
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.248.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.248.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8074 (7.9K) [text/plain]
Saving to: ‘queries.sql’
queries.sql 100%[========================================================================================================================>] 7.88K --.-KB/s in 0s
2020-08-12 22:11:12 (135 MB/s) - ‘queries.sql’ saved [8074/8074]
ubuntu@ip-▉▉▉:~$ wget https://clickhouse-datasets.s3.yandex.net/hits/partitions/hits_100m_obfuscated_v1.tar.xz
--2020-08-12 22:11:27-- https://clickhouse-datasets.s3.yandex.net/hits/partitions/hits_100m_obfuscated_v1.tar.xz
Resolving clickhouse-datasets.s3.yandex.net (clickhouse-datasets.s3.yandex.net)... 93.158.134.158, 2a02:6b8::2:158
Connecting to clickhouse-datasets.s3.yandex.net (clickhouse-datasets.s3.yandex.net)|93.158.134.158|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9722280160 (9.1G) [application/octet-stream]
Saving to: ‘hits_100m_obfuscated_v1.tar.xz’
hits_100m_obfuscated_v1.tar.xz 100%[========================================================================================================================>] 9.05G 10.8MB/s in 13m 7s
2020-08-12 22:24:35 (11.8 MB/s) - ‘hits_100m_obfuscated_v1.tar.xz’ saved [9722280160/9722280160]
ubuntu@ip-▉▉▉:~$ tar xvf hits_100m_obfuscated_v1.tar.xz -C .
hits_100m_obfuscated_v1/data/
hits_100m_obfuscated_v1/data/default/
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsParameter.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/WithHash.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/WatchID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Interests.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SocialNetwork.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsLink.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/RefererCategoryID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ResponseStartTiming.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ParamCurrency.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FetchTiming.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/MobilePhone.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/HistoryLength.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Params.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/CookieEnable.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ParamOrderID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Refresh.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/HasGCLID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsNotBounce.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/EventDate.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SocialAction.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/RemoteIP.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ResolutionWidth.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsEvent.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/WithHash.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SocialSourceNetworkID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/CLID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/DontCountHits.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/EventTime.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FlashMinor.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UTMMedium.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/WatchID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FlashMinor2.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/CLID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/URLCategoryID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/URLRegionID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsEvent.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsParameter.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/WindowClientHeight.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SocialSourcePage.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/checksums.txt
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/CounterID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SilverlightVersion1.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SocialSourceNetworkID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Sex.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/PageCharset.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UserAgentMajor.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ResolutionDepth.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OpenstatCampaignID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IPNetworkID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OriginalURL.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsArtifical.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/CounterClass.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Refresh.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UTMSource.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/BrowserLanguage.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/HTTPError.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/NetMajor.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UTMTerm.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OpenerName.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Robotness.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/GoodEvent.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FetchTiming.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UserAgent.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SilverlightVersion2.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ClientEventTime.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/EventTime.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/WindowName.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OriginalURL.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Params.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SearchEngineID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/URLRegionID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Age.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ParamCurrencyID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FlashMajor.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FUniqID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SilverlightVersion3.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UTMTerm.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/EventDate.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SilverlightVersion4.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/URLCategoryID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsOldCounter.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/WindowClientHeight.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Referer.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/RefererCategoryID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsDownload.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ClientTimeZone.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ParamPrice.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UTMCampaign.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsArtifical.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/HitColor.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/HistoryLength.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/RegionID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsLink.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Title.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FUniqID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SilverlightVersion2.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ConnectTiming.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ResolutionWidth.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/LocalEventTime.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/columns.txt
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/minmax_EventDate.idx
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UTMMedium.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/TraficSourceID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ClientTimeZone.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OpenstatSourceID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/NetMinor.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/RegionID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/HTTPError.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/URLHash.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsMobile.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/CodeVersion.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Age.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/URL.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/JavascriptEnable.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsOldCounter.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FlashMajor.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/WindowName.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/JavaEnable.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/CounterClass.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SocialSourcePage.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/RefererRegionID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/partition.dat
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/DNSTiming.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OpenstatServiceName.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UserAgentMinor.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/MobilePhoneModel.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OpenstatServiceName.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ResolutionHeight.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ParamCurrency.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Robotness.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SendTiming.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ConnectTiming.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FromTag.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OpenstatAdID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/URLHash.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/DontCountHits.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SendTiming.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ResponseEndTiming.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ParamOrderID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UTMCampaign.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsMobile.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/MobilePhone.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/RefererHash.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Referer.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/BrowserCountry.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/CookieEnable.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UserAgentMinor.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FlashMinor.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/GoodEvent.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/AdvEngineID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OS.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/HasGCLID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OS.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ResolutionHeight.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ResolutionDepth.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/BrowserLanguage.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/MobilePhoneModel.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OpenstatSourceID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsDownload.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Sex.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ClientIP.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/DNSTiming.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FlashMinor2.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/count.txt
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ClientEventTime.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/LocalEventTime.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UserAgentMajor.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/AdvEngineID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ResponseEndTiming.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/HitColor.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ParamCurrencyID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Title.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/WindowClientWidth.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SilverlightVersion4.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Income.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UTMContent.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/HID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SilverlightVersion1.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/NetMajor.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UserID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/FromTag.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/TraficSourceID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/primary.idx
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UTMContent.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/WindowClientWidth.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UserAgent.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SocialAction.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/JavascriptEnable.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SilverlightVersion3.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OpenstatCampaignID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/RefererHash.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OpenerName.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/BrowserCountry.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/JavaEnable.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Interests.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ResponseStartTiming.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/RefererRegionID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/RemoteIP.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IsNotBounce.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ClientIP.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/CounterID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UserID.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SearchPhrase.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/CodeVersion.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SearchPhrase.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/Income.bin
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/ParamPrice.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/PageCharset.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SearchEngineID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/SocialNetwork.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/OpenstatAdID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/URL.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/IPNetworkID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/HID.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/UTMSource.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/201307_1_96_4/NetMinor.mrk2
hits_100m_obfuscated_v1/data/default/hits_100m_obfuscated/format_version.txt
hits_100m_obfuscated_v1/metadata/
hits_100m_obfuscated_v1/metadata/default/
hits_100m_obfuscated_v1/metadata/default/hits_100m_obfuscated.sql
ubuntu@ip-▉▉▉:~$ mv hits_100m_obfuscated_v1/* .
mv: cannot move 'hits_100m_obfuscated_v1/data' to './data': Directory not empty
mv: cannot move 'hits_100m_obfuscated_v1/metadata' to './metadata': Directory not empty
ubuntu@ip-▉▉▉:~$ rm -rf data metadata
ubuntu@ip-▉▉▉:~$ mv hits_100m_obfuscated_v1/* .
ubuntu@ip-▉▉▉:~$ ./clickhouse client --query "SELECT count() FROM hits_100m_obfuscated"
100000000
ubuntu@ip-▉▉▉:~$ mcedit benchmark-new.sh
Command 'mcedit' not found, but can be installed with:
sudo apt install mc
ubuntu@ip-▉▉▉:~$ nanobenchmark-new.sh
nanobenchmark-new.sh: command not found
ubuntu@ip-▉▉▉:~$ nano benchmark-new.sh
ubuntu@ip-▉▉▉:~$ ./benchmark-new.sh hits_100m_obfuscated
[0.015, 0.001, 0.001],
[0.038, 0.015, 0.015],
[0.072, 0.033, 0.033],
[0.198, 0.052, 0.051],
[0.338, 0.166, 0.120],
[1.036, 0.252, 0.264],
[0.034, 0.020, 0.020],
[0.020, 0.015, 0.015],
[0.521, 0.414, 0.391],
[0.563, 0.496, 0.497],
[0.242, 0.143, 0.143],
[0.261, 0.168, 0.168],
[1.067, 0.574, 0.567],
[1.537, 0.772, 0.759],
[0.802, 0.736, 0.752],
[0.904, 0.797, 0.742],
[2.229, 1.795, 1.867],
[1.314, 0.987, 0.962],
[5.216, 4.149, 5.540],
[0.208, 0.057, 0.036],
[9.238, 0.660, 0.609],
[10.105, 0.685, 0.668],
[19.544, 1.851, 1.766],
[19.690, 0.943, 0.817],
[1.834, 0.228, 0.217],
[0.757, 0.177, 0.176],
[2.331, 0.224, 0.214],
[9.174, 0.728, 0.688],
[7.330, 1.130, 1.108],
[1.834, 1.810, 1.818],
[1.429, 0.547, 0.550],
[4.518, 0.879, 0.895],
[6.157, 5.540, 5.547],
[9.846, 3.033, 3.044],
[9.847, 3.061, 3.016],
[1.157, 1.086, 1.117],
[0.238, 0.169, 0.175],
[0.094, 0.072, 0.072],
[0.041, 0.037, 0.041],
[0.453, 0.364, 0.345],
[0.054, 0.015, 0.019],
[0.024, 0.010, 0.010],
[0.012, 0.006, 0.007],
ubuntu@ip-▉▉▉:~$ ./clickhouse client
ClickHouse client version 20.8.1.4338 (official build).
Connecting to localhost:9000 as user default.
Connected to ClickHouse server version 20.8.1 revision 54438.
ip-▉▉▉.us-east-2.compute.internal :) SELECT
:-] toInt64(WatchID) = -9223372036854775808 ? -9223372036854775807 : toInt64(WatchID),
:-] toInt8(JavaEnable) = -128 ? -127 : toInt8(JavaEnable),
:-] toValidUTF8(toString(Title)),
:-] toInt16(GoodEvent) = -32768 ? -32767 : toInt16(GoodEvent),
:-] EventTime,
:-] EventDate,
:-] toInt32(CounterID) = -2147483648 ? -2147483647 : toInt32(CounterID),
:-] toInt32(ClientIP) = -2147483648 ? -2147483647 : toInt32(ClientIP),
:-] toInt32(RegionID) = -2147483648 ? -2147483647 : toInt32(RegionID),
:-] toInt64(UserID) = -9223372036854775808 ? -9223372036854775807 : toInt64(UserID),
:-] toInt8(CounterClass) = -128 ? -127 : toInt8(CounterClass),
:-] toInt8(OS) = -128 ? -127 : toInt8(OS),
:-] toInt8(UserAgent) = -128 ? -127 : toInt8(UserAgent),
:-] toValidUTF8(toString(URL)),
:-] toValidUTF8(toString(Referer)),
:-] toInt8(Refresh) = -128 ? -127 : toInt8(Refresh),
:-] toInt16(RefererCategoryID) = -32768 ? -32767 : toInt16(RefererCategoryID),
:-] toInt32(RefererRegionID) = -2147483648 ? -2147483647 : toInt32(RefererRegionID),
:-] toInt16(URLCategoryID) = -32768 ? -32767 : toInt16(URLCategoryID),
:-] toInt32(URLRegionID) = -2147483648 ? -2147483647 : toInt32(URLRegionID),
:-] toInt16(ResolutionWidth) = -32768 ? -32767 : toInt16(ResolutionWidth),
:-] toInt16(ResolutionHeight) = -32768 ? -32767 : toInt16(ResolutionHeight),
:-] toInt8(ResolutionDepth) = -128 ? -127 : toInt8(ResolutionDepth),
:-] toInt8(FlashMajor) = -128 ? -127 : toInt8(FlashMajor),
:-] toInt8(FlashMinor) = -128 ? -127 : toInt8(FlashMinor),
:-] toValidUTF8(toString(FlashMinor2)),
:-] toInt8(NetMajor) = -128 ? -127 : toInt8(NetMajor),
:-] toInt8(NetMinor) = -128 ? -127 : toInt8(NetMinor),
:-] toInt16(UserAgentMajor) = -32768 ? -32767 : toInt16(UserAgentMajor),
:-] toValidUTF8(toString(UserAgentMinor)),
:-] toInt8(CookieEnable) = -128 ? -127 : toInt8(CookieEnable),
:-] toInt8(JavascriptEnable) = -128 ? -127 : toInt8(JavascriptEnable),
:-] toInt8(IsMobile) = -128 ? -127 : toInt8(IsMobile),
:-] toInt8(MobilePhone) = -128 ? -127 : toInt8(MobilePhone),
:-] toValidUTF8(toString(MobilePhoneModel)),
:-] toValidUTF8(toString(Params)),
:-] toInt32(IPNetworkID) = -2147483648 ? -2147483647 : toInt32(IPNetworkID),
:-] toInt8(TraficSourceID) = -128 ? -127 : toInt8(TraficSourceID),
:-] toInt16(SearchEngineID) = -32768 ? -32767 : toInt16(SearchEngineID),
:-] toValidUTF8(toString(SearchPhrase)),
:-] toInt8(AdvEngineID) = -128 ? -127 : toInt8(AdvEngineID),
:-] toInt8(IsArtifical) = -128 ? -127 : toInt8(IsArtifical),
:-] toInt16(WindowClientWidth) = -32768 ? -32767 : toInt16(WindowClientWidth),
:-] toInt16(WindowClientHeight) = -32768 ? -32767 : toInt16(WindowClientHeight),
:-] toInt16(ClientTimeZone) = -32768 ? -32767 : toInt16(ClientTimeZone),
:-] ClientEventTime,
:-] toInt8(SilverlightVersion1) = -128 ? -127 : toInt8(SilverlightVersion1),
:-] toInt8(SilverlightVersion2) = -128 ? -127 : toInt8(SilverlightVersion2),
:-] toInt32(SilverlightVersion3) = -2147483648 ? -2147483647 : toInt32(SilverlightVersion3),
:-] toInt16(SilverlightVersion4) = -32768 ? -32767 : toInt16(SilverlightVersion4),
:-] toValidUTF8(toString(PageCharset)),
:-] toInt32(CodeVersion) = -2147483648 ? -2147483647 : toInt32(CodeVersion),
:-] toInt8(IsLink) = -128 ? -127 : toInt8(IsLink),
:-] toInt8(IsDownload) = -128 ? -127 : toInt8(IsDownload),
:-] toInt8(IsNotBounce) = -128 ? -127 : toInt8(IsNotBounce),
:-] toInt64(FUniqID) = -9223372036854775808 ? -9223372036854775807 : toInt64(FUniqID),
:-] toValidUTF8(toString(OriginalURL)),
:-] toInt32(HID) = -2147483648 ? -2147483647 : toInt32(HID),
:-] toInt8(IsOldCounter) = -128 ? -127 : toInt8(IsOldCounter),
:-] toInt8(IsEvent) = -128 ? -127 : toInt8(IsEvent),
:-] toInt8(IsParameter) = -128 ? -127 : toInt8(IsParameter),
:-] toInt8(DontCountHits) = -128 ? -127 : toInt8(DontCountHits),
:-] toInt8(WithHash) = -128 ? -127 : toInt8(WithHash),
:-] toValidUTF8(toString(HitColor)),
:-] LocalEventTime,
:-] toInt8(Age) = -128 ? -127 : toInt8(Age),
:-] toInt8(Sex) = -128 ? -127 : toInt8(Sex),
:-] toInt8(Income) = -128 ? -127 : toInt8(Income),
:-] toInt16(Interests) = -32768 ? -32767 : toInt16(Interests),
:-] toInt8(Robotness) = -128 ? -127 : toInt8(Robotness),
:-] toInt32(RemoteIP) = -2147483648 ? -2147483647 : toInt32(RemoteIP),
:-] toInt32(WindowName) = -2147483648 ? -2147483647 : toInt32(WindowName),
:-] toInt32(OpenerName) = -2147483648 ? -2147483647 : toInt32(OpenerName),
:-] toInt16(HistoryLength) = -32768 ? -32767 : toInt16(HistoryLength),
:-] toValidUTF8(toString(BrowserLanguage)),
:-] toValidUTF8(toString(BrowserCountry)),
:-] toValidUTF8(toString(SocialNetwork)),
:-] toValidUTF8(toString(SocialAction)),
:-] toInt16(HTTPError) = -32768 ? -32767 : toInt16(HTTPError),
:-] toInt32(SendTiming) = -2147483648 ? -2147483647 : toInt32(SendTiming),
:-] toInt32(DNSTiming) = -2147483648 ? -2147483647 : toInt32(DNSTiming),
:-] toInt32(ConnectTiming) = -2147483648 ? -2147483647 : toInt32(ConnectTiming),
:-] toInt32(ResponseStartTiming) = -2147483648 ? -2147483647 : toInt32(ResponseStartTiming),
:-] toInt32(ResponseEndTiming) = -2147483648 ? -2147483647 : toInt32(ResponseEndTiming),
:-] toInt32(FetchTiming) = -2147483648 ? -2147483647 : toInt32(FetchTiming),
:-] toInt8(SocialSourceNetworkID) = -128 ? -127 : toInt8(SocialSourceNetworkID),
:-] toValidUTF8(toString(SocialSourcePage)),
:-] toInt64(ParamPrice) = -9223372036854775808 ? -9223372036854775807 : toInt64(ParamPrice),
:-] toValidUTF8(toString(ParamOrderID)),
:-] toValidUTF8(toString(ParamCurrency)),
:-] toInt16(ParamCurrencyID) = -32768 ? -32767 : toInt16(ParamCurrencyID),
:-] toValidUTF8(toString(OpenstatServiceName)),
:-] toValidUTF8(toString(OpenstatCampaignID)),
:-] toValidUTF8(toString(OpenstatAdID)),
:-] toValidUTF8(toString(OpenstatSourceID)),
:-] toValidUTF8(toString(UTMSource)),
:-] toValidUTF8(toString(UTMMedium)),
:-] toValidUTF8(toString(UTMCampaign)),
:-] toValidUTF8(toString(UTMContent)),
:-] toValidUTF8(toString(UTMTerm)),
:-] toValidUTF8(toString(FromTag)),
:-] toInt8(HasGCLID) = -128 ? -127 : toInt8(HasGCLID),
:-] toInt64(RefererHash) = -9223372036854775808 ? -9223372036854775807 : toInt64(RefererHash),
:-] toInt64(URLHash) = -9223372036854775808 ? -9223372036854775807 : toInt64(URLHash),
:-] toInt32(CLID) = -2147483648 ? -2147483647 : toInt32(CLID)
:-] FROM hits_100m_obfuscated
:-] INTO OUTFILE '/home/ubuntu/hits_100m_obfuscated_monetdb.tsv'
:-] FORMAT TSV;
SELECT
if(toInt64(WatchID) = -9223372036854775808, -9223372036854775807, toInt64(WatchID)),
if(toInt8(JavaEnable) = -128, -127, toInt8(JavaEnable)),
toValidUTF8(toString(Title)),
if(toInt16(GoodEvent) = -32768, -32767, toInt16(GoodEvent)),
EventTime,
EventDate,
if(toInt32(CounterID) = -2147483648, -2147483647, toInt32(CounterID)),
if(toInt32(ClientIP) = -2147483648, -2147483647, toInt32(ClientIP)),
if(toInt32(RegionID) = -2147483648, -2147483647, toInt32(RegionID)),
if(toInt64(UserID) = -9223372036854775808, -9223372036854775807, toInt64(UserID)),
if(toInt8(CounterClass) = -128, -127, toInt8(CounterClass)),
if(toInt8(OS) = -128, -127, toInt8(OS)),
if(toInt8(UserAgent) = -128, -127, toInt8(UserAgent)),
toValidUTF8(toString(URL)),
toValidUTF8(toString(Referer)),
if(toInt8(Refresh) = -128, -127, toInt8(Refresh)),
if(toInt16(RefererCategoryID) = -32768, -32767, toInt16(RefererCategoryID)),
if(toInt32(RefererRegionID) = -2147483648, -2147483647, toInt32(RefererRegionID)),
if(toInt16(URLCategoryID) = -32768, -32767, toInt16(URLCategoryID)),
if(toInt32(URLRegionID) = -2147483648, -2147483647, toInt32(URLRegionID)),
if(toInt16(ResolutionWidth) = -32768, -32767, toInt16(ResolutionWidth)),
if(toInt16(ResolutionHeight) = -32768, -32767, toInt16(ResolutionHeight)),
if(toInt8(ResolutionDepth) = -128, -127, toInt8(ResolutionDepth)),
if(toInt8(FlashMajor) = -128, -127, toInt8(FlashMajor)),
if(toInt8(FlashMinor) = -128, -127, toInt8(FlashMinor)),
toValidUTF8(toString(FlashMinor2)),
if(toInt8(NetMajor) = -128, -127, toInt8(NetMajor)),
if(toInt8(NetMinor) = -128, -127, toInt8(NetMinor)),
if(toInt16(UserAgentMajor) = -32768, -32767, toInt16(UserAgentMajor)),
toValidUTF8(toString(UserAgentMinor)),
if(toInt8(CookieEnable) = -128, -127, toInt8(CookieEnable)),
if(toInt8(JavascriptEnable) = -128, -127, toInt8(JavascriptEnable)),
if(toInt8(IsMobile) = -128, -127, toInt8(IsMobile)),
if(toInt8(MobilePhone) = -128, -127, toInt8(MobilePhone)),
toValidUTF8(toString(MobilePhoneModel)),
toValidUTF8(toString(Params)),
if(toInt32(IPNetworkID) = -2147483648, -2147483647, toInt32(IPNetworkID)),
if(toInt8(TraficSourceID) = -128, -127, toInt8(TraficSourceID)),
if(toInt16(SearchEngineID) = -32768, -32767, toInt16(SearchEngineID)),
toValidUTF8(toString(SearchPhrase)),
if(toInt8(AdvEngineID) = -128, -127, toInt8(AdvEngineID)),
if(toInt8(IsArtifical) = -128, -127, toInt8(IsArtifical)),
if(toInt16(WindowClientWidth) = -32768, -32767, toInt16(WindowClientWidth)),
if(toInt16(WindowClientHeight) = -32768, -32767, toInt16(WindowClientHeight)),
if(toInt16(ClientTimeZone) = -32768, -32767, toInt16(ClientTimeZone)),
ClientEventTime,
if(toInt8(SilverlightVersion1) = -128, -127, toInt8(SilverlightVersion1)),
if(toInt8(SilverlightVersion2) = -128, -127, toInt8(SilverlightVersion2)),
if(toInt32(SilverlightVersion3) = -2147483648, -2147483647, toInt32(SilverlightVersion3)),
if(toInt16(SilverlightVersion4) = -32768, -32767, toInt16(SilverlightVersion4)),
toValidUTF8(toString(PageCharset)),
if(toInt32(CodeVersion) = -2147483648, -2147483647, toInt32(CodeVersion)),
if(toInt8(IsLink) = -128, -127, toInt8(IsLink)),
if(toInt8(IsDownload) = -128, -127, toInt8(IsDownload)),
if(toInt8(IsNotBounce) = -128, -127, toInt8(IsNotBounce)),
if(toInt64(FUniqID) = -9223372036854775808, -9223372036854775807, toInt64(FUniqID)),
toValidUTF8(toString(OriginalURL)),
if(toInt32(HID) = -2147483648, -2147483647, toInt32(HID)),
if(toInt8(IsOldCounter) = -128, -127, toInt8(IsOldCounter)),
if(toInt8(IsEvent) = -128, -127, toInt8(IsEvent)),
if(toInt8(IsParameter) = -128, -127, toInt8(IsParameter)),
if(toInt8(DontCountHits) = -128, -127, toInt8(DontCountHits)),
if(toInt8(WithHash) = -128, -127, toInt8(WithHash)),
toValidUTF8(toString(HitColor)),
LocalEventTime,
if(toInt8(Age) = -128, -127, toInt8(Age)),
if(toInt8(Sex) = -128, -127, toInt8(Sex)),
if(toInt8(Income) = -128, -127, toInt8(Income)),
if(toInt16(Interests) = -32768, -32767, toInt16(Interests)),
if(toInt8(Robotness) = -128, -127, toInt8(Robotness)),
if(toInt32(RemoteIP) = -2147483648, -2147483647, toInt32(RemoteIP)),
if(toInt32(WindowName) = -2147483648, -2147483647, toInt32(WindowName)),
if(toInt32(OpenerName) = -2147483648, -2147483647, toInt32(OpenerName)),
if(toInt16(HistoryLength) = -32768, -32767, toInt16(HistoryLength)),
toValidUTF8(toString(BrowserLanguage)),
toValidUTF8(toString(BrowserCountry)),
toValidUTF8(toString(SocialNetwork)),
toValidUTF8(toString(SocialAction)),
if(toInt16(HTTPError) = -32768, -32767, toInt16(HTTPError)),
if(toInt32(SendTiming) = -2147483648, -2147483647, toInt32(SendTiming)),
if(toInt32(DNSTiming) = -2147483648, -2147483647, toInt32(DNSTiming)),
if(toInt32(ConnectTiming) = -2147483648, -2147483647, toInt32(ConnectTiming)),
if(toInt32(ResponseStartTiming) = -2147483648, -2147483647, toInt32(ResponseStartTiming)),
if(toInt32(ResponseEndTiming) = -2147483648, -2147483647, toInt32(ResponseEndTiming)),
if(toInt32(FetchTiming) = -2147483648, -2147483647, toInt32(FetchTiming)),
if(toInt8(SocialSourceNetworkID) = -128, -127, toInt8(SocialSourceNetworkID)),
toValidUTF8(toString(SocialSourcePage)),
if(toInt64(ParamPrice) = -9223372036854775808, -9223372036854775807, toInt64(ParamPrice)),
toValidUTF8(toString(ParamOrderID)),
toValidUTF8(toString(ParamCurrency)),
if(toInt16(ParamCurrencyID) = -32768, -32767, toInt16(ParamCurrencyID)),
toValidUTF8(toString(OpenstatServiceName)),
toValidUTF8(toString(OpenstatCampaignID)),
toValidUTF8(toString(OpenstatAdID)),
toValidUTF8(toString(OpenstatSourceID)),
toValidUTF8(toString(UTMSource)),
toValidUTF8(toString(UTMMedium)),
toValidUTF8(toString(UTMCampaign)),
toValidUTF8(toString(UTMContent)),
toValidUTF8(toString(UTMTerm)),
toValidUTF8(toString(FromTag)),
if(toInt8(HasGCLID) = -128, -127, toInt8(HasGCLID)),
if(toInt64(RefererHash) = -9223372036854775808, -9223372036854775807, toInt64(RefererHash)),
if(toInt64(URLHash) = -9223372036854775808, -9223372036854775807, toInt64(URLHash)),
if(toInt32(CLID) = -2147483648, -2147483647, toInt32(CLID))
FROM hits_100m_obfuscated
INTO OUTFILE '/home/ubuntu/hits_100m_obfuscated_monetdb.tsv'
FORMAT TSV
100000000 rows in set. Elapsed: 460.329 sec. Processed 100.00 million rows, 74.69 GB (217.24 thousand rows/s., 162.25 MB/s.)
ip-▉▉▉.us-east-2.compute.internal :) Bye.
ubuntu@ip-▉▉▉:~$ ls -l
total 86156956
-rwxrwxr-x 1 ubuntu ubuntu 502 Aug 12 22:37 benchmark-new.sh
-rwxrwxr-x 1 ubuntu ubuntu 3676614664 Aug 11 02:05 clickhouse
drwxrwxr-x 4 ubuntu ubuntu 4096 Aug 12 22:36 data
drwxr-x--- 2 ubuntu ubuntu 4096 Aug 12 22:10 dictionaries_lib
drwxr-x--- 2 ubuntu ubuntu 4096 Aug 12 22:10 flags
drwxr-x--- 2 ubuntu ubuntu 4096 Aug 12 22:10 format_schemas
-rw-rw-r-- 1 ubuntu ubuntu 74825746736 Aug 12 22:50 hits_100m_obfuscated_monetdb.tsv
drwxrwxr-x 2 ubuntu ubuntu 4096 Aug 12 22:36 hits_100m_obfuscated_v1
-rw-rw-r-- 1 ubuntu ubuntu 9722280160 Aug 7 2019 hits_100m_obfuscated_v1.tar.xz
drwxrwxr-x 4 ubuntu ubuntu 4096 Aug 12 22:36 metadata
drwxr-x--- 2 ubuntu ubuntu 4096 Aug 12 22:10 metadata_dropped
drwxr-x--- 2 ubuntu ubuntu 4096 Aug 12 22:10 preprocessed_configs
-rw-rw-r-- 1 ubuntu ubuntu 8074 Aug 12 22:11 queries.sql
-rw-r----- 1 ubuntu ubuntu 58 Aug 12 22:36 status
drwxr-x--- 2 ubuntu ubuntu 4096 Aug 12 22:10 tmp
drwxr-x--- 2 ubuntu ubuntu 4096 Aug 12 22:10 user_files
ubuntu@ip-▉▉▉:~$ sudo nano /etc/apt/sources.list.d/monetdb.list
ubuntu@ip-▉▉▉:~$ wget --output-document=- https://www.monetdb.org/downloads/MonetDB-GPG-KEY | sudo apt-key add -
--2020-08-12 22:51:53-- https://www.monetdb.org/downloads/MonetDB-GPG-KEY
Resolving www.monetdb.org (www.monetdb.org)... 192.16.197.137
Connecting to www.monetdb.org (www.monetdb.org)|192.16.197.137|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6738 (6.6K)
Saving to: ‘STDOUT’
- 100%[========================================================================================================================>] 6.58K --.-KB/s in 0s
2020-08-12 22:51:53 (2.24 GB/s) - written to stdout [6738/6738]
OK
ubuntu@ip-▉▉▉:~$ sudo apt update
Hit:1 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic InRelease
Get:2 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:3 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:4 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:5 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [8570 kB]
Get:6 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic/universe Translation-en [4941 kB]
Get:7 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [151 kB]
Get:8 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic/multiverse Translation-en [108 kB]
Get:9 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1032 kB]
Get:10 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1097 kB]
Get:11 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [342 kB]
Get:12 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [19.2 kB]
Get:13 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6712 B]
Get:14 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [7516 B]
Get:15 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-backports/main Translation-en [4764 B]
Get:16 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [7736 B]
Get:17 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic-backports/universe Translation-en [4588 B]
Get:18 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [809 kB]
Get:19 https://dev.monetdb.org/downloads/deb bionic InRelease [4457 B]
Get:20 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [254 kB]
Get:21 https://dev.monetdb.org/downloads/deb bionic/monetdb amd64 Packages [72.8 kB]
Get:22 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [689 kB]
Get:23 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [228 kB]
Get:24 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [8100 B]
Get:25 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2852 B]
Fetched 18.6 MB in 3s (6053 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
2 packages can be upgraded. Run 'apt list --upgradable' to see them.
ubuntu@ip-▉▉▉:~$ sudo apt install monetdb5-sql monetdb-client
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libmonetdb-client12 libmonetdb-stream13 libmonetdb20 monetdb5-server monetdb5-server-hugeint monetdb5-sql-hugeint
The following NEW packages will be installed:
libmonetdb-client12 libmonetdb-stream13 libmonetdb20 monetdb-client monetdb5-server monetdb5-server-hugeint monetdb5-sql monetdb5-sql-hugeint
0 upgraded, 8 newly installed, 0 to remove and 2 not upgraded.
Need to get 3315 kB of archives.
After this operation, 13.6 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 https://dev.monetdb.org/downloads/deb bionic/monetdb amd64 libmonetdb-client12 amd64 11.37.11 [104 kB]
Get:2 https://dev.monetdb.org/downloads/deb bionic/monetdb amd64 libmonetdb-stream13 amd64 11.37.11 [92.4 kB]
Get:3 https://dev.monetdb.org/downloads/deb bionic/monetdb amd64 libmonetdb20 amd64 11.37.11 [1229 kB]
Get:4 https://dev.monetdb.org/downloads/deb bionic/monetdb amd64 monetdb-client amd64 11.37.11 [149 kB]
Get:5 https://dev.monetdb.org/downloads/deb bionic/monetdb amd64 monetdb5-server amd64 11.37.11 [695 kB]
Get:6 https://dev.monetdb.org/downloads/deb bionic/monetdb amd64 monetdb5-server-hugeint amd64 11.37.11 [92.4 kB]
Get:7 https://dev.monetdb.org/downloads/deb bionic/monetdb amd64 monetdb5-sql amd64 11.37.11 [875 kB]
Get:8 https://dev.monetdb.org/downloads/deb bionic/monetdb amd64 monetdb5-sql-hugeint amd64 11.37.11 [77.7 kB]
Get:8 https://dev.monetdb.org/downloads/deb bionic/monetdb amd64 monetdb5-sql-hugeint amd64 11.37.11 [77.7 kB]
Fetched 3250 kB in 36s (90.3 kB/s)
Selecting previously unselected package libmonetdb-client12.
(Reading database ... 57084 files and directories currently installed.)
Preparing to unpack .../0-libmonetdb-client12_11.37.11_amd64.deb ...
Unpacking libmonetdb-client12 (11.37.11) ...
Selecting previously unselected package libmonetdb-stream13.
Preparing to unpack .../1-libmonetdb-stream13_11.37.11_amd64.deb ...
Unpacking libmonetdb-stream13 (11.37.11) ...
Selecting previously unselected package libmonetdb20.
Preparing to unpack .../2-libmonetdb20_11.37.11_amd64.deb ...
Unpacking libmonetdb20 (11.37.11) ...
Selecting previously unselected package monetdb-client.
Preparing to unpack .../3-monetdb-client_11.37.11_amd64.deb ...
Unpacking monetdb-client (11.37.11) ...
Selecting previously unselected package monetdb5-server.
Preparing to unpack .../4-monetdb5-server_11.37.11_amd64.deb ...
Unpacking monetdb5-server (11.37.11) ...
Selecting previously unselected package monetdb5-server-hugeint.
Preparing to unpack .../5-monetdb5-server-hugeint_11.37.11_amd64.deb ...
Unpacking monetdb5-server-hugeint (11.37.11) ...
Selecting previously unselected package monetdb5-sql.
Preparing to unpack .../6-monetdb5-sql_11.37.11_amd64.deb ...
Unpacking monetdb5-sql (11.37.11) ...
Selecting previously unselected package monetdb5-sql-hugeint.
Preparing to unpack .../7-monetdb5-sql-hugeint_11.37.11_amd64.deb ...
Unpacking monetdb5-sql-hugeint (11.37.11) ...
Setting up libmonetdb20 (11.37.11) ...
Setting up libmonetdb-client12 (11.37.11) ...
Setting up libmonetdb-stream13 (11.37.11) ...
Setting up monetdb-client (11.37.11) ...
Setting up monetdb5-server (11.37.11) ...
Adding group `monetdb' (GID 115) ...
Done.
Warning: The home dir /var/lib/monetdb you specified already exists.
Adding system user `monetdb' (UID 111) ...
Adding new user `monetdb' (UID 111) with group `monetdb' ...
The home directory `/var/lib/monetdb' already exists. Not copying from `/etc/skel'.
adduser: Warning: The home directory `/var/lib/monetdb' does not belong to the user you are currently creating.
Setting up monetdb5-sql (11.37.11) ...
Setting up monetdb5-server-hugeint (11.37.11) ...
Setting up monetdb5-sql-hugeint (11.37.11) ...
Processing triggers for systemd (237-3ubuntu10.42) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for ureadahead (0.100.0-21) ...
Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
ubuntu@ip-▉▉▉:~$ sudo systemctl enable monetdbd
Created symlink /etc/systemd/system/multi-user.target.wants/monetdbd.service → /lib/systemd/system/monetdbd.service.
ubuntu@ip-▉▉▉:~$ sudo systemctl start monetdbd
ubuntu@ip-▉▉▉:~$ sudo usermod -a -G monetdb $USER
ubuntu@ip-▉▉▉:~$ logout
Connection to ec2-▉▉▉.us-east-2.compute.amazonaws.com closed.
milovidov@milovidov-desktop:~$ ssh -i ~/.ssh/aws_milovidov.pem ubuntu@ec2-▉▉▉.us-east-2.compute.amazonaws.com
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 5.3.0-1032-aws x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Wed Aug 12 22:53:13 UTC 2020
System load: 0.31 Processes: 329
Usage of /: 50.1% of 193.82GB Users logged in: 1
Memory usage: 2% IP address for ens5: ▉▉▉
Swap usage: 0%
2 packages can be updated.
2 updates are security updates.
Last login: Wed Aug 12 22:11:01 2020 from 109.252.51.97
ubuntu@ip-▉▉▉:~$ monetdbd create /opt/monetdb
unable to create directory '/opt/monetdb': Permission denied
ubuntu@ip-▉▉▉:~$ sudo monetdbd create /opt/monetdb
ubuntu@ip-▉▉▉:~$ sudo monetdbd start /opt/monetdb
monetdbd: binding to stream socket port 50000 failed: Address already in use
ubuntu@ip-▉▉▉:~$ monetdb create test
monetdb: cannot connect: control socket does not exist
ubuntu@ip-▉▉▉:~$ sudo systemctl start monetdbd
ubuntu@ip-▉▉▉:~$ monetdb create test
monetdb: cannot connect: control socket does not exist
ubuntu@ip-▉▉▉:~$ sudo systemctl start monetdbd
ubuntu@ip-▉▉▉:~$ sudo monetdb create test
monetdb: cannot connect: control socket does not exist
ubuntu@ip-▉▉▉:~$ less /var/log/monetdb/merovingian.log
/var/log/monetdb/merovingian.log: Permission denied
ubuntu@ip-▉▉▉:~$ sudo less /var/log/monetdb/merovingian.log
ubuntu@ip-▉▉▉:~$ monetdb create test
monetdb: cannot connect: control socket does not exist
ubuntu@ip-▉▉▉:~$ sudo systemctl stop monetdbd
ubuntu@ip-▉▉▉:~$ sudo monetdbd start /opt/monetdb
ubuntu@ip-▉▉▉:~$ monetdb create test
monetdb: cannot connect: no permission to access control socket
ubuntu@ip-▉▉▉:~$ sudo monetdb create test
created database in maintenance mode: test
ubuntu@ip-▉▉▉:~$ sudo monetdb release test
taken database out of maintenance mode: test
ubuntu@ip-▉▉▉:~$ mclient -u monetdb -d test
password:
Welcome to mclient, the MonetDB/SQL interactive terminal (Jun2020-SP1)
Database: MonetDB v11.37.11 (Jun2020-SP1), 'mapi:monetdb://ip-▉▉▉:50000/test'
FOLLOW US on https://twitter.com/MonetDB or https://github.com/MonetDB/MonetDB
Type \q to quit, \? for a list of available commands
auto commit mode: on
sql>CREATE TABLE hits
more>(
more> "WatchID" BIGINT,
more> "JavaEnable" TINYINT,
more> "Title" TEXT,
more> "GoodEvent" SMALLINT,
more> "EventTime" TIMESTAMP,
more> "EventDate" Date,
more> "CounterID" INTEGER,
more> "ClientIP" INTEGER,
more> "RegionID" INTEGER,
more> "UserID" BIGINT,
more> "CounterClass" TINYINT,
more> "OS" TINYINT,
more> "UserAgent" TINYINT,
more> "URL" TEXT,
more> "Referer" TEXT,
more> "Refresh" TINYINT,
more> "RefererCategoryID" SMALLINT,
more> "RefererRegionID" INTEGER,
more> "URLCategoryID" SMALLINT,
more> "URLRegionID" INTEGER,
more> "ResolutionWidth" SMALLINT,
more> "ResolutionHeight" SMALLINT,
more> "ResolutionDepth" TINYINT,
more> "FlashMajor" TINYINT,
more> "FlashMinor" TINYINT,
more> "FlashMinor2" TEXT,
more> "NetMajor" TINYINT,
more> "NetMinor" TINYINT,
more> "UserAgentMajor" SMALLINT,
more> "UserAgentMinor" TEXT,
more> "CookieEnable" TINYINT,
more> "JavascriptEnable" TINYINT,
more> "IsMobile" TINYINT,
more> "MobilePhone" TINYINT,
more> "MobilePhoneModel" TEXT,
more> "Params" TEXT,
more> "IPNetworkID" INTEGER,
more> "TraficSourceID" TINYINT,
more> "SearchEngineID" SMALLINT,
more> "SearchPhrase" TEXT,
more> "AdvEngineID" TINYINT,
more> "IsArtifical" TINYINT,
more> "WindowClientWidth" SMALLINT,
more> "WindowClientHeight" SMALLINT,
more> "ClientTimeZone" SMALLINT,
more> "ClientEventTime" TIMESTAMP,
more> "SilverlightVersion1" TINYINT,
more> "SilverlightVersion2" TINYINT,
more> "SilverlightVersion3" INTEGER,
more> "SilverlightVersion4" SMALLINT,
more> "PageCharset" TEXT,
more> "CodeVersion" INTEGER,
more> "IsLink" TINYINT,
more> "IsDownload" TINYINT,
more> "IsNotBounce" TINYINT,
more> "FUniqID" BIGINT,
more> "OriginalURL" TEXT,
more> "HID" INTEGER,
more> "IsOldCounter" TINYINT,
more> "IsEvent" TINYINT,
more> "IsParameter" TINYINT,
more> "DontCountHits" TINYINT,
more> "WithHash" TINYINT,
more> "HitColor" TEXT,
more> "LocalEventTime" TIMESTAMP,
more> "Age" TINYINT,
more> "Sex" TINYINT,
more> "Income" TINYINT,
more> "Interests" SMALLINT,
more> "Robotness" TINYINT,
more> "RemoteIP" INTEGER,
more> "WindowName" INTEGER,
more> "OpenerName" INTEGER,
more> "HistoryLength" SMALLINT,
more> "BrowserLanguage" TEXT,
more> "BrowserCountry" TEXT,
more> "SocialNetwork" TEXT,
more> "SocialAction" TEXT,
more> "HTTPError" SMALLINT,
more> "SendTiming" INTEGER,
more> "DNSTiming" INTEGER,
more> "ConnectTiming" INTEGER,
more> "ResponseStartTiming" INTEGER,
more> "ResponseEndTiming" INTEGER,
more> "FetchTiming" INTEGER,
more> "SocialSourceNetworkID" TINYINT,
more> "SocialSourcePage" TEXT,
more> "ParamPrice" BIGINT,
more> "ParamOrderID" TEXT,
more> "ParamCurrency" TEXT,
more> "ParamCurrencyID" SMALLINT,
more> "OpenstatServiceName" TEXT,
more> "OpenstatCampaignID" TEXT,
more> "OpenstatAdID" TEXT,
more> "OpenstatSourceID" TEXT,
more> "UTMSource" TEXT,
more> "UTMMedium" TEXT,
more> "UTMCampaign" TEXT,
more> "UTMContent" TEXT,
more> "UTMTerm" TEXT,
more> "FromTag" TEXT,
more> "HasGCLID" TINYINT,
more> "RefererHash" BIGINT,
more> "URLHash" BIGINT,
more> "CLID" INTEGER
more>);
operation successful
sql>ubuntu@ip-▉▉▉:~$ mclient -u monetdb -d test
password:
Welcome to mclient, the MonetDB/SQL interactive terminal (Jun2020-SP1)
Database: MonetDB v11.37.11 (Jun2020-SP1), 'mapi:monetdb://ip-▉▉▉:50000/test'
FOLLOW US on https://twitter.com/MonetDB or https://github.com/MonetDB/MonetDB
Type \q to quit, \? for a list of available commands
auto commit mode: on
sql>COPY INTO hits FROM '/home/milovidov/example_datasets/hits_100m_obfuscated_monetdb.tsv' USING DELIMITERS '\t';
Cannot open file '/home/milovidov/example_datasets/hits_100m_obfuscated_monetdb.tsv': No such file or directory
sql>COPY INTO hits FROM '/home/ubuntu/hits_100m_obfuscated_monetdb.tsv' USING DELIMITERS '\t';
100000000 affected rows
sql>SELECT count(*) FROM hits;
+-----------+
| %1 |
+===========+
| 100000000 |
+-----------+
1 tuple
sql>CREATE INDEX hits_idx ON hits ("CounterID", "EventDate");
operation successful
sql>ubuntu@ip-▉▉▉:~$ wget https://raw.githubusercontent.com/ClickHouse/ClickHouse/master/benchmark/monetdb/{queries.sql,benchmark.sh,send-query}
--2020-08-12 23:28:53-- https://raw.githubusercontent.com/ClickHouse/ClickHouse/master/benchmark/monetdb/queries.sql
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.200.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.200.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8964 (8.8K) [text/plain]
Saving to: ‘queries.sql.1’
queries.sql.1 100%[========================================================================================================================>] 8.75K --.-KB/s in 0s
2020-08-12 23:28:58 (93.2 MB/s) - ‘queries.sql.1’ saved [8964/8964]
--2020-08-12 23:28:58-- https://raw.githubusercontent.com/ClickHouse/ClickHouse/master/benchmark/monetdb/benchmark.sh
Reusing existing connection to raw.githubusercontent.com:443.
HTTP request sent, awaiting response... 200 OK
Length: 285 [text/plain]
Saving to: ‘benchmark.sh’
benchmark.sh 100%[========================================================================================================================>] 285 --.-KB/s in 0s
2020-08-12 23:28:58 (26.2 MB/s) - ‘benchmark.sh’ saved [285/285]
--2020-08-12 23:28:58-- https://raw.githubusercontent.com/ClickHouse/ClickHouse/master/benchmark/monetdb/send-query
Reusing existing connection to raw.githubusercontent.com:443.
HTTP request sent, awaiting response... 200 OK
Length: 249 [text/plain]
Saving to: ‘send-query’
send-query 100%[========================================================================================================================>] 249 --.-KB/s in 0s
2020-08-12 23:28:58 (24.4 MB/s) - ‘send-query’ saved [249/249]
FINISHED --2020-08-12 23:28:58--
Total wall clock time: 5.4s
Downloaded: 3 files, 9.3K in 0s (81.0 MB/s)
ubuntu@ip-▉▉▉:~$ chmod a+x benchmark.sh send-query
ubuntu@ip-▉▉▉:~$ ./benchmark.sh | tee log.txt
3
SELECT count() FROM hits;
3
SELECT count() FROM hits WHERE AdvEngineID != 0;
3
SELECT sum(AdvEngineID), count(), avg(ResolutionWidth) FROM hits ;
3
SELECT sum(UserID) FROM hits ;
3
SELECT uniq(UserID) FROM hits ;
3
SELECT uniq(SearchPhrase) FROM hits ;
3
SELECT min(EventDate), max(EventDate) FROM hits ;
3
SELECT AdvEngineID, count() FROM hits WHERE AdvEngineID != 0 GROUP BY AdvEngineID ORDER BY count() DESC;
3
SELECT RegionID, uniq(UserID) AS u FROM hits GROUP BY RegionID ORDER BY u DESC LIMIT 10;
3
SELECT RegionID, sum(AdvEngineID), count() AS c, avg(ResolutionWidth), uniq(UserID) FROM hits GROUP BY RegionID ORDER BY c DESC LIMIT 10;
3
SELECT MobilePhoneModel, uniq(UserID) AS u FROM hits WHERE MobilePhoneModel != '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10;
3
SELECT MobilePhone, MobilePhoneModel, uniq(UserID) AS u FROM hits WHERE MobilePhoneModel != '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10;
3
SELECT SearchPhrase, count() AS c FROM hits WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
3
SELECT SearchPhrase, uniq(UserID) AS u FROM hits WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10;
3
SELECT SearchEngineID, SearchPhrase, count() AS c FROM hits WHERE SearchPhrase != '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10;
3
SELECT UserID, count() FROM hits GROUP BY UserID ORDER BY count() DESC LIMIT 10;
3
SELECT UserID, SearchPhrase, count() FROM hits GROUP BY UserID, SearchPhrase ORDER BY count() DESC LIMIT 10;
3
SELECT UserID, SearchPhrase, count() FROM hits GROUP BY UserID, SearchPhrase LIMIT 10;
3
SELECT UserID, toMinute(EventTime) AS m, SearchPhrase, count() FROM hits GROUP BY UserID, m, SearchPhrase ORDER BY count() DESC LIMIT 10;
3
SELECT UserID FROM hits WHERE UserID = 12345678901234567890;
3
SELECT count() FROM hits WHERE URL LIKE '%metrika%';
3
SELECT SearchPhrase, any(URL), count() AS c FROM hits WHERE URL LIKE '%metrika%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
3
SELECT SearchPhrase, any(URL), any(Title), count() AS c, uniq(UserID) FROM hits WHERE Title LIKE '%Яндекс%' AND URL NOT LIKE '%.yandex.%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
3
SELECT * FROM hits WHERE URL LIKE '%metrika%' ORDER BY EventTime LIMIT 10;
3
SELECT SearchPhrase FROM hits WHERE SearchPhrase != '' ORDER BY EventTime LIMIT 10;
3
SELECT SearchPhrase FROM hits WHERE SearchPhrase != '' ORDER BY SearchPhrase LIMIT 10;
3
SELECT SearchPhrase FROM hits WHERE SearchPhrase != '' ORDER BY EventTime, SearchPhrase LIMIT 10;
3
SELECT CounterID, avg(length(URL)) AS l, count() AS c FROM hits WHERE URL != '' GROUP BY CounterID HAVING c > 100000 ORDER BY l DESC LIMIT 25;
3
SELECT domainWithoutWWW(Referer) AS key, avg(length(Referer)) AS l, count() AS c, any(Referer) FROM hits WHERE Referer != '' GROUP BY key HAVING c > 100000 ORDER BY l DESC LIMIT 25;
3
SELECT sum(ResolutionWidth), sum(ResolutionWidth + 1), sum(ResolutionWidth + 2), sum(ResolutionWidth + 3), sum(ResolutionWidth + 4), sum(ResolutionWidth + 5), sum(ResolutionWidth + 6), sum(ResolutionWidth + 7), sum(ResolutionWidth + 8), sum(ResolutionWidth + 9), sum(ResolutionWidth + 10), sum(ResolutionWidth + 11), sum(ResolutionWidth + 12), sum(ResolutionWidth + 13), sum(ResolutionWidth + 14), sum(ResolutionWidth + 15), sum(ResolutionWidth + 16), sum(ResolutionWidth + 17), sum(ResolutionWidth + 18), sum(ResolutionWidth + 19), sum(ResolutionWidth + 20), sum(ResolutionWidth + 21), sum(ResolutionWidth + 22), sum(ResolutionWidth + 23), sum(ResolutionWidth + 24), sum(ResolutionWidth + 25), sum(ResolutionWidth + 26), sum(ResolutionWidth + 27), sum(ResolutionWidth + 28), sum(ResolutionWidth + 29), sum(ResolutionWidth + 30), sum(ResolutionWidth + 31), sum(ResolutionWidth + 32), sum(ResolutionWidth + 33), sum(ResolutionWidth + 34), sum(ResolutionWidth + 35), sum(ResolutionWidth + 36), sum(ResolutionWidth + 37), sum(ResolutionWidth + 38), sum(ResolutionWidth + 39), sum(ResolutionWidth + 40), sum(ResolutionWidth + 41), sum(ResolutionWidth + 42), sum(ResolutionWidth + 43), sum(ResolutionWidth + 44), sum(ResolutionWidth + 45), sum(ResolutionWidth + 46), sum(ResolutionWidth + 47), sum(ResolutionWidth + 48), sum(ResolutionWidth + 49), sum(ResolutionWidth + 50), sum(ResolutionWidth + 51), sum(ResolutionWidth + 52), sum(ResolutionWidth + 53), sum(ResolutionWidth + 54), sum(ResolutionWidth + 55), sum(ResolutionWidth + 56), sum(ResolutionWidth + 57), sum(ResolutionWidth + 58), sum(ResolutionWidth + 59), sum(ResolutionWidth + 60), sum(ResolutionWidth + 61), sum(ResolutionWidth + 62), sum(ResolutionWidth + 63), sum(ResolutionWidth + 64), sum(ResolutionWidth + 65), sum(ResolutionWidth + 66), sum(ResolutionWidth + 67), sum(ResolutionWidth + 68), sum(ResolutionWidth + 69), sum(ResolutionWidth + 70), sum(ResolutionWidth + 71), sum(ResolutionWidth + 72), sum(ResolutionWidth + 73), sum(ResolutionWidth + 74), sum(ResolutionWidth + 75), sum(ResolutionWidth + 76), sum(ResolutionWidth + 77), sum(ResolutionWidth + 78), sum(ResolutionWidth + 79), sum(ResolutionWidth + 80), sum(ResolutionWidth + 81), sum(ResolutionWidth + 82), sum(ResolutionWidth + 83), sum(ResolutionWidth + 84), sum(ResolutionWidth + 85), sum(ResolutionWidth + 86), sum(ResolutionWidth + 87), sum(ResolutionWidth + 88), sum(ResolutionWidth + 89) FROM hits;
3
SELECT SearchEngineID, ClientIP, count() AS c, sum(Refresh), avg(ResolutionWidth) FROM hits WHERE SearchPhrase != '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10;
3
SELECT WatchID, ClientIP, count() AS c, sum(Refresh), avg(ResolutionWidth) FROM hits WHERE SearchPhrase != '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
3
SELECT WatchID, ClientIP, count() AS c, sum(Refresh), avg(ResolutionWidth) FROM hits GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
3
SELECT URL, count() AS c FROM hits GROUP BY URL ORDER BY c DESC LIMIT 10;
3
SELECT 1, URL, count() AS c FROM hits GROUP BY 1, URL ORDER BY c DESC LIMIT 10;
3
SELECT ClientIP AS x, x - 1, x - 2, x - 3, count() AS c FROM hits GROUP BY x, x - 1, x - 2, x - 3 ORDER BY c DESC LIMIT 10;
3
SELECT URL, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT DontCountHits AND NOT Refresh AND notEmpty(URL) GROUP BY URL ORDER BY PageViews DESC LIMIT 10;
3
SELECT Title, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT DontCountHits AND NOT Refresh AND notEmpty(Title) GROUP BY Title ORDER BY PageViews DESC LIMIT 10;
3
SELECT URL, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh AND IsLink AND NOT IsDownload GROUP BY URL ORDER BY PageViews DESC LIMIT 1000;
3
SELECT TraficSourceID, SearchEngineID, AdvEngineID, ((SearchEngineID = 0 AND AdvEngineID = 0) ? Referer : '') AS Src, URL AS Dst, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, Src, Dst ORDER BY PageViews DESC LIMIT 1000;
3
SELECT URLHash, EventDate, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh AND TraficSourceID IN (-1, 6) AND RefererHash = halfMD5('http://example.ru/') GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 100;
3
SELECT WindowClientWidth, WindowClientHeight, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh AND NOT DontCountHits AND URLHash = halfMD5('http://example.ru/') GROUP BY WindowClientWidth, WindowClientHeight ORDER BY PageViews DESC LIMIT 10000;
3
SELECT toStartOfMinute(EventTime) AS Minute, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-02' AND NOT Refresh AND NOT DontCountHits GROUP BY Minute ORDER BY Minute;
ubuntu@ip-▉▉▉:~$ mclient -u monetdb -d test
password:
Welcome to mclient, the MonetDB/SQL interactive terminal (Jun2020-SP1)
Database: MonetDB v11.37.11 (Jun2020-SP1), 'mapi:monetdb://ip-▉▉▉:50000/test'
FOLLOW US on https://twitter.com/MonetDB or https://github.com/MonetDB/MonetDB
Type \q to quit, \? for a list of available commands
auto commit mode: on
sql>SELECT COUNT(DISTINCT "SearchPhrase") FROM hits;
+---------+
| %1 |
+=========+
| 6019589 |
+---------+
1 tuple
sql>ubuntu@ip-▉▉▉:~$ expect
Command 'expect' not found, but can be installed with:
sudo snap install expect # version 5.45-7snap0, or
sudo apt install expect
See 'snap info expect' for additional versions.
ubuntu@ip-▉▉▉:~$ sudo apt install expect
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libtcl8.6 tcl-expect tcl8.6
Suggested packages:
tk8.6 tcl-tclreadline
The following NEW packages will be installed:
expect libtcl8.6 tcl-expect tcl8.6
0 upgraded, 4 newly installed, 0 to remove and 2 not upgraded.
Need to get 1138 kB of archives.
After this operation, 4598 kB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic/main amd64 libtcl8.6 amd64 8.6.8+dfsg-3 [881 kB]
Get:2 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic/universe amd64 tcl-expect amd64 5.45.4-1 [105 kB]
Get:3 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic/universe amd64 expect amd64 5.45.4-1 [137 kB]
Get:4 http://us-east-2.ec2.archive.ubuntu.com/ubuntu bionic/main amd64 tcl8.6 amd64 8.6.8+dfsg-3 [14.4 kB]
Fetched 1138 kB in 0s (31.4 MB/s)
Selecting previously unselected package libtcl8.6:amd64.
(Reading database ... 57274 files and directories currently installed.)
Preparing to unpack .../libtcl8.6_8.6.8+dfsg-3_amd64.deb ...
Unpacking libtcl8.6:amd64 (8.6.8+dfsg-3) ...
Selecting previously unselected package tcl-expect:amd64.
Preparing to unpack .../tcl-expect_5.45.4-1_amd64.deb ...
Unpacking tcl-expect:amd64 (5.45.4-1) ...
Selecting previously unselected package expect.
Preparing to unpack .../expect_5.45.4-1_amd64.deb ...
Unpacking expect (5.45.4-1) ...
Selecting previously unselected package tcl8.6.
Preparing to unpack .../tcl8.6_8.6.8+dfsg-3_amd64.deb ...
Unpacking tcl8.6 (8.6.8+dfsg-3) ...
Setting up libtcl8.6:amd64 (8.6.8+dfsg-3) ...
Setting up tcl-expect:amd64 (5.45.4-1) ...
Setting up tcl8.6 (8.6.8+dfsg-3) ...
Setting up expect (5.45.4-1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
^[[Aubuntu@ip-▉▉▉:~$ ./benchmark.sh | tee log.txt
3
SELECT count() FROM hits;
clk: 0.299 ms
clk: 0.380 ms
clk: 0.471 ms
3
SELECT count() FROM hits WHERE AdvEngineID != 0;
clk: 0.297 ms
clk: 0.342 ms
clk: 0.363 ms
3
SELECT sum(AdvEngineID), count(), avg(ResolutionWidth) FROM hits ;
SELECT: identifier 'advengineid' unknown
clk: 0.485 ms
SELECT: identifier 'advengineid' unknown
clk: 0.228 ms
SELECT: identifier 'advengineid' unknown
clk: 0.233 ms
3
SELECT sum(UserID) FROM hits ;
SELECT: identifier 'userid' unknown
clk: 0.239 ms
SELECT: identifier 'userid' unknown
clk: 0.492 ms
SELECT: identifier 'userid' unknown
clk: 0.388 ms
3
SELECT uniq(UserID) FROM hits ;
SELECT: identifier 'userid' unknown
clk: 0.267 ms
SELECT: identifier 'userid' unknown
clk: 0.222 ms
SELECT: identifier 'userid' unknown
clk: 0.212 ms
3
SELECT uniq(SearchPhrase) FROM hits ;
SELECT: identifier 'searchphrase' unknown
clk: 0.498 ms
SELECT: identifier 'searchphrase' unknown
clk: 0.255 ms
SELECT: identifier 'searchphrase' unknown
clk: 0.224 ms
3
SELECT min(EventDate), max(EventDate) FROM hits ;
SELECT: identifier 'eventdate' unknown
clk: 0.272 ms
SELECT: identifier 'eventdate' unknown
clk: 0.205 ms
SELECT: identifier 'eventdate' unknown
clk: 0.209 ms
3
SELECT AdvEngineID, count() FROM hits WHERE AdvEngineID != 0 GROUP BY AdvEngineID ORDER BY count() DESC;
clk: 0.439 ms
clk: 0.185 ms
clk: 0.235 ms
3
SELECT RegionID, uniq(UserID) AS u FROM hits GROUP BY RegionID ORDER BY u DESC LIMIT 10;
SELECT: identifier 'regionid' unknown
clk: 0.283 ms
SELECT: identifier 'regionid' unknown
clk: 0.207 ms
SELECT: identifier 'regionid' unknown
clk: 0.212 ms
3
SELECT RegionID, sum(AdvEngineID), count() AS c, avg(ResolutionWidth), uniq(UserID) FROM hits GROUP BY RegionID ORDER BY c DESC LIMIT 10;
SELECT: identifier 'regionid' unknown
clk: 0.271 ms
SELECT: identifier 'regionid' unknown
clk: 0.231 ms
SELECT: identifier 'regionid' unknown
clk: 0.227 ms
3
SELECT MobilePhoneModel, uniq(UserID) AS u FROM hits WHERE MobilePhoneModel != '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10;
clk: 0.256 ms
clk: 0.229 ms
clk: 0.223 ms
3
SELECT MobilePhone, MobilePhoneModel, uniq(UserID) AS u FROM hits WHERE MobilePhoneModel != '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10;
clk: 0.244 ms
clk: 0.201 ms
clk: 0.234 ms
3
SELECT SearchPhrase, count() AS c FROM hits WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
clk: 0.223 ms
clk: 0.210 ms
clk: 0.207 ms
3
SELECT SearchPhrase, uniq(UserID) AS u FROM hits WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10;
clk: 0.220 ms
clk: 0.231 ms
clk: 0.189 ms
3
SELECT SearchEngineID, SearchPhrase, count() AS c FROM hits WHERE SearchPhrase != '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10;
clk: 0.238 ms
clk: 0.226 ms
clk: 0.221 ms
3
SELECT UserID, count() FROM hits GROUP BY UserID ORDER BY count() DESC LIMIT 10;
SELECT: identifier 'userid' unknown
clk: 0.216 ms
SELECT: identifier 'userid' unknown
clk: 0.214 ms
SELECT: identifier 'userid' unknown
clk: 0.465 ms
3
SELECT UserID, SearchPhrase, count() FROM hits GROUP BY UserID, SearchPhrase ORDER BY count() DESC LIMIT 10;
SELECT: identifier 'userid' unknown
clk: 0.250 ms
SELECT: identifier 'userid' unknown
clk: 0.304 ms
SELECT: identifier 'userid' unknown
clk: 0.239 ms
3
SELECT UserID, SearchPhrase, count() FROM hits GROUP BY UserID, SearchPhrase LIMIT 10;
SELECT: identifier 'userid' unknown
clk: 0.203 ms
SELECT: identifier 'userid' unknown
clk: 0.209 ms
SELECT: identifier 'userid' unknown
clk: 0.221 ms
3
SELECT UserID, toMinute(EventTime) AS m, SearchPhrase, count() FROM hits GROUP BY UserID, m, SearchPhrase ORDER BY count() DESC LIMIT 10;
SELECT: identifier 'userid' unknown
clk: 0.238 ms
SELECT: identifier 'userid' unknown
clk: 0.249 ms
SELECT: identifier 'userid' unknown
clk: 0.258 ms
3
SELECT UserID FROM hits WHERE UserID = 12345678901234567890;
SELECT: identifier 'userid' unknown
clk: 0.204 ms
SELECT: identifier 'userid' unknown
clk: 0.187 ms
SELECT: identifier 'userid' unknown
clk: 0.195 ms
3
SELECT count() FROM hits WHERE URL LIKE '%metrika%';
SELECT: identifier 'url' unknown
clk: 0.575 ms
SELECT: identifier 'url' unknown
clk: 0.191 ms
SELECT: identifier 'url' unknown
clk: 0.191 ms
3
SELECT SearchPhrase, any(URL), count() AS c FROM hits WHERE URL LIKE '%metrika%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
syntax error, unexpected ANY in: "select searchphrase, any"
clk: 0.411 ms
syntax error, unexpected ANY in: "select searchphrase, any"
clk: 0.233 ms
syntax error, unexpected ANY in: "select searchphrase, any"
clk: 0.208 ms
3
SELECT SearchPhrase, any(URL), any(Title), count() AS c, uniq(UserID) FROM hits WHERE Title LIKE '%Яндекс%' AND URL NOT LIKE '%.yandex.%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
syntax error, unexpected ANY in: "select searchphrase, any"
clk: 0.266 ms
syntax error, unexpected ANY in: "select searchphrase, any"
clk: 0.238 ms
syntax error, unexpected ANY in: "select searchphrase, any"
clk: 0.311 ms
3
SELECT * FROM hits WHERE URL LIKE '%metrika%' ORDER BY EventTime LIMIT 10;
SELECT: identifier 'url' unknown
clk: 0.221 ms
SELECT: identifier 'url' unknown
clk: 0.205 ms
SELECT: identifier 'url' unknown
clk: 0.201 ms
3
SELECT SearchPhrase FROM hits WHERE SearchPhrase != '' ORDER BY EventTime LIMIT 10;
clk: 0.183 ms
clk: 0.539 ms
clk: 0.183 ms
3
SELECT SearchPhrase FROM hits WHERE SearchPhrase != '' ORDER BY SearchPhrase LIMIT 10;
clk: 0.227 ms
clk: 0.187 ms
clk: 0.201 ms
3
SELECT SearchPhrase FROM hits WHERE SearchPhrase != '' ORDER BY EventTime, SearchPhrase LIMIT 10;
clk: 0.214 ms
clk: 0.192 ms
clk: 0.230 ms
3
SELECT CounterID, avg(length(URL)) AS l, count() AS c FROM hits WHERE URL != '' GROUP BY CounterID HAVING c > 100000 ORDER BY l DESC LIMIT 25;
clk: 0.245 ms
clk: 0.244 ms
clk: 0.216 ms
3
SELECT domainWithoutWWW(Referer) AS key, avg(length(Referer)) AS l, count() AS c, any(Referer) FROM hits WHERE Referer != '' GROUP BY key HAVING c > 100000 ORDER BY l DESC LIMIT 25;
syntax error, unexpected ANY in: "select domainwithoutwww(referer) as key, avg(length(referer)) as l, count() as c"
clk: 0.415 ms
syntax error, unexpected ANY in: "select domainwithoutwww(referer) as key, avg(length(referer)) as l, count() as c"
clk: 0.221 ms
syntax error, unexpected ANY in: "select domainwithoutwww(referer) as key, avg(length(referer)) as l, count() as c"
clk: 0.232 ms
3
SELECT sum(ResolutionWidth), sum(ResolutionWidth + 1), sum(ResolutionWidth + 2), sum(ResolutionWidth + 3), sum(ResolutionWidth + 4), sum(ResolutionWidth + 5), sum(ResolutionWidth + 6), sum(ResolutionWidth + 7), sum(ResolutionWidth + 8), sum(ResolutionWidth + 9), sum(ResolutionWidth + 10), sum(ResolutionWidth + 11), sum(ResolutionWidth + 12), sum(ResolutionWidth + 13), sum(ResolutionWidth + 14), sum(ResolutionWidth + 15), sum(ResolutionWidth + 16), sum(ResolutionWidth + 17), sum(ResolutionWidth + 18), sum(ResolutionWidth + 19), sum(ResolutionWidth + 20), sum(ResolutionWidth + 21), sum(ResolutionWidth + 22), sum(ResolutionWidth + 23), sum(ResolutionWidth + 24), sum(ResolutionWidth + 25), sum(ResolutionWidth + 26), sum(ResolutionWidth + 27), sum(ResolutionWidth + 28), sum(ResolutionWidth + 29), sum(ResolutionWidth + 30), sum(ResolutionWidth + 31), sum(ResolutionWidth + 32), sum(ResolutionWidth + 33), sum(ResolutionWidth + 34), sum(ResolutionWidth + 35), sum(ResolutionWidth + 36), sum(ResolutionWidth + 37), sum(ResolutionWidth + 38), sum(ResolutionWidth + 39), sum(ResolutionWidth + 40), sum(ResolutionWidth + 41), sum(ResolutionWidth + 42), sum(ResolutionWidth + 43), sum(ResolutionWidth + 44), sum(ResolutionWidth + 45), sum(ResolutionWidth + 46), sum(ResolutionWidth + 47), sum(ResolutionWidth + 48), sum(ResolutionWidth + 49), sum(ResolutionWidth + 50), sum(ResolutionWidth + 51), sum(ResolutionWidth + 52), sum(ResolutionWidth + 53), sum(ResolutionWidth + 54), sum(ResolutionWidth + 55), sum(ResolutionWidth + 56), sum(ResolutionWidth + 57), sum(ResolutionWidth + 58), sum(ResolutionWidth + 59), sum(ResolutionWidth + 60), sum(ResolutionWidth + 61), sum(ResolutionWidth + 62), sum(ResolutionWidth + 63), sum(ResolutionWidth + 64), sum(ResolutionWidth + 65), sum(ResolutionWidth + 66), sum(ResolutionWidth + 67), sum(ResolutionWidth + 68), sum(ResolutionWidth + 69), sum(ResolutionWidth + 70), sum(ResolutionWidth + 71), sum(ResolutionWidth + 72), sum(ResolutionWidth + 73), sum(ResolutionWidth + 74), sum(ResolutionWidth + 75), sum(ResolutionWidth + 76), sum(ResolutionWidth + 77), sum(ResolutionWidth + 78), sum(ResolutionWidth + 79), sum(ResolutionWidth + 80), sum(ResolutionWidth + 81), sum(ResolutionWidth + 82), sum(ResolutionWidth + 83), sum(ResolutionWidth + 84), sum(ResolutionWidth + 85), sum(ResolutionWidth + 86), sum(ResolutionWidth + 87), sum(ResolutionWidth + 88), sum(ResolutionWidth + 89) FROM hits;
SELECT: identifier 'resolutionwidth' unknown
clk: 0.869 ms
SELECT: identifier 'resolutionwidth' unknown
clk: 0.705 ms
SELECT: identifier 'resolutionwidth' unknown
clk: 0.780 ms
3
SELECT SearchEngineID, ClientIP, count() AS c, sum(Refresh), avg(ResolutionWidth) FROM hits WHERE SearchPhrase != '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10;
clk: 0.251 ms
clk: 0.239 ms
clk: 0.231 ms
3
SELECT WatchID, ClientIP, count() AS c, sum(Refresh), avg(ResolutionWidth) FROM hits WHERE SearchPhrase != '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
clk: 0.237 ms
clk: 0.234 ms
clk: 0.209 ms
3
SELECT WatchID, ClientIP, count() AS c, sum(Refresh), avg(ResolutionWidth) FROM hits GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
SELECT: identifier 'watchid' unknown
clk: 0.252 ms
SELECT: identifier 'watchid' unknown
clk: 0.261 ms
SELECT: identifier 'watchid' unknown
clk: 0.226 ms
3
SELECT URL, count() AS c FROM hits GROUP BY URL ORDER BY c DESC LIMIT 10;
SELECT: identifier 'url' unknown
clk: 0.459 ms
SELECT: identifier 'url' unknown
clk: 0.195 ms
SELECT: identifier 'url' unknown
clk: 0.206 ms
3
SELECT 1, URL, count() AS c FROM hits GROUP BY 1, URL ORDER BY c DESC LIMIT 10;
SELECT: identifier 'url' unknown
clk: 0.254 ms
SELECT: identifier 'url' unknown
clk: 0.204 ms
SELECT: identifier 'url' unknown
clk: 0.238 ms
3
SELECT ClientIP AS x, x - 1, x - 2, x - 3, count() AS c FROM hits GROUP BY x, x - 1, x - 2, x - 3 ORDER BY c DESC LIMIT 10;
SELECT: identifier 'clientip' unknown
clk: 0.275 ms
SELECT: identifier 'clientip' unknown
clk: 0.254 ms
SELECT: identifier 'clientip' unknown
clk: 0.230 ms
3
SELECT URL, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT DontCountHits AND NOT Refresh AND notEmpty(URL) GROUP BY URL ORDER BY PageViews DESC LIMIT 10;
SELECT: identifier 'counterid' unknown
clk: 0.446 ms
SELECT: identifier 'counterid' unknown
clk: 0.276 ms
SELECT: identifier 'counterid' unknown
clk: 0.255 ms
3
SELECT Title, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT DontCountHits AND NOT Refresh AND notEmpty(Title) GROUP BY Title ORDER BY PageViews DESC LIMIT 10;
SELECT: identifier 'counterid' unknown
clk: 0.248 ms
SELECT: identifier 'counterid' unknown
clk: 0.237 ms
SELECT: identifier 'counterid' unknown
clk: 0.294 ms
3
SELECT URL, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh AND IsLink AND NOT IsDownload GROUP BY URL ORDER BY PageViews DESC LIMIT 1000;
SELECT: identifier 'counterid' unknown
clk: 0.302 ms
SELECT: identifier 'counterid' unknown
clk: 0.234 ms
SELECT: identifier 'counterid' unknown
clk: 0.274 ms
3
SELECT TraficSourceID, SearchEngineID, AdvEngineID, ((SearchEngineID = 0 AND AdvEngineID = 0) ? Referer : '') AS Src, URL AS Dst, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, Src, Dst ORDER BY PageViews DESC LIMIT 1000;
syntax error, unexpected '?', expecting ')' or OR in: "select traficsourceid, searchengineid, advengineid, ((searchengineid = 0 and adv"
clk: 0.446 ms
syntax error, unexpected '?', expecting ')' or OR in: "select traficsourceid, searchengineid, advengineid, ((searchengineid = 0 and adv"
clk: 0.239 ms
syntax error, unexpected '?', expecting ')' or OR in: "select traficsourceid, searchengineid, advengineid, ((searchengineid = 0 and adv"
clk: 0.232 ms
3
SELECT URLHash, EventDate, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh AND TraficSourceID IN (-1, 6) AND RefererHash = halfMD5('http://example.ru/') GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 100;
SELECT: identifier 'counterid' unknown
clk: 0.259 ms
SELECT: identifier 'counterid' unknown
clk: 0.263 ms
SELECT: identifier 'counterid' unknown
clk: 0.267 ms
3
SELECT WindowClientWidth, WindowClientHeight, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh AND NOT DontCountHits AND URLHash = halfMD5('http://example.ru/') GROUP BY WindowClientWidth, WindowClientHeight ORDER BY PageViews DESC LIMIT 10000;
SELECT: identifier 'counterid' unknown
clk: 0.253 ms
SELECT: identifier 'counterid' unknown
clk: 0.281 ms
SELECT: identifier 'counterid' unknown
clk: 0.556 ms
3
SELECT toStartOfMinute(EventTime) AS Minute, count() AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-02' AND NOT Refresh AND NOT DontCountHits GROUP BY Minute ORDER BY Minute;
syntax error, unexpected MINUTE, expecting SCOLON in: "select tostartofminute(eventtime) as minute"
clk: 0.239 ms
syntax error, unexpected MINUTE, expecting SCOLON in: "select tostartofminute(eventtime) as minute"
clk: 0.228 ms
syntax error, unexpected MINUTE, expecting SCOLON in: "select tostartofminute(eventtime) as minute"
clk: 0.223 ms
ubuntu@ip-▉▉▉:~$ less queries.sql
ubuntu@ip-▉▉▉:~$ rm queries.sql
ubuntu@ip-▉▉▉:~$ mv queries.sql.1 queries.sql
ubuntu@ip-▉▉▉:~$ ./benchmark.sh | tee log.txt
3
SELECT count(*) FROM hits;
1 tuple
clk: 2.346 ms
1 tuple
clk: 1.873 ms
1 tuple
clk: 1.862 ms
3
SELECT count(*) FROM hits WHERE "AdvEngineID" <> 0;
1 tuple
clk: 1.137 sec
1 tuple
clk: 36.303 ms
1 tuple
clk: 35.399 ms
3
SELECT sum("AdvEngineID"), count(*), avg("ResolutionWidth") FROM hits;
1 tuple
clk: 194.816 ms
1 tuple
clk: 54.757 ms
1 tuple
clk: 51.177 ms
3
SELECT sum("UserID") FROM hits;
overflow in calculation.
clk: 7.495 ms
overflow in calculation.
clk: 2.967 ms
overflow in calculation.
clk: 3.073 ms
3
SELECT COUNT(DISTINCT "UserID") FROM hits;
1 tuple
clk: 5.556 sec
1 tuple
clk: 3.550 sec
1 tuple
clk: 3.797 sec
3
SELECT COUNT(DISTINCT "SearchPhrase") FROM hits;
1 tuple
clk: 6.228 sec
1 tuple
clk: 4.042 sec
1 tuple
clk: 3.967 sec
3
SELECT min("EventDate"), max("EventDate") FROM hits;
1 tuple
clk: 67.704 ms
1 tuple
clk: 50.072 ms
1 tuple
clk: 51.354 ms
3
SELECT "AdvEngineID", count(*) FROM hits WHERE "AdvEngineID" <> 0 GROUP BY "AdvEngineID" ORDER BY count(*) DESC;
18 tuples
clk: 93.070 ms
18 tuples
clk: 37.996 ms
18 tuples
clk: 31.543 ms
3
SELECT "RegionID", COUNT(DISTINCT "UserID") AS u FROM hits GROUP BY "RegionID" ORDER BY u DESC LIMIT 10;
10 tuples
clk: 8.111 sec
10 tuples
clk: 4.839 sec
10 tuples
clk: 5.105 sec
3
SELECT "RegionID", sum("AdvEngineID"), count(*) AS c, avg("ResolutionWidth"), COUNT(DISTINCT "UserID") FROM hits GROUP BY "RegionID" ORDER BY c DESC LIMIT 10;
10 tuples
clk: 11.571 sec
10 tuples
clk: 7.454 sec
10 tuples
clk: 7.433 sec
3
SELECT "MobilePhoneModel", COUNT(DISTINCT "UserID") AS u FROM hits WHERE "MobilePhoneModel" <> '' GROUP BY "MobilePhoneModel" ORDER BY u DESC LIMIT 10;
10 tuples
clk: 3.080 sec
10 tuples
clk: 387.757 ms
10 tuples
clk: 348.083 ms
3
SELECT "MobilePhone", "MobilePhoneModel", COUNT(DISTINCT "UserID") AS u FROM hits WHERE "MobilePhoneModel" <> '' GROUP BY "MobilePhone", "MobilePhoneModel" ORDER BY u DESC LIMIT 10;
10 tuples
clk: 3.456 sec
10 tuples
clk: 438.178 ms
10 tuples
clk: 393.357 ms
3
SELECT "SearchPhrase", count(*) AS c FROM hits WHERE "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY c DESC LIMIT 10;
10 tuples
clk: 8.111 sec
10 tuples
clk: 5.833 sec
10 tuples
clk: 5.808 sec
3
SELECT "SearchPhrase", COUNT(DISTINCT "UserID") AS u FROM hits WHERE "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY u DESC LIMIT 10;
10 tuples
clk: 16.087 sec
10 tuples
clk: 10.259 sec
10 tuples
clk: 10.159 sec
3
SELECT "SearchEngineID", "SearchPhrase", count(*) AS c FROM hits WHERE "SearchPhrase" <> '' GROUP BY "SearchEngineID", "SearchPhrase" ORDER BY c DESC LIMIT 10;
10 tuples
clk: 9.149 sec
10 tuples
clk: 6.234 sec
10 tuples
clk: 6.197 sec
3
SELECT "UserID", count(*) FROM hits GROUP BY "UserID" ORDER BY count(*) DESC LIMIT 10;
10 tuples
clk: 5.809 sec
10 tuples
clk: 4.814 sec
10 tuples
clk: 4.413 sec
3
SELECT "UserID", "SearchPhrase", count(*) FROM hits GROUP BY "UserID", "SearchPhrase" ORDER BY count(*) DESC LIMIT 10;
10 tuples
clk: 15.222 sec
10 tuples
clk: 11.985 sec
10 tuples
clk: 10.959 sec
3
SELECT "UserID", "SearchPhrase", count(*) FROM hits GROUP BY "UserID", "SearchPhrase" LIMIT 10;
10 tuples
clk: 16.143 sec
10 tuples
clk: 10.903 sec
10 tuples
clk: 11.900 sec
3
SELECT "UserID", extract(minute FROM "EventTime") AS m, "SearchPhrase", count(*) FROM hits GROUP BY "UserID", m, "SearchPhrase" ORDER BY count(*) DESC LIMIT 10;
10 tuples
clk: 28.492 sec
10 tuples
clk: 22.938 sec
10 tuples
clk: 20.756 sec
3
SELECT "UserID" FROM hits WHERE "UserID" = -6101065172474983726;
0 tuples
clk: 5.792 sec
0 tuples
clk: 1.764 sec
0 tuples
clk: 0.574 ms
3
SELECT count(*) FROM hits WHERE "URL" LIKE '%metrika%';
1 tuple
clk: 22.995 sec
1 tuple
clk: 1.745 sec
1 tuple
clk: 1.597 sec
3
SELECT "SearchPhrase", min("URL"), count(*) AS c FROM hits WHERE "URL" LIKE '%metrika%' AND "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY c DESC LIMIT 10;
10 tuples
clk: 44.995 sec
10 tuples
clk: 14.185 sec
10 tuples
clk: 15.712 sec
3
SELECT "SearchPhrase", min("URL"), min("Title"), count(*) AS c, COUNT(DISTINCT "UserID") FROM hits WHERE "Title" LIKE '%Яндекс%' AND "URL" NOT LIKE '%.yandex.%' AND "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY c DESC LIMIT 10;
10 tuples
clk: 44.542 sec
10 tuples
clk: 1.850 sec
10 tuples
clk: 1.552 sec
3
SELECT * FROM hits WHERE "URL" LIKE '%metrika%' ORDER BY "EventTime" LIMIT 10;
10 tuples !85 columns dropped, 29 fields truncated!
clk: 29.023 sec
10 tuples !85 columns dropped, 29 fields truncated!
clk: 1.696 sec
10 tuples !85 columns dropped, 29 fields truncated!
clk: 1.459 sec
3
SELECT "SearchPhrase" FROM hits WHERE "SearchPhrase" <> '' ORDER BY "EventTime" LIMIT 10;
10 tuples
clk: 5.979 sec
10 tuples
clk: 319.889 ms
10 tuples
clk: 288.515 ms
3
SELECT "SearchPhrase" FROM hits WHERE "SearchPhrase" <> '' ORDER BY "SearchPhrase" LIMIT 10;
10 tuples
clk: 3.500 sec
10 tuples
clk: 860.218 ms
10 tuples
clk: 852.812 ms
3
SELECT "SearchPhrase" FROM hits WHERE "SearchPhrase" <> '' ORDER BY "EventTime", "SearchPhrase" LIMIT 10;
10 tuples
clk: 6.540 sec
10 tuples
clk: 853.858 ms
10 tuples
clk: 765.395 ms
3
SELECT "CounterID", avg(length("URL")) AS l, count(*) AS c FROM hits WHERE "URL" <> '' GROUP BY "CounterID" HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;
25 tuples
clk: 24.407 sec
25 tuples
clk: 2.111 sec
25 tuples
clk: 2.085 sec
3
SELECT sys.getdomain("Referer") AS key, avg(length("Referer")) AS l, count(*) AS c, min("Referer") FROM hits WHERE "Referer" <> '' GROUP BY key HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;
clk: 16.458 sec
clk: 7.022 sec
clk: 8.084 sec
3
SELECT sum("ResolutionWidth"), sum("ResolutionWidth" + 1), sum("ResolutionWidth" + 2), sum("ResolutionWidth" + 3), sum("ResolutionWidth" + 4), sum("ResolutionWidth" + 5), sum("ResolutionWidth" + 6), sum("ResolutionWidth" + 7), sum("ResolutionWidth" + 8), sum("ResolutionWidth" + 9), sum("ResolutionWidth" + 10), sum("ResolutionWidth" + 11), sum("ResolutionWidth" + 12), sum("ResolutionWidth" + 13), sum("ResolutionWidth" + 14), sum("ResolutionWidth" + 15), sum("ResolutionWidth" + 16), sum("ResolutionWidth" + 17), sum("ResolutionWidth" + 18), sum("ResolutionWidth" + 19), sum("ResolutionWidth" + 20), sum("ResolutionWidth" + 21), sum("ResolutionWidth" + 22), sum("ResolutionWidth" + 23), sum("ResolutionWidth" + 24), sum("ResolutionWidth" + 25), sum("ResolutionWidth" + 26), sum("ResolutionWidth" + 27), sum("ResolutionWidth" + 28), sum("ResolutionWidth" + 29), sum("ResolutionWidth" + 30), sum("ResolutionWidth" + 31), sum("ResolutionWidth" + 32), sum("ResolutionWidth" + 33), sum("ResolutionWidth" + 34), sum("ResolutionWidth" + 35), sum("ResolutionWidth" + 36), sum("ResolutionWidth" + 37), sum("ResolutionWidth" + 38), sum("ResolutionWidth" + 39), sum("ResolutionWidth" + 40), sum("ResolutionWidth" + 41), sum("ResolutionWidth" + 42), sum("ResolutionWidth" + 43), sum("ResolutionWidth" + 44), sum("ResolutionWidth" + 45), sum("ResolutionWidth" + 46), sum("ResolutionWidth" + 47), sum("ResolutionWidth" + 48), sum("ResolutionWidth" + 49), sum("ResolutionWidth" + 50), sum("ResolutionWidth" + 51), sum("ResolutionWidth" + 52), sum("ResolutionWidth" + 53), sum("ResolutionWidth" + 54), sum("ResolutionWidth" + 55), sum("ResolutionWidth" + 56), sum("ResolutionWidth" + 57), sum("ResolutionWidth" + 58), sum("ResolutionWidth" + 59), sum("ResolutionWidth" + 60), sum("ResolutionWidth" + 61), sum("ResolutionWidth" + 62), sum("ResolutionWidth" + 63), sum("ResolutionWidth" + 64), sum("ResolutionWidth" + 65), sum("ResolutionWidth" + 66), sum("ResolutionWidth" + 67), sum("ResolutionWidth" + 68), sum("ResolutionWidth" + 69), sum("ResolutionWidth" + 70), sum("ResolutionWidth" + 71), sum("ResolutionWidth" + 72), sum("ResolutionWidth" + 73), sum("ResolutionWidth" + 74), sum("ResolutionWidth" + 75), sum("ResolutionWidth" + 76), sum("ResolutionWidth" + 77), sum("ResolutionWidth" + 78), sum("ResolutionWidth" + 79), sum("ResolutionWidth" + 80), sum("ResolutionWidth" + 81), sum("ResolutionWidth" + 82), sum("ResolutionWidth" + 83), sum("ResolutionWidth" + 84), sum("ResolutionWidth" + 85), sum("ResolutionWidth" + 86), sum("ResolutionWidth" + 87), sum("ResolutionWidth" + 88), sum("ResolutionWidth" + 89) FROM hits;
1 tuple !76 columns dropped!
clk: 3.026 sec
1 tuple !76 columns dropped!
clk: 2.862 sec
1 tuple !76 columns dropped!
clk: 2.846 sec
3
SELECT "SearchEngineID", "ClientIP", count(*) AS c, sum("Refresh"), avg("ResolutionWidth") FROM hits WHERE "SearchPhrase" <> '' GROUP BY "SearchEngineID", "ClientIP" ORDER BY c DESC LIMIT 10;
10 tuples
clk: 9.145 sec
10 tuples
clk: 3.225 sec
10 tuples
clk: 3.061 sec
3
SELECT "WatchID", "ClientIP", count(*) AS c, sum("Refresh"), avg("ResolutionWidth") FROM hits WHERE "SearchPhrase" <> '' GROUP BY "WatchID", "ClientIP" ORDER BY c DESC LIMIT 10;
10 tuples
clk: 12.443 sec
10 tuples
clk: 4.298 sec
10 tuples
clk: 4.308 sec
3
SELECT "WatchID", "ClientIP", count(*) AS c, sum("Refresh"), avg("ResolutionWidth") FROM hits GROUP BY "WatchID", "ClientIP" ORDER BY c DESC LIMIT 10;
10 tuples
clk: 28.515 sec
10 tuples
clk: 27.195 sec
10 tuples
clk: 27.687 sec
3
SELECT "URL", count(*) AS c FROM hits GROUP BY "URL" ORDER BY c DESC LIMIT 10;
10 tuples
clk: 42.507 sec
10 tuples
clk: 22.997 sec
10 tuples
clk: 21.922 sec
3
SELECT 1, "URL", count(*) AS c FROM hits GROUP BY 1, "URL" ORDER BY c DESC LIMIT 10;
10 tuples
clk: 42.009 sec
10 tuples
clk: 22.347 sec
10 tuples
clk: 21.288 sec
3
SELECT "ClientIP", "ClientIP" - 1, "ClientIP" - 2, "ClientIP" - 3, count(*) AS c FROM hits GROUP BY "ClientIP", "ClientIP" - 1, "ClientIP" - 2, "ClientIP" - 3 ORDER BY c DESC LIMIT 10;
10 tuples
clk: 16.540 sec
10 tuples
clk: 14.959 sec
10 tuples
clk: 14.100 sec
3
SELECT "URL", count(*) AS "PageViews" FROM hits WHERE "CounterID" = 62 AND "EventDate" >= '2013-07-01' AND "EventDate" <= '2013-07-31' AND "DontCountHits" = 0 AND "Refresh" = 0 AND "URL" <> '' GROUP BY "URL" ORDER BY "PageViews" DESC LIMIT 10;
10 tuples
clk: 2:08 min
10 tuples
clk: 24.004 sec
10 tuples
clk: 23.890 sec
3
SELECT "Title", count(*) AS "PageViews" FROM hits WHERE "CounterID" = 62 AND "EventDate" >= '2013-07-01' AND "EventDate" <= '2013-07-31' AND "DontCountHits" = 0 AND "Refresh" = 0 AND "Title" <> '' GROUP BY "Title" ORDER BY "PageViews" DESC LIMIT 10;
10 tuples
clk: 1.893 sec
10 tuples
clk: 245.259 ms
10 tuples
clk: 226.849 ms
3
SELECT "URL", count(*) AS "PageViews" FROM hits WHERE "CounterID" = 62 AND "EventDate" >= '2013-07-01' AND "EventDate" <= '2013-07-31' AND "Refresh" = 0 AND "IsLink" <> 0 AND "IsDownload" = 0 GROUP BY "URL" ORDER BY "PageViews" DESC LIMIT 1000;
1000 tuples
clk: 2:09 min
1000 tuples
clk: 22.917 sec
1000 tuples
clk: 23.099 sec
3
SELECT "TraficSourceID", "SearchEngineID", "AdvEngineID", CASE WHEN ("SearchEngineID" = 0 AND "AdvEngineID" = 0) THEN "Referer" ELSE '' END AS Src, "URL" AS Dst, count(*) AS "PageViews" FROM hits WHERE "CounterID" = 62 AND "EventDate" >= '2013-07-01' AND "EventDate" <= '2013-07-31' AND "Refresh" = 0 GROUP BY "TraficSourceID", "SearchEngineID", "AdvEngineID", CASE WHEN ("SearchEngineID" = 0 AND "AdvEngineID" = 0) THEN "Referer" ELSE '' END, "URL" ORDER BY "PageViews" DESC LIMIT 1000;
1000 tuples
clk: 2:09 min
1000 tuples
clk: 23.975 sec
1000 tuples
clk: 22.763 sec
3
SELECT "URLHash", "EventDate", count(*) AS "PageViews" FROM hits WHERE "CounterID" = 62 AND "EventDate" >= '2013-07-01' AND "EventDate" <= '2013-07-31' AND "Refresh" = 0 AND "TraficSourceID" IN (-1, 6) AND "RefererHash" = 686716256552154761 GROUP BY "URLHash", "EventDate" ORDER BY "PageViews" DESC LIMIT 100;
0 tuples
clk: 1.148 sec
0 tuples
clk: 114.740 ms
0 tuples
clk: 114.258 ms
3
SELECT "WindowClientWidth", "WindowClientHeight", count(*) AS "PageViews" FROM hits WHERE "CounterID" = 62 AND "EventDate" >= '2013-07-01' AND "EventDate" <= '2013-07-31' AND "Refresh" = 0 AND "DontCountHits" = 0 AND "URLHash" = 686716256552154761 GROUP BY "WindowClientWidth", "WindowClientHeight" ORDER BY "PageViews" DESC LIMIT 10000;
0 tuples
clk: 2.079 sec
0 tuples
clk: 235.220 ms
0 tuples
clk: 163.850 ms
3
SELECT DATE_TRUNC('minute', "EventTime") AS "Minute", count(*) AS "PageViews" FROM hits WHERE "CounterID" = 62 AND "EventDate" >= '2013-07-01' AND "EventDate" <= '2013-07-02' AND "Refresh" = 0 AND "DontCountHits" = 0 GROUP BY DATE_TRUNC('minute', "EventTime") ORDER BY DATE_TRUNC('minute', "EventTime");
0 tuples
clk: 1.690 sec
0 tuples
clk: 148.379 ms
0 tuples
clk: 166.565 ms
ubuntu@ip-▉▉▉:~$ grep clk log.txt | tr -d '\r' | awk '{ if ($3 == "ms") { print $2 / 1000; } else if ($3 == "sec") { print $2 } else { print } }' > tmp.txt
ubuntu@ip-▉▉▉:~$ nano tmp.txt
ubuntu@ip-▉▉▉:~$ awk '{
> if (i % 3 == 0) { a = $1 }
> else if (i % 3 == 1) { b = $1 }
> else if (i % 3 == 2) { c = $1; print "[" a ", " b ", " c "]," };
> ++i; }' < tmp.txt
[0.002346, 0.001873, 0.001862],
[1.137, 0.036303, 0.035399],
[0.194816, 0.054757, 0.051177],
[0.007495, 0.002967, 0.003073],
[5.556, 3.550, 3.797],
[6.228, 4.042, 3.967],
[0.067704, 0.050072, 0.051354],
[0.09307, 0.037996, 0.031543],
[8.111, 4.839, 5.105],
[11.571, 7.454, 7.433],
[3.080, 0.387757, 0.348083],
[3.456, 0.438178, 0.393357],
[8.111, 5.833, 5.808],
[16.087, 10.259, 10.159],
[9.149, 6.234, 6.197],
[5.809, 4.814, 4.413],
[15.222, 11.985, 10.959],
[16.143, 10.903, 11.900],
[28.492, 22.938, 20.756],
[5.792, 1.764, 0.000574],
[22.995, 1.745, 1.597],
[44.995, 14.185, 15.712],
[44.542, 1.850, 1.552],
[29.023, 1.696, 1.459],
[5.979, 0.319889, 0.288515],
[3.500, 0.860218, 0.852812],
[6.540, 0.853858, 0.765395],
[24.407, 2.111, 2.085],
[16.458, 7.022, 8.084],
[3.026, 2.862, 2.846],
[9.145, 3.225, 3.061],
[12.443, 4.298, 4.308],
[28.515, 27.195, 27.687],
[42.507, 22.997, 21.922],
[42.009, 22.347, 21.288],
[16.540, 14.959, 14.100],
[128, 24.004, 23.890],
[1.893, 0.245259, 0.226849],
[129, 22.917, 23.099],
[129, 23.975, 22.763],
[1.148, 0.11474, 0.114258],
[2.079, 0.23522, 0.16385],
[1.690, 0.148379, 0.166565],
ubuntu@ip-▉▉▉:~$
Reference in New Issue Copy Permalink