mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-17 13:13:36 +00:00
371ecb4fe6
* Remove machine_translated * Add 'For non-Linux operating systems and for AArch64 CPU' section * Translate getting-started/install.md to ja * Fix index.md anchors * Translate index.md via GitLocalize * Translate index.md via GitLocalize * Translate getting-started/playground.md to ja * Translate getting-started/index.md to ja * Translate experiment to 試す in ja * Translate getting-started/tutoial.md to ja * Translate getting-started/tutoial.md to ja * Translate getting-started/example-datasets to ja * Fix original article path * Fix getting-started/index.md * Revert ja/index.md * Fix Input/Output to 入出力 * Fix Original argicle en * Fix missing links md files. `getting-started/sql-reference/statements/optimize.md` is not placed in ja directory. Currentry, it refer to english one. * Change link to optimize.md copied from en * Change link to alter.md ja
1.3 KiB
1.3 KiB
| toc_priority | toc_title |
|---|---|
| 18 | WikiStat |
WikiStat
参照: http://dumps.wikimedia.org/other/pagecounts-raw/
テーブルの作成:
CREATE TABLE wikistat
(
date Date,
time DateTime,
project String,
subproject String,
path String,
hits UInt64,
size UInt64
) ENGINE = MergeTree(date, (path, time), 8192);
データのロード:
$ for i in {2007..2016}; do for j in {01..12}; do echo $i-$j >&2; curl -sSL "http://dumps.wikimedia.org/other/pagecounts-raw/$i/$i-$j/" | grep -oE 'pagecounts-[0-9]+-[0-9]+\.gz'; done; done | sort | uniq | tee links.txt
$ cat links.txt | while read link; do wget http://dumps.wikimedia.org/other/pagecounts-raw/$(echo $link | sed -r 's/pagecounts-([0-9]{4})([0-9]{2})[0-9]{2}-[0-9]+\.gz/\1/')/$(echo $link | sed -r 's/pagecounts-([0-9]{4})([0-9]{2})[0-9]{2}-[0-9]+\.gz/\1-\2/')/$link; done
$ ls -1 /opt/wikistat/ | grep gz | while read i; do echo $i; gzip -cd /opt/wikistat/$i | ./wikistat-loader --time="$(echo -n $i | sed -r 's/pagecounts-([0-9]{4})([0-9]{2})([0-9]{2})-([0-9]{2})([0-9]{2})([0-9]{2})\.gz/\1-\2-\3 \4-00-00/')" | clickhouse-client --query="INSERT INTO wikistat FORMAT TabSeparated"; done