ClickHouse/benchmark/hive/log/log_100m_tuned

Not found: /etc/init.d/service with path=
start time: Вт. сент. 17 22:55:26 MSK 2013

times: 1
query: SELECT count(*) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_18854@mturlrep13_201309172255_1961113475.txt
hive> SELECT count(*) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0004
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 22:56:16,852 Stage-1 map = 0%,  reduce = 0%
2013-09-17 22:56:37,989 Stage-1 map = 1%,  reduce = 0%
2013-09-17 22:56:41,077 Stage-1 map = 6%,  reduce = 0%
2013-09-17 22:56:43,088 Stage-1 map = 10%,  reduce = 0%
2013-09-17 22:56:44,095 Stage-1 map = 14%,  reduce = 0%
2013-09-17 22:56:47,115 Stage-1 map = 16%,  reduce = 0%
2013-09-17 22:56:48,121 Stage-1 map = 17%,  reduce = 0%
2013-09-17 22:56:50,131 Stage-1 map = 24%,  reduce = 0%
2013-09-17 22:56:51,144 Stage-1 map = 25%,  reduce = 0%
2013-09-17 22:56:53,160 Stage-1 map = 30%,  reduce = 0%
2013-09-17 22:56:54,167 Stage-1 map = 32%,  reduce = 0%
2013-09-17 22:56:55,173 Stage-1 map = 33%,  reduce = 0%
2013-09-17 22:56:56,179 Stage-1 map = 36%,  reduce = 0%
2013-09-17 22:56:57,185 Stage-1 map = 37%,  reduce = 0%
2013-09-17 22:56:58,199 Stage-1 map = 39%,  reduce = 0%
2013-09-17 22:57:00,209 Stage-1 map = 45%,  reduce = 0%
2013-09-17 22:57:01,214 Stage-1 map = 46%,  reduce = 0%
2013-09-17 22:57:03,223 Stage-1 map = 51%,  reduce = 0%
2013-09-17 22:57:04,230 Stage-1 map = 52%,  reduce = 0%
2013-09-17 22:57:05,693 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 36.54 sec
2013-09-17 22:57:06,700 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 36.54 sec
2013-09-17 22:57:07,707 Stage-1 map = 58%,  reduce = 0%, Cumulative CPU 36.54 sec
2013-09-17 22:57:08,725 Stage-1 map = 58%,  reduce = 0%, Cumulative CPU 36.54 sec
2013-09-17 22:57:09,731 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 36.54 sec
2013-09-17 22:57:10,948 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 36.54 sec
2013-09-17 22:57:11,958 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 36.54 sec
2013-09-17 22:57:12,970 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 36.54 sec
2013-09-17 22:57:13,990 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 1483.14 sec
2013-09-17 22:57:15,025 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 1484.59 sec
2013-09-17 22:57:16,048 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 1484.59 sec
2013-09-17 22:57:17,060 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 1484.59 sec
2013-09-17 22:57:18,095 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 1502.09 sec
2013-09-17 22:57:19,123 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 1512.83 sec
2013-09-17 22:57:20,133 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 1530.9 sec
2013-09-17 22:57:21,147 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 1580.07 sec
2013-09-17 22:57:22,153 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 1640.62 sec
2013-09-17 22:57:23,159 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 1677.56 sec
2013-09-17 22:57:24,166 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 1722.04 sec
2013-09-17 22:57:25,267 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 1733.88 sec
2013-09-17 22:57:26,274 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1760.11 sec
2013-09-17 22:57:27,280 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1772.82 sec
2013-09-17 22:57:28,286 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1786.49 sec
2013-09-17 22:57:29,291 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1786.49 sec
2013-09-17 22:57:30,297 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1786.49 sec
2013-09-17 22:57:31,302 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1786.49 sec
2013-09-17 22:57:32,307 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1786.49 sec
2013-09-17 22:57:33,313 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1786.49 sec
2013-09-17 22:57:34,319 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1786.49 sec
2013-09-17 22:57:35,337 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1789.45 sec
2013-09-17 22:57:36,344 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1789.45 sec
2013-09-17 22:57:37,350 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1789.45 sec
MapReduce Total cumulative CPU time: 29 minutes 49 seconds 450 msec
Ended Job = job_201309172235_0004
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 1789.45 sec   HDFS Read: 10081441390 HDFS Write: 10 SUCCESS
Total MapReduce CPU Time Spent: 29 minutes 49 seconds 450 msec
OK
100000000
Time taken: 110.676 seconds, Fetched: 1 row(s)
hive> quit;

times: 2
query: SELECT count(*) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_23942@mturlrep13_201309172257_1656900452.txt
hive> SELECT count(*) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0005
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 22:58:08,971 Stage-1 map = 0%,  reduce = 0%
2013-09-17 22:58:29,310 Stage-1 map = 1%,  reduce = 0%
2013-09-17 22:58:32,327 Stage-1 map = 4%,  reduce = 0%
2013-09-17 22:58:33,338 Stage-1 map = 5%,  reduce = 0%
2013-09-17 22:58:34,345 Stage-1 map = 6%,  reduce = 0%
2013-09-17 22:58:35,354 Stage-1 map = 8%,  reduce = 0%
2013-09-17 22:58:36,360 Stage-1 map = 13%,  reduce = 0%
2013-09-17 22:58:37,376 Stage-1 map = 15%,  reduce = 0%
2013-09-17 22:58:39,389 Stage-1 map = 17%,  reduce = 0%
2013-09-17 22:58:40,408 Stage-1 map = 18%,  reduce = 0%
2013-09-17 22:58:41,420 Stage-1 map = 20%,  reduce = 0%
2013-09-17 22:58:42,432 Stage-1 map = 25%,  reduce = 0%
2013-09-17 22:58:43,444 Stage-1 map = 27%,  reduce = 0%
2013-09-17 22:58:44,461 Stage-1 map = 29%,  reduce = 0%
2013-09-17 22:58:45,468 Stage-1 map = 31%,  reduce = 0%
2013-09-17 22:58:46,488 Stage-1 map = 33%,  reduce = 0%
2013-09-17 22:58:48,499 Stage-1 map = 36%,  reduce = 0%
2013-09-17 22:58:49,507 Stage-1 map = 38%,  reduce = 0%
2013-09-17 22:58:50,513 Stage-1 map = 41%,  reduce = 0%
2013-09-17 22:58:51,519 Stage-1 map = 42%,  reduce = 0%
2013-09-17 22:58:52,524 Stage-1 map = 45%,  reduce = 0%
2013-09-17 22:58:53,529 Stage-1 map = 46%,  reduce = 0%
2013-09-17 22:58:54,535 Stage-1 map = 48%,  reduce = 0%
2013-09-17 22:58:55,575 Stage-1 map = 51%,  reduce = 0%
2013-09-17 22:58:56,602 Stage-1 map = 53%,  reduce = 0%
2013-09-17 22:58:57,639 Stage-1 map = 54%,  reduce = 0%
2013-09-17 22:58:58,645 Stage-1 map = 57%,  reduce = 0%
2013-09-17 22:58:59,664 Stage-1 map = 59%,  reduce = 0%
2013-09-17 22:59:01,690 Stage-1 map = 65%,  reduce = 0%
2013-09-17 22:59:02,722 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 36.1 sec
2013-09-17 22:59:03,750 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 36.1 sec
2013-09-17 22:59:04,761 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 36.1 sec
2013-09-17 22:59:05,770 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 36.1 sec
2013-09-17 22:59:06,784 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 36.1 sec
2013-09-17 22:59:08,117 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 175.85 sec
2013-09-17 22:59:09,124 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 221.51 sec
2013-09-17 22:59:10,130 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 314.14 sec
2013-09-17 22:59:11,138 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 650.01 sec
2013-09-17 22:59:12,145 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 788.39 sec
2013-09-17 22:59:13,149 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 1123.31 sec
2013-09-17 22:59:14,155 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 1744.89 sec
2013-09-17 22:59:15,161 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 1753.27 sec
2013-09-17 22:59:16,167 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1760.31 sec
2013-09-17 22:59:17,174 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1765.77 sec
2013-09-17 22:59:18,179 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1773.76 sec
2013-09-17 22:59:19,185 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1773.76 sec
2013-09-17 22:59:20,191 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1773.76 sec
2013-09-17 22:59:21,196 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1773.76 sec
2013-09-17 22:59:22,202 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1773.76 sec
2013-09-17 22:59:23,215 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1773.76 sec
2013-09-17 22:59:24,221 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1773.76 sec
2013-09-17 22:59:25,226 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1773.76 sec
2013-09-17 22:59:26,234 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1776.78 sec
2013-09-17 22:59:27,552 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1776.78 sec
MapReduce Total cumulative CPU time: 29 minutes 36 seconds 780 msec
Ended Job = job_201309172235_0005
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 1776.78 sec   HDFS Read: 10081441390 HDFS Write: 10 SUCCESS
Total MapReduce CPU Time Spent: 29 minutes 36 seconds 780 msec
OK
100000000
Time taken: 105.13 seconds, Fetched: 1 row(s)
hive> quit;

times: 3
query: SELECT count(*) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_28935@mturlrep13_201309172259_1582946547.txt
hive> SELECT count(*) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0006
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 22:59:59,626 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:00:21,889 Stage-1 map = 1%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:22,898 Stage-1 map = 1%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:24,158 Stage-1 map = 5%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:25,172 Stage-1 map = 5%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:26,201 Stage-1 map = 6%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:27,208 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:28,215 Stage-1 map = 13%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:29,226 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:30,236 Stage-1 map = 18%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:31,983 Stage-1 map = 18%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:32,998 Stage-1 map = 22%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:34,970 Stage-1 map = 24%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:35,996 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:37,017 Stage-1 map = 29%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:38,034 Stage-1 map = 30%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:39,050 Stage-1 map = 31%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:40,078 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:41,092 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:42,097 Stage-1 map = 39%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:43,104 Stage-1 map = 41%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:44,109 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:45,115 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:46,124 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:47,148 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:48,154 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:49,162 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:50,172 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 106.84 sec
2013-09-17 23:00:51,207 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 140.29 sec
2013-09-17 23:00:52,224 Stage-1 map = 62%,  reduce = 0%, Cumulative CPU 140.29 sec
2013-09-17 23:00:53,230 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 140.29 sec
2013-09-17 23:00:54,237 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 140.29 sec
2013-09-17 23:00:55,247 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 140.29 sec
2013-09-17 23:00:56,254 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 140.29 sec
2013-09-17 23:00:57,261 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 140.29 sec
2013-09-17 23:00:58,268 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 178.19 sec
2013-09-17 23:00:59,275 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 178.19 sec
2013-09-17 23:01:00,282 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 178.19 sec
2013-09-17 23:01:01,288 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 309.14 sec
2013-09-17 23:01:02,448 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 570.2 sec
2013-09-17 23:01:03,547 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 708.2 sec
2013-09-17 23:01:04,555 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 845.2 sec
2013-09-17 23:01:05,562 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 982.42 sec
2013-09-17 23:01:06,568 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 1312.62 sec
2013-09-17 23:01:07,574 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 1599.05 sec
2013-09-17 23:01:08,580 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 1599.05 sec
2013-09-17 23:01:09,585 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1697.63 sec
2013-09-17 23:01:10,590 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1749.84 sec
2013-09-17 23:01:11,595 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1749.84 sec
2013-09-17 23:01:12,601 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1802.81 sec
2013-09-17 23:01:13,606 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1802.81 sec
2013-09-17 23:01:14,611 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1803.33 sec
2013-09-17 23:01:15,617 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1803.33 sec
2013-09-17 23:01:16,624 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1803.33 sec
2013-09-17 23:01:17,632 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1805.73 sec
2013-09-17 23:01:18,638 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1805.73 sec
2013-09-17 23:01:19,644 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1805.73 sec
MapReduce Total cumulative CPU time: 30 minutes 5 seconds 730 msec
Ended Job = job_201309172235_0006
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 1805.73 sec   HDFS Read: 10081441390 HDFS Write: 10 SUCCESS
Total MapReduce CPU Time Spent: 30 minutes 5 seconds 730 msec
OK
100000000
Time taken: 107.358 seconds, Fetched: 1 row(s)
hive> quit;

times: 1
query: SELECT count(*) FROM hits_100m WHERE AdvEngineID != 0;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_2345@mturlrep13_201309172301_1950916943.txt
hive> SELECT count(*) FROM hits_100m WHERE AdvEngineID != 0;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0007
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:02:11,138 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:02:22,387 Stage-1 map = 4%,  reduce = 0%
2013-09-17 23:02:23,396 Stage-1 map = 11%,  reduce = 0%
2013-09-17 23:02:24,402 Stage-1 map = 30%,  reduce = 0%
2013-09-17 23:02:25,700 Stage-1 map = 35%,  reduce = 0%
2013-09-17 23:02:26,723 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 46.24 sec
2013-09-17 23:02:27,736 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 173.19 sec
2013-09-17 23:02:28,746 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.36 sec
2013-09-17 23:02:29,751 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.36 sec
2013-09-17 23:02:30,756 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.36 sec
2013-09-17 23:02:31,761 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.36 sec
2013-09-17 23:02:32,768 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.36 sec
2013-09-17 23:02:33,774 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.36 sec
2013-09-17 23:02:34,943 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.36 sec
2013-09-17 23:02:35,950 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.36 sec
2013-09-17 23:02:36,959 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 297.32 sec
2013-09-17 23:02:37,967 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 297.32 sec
MapReduce Total cumulative CPU time: 4 minutes 57 seconds 320 msec
Ended Job = job_201309172235_0007
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 297.32 sec   HDFS Read: 6282387 HDFS Write: 7 SUCCESS
Total MapReduce CPU Time Spent: 4 minutes 57 seconds 320 msec
OK
630535
Time taken: 55.195 seconds, Fetched: 1 row(s)
hive> quit;

times: 2
query: SELECT count(*) FROM hits_100m WHERE AdvEngineID != 0;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_7709@mturlrep13_201309172302_964147901.txt
hive> SELECT count(*) FROM hits_100m WHERE AdvEngineID != 0;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0008
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:02:52,920 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:03:03,995 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:03:05,016 Stage-1 map = 3%,  reduce = 0%
2013-09-17 23:03:06,076 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 6.7 sec
2013-09-17 23:03:07,088 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 210.4 sec
2013-09-17 23:03:08,096 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.76 sec
2013-09-17 23:03:09,102 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.76 sec
2013-09-17 23:03:10,109 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.76 sec
2013-09-17 23:03:11,114 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.76 sec
2013-09-17 23:03:12,127 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.76 sec
2013-09-17 23:03:13,133 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 294.76 sec
2013-09-17 23:03:14,139 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 294.76 sec
2013-09-17 23:03:15,146 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 295.45 sec
2013-09-17 23:03:16,154 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 297.51 sec
2013-09-17 23:03:17,160 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 297.51 sec
MapReduce Total cumulative CPU time: 4 minutes 57 seconds 510 msec
Ended Job = job_201309172235_0008
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 297.51 sec   HDFS Read: 6282387 HDFS Write: 7 SUCCESS
Total MapReduce CPU Time Spent: 4 minutes 57 seconds 510 msec
OK
630535
Time taken: 36.435 seconds, Fetched: 1 row(s)
hive> quit;

times: 3
query: SELECT count(*) FROM hits_100m WHERE AdvEngineID != 0;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_12665@mturlrep13_201309172303_2013045725.txt
hive> SELECT count(*) FROM hits_100m WHERE AdvEngineID != 0;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0009
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:03:27,027 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:03:37,093 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:03:40,143 Stage-1 map = 41%,  reduce = 0%
2013-09-17 23:03:41,161 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 171.9 sec
2013-09-17 23:03:42,169 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 298.73 sec
2013-09-17 23:03:43,177 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 298.73 sec
2013-09-17 23:03:44,184 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 298.73 sec
2013-09-17 23:03:45,190 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 298.73 sec
2013-09-17 23:03:46,196 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 298.73 sec
2013-09-17 23:03:47,202 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 298.73 sec
2013-09-17 23:03:48,208 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 298.73 sec
2013-09-17 23:03:49,214 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 298.73 sec
2013-09-17 23:03:50,223 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 301.9 sec
2013-09-17 23:03:51,229 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 301.9 sec
MapReduce Total cumulative CPU time: 5 minutes 1 seconds 900 msec
Ended Job = job_201309172235_0009
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 301.9 sec   HDFS Read: 6282387 HDFS Write: 7 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 1 seconds 900 msec
OK
630535
Time taken: 32.201 seconds, Fetched: 1 row(s)
hive> quit;

times: 1
query: SELECT sum(AdvEngineID), count(*), avg(ResolutionWidth) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_17646@mturlrep13_201309172303_809195796.txt
hive> SELECT sum(AdvEngineID), count(*), avg(ResolutionWidth) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0010
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:04:11,044 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:04:30,217 Stage-1 map = 21%,  reduce = 0%, Cumulative CPU 29.16 sec
2013-09-17 23:04:31,225 Stage-1 map = 35%,  reduce = 0%, Cumulative CPU 90.03 sec
2013-09-17 23:04:32,231 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 230.36 sec
2013-09-17 23:04:33,237 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 349.12 sec
2013-09-17 23:04:34,243 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 391.27 sec
2013-09-17 23:04:35,249 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 391.27 sec
2013-09-17 23:04:36,255 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 391.27 sec
2013-09-17 23:04:37,261 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 391.27 sec
2013-09-17 23:04:38,267 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 391.27 sec
2013-09-17 23:04:39,276 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 393.94 sec
2013-09-17 23:04:40,283 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 393.94 sec
MapReduce Total cumulative CPU time: 6 minutes 33 seconds 940 msec
Ended Job = job_201309172235_0010
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 393.94 sec   HDFS Read: 78754487 HDFS Write: 32 SUCCESS
Total MapReduce CPU Time Spent: 6 minutes 33 seconds 940 msec
OK
Time taken: 39.991 seconds, Fetched: 1 row(s)
hive> quit;

times: 2
query: SELECT sum(AdvEngineID), count(*), avg(ResolutionWidth) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_22907@mturlrep13_201309172304_403247887.txt
hive> SELECT sum(AdvEngineID), count(*), avg(ResolutionWidth) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0011
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:04:50,128 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:05:02,260 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:05:03,278 Stage-1 map = 13%,  reduce = 0%
2013-09-17 23:05:04,299 Stage-1 map = 25%,  reduce = 0%
2013-09-17 23:05:05,308 Stage-1 map = 30%,  reduce = 0%
2013-09-17 23:05:06,333 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 9.21 sec
2013-09-17 23:05:07,352 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 39.3 sec
2013-09-17 23:05:08,369 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 377.82 sec
2013-09-17 23:05:09,375 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 401.7 sec
2013-09-17 23:05:10,381 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 401.7 sec
2013-09-17 23:05:11,387 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 401.7 sec
2013-09-17 23:05:12,393 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 401.7 sec
2013-09-17 23:05:13,399 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 401.7 sec
2013-09-17 23:05:14,405 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 401.7 sec
2013-09-17 23:05:15,411 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 401.7 sec
2013-09-17 23:05:16,419 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 404.35 sec
2013-09-17 23:05:17,426 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 404.35 sec
MapReduce Total cumulative CPU time: 6 minutes 44 seconds 350 msec
Ended Job = job_201309172235_0011
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 404.35 sec   HDFS Read: 78754487 HDFS Write: 32 SUCCESS
Total MapReduce CPU Time Spent: 6 minutes 44 seconds 350 msec
OK
Time taken: 35.143 seconds, Fetched: 1 row(s)
hive> quit;

times: 3
query: SELECT sum(AdvEngineID), count(*), avg(ResolutionWidth) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_28807@mturlrep13_201309172305_1471136727.txt
hive> SELECT sum(AdvEngineID), count(*), avg(ResolutionWidth) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0012
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:05:28,152 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:05:40,255 Stage-1 map = 7%,  reduce = 0%
2013-09-17 23:05:41,260 Stage-1 map = 22%,  reduce = 0%
2013-09-17 23:05:42,266 Stage-1 map = 28%,  reduce = 0%
2013-09-17 23:05:43,279 Stage-1 map = 39%,  reduce = 0%, Cumulative CPU 9.21 sec
2013-09-17 23:05:44,286 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 90.45 sec
2013-09-17 23:05:45,294 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 363.6 sec
2013-09-17 23:05:46,301 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 399.15 sec
2013-09-17 23:05:47,307 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 399.15 sec
2013-09-17 23:05:48,312 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 399.15 sec
2013-09-17 23:05:49,317 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 399.15 sec
2013-09-17 23:05:50,323 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 399.15 sec
2013-09-17 23:05:51,329 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 399.15 sec
2013-09-17 23:05:52,334 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 399.15 sec
2013-09-17 23:05:53,343 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 402.09 sec
2013-09-17 23:05:54,348 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 402.09 sec
MapReduce Total cumulative CPU time: 6 minutes 42 seconds 90 msec
Ended Job = job_201309172235_0012
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 402.09 sec   HDFS Read: 78754487 HDFS Write: 32 SUCCESS
Total MapReduce CPU Time Spent: 6 minutes 42 seconds 90 msec
OK
Time taken: 35.085 seconds, Fetched: 1 row(s)
hive> quit;

times: 1
query: SELECT sum(UserID) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_1699@mturlrep13_201309172306_1224643663.txt
hive> SELECT sum(UserID) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0013
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:06:12,215 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:06:22,307 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:06:24,332 Stage-1 map = 5%,  reduce = 0%
2013-09-17 23:06:26,810 Stage-1 map = 15%,  reduce = 0%
2013-09-17 23:06:27,822 Stage-1 map = 17%,  reduce = 0%, Cumulative CPU 7.28 sec
2013-09-17 23:06:28,829 Stage-1 map = 17%,  reduce = 0%, Cumulative CPU 7.28 sec
2013-09-17 23:06:29,837 Stage-1 map = 17%,  reduce = 0%, Cumulative CPU 7.28 sec
2013-09-17 23:06:32,109 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 25.3 sec
2013-09-17 23:06:33,115 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 25.3 sec
2013-09-17 23:06:35,953 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 34.74 sec
2013-09-17 23:06:36,962 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 352.67 sec
2013-09-17 23:06:37,968 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 352.67 sec
2013-09-17 23:06:38,974 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 352.67 sec
2013-09-17 23:06:39,979 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 352.67 sec
2013-09-17 23:06:40,985 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 352.67 sec
2013-09-17 23:06:41,991 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 352.67 sec
2013-09-17 23:06:42,997 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 352.67 sec
2013-09-17 23:06:44,006 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 355.35 sec
2013-09-17 23:06:45,013 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 355.35 sec
2013-09-17 23:06:46,025 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 355.35 sec
MapReduce Total cumulative CPU time: 5 minutes 55 seconds 350 msec
Ended Job = job_201309172235_0013
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 355.35 sec   HDFS Read: 663537300 HDFS Write: 21 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 55 seconds 350 msec
OK
-2020069000640784227
Time taken: 44.465 seconds, Fetched: 1 row(s)
hive> quit;

times: 2
query: SELECT sum(UserID) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_7091@mturlrep13_201309172306_2064459587.txt
hive> SELECT sum(UserID) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0014
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:06:55,801 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:07:07,899 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:07:08,910 Stage-1 map = 20%,  reduce = 0%
2013-09-17 23:07:09,919 Stage-1 map = 38%,  reduce = 0%
2013-09-17 23:07:10,933 Stage-1 map = 45%,  reduce = 0%, Cumulative CPU 26.96 sec
2013-09-17 23:07:11,941 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 277.91 sec
2013-09-17 23:07:12,948 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.42 sec
2013-09-17 23:07:13,954 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.42 sec
2013-09-17 23:07:14,960 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.42 sec
2013-09-17 23:07:15,966 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.42 sec
2013-09-17 23:07:16,972 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.42 sec
2013-09-17 23:07:17,978 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.42 sec
2013-09-17 23:07:18,984 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 349.42 sec
2013-09-17 23:07:19,992 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 352.35 sec
2013-09-17 23:07:20,998 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 352.35 sec
2013-09-17 23:07:22,005 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 352.35 sec
MapReduce Total cumulative CPU time: 5 minutes 52 seconds 350 msec
Ended Job = job_201309172235_0014
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 352.35 sec   HDFS Read: 663537300 HDFS Write: 21 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 52 seconds 350 msec
OK
-2020069000640784227
Time taken: 34.131 seconds, Fetched: 1 row(s)
hive> quit;

times: 3
query: SELECT sum(UserID) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_12067@mturlrep13_201309172307_240829598.txt
hive> SELECT sum(UserID) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0015
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:07:31,722 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:07:43,798 Stage-1 map = 3%,  reduce = 0%
2013-09-17 23:07:44,804 Stage-1 map = 24%,  reduce = 0%
2013-09-17 23:07:45,810 Stage-1 map = 38%,  reduce = 0%
2013-09-17 23:07:46,822 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 33.64 sec
2013-09-17 23:07:47,830 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 257.18 sec
2013-09-17 23:07:48,837 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.23 sec
2013-09-17 23:07:49,842 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.23 sec
2013-09-17 23:07:50,847 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.23 sec
2013-09-17 23:07:51,852 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.23 sec
2013-09-17 23:07:52,858 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.23 sec
2013-09-17 23:07:53,863 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 349.23 sec
2013-09-17 23:07:54,869 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 349.23 sec
2013-09-17 23:07:55,877 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 352.02 sec
2013-09-17 23:07:56,884 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 352.02 sec
2013-09-17 23:07:57,889 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 352.02 sec
MapReduce Total cumulative CPU time: 5 minutes 52 seconds 20 msec
Ended Job = job_201309172235_0015
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 352.02 sec   HDFS Read: 663537300 HDFS Write: 21 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 52 seconds 20 msec
OK
-2020069000640784227
Time taken: 34.032 seconds, Fetched: 1 row(s)
hive> quit;

times: 1
query: SELECT count(DISTINCT UserID) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_17039@mturlrep13_201309172308_800014414.txt
hive> SELECT count(DISTINCT UserID) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0016
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:08:15,876 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:08:25,047 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:08:27,066 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:08:28,077 Stage-1 map = 6%,  reduce = 0%
2013-09-17 23:08:29,083 Stage-1 map = 11%,  reduce = 0%
2013-09-17 23:08:30,088 Stage-1 map = 18%,  reduce = 0%
2013-09-17 23:08:31,110 Stage-1 map = 19%,  reduce = 0%
2013-09-17 23:08:32,125 Stage-1 map = 29%,  reduce = 0%
2013-09-17 23:08:33,130 Stage-1 map = 34%,  reduce = 0%
2013-09-17 23:08:35,158 Stage-1 map = 47%,  reduce = 0%
2013-09-17 23:08:36,166 Stage-1 map = 51%,  reduce = 0%
2013-09-17 23:08:38,177 Stage-1 map = 65%,  reduce = 0%
2013-09-17 23:08:39,189 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 29.98 sec
2013-09-17 23:08:40,216 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 101.62 sec
2013-09-17 23:08:41,230 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 101.62 sec
2013-09-17 23:08:42,264 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 101.62 sec
2013-09-17 23:08:43,274 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 179.43 sec
2013-09-17 23:08:44,289 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 250.32 sec
2013-09-17 23:08:45,300 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 338.89 sec
2013-09-17 23:08:46,306 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 456.84 sec
2013-09-17 23:08:47,312 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 496.19 sec
2013-09-17 23:08:48,318 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 522.86 sec
2013-09-17 23:08:49,323 Stage-1 map = 98%,  reduce = 1%, Cumulative CPU 600.26 sec
2013-09-17 23:08:50,329 Stage-1 map = 99%,  reduce = 1%, Cumulative CPU 653.17 sec
2013-09-17 23:08:51,335 Stage-1 map = 99%,  reduce = 1%, Cumulative CPU 653.17 sec
2013-09-17 23:08:52,341 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 708.39 sec
2013-09-17 23:08:53,347 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 708.39 sec
2013-09-17 23:08:54,352 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 708.39 sec
2013-09-17 23:08:55,359 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 708.39 sec
2013-09-17 23:08:56,365 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 708.39 sec
2013-09-17 23:08:57,371 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 708.39 sec
2013-09-17 23:08:58,377 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 708.39 sec
2013-09-17 23:08:59,383 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 708.39 sec
2013-09-17 23:09:00,389 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 708.39 sec
2013-09-17 23:09:01,395 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:02,401 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:03,406 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:04,412 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:05,418 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:06,424 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:07,430 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:08,436 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:09,442 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:10,447 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:11,453 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:12,459 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:13,464 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:14,470 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 708.39 sec
2013-09-17 23:09:15,476 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 733.92 sec
2013-09-17 23:09:16,482 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 733.92 sec
2013-09-17 23:09:17,488 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 733.92 sec
2013-09-17 23:09:18,494 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 733.92 sec
2013-09-17 23:09:19,500 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 733.92 sec
2013-09-17 23:09:20,506 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 733.92 sec
2013-09-17 23:09:21,511 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 733.92 sec
2013-09-17 23:09:22,517 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 733.92 sec
2013-09-17 23:09:23,523 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 733.92 sec
2013-09-17 23:09:24,595 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 733.92 sec
2013-09-17 23:09:25,601 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 733.92 sec
2013-09-17 23:09:26,607 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 733.92 sec
2013-09-17 23:09:27,613 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 733.92 sec
2013-09-17 23:09:28,619 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 733.92 sec
2013-09-17 23:09:29,625 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 733.92 sec
2013-09-17 23:09:30,631 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 733.92 sec
2013-09-17 23:09:31,637 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 733.92 sec
2013-09-17 23:09:32,643 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 733.92 sec
2013-09-17 23:09:33,650 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 733.92 sec
2013-09-17 23:09:36,622 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 733.92 sec
2013-09-17 23:09:37,628 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 733.92 sec
2013-09-17 23:09:38,634 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 733.92 sec
2013-09-17 23:09:39,648 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 733.92 sec
2013-09-17 23:09:40,654 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 733.92 sec
2013-09-17 23:09:41,660 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 733.92 sec
2013-09-17 23:09:42,666 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 733.92 sec
2013-09-17 23:09:43,672 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 733.92 sec
2013-09-17 23:09:44,677 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 733.92 sec
2013-09-17 23:09:45,683 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 733.92 sec
2013-09-17 23:09:46,689 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 733.92 sec
2013-09-17 23:09:47,695 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 733.92 sec
2013-09-17 23:09:48,702 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 733.92 sec
2013-09-17 23:09:49,708 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 733.92 sec
2013-09-17 23:09:50,714 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 733.92 sec
2013-09-17 23:09:51,720 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 733.92 sec
2013-09-17 23:09:52,726 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 733.92 sec
2013-09-17 23:09:53,732 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 733.92 sec
2013-09-17 23:09:54,740 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 775.12 sec
2013-09-17 23:09:55,746 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 775.12 sec
MapReduce Total cumulative CPU time: 12 minutes 55 seconds 120 msec
Ended Job = job_201309172235_0016
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 775.12 sec   HDFS Read: 663537300 HDFS Write: 9 SUCCESS
Total MapReduce CPU Time Spent: 12 minutes 55 seconds 120 msec
OK
17630976
Time taken: 110.69 seconds, Fetched: 1 row(s)
hive> quit;

times: 2
query: SELECT count(DISTINCT UserID) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_22191@mturlrep13_201309172309_1602577058.txt
hive> SELECT count(DISTINCT UserID) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0017
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:10:05,908 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:10:18,029 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:10:19,113 Stage-1 map = 16%,  reduce = 0%
2013-09-17 23:10:20,139 Stage-1 map = 18%,  reduce = 0%
2013-09-17 23:10:21,150 Stage-1 map = 19%,  reduce = 0%
2013-09-17 23:10:22,158 Stage-1 map = 46%,  reduce = 0%
2013-09-17 23:10:23,166 Stage-1 map = 56%,  reduce = 0%
2013-09-17 23:10:24,186 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 10.62 sec
2013-09-17 23:10:25,199 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 10.62 sec
2013-09-17 23:10:26,206 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 10.62 sec
2013-09-17 23:10:27,222 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 37.91 sec
2013-09-17 23:10:28,230 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 79.92 sec
2013-09-17 23:10:29,287 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 172.39 sec
2013-09-17 23:10:30,300 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 220.22 sec
2013-09-17 23:10:31,311 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 326.41 sec
2013-09-17 23:10:32,317 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 441.77 sec
2013-09-17 23:10:33,324 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 481.76 sec
2013-09-17 23:10:34,330 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 481.76 sec
2013-09-17 23:10:35,337 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 481.76 sec
2013-09-17 23:10:36,344 Stage-1 map = 97%,  reduce = 11%, Cumulative CPU 507.3 sec
2013-09-17 23:10:37,351 Stage-1 map = 98%,  reduce = 11%, Cumulative CPU 610.94 sec
2013-09-17 23:10:38,357 Stage-1 map = 99%,  reduce = 11%, Cumulative CPU 610.94 sec
2013-09-17 23:10:39,362 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 693.79 sec
2013-09-17 23:10:40,368 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 693.79 sec
2013-09-17 23:10:41,375 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 693.79 sec
2013-09-17 23:10:42,381 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 693.79 sec
2013-09-17 23:10:43,388 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 693.79 sec
2013-09-17 23:10:44,393 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 693.79 sec
2013-09-17 23:10:45,399 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:46,404 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:47,409 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:48,414 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:49,419 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:50,424 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:51,429 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:52,435 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:53,440 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:54,446 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:55,451 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:56,456 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:57,461 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:58,836 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:10:59,948 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:11:00,953 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:11:01,958 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:11:02,963 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:11:05,322 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 693.79 sec
2013-09-17 23:11:06,328 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 693.79 sec
2013-09-17 23:11:07,333 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 693.79 sec
2013-09-17 23:11:08,339 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 693.79 sec
2013-09-17 23:11:09,344 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 693.79 sec
2013-09-17 23:11:10,349 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 693.79 sec
2013-09-17 23:11:12,853 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 693.79 sec
2013-09-17 23:11:13,858 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 693.79 sec
2013-09-17 23:11:14,864 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 693.79 sec
2013-09-17 23:11:21,480 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 734.95 sec
2013-09-17 23:11:22,485 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 734.95 sec
2013-09-17 23:11:23,490 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 734.95 sec
2013-09-17 23:11:24,496 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 734.95 sec
2013-09-17 23:11:25,501 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 734.95 sec
2013-09-17 23:11:26,506 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 734.95 sec
2013-09-17 23:11:27,512 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 734.95 sec
2013-09-17 23:11:28,517 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 734.95 sec
2013-09-17 23:11:29,522 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 734.95 sec
2013-09-17 23:11:30,527 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 734.95 sec
2013-09-17 23:11:31,543 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 734.95 sec
2013-09-17 23:11:32,548 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 734.95 sec
2013-09-17 23:11:33,553 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 734.95 sec
2013-09-17 23:11:34,558 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 734.95 sec
2013-09-17 23:11:35,562 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 734.95 sec
2013-09-17 23:11:36,568 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 734.95 sec
2013-09-17 23:11:37,573 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 734.95 sec
2013-09-17 23:11:38,578 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 734.95 sec
2013-09-17 23:11:39,583 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 734.95 sec
2013-09-17 23:11:40,588 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 734.95 sec
2013-09-17 23:11:41,593 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 761.58 sec
2013-09-17 23:11:42,599 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 761.58 sec
2013-09-17 23:11:43,605 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 761.58 sec
MapReduce Total cumulative CPU time: 12 minutes 41 seconds 580 msec
Ended Job = job_201309172235_0017
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 761.58 sec   HDFS Read: 663537300 HDFS Write: 9 SUCCESS
Total MapReduce CPU Time Spent: 12 minutes 41 seconds 580 msec
OK
17630976
Time taken: 105.953 seconds, Fetched: 1 row(s)
hive> quit;

times: 3
query: SELECT count(DISTINCT UserID) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_28018@mturlrep13_201309172311_1489682234.txt
hive> SELECT count(DISTINCT UserID) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0018
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:11:54,362 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:12:06,510 Stage-1 map = 9%,  reduce = 0%
2013-09-17 23:12:07,520 Stage-1 map = 20%,  reduce = 0%
2013-09-17 23:12:08,525 Stage-1 map = 24%,  reduce = 0%
2013-09-17 23:12:09,532 Stage-1 map = 35%,  reduce = 0%
2013-09-17 23:12:10,537 Stage-1 map = 55%,  reduce = 0%
2013-09-17 23:12:11,543 Stage-1 map = 58%,  reduce = 0%
2013-09-17 23:12:12,555 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 11.81 sec
2013-09-17 23:12:13,575 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 11.81 sec
2013-09-17 23:12:14,581 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 11.81 sec
2013-09-17 23:12:15,598 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 81.59 sec
2013-09-17 23:12:16,636 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 141.75 sec
2013-09-17 23:12:17,658 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 223.95 sec
2013-09-17 23:12:18,665 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 293.32 sec
2013-09-17 23:12:19,671 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 427.68 sec
2013-09-17 23:12:20,678 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 487.22 sec
2013-09-17 23:12:21,684 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 642.1 sec
2013-09-17 23:12:22,691 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 642.1 sec
2013-09-17 23:12:23,697 Stage-1 map = 96%,  reduce = 11%, Cumulative CPU 642.1 sec
2013-09-17 23:12:24,703 Stage-1 map = 98%,  reduce = 11%, Cumulative CPU 665.75 sec
2013-09-17 23:12:25,709 Stage-1 map = 99%,  reduce = 11%, Cumulative CPU 680.97 sec
2013-09-17 23:12:26,715 Stage-1 map = 100%,  reduce = 11%, Cumulative CPU 699.12 sec
2013-09-17 23:12:27,720 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 699.12 sec
2013-09-17 23:12:28,725 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 699.12 sec
2013-09-17 23:12:29,730 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 699.12 sec
2013-09-17 23:12:30,736 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 699.12 sec
2013-09-17 23:12:31,741 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 699.12 sec
2013-09-17 23:12:32,747 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 699.12 sec
2013-09-17 23:12:33,752 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:34,757 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:35,762 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:36,766 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:37,771 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:38,776 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:39,780 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:40,785 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:41,789 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:42,794 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:43,798 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:44,803 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:45,808 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:46,813 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:48,011 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:49,016 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:50,021 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:51,025 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:52,030 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:53,035 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:54,041 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 699.12 sec
2013-09-17 23:12:55,046 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 699.12 sec
2013-09-17 23:12:56,051 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 699.12 sec
2013-09-17 23:12:57,056 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 699.12 sec
2013-09-17 23:12:58,061 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 699.12 sec
2013-09-17 23:12:59,066 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 699.12 sec
2013-09-17 23:13:00,071 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 699.12 sec
2013-09-17 23:13:01,076 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 699.12 sec
2013-09-17 23:13:02,081 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 699.12 sec
2013-09-17 23:13:03,086 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 699.12 sec
2013-09-17 23:13:09,756 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 699.12 sec
2013-09-17 23:13:10,762 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 699.12 sec
2013-09-17 23:13:11,767 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 699.12 sec
2013-09-17 23:13:12,773 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 699.12 sec
2013-09-17 23:13:13,779 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 699.12 sec
2013-09-17 23:13:14,784 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 699.12 sec
2013-09-17 23:13:15,790 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 699.12 sec
2013-09-17 23:13:16,795 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 699.12 sec
2013-09-17 23:13:17,800 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 699.12 sec
2013-09-17 23:13:18,805 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 699.12 sec
2013-09-17 23:13:19,815 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 699.12 sec
2013-09-17 23:13:20,820 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 699.12 sec
2013-09-17 23:13:21,827 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 755.01 sec
2013-09-17 23:13:22,832 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 755.01 sec
2013-09-17 23:13:23,837 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 755.01 sec
2013-09-17 23:13:24,844 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 755.01 sec
2013-09-17 23:13:25,849 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 755.01 sec
2013-09-17 23:13:26,855 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 755.01 sec
2013-09-17 23:13:27,860 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 755.01 sec
2013-09-17 23:13:28,866 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 755.01 sec
2013-09-17 23:13:29,872 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 755.01 sec
2013-09-17 23:13:30,877 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 764.66 sec
2013-09-17 23:13:31,883 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 764.66 sec
2013-09-17 23:13:32,888 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 764.66 sec
MapReduce Total cumulative CPU time: 12 minutes 44 seconds 660 msec
Ended Job = job_201309172235_0018
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 764.66 sec   HDFS Read: 663537300 HDFS Write: 9 SUCCESS
Total MapReduce CPU Time Spent: 12 minutes 44 seconds 660 msec
OK
17630976
Time taken: 107.343 seconds, Fetched: 1 row(s)
hive> quit;

times: 1
query: SELECT count(DISTINCT SearchPhrase) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_709@mturlrep13_201309172313_1257295877.txt
hive> SELECT count(DISTINCT SearchPhrase) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0019
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:13:50,358 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:14:00,485 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:14:01,491 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:14:02,497 Stage-1 map = 11%,  reduce = 0%
2013-09-17 23:14:04,042 Stage-1 map = 24%,  reduce = 0%
2013-09-17 23:14:05,047 Stage-1 map = 26%,  reduce = 0%
2013-09-17 23:14:06,057 Stage-1 map = 35%,  reduce = 0%
2013-09-17 23:14:07,069 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 9.77 sec
2013-09-17 23:14:08,077 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 9.77 sec
2013-09-17 23:14:09,260 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 9.77 sec
2013-09-17 23:14:10,285 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 54.19 sec
2013-09-17 23:14:11,298 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 98.66 sec
2013-09-17 23:14:12,311 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 229.9 sec
2013-09-17 23:14:13,317 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 354.88 sec
2013-09-17 23:14:14,323 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 470.21 sec
2013-09-17 23:14:15,329 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 470.21 sec
2013-09-17 23:14:16,334 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 470.21 sec
2013-09-17 23:14:17,340 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 470.21 sec
2013-09-17 23:14:18,346 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 470.21 sec
2013-09-17 23:14:19,353 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 470.21 sec
2013-09-17 23:14:20,359 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 470.21 sec
2013-09-17 23:14:21,365 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 470.21 sec
2013-09-17 23:14:22,370 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 477.68 sec
2013-09-17 23:14:23,376 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 477.68 sec
2013-09-17 23:14:24,382 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 477.68 sec
2013-09-17 23:14:25,387 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 477.68 sec
2013-09-17 23:14:26,393 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 477.68 sec
2013-09-17 23:14:27,399 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 477.68 sec
2013-09-17 23:14:28,404 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 477.68 sec
2013-09-17 23:14:29,410 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 477.68 sec
2013-09-17 23:14:30,416 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 477.68 sec
2013-09-17 23:14:31,421 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 477.68 sec
2013-09-17 23:14:32,425 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 477.68 sec
2013-09-17 23:14:33,430 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 477.68 sec
2013-09-17 23:14:34,436 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 477.68 sec
2013-09-17 23:14:35,440 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 477.68 sec
2013-09-17 23:14:36,446 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 477.68 sec
2013-09-17 23:14:37,451 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 477.68 sec
2013-09-17 23:14:38,456 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 477.68 sec
2013-09-17 23:14:39,479 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 477.68 sec
2013-09-17 23:14:40,484 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 477.68 sec
2013-09-17 23:14:41,489 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 477.68 sec
2013-09-17 23:14:42,494 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 477.68 sec
2013-09-17 23:14:43,499 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 477.68 sec
2013-09-17 23:14:44,506 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 501.74 sec
2013-09-17 23:14:45,512 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 501.74 sec
MapReduce Total cumulative CPU time: 8 minutes 21 seconds 740 msec
Ended Job = job_201309172235_0019
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 501.74 sec   HDFS Read: 182281719 HDFS Write: 8 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 21 seconds 740 msec
OK
6404441
Time taken: 68.119 seconds, Fetched: 1 row(s)
hive> quit;

times: 2
query: SELECT count(DISTINCT SearchPhrase) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_5905@mturlrep13_201309172314_300988988.txt
hive> SELECT count(DISTINCT SearchPhrase) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0020
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:14:57,829 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:15:09,948 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:15:10,963 Stage-1 map = 17%,  reduce = 0%
2013-09-17 23:15:11,975 Stage-1 map = 37%,  reduce = 0%
2013-09-17 23:15:12,995 Stage-1 map = 38%,  reduce = 0%
2013-09-17 23:15:14,001 Stage-1 map = 61%,  reduce = 0%
2013-09-17 23:15:15,016 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 20.56 sec
2013-09-17 23:15:16,039 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 120.41 sec
2013-09-17 23:15:17,051 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 219.11 sec
2013-09-17 23:15:18,058 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 287.4 sec
2013-09-17 23:15:19,064 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 475.43 sec
2013-09-17 23:15:20,070 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 475.43 sec
2013-09-17 23:15:21,075 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 475.43 sec
2013-09-17 23:15:22,081 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 475.43 sec
2013-09-17 23:15:23,087 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 475.43 sec
2013-09-17 23:15:24,093 Stage-1 map = 100%,  reduce = 17%, Cumulative CPU 475.43 sec
2013-09-17 23:15:25,099 Stage-1 map = 100%,  reduce = 17%, Cumulative CPU 475.43 sec
2013-09-17 23:15:26,104 Stage-1 map = 100%,  reduce = 17%, Cumulative CPU 475.43 sec
2013-09-17 23:15:27,109 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 475.43 sec
2013-09-17 23:15:28,115 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 475.43 sec
2013-09-17 23:15:29,120 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 475.43 sec
2013-09-17 23:15:30,126 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 475.43 sec
2013-09-17 23:15:31,132 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 475.43 sec
2013-09-17 23:15:32,138 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 475.43 sec
2013-09-17 23:15:33,144 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 475.43 sec
2013-09-17 23:15:34,150 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 475.43 sec
2013-09-17 23:15:35,156 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 475.43 sec
2013-09-17 23:15:36,161 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 475.43 sec
2013-09-17 23:15:37,167 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 475.43 sec
2013-09-17 23:15:38,172 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 475.43 sec
2013-09-17 23:15:39,177 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 475.43 sec
2013-09-17 23:15:40,182 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 475.43 sec
2013-09-17 23:15:41,187 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 475.43 sec
2013-09-17 23:15:42,192 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 475.43 sec
2013-09-17 23:15:43,198 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 475.43 sec
2013-09-17 23:15:44,203 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 475.43 sec
2013-09-17 23:15:45,208 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 475.43 sec
2013-09-17 23:15:46,214 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 475.43 sec
2013-09-17 23:15:47,219 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 475.43 sec
2013-09-17 23:15:48,224 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 475.43 sec
2013-09-17 23:15:49,229 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 475.43 sec
2013-09-17 23:15:50,237 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 508.08 sec
2013-09-17 23:15:53,774 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 508.08 sec
2013-09-17 23:15:54,834 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 508.08 sec
MapReduce Total cumulative CPU time: 8 minutes 28 seconds 80 msec
Ended Job = job_201309172235_0020
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 508.08 sec   HDFS Read: 182281719 HDFS Write: 8 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 28 seconds 80 msec
OK
6404441
Time taken: 64.831 seconds, Fetched: 1 row(s)
hive> quit;

times: 3
query: SELECT count(DISTINCT SearchPhrase) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_11972@mturlrep13_201309172315_1325234581.txt
hive> SELECT count(DISTINCT SearchPhrase) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0021
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:16:04,362 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:16:16,459 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:16:17,471 Stage-1 map = 17%,  reduce = 0%
2013-09-17 23:16:18,484 Stage-1 map = 31%,  reduce = 0%
2013-09-17 23:16:19,512 Stage-1 map = 36%,  reduce = 0%
2013-09-17 23:16:20,525 Stage-1 map = 66%,  reduce = 0%
2013-09-17 23:16:21,542 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 31.11 sec
2013-09-17 23:16:22,565 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 364.53 sec
2013-09-17 23:16:23,583 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 382.31 sec
2013-09-17 23:16:24,593 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 432.98 sec
2013-09-17 23:16:25,599 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 472.89 sec
2013-09-17 23:16:26,604 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 472.89 sec
2013-09-17 23:16:27,610 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 472.89 sec
2013-09-17 23:16:28,615 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 472.89 sec
2013-09-17 23:16:29,621 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 472.89 sec
2013-09-17 23:16:30,626 Stage-1 map = 100%,  reduce = 16%, Cumulative CPU 472.89 sec
2013-09-17 23:16:31,632 Stage-1 map = 100%,  reduce = 16%, Cumulative CPU 472.89 sec
2013-09-17 23:16:32,638 Stage-1 map = 100%,  reduce = 16%, Cumulative CPU 472.89 sec
2013-09-17 23:16:33,643 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 472.89 sec
2013-09-17 23:16:34,649 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 472.89 sec
2013-09-17 23:16:35,654 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 472.89 sec
2013-09-17 23:16:36,660 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 472.89 sec
2013-09-17 23:16:37,665 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 472.89 sec
2013-09-17 23:16:38,671 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 472.89 sec
2013-09-17 23:16:39,677 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 472.89 sec
2013-09-17 23:16:40,682 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 472.89 sec
2013-09-17 23:16:41,688 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 472.89 sec
2013-09-17 23:16:42,694 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 472.89 sec
2013-09-17 23:16:43,699 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 472.89 sec
2013-09-17 23:16:44,704 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 472.89 sec
2013-09-17 23:16:45,710 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 472.89 sec
2013-09-17 23:16:46,715 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 472.89 sec
2013-09-17 23:16:47,721 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 472.89 sec
2013-09-17 23:16:48,726 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 472.89 sec
2013-09-17 23:16:49,731 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 472.89 sec
2013-09-17 23:16:50,736 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 472.89 sec
2013-09-17 23:16:51,741 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 472.89 sec
2013-09-17 23:16:52,746 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 472.89 sec
2013-09-17 23:16:53,751 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 472.89 sec
2013-09-17 23:16:54,770 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 472.89 sec
2013-09-17 23:16:55,775 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 472.89 sec
2013-09-17 23:16:56,783 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 504.75 sec
2013-09-17 23:16:57,788 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 504.75 sec
MapReduce Total cumulative CPU time: 8 minutes 24 seconds 750 msec
Ended Job = job_201309172235_0021
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 504.75 sec   HDFS Read: 182281719 HDFS Write: 8 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 24 seconds 750 msec
OK
6404441
Time taken: 64.269 seconds, Fetched: 1 row(s)
hive> quit;

times: 1
query: SELECT min(EventDate), max(EventDate) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_17050@mturlrep13_201309172317_1762378997.txt
hive> SELECT min(EventDate), max(EventDate) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0022
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:17:18,068 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:17:30,186 Stage-1 map = 4%,  reduce = 0%
2013-09-17 23:17:31,192 Stage-1 map = 30%,  reduce = 0%
2013-09-17 23:17:32,197 Stage-1 map = 31%,  reduce = 0%
2013-09-17 23:17:33,206 Stage-1 map = 36%,  reduce = 0%
2013-09-17 23:17:34,211 Stage-1 map = 71%,  reduce = 0%
2013-09-17 23:17:35,223 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 50.55 sec
2013-09-17 23:17:36,231 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 318.5 sec
2013-09-17 23:17:37,237 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 318.5 sec
2013-09-17 23:17:38,242 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 318.5 sec
2013-09-17 23:17:39,248 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 318.5 sec
2013-09-17 23:17:40,253 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 318.5 sec
2013-09-17 23:17:41,259 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 318.5 sec
2013-09-17 23:17:42,265 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 318.5 sec
2013-09-17 23:17:43,271 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 318.5 sec
2013-09-17 23:17:44,278 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 321.17 sec
2013-09-17 23:17:45,284 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 321.17 sec
MapReduce Total cumulative CPU time: 5 minutes 21 seconds 170 msec
Ended Job = job_201309172235_0022
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 321.17 sec   HDFS Read: 4951094 HDFS Write: 6 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 21 seconds 170 msec
OK
Time taken: 37.809 seconds, Fetched: 1 row(s)
hive> quit;

times: 2
query: SELECT min(EventDate), max(EventDate) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_22045@mturlrep13_201309172317_1976013998.txt
hive> SELECT min(EventDate), max(EventDate) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0023
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:17:55,984 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:18:07,051 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:18:08,056 Stage-1 map = 37%,  reduce = 0%
2013-09-17 23:18:09,084 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 15.31 sec
2013-09-17 23:18:10,092 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 213.11 sec
2013-09-17 23:18:11,100 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.89 sec
2013-09-17 23:18:12,105 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.89 sec
2013-09-17 23:18:13,111 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.89 sec
2013-09-17 23:18:14,117 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.89 sec
2013-09-17 23:18:15,123 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.89 sec
2013-09-17 23:18:16,128 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.89 sec
2013-09-17 23:18:17,134 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 322.89 sec
2013-09-17 23:18:18,141 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 325.69 sec
2013-09-17 23:18:19,147 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 325.69 sec
2013-09-17 23:18:20,153 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 325.69 sec
MapReduce Total cumulative CPU time: 5 minutes 25 seconds 690 msec
Ended Job = job_201309172235_0023
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 325.69 sec   HDFS Read: 4951094 HDFS Write: 6 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 25 seconds 690 msec
OK
Time taken: 33.021 seconds, Fetched: 1 row(s)
hive> quit;

times: 3
query: SELECT min(EventDate), max(EventDate) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_26986@mturlrep13_201309172318_1806287056.txt
hive> SELECT min(EventDate), max(EventDate) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0024
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-17 23:18:30,852 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:18:42,000 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:18:43,008 Stage-1 map = 28%,  reduce = 0%
2013-09-17 23:18:44,021 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 41.29 sec
2013-09-17 23:18:45,035 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 247.75 sec
2013-09-17 23:18:46,043 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.15 sec
2013-09-17 23:18:47,048 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.15 sec
2013-09-17 23:18:48,053 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.15 sec
2013-09-17 23:18:49,059 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.15 sec
2013-09-17 23:18:50,064 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.15 sec
2013-09-17 23:18:51,070 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 322.15 sec
2013-09-17 23:18:52,076 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 322.15 sec
2013-09-17 23:18:53,081 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 322.15 sec
2013-09-17 23:18:54,089 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 325.35 sec
2013-09-17 23:18:55,095 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 325.35 sec
MapReduce Total cumulative CPU time: 5 minutes 25 seconds 350 msec
Ended Job = job_201309172235_0024
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 325.35 sec   HDFS Read: 4951094 HDFS Write: 6 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 25 seconds 350 msec
OK
Time taken: 33.13 seconds, Fetched: 1 row(s)
hive> quit;

times: 1
query: SELECT AdvEngineID, count(*) AS c FROM hits_100m WHERE AdvEngineID != 0 GROUP BY AdvEngineID ORDER BY c  DESC;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_31945@mturlrep13_201309172319_144240708.txt
hive> SELECT AdvEngineID, count(*) AS c FROM hits_100m WHERE AdvEngineID != 0 GROUP BY AdvEngineID ORDER BY c  DESC;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0025
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:19:12,376 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:19:22,455 Stage-1 map = 1%,  reduce = 0%, Cumulative CPU 53.86 sec
2013-09-17 23:19:23,463 Stage-1 map = 1%,  reduce = 0%, Cumulative CPU 53.86 sec
2013-09-17 23:19:24,470 Stage-1 map = 13%,  reduce = 0%, Cumulative CPU 53.86 sec
2013-09-17 23:19:25,992 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 55.65 sec
2013-09-17 23:19:27,002 Stage-1 map = 37%,  reduce = 0%, Cumulative CPU 55.65 sec
2013-09-17 23:19:28,010 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 97.8 sec
2013-09-17 23:19:29,023 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 216.67 sec
2013-09-17 23:19:30,033 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 288.99 sec
2013-09-17 23:19:31,040 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 288.99 sec
2013-09-17 23:19:32,046 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 288.99 sec
2013-09-17 23:19:33,052 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 288.99 sec
2013-09-17 23:19:34,058 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 288.99 sec
2013-09-17 23:19:35,065 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 288.99 sec
2013-09-17 23:19:36,070 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 288.99 sec
2013-09-17 23:19:37,080 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 292.1 sec
2013-09-17 23:19:38,087 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 310.42 sec
2013-09-17 23:19:39,094 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 328.04 sec
2013-09-17 23:19:40,101 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 328.04 sec
2013-09-17 23:19:41,111 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 328.04 sec
MapReduce Total cumulative CPU time: 5 minutes 28 seconds 40 msec
Ended Job = job_201309172235_0025
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0026
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:19:43,669 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:19:45,678 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.75 sec
2013-09-17 23:19:46,683 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.75 sec
2013-09-17 23:19:47,689 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.75 sec
2013-09-17 23:19:48,694 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.75 sec
2013-09-17 23:19:49,698 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.75 sec
2013-09-17 23:19:50,703 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.75 sec
2013-09-17 23:19:51,709 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.75 sec
2013-09-17 23:19:52,715 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.75 sec
2013-09-17 23:19:53,721 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.37 sec
2013-09-17 23:19:54,727 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.37 sec
2013-09-17 23:19:55,733 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.37 sec
MapReduce Total cumulative CPU time: 2 seconds 370 msec
Ended Job = job_201309172235_0026
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 328.04 sec   HDFS Read: 6282387 HDFS Write: 1443 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.37 sec   HDFS Read: 5020 HDFS Write: 129 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 30 seconds 410 msec
OK
Time taken: 53.788 seconds, Fetched: 18 row(s)
hive> quit;

times: 2
query: SELECT AdvEngineID, count(*) AS c FROM hits_100m WHERE AdvEngineID != 0 GROUP BY AdvEngineID ORDER BY c  DESC;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_6554@mturlrep13_201309172319_1947847771.txt
hive> SELECT AdvEngineID, count(*) AS c FROM hits_100m WHERE AdvEngineID != 0 GROUP BY AdvEngineID ORDER BY c  DESC;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0027
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:20:07,904 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:20:19,166 Stage-1 map = 4%,  reduce = 0%
2013-09-17 23:20:20,189 Stage-1 map = 42%,  reduce = 0%
2013-09-17 23:20:21,209 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 147.41 sec
2013-09-17 23:20:22,217 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 296.25 sec
2013-09-17 23:20:23,226 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 296.25 sec
2013-09-17 23:20:24,237 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 296.25 sec
2013-09-17 23:20:25,244 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 296.25 sec
2013-09-17 23:20:26,251 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 296.25 sec
2013-09-17 23:20:27,257 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 296.25 sec
2013-09-17 23:20:28,264 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 296.25 sec
2013-09-17 23:20:29,270 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 296.25 sec
2013-09-17 23:20:30,279 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 305.77 sec
2013-09-17 23:20:31,286 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 315.54 sec
2013-09-17 23:20:32,293 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 327.93 sec
2013-09-17 23:20:33,300 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 331.3 sec
2013-09-17 23:20:34,306 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 331.3 sec
MapReduce Total cumulative CPU time: 5 minutes 31 seconds 300 msec
Ended Job = job_201309172235_0027
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0028
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:20:36,785 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:20:38,794 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-17 23:20:39,799 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-17 23:20:40,803 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-17 23:20:41,808 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-17 23:20:42,812 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-17 23:20:43,817 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-17 23:20:44,821 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-17 23:20:45,826 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.72 sec
2013-09-17 23:20:46,831 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.34 sec
2013-09-17 23:20:47,837 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.34 sec
2013-09-17 23:20:48,842 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.34 sec
MapReduce Total cumulative CPU time: 2 seconds 340 msec
Ended Job = job_201309172235_0028
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 331.3 sec   HDFS Read: 6282387 HDFS Write: 1443 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.34 sec   HDFS Read: 5020 HDFS Write: 129 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 33 seconds 640 msec
OK
Time taken: 51.261 seconds, Fetched: 18 row(s)
hive> quit;

times: 3
query: SELECT AdvEngineID, count(*) AS c FROM hits_100m WHERE AdvEngineID != 0 GROUP BY AdvEngineID ORDER BY c  DESC;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_13828@mturlrep13_201309172320_1002752164.txt
hive> SELECT AdvEngineID, count(*) AS c FROM hits_100m WHERE AdvEngineID != 0 GROUP BY AdvEngineID ORDER BY c  DESC;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0029
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:20:59,464 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:21:08,619 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:21:10,697 Stage-1 map = 4%,  reduce = 0%, Cumulative CPU 7.3 sec
2013-09-17 23:21:11,718 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 28.78 sec
2013-09-17 23:21:12,731 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 254.07 sec
2013-09-17 23:21:13,739 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 287.86 sec
2013-09-17 23:21:14,748 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 287.86 sec
2013-09-17 23:21:15,755 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 287.86 sec
2013-09-17 23:21:16,762 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 287.86 sec
2013-09-17 23:21:17,769 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 287.86 sec
2013-09-17 23:21:18,776 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 287.86 sec
2013-09-17 23:21:19,782 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 287.86 sec
2013-09-17 23:21:20,789 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 287.86 sec
2013-09-17 23:21:21,798 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 307.51 sec
2013-09-17 23:21:22,804 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 319.28 sec
2013-09-17 23:21:23,811 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 323.91 sec
2013-09-17 23:21:24,817 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 323.91 sec
MapReduce Total cumulative CPU time: 5 minutes 23 seconds 910 msec
Ended Job = job_201309172235_0029
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0030
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:21:28,286 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:21:29,293 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.69 sec
2013-09-17 23:21:30,299 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.69 sec
2013-09-17 23:21:31,304 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.69 sec
2013-09-17 23:21:32,309 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.69 sec
2013-09-17 23:21:33,314 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.69 sec
2013-09-17 23:21:34,320 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.69 sec
2013-09-17 23:21:35,325 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.69 sec
2013-09-17 23:21:36,330 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.69 sec
2013-09-17 23:21:37,336 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.13 sec
2013-09-17 23:21:38,341 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.13 sec
2013-09-17 23:21:39,348 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.13 sec
MapReduce Total cumulative CPU time: 2 seconds 130 msec
Ended Job = job_201309172235_0030
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 323.91 sec   HDFS Read: 6282387 HDFS Write: 1443 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.13 sec   HDFS Read: 5009 HDFS Write: 129 SUCCESS
Total MapReduce CPU Time Spent: 5 minutes 26 seconds 40 msec
OK
Time taken: 48.653 seconds, Fetched: 18 row(s)
hive> quit;
-- мощная фильтрация. После фильтрации почти ничего не остаётся, но делаем ещё агрегацию.;


times: 1
query: SELECT RegionID, count(DISTINCT UserID) AS u FROM hits_100m GROUP BY RegionID ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_20466@mturlrep13_201309172321_1193168986.txt
hive> SELECT RegionID, count(DISTINCT UserID) AS u FROM hits_100m GROUP BY RegionID ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0031
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:21:56,900 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:22:07,016 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:22:08,024 Stage-1 map = 3%,  reduce = 0%
2013-09-17 23:22:09,036 Stage-1 map = 4%,  reduce = 0%
2013-09-17 23:22:10,195 Stage-1 map = 17%,  reduce = 0%
2013-09-17 23:22:11,201 Stage-1 map = 18%,  reduce = 0%
2013-09-17 23:22:12,207 Stage-1 map = 24%,  reduce = 0%
2013-09-17 23:22:13,212 Stage-1 map = 32%,  reduce = 0%
2013-09-17 23:22:14,220 Stage-1 map = 36%,  reduce = 0%
2013-09-17 23:22:16,775 Stage-1 map = 52%,  reduce = 0%
2013-09-17 23:22:18,821 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 39.61 sec
2013-09-17 23:22:19,842 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 39.61 sec
2013-09-17 23:22:20,847 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 39.61 sec
2013-09-17 23:22:21,854 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 59.18 sec
2013-09-17 23:22:22,860 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 538.37 sec
2013-09-17 23:22:23,900 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 540.67 sec
2013-09-17 23:22:25,023 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 541.84 sec
2013-09-17 23:22:26,034 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 571.25 sec
2013-09-17 23:22:27,044 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 595.16 sec
2013-09-17 23:22:28,050 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 618.83 sec
2013-09-17 23:22:29,057 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 647.47 sec
2013-09-17 23:22:30,063 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 664.76 sec
2013-09-17 23:22:31,070 Stage-1 map = 96%,  reduce = 1%, Cumulative CPU 664.76 sec
2013-09-17 23:22:32,076 Stage-1 map = 98%,  reduce = 12%, Cumulative CPU 685.09 sec
2013-09-17 23:22:33,082 Stage-1 map = 99%,  reduce = 12%, Cumulative CPU 710.3 sec
2013-09-17 23:22:34,088 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 722.47 sec
2013-09-17 23:22:35,094 Stage-1 map = 99%,  reduce = 19%, Cumulative CPU 735.48 sec
2013-09-17 23:22:36,100 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 749.67 sec
2013-09-17 23:22:37,107 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 749.67 sec
2013-09-17 23:22:38,113 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 749.67 sec
2013-09-17 23:22:39,119 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 749.67 sec
2013-09-17 23:22:40,125 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 749.67 sec
2013-09-17 23:22:41,132 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 749.67 sec
2013-09-17 23:22:42,139 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 749.67 sec
2013-09-17 23:22:43,147 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 759.31 sec
2013-09-17 23:22:44,153 Stage-1 map = 100%,  reduce = 62%, Cumulative CPU 792.97 sec
2013-09-17 23:22:45,159 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 792.97 sec
2013-09-17 23:22:46,166 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 805.88 sec
2013-09-17 23:22:47,172 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 816.79 sec
2013-09-17 23:22:48,178 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 816.79 sec
2013-09-17 23:22:49,184 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 829.11 sec
2013-09-17 23:22:50,190 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 843.07 sec
2013-09-17 23:22:51,196 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 872.26 sec
2013-09-17 23:22:52,202 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 872.26 sec
2013-09-17 23:22:53,207 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 872.26 sec
2013-09-17 23:22:54,219 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 872.26 sec
2013-09-17 23:22:55,224 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 872.26 sec
2013-09-17 23:22:56,230 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 897.41 sec
2013-09-17 23:22:57,236 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 897.41 sec
2013-09-17 23:22:58,242 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 897.41 sec
2013-09-17 23:22:59,248 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 897.41 sec
MapReduce Total cumulative CPU time: 14 minutes 57 seconds 410 msec
Ended Job = job_201309172235_0031
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0032
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:23:01,757 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:23:04,768 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:23:05,774 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:23:06,779 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:23:07,783 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:23:08,787 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:23:09,792 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:23:10,796 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:23:11,801 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.62 sec
2013-09-17 23:23:12,806 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.6 sec
2013-09-17 23:23:13,810 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.6 sec
MapReduce Total cumulative CPU time: 3 seconds 600 msec
Ended Job = job_201309172235_0032
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 897.41 sec   HDFS Read: 773347596 HDFS Write: 200700 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.6 sec   HDFS Read: 204277 HDFS Write: 106 SUCCESS
Total MapReduce CPU Time Spent: 15 minutes 1 seconds 10 msec
OK
Time taken: 87.479 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT RegionID, count(DISTINCT UserID) AS u FROM hits_100m GROUP BY RegionID ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_27218@mturlrep13_201309172323_828143909.txt
hive> SELECT RegionID, count(DISTINCT UserID) AS u FROM hits_100m GROUP BY RegionID ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0033
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:23:24,453 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:23:35,535 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:23:36,541 Stage-1 map = 11%,  reduce = 0%
2013-09-17 23:23:37,547 Stage-1 map = 15%,  reduce = 0%
2013-09-17 23:23:38,823 Stage-1 map = 16%,  reduce = 0%
2013-09-17 23:23:39,831 Stage-1 map = 44%,  reduce = 0%
2013-09-17 23:23:40,851 Stage-1 map = 52%,  reduce = 0%
2013-09-17 23:23:41,857 Stage-1 map = 53%,  reduce = 0%
2013-09-17 23:23:42,879 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 11.88 sec
2013-09-17 23:23:43,887 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 11.88 sec
2013-09-17 23:23:44,916 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 40.11 sec
2013-09-17 23:23:45,931 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 54.11 sec
2013-09-17 23:23:46,948 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 83.97 sec
2013-09-17 23:23:47,955 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 116.76 sec
2013-09-17 23:23:48,966 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 237.46 sec
2013-09-17 23:23:49,982 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 292.37 sec
2013-09-17 23:23:50,990 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 331.01 sec
2013-09-17 23:23:51,996 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 474.07 sec
2013-09-17 23:23:53,002 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 516.53 sec
2013-09-17 23:23:54,009 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 516.53 sec
2013-09-17 23:23:55,018 Stage-1 map = 97%,  reduce = 2%, Cumulative CPU 516.53 sec
2013-09-17 23:23:56,024 Stage-1 map = 97%,  reduce = 10%, Cumulative CPU 516.53 sec
2013-09-17 23:23:57,030 Stage-1 map = 98%,  reduce = 22%, Cumulative CPU 599.75 sec
2013-09-17 23:23:58,037 Stage-1 map = 99%,  reduce = 24%, Cumulative CPU 656.33 sec
2013-09-17 23:23:59,043 Stage-1 map = 99%,  reduce = 26%, Cumulative CPU 717.14 sec
2013-09-17 23:24:00,049 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 746.73 sec
2013-09-17 23:24:01,056 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 746.73 sec
2013-09-17 23:24:02,063 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 746.73 sec
2013-09-17 23:24:03,069 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 746.73 sec
2013-09-17 23:24:04,075 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 746.73 sec
2013-09-17 23:24:05,081 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 746.73 sec
2013-09-17 23:24:06,087 Stage-1 map = 100%,  reduce = 53%, Cumulative CPU 746.73 sec
2013-09-17 23:24:07,094 Stage-1 map = 100%,  reduce = 53%, Cumulative CPU 746.73 sec
2013-09-17 23:24:08,102 Stage-1 map = 100%,  reduce = 59%, Cumulative CPU 766.33 sec
2013-09-17 23:24:09,109 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 790.42 sec
2013-09-17 23:24:10,115 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 790.42 sec
2013-09-17 23:24:11,122 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 817.26 sec
2013-09-17 23:24:12,128 Stage-1 map = 100%,  reduce = 87%, Cumulative CPU 817.26 sec
2013-09-17 23:24:13,134 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 828.67 sec
2013-09-17 23:24:14,141 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 840.81 sec
2013-09-17 23:24:15,147 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 869.4 sec
2013-09-17 23:24:16,152 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 869.4 sec
2013-09-17 23:24:17,158 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 869.4 sec
2013-09-17 23:24:18,592 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 869.4 sec
2013-09-17 23:24:19,724 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 869.4 sec
2013-09-17 23:24:20,729 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 869.4 sec
2013-09-17 23:24:21,735 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 894.47 sec
2013-09-17 23:24:22,742 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 894.47 sec
MapReduce Total cumulative CPU time: 14 minutes 54 seconds 470 msec
Ended Job = job_201309172235_0033
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0034
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:24:28,624 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:24:30,631 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.65 sec
2013-09-17 23:24:31,636 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.65 sec
2013-09-17 23:24:32,641 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.65 sec
2013-09-17 23:24:33,646 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.65 sec
2013-09-17 23:24:34,650 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.65 sec
2013-09-17 23:24:35,655 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.65 sec
2013-09-17 23:24:36,660 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.65 sec
2013-09-17 23:24:37,666 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.65 sec
2013-09-17 23:24:38,671 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.47 sec
2013-09-17 23:24:39,676 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.47 sec
2013-09-17 23:24:40,682 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.47 sec
MapReduce Total cumulative CPU time: 3 seconds 470 msec
Ended Job = job_201309172235_0034
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 894.47 sec   HDFS Read: 773347596 HDFS Write: 200700 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.47 sec   HDFS Read: 204277 HDFS Write: 106 SUCCESS
Total MapReduce CPU Time Spent: 14 minutes 57 seconds 940 msec
OK
Time taken: 85.062 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT RegionID, count(DISTINCT UserID) AS u FROM hits_100m GROUP BY RegionID ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_1610@mturlrep13_201309172324_625629875.txt
hive> SELECT RegionID, count(DISTINCT UserID) AS u FROM hits_100m GROUP BY RegionID ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0035
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:24:50,386 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:25:02,507 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:25:03,535 Stage-1 map = 8%,  reduce = 0%
2013-09-17 23:25:04,541 Stage-1 map = 14%,  reduce = 0%
2013-09-17 23:25:05,547 Stage-1 map = 22%,  reduce = 0%
2013-09-17 23:25:06,553 Stage-1 map = 39%,  reduce = 0%
2013-09-17 23:25:07,559 Stage-1 map = 47%,  reduce = 0%
2013-09-17 23:25:08,571 Stage-1 map = 56%,  reduce = 0%
2013-09-17 23:25:09,583 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 12.22 sec
2013-09-17 23:25:10,598 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 12.22 sec
2013-09-17 23:25:11,612 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 12.22 sec
2013-09-17 23:25:12,618 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 55.11 sec
2013-09-17 23:25:13,632 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 55.11 sec
2013-09-17 23:25:14,646 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 117.12 sec
2013-09-17 23:25:15,672 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 186.45 sec
2013-09-17 23:25:16,681 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 275.56 sec
2013-09-17 23:25:17,690 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 373.55 sec
2013-09-17 23:25:18,698 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 434.47 sec
2013-09-17 23:25:19,704 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 497.78 sec
2013-09-17 23:25:20,710 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 518.9 sec
2013-09-17 23:25:21,717 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 518.9 sec
2013-09-17 23:25:22,723 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 518.9 sec
2013-09-17 23:25:23,729 Stage-1 map = 97%,  reduce = 15%, Cumulative CPU 722.2 sec
2013-09-17 23:25:24,736 Stage-1 map = 98%,  reduce = 20%, Cumulative CPU 729.94 sec
2013-09-17 23:25:25,742 Stage-1 map = 99%,  reduce = 24%, Cumulative CPU 745.51 sec
2013-09-17 23:25:26,748 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 754.8 sec
2013-09-17 23:25:27,760 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 754.8 sec
2013-09-17 23:25:28,769 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 754.8 sec
2013-09-17 23:25:29,776 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 754.8 sec
2013-09-17 23:25:30,782 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 754.8 sec
2013-09-17 23:25:31,787 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 754.8 sec
2013-09-17 23:25:32,794 Stage-1 map = 100%,  reduce = 48%, Cumulative CPU 754.8 sec
2013-09-17 23:25:33,800 Stage-1 map = 100%,  reduce = 56%, Cumulative CPU 754.8 sec
2013-09-17 23:25:34,806 Stage-1 map = 100%,  reduce = 60%, Cumulative CPU 754.8 sec
2013-09-17 23:25:35,814 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 775.22 sec
2013-09-17 23:25:36,820 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 800.08 sec
2013-09-17 23:25:37,826 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 813.13 sec
2013-09-17 23:25:38,833 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 854.96 sec
2013-09-17 23:25:39,839 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 881.01 sec
2013-09-17 23:25:40,845 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 881.01 sec
2013-09-17 23:25:41,850 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 881.01 sec
2013-09-17 23:25:42,856 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 881.01 sec
2013-09-17 23:25:43,862 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 881.01 sec
2013-09-17 23:25:45,177 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 881.01 sec
2013-09-17 23:25:46,251 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 881.01 sec
2013-09-17 23:25:47,256 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 881.01 sec
2013-09-17 23:25:48,262 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 881.01 sec
2013-09-17 23:25:49,268 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 905.59 sec
2013-09-17 23:25:50,273 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 905.59 sec
MapReduce Total cumulative CPU time: 15 minutes 5 seconds 590 msec
Ended Job = job_201309172235_0035
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0036
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:25:55,561 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:25:57,569 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:25:58,574 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:25:59,579 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:26:00,584 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:26:01,588 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:26:02,592 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:26:03,597 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.62 sec
2013-09-17 23:26:04,603 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.62 sec
2013-09-17 23:26:05,609 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.52 sec
2013-09-17 23:26:06,614 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.52 sec
2013-09-17 23:26:07,619 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.52 sec
MapReduce Total cumulative CPU time: 3 seconds 520 msec
Ended Job = job_201309172235_0036
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 905.59 sec   HDFS Read: 773347596 HDFS Write: 200700 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.52 sec   HDFS Read: 204266 HDFS Write: 106 SUCCESS
Total MapReduce CPU Time Spent: 15 minutes 9 seconds 110 msec
OK
Time taken: 85.039 seconds, Fetched: 10 row(s)
hive> quit;
-- агрегация, среднее количество ключей.;


times: 1
query: SELECT RegionID, sum(AdvEngineID), count(*) AS c, avg(ResolutionWidth), count(DISTINCT UserID) FROM hits_100m GROUP BY RegionID ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_9511@mturlrep13_201309172326_2040723350.txt
hive> SELECT RegionID, sum(AdvEngineID), count(*) AS c, avg(ResolutionWidth), count(DISTINCT UserID) FROM hits_100m GROUP BY RegionID ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0037
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:26:25,696 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:26:35,814 Stage-1 map = 3%,  reduce = 0%
2013-09-17 23:26:38,841 Stage-1 map = 10%,  reduce = 0%
2013-09-17 23:26:39,848 Stage-1 map = 13%,  reduce = 0%
2013-09-17 23:26:40,855 Stage-1 map = 16%,  reduce = 0%
2013-09-17 23:26:41,883 Stage-1 map = 19%,  reduce = 0%
2013-09-17 23:26:42,890 Stage-1 map = 29%,  reduce = 0%
2013-09-17 23:26:43,897 Stage-1 map = 32%,  reduce = 0%
2013-09-17 23:26:45,223 Stage-1 map = 38%,  reduce = 0%
2013-09-17 23:26:46,236 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 14.75 sec
2013-09-17 23:26:47,243 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 14.75 sec
2013-09-17 23:26:48,308 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 14.75 sec
2013-09-17 23:26:49,329 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 14.75 sec
2013-09-17 23:26:50,342 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 14.75 sec
2013-09-17 23:26:51,354 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 14.75 sec
2013-09-17 23:26:52,368 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 14.75 sec
2013-09-17 23:26:53,382 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 74.49 sec
2013-09-17 23:26:54,393 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 113.4 sec
2013-09-17 23:26:55,401 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 135.12 sec
2013-09-17 23:26:56,414 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 176.43 sec
2013-09-17 23:26:57,430 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 194.75 sec
2013-09-17 23:26:58,444 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 238.4 sec
2013-09-17 23:26:59,451 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 277.62 sec
2013-09-17 23:27:00,458 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 418.29 sec
2013-09-17 23:27:01,465 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 447.59 sec
2013-09-17 23:27:02,472 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 578.54 sec
2013-09-17 23:27:03,478 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 657.91 sec
2013-09-17 23:27:04,486 Stage-1 map = 96%,  reduce = 2%, Cumulative CPU 657.91 sec
2013-09-17 23:27:05,497 Stage-1 map = 97%,  reduce = 12%, Cumulative CPU 726.48 sec
2013-09-17 23:27:06,503 Stage-1 map = 98%,  reduce = 12%, Cumulative CPU 760.94 sec
2013-09-17 23:27:07,509 Stage-1 map = 99%,  reduce = 17%, Cumulative CPU 867.41 sec
2013-09-17 23:27:08,515 Stage-1 map = 99%,  reduce = 24%, Cumulative CPU 867.41 sec
2013-09-17 23:27:09,521 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 941.56 sec
2013-09-17 23:27:10,528 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 941.56 sec
2013-09-17 23:27:11,534 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 941.56 sec
2013-09-17 23:27:12,540 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 941.56 sec
2013-09-17 23:27:13,546 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 941.56 sec
2013-09-17 23:27:14,552 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 941.56 sec
2013-09-17 23:27:15,559 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 941.56 sec
2013-09-17 23:27:16,566 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 941.56 sec
2013-09-17 23:27:17,602 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 941.56 sec
2013-09-17 23:27:18,609 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 941.56 sec
2013-09-17 23:27:19,615 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 941.56 sec
2013-09-17 23:27:20,621 Stage-1 map = 100%,  reduce = 63%, Cumulative CPU 941.56 sec
2013-09-17 23:27:21,628 Stage-1 map = 100%,  reduce = 63%, Cumulative CPU 941.56 sec
2013-09-17 23:27:22,633 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 941.56 sec
2013-09-17 23:27:23,642 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 1072.24 sec
2013-09-17 23:27:24,649 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1080.16 sec
2013-09-17 23:27:25,655 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1080.16 sec
2013-09-17 23:27:26,661 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1087.79 sec
2013-09-17 23:27:27,668 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1097.18 sec
2013-09-17 23:27:28,675 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1097.18 sec
2013-09-17 23:27:29,682 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1106.96 sec
2013-09-17 23:27:30,688 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1106.96 sec
2013-09-17 23:27:31,695 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1106.96 sec
2013-09-17 23:27:32,701 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1106.96 sec
2013-09-17 23:27:33,923 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1136.44 sec
2013-09-17 23:27:34,929 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1136.44 sec
2013-09-17 23:27:35,935 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1136.44 sec
2013-09-17 23:27:36,940 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1136.44 sec
2013-09-17 23:27:37,946 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1136.44 sec
2013-09-17 23:27:41,680 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1136.44 sec
2013-09-17 23:27:42,685 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1136.44 sec
2013-09-17 23:27:43,690 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1136.44 sec
2013-09-17 23:27:45,033 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1155.81 sec
2013-09-17 23:27:46,039 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1155.81 sec
2013-09-17 23:27:47,045 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1155.81 sec
MapReduce Total cumulative CPU time: 19 minutes 15 seconds 810 msec
Ended Job = job_201309172235_0037
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0038
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:27:49,557 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:27:52,569 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.83 sec
2013-09-17 23:27:53,574 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.83 sec
2013-09-17 23:27:54,579 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.83 sec
2013-09-17 23:27:55,583 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.83 sec
2013-09-17 23:27:56,588 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.83 sec
2013-09-17 23:27:57,592 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.83 sec
2013-09-17 23:27:58,596 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.83 sec
2013-09-17 23:27:59,601 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.83 sec
2013-09-17 23:28:00,607 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.83 sec
2013-09-17 23:28:01,613 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.83 sec
MapReduce Total cumulative CPU time: 3 seconds 830 msec
Ended Job = job_201309172235_0038
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1155.81 sec   HDFS Read: 847155129 HDFS Write: 298189 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.83 sec   HDFS Read: 301766 HDFS Write: 437 SUCCESS
Total MapReduce CPU Time Spent: 19 minutes 19 seconds 640 msec
OK
Time taken: 106.577 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT RegionID, sum(AdvEngineID), count(*) AS c, avg(ResolutionWidth), count(DISTINCT UserID) FROM hits_100m GROUP BY RegionID ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_16855@mturlrep13_201309172328_909540588.txt
hive> SELECT RegionID, sum(AdvEngineID), count(*) AS c, avg(ResolutionWidth), count(DISTINCT UserID) FROM hits_100m GROUP BY RegionID ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0039
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:28:12,290 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:28:23,414 Stage-1 map = 1%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:24,442 Stage-1 map = 6%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:25,480 Stage-1 map = 8%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:26,503 Stage-1 map = 10%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:27,510 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:28,517 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:29,524 Stage-1 map = 30%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:30,530 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:31,537 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:32,543 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:33,550 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 51.91 sec
2013-09-17 23:28:34,558 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 66.92 sec
2013-09-17 23:28:35,564 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 66.92 sec
2013-09-17 23:28:36,571 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 66.92 sec
2013-09-17 23:28:37,576 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 91.33 sec
2013-09-17 23:28:38,583 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 128.84 sec
2013-09-17 23:28:39,589 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 167.09 sec
2013-09-17 23:28:40,601 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 202.64 sec
2013-09-17 23:28:41,608 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 265.38 sec
2013-09-17 23:28:42,615 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 331.39 sec
2013-09-17 23:28:43,629 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 434.98 sec
2013-09-17 23:28:44,639 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 460.38 sec
2013-09-17 23:28:45,647 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 605.85 sec
2013-09-17 23:28:46,653 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 658.68 sec
2013-09-17 23:28:47,659 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 658.68 sec
2013-09-17 23:28:48,665 Stage-1 map = 95%,  reduce = 1%, Cumulative CPU 658.68 sec
2013-09-17 23:28:49,672 Stage-1 map = 97%,  reduce = 12%, Cumulative CPU 658.68 sec
2013-09-17 23:28:50,678 Stage-1 map = 97%,  reduce = 17%, Cumulative CPU 658.68 sec
2013-09-17 23:28:51,684 Stage-1 map = 98%,  reduce = 21%, Cumulative CPU 795.29 sec
2013-09-17 23:28:52,690 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 927.85 sec
2013-09-17 23:28:53,696 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 927.85 sec
2013-09-17 23:28:54,702 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 927.85 sec
2013-09-17 23:28:55,712 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 927.85 sec
2013-09-17 23:28:56,718 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 927.85 sec
2013-09-17 23:28:57,724 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 927.85 sec
2013-09-17 23:28:58,730 Stage-1 map = 100%,  reduce = 48%, Cumulative CPU 927.85 sec
2013-09-17 23:28:59,737 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 927.85 sec
2013-09-17 23:29:00,765 Stage-1 map = 100%,  reduce = 56%, Cumulative CPU 927.85 sec
2013-09-17 23:29:01,771 Stage-1 map = 100%,  reduce = 65%, Cumulative CPU 927.85 sec
2013-09-17 23:29:02,777 Stage-1 map = 100%,  reduce = 65%, Cumulative CPU 927.85 sec
2013-09-17 23:29:03,785 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 940.95 sec
2013-09-17 23:29:04,793 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 954.71 sec
2013-09-17 23:29:05,799 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 954.71 sec
2013-09-17 23:29:06,958 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1004.24 sec
2013-09-17 23:29:08,056 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1004.24 sec
2013-09-17 23:29:09,063 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1022.86 sec
2013-09-17 23:29:10,069 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1079.45 sec
2013-09-17 23:29:11,075 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1079.45 sec
2013-09-17 23:29:12,081 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1079.45 sec
2013-09-17 23:29:13,086 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1079.45 sec
2013-09-17 23:29:14,092 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1079.45 sec
2013-09-17 23:29:15,098 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1079.45 sec
2013-09-17 23:29:16,103 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1102.32 sec
2013-09-17 23:29:17,109 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1102.32 sec
2013-09-17 23:29:18,115 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1102.32 sec
2013-09-17 23:29:19,120 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1102.32 sec
2013-09-17 23:29:20,126 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1102.32 sec
2013-09-17 23:29:21,131 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1102.32 sec
2013-09-17 23:29:22,137 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1102.32 sec
2013-09-17 23:29:23,142 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1102.32 sec
2013-09-17 23:29:24,148 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1135.97 sec
2013-09-17 23:29:25,154 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1135.97 sec
2013-09-17 23:29:26,159 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1135.97 sec
2013-09-17 23:29:28,761 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1141.12 sec
2013-09-17 23:29:29,766 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1141.12 sec
2013-09-17 23:29:30,771 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1141.12 sec
MapReduce Total cumulative CPU time: 19 minutes 1 seconds 120 msec
Ended Job = job_201309172235_0039
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0040
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:29:33,329 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:29:36,340 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.71 sec
2013-09-17 23:29:37,344 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.71 sec
2013-09-17 23:29:38,348 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.71 sec
2013-09-17 23:29:39,352 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.71 sec
2013-09-17 23:29:40,356 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.71 sec
2013-09-17 23:29:41,360 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.71 sec
2013-09-17 23:29:42,364 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.71 sec
2013-09-17 23:29:43,369 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.71 sec
2013-09-17 23:29:44,373 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.59 sec
2013-09-17 23:29:45,378 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.59 sec
2013-09-17 23:29:46,383 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.59 sec
MapReduce Total cumulative CPU time: 3 seconds 590 msec
Ended Job = job_201309172235_0040
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1141.12 sec   HDFS Read: 847155129 HDFS Write: 298189 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.59 sec   HDFS Read: 301766 HDFS Write: 437 SUCCESS
Total MapReduce CPU Time Spent: 19 minutes 4 seconds 710 msec
OK
Time taken: 102.879 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT RegionID, sum(AdvEngineID), count(*) AS c, avg(ResolutionWidth), count(DISTINCT UserID) FROM hits_100m GROUP BY RegionID ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_24207@mturlrep13_201309172329_822092081.txt
hive> SELECT RegionID, sum(AdvEngineID), count(*) AS c, avg(ResolutionWidth), count(DISTINCT UserID) FROM hits_100m GROUP BY RegionID ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0041
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:29:56,995 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:30:09,088 Stage-1 map = 5%,  reduce = 0%
2013-09-17 23:30:10,094 Stage-1 map = 6%,  reduce = 0%
2013-09-17 23:30:11,105 Stage-1 map = 7%,  reduce = 0%
2013-09-17 23:30:12,113 Stage-1 map = 15%,  reduce = 0%
2013-09-17 23:30:13,119 Stage-1 map = 26%,  reduce = 0%
2013-09-17 23:30:14,124 Stage-1 map = 29%,  reduce = 0%
2013-09-17 23:30:15,133 Stage-1 map = 33%,  reduce = 0%
2013-09-17 23:30:16,147 Stage-1 map = 47%,  reduce = 0%
2013-09-17 23:30:17,156 Stage-1 map = 52%,  reduce = 0%
2013-09-17 23:30:18,165 Stage-1 map = 54%,  reduce = 0%
2013-09-17 23:30:19,192 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 14.3 sec
2013-09-17 23:30:20,200 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 14.3 sec
2013-09-17 23:30:21,213 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 14.3 sec
2013-09-17 23:30:22,219 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 32.42 sec
2013-09-17 23:30:23,227 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 70.55 sec
2013-09-17 23:30:24,234 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 668.79 sec
2013-09-17 23:30:25,259 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 673.33 sec
2013-09-17 23:30:26,274 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 678.94 sec
2013-09-17 23:30:27,300 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 702.3 sec
2013-09-17 23:30:28,312 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 712.73 sec
2013-09-17 23:30:29,319 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 738.35 sec
2013-09-17 23:30:30,326 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 769.71 sec
2013-09-17 23:30:31,333 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 778.96 sec
2013-09-17 23:30:32,339 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 797.52 sec
2013-09-17 23:30:33,345 Stage-1 map = 95%,  reduce = 1%, Cumulative CPU 797.52 sec
2013-09-17 23:30:34,354 Stage-1 map = 95%,  reduce = 8%, Cumulative CPU 797.52 sec
2013-09-17 23:30:35,361 Stage-1 map = 97%,  reduce = 13%, Cumulative CPU 797.52 sec
2013-09-17 23:30:36,367 Stage-1 map = 98%,  reduce = 15%, Cumulative CPU 841.63 sec
2013-09-17 23:30:37,373 Stage-1 map = 98%,  reduce = 21%, Cumulative CPU 857.24 sec
2013-09-17 23:30:38,379 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 928.19 sec
2013-09-17 23:30:39,385 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 928.19 sec
2013-09-17 23:30:40,391 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 928.19 sec
2013-09-17 23:30:41,396 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 928.19 sec
2013-09-17 23:30:42,403 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 928.19 sec
2013-09-17 23:30:43,409 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 928.19 sec
2013-09-17 23:30:44,415 Stage-1 map = 100%,  reduce = 37%, Cumulative CPU 928.19 sec
2013-09-17 23:30:45,421 Stage-1 map = 100%,  reduce = 37%, Cumulative CPU 928.19 sec
2013-09-17 23:30:46,428 Stage-1 map = 100%,  reduce = 37%, Cumulative CPU 928.19 sec
2013-09-17 23:30:47,451 Stage-1 map = 100%,  reduce = 44%, Cumulative CPU 928.19 sec
2013-09-17 23:30:48,460 Stage-1 map = 100%,  reduce = 48%, Cumulative CPU 940.63 sec
2013-09-17 23:30:49,467 Stage-1 map = 100%,  reduce = 66%, Cumulative CPU 940.63 sec
2013-09-17 23:30:50,474 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 940.63 sec
2013-09-17 23:30:51,480 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 940.63 sec
2013-09-17 23:30:52,487 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 940.63 sec
2013-09-17 23:30:53,494 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 954.04 sec
2013-09-17 23:30:54,500 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 989.38 sec
2013-09-17 23:30:55,507 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 989.38 sec
2013-09-17 23:30:56,514 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1005.72 sec
2013-09-17 23:30:57,520 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1005.72 sec
2013-09-17 23:30:58,526 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1040.57 sec
2013-09-17 23:30:59,532 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1061.53 sec
2013-09-17 23:31:00,538 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1081.7 sec
2013-09-17 23:31:01,544 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1104.84 sec
2013-09-17 23:31:02,550 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1104.84 sec
2013-09-17 23:31:03,556 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1104.84 sec
2013-09-17 23:31:04,562 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1104.84 sec
2013-09-17 23:31:05,568 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1104.84 sec
2013-09-17 23:31:06,573 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1104.84 sec
2013-09-17 23:31:07,579 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1104.84 sec
2013-09-17 23:31:08,585 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1104.84 sec
2013-09-17 23:31:09,591 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1104.84 sec
2013-09-17 23:31:10,597 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1104.84 sec
2013-09-17 23:31:13,163 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1144.43 sec
2013-09-17 23:31:14,168 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1144.43 sec
MapReduce Total cumulative CPU time: 19 minutes 4 seconds 430 msec
Ended Job = job_201309172235_0041
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0042
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:31:17,701 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:31:19,709 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.75 sec
2013-09-17 23:31:20,714 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.75 sec
2013-09-17 23:31:21,719 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.75 sec
2013-09-17 23:31:22,724 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.75 sec
2013-09-17 23:31:23,728 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.75 sec
2013-09-17 23:31:24,733 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.75 sec
2013-09-17 23:31:25,738 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.75 sec
2013-09-17 23:31:26,743 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.75 sec
2013-09-17 23:31:27,748 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.69 sec
2013-09-17 23:31:28,754 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.69 sec
2013-09-17 23:31:29,759 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.69 sec
MapReduce Total cumulative CPU time: 3 seconds 690 msec
Ended Job = job_201309172235_0042
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1144.43 sec   HDFS Read: 847155129 HDFS Write: 298189 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.69 sec   HDFS Read: 301766 HDFS Write: 437 SUCCESS
Total MapReduce CPU Time Spent: 19 minutes 8 seconds 120 msec
OK
Time taken: 101.705 seconds, Fetched: 10 row(s)
hive> quit;
-- агрегация, среднее количество ключей, несколько агрегатных функций.;


times: 1
query: SELECT MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_32236@mturlrep13_201309172331_1074251986.txt
hive> SELECT MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0043
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:31:47,285 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:31:57,397 Stage-1 map = 7%,  reduce = 0%
2013-09-17 23:31:58,403 Stage-1 map = 8%,  reduce = 0%
2013-09-17 23:32:01,416 Stage-1 map = 23%,  reduce = 0%
2013-09-17 23:32:02,422 Stage-1 map = 24%,  reduce = 0%
2013-09-17 23:32:03,436 Stage-1 map = 25%,  reduce = 0%, Cumulative CPU 7.06 sec
2013-09-17 23:32:04,442 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 7.06 sec
2013-09-17 23:32:06,734 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 17.18 sec
2013-09-17 23:32:07,740 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 57.06 sec
2013-09-17 23:32:08,752 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 57.06 sec
2013-09-17 23:32:09,767 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 67.22 sec
2013-09-17 23:32:10,784 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 135.37 sec
2013-09-17 23:32:11,792 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 293.42 sec
2013-09-17 23:32:12,798 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 369.89 sec
2013-09-17 23:32:13,805 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 369.89 sec
2013-09-17 23:32:14,811 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 369.89 sec
2013-09-17 23:32:15,817 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 369.89 sec
2013-09-17 23:32:16,825 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 374.04 sec
2013-09-17 23:32:17,832 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 378.73 sec
2013-09-17 23:32:18,840 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 387.59 sec
2013-09-17 23:32:19,848 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 396.86 sec
2013-09-17 23:32:20,855 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 418.82 sec
2013-09-17 23:32:21,861 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 425.02 sec
2013-09-17 23:32:22,868 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 425.02 sec
MapReduce Total cumulative CPU time: 7 minutes 5 seconds 20 msec
Ended Job = job_201309172235_0043
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0044
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:32:26,376 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:32:28,387 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.98 sec
2013-09-17 23:32:29,392 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.98 sec
2013-09-17 23:32:30,398 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.98 sec
2013-09-17 23:32:31,403 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.98 sec
2013-09-17 23:32:32,408 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.98 sec
2013-09-17 23:32:33,413 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.98 sec
2013-09-17 23:32:34,419 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.98 sec
2013-09-17 23:32:35,425 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.98 sec
2013-09-17 23:32:36,430 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.64 sec
2013-09-17 23:32:37,437 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.64 sec
MapReduce Total cumulative CPU time: 2 seconds 640 msec
Ended Job = job_201309172235_0044
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 425.02 sec   HDFS Read: 673615476 HDFS Write: 41947 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.64 sec   HDFS Read: 45524 HDFS Write: 131 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 7 seconds 660 msec
OK
Time taken: 60.4 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_7129@mturlrep13_201309172332_718169783.txt
hive> SELECT MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0045
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:32:46,858 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:32:58,979 Stage-1 map = 8%,  reduce = 0%
2013-09-17 23:32:59,985 Stage-1 map = 18%,  reduce = 0%
2013-09-17 23:33:00,999 Stage-1 map = 22%,  reduce = 0%
2013-09-17 23:33:02,014 Stage-1 map = 30%,  reduce = 0%
2013-09-17 23:33:03,035 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 39.47 sec
2013-09-17 23:33:04,061 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 127.5 sec
2013-09-17 23:33:05,075 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 389.68 sec
2013-09-17 23:33:06,082 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 414.94 sec
2013-09-17 23:33:07,089 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 414.94 sec
2013-09-17 23:33:08,095 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 414.94 sec
2013-09-17 23:33:09,101 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 414.94 sec
2013-09-17 23:33:10,108 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 414.94 sec
2013-09-17 23:33:11,114 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 414.94 sec
2013-09-17 23:33:12,123 Stage-1 map = 100%,  reduce = 15%, Cumulative CPU 414.94 sec
2013-09-17 23:33:13,130 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 414.94 sec
2013-09-17 23:33:14,145 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 436.06 sec
2013-09-17 23:33:15,152 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 446.54 sec
2013-09-17 23:33:16,160 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 475.11 sec
2013-09-17 23:33:17,166 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 475.11 sec
2013-09-17 23:33:18,173 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 475.11 sec
MapReduce Total cumulative CPU time: 7 minutes 55 seconds 110 msec
Ended Job = job_201309172235_0045
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0046
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:33:21,703 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:33:23,714 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.0 sec
2013-09-17 23:33:24,719 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.0 sec
2013-09-17 23:33:25,724 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.0 sec
2013-09-17 23:33:26,729 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.0 sec
2013-09-17 23:33:27,733 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.0 sec
2013-09-17 23:33:28,738 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.0 sec
2013-09-17 23:33:29,743 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.0 sec
2013-09-17 23:33:30,749 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.71 sec
2013-09-17 23:33:31,755 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.71 sec
2013-09-17 23:33:32,761 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.71 sec
MapReduce Total cumulative CPU time: 2 seconds 710 msec
Ended Job = job_201309172235_0046
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 475.11 sec   HDFS Read: 673615476 HDFS Write: 41947 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.71 sec   HDFS Read: 45524 HDFS Write: 131 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 57 seconds 820 msec
OK
Time taken: 53.498 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_13985@mturlrep13_201309172333_212079891.txt
hive> SELECT MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0047
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:33:42,330 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:33:53,413 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:33:54,432 Stage-1 map = 7%,  reduce = 0%
2013-09-17 23:33:55,462 Stage-1 map = 15%,  reduce = 0%
2013-09-17 23:33:56,469 Stage-1 map = 23%,  reduce = 0%
2013-09-17 23:33:57,482 Stage-1 map = 42%,  reduce = 0%
2013-09-17 23:33:58,503 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 9.27 sec
2013-09-17 23:33:59,519 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 114.64 sec
2013-09-17 23:34:00,529 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 388.28 sec
2013-09-17 23:34:01,538 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 412.66 sec
2013-09-17 23:34:02,545 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 412.66 sec
2013-09-17 23:34:03,551 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 412.66 sec
2013-09-17 23:34:04,558 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 412.66 sec
2013-09-17 23:34:05,563 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 412.66 sec
2013-09-17 23:34:06,569 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 412.66 sec
2013-09-17 23:34:07,576 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 412.66 sec
2013-09-17 23:34:08,585 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 412.66 sec
2013-09-17 23:34:09,595 Stage-1 map = 100%,  reduce = 45%, Cumulative CPU 422.58 sec
2013-09-17 23:34:10,603 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 442.18 sec
2013-09-17 23:34:11,609 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 464.06 sec
2013-09-17 23:34:12,616 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 470.59 sec
2013-09-17 23:34:13,623 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 470.59 sec
MapReduce Total cumulative CPU time: 7 minutes 50 seconds 590 msec
Ended Job = job_201309172235_0047
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0048
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:34:16,130 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:34:19,144 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.95 sec
2013-09-17 23:34:20,150 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.95 sec
2013-09-17 23:34:21,155 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.95 sec
2013-09-17 23:34:22,160 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.95 sec
2013-09-17 23:34:23,165 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.95 sec
2013-09-17 23:34:24,169 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.95 sec
2013-09-17 23:34:25,174 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.95 sec
2013-09-17 23:34:26,180 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.57 sec
2013-09-17 23:34:27,186 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.57 sec
2013-09-17 23:34:28,191 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.57 sec
MapReduce Total cumulative CPU time: 2 seconds 570 msec
Ended Job = job_201309172235_0048
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 470.59 sec   HDFS Read: 673615476 HDFS Write: 41947 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.57 sec   HDFS Read: 45524 HDFS Write: 131 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 53 seconds 160 msec
OK
Time taken: 53.516 seconds, Fetched: 10 row(s)
hive> quit;
-- мощная фильтрация по строкам, затем агрегация по строкам.;


times: 1
query: SELECT MobilePhone, MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_20877@mturlrep13_201309172334_738248079.txt
hive> SELECT MobilePhone, MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0049
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:34:45,537 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:34:54,639 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:34:55,649 Stage-1 map = 6%,  reduce = 0%
2013-09-17 23:34:56,654 Stage-1 map = 9%,  reduce = 0%
2013-09-17 23:34:59,668 Stage-1 map = 21%,  reduce = 0%
2013-09-17 23:35:01,463 Stage-1 map = 22%,  reduce = 0%
2013-09-17 23:35:02,469 Stage-1 map = 43%,  reduce = 0%
2013-09-17 23:35:05,662 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 19.08 sec
2013-09-17 23:35:06,670 Stage-1 map = 62%,  reduce = 0%, Cumulative CPU 64.52 sec
2013-09-17 23:35:07,678 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 81.56 sec
2013-09-17 23:35:08,689 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 92.37 sec
2013-09-17 23:35:09,697 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 161.79 sec
2013-09-17 23:35:10,703 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 299.59 sec
2013-09-17 23:35:11,709 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 358.46 sec
2013-09-17 23:35:12,716 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 358.46 sec
2013-09-17 23:35:13,722 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 358.46 sec
2013-09-17 23:35:14,729 Stage-1 map = 100%,  reduce = 15%, Cumulative CPU 358.46 sec
2013-09-17 23:35:15,735 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 358.46 sec
2013-09-17 23:35:16,745 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 372.22 sec
2013-09-17 23:35:17,752 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 377.09 sec
2013-09-17 23:35:18,760 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 394.56 sec
2013-09-17 23:35:19,766 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 415.94 sec
2013-09-17 23:35:20,773 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 415.94 sec
2013-09-17 23:35:21,779 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 415.94 sec
MapReduce Total cumulative CPU time: 6 minutes 55 seconds 940 msec
Ended Job = job_201309172235_0049
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0050
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:35:24,308 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:35:27,321 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.12 sec
2013-09-17 23:35:28,327 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.12 sec
2013-09-17 23:35:29,332 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.12 sec
2013-09-17 23:35:30,337 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.12 sec
2013-09-17 23:35:31,341 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.12 sec
2013-09-17 23:35:32,346 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.12 sec
2013-09-17 23:35:33,351 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.12 sec
2013-09-17 23:35:34,356 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.12 sec
2013-09-17 23:35:35,362 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.78 sec
2013-09-17 23:35:36,367 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.78 sec
MapReduce Total cumulative CPU time: 2 seconds 780 msec
Ended Job = job_201309172235_0050
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 415.94 sec   HDFS Read: 683872957 HDFS Write: 45535 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.78 sec   HDFS Read: 49112 HDFS Write: 154 SUCCESS
Total MapReduce CPU Time Spent: 6 minutes 58 seconds 720 msec
OK
Time taken: 61.275 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT MobilePhone, MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_28396@mturlrep13_201309172335_1701222232.txt
hive> SELECT MobilePhone, MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0051
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:35:47,068 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:35:58,144 Stage-1 map = 4%,  reduce = 0%
2013-09-17 23:35:59,176 Stage-1 map = 24%,  reduce = 0%
2013-09-17 23:36:00,190 Stage-1 map = 27%,  reduce = 0%
2013-09-17 23:36:01,199 Stage-1 map = 35%,  reduce = 0%
2013-09-17 23:36:02,206 Stage-1 map = 66%,  reduce = 0%
2013-09-17 23:36:03,233 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 156.88 sec
2013-09-17 23:36:04,245 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 396.77 sec
2013-09-17 23:36:05,254 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 408.98 sec
2013-09-17 23:36:06,262 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 408.98 sec
2013-09-17 23:36:07,268 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 408.98 sec
2013-09-17 23:36:08,275 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 408.98 sec
2013-09-17 23:36:09,282 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 408.98 sec
2013-09-17 23:36:10,289 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 408.98 sec
2013-09-17 23:36:11,296 Stage-1 map = 100%,  reduce = 15%, Cumulative CPU 408.98 sec
2013-09-17 23:36:12,305 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 408.98 sec
2013-09-17 23:36:13,315 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 424.36 sec
2013-09-17 23:36:14,324 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 435.5 sec
2013-09-17 23:36:15,330 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 463.94 sec
2013-09-17 23:36:16,337 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 471.88 sec
2013-09-17 23:36:17,343 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 471.88 sec
MapReduce Total cumulative CPU time: 7 minutes 51 seconds 880 msec
Ended Job = job_201309172235_0051
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0052
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:36:20,861 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:36:22,870 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.06 sec
2013-09-17 23:36:23,876 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.06 sec
2013-09-17 23:36:24,881 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.06 sec
2013-09-17 23:36:25,886 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.06 sec
2013-09-17 23:36:26,892 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.06 sec
2013-09-17 23:36:27,897 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.06 sec
2013-09-17 23:36:28,902 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.06 sec
2013-09-17 23:36:29,907 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.06 sec
2013-09-17 23:36:30,914 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.64 sec
2013-09-17 23:36:31,919 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.64 sec
MapReduce Total cumulative CPU time: 2 seconds 640 msec
Ended Job = job_201309172235_0052
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 471.88 sec   HDFS Read: 683872957 HDFS Write: 45535 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.64 sec   HDFS Read: 49112 HDFS Write: 154 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 54 seconds 520 msec
OK
Time taken: 53.698 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT MobilePhone, MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_2932@mturlrep13_201309172336_301621711.txt
hive> SELECT MobilePhone, MobilePhoneModel, count(DISTINCT UserID) AS u FROM hits_100m WHERE MobilePhoneModel != '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0053
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:36:42,472 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:36:53,559 Stage-1 map = 3%,  reduce = 0%
2013-09-17 23:36:54,579 Stage-1 map = 23%,  reduce = 0%
2013-09-17 23:36:55,590 Stage-1 map = 26%,  reduce = 0%
2013-09-17 23:36:56,627 Stage-1 map = 32%,  reduce = 0%
2013-09-17 23:36:57,654 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 9.3 sec
2013-09-17 23:36:58,675 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 127.12 sec
2013-09-17 23:36:59,688 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 387.4 sec
2013-09-17 23:37:00,695 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 400.81 sec
2013-09-17 23:37:01,701 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 413.22 sec
2013-09-17 23:37:02,706 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 413.22 sec
2013-09-17 23:37:03,712 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 413.22 sec
2013-09-17 23:37:04,717 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 413.22 sec
2013-09-17 23:37:05,723 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 413.22 sec
2013-09-17 23:37:06,729 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 413.22 sec
2013-09-17 23:37:07,737 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 413.22 sec
2013-09-17 23:37:08,746 Stage-1 map = 100%,  reduce = 42%, Cumulative CPU 422.72 sec
2013-09-17 23:37:09,754 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 432.93 sec
2013-09-17 23:37:10,760 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 448.36 sec
2013-09-17 23:37:11,768 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 474.11 sec
2013-09-17 23:37:12,775 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 474.11 sec
MapReduce Total cumulative CPU time: 7 minutes 54 seconds 110 msec
Ended Job = job_201309172235_0053
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0054
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-17 23:37:15,239 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:37:18,252 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.03 sec
2013-09-17 23:37:19,258 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.03 sec
2013-09-17 23:37:20,263 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.03 sec
2013-09-17 23:37:21,269 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.03 sec
2013-09-17 23:37:22,274 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.03 sec
2013-09-17 23:37:23,279 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.03 sec
2013-09-17 23:37:24,284 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.03 sec
2013-09-17 23:37:25,289 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.5 sec
2013-09-17 23:37:26,295 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.5 sec
2013-09-17 23:37:27,301 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.5 sec
MapReduce Total cumulative CPU time: 2 seconds 500 msec
Ended Job = job_201309172235_0054
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 474.11 sec   HDFS Read: 683872957 HDFS Write: 45535 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.5 sec   HDFS Read: 49101 HDFS Write: 154 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 56 seconds 610 msec
OK
Time taken: 53.577 seconds, Fetched: 10 row(s)
hive> quit;
-- мощная фильтрация по строкам, затем агрегация по паре из числа и строки.;


times: 1
query: SELECT SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_10187@mturlrep13_201309172337_1774589050.txt
hive> SELECT SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0055
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:37:44,238 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:37:54,302 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:37:55,311 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:37:56,318 Stage-1 map = 5%,  reduce = 0%
2013-09-17 23:37:57,890 Stage-1 map = 19%,  reduce = 0%
2013-09-17 23:37:58,905 Stage-1 map = 21%,  reduce = 0%
2013-09-17 23:37:59,930 Stage-1 map = 25%,  reduce = 0%
2013-09-17 23:38:00,937 Stage-1 map = 47%,  reduce = 0%
2013-09-17 23:38:01,955 Stage-1 map = 50%,  reduce = 0%
2013-09-17 23:38:02,982 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 30.99 sec
2013-09-17 23:38:04,003 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 125.27 sec
2013-09-17 23:38:05,023 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 170.25 sec
2013-09-17 23:38:06,039 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 236.5 sec
2013-09-17 23:38:07,052 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 320.67 sec
2013-09-17 23:38:08,061 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 455.12 sec
2013-09-17 23:38:09,067 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.66 sec
2013-09-17 23:38:10,073 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.66 sec
2013-09-17 23:38:11,079 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.66 sec
2013-09-17 23:38:12,085 Stage-1 map = 100%,  reduce = 1%, Cumulative CPU 469.66 sec
2013-09-17 23:38:13,094 Stage-1 map = 100%,  reduce = 10%, Cumulative CPU 469.66 sec
2013-09-17 23:38:14,102 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 469.66 sec
2013-09-17 23:38:15,109 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 469.66 sec
2013-09-17 23:38:16,116 Stage-1 map = 100%,  reduce = 42%, Cumulative CPU 469.66 sec
2013-09-17 23:38:17,126 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 469.66 sec
2013-09-17 23:38:18,133 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 469.66 sec
2013-09-17 23:38:19,141 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 469.66 sec
2013-09-17 23:38:20,149 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 545.11 sec
2013-09-17 23:38:21,156 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 588.58 sec
2013-09-17 23:38:22,163 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 588.58 sec
MapReduce Total cumulative CPU time: 9 minutes 48 seconds 580 msec
Ended Job = job_201309172235_0055
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0056
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:38:25,617 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:38:32,641 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:38:35,652 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:38:38,663 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:38:41,672 Stage-2 map = 73%,  reduce = 0%
2013-09-17 23:38:44,684 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.74 sec
2013-09-17 23:38:45,690 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.74 sec
2013-09-17 23:38:46,696 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.74 sec
2013-09-17 23:38:47,700 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.5 sec
2013-09-17 23:38:48,705 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.5 sec
2013-09-17 23:38:49,711 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.5 sec
2013-09-17 23:38:50,716 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.5 sec
2013-09-17 23:38:51,722 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.5 sec
2013-09-17 23:38:52,728 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.5 sec
2013-09-17 23:38:53,732 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.5 sec
2013-09-17 23:38:54,738 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.5 sec
2013-09-17 23:38:55,744 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.5 sec
2013-09-17 23:38:56,749 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.5 sec
2013-09-17 23:38:57,754 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.5 sec
2013-09-17 23:38:58,759 Stage-2 map = 100%,  reduce = 74%, Cumulative CPU 54.5 sec
2013-09-17 23:38:59,764 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.07 sec
2013-09-17 23:39:00,769 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.07 sec
2013-09-17 23:39:01,774 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.07 sec
MapReduce Total cumulative CPU time: 1 minutes 4 seconds 70 msec
Ended Job = job_201309172235_0056
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 588.58 sec   HDFS Read: 182281719 HDFS Write: 470238755 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 64.07 sec   HDFS Read: 470242477 HDFS Write: 294 SUCCESS
Total MapReduce CPU Time Spent: 10 minutes 52 seconds 650 msec
OK
Time taken: 87.924 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_16997@mturlrep13_201309172339_969560501.txt
hive> SELECT SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0057
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:39:11,317 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:39:22,427 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:39:23,436 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:39:24,449 Stage-1 map = 17%,  reduce = 0%
2013-09-17 23:39:25,489 Stage-1 map = 27%,  reduce = 0%, Cumulative CPU 238.77 sec
2013-09-17 23:39:26,502 Stage-1 map = 31%,  reduce = 0%, Cumulative CPU 238.77 sec
2013-09-17 23:39:27,523 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 240.99 sec
2013-09-17 23:39:28,534 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 257.1 sec
2013-09-17 23:39:29,549 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 285.48 sec
2013-09-17 23:39:30,573 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 324.5 sec
2013-09-17 23:39:31,585 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 386.01 sec
2013-09-17 23:39:32,592 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.29 sec
2013-09-17 23:39:33,597 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.29 sec
2013-09-17 23:39:34,603 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.29 sec
2013-09-17 23:39:35,608 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.29 sec
2013-09-17 23:39:36,614 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.29 sec
2013-09-17 23:39:37,621 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 469.29 sec
2013-09-17 23:39:38,629 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 469.29 sec
2013-09-17 23:39:39,637 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 469.29 sec
2013-09-17 23:39:40,644 Stage-1 map = 100%,  reduce = 42%, Cumulative CPU 469.29 sec
2013-09-17 23:39:41,652 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 469.29 sec
2013-09-17 23:39:42,659 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 469.29 sec
2013-09-17 23:39:43,669 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 479.78 sec
2013-09-17 23:39:44,677 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 523.18 sec
2013-09-17 23:39:45,684 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 589.52 sec
2013-09-17 23:39:46,692 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 589.52 sec
2013-09-17 23:39:47,699 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 589.52 sec
MapReduce Total cumulative CPU time: 9 minutes 49 seconds 520 msec
Ended Job = job_201309172235_0057
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0058
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:39:50,186 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:39:58,217 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:40:01,228 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:40:04,240 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:40:07,251 Stage-2 map = 82%,  reduce = 0%
2013-09-17 23:40:11,265 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.49 sec
2013-09-17 23:40:12,270 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.49 sec
2013-09-17 23:40:13,276 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.49 sec
2013-09-17 23:40:14,281 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.65 sec
2013-09-17 23:40:15,286 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.65 sec
2013-09-17 23:40:16,291 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.65 sec
2013-09-17 23:40:18,615 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.65 sec
2013-09-17 23:40:19,621 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.65 sec
2013-09-17 23:40:20,627 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.65 sec
2013-09-17 23:40:21,632 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.65 sec
2013-09-17 23:40:22,637 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.65 sec
2013-09-17 23:40:23,643 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.65 sec
2013-09-17 23:40:24,649 Stage-2 map = 100%,  reduce = 74%, Cumulative CPU 54.65 sec
2013-09-17 23:40:25,654 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.01 sec
2013-09-17 23:40:26,660 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.01 sec
MapReduce Total cumulative CPU time: 1 minutes 4 seconds 10 msec
Ended Job = job_201309172235_0058
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 589.52 sec   HDFS Read: 182281719 HDFS Write: 470238755 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 64.01 sec   HDFS Read: 470242477 HDFS Write: 294 SUCCESS
Total MapReduce CPU Time Spent: 10 minutes 53 seconds 530 msec
OK
Time taken: 82.999 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_24505@mturlrep13_201309172340_635324541.txt
hive> SELECT SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0059
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:40:37,367 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:40:48,543 Stage-1 map = 3%,  reduce = 0%
2013-09-17 23:40:49,571 Stage-1 map = 11%,  reduce = 0%
2013-09-17 23:40:50,591 Stage-1 map = 26%,  reduce = 0%
2013-09-17 23:40:51,619 Stage-1 map = 32%,  reduce = 0%
2013-09-17 23:40:52,626 Stage-1 map = 43%,  reduce = 0%
2013-09-17 23:40:53,651 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 28.49 sec
2013-09-17 23:40:54,672 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 114.43 sec
2013-09-17 23:40:55,681 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 171.5 sec
2013-09-17 23:40:56,691 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 279.83 sec
2013-09-17 23:40:57,706 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 424.96 sec
2013-09-17 23:40:58,712 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.69 sec
2013-09-17 23:40:59,717 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.69 sec
2013-09-17 23:41:00,723 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.69 sec
2013-09-17 23:41:01,728 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.69 sec
2013-09-17 23:41:02,735 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 469.69 sec
2013-09-17 23:41:03,743 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 469.69 sec
2013-09-17 23:41:04,751 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 469.69 sec
2013-09-17 23:41:05,758 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 469.69 sec
2013-09-17 23:41:06,765 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 469.69 sec
2013-09-17 23:41:07,772 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 469.69 sec
2013-09-17 23:41:08,779 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 469.69 sec
2013-09-17 23:41:09,788 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 513.51 sec
2013-09-17 23:41:10,794 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 579.41 sec
2013-09-17 23:41:11,800 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 590.63 sec
2013-09-17 23:41:12,807 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 590.63 sec
MapReduce Total cumulative CPU time: 9 minutes 50 seconds 630 msec
Ended Job = job_201309172235_0059
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0060
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:41:15,266 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:41:23,289 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:41:26,298 Stage-2 map = 37%,  reduce = 0%, Cumulative CPU 21.93 sec
2013-09-17 23:41:27,302 Stage-2 map = 37%,  reduce = 0%, Cumulative CPU 21.93 sec
2013-09-17 23:41:28,306 Stage-2 map = 37%,  reduce = 0%, Cumulative CPU 21.93 sec
2013-09-17 23:41:29,310 Stage-2 map = 55%,  reduce = 0%, Cumulative CPU 21.93 sec
2013-09-17 23:41:30,314 Stage-2 map = 55%,  reduce = 0%, Cumulative CPU 21.93 sec
2013-09-17 23:41:31,318 Stage-2 map = 55%,  reduce = 0%, Cumulative CPU 21.93 sec
2013-09-17 23:41:32,322 Stage-2 map = 73%,  reduce = 0%, Cumulative CPU 21.93 sec
2013-09-17 23:41:33,326 Stage-2 map = 73%,  reduce = 0%, Cumulative CPU 21.93 sec
2013-09-17 23:41:34,334 Stage-2 map = 73%,  reduce = 0%, Cumulative CPU 21.93 sec
2013-09-17 23:41:35,338 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 36.41 sec
2013-09-17 23:41:36,342 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 36.41 sec
2013-09-17 23:41:37,347 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 36.41 sec
2013-09-17 23:41:38,351 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 36.41 sec
2013-09-17 23:41:39,355 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.78 sec
2013-09-17 23:41:40,359 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.78 sec
2013-09-17 23:41:41,363 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.78 sec
2013-09-17 23:41:42,368 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.78 sec
2013-09-17 23:41:43,372 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.78 sec
2013-09-17 23:41:44,376 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.78 sec
2013-09-17 23:41:45,381 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.78 sec
2013-09-17 23:41:46,386 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.78 sec
2013-09-17 23:41:47,390 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.78 sec
2013-09-17 23:41:48,394 Stage-2 map = 100%,  reduce = 74%, Cumulative CPU 54.78 sec
2013-09-17 23:41:49,399 Stage-2 map = 100%,  reduce = 74%, Cumulative CPU 54.78 sec
2013-09-17 23:41:50,404 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.36 sec
2013-09-17 23:41:51,408 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.36 sec
MapReduce Total cumulative CPU time: 1 minutes 4 seconds 360 msec
Ended Job = job_201309172235_0060
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 590.63 sec   HDFS Read: 182281719 HDFS Write: 470238755 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 64.36 sec   HDFS Read: 470242477 HDFS Write: 294 SUCCESS
Total MapReduce CPU Time Spent: 10 minutes 54 seconds 990 msec
OK
Time taken: 82.867 seconds, Fetched: 10 row(s)
hive> quit;
-- средняя фильтрация по строкам, затем агрегация по строкам, большое количество ключей.;


times: 1
query: SELECT SearchPhrase, count(DISTINCT UserID) AS u FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_31334@mturlrep13_201309172342_322265508.txt
hive> SELECT SearchPhrase, count(DISTINCT UserID) AS u FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0061
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:42:11,682 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:42:21,798 Stage-1 map = 7%,  reduce = 0%
2013-09-17 23:42:22,804 Stage-1 map = 8%,  reduce = 0%
2013-09-17 23:42:24,816 Stage-1 map = 15%,  reduce = 0%
2013-09-17 23:42:27,083 Stage-1 map = 22%,  reduce = 0%, Cumulative CPU 228.9 sec
2013-09-17 23:42:28,092 Stage-1 map = 32%,  reduce = 0%, Cumulative CPU 228.9 sec
2013-09-17 23:42:29,101 Stage-1 map = 35%,  reduce = 0%, Cumulative CPU 228.9 sec
2013-09-17 23:42:31,162 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 228.9 sec
2013-09-17 23:42:32,201 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 231.63 sec
2013-09-17 23:42:33,214 Stage-1 map = 51%,  reduce = 0%, Cumulative CPU 234.97 sec
2013-09-17 23:42:34,223 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 254.73 sec
2013-09-17 23:42:35,315 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 254.73 sec
2013-09-17 23:42:36,327 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 260.85 sec
2013-09-17 23:42:37,339 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 292.13 sec
2013-09-17 23:42:38,347 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 320.32 sec
2013-09-17 23:42:39,358 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 396.48 sec
2013-09-17 23:42:40,364 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 484.46 sec
2013-09-17 23:42:41,369 Stage-1 map = 100%,  reduce = 2%, Cumulative CPU 517.16 sec
2013-09-17 23:42:42,376 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 517.16 sec
2013-09-17 23:42:43,383 Stage-1 map = 100%,  reduce = 16%, Cumulative CPU 517.16 sec
2013-09-17 23:42:44,391 Stage-1 map = 100%,  reduce = 25%, Cumulative CPU 517.16 sec
2013-09-17 23:42:45,397 Stage-1 map = 100%,  reduce = 37%, Cumulative CPU 517.16 sec
2013-09-17 23:42:46,404 Stage-1 map = 100%,  reduce = 48%, Cumulative CPU 517.16 sec
2013-09-17 23:42:47,411 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 517.16 sec
2013-09-17 23:42:48,419 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 517.16 sec
2013-09-17 23:42:49,427 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 528.88 sec
2013-09-17 23:42:50,435 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 540.66 sec
2013-09-17 23:42:51,442 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 589.36 sec
2013-09-17 23:42:52,449 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 626.57 sec
2013-09-17 23:42:53,455 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 626.57 sec
2013-09-17 23:42:54,461 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 651.04 sec
2013-09-17 23:42:55,468 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 651.04 sec
MapReduce Total cumulative CPU time: 10 minutes 51 seconds 40 msec
Ended Job = job_201309172235_0061
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0062
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:42:57,983 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:43:06,006 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:43:09,017 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:43:12,028 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:43:15,039 Stage-2 map = 73%,  reduce = 0%
2013-09-17 23:43:18,050 Stage-2 map = 82%,  reduce = 0%
2013-09-17 23:43:19,055 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.26 sec
2013-09-17 23:43:20,061 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.26 sec
2013-09-17 23:43:21,068 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 53.98 sec
2013-09-17 23:43:22,073 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 53.98 sec
2013-09-17 23:43:23,077 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 53.98 sec
2013-09-17 23:43:24,082 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 53.98 sec
2013-09-17 23:43:25,087 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 53.98 sec
2013-09-17 23:43:26,092 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 53.98 sec
2013-09-17 23:43:27,096 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 53.98 sec
2013-09-17 23:43:28,101 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 53.98 sec
2013-09-17 23:43:29,106 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 53.98 sec
2013-09-17 23:43:30,110 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 53.98 sec
2013-09-17 23:43:31,114 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 53.98 sec
2013-09-17 23:43:32,119 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 53.98 sec
2013-09-17 23:43:33,124 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 63.69 sec
2013-09-17 23:43:34,129 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 63.69 sec
2013-09-17 23:43:35,135 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 63.69 sec
MapReduce Total cumulative CPU time: 1 minutes 3 seconds 690 msec
Ended Job = job_201309172235_0062
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 651.04 sec   HDFS Read: 840872065 HDFS Write: 470236967 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 63.69 sec   HDFS Read: 470240689 HDFS Write: 291 SUCCESS
Total MapReduce CPU Time Spent: 11 minutes 54 seconds 730 msec
OK
Time taken: 94.281 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT SearchPhrase, count(DISTINCT UserID) AS u FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_6246@mturlrep13_201309172343_752416466.txt
hive> SELECT SearchPhrase, count(DISTINCT UserID) AS u FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0063
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:43:46,269 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:43:55,322 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:43:56,328 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:43:57,335 Stage-1 map = 8%,  reduce = 0%
2013-09-17 23:43:58,347 Stage-1 map = 15%,  reduce = 0%
2013-09-17 23:43:59,353 Stage-1 map = 20%,  reduce = 0%
2013-09-17 23:44:00,368 Stage-1 map = 29%,  reduce = 0%
2013-09-17 23:44:01,375 Stage-1 map = 40%,  reduce = 0%
2013-09-17 23:44:02,399 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 11.32 sec
2013-09-17 23:44:03,419 Stage-1 map = 54%,  reduce = 0%, Cumulative CPU 45.05 sec
2013-09-17 23:44:04,431 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 68.44 sec
2013-09-17 23:44:05,439 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 146.24 sec
2013-09-17 23:44:06,446 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 213.64 sec
2013-09-17 23:44:07,457 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 292.27 sec
2013-09-17 23:44:08,464 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 443.09 sec
2013-09-17 23:44:09,470 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 549.17 sec
2013-09-17 23:44:10,476 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 549.17 sec
2013-09-17 23:44:11,482 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 549.17 sec
2013-09-17 23:44:12,489 Stage-1 map = 100%,  reduce = 1%, Cumulative CPU 549.17 sec
2013-09-17 23:44:13,497 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 549.17 sec
2013-09-17 23:44:14,506 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 549.17 sec
2013-09-17 23:44:15,513 Stage-1 map = 100%,  reduce = 34%, Cumulative CPU 549.17 sec
2013-09-17 23:44:16,520 Stage-1 map = 100%,  reduce = 45%, Cumulative CPU 549.17 sec
2013-09-17 23:44:17,527 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 549.17 sec
2013-09-17 23:44:18,534 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 549.17 sec
2013-09-17 23:44:19,541 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 549.17 sec
2013-09-17 23:44:20,550 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 561.32 sec
2013-09-17 23:44:21,556 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 585.6 sec
2013-09-17 23:44:22,563 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 687.26 sec
2013-09-17 23:44:23,570 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 687.26 sec
2013-09-17 23:44:24,576 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 687.26 sec
MapReduce Total cumulative CPU time: 11 minutes 27 seconds 260 msec
Ended Job = job_201309172235_0063
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0064
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:44:28,140 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:44:35,163 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:44:38,174 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:44:41,183 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:44:44,194 Stage-2 map = 82%,  reduce = 0%
2013-09-17 23:44:47,205 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.21 sec
2013-09-17 23:44:48,211 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.21 sec
2013-09-17 23:44:49,215 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.21 sec
2013-09-17 23:44:50,916 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.21 sec
2013-09-17 23:44:51,921 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.96 sec
2013-09-17 23:44:52,925 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.96 sec
2013-09-17 23:44:53,931 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.96 sec
2013-09-17 23:44:54,937 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.96 sec
2013-09-17 23:44:55,942 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.96 sec
2013-09-17 23:44:56,947 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.96 sec
2013-09-17 23:44:57,951 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.96 sec
2013-09-17 23:44:58,956 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.96 sec
2013-09-17 23:44:59,960 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 54.96 sec
2013-09-17 23:45:00,964 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 54.96 sec
2013-09-17 23:45:01,970 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.36 sec
2013-09-17 23:45:02,975 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.36 sec
2013-09-17 23:45:03,981 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.36 sec
MapReduce Total cumulative CPU time: 1 minutes 4 seconds 360 msec
Ended Job = job_201309172235_0064
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 687.26 sec   HDFS Read: 840872065 HDFS Write: 470236967 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 64.36 sec   HDFS Read: 470240689 HDFS Write: 291 SUCCESS
Total MapReduce CPU Time Spent: 12 minutes 31 seconds 620 msec
OK
Time taken: 86.991 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT SearchPhrase, count(DISTINCT UserID) AS u FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_13649@mturlrep13_201309172345_1690142009.txt
hive> SELECT SearchPhrase, count(DISTINCT UserID) AS u FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0065
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:45:13,718 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:45:24,809 Stage-1 map = 3%,  reduce = 0%
2013-09-17 23:45:25,833 Stage-1 map = 9%,  reduce = 0%, Cumulative CPU 168.95 sec
2013-09-17 23:45:26,845 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 168.95 sec
2013-09-17 23:45:27,853 Stage-1 map = 23%,  reduce = 0%, Cumulative CPU 168.95 sec
2013-09-17 23:45:28,867 Stage-1 map = 29%,  reduce = 0%, Cumulative CPU 168.95 sec
2013-09-17 23:45:29,874 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 168.95 sec
2013-09-17 23:45:30,887 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 173.1 sec
2013-09-17 23:45:31,894 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 190.37 sec
2013-09-17 23:45:32,900 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 207.59 sec
2013-09-17 23:45:33,913 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 266.9 sec
2013-09-17 23:45:34,921 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 286.81 sec
2013-09-17 23:45:35,928 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 319.49 sec
2013-09-17 23:45:36,935 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 429.06 sec
2013-09-17 23:45:37,943 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 545.45 sec
2013-09-17 23:45:38,949 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 545.45 sec
2013-09-17 23:45:39,955 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 545.45 sec
2013-09-17 23:45:40,961 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 545.45 sec
2013-09-17 23:45:41,968 Stage-1 map = 100%,  reduce = 5%, Cumulative CPU 545.45 sec
2013-09-17 23:45:42,978 Stage-1 map = 100%,  reduce = 17%, Cumulative CPU 545.45 sec
2013-09-17 23:45:43,985 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 545.45 sec
2013-09-17 23:45:44,992 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 545.45 sec
2013-09-17 23:45:45,999 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 545.45 sec
2013-09-17 23:45:47,006 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 545.45 sec
2013-09-17 23:45:48,013 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 545.45 sec
2013-09-17 23:45:49,019 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 545.45 sec
2013-09-17 23:45:50,027 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 595.55 sec
2013-09-17 23:45:51,034 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 670.19 sec
2013-09-17 23:45:52,040 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 682.37 sec
2013-09-17 23:45:53,047 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 682.37 sec
MapReduce Total cumulative CPU time: 11 minutes 22 seconds 370 msec
Ended Job = job_201309172235_0065
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0066
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:45:56,566 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:46:03,591 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:46:06,601 Stage-2 map = 28%,  reduce = 0%
2013-09-17 23:46:07,606 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:46:10,617 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:46:13,628 Stage-2 map = 73%,  reduce = 0%
2013-09-17 23:46:16,640 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.88 sec
2013-09-17 23:46:17,646 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.88 sec
2013-09-17 23:46:18,651 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 25.88 sec
2013-09-17 23:46:19,840 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.97 sec
2013-09-17 23:46:20,844 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.97 sec
2013-09-17 23:46:21,848 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.97 sec
2013-09-17 23:46:22,853 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.97 sec
2013-09-17 23:46:23,857 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 54.97 sec
2013-09-17 23:46:24,862 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 54.97 sec
2013-09-17 23:46:25,866 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 56.42 sec
2013-09-17 23:46:26,871 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 56.42 sec
2013-09-17 23:46:27,875 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 56.42 sec
2013-09-17 23:46:28,879 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 56.42 sec
2013-09-17 23:46:29,883 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 56.42 sec
2013-09-17 23:46:30,888 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 56.42 sec
2013-09-17 23:46:31,893 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.4 sec
2013-09-17 23:46:32,899 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 64.4 sec
MapReduce Total cumulative CPU time: 1 minutes 4 seconds 400 msec
Ended Job = job_201309172235_0066
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 682.37 sec   HDFS Read: 840872065 HDFS Write: 470236967 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 64.4 sec   HDFS Read: 470240689 HDFS Write: 291 SUCCESS
Total MapReduce CPU Time Spent: 12 minutes 26 seconds 770 msec
OK
Time taken: 87.084 seconds, Fetched: 10 row(s)
hive> quit;
-- агрегация чуть сложнее.;


times: 1
query: SELECT SearchEngineID, SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_21386@mturlrep13_201309172346_1552848937.txt
hive> SELECT SearchEngineID, SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0067
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:46:53,783 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:47:03,859 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:47:05,878 Stage-1 map = 5%,  reduce = 0%
2013-09-17 23:47:07,199 Stage-1 map = 21%,  reduce = 0%
2013-09-17 23:47:08,207 Stage-1 map = 24%,  reduce = 0%
2013-09-17 23:47:09,214 Stage-1 map = 30%,  reduce = 0%
2013-09-17 23:47:10,223 Stage-1 map = 48%,  reduce = 0%
2013-09-17 23:47:11,239 Stage-1 map = 50%,  reduce = 0%
2013-09-17 23:47:12,265 Stage-1 map = 62%,  reduce = 0%, Cumulative CPU 41.92 sec
2013-09-17 23:47:13,282 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 108.6 sec
2013-09-17 23:47:14,295 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 181.12 sec
2013-09-17 23:47:15,307 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 247.1 sec
2013-09-17 23:47:16,313 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 290.6 sec
2013-09-17 23:47:17,320 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 460.07 sec
2013-09-17 23:47:18,326 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 491.41 sec
2013-09-17 23:47:19,332 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 491.41 sec
2013-09-17 23:47:20,338 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 491.41 sec
2013-09-17 23:47:21,345 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 491.41 sec
2013-09-17 23:47:22,352 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 491.41 sec
2013-09-17 23:47:23,359 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 491.41 sec
2013-09-17 23:47:24,369 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 491.41 sec
2013-09-17 23:47:25,377 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 491.41 sec
2013-09-17 23:47:26,389 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 543.54 sec
2013-09-17 23:47:27,397 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 543.54 sec
2013-09-17 23:47:28,404 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 543.54 sec
2013-09-17 23:47:29,411 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 560.73 sec
2013-09-17 23:47:30,418 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 619.13 sec
2013-09-17 23:47:31,425 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 619.13 sec
2013-09-17 23:47:32,432 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 619.13 sec
MapReduce Total cumulative CPU time: 10 minutes 19 seconds 130 msec
Ended Job = job_201309172235_0067
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0068
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:47:35,954 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:47:42,978 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:47:45,991 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:47:49,003 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:47:52,014 Stage-2 map = 73%,  reduce = 0%
2013-09-17 23:47:55,025 Stage-2 map = 82%,  reduce = 0%
2013-09-17 23:47:57,034 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 27.07 sec
2013-09-17 23:47:58,040 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 27.07 sec
2013-09-17 23:47:59,046 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 27.07 sec
2013-09-17 23:48:00,051 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 27.07 sec
2013-09-17 23:48:01,056 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 58.79 sec
2013-09-17 23:48:02,060 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 58.79 sec
2013-09-17 23:48:03,066 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 58.79 sec
2013-09-17 23:48:04,072 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 58.79 sec
2013-09-17 23:48:05,077 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 58.79 sec
2013-09-17 23:48:06,082 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 58.79 sec
2013-09-17 23:48:07,087 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 58.79 sec
2013-09-17 23:48:08,092 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 58.79 sec
2013-09-17 23:48:09,096 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 58.79 sec
2013-09-17 23:48:10,102 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 58.79 sec
2013-09-17 23:48:11,107 Stage-2 map = 100%,  reduce = 74%, Cumulative CPU 58.79 sec
2013-09-17 23:48:12,112 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 68.73 sec
2013-09-17 23:48:13,119 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 68.73 sec
2013-09-17 23:48:14,124 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 68.73 sec
MapReduce Total cumulative CPU time: 1 minutes 8 seconds 730 msec
Ended Job = job_201309172235_0068
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 619.13 sec   HDFS Read: 200023859 HDFS Write: 499984003 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 68.73 sec   HDFS Read: 499987725 HDFS Write: 286 SUCCESS
Total MapReduce CPU Time Spent: 11 minutes 27 seconds 860 msec
OK
Time taken: 91.05 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT SearchEngineID, SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_28179@mturlrep13_201309172348_1562230786.txt
hive> SELECT SearchEngineID, SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0069
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:48:23,881 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:48:35,995 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:48:37,001 Stage-1 map = 16%,  reduce = 0%
2013-09-17 23:48:38,011 Stage-1 map = 20%,  reduce = 0%
2013-09-17 23:48:39,018 Stage-1 map = 24%,  reduce = 0%
2013-09-17 23:48:40,024 Stage-1 map = 39%,  reduce = 0%
2013-09-17 23:48:41,052 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 10.64 sec
2013-09-17 23:48:42,094 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 99.96 sec
2013-09-17 23:48:43,114 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 158.73 sec
2013-09-17 23:48:44,140 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 239.0 sec
2013-09-17 23:48:45,149 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 313.14 sec
2013-09-17 23:48:46,156 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 453.86 sec
2013-09-17 23:48:47,162 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 503.27 sec
2013-09-17 23:48:48,168 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 503.27 sec
2013-09-17 23:48:49,174 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 503.27 sec
2013-09-17 23:48:50,181 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 503.27 sec
2013-09-17 23:48:51,188 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 503.27 sec
2013-09-17 23:48:52,194 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 503.27 sec
2013-09-17 23:48:53,201 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 503.27 sec
2013-09-17 23:48:54,207 Stage-1 map = 100%,  reduce = 45%, Cumulative CPU 503.27 sec
2013-09-17 23:48:55,214 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 503.27 sec
2013-09-17 23:48:56,221 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 503.27 sec
2013-09-17 23:48:57,228 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 503.27 sec
2013-09-17 23:48:58,237 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 548.17 sec
2013-09-17 23:48:59,244 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 606.99 sec
2013-09-17 23:49:00,251 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 630.32 sec
2013-09-17 23:49:01,257 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 630.32 sec
MapReduce Total cumulative CPU time: 10 minutes 30 seconds 320 msec
Ended Job = job_201309172235_0069
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0070
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:49:04,779 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:49:11,801 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:49:14,812 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:49:17,821 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:49:20,831 Stage-2 map = 73%,  reduce = 0%
2013-09-17 23:49:23,839 Stage-2 map = 82%,  reduce = 0%
2013-09-17 23:49:26,850 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 52.88 sec
2013-09-17 23:49:27,856 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 52.88 sec
2013-09-17 23:49:28,860 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 52.88 sec
2013-09-17 23:49:29,866 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 52.88 sec
2013-09-17 23:49:32,220 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 60.11 sec
2013-09-17 23:49:33,226 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 60.11 sec
2013-09-17 23:49:34,231 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 60.11 sec
2013-09-17 23:49:35,237 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 60.11 sec
2013-09-17 23:49:36,242 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 60.11 sec
2013-09-17 23:49:37,248 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 60.11 sec
2013-09-17 23:49:38,253 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 60.11 sec
2013-09-17 23:49:39,259 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 60.11 sec
2013-09-17 23:49:40,265 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 60.11 sec
2013-09-17 23:49:41,270 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 70.2 sec
2013-09-17 23:49:42,276 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 70.2 sec
2013-09-17 23:49:43,281 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 70.2 sec
MapReduce Total cumulative CPU time: 1 minutes 10 seconds 200 msec
Ended Job = job_201309172235_0070
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 630.32 sec   HDFS Read: 200023859 HDFS Write: 499984003 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 70.2 sec   HDFS Read: 499987725 HDFS Write: 286 SUCCESS
Total MapReduce CPU Time Spent: 11 minutes 40 seconds 520 msec
OK
Time taken: 87.267 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT SearchEngineID, SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_2628@mturlrep13_201309172349_1108098215.txt
hive> SELECT SearchEngineID, SearchPhrase, count(*) AS c FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0071
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:49:52,985 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:50:05,128 Stage-1 map = 2%,  reduce = 0%
2013-09-17 23:50:06,133 Stage-1 map = 15%,  reduce = 0%
2013-09-17 23:50:07,151 Stage-1 map = 22%,  reduce = 0%
2013-09-17 23:50:08,158 Stage-1 map = 23%,  reduce = 0%
2013-09-17 23:50:09,175 Stage-1 map = 50%,  reduce = 0%
2013-09-17 23:50:10,194 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 31.85 sec
2013-09-17 23:50:11,210 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 65.05 sec
2013-09-17 23:50:12,225 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 147.44 sec
2013-09-17 23:50:13,259 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 212.15 sec
2013-09-17 23:50:14,277 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 269.6 sec
2013-09-17 23:50:15,286 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 409.48 sec
2013-09-17 23:50:16,293 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 490.26 sec
2013-09-17 23:50:17,299 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 507.73 sec
2013-09-17 23:50:18,304 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 507.73 sec
2013-09-17 23:50:19,311 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 507.73 sec
2013-09-17 23:50:20,317 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 507.73 sec
2013-09-17 23:50:21,323 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 507.73 sec
2013-09-17 23:50:22,331 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 507.73 sec
2013-09-17 23:50:23,337 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 507.73 sec
2013-09-17 23:50:24,344 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 507.73 sec
2013-09-17 23:50:25,351 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 507.73 sec
2013-09-17 23:50:26,357 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 507.73 sec
2013-09-17 23:50:27,367 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 592.09 sec
2013-09-17 23:50:28,374 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 620.73 sec
2013-09-17 23:50:29,381 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 636.3 sec
2013-09-17 23:50:30,387 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 636.3 sec
MapReduce Total cumulative CPU time: 10 minutes 36 seconds 300 msec
Ended Job = job_201309172235_0071
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0072
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:50:33,867 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:50:40,890 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:50:43,901 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:50:46,911 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:50:49,921 Stage-2 map = 73%,  reduce = 0%
2013-09-17 23:50:53,934 Stage-2 map = 82%,  reduce = 0%
2013-09-17 23:50:55,943 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 27.87 sec
2013-09-17 23:50:56,949 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 27.87 sec
2013-09-17 23:50:57,955 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 27.87 sec
2013-09-17 23:50:59,680 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 27.87 sec
2013-09-17 23:51:00,685 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 61.27 sec
2013-09-17 23:51:01,690 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 61.27 sec
2013-09-17 23:51:02,695 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 61.27 sec
2013-09-17 23:51:03,700 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 61.27 sec
2013-09-17 23:51:04,705 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 61.27 sec
2013-09-17 23:51:05,710 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 61.27 sec
2013-09-17 23:51:06,715 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 61.27 sec
2013-09-17 23:51:07,719 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 61.27 sec
2013-09-17 23:51:08,725 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 61.27 sec
2013-09-17 23:51:09,730 Stage-2 map = 100%,  reduce = 74%, Cumulative CPU 61.27 sec
2013-09-17 23:51:10,735 Stage-2 map = 100%,  reduce = 74%, Cumulative CPU 61.27 sec
2013-09-17 23:51:11,740 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 71.17 sec
2013-09-17 23:51:12,746 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 71.17 sec
MapReduce Total cumulative CPU time: 1 minutes 11 seconds 170 msec
Ended Job = job_201309172235_0072
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 636.3 sec   HDFS Read: 200023859 HDFS Write: 499984003 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 71.17 sec   HDFS Read: 499987725 HDFS Write: 286 SUCCESS
Total MapReduce CPU Time Spent: 11 minutes 47 seconds 470 msec
OK
Time taken: 87.731 seconds, Fetched: 10 row(s)
hive> quit;
-- агрегация по числу и строке, большое количество ключей.;


times: 1
query: SELECT UserID, count(*) AS c FROM hits_100m GROUP BY UserID ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_10539@mturlrep13_201309172351_1364032363.txt
hive> SELECT UserID, count(*) AS c FROM hits_100m GROUP BY UserID ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0073
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:51:32,893 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:51:43,007 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:51:45,021 Stage-1 map = 4%,  reduce = 0%
2013-09-17 23:51:47,070 Stage-1 map = 11%,  reduce = 0%
2013-09-17 23:51:48,076 Stage-1 map = 14%,  reduce = 0%
2013-09-17 23:51:50,095 Stage-1 map = 33%,  reduce = 0%
2013-09-17 23:51:51,260 Stage-1 map = 39%,  reduce = 0%
2013-09-17 23:51:53,285 Stage-1 map = 51%,  reduce = 0%
2013-09-17 23:51:54,299 Stage-1 map = 58%,  reduce = 0%
2013-09-17 23:51:55,943 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 13.64 sec
2013-09-17 23:51:56,974 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 25.01 sec
2013-09-17 23:51:57,984 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 38.32 sec
2013-09-17 23:51:58,990 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 80.19 sec
2013-09-17 23:51:59,996 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 80.19 sec
2013-09-17 23:52:01,007 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 95.79 sec
2013-09-17 23:52:02,036 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 193.09 sec
2013-09-17 23:52:03,054 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 243.99 sec
2013-09-17 23:52:04,061 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 426.96 sec
2013-09-17 23:52:05,067 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 468.05 sec
2013-09-17 23:52:06,074 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 487.46 sec
2013-09-17 23:52:07,080 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 685.94 sec
2013-09-17 23:52:08,087 Stage-1 map = 100%,  reduce = 2%, Cumulative CPU 685.94 sec
2013-09-17 23:52:09,093 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 685.94 sec
2013-09-17 23:52:10,100 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 685.94 sec
2013-09-17 23:52:11,107 Stage-1 map = 100%,  reduce = 25%, Cumulative CPU 685.94 sec
2013-09-17 23:52:12,114 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 685.94 sec
2013-09-17 23:52:13,122 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 685.94 sec
2013-09-17 23:52:14,128 Stage-1 map = 100%,  reduce = 40%, Cumulative CPU 685.94 sec
2013-09-17 23:52:15,135 Stage-1 map = 100%,  reduce = 51%, Cumulative CPU 685.94 sec
2013-09-17 23:52:16,141 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 685.94 sec
2013-09-17 23:52:17,148 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 685.94 sec
2013-09-17 23:52:18,153 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 685.94 sec
2013-09-17 23:52:19,170 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 700.91 sec
2013-09-17 23:52:20,177 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 810.21 sec
2013-09-17 23:52:21,183 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 825.67 sec
2013-09-17 23:52:22,188 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 825.67 sec
2013-09-17 23:52:23,194 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 855.09 sec
2013-09-17 23:52:24,198 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 855.09 sec
MapReduce Total cumulative CPU time: 14 minutes 15 seconds 90 msec
Ended Job = job_201309172235_0073
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0074
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:52:27,714 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:52:40,754 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:52:48,779 Stage-2 map = 27%,  reduce = 0%
2013-09-17 23:52:49,783 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:52:52,793 Stage-2 map = 45%,  reduce = 0%
2013-09-17 23:52:55,803 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:53:01,889 Stage-2 map = 73%,  reduce = 0%
2013-09-17 23:53:07,908 Stage-2 map = 82%,  reduce = 0%
2013-09-17 23:53:13,929 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 55.32 sec
2013-09-17 23:53:14,935 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 55.32 sec
2013-09-17 23:53:15,940 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 55.32 sec
2013-09-17 23:53:16,946 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 55.32 sec
2013-09-17 23:53:17,951 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 55.32 sec
2013-09-17 23:53:18,956 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 55.32 sec
2013-09-17 23:53:19,961 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 118.9 sec
2013-09-17 23:53:20,966 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 118.9 sec
2013-09-17 23:53:21,971 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 118.9 sec
2013-09-17 23:53:22,975 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 118.9 sec
2013-09-17 23:53:23,979 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 118.9 sec
2013-09-17 23:53:24,983 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.9 sec
2013-09-17 23:53:25,987 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.9 sec
2013-09-17 23:53:26,992 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 123.85 sec
2013-09-17 23:53:27,996 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 123.85 sec
2013-09-17 23:53:29,001 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 123.85 sec
2013-09-17 23:53:30,005 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 123.85 sec
2013-09-17 23:53:31,010 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 123.85 sec
2013-09-17 23:53:32,014 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 123.85 sec
2013-09-17 23:53:33,019 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 123.85 sec
2013-09-17 23:53:34,029 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 133.7 sec
2013-09-17 23:53:35,034 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 133.7 sec
MapReduce Total cumulative CPU time: 2 minutes 13 seconds 700 msec
Ended Job = job_201309172235_0074
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 855.09 sec   HDFS Read: 663537300 HDFS Write: 480155183 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 133.7 sec   HDFS Read: 480158905 HDFS Write: 251 SUCCESS
Total MapReduce CPU Time Spent: 16 minutes 28 seconds 790 msec
OK
Time taken: 132.697 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT UserID, count(*) AS c FROM hits_100m GROUP BY UserID ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_17435@mturlrep13_201309172353_888901844.txt
hive> SELECT UserID, count(*) AS c FROM hits_100m GROUP BY UserID ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0075
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:53:45,965 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:53:57,063 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:53:58,076 Stage-1 map = 9%,  reduce = 0%
2013-09-17 23:53:59,095 Stage-1 map = 20%,  reduce = 0%
2013-09-17 23:54:00,108 Stage-1 map = 23%,  reduce = 0%
2013-09-17 23:54:01,121 Stage-1 map = 37%,  reduce = 0%
2013-09-17 23:54:02,132 Stage-1 map = 58%,  reduce = 0%
2013-09-17 23:54:03,151 Stage-1 map = 59%,  reduce = 0%
2013-09-17 23:54:04,163 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 10.96 sec
2013-09-17 23:54:05,184 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 51.58 sec
2013-09-17 23:54:06,196 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 65.4 sec
2013-09-17 23:54:07,205 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 123.94 sec
2013-09-17 23:54:08,310 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 137.81 sec
2013-09-17 23:54:09,332 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 234.2 sec
2013-09-17 23:54:10,342 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 283.15 sec
2013-09-17 23:54:11,350 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 399.5 sec
2013-09-17 23:54:12,356 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 459.78 sec
2013-09-17 23:54:13,365 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 479.7 sec
2013-09-17 23:54:14,372 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 479.7 sec
2013-09-17 23:54:15,378 Stage-1 map = 98%,  reduce = 5%, Cumulative CPU 553.18 sec
2013-09-17 23:54:16,383 Stage-1 map = 99%,  reduce = 17%, Cumulative CPU 654.99 sec
2013-09-17 23:54:17,388 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 681.14 sec
2013-09-17 23:54:18,394 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 681.14 sec
2013-09-17 23:54:19,399 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 681.14 sec
2013-09-17 23:54:20,405 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 681.14 sec
2013-09-17 23:54:21,411 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 681.14 sec
2013-09-17 23:54:22,417 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 681.14 sec
2013-09-17 23:54:23,423 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 681.14 sec
2013-09-17 23:54:24,430 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 681.14 sec
2013-09-17 23:54:25,442 Stage-1 map = 100%,  reduce = 56%, Cumulative CPU 681.14 sec
2013-09-17 23:54:26,448 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 681.14 sec
2013-09-17 23:54:27,456 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 758.79 sec
2013-09-17 23:54:28,462 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 758.79 sec
2013-09-17 23:54:29,468 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 758.79 sec
2013-09-17 23:54:30,475 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 758.79 sec
2013-09-17 23:54:31,481 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 792.19 sec
2013-09-17 23:54:32,487 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 841.44 sec
2013-09-17 23:54:33,493 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 849.83 sec
2013-09-17 23:54:34,499 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 849.83 sec
MapReduce Total cumulative CPU time: 14 minutes 9 seconds 830 msec
Ended Job = job_201309172235_0075
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0076
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:54:36,978 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:54:51,026 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:54:57,047 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:55:04,073 Stage-2 map = 47%,  reduce = 0%
2013-09-17 23:55:07,084 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:55:10,096 Stage-2 map = 65%,  reduce = 0%
2013-09-17 23:55:14,109 Stage-2 map = 73%,  reduce = 0%
2013-09-17 23:55:20,130 Stage-2 map = 82%,  reduce = 0%
2013-09-17 23:55:23,142 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 54.08 sec
2013-09-17 23:55:24,147 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 54.08 sec
2013-09-17 23:55:25,151 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 54.08 sec
2013-09-17 23:55:26,156 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 54.08 sec
2013-09-17 23:55:27,160 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 112.2 sec
2013-09-17 23:55:28,165 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 112.2 sec
2013-09-17 23:55:29,170 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 112.2 sec
2013-09-17 23:55:30,175 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 112.2 sec
2013-09-17 23:55:31,180 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 112.2 sec
2013-09-17 23:55:32,185 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 118.52 sec
2013-09-17 23:55:33,190 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 118.52 sec
2013-09-17 23:55:34,194 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 118.52 sec
2013-09-17 23:55:35,198 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 118.52 sec
2013-09-17 23:55:36,202 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 118.52 sec
2013-09-17 23:55:37,207 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 118.52 sec
2013-09-17 23:55:38,212 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 118.52 sec
2013-09-17 23:55:39,217 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 118.52 sec
2013-09-17 23:55:40,222 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.52 sec
2013-09-17 23:55:41,227 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.52 sec
2013-09-17 23:55:42,233 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.52 sec
2013-09-17 23:55:43,248 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.52 sec
2013-09-17 23:55:44,252 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.52 sec
2013-09-17 23:55:45,257 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.52 sec
2013-09-17 23:55:46,262 Stage-2 map = 100%,  reduce = 80%, Cumulative CPU 118.52 sec
2013-09-17 23:55:47,267 Stage-2 map = 100%,  reduce = 80%, Cumulative CPU 118.52 sec
2013-09-17 23:55:48,272 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 133.0 sec
2013-09-17 23:55:49,278 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 133.0 sec
MapReduce Total cumulative CPU time: 2 minutes 13 seconds 0 msec
Ended Job = job_201309172235_0076
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 849.83 sec   HDFS Read: 663537300 HDFS Write: 480155183 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 133.0 sec   HDFS Read: 480158905 HDFS Write: 251 SUCCESS
Total MapReduce CPU Time Spent: 16 minutes 22 seconds 830 msec
OK
Time taken: 132.306 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT UserID, count(*) AS c FROM hits_100m GROUP BY UserID ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_25039@mturlrep13_201309172355_777652129.txt
hive> SELECT UserID, count(*) AS c FROM hits_100m GROUP BY UserID ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0077
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:55:58,731 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:56:11,863 Stage-1 map = 12%,  reduce = 0%
2013-09-17 23:56:12,877 Stage-1 map = 20%,  reduce = 0%
2013-09-17 23:56:13,887 Stage-1 map = 22%,  reduce = 0%
2013-09-17 23:56:14,900 Stage-1 map = 40%,  reduce = 0%
2013-09-17 23:56:15,915 Stage-1 map = 57%,  reduce = 0%
2013-09-17 23:56:16,938 Stage-1 map = 58%,  reduce = 0%, Cumulative CPU 10.56 sec
2013-09-17 23:56:17,950 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 10.56 sec
2013-09-17 23:56:18,963 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 10.56 sec
2013-09-17 23:56:19,993 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 52.39 sec
2013-09-17 23:56:20,999 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 109.68 sec
2013-09-17 23:56:22,006 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 171.28 sec
2013-09-17 23:56:23,012 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 218.43 sec
2013-09-17 23:56:24,036 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 285.81 sec
2013-09-17 23:56:25,051 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 343.66 sec
2013-09-17 23:56:26,057 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 482.51 sec
2013-09-17 23:56:27,063 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 635.88 sec
2013-09-17 23:56:28,070 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 635.88 sec
2013-09-17 23:56:29,077 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 651.74 sec
2013-09-17 23:56:30,084 Stage-1 map = 100%,  reduce = 7%, Cumulative CPU 682.92 sec
2013-09-17 23:56:31,091 Stage-1 map = 100%,  reduce = 19%, Cumulative CPU 682.92 sec
2013-09-17 23:56:32,097 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 682.92 sec
2013-09-17 23:56:33,104 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 682.92 sec
2013-09-17 23:56:34,110 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 682.92 sec
2013-09-17 23:56:35,117 Stage-1 map = 100%,  reduce = 40%, Cumulative CPU 682.92 sec
2013-09-17 23:56:36,123 Stage-1 map = 100%,  reduce = 40%, Cumulative CPU 682.92 sec
2013-09-17 23:56:37,129 Stage-1 map = 100%,  reduce = 40%, Cumulative CPU 682.92 sec
2013-09-17 23:56:38,136 Stage-1 map = 100%,  reduce = 43%, Cumulative CPU 682.92 sec
2013-09-17 23:56:39,142 Stage-1 map = 100%,  reduce = 54%, Cumulative CPU 682.92 sec
2013-09-17 23:56:40,150 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 682.92 sec
2013-09-17 23:56:41,155 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 682.92 sec
2013-09-17 23:56:42,163 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 725.63 sec
2013-09-17 23:56:43,170 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 725.63 sec
2013-09-17 23:56:44,176 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 725.63 sec
2013-09-17 23:56:45,182 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 755.1 sec
2013-09-17 23:56:46,188 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 785.47 sec
2013-09-17 23:56:47,195 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 845.55 sec
2013-09-17 23:56:48,201 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 845.55 sec
2013-09-17 23:56:49,207 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 845.55 sec
MapReduce Total cumulative CPU time: 14 minutes 5 seconds 550 msec
Ended Job = job_201309172235_0077
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0078
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-17 23:56:51,716 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:57:05,763 Stage-2 map = 18%,  reduce = 0%
2013-09-17 23:57:11,783 Stage-2 map = 37%,  reduce = 0%
2013-09-17 23:57:17,803 Stage-2 map = 55%,  reduce = 0%
2013-09-17 23:57:23,823 Stage-2 map = 73%,  reduce = 0%
2013-09-17 23:57:29,845 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 87.94 sec
2013-09-17 23:57:30,850 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 87.94 sec
2013-09-17 23:57:31,855 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 87.94 sec
2013-09-17 23:57:32,860 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 87.94 sec
2013-09-17 23:57:33,865 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 87.94 sec
2013-09-17 23:57:34,870 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 87.94 sec
2013-09-17 23:57:35,876 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 97.95 sec
2013-09-17 23:57:36,881 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 97.95 sec
2013-09-17 23:57:37,887 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 97.95 sec
2013-09-17 23:57:38,893 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 97.95 sec
2013-09-17 23:57:39,898 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 97.95 sec
2013-09-17 23:57:40,903 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 97.95 sec
2013-09-17 23:57:41,908 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 97.95 sec
2013-09-17 23:57:42,914 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 97.95 sec
2013-09-17 23:57:43,919 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 97.95 sec
2013-09-17 23:57:44,924 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 118.7 sec
2013-09-17 23:57:45,929 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 118.7 sec
2013-09-17 23:57:46,934 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 118.7 sec
2013-09-17 23:57:47,939 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 118.7 sec
2013-09-17 23:57:48,944 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 118.7 sec
2013-09-17 23:57:49,948 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.7 sec
2013-09-17 23:57:50,960 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.7 sec
2013-09-17 23:57:51,965 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.7 sec
2013-09-17 23:57:52,969 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.7 sec
2013-09-17 23:57:53,974 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.7 sec
2013-09-17 23:57:54,978 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.7 sec
2013-09-17 23:57:55,982 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.7 sec
2013-09-17 23:57:56,986 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.7 sec
2013-09-17 23:57:57,991 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 118.7 sec
2013-09-17 23:57:58,995 Stage-2 map = 100%,  reduce = 80%, Cumulative CPU 118.7 sec
2013-09-17 23:58:00,000 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 133.21 sec
2013-09-17 23:58:01,005 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 133.21 sec
2013-09-17 23:58:02,010 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 133.21 sec
MapReduce Total cumulative CPU time: 2 minutes 13 seconds 210 msec
Ended Job = job_201309172235_0078
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 845.55 sec   HDFS Read: 663537300 HDFS Write: 480155183 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 133.21 sec   HDFS Read: 480158905 HDFS Write: 251 SUCCESS
Total MapReduce CPU Time Spent: 16 minutes 18 seconds 760 msec
OK
Time taken: 130.91 seconds, Fetched: 10 row(s)
hive> quit;
-- агрегация по очень большому количеству ключей, может не хватить оперативки.;


times: 1
query: SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_31945@mturlrep13_201309172358_1169432593.txt
hive> SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0079
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-17 23:58:22,443 Stage-1 map = 0%,  reduce = 0%
2013-09-17 23:58:32,503 Stage-1 map = 1%,  reduce = 0%
2013-09-17 23:58:36,545 Stage-1 map = 12%,  reduce = 0%
2013-09-17 23:58:39,582 Stage-1 map = 18%,  reduce = 0%
2013-09-17 23:58:41,277 Stage-1 map = 26%,  reduce = 0%
2013-09-17 23:58:43,786 Stage-1 map = 40%,  reduce = 0%
2013-09-17 23:58:46,818 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 15.02 sec
2013-09-17 23:58:47,825 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 15.02 sec
2013-09-17 23:58:48,836 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 15.02 sec
2013-09-17 23:58:49,842 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 31.76 sec
2013-09-17 23:58:50,856 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 31.76 sec
2013-09-17 23:58:51,862 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 83.8 sec
2013-09-17 23:58:52,870 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 99.52 sec
2013-09-17 23:58:53,884 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 99.52 sec
2013-09-17 23:58:54,894 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 117.13 sec
2013-09-17 23:58:55,909 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 213.91 sec
2013-09-17 23:58:56,924 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 324.7 sec
2013-09-17 23:58:57,930 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 449.36 sec
2013-09-17 23:58:58,936 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 491.93 sec
2013-09-17 23:58:59,943 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 542.45 sec
2013-09-17 23:59:00,949 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 591.47 sec
2013-09-17 23:59:01,955 Stage-1 map = 97%,  reduce = 3%, Cumulative CPU 620.12 sec
2013-09-17 23:59:02,961 Stage-1 map = 99%,  reduce = 10%, Cumulative CPU 796.58 sec
2013-09-17 23:59:03,967 Stage-1 map = 99%,  reduce = 10%, Cumulative CPU 796.58 sec
2013-09-17 23:59:04,972 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 827.74 sec
2013-09-17 23:59:05,980 Stage-1 map = 100%,  reduce = 17%, Cumulative CPU 827.74 sec
2013-09-17 23:59:06,987 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 827.74 sec
2013-09-17 23:59:07,994 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 827.74 sec
2013-09-17 23:59:09,000 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 827.74 sec
2013-09-17 23:59:10,007 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 827.74 sec
2013-09-17 23:59:11,013 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 827.74 sec
2013-09-17 23:59:12,020 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 827.74 sec
2013-09-17 23:59:13,027 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 827.74 sec
2013-09-17 23:59:14,033 Stage-1 map = 100%,  reduce = 56%, Cumulative CPU 827.74 sec
2013-09-17 23:59:15,039 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 827.74 sec
2013-09-17 23:59:16,046 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 827.74 sec
2013-09-17 23:59:17,052 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 827.74 sec
2013-09-17 23:59:18,059 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 827.74 sec
2013-09-17 23:59:19,065 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 827.74 sec
2013-09-17 23:59:20,072 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 827.74 sec
2013-09-17 23:59:21,077 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 827.74 sec
2013-09-17 23:59:22,114 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 827.74 sec
2013-09-17 23:59:23,122 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 871.5 sec
2013-09-17 23:59:24,133 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1068.13 sec
2013-09-17 23:59:25,139 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1068.13 sec
2013-09-17 23:59:26,143 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1068.13 sec
MapReduce Total cumulative CPU time: 17 minutes 48 seconds 130 msec
Ended Job = job_201309172235_0079
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0080
Hadoop job information for Stage-2: number of mappers: 5; number of reducers: 1
2013-09-17 23:59:29,594 Stage-2 map = 0%,  reduce = 0%
2013-09-17 23:59:39,626 Stage-2 map = 6%,  reduce = 0%
2013-09-17 23:59:42,636 Stage-2 map = 28%,  reduce = 0%
2013-09-17 23:59:45,462 Stage-2 map = 39%,  reduce = 0%
2013-09-17 23:59:46,467 Stage-2 map = 44%,  reduce = 0%
2013-09-17 23:59:47,945 Stage-2 map = 52%,  reduce = 0%, Cumulative CPU 23.32 sec
2013-09-17 23:59:48,950 Stage-2 map = 58%,  reduce = 0%, Cumulative CPU 23.32 sec
2013-09-17 23:59:49,955 Stage-2 map = 58%,  reduce = 0%, Cumulative CPU 23.32 sec
2013-09-17 23:59:50,960 Stage-2 map = 58%,  reduce = 0%, Cumulative CPU 23.32 sec
2013-09-17 23:59:51,965 Stage-2 map = 63%,  reduce = 0%, Cumulative CPU 23.32 sec
2013-09-17 23:59:52,970 Stage-2 map = 63%,  reduce = 0%, Cumulative CPU 23.32 sec
2013-09-17 23:59:53,976 Stage-2 map = 63%,  reduce = 0%, Cumulative CPU 23.32 sec
2013-09-17 23:59:54,981 Stage-2 map = 78%,  reduce = 7%, Cumulative CPU 23.32 sec
2013-09-17 23:59:55,986 Stage-2 map = 78%,  reduce = 7%, Cumulative CPU 23.32 sec
2013-09-17 23:59:56,992 Stage-2 map = 78%,  reduce = 7%, Cumulative CPU 23.32 sec
2013-09-17 23:59:57,997 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 23.32 sec
2013-09-17 23:59:59,002 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 23.32 sec
2013-09-18 00:00:00,007 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 23.32 sec
2013-09-18 00:00:01,028 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 23.32 sec
2013-09-18 00:00:02,033 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 23.32 sec
2013-09-18 00:00:03,039 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 23.32 sec
2013-09-18 00:00:04,043 Stage-2 map = 87%,  reduce = 7%, Cumulative CPU 66.8 sec
2013-09-18 00:00:05,049 Stage-2 map = 91%,  reduce = 7%, Cumulative CPU 111.39 sec
2013-09-18 00:00:06,054 Stage-2 map = 91%,  reduce = 7%, Cumulative CPU 111.39 sec
2013-09-18 00:00:07,058 Stage-2 map = 95%,  reduce = 7%, Cumulative CPU 158.23 sec
2013-09-18 00:00:08,064 Stage-2 map = 95%,  reduce = 7%, Cumulative CPU 158.23 sec
2013-09-18 00:00:09,069 Stage-2 map = 100%,  reduce = 7%, Cumulative CPU 206.95 sec
2013-09-18 00:00:10,074 Stage-2 map = 100%,  reduce = 7%, Cumulative CPU 206.95 sec
2013-09-18 00:00:11,079 Stage-2 map = 100%,  reduce = 7%, Cumulative CPU 206.95 sec
2013-09-18 00:00:12,088 Stage-2 map = 100%,  reduce = 7%, Cumulative CPU 206.95 sec
2013-09-18 00:00:13,092 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:14,097 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:15,102 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:16,106 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:17,110 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:18,114 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:19,118 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:20,123 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:21,127 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:22,131 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:23,135 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:24,139 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:25,144 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:26,148 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:27,152 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.95 sec
2013-09-18 00:00:28,157 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 226.88 sec
2013-09-18 00:00:29,161 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 226.88 sec
2013-09-18 00:00:30,166 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 226.88 sec
2013-09-18 00:00:31,170 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 226.88 sec
2013-09-18 00:00:32,175 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 233.37 sec
2013-09-18 00:00:33,180 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 233.37 sec
MapReduce Total cumulative CPU time: 3 minutes 53 seconds 370 msec
Ended Job = job_201309172235_0080
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1068.13 sec   HDFS Read: 840872065 HDFS Write: 1196040653 SUCCESS
Job 1: Map: 5  Reduce: 1   Cumulative CPU: 233.37 sec   HDFS Read: 1196066509 HDFS Write: 261 SUCCESS
Total MapReduce CPU Time Spent: 21 minutes 41 seconds 500 msec
OK
Time taken: 141.357 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_8249@mturlrep13_201309180000_1745995125.txt
hive> SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0081
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:00:43,931 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:00:56,183 Stage-1 map = 6%,  reduce = 0%
2013-09-18 00:00:57,190 Stage-1 map = 7%,  reduce = 0%
2013-09-18 00:00:58,202 Stage-1 map = 11%,  reduce = 0%
2013-09-18 00:00:59,212 Stage-1 map = 29%,  reduce = 0%
2013-09-18 00:01:00,225 Stage-1 map = 34%,  reduce = 0%
2013-09-18 00:01:01,231 Stage-1 map = 37%,  reduce = 0%
2013-09-18 00:01:02,237 Stage-1 map = 45%,  reduce = 0%
2013-09-18 00:01:03,242 Stage-1 map = 58%,  reduce = 0%
2013-09-18 00:01:04,254 Stage-1 map = 62%,  reduce = 0%
2013-09-18 00:01:05,260 Stage-1 map = 66%,  reduce = 0%
2013-09-18 00:01:06,286 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 14.75 sec
2013-09-18 00:01:07,294 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 30.84 sec
2013-09-18 00:01:08,306 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 81.13 sec
2013-09-18 00:01:09,336 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 133.53 sec
2013-09-18 00:01:10,343 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 152.09 sec
2013-09-18 00:01:11,353 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 212.05 sec
2013-09-18 00:01:12,359 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 251.83 sec
2013-09-18 00:01:13,366 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 359.32 sec
2013-09-18 00:01:14,373 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 452.13 sec
2013-09-18 00:01:15,379 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 547.58 sec
2013-09-18 00:01:16,387 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 547.58 sec
2013-09-18 00:01:17,393 Stage-1 map = 97%,  reduce = 2%, Cumulative CPU 603.6 sec
2013-09-18 00:01:18,400 Stage-1 map = 98%,  reduce = 6%, Cumulative CPU 715.99 sec
2013-09-18 00:01:19,407 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 744.69 sec
2013-09-18 00:01:20,413 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 837.42 sec
2013-09-18 00:01:21,419 Stage-1 map = 100%,  reduce = 25%, Cumulative CPU 837.42 sec
2013-09-18 00:01:22,426 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 837.42 sec
2013-09-18 00:01:23,432 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 837.42 sec
2013-09-18 00:01:24,439 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 837.42 sec
2013-09-18 00:01:25,446 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 837.42 sec
2013-09-18 00:01:26,452 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 837.42 sec
2013-09-18 00:01:27,460 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 837.42 sec
2013-09-18 00:01:28,467 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 890.75 sec
2013-09-18 00:01:29,474 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 890.75 sec
2013-09-18 00:01:30,489 Stage-1 map = 100%,  reduce = 53%, Cumulative CPU 890.75 sec
2013-09-18 00:01:31,496 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 890.75 sec
2013-09-18 00:01:32,503 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 890.75 sec
2013-09-18 00:01:33,509 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 890.75 sec
2013-09-18 00:01:34,515 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 890.75 sec
2013-09-18 00:01:35,522 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 890.75 sec
2013-09-18 00:01:36,528 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 890.75 sec
2013-09-18 00:01:37,534 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 890.75 sec
2013-09-18 00:01:38,541 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 890.75 sec
2013-09-18 00:01:39,555 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 998.36 sec
2013-09-18 00:01:40,562 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 998.36 sec
2013-09-18 00:01:41,568 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1035.02 sec
2013-09-18 00:01:42,574 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1070.49 sec
2013-09-18 00:01:43,580 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1070.49 sec
MapReduce Total cumulative CPU time: 17 minutes 50 seconds 490 msec
Ended Job = job_201309172235_0081
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0082
Hadoop job information for Stage-2: number of mappers: 5; number of reducers: 1
2013-09-18 00:01:47,035 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:01:57,066 Stage-2 map = 6%,  reduce = 0%
2013-09-18 00:02:00,076 Stage-2 map = 28%,  reduce = 0%
2013-09-18 00:02:05,092 Stage-2 map = 49%,  reduce = 0%, Cumulative CPU 22.46 sec
2013-09-18 00:02:06,098 Stage-2 map = 49%,  reduce = 0%, Cumulative CPU 22.46 sec
2013-09-18 00:02:07,104 Stage-2 map = 49%,  reduce = 0%, Cumulative CPU 22.46 sec
2013-09-18 00:02:10,085 Stage-2 map = 52%,  reduce = 0%, Cumulative CPU 22.46 sec
2013-09-18 00:02:11,090 Stage-2 map = 52%,  reduce = 0%, Cumulative CPU 22.46 sec
2013-09-18 00:02:12,095 Stage-2 map = 52%,  reduce = 0%, Cumulative CPU 22.46 sec
2013-09-18 00:02:13,100 Stage-2 map = 58%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:14,106 Stage-2 map = 58%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:15,111 Stage-2 map = 58%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:16,117 Stage-2 map = 68%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:17,122 Stage-2 map = 68%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:18,128 Stage-2 map = 68%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:19,133 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:20,166 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:21,171 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:22,176 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:23,273 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:24,279 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:25,284 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 22.46 sec
2013-09-18 00:02:26,289 Stage-2 map = 91%,  reduce = 7%, Cumulative CPU 110.5 sec
2013-09-18 00:02:27,294 Stage-2 map = 91%,  reduce = 7%, Cumulative CPU 110.5 sec
2013-09-18 00:02:28,412 Stage-2 map = 91%,  reduce = 7%, Cumulative CPU 203.81 sec
2013-09-18 00:02:29,416 Stage-2 map = 100%,  reduce = 7%, Cumulative CPU 206.44 sec
2013-09-18 00:02:30,420 Stage-2 map = 100%,  reduce = 7%, Cumulative CPU 206.44 sec
2013-09-18 00:02:31,425 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 206.44 sec
2013-09-18 00:02:32,429 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 206.44 sec
2013-09-18 00:02:33,989 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 206.44 sec
2013-09-18 00:02:34,994 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 206.44 sec
2013-09-18 00:02:35,999 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 206.44 sec
2013-09-18 00:02:37,003 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 206.44 sec
2013-09-18 00:02:38,007 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 206.44 sec
2013-09-18 00:02:39,011 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 206.44 sec
2013-09-18 00:02:40,015 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.44 sec
2013-09-18 00:02:41,020 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.44 sec
2013-09-18 00:02:42,023 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.44 sec
2013-09-18 00:02:43,027 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.44 sec
2013-09-18 00:02:44,031 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.44 sec
2013-09-18 00:02:45,036 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.44 sec
2013-09-18 00:02:46,040 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.44 sec
2013-09-18 00:02:47,045 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.44 sec
2013-09-18 00:02:48,049 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.44 sec
2013-09-18 00:02:49,054 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 206.44 sec
2013-09-18 00:02:50,058 Stage-2 map = 100%,  reduce = 69%, Cumulative CPU 206.44 sec
2013-09-18 00:02:51,063 Stage-2 map = 100%,  reduce = 69%, Cumulative CPU 206.44 sec
2013-09-18 00:02:52,067 Stage-2 map = 100%,  reduce = 69%, Cumulative CPU 206.44 sec
2013-09-18 00:02:53,071 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 206.44 sec
2013-09-18 00:02:54,395 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 206.44 sec
2013-09-18 00:02:55,399 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 206.44 sec
2013-09-18 00:02:56,403 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 206.44 sec
2013-09-18 00:02:57,408 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 206.44 sec
2013-09-18 00:03:01,104 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 232.56 sec
2013-09-18 00:03:02,108 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 232.56 sec
MapReduce Total cumulative CPU time: 3 minutes 52 seconds 560 msec
Ended Job = job_201309172235_0082
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1070.49 sec   HDFS Read: 840872065 HDFS Write: 1196040653 SUCCESS
Job 1: Map: 5  Reduce: 1   Cumulative CPU: 232.56 sec   HDFS Read: 1196066509 HDFS Write: 261 SUCCESS
Total MapReduce CPU Time Spent: 21 minutes 43 seconds 50 msec
OK
Time taken: 147.059 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_15858@mturlrep13_201309180003_1278146290.txt
hive> SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0083
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:03:11,727 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:03:23,895 Stage-1 map = 2%,  reduce = 0%
2013-09-18 00:03:24,924 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:03:25,943 Stage-1 map = 6%,  reduce = 0%
2013-09-18 00:03:26,979 Stage-1 map = 13%,  reduce = 0%
2013-09-18 00:03:28,028 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 341.29 sec
2013-09-18 00:03:29,044 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 341.29 sec
2013-09-18 00:03:30,053 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 341.29 sec
2013-09-18 00:03:31,062 Stage-1 map = 54%,  reduce = 0%, Cumulative CPU 341.29 sec
2013-09-18 00:03:32,074 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 341.29 sec
2013-09-18 00:03:33,090 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 346.1 sec
2013-09-18 00:03:34,112 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 346.1 sec
2013-09-18 00:03:35,122 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 346.1 sec
2013-09-18 00:03:36,133 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 346.1 sec
2013-09-18 00:03:37,145 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 394.29 sec
2013-09-18 00:03:38,153 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 402.8 sec
2013-09-18 00:03:39,160 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 413.32 sec
2013-09-18 00:03:40,175 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 435.9 sec
2013-09-18 00:03:41,200 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 505.2 sec
2013-09-18 00:03:42,209 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 544.32 sec
2013-09-18 00:03:43,217 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 600.85 sec
2013-09-18 00:03:44,223 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 629.83 sec
2013-09-18 00:03:45,230 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 645.26 sec
2013-09-18 00:03:46,236 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 682.7 sec
2013-09-18 00:03:47,243 Stage-1 map = 98%,  reduce = 3%, Cumulative CPU 758.48 sec
2013-09-18 00:03:48,250 Stage-1 map = 99%,  reduce = 17%, Cumulative CPU 799.32 sec
2013-09-18 00:03:49,257 Stage-1 map = 100%,  reduce = 22%, Cumulative CPU 841.01 sec
2013-09-18 00:03:50,263 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 841.01 sec
2013-09-18 00:03:51,269 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 841.01 sec
2013-09-18 00:03:52,275 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 841.01 sec
2013-09-18 00:03:53,281 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 841.01 sec
2013-09-18 00:03:54,287 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 841.01 sec
2013-09-18 00:03:55,293 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 841.01 sec
2013-09-18 00:03:56,300 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 841.01 sec
2013-09-18 00:03:57,306 Stage-1 map = 100%,  reduce = 42%, Cumulative CPU 841.01 sec
2013-09-18 00:03:58,313 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 841.01 sec
2013-09-18 00:03:59,319 Stage-1 map = 100%,  reduce = 53%, Cumulative CPU 841.01 sec
2013-09-18 00:04:00,325 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 841.01 sec
2013-09-18 00:04:01,331 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 841.01 sec
2013-09-18 00:04:02,338 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 841.01 sec
2013-09-18 00:04:03,344 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 841.01 sec
2013-09-18 00:04:04,350 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 841.01 sec
2013-09-18 00:04:05,357 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 841.01 sec
2013-09-18 00:04:06,363 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 841.01 sec
2013-09-18 00:04:07,382 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 841.01 sec
2013-09-18 00:04:08,391 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 924.61 sec
2013-09-18 00:04:09,396 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1053.16 sec
2013-09-18 00:04:10,402 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1074.59 sec
2013-09-18 00:04:11,408 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1074.59 sec
MapReduce Total cumulative CPU time: 17 minutes 54 seconds 590 msec
Ended Job = job_201309172235_0083
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0084
Hadoop job information for Stage-2: number of mappers: 5; number of reducers: 1
2013-09-18 00:04:16,454 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:04:27,489 Stage-2 map = 6%,  reduce = 0%
2013-09-18 00:04:30,502 Stage-2 map = 28%,  reduce = 0%, Cumulative CPU 71.89 sec
2013-09-18 00:04:31,507 Stage-2 map = 28%,  reduce = 0%, Cumulative CPU 71.89 sec
2013-09-18 00:04:32,513 Stage-2 map = 28%,  reduce = 0%, Cumulative CPU 71.89 sec
2013-09-18 00:04:33,518 Stage-2 map = 28%,  reduce = 0%, Cumulative CPU 71.89 sec
2013-09-18 00:04:34,523 Stage-2 map = 41%,  reduce = 0%, Cumulative CPU 71.89 sec
2013-09-18 00:04:35,529 Stage-2 map = 49%,  reduce = 0%, Cumulative CPU 81.02 sec
2013-09-18 00:04:36,534 Stage-2 map = 49%,  reduce = 0%, Cumulative CPU 81.02 sec
2013-09-18 00:04:37,539 Stage-2 map = 49%,  reduce = 0%, Cumulative CPU 81.02 sec
2013-09-18 00:04:38,545 Stage-2 map = 58%,  reduce = 0%, Cumulative CPU 81.02 sec
2013-09-18 00:04:39,550 Stage-2 map = 58%,  reduce = 0%, Cumulative CPU 81.02 sec
2013-09-18 00:04:41,187 Stage-2 map = 68%,  reduce = 0%, Cumulative CPU 81.02 sec
2013-09-18 00:04:42,265 Stage-2 map = 68%,  reduce = 0%, Cumulative CPU 81.02 sec
2013-09-18 00:04:43,270 Stage-2 map = 68%,  reduce = 7%, Cumulative CPU 81.02 sec
2013-09-18 00:04:44,275 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 81.02 sec
2013-09-18 00:04:45,281 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 81.02 sec
2013-09-18 00:04:46,286 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 81.02 sec
2013-09-18 00:04:47,291 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 81.02 sec
2013-09-18 00:04:48,302 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 81.02 sec
2013-09-18 00:04:49,306 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 81.02 sec
2013-09-18 00:04:50,311 Stage-2 map = 84%,  reduce = 7%, Cumulative CPU 81.02 sec
2013-09-18 00:04:51,316 Stage-2 map = 87%,  reduce = 7%, Cumulative CPU 108.7 sec
2013-09-18 00:04:52,321 Stage-2 map = 91%,  reduce = 7%, Cumulative CPU 136.95 sec
2013-09-18 00:04:53,326 Stage-2 map = 91%,  reduce = 7%, Cumulative CPU 136.95 sec
2013-09-18 00:04:54,331 Stage-2 map = 91%,  reduce = 7%, Cumulative CPU 136.95 sec
2013-09-18 00:04:55,335 Stage-2 map = 95%,  reduce = 7%, Cumulative CPU 169.96 sec
2013-09-18 00:04:56,340 Stage-2 map = 100%,  reduce = 7%, Cumulative CPU 203.62 sec
2013-09-18 00:04:57,344 Stage-2 map = 100%,  reduce = 7%, Cumulative CPU 203.62 sec
2013-09-18 00:04:58,348 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 203.62 sec
2013-09-18 00:04:59,353 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 203.62 sec
2013-09-18 00:05:00,358 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 203.62 sec
2013-09-18 00:05:01,362 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 203.62 sec
2013-09-18 00:05:02,367 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 203.62 sec
2013-09-18 00:05:03,371 Stage-2 map = 100%,  reduce = 20%, Cumulative CPU 203.62 sec
2013-09-18 00:05:04,376 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:05,380 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:06,384 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:07,389 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:08,393 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:09,397 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:10,402 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:11,406 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:12,410 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:13,415 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:14,419 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:15,422 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 203.62 sec
2013-09-18 00:05:16,426 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 203.62 sec
2013-09-18 00:05:17,430 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 203.62 sec
2013-09-18 00:05:18,433 Stage-2 map = 100%,  reduce = 77%, Cumulative CPU 203.62 sec
2013-09-18 00:05:19,439 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 228.63 sec
2013-09-18 00:05:20,443 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 228.63 sec
2013-09-18 00:05:21,448 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 228.63 sec
2013-09-18 00:05:24,696 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 228.63 sec
MapReduce Total cumulative CPU time: 3 minutes 48 seconds 630 msec
Ended Job = job_201309172235_0084
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1074.59 sec   HDFS Read: 840872065 HDFS Write: 1196040653 SUCCESS
Job 1: Map: 5  Reduce: 1   Cumulative CPU: 228.63 sec   HDFS Read: 1196066487 HDFS Write: 261 SUCCESS
Total MapReduce CPU Time Spent: 21 minutes 43 seconds 220 msec
OK
Time taken: 140.75 seconds, Fetched: 10 row(s)
hive> quit;
-- ещё более сложная агрегация.;


times: 1
query: SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_24153@mturlrep13_201309180005_705271963.txt
hive> SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0085
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:05:46,233 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:05:56,352 Stage-1 map = 3%,  reduce = 0%
2013-09-18 00:05:59,390 Stage-1 map = 10%,  reduce = 0%
2013-09-18 00:06:00,398 Stage-1 map = 17%,  reduce = 0%
2013-09-18 00:06:02,423 Stage-1 map = 21%,  reduce = 0%
2013-09-18 00:06:03,429 Stage-1 map = 28%,  reduce = 0%
2013-09-18 00:06:04,764 Stage-1 map = 32%,  reduce = 0%
2013-09-18 00:06:05,770 Stage-1 map = 40%,  reduce = 0%
2013-09-18 00:06:06,782 Stage-1 map = 46%,  reduce = 0%
2013-09-18 00:06:07,789 Stage-1 map = 49%,  reduce = 0%
2013-09-18 00:06:08,936 Stage-1 map = 57%,  reduce = 0%
2013-09-18 00:06:09,962 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 31.6 sec
2013-09-18 00:06:11,308 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 48.37 sec
2013-09-18 00:06:12,316 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 48.37 sec
2013-09-18 00:06:13,322 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 70.19 sec
2013-09-18 00:06:14,328 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 89.75 sec
2013-09-18 00:06:15,335 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 106.46 sec
2013-09-18 00:06:16,340 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 181.45 sec
2013-09-18 00:06:17,350 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 234.9 sec
2013-09-18 00:06:18,357 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 234.9 sec
2013-09-18 00:06:19,363 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 376.71 sec
2013-09-18 00:06:20,369 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 461.07 sec
2013-09-18 00:06:21,375 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 534.4 sec
2013-09-18 00:06:22,382 Stage-1 map = 96%,  reduce = 1%, Cumulative CPU 584.98 sec
2013-09-18 00:06:23,388 Stage-1 map = 97%,  reduce = 5%, Cumulative CPU 641.71 sec
2013-09-18 00:06:24,393 Stage-1 map = 99%,  reduce = 5%, Cumulative CPU 730.93 sec
2013-09-18 00:06:25,398 Stage-1 map = 99%,  reduce = 5%, Cumulative CPU 792.13 sec
2013-09-18 00:06:26,404 Stage-1 map = 100%,  reduce = 16%, Cumulative CPU 822.69 sec
2013-09-18 00:06:27,410 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 822.69 sec
2013-09-18 00:06:28,416 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 822.69 sec
2013-09-18 00:06:29,422 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 853.5 sec
2013-09-18 00:06:30,427 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 853.5 sec
2013-09-18 00:06:31,434 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 853.5 sec
2013-09-18 00:06:32,440 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 853.5 sec
2013-09-18 00:06:33,447 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 890.94 sec
2013-09-18 00:06:34,456 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 928.82 sec
2013-09-18 00:06:35,461 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 928.82 sec
2013-09-18 00:06:36,466 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 928.82 sec
MapReduce Total cumulative CPU time: 15 minutes 28 seconds 820 msec
Ended Job = job_201309172235_0085
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 928.82 sec   HDFS Read: 840872065 HDFS Write: 5012 SUCCESS
Total MapReduce CPU Time Spent: 15 minutes 28 seconds 820 msec
OK
Time taken: 60.884 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_30696@mturlrep13_201309180006_357154034.txt
hive> SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0086
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:06:47,336 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:06:58,492 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:06:59,507 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:07:00,512 Stage-1 map = 8%,  reduce = 0%
2013-09-18 00:07:01,519 Stage-1 map = 11%,  reduce = 0%
2013-09-18 00:07:02,525 Stage-1 map = 30%,  reduce = 0%
2013-09-18 00:07:03,531 Stage-1 map = 33%,  reduce = 0%
2013-09-18 00:07:04,537 Stage-1 map = 36%,  reduce = 0%
2013-09-18 00:07:05,542 Stage-1 map = 51%,  reduce = 0%
2013-09-18 00:07:06,547 Stage-1 map = 56%,  reduce = 0%
2013-09-18 00:07:07,553 Stage-1 map = 60%,  reduce = 0%
2013-09-18 00:07:08,565 Stage-1 map = 66%,  reduce = 0%
2013-09-18 00:07:09,577 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 14.92 sec
2013-09-18 00:07:10,584 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 46.57 sec
2013-09-18 00:07:11,590 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 80.51 sec
2013-09-18 00:07:12,596 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 131.92 sec
2013-09-18 00:07:13,713 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 131.92 sec
2013-09-18 00:07:14,736 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 152.9 sec
2013-09-18 00:07:15,752 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 254.96 sec
2013-09-18 00:07:16,766 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 338.32 sec
2013-09-18 00:07:17,773 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 429.15 sec
2013-09-18 00:07:18,778 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 523.97 sec
2013-09-18 00:07:19,784 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 548.71 sec
2013-09-18 00:07:20,791 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 604.14 sec
2013-09-18 00:07:21,798 Stage-1 map = 97%,  reduce = 2%, Cumulative CPU 660.44 sec
2013-09-18 00:07:22,804 Stage-1 map = 99%,  reduce = 13%, Cumulative CPU 747.55 sec
2013-09-18 00:07:23,810 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 839.86 sec
2013-09-18 00:07:24,816 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 839.86 sec
2013-09-18 00:07:25,822 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 839.86 sec
2013-09-18 00:07:26,829 Stage-1 map = 100%,  reduce = 25%, Cumulative CPU 839.86 sec
2013-09-18 00:07:27,835 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 839.86 sec
2013-09-18 00:07:28,841 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 839.86 sec
2013-09-18 00:07:29,847 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 871.07 sec
2013-09-18 00:07:30,853 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 871.07 sec
2013-09-18 00:07:31,859 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 871.07 sec
2013-09-18 00:07:32,868 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 898.81 sec
2013-09-18 00:07:33,873 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 949.24 sec
2013-09-18 00:07:34,879 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 949.24 sec
2013-09-18 00:07:35,885 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 949.24 sec
MapReduce Total cumulative CPU time: 15 minutes 49 seconds 240 msec
Ended Job = job_201309172235_0086
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 949.24 sec   HDFS Read: 840872065 HDFS Write: 5012 SUCCESS
Total MapReduce CPU Time Spent: 15 minutes 49 seconds 240 msec
OK
Time taken: 57.376 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_4908@mturlrep13_201309180007_865221313.txt
hive> SELECT UserID, SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, SearchPhrase LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0087
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:07:46,553 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:07:56,648 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:07:58,657 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:07:59,663 Stage-1 map = 7%,  reduce = 0%
2013-09-18 00:08:00,669 Stage-1 map = 12%,  reduce = 0%
2013-09-18 00:08:01,675 Stage-1 map = 29%,  reduce = 0%
2013-09-18 00:08:02,699 Stage-1 map = 32%,  reduce = 0%
2013-09-18 00:08:03,714 Stage-1 map = 37%,  reduce = 0%
2013-09-18 00:08:04,734 Stage-1 map = 46%,  reduce = 0%
2013-09-18 00:08:05,741 Stage-1 map = 55%,  reduce = 0%
2013-09-18 00:08:06,749 Stage-1 map = 61%,  reduce = 0%
2013-09-18 00:08:07,780 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 14.11 sec
2013-09-18 00:08:08,868 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 14.11 sec
2013-09-18 00:08:09,882 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 14.11 sec
2013-09-18 00:08:10,889 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 64.17 sec
2013-09-18 00:08:11,894 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 133.33 sec
2013-09-18 00:08:12,901 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 152.87 sec
2013-09-18 00:08:13,906 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 152.87 sec
2013-09-18 00:08:14,926 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 318.88 sec
2013-09-18 00:08:15,942 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 382.36 sec
2013-09-18 00:08:16,947 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 498.71 sec
2013-09-18 00:08:17,953 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 522.53 sec
2013-09-18 00:08:18,958 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 546.88 sec
2013-09-18 00:08:19,985 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 603.23 sec
2013-09-18 00:08:20,991 Stage-1 map = 97%,  reduce = 2%, Cumulative CPU 630.86 sec
2013-09-18 00:08:21,996 Stage-1 map = 99%,  reduce = 13%, Cumulative CPU 777.69 sec
2013-09-18 00:08:23,002 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 839.76 sec
2013-09-18 00:08:24,008 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 839.76 sec
2013-09-18 00:08:25,013 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 839.76 sec
2013-09-18 00:08:26,019 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 839.76 sec
2013-09-18 00:08:27,026 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 839.76 sec
2013-09-18 00:08:28,032 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 839.76 sec
2013-09-18 00:08:29,038 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 839.76 sec
2013-09-18 00:08:30,044 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 872.39 sec
2013-09-18 00:08:31,050 Stage-1 map = 100%,  reduce = 38%, Cumulative CPU 872.39 sec
2013-09-18 00:08:32,059 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 881.87 sec
2013-09-18 00:08:33,065 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 922.01 sec
2013-09-18 00:08:34,070 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 945.45 sec
2013-09-18 00:08:35,076 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 945.45 sec
MapReduce Total cumulative CPU time: 15 minutes 45 seconds 450 msec
Ended Job = job_201309172235_0087
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 945.45 sec   HDFS Read: 840872065 HDFS Write: 5012 SUCCESS
Total MapReduce CPU Time Spent: 15 minutes 45 seconds 450 msec
OK
Time taken: 57.367 seconds, Fetched: 10 row(s)
hive> quit;
-- то же самое, но без сортировки.;


times: 1
query: SELECT UserID, minute(EventTime), SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, minute(EventTime), SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_11923@mturlrep13_201309180008_1140241874.txt
hive> SELECT UserID, minute(EventTime), SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, minute(EventTime), SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0088
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:09:03,020 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:09:16,111 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:09:17,117 Stage-1 map = 2%,  reduce = 0%
2013-09-18 00:09:19,132 Stage-1 map = 3%,  reduce = 0%
2013-09-18 00:09:20,138 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:09:21,154 Stage-1 map = 8%,  reduce = 0%
2013-09-18 00:09:22,176 Stage-1 map = 11%,  reduce = 0%
2013-09-18 00:09:23,188 Stage-1 map = 12%,  reduce = 0%
2013-09-18 00:09:24,193 Stage-1 map = 15%,  reduce = 0%
2013-09-18 00:09:25,203 Stage-1 map = 22%,  reduce = 0%
2013-09-18 00:09:26,210 Stage-1 map = 24%,  reduce = 0%
2013-09-18 00:09:27,313 Stage-1 map = 27%,  reduce = 0%
2013-09-18 00:09:28,319 Stage-1 map = 34%,  reduce = 0%
2013-09-18 00:09:29,325 Stage-1 map = 37%,  reduce = 0%
2013-09-18 00:09:30,337 Stage-1 map = 38%,  reduce = 0%, Cumulative CPU 616.5 sec
2013-09-18 00:09:31,344 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 616.5 sec
2013-09-18 00:09:32,351 Stage-1 map = 45%,  reduce = 0%, Cumulative CPU 616.5 sec
2013-09-18 00:09:33,366 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 616.5 sec
2013-09-18 00:09:34,373 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 616.5 sec
2013-09-18 00:09:35,382 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 628.21 sec
2013-09-18 00:09:36,418 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 628.21 sec
2013-09-18 00:09:37,437 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 628.21 sec
2013-09-18 00:09:38,446 Stage-1 map = 62%,  reduce = 0%, Cumulative CPU 628.21 sec
2013-09-18 00:09:39,465 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 628.21 sec
2013-09-18 00:09:40,473 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 628.21 sec
2013-09-18 00:09:41,481 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 628.21 sec
2013-09-18 00:09:42,488 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 641.75 sec
2013-09-18 00:09:43,500 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 655.91 sec
2013-09-18 00:09:44,507 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 655.91 sec
2013-09-18 00:09:45,524 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 655.91 sec
2013-09-18 00:09:46,548 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 710.09 sec
2013-09-18 00:09:47,560 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 729.24 sec
2013-09-18 00:09:48,567 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 746.21 sec
2013-09-18 00:09:49,578 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 781.66 sec
2013-09-18 00:09:50,585 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 893.41 sec
2013-09-18 00:09:51,590 Stage-1 map = 94%,  reduce = 1%, Cumulative CPU 997.39 sec
2013-09-18 00:09:52,799 Stage-1 map = 96%,  reduce = 5%, Cumulative CPU 1063.45 sec
2013-09-18 00:09:53,805 Stage-1 map = 98%,  reduce = 7%, Cumulative CPU 1196.08 sec
2013-09-18 00:09:54,811 Stage-1 map = 99%,  reduce = 12%, Cumulative CPU 1239.77 sec
2013-09-18 00:09:55,817 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 1264.22 sec
2013-09-18 00:09:56,822 Stage-1 map = 99%,  reduce = 20%, Cumulative CPU 1289.61 sec
2013-09-18 00:09:57,828 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 1316.56 sec
2013-09-18 00:09:58,847 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 1316.56 sec
2013-09-18 00:09:59,853 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:00,859 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:01,866 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:02,872 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:03,879 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:04,885 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:05,892 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:06,899 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:07,908 Stage-1 map = 100%,  reduce = 42%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:08,915 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:09,921 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:11,093 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:12,099 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:13,105 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:14,111 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:15,117 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:19,314 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:20,320 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:21,802 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:22,808 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:23,814 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:24,820 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:25,825 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:27,283 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:28,288 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:29,293 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1316.56 sec
2013-09-18 00:10:30,301 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:31,307 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:32,316 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:33,323 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:34,609 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:35,615 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:36,620 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:37,750 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:38,756 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:39,762 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:40,767 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:41,772 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:43,972 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:44,978 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:45,984 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:46,990 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:48,166 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:49,171 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:50,176 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1610.02 sec
2013-09-18 00:10:51,182 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1776.03 sec
2013-09-18 00:10:52,188 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1795.3 sec
2013-09-18 00:10:53,193 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1795.3 sec
MapReduce Total cumulative CPU time: 29 minutes 55 seconds 300 msec
Ended Job = job_201309172235_0088
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0089
Hadoop job information for Stage-2: number of mappers: 9; number of reducers: 1
2013-09-18 00:10:56,790 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:11:13,933 Stage-2 map = 22%,  reduce = 0%, Cumulative CPU 22.42 sec
2013-09-18 00:11:14,937 Stage-2 map = 22%,  reduce = 0%, Cumulative CPU 22.42 sec
2013-09-18 00:11:15,942 Stage-2 map = 22%,  reduce = 0%, Cumulative CPU 22.42 sec
2013-09-18 00:11:16,948 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 22.42 sec
2013-09-18 00:11:17,953 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 22.42 sec
2013-09-18 00:11:18,959 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 22.42 sec
2013-09-18 00:11:19,964 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 22.42 sec
2013-09-18 00:11:20,969 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 22.42 sec
2013-09-18 00:11:21,974 Stage-2 map = 33%,  reduce = 4%, Cumulative CPU 22.42 sec
2013-09-18 00:11:22,980 Stage-2 map = 47%,  reduce = 4%, Cumulative CPU 22.42 sec
2013-09-18 00:11:23,985 Stage-2 map = 47%,  reduce = 4%, Cumulative CPU 22.42 sec
2013-09-18 00:11:24,991 Stage-2 map = 47%,  reduce = 4%, Cumulative CPU 22.42 sec
2013-09-18 00:11:25,996 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.42 sec
2013-09-18 00:11:27,001 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.42 sec
2013-09-18 00:11:28,007 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.42 sec
2013-09-18 00:11:29,012 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.42 sec
2013-09-18 00:11:30,017 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:31,022 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:32,027 Stage-2 map = 67%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:33,031 Stage-2 map = 67%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:34,036 Stage-2 map = 67%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:35,040 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:36,045 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:37,050 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:38,080 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:39,085 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:40,099 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:41,104 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:42,109 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:43,114 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:44,120 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:45,125 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 339.47 sec
2013-09-18 00:11:46,130 Stage-2 map = 81%,  reduce = 4%, Cumulative CPU 358.69 sec
2013-09-18 00:11:47,135 Stage-2 map = 89%,  reduce = 4%, Cumulative CPU 418.36 sec
2013-09-18 00:11:48,139 Stage-2 map = 89%,  reduce = 4%, Cumulative CPU 418.36 sec
2013-09-18 00:11:49,144 Stage-2 map = 89%,  reduce = 4%, Cumulative CPU 418.36 sec
2013-09-18 00:11:50,150 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 510.55 sec
2013-09-18 00:11:51,155 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 510.55 sec
2013-09-18 00:11:52,160 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 510.55 sec
2013-09-18 00:11:53,164 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 510.55 sec
2013-09-18 00:11:54,169 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 510.55 sec
2013-09-18 00:11:55,174 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 510.55 sec
2013-09-18 00:11:56,178 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 510.55 sec
2013-09-18 00:11:57,181 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 510.55 sec
2013-09-18 00:11:58,185 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 510.55 sec
2013-09-18 00:11:59,189 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 510.55 sec
2013-09-18 00:12:00,194 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 510.55 sec
2013-09-18 00:12:01,199 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 510.55 sec
2013-09-18 00:12:02,202 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 510.55 sec
2013-09-18 00:12:03,206 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 510.55 sec
2013-09-18 00:12:04,210 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 510.55 sec
2013-09-18 00:12:05,214 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 510.55 sec
2013-09-18 00:12:06,218 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 510.55 sec
2013-09-18 00:12:07,222 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:08,226 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:09,230 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:10,234 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:11,238 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:12,241 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:13,245 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:14,250 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:15,253 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:16,257 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:17,261 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:18,265 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:19,826 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:20,830 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:21,833 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:24,804 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 510.55 sec
2013-09-18 00:12:31,735 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 545.43 sec
2013-09-18 00:12:32,740 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 545.43 sec
2013-09-18 00:12:33,744 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 545.43 sec
2013-09-18 00:12:34,748 Stage-2 map = 100%,  reduce = 71%, Cumulative CPU 545.43 sec
2013-09-18 00:12:35,753 Stage-2 map = 100%,  reduce = 71%, Cumulative CPU 545.43 sec
2013-09-18 00:12:36,757 Stage-2 map = 100%,  reduce = 71%, Cumulative CPU 545.43 sec
2013-09-18 00:12:37,762 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 545.43 sec
2013-09-18 00:12:39,183 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 545.43 sec
2013-09-18 00:12:40,188 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 545.43 sec
2013-09-18 00:12:41,192 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 545.43 sec
2013-09-18 00:12:42,196 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 545.43 sec
2013-09-18 00:12:43,622 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 545.43 sec
2013-09-18 00:12:44,627 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 545.43 sec
2013-09-18 00:12:45,631 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 545.43 sec
2013-09-18 00:12:46,636 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 545.43 sec
2013-09-18 00:12:47,640 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 559.08 sec
2013-09-18 00:12:48,644 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 559.08 sec
2013-09-18 00:12:49,648 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 559.08 sec
MapReduce Total cumulative CPU time: 9 minutes 19 seconds 80 msec
Ended Job = job_201309172235_0089
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1795.3 sec   HDFS Read: 843455069 HDFS Write: 2238428518 SUCCESS
Job 1: Map: 9  Reduce: 1   Cumulative CPU: 559.08 sec   HDFS Read: 2238513761 HDFS Write: 278 SUCCESS
Total MapReduce CPU Time Spent: 39 minutes 14 seconds 380 msec
OK
Time taken: 237.554 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT UserID, minute(EventTime), SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, minute(EventTime), SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_21185@mturlrep13_201309180012_506617682.txt
hive> SELECT UserID, minute(EventTime), SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, minute(EventTime), SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0090
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:12:59,255 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:13:15,399 Stage-1 map = 2%,  reduce = 0%
2013-09-18 00:13:16,405 Stage-1 map = 8%,  reduce = 0%
2013-09-18 00:13:18,423 Stage-1 map = 12%,  reduce = 0%
2013-09-18 00:13:19,429 Stage-1 map = 19%,  reduce = 0%
2013-09-18 00:13:21,449 Stage-1 map = 21%,  reduce = 0%
2013-09-18 00:13:22,454 Stage-1 map = 31%,  reduce = 0%
2013-09-18 00:13:23,475 Stage-1 map = 32%,  reduce = 0%
2013-09-18 00:13:25,512 Stage-1 map = 40%,  reduce = 0%
2013-09-18 00:13:26,534 Stage-1 map = 41%,  reduce = 0%
2013-09-18 00:13:28,545 Stage-1 map = 50%,  reduce = 0%
2013-09-18 00:13:29,557 Stage-1 map = 52%,  reduce = 0%
2013-09-18 00:13:30,834 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 21.31 sec
2013-09-18 00:13:31,841 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 814.57 sec
2013-09-18 00:13:32,847 Stage-1 map = 62%,  reduce = 0%, Cumulative CPU 814.57 sec
2013-09-18 00:13:33,857 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 816.31 sec
2013-09-18 00:13:34,872 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 816.31 sec
2013-09-18 00:13:35,886 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 816.31 sec
2013-09-18 00:13:36,894 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 816.31 sec
2013-09-18 00:13:37,900 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 824.05 sec
2013-09-18 00:13:38,913 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 830.96 sec
2013-09-18 00:13:39,925 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 830.96 sec
2013-09-18 00:13:40,931 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 830.96 sec
2013-09-18 00:13:41,938 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 830.96 sec
2013-09-18 00:13:42,978 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 865.43 sec
2013-09-18 00:13:43,986 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 875.5 sec
2013-09-18 00:13:44,998 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 908.75 sec
2013-09-18 00:13:46,004 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 1001.65 sec
2013-09-18 00:13:47,011 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 1063.43 sec
2013-09-18 00:13:48,018 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 1202.46 sec
2013-09-18 00:13:49,023 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1255.44 sec
2013-09-18 00:13:50,029 Stage-1 map = 99%,  reduce = 3%, Cumulative CPU 1272.82 sec
2013-09-18 00:13:51,035 Stage-1 map = 99%,  reduce = 7%, Cumulative CPU 1272.82 sec
2013-09-18 00:13:52,040 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 1316.55 sec
2013-09-18 00:13:53,046 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 1316.55 sec
2013-09-18 00:13:54,052 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 1316.55 sec
2013-09-18 00:13:55,065 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1316.55 sec
2013-09-18 00:13:56,071 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1316.55 sec
2013-09-18 00:13:57,077 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1316.55 sec
2013-09-18 00:13:58,083 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1316.55 sec
2013-09-18 00:13:59,109 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:00,116 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:01,122 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:02,127 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:03,133 Stage-1 map = 100%,  reduce = 40%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:04,139 Stage-1 map = 100%,  reduce = 40%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:05,145 Stage-1 map = 100%,  reduce = 43%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:06,152 Stage-1 map = 100%,  reduce = 50%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:07,158 Stage-1 map = 100%,  reduce = 59%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:08,164 Stage-1 map = 100%,  reduce = 66%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:09,702 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:10,707 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:11,713 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:12,719 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:13,725 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:14,731 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:15,959 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:16,964 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:17,970 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:18,976 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:19,982 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:21,672 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:24,777 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:25,782 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:26,787 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:30,291 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1316.55 sec
2013-09-18 00:14:34,570 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:35,576 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:36,582 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:37,588 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:38,594 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:39,628 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:40,634 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:41,640 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:42,645 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:43,651 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:44,657 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1638.29 sec
2013-09-18 00:14:47,811 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1648.84 sec
2013-09-18 00:14:48,817 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1725.98 sec
2013-09-18 00:14:49,823 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1725.98 sec
2013-09-18 00:14:54,762 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1725.98 sec
2013-09-18 00:14:56,039 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1784.82 sec
2013-09-18 00:14:57,044 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1784.82 sec
MapReduce Total cumulative CPU time: 29 minutes 44 seconds 820 msec
Ended Job = job_201309172235_0090
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0091
Hadoop job information for Stage-2: number of mappers: 9; number of reducers: 1
2013-09-18 00:15:00,513 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:15:17,637 Stage-2 map = 30%,  reduce = 0%
2013-09-18 00:15:18,642 Stage-2 map = 31%,  reduce = 0%, Cumulative CPU 21.69 sec
2013-09-18 00:15:19,647 Stage-2 map = 31%,  reduce = 0%, Cumulative CPU 21.69 sec
2013-09-18 00:15:20,652 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 21.69 sec
2013-09-18 00:15:21,657 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 21.69 sec
2013-09-18 00:15:22,662 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 21.69 sec
2013-09-18 00:15:23,667 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 21.69 sec
2013-09-18 00:15:24,672 Stage-2 map = 33%,  reduce = 0%, Cumulative CPU 21.69 sec
2013-09-18 00:15:25,678 Stage-2 map = 33%,  reduce = 4%, Cumulative CPU 21.69 sec
2013-09-18 00:15:26,683 Stage-2 map = 47%,  reduce = 4%, Cumulative CPU 21.69 sec
2013-09-18 00:15:27,687 Stage-2 map = 47%,  reduce = 4%, Cumulative CPU 21.69 sec
2013-09-18 00:15:28,692 Stage-2 map = 47%,  reduce = 4%, Cumulative CPU 21.69 sec
2013-09-18 00:15:29,697 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 21.69 sec
2013-09-18 00:15:30,702 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 21.69 sec
2013-09-18 00:15:31,707 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 21.69 sec
2013-09-18 00:15:32,712 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 21.69 sec
2013-09-18 00:15:33,717 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 21.69 sec
2013-09-18 00:15:34,722 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:35,728 Stage-2 map = 70%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:36,733 Stage-2 map = 70%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:37,738 Stage-2 map = 70%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:38,744 Stage-2 map = 72%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:39,749 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:40,754 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:41,906 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:42,911 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:43,916 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:44,921 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:45,951 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:46,956 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:47,961 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:48,977 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:49,982 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:50,988 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:52,053 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 343.13 sec
2013-09-18 00:15:53,058 Stage-2 map = 92%,  reduce = 4%, Cumulative CPU 444.1 sec
2013-09-18 00:15:54,063 Stage-2 map = 94%,  reduce = 4%, Cumulative CPU 466.73 sec
2013-09-18 00:15:55,109 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 513.25 sec
2013-09-18 00:15:56,114 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 513.25 sec
2013-09-18 00:15:57,119 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 513.25 sec
2013-09-18 00:15:58,124 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 513.25 sec
2013-09-18 00:15:59,129 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 513.25 sec
2013-09-18 00:16:00,133 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 513.25 sec
2013-09-18 00:16:01,138 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 513.25 sec
2013-09-18 00:16:02,142 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 513.25 sec
2013-09-18 00:16:03,148 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 513.25 sec
2013-09-18 00:16:04,152 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 513.25 sec
2013-09-18 00:16:05,157 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 513.25 sec
2013-09-18 00:16:06,161 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 513.25 sec
2013-09-18 00:16:07,165 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 513.25 sec
2013-09-18 00:16:08,170 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 513.25 sec
2013-09-18 00:16:09,174 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 513.25 sec
2013-09-18 00:16:10,179 Stage-2 map = 100%,  reduce = 30%, Cumulative CPU 513.25 sec
2013-09-18 00:16:11,183 Stage-2 map = 100%,  reduce = 30%, Cumulative CPU 513.25 sec
2013-09-18 00:16:12,188 Stage-2 map = 100%,  reduce = 30%, Cumulative CPU 513.25 sec
2013-09-18 00:16:13,191 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:14,196 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:15,200 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:16,204 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:18,269 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:19,273 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:20,276 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:21,281 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:27,554 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:28,559 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:29,564 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 513.25 sec
2013-09-18 00:16:33,769 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 513.25 sec
2013-09-18 00:16:34,774 Stage-2 map = 100%,  reduce = 71%, Cumulative CPU 552.24 sec
2013-09-18 00:16:35,778 Stage-2 map = 100%,  reduce = 71%, Cumulative CPU 552.24 sec
2013-09-18 00:16:36,782 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 552.24 sec
2013-09-18 00:16:37,787 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 552.24 sec
2013-09-18 00:16:38,792 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 552.24 sec
2013-09-18 00:16:39,796 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 552.24 sec
2013-09-18 00:16:40,801 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 552.24 sec
2013-09-18 00:16:41,805 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 552.24 sec
2013-09-18 00:16:42,809 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 552.24 sec
2013-09-18 00:16:43,814 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 552.24 sec
2013-09-18 00:16:44,819 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 562.94 sec
2013-09-18 00:16:45,823 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 562.94 sec
MapReduce Total cumulative CPU time: 9 minutes 22 seconds 940 msec
Ended Job = job_201309172235_0091
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1784.82 sec   HDFS Read: 843455069 HDFS Write: 2238428518 SUCCESS
Job 1: Map: 9  Reduce: 1   Cumulative CPU: 562.94 sec   HDFS Read: 2238513761 HDFS Write: 278 SUCCESS
Total MapReduce CPU Time Spent: 39 minutes 7 seconds 760 msec
OK
Time taken: 234.361 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT UserID, minute(EventTime), SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, minute(EventTime), SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_30407@mturlrep13_201309180016_1665474298.txt
hive> SELECT UserID, minute(EventTime), SearchPhrase, count(*) AS c FROM hits_100m GROUP BY UserID, minute(EventTime), SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0092
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:16:56,793 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:17:10,896 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:17:11,902 Stage-1 map = 4%,  reduce = 0%
2013-09-18 00:17:12,907 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:17:13,917 Stage-1 map = 10%,  reduce = 0%
2013-09-18 00:17:14,928 Stage-1 map = 15%,  reduce = 0%
2013-09-18 00:17:15,933 Stage-1 map = 17%,  reduce = 0%
2013-09-18 00:17:16,939 Stage-1 map = 20%,  reduce = 0%
2013-09-18 00:17:17,944 Stage-1 map = 24%,  reduce = 0%
2013-09-18 00:17:18,950 Stage-1 map = 27%,  reduce = 0%
2013-09-18 00:17:19,955 Stage-1 map = 31%,  reduce = 0%
2013-09-18 00:17:20,961 Stage-1 map = 33%,  reduce = 0%
2013-09-18 00:17:21,966 Stage-1 map = 37%,  reduce = 0%
2013-09-18 00:17:22,971 Stage-1 map = 40%,  reduce = 0%
2013-09-18 00:17:24,001 Stage-1 map = 44%,  reduce = 0%
2013-09-18 00:17:25,019 Stage-1 map = 47%,  reduce = 0%
2013-09-18 00:17:26,029 Stage-1 map = 50%,  reduce = 0%
2013-09-18 00:17:27,037 Stage-1 map = 53%,  reduce = 0%
2013-09-18 00:17:28,044 Stage-1 map = 58%,  reduce = 0%
2013-09-18 00:17:29,062 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 21.75 sec
2013-09-18 00:17:30,082 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 21.75 sec
2013-09-18 00:17:31,094 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 21.75 sec
2013-09-18 00:17:32,114 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 21.75 sec
2013-09-18 00:17:33,129 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 21.75 sec
2013-09-18 00:17:34,140 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 105.0 sec
2013-09-18 00:17:35,156 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 1037.71 sec
2013-09-18 00:17:36,167 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 1037.71 sec
2013-09-18 00:17:37,173 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 1037.71 sec
2013-09-18 00:17:38,180 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 1037.71 sec
2013-09-18 00:17:39,186 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 1037.71 sec
2013-09-18 00:17:40,198 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 1047.34 sec
2013-09-18 00:17:41,214 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 1064.75 sec
2013-09-18 00:17:42,227 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 1113.1 sec
2013-09-18 00:17:43,233 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 1183.03 sec
2013-09-18 00:17:44,240 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 1236.75 sec
2013-09-18 00:17:45,256 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1266.7 sec
2013-09-18 00:17:46,269 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1290.62 sec
2013-09-18 00:17:47,277 Stage-1 map = 99%,  reduce = 1%, Cumulative CPU 1290.62 sec
2013-09-18 00:17:48,283 Stage-1 map = 99%,  reduce = 3%, Cumulative CPU 1304.6 sec
2013-09-18 00:17:49,289 Stage-1 map = 100%,  reduce = 11%, Cumulative CPU 1320.25 sec
2013-09-18 00:17:50,296 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 1320.25 sec
2013-09-18 00:17:51,313 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 1320.25 sec
2013-09-18 00:17:52,319 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 1320.25 sec
2013-09-18 00:17:53,325 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1320.25 sec
2013-09-18 00:17:54,331 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1320.25 sec
2013-09-18 00:17:55,338 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1320.25 sec
2013-09-18 00:17:56,344 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1320.25 sec
2013-09-18 00:17:57,350 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1320.25 sec
2013-09-18 00:17:58,356 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1320.25 sec
2013-09-18 00:17:59,362 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:00,368 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:01,374 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:02,381 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:03,387 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:04,392 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:07,247 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:08,252 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:09,259 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:10,266 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:11,272 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:12,762 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:13,767 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:14,772 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:15,778 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:18,513 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:20,464 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:21,470 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:22,477 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:23,674 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:24,680 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:26,496 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:27,502 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:28,508 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:31,532 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:32,538 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:33,543 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1320.25 sec
2013-09-18 00:18:34,555 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1721.3 sec
2013-09-18 00:18:36,736 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1721.3 sec
2013-09-18 00:18:37,742 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1721.3 sec
2013-09-18 00:18:38,747 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1721.3 sec
2013-09-18 00:18:39,753 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1721.3 sec
2013-09-18 00:18:40,759 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1724.11 sec
2013-09-18 00:18:41,765 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1724.11 sec
2013-09-18 00:18:42,770 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1724.11 sec
2013-09-18 00:18:43,776 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1724.11 sec
2013-09-18 00:18:44,782 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1743.27 sec
2013-09-18 00:18:45,788 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1754.65 sec
2013-09-18 00:18:48,727 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1754.65 sec
2013-09-18 00:18:49,733 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1786.03 sec
2013-09-18 00:18:50,739 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1786.03 sec
2013-09-18 00:18:51,754 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1786.03 sec
MapReduce Total cumulative CPU time: 29 minutes 46 seconds 30 msec
Ended Job = job_201309172235_0092
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0093
Hadoop job information for Stage-2: number of mappers: 9; number of reducers: 1
2013-09-18 00:18:55,132 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:19:13,198 Stage-2 map = 33%,  reduce = 0%
2013-09-18 00:19:21,228 Stage-2 map = 33%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:22,233 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:23,238 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:24,243 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:25,248 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:26,252 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:27,257 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:28,262 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:29,267 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:30,271 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:31,276 Stage-2 map = 56%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:32,280 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:33,285 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:34,290 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 22.55 sec
2013-09-18 00:19:35,295 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 398.54 sec
2013-09-18 00:19:36,299 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 398.54 sec
2013-09-18 00:19:37,305 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 398.54 sec
2013-09-18 00:19:38,310 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 398.54 sec
2013-09-18 00:19:39,315 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 398.54 sec
2013-09-18 00:19:40,320 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 398.54 sec
2013-09-18 00:19:41,326 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 398.54 sec
2013-09-18 00:19:42,331 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 398.54 sec
2013-09-18 00:19:43,349 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 398.54 sec
2013-09-18 00:19:44,354 Stage-2 map = 78%,  reduce = 4%, Cumulative CPU 398.54 sec
2013-09-18 00:19:46,244 Stage-2 map = 84%,  reduce = 4%, Cumulative CPU 423.18 sec
2013-09-18 00:19:47,248 Stage-2 map = 94%,  reduce = 4%, Cumulative CPU 476.82 sec
2013-09-18 00:19:48,253 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 505.58 sec
2013-09-18 00:19:49,258 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 505.58 sec
2013-09-18 00:19:50,262 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 505.58 sec
2013-09-18 00:19:51,267 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 505.58 sec
2013-09-18 00:19:52,273 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 505.58 sec
2013-09-18 00:19:53,277 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 505.58 sec
2013-09-18 00:19:54,281 Stage-2 map = 100%,  reduce = 4%, Cumulative CPU 505.58 sec
2013-09-18 00:19:55,285 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 505.58 sec
2013-09-18 00:19:56,289 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 505.58 sec
2013-09-18 00:19:57,293 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 505.58 sec
2013-09-18 00:19:58,296 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 505.58 sec
2013-09-18 00:19:59,300 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 505.58 sec
2013-09-18 00:20:00,305 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 505.58 sec
2013-09-18 00:20:01,309 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 505.58 sec
2013-09-18 00:20:02,313 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 505.58 sec
2013-09-18 00:20:03,317 Stage-2 map = 100%,  reduce = 22%, Cumulative CPU 505.58 sec
2013-09-18 00:20:04,322 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 505.58 sec
2013-09-18 00:20:05,326 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 505.58 sec
2013-09-18 00:20:06,329 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 505.58 sec
2013-09-18 00:20:07,334 Stage-2 map = 100%,  reduce = 30%, Cumulative CPU 505.58 sec
2013-09-18 00:20:08,338 Stage-2 map = 100%,  reduce = 30%, Cumulative CPU 505.58 sec
2013-09-18 00:20:09,342 Stage-2 map = 100%,  reduce = 30%, Cumulative CPU 505.58 sec
2013-09-18 00:20:10,347 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:11,357 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:12,362 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:13,366 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:14,371 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:15,376 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:16,381 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:17,386 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:18,391 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:19,396 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:20,401 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:21,405 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:22,410 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:24,399 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:25,403 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:26,407 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 505.58 sec
2013-09-18 00:20:27,761 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 505.58 sec
2013-09-18 00:20:28,766 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 505.58 sec
2013-09-18 00:20:29,947 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 505.58 sec
2013-09-18 00:20:30,952 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 505.58 sec
2013-09-18 00:20:31,956 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 505.58 sec
2013-09-18 00:20:32,961 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 505.58 sec
2013-09-18 00:20:33,966 Stage-2 map = 100%,  reduce = 71%, Cumulative CPU 505.58 sec
2013-09-18 00:20:34,970 Stage-2 map = 100%,  reduce = 71%, Cumulative CPU 544.29 sec
2013-09-18 00:20:35,975 Stage-2 map = 100%,  reduce = 71%, Cumulative CPU 544.29 sec
2013-09-18 00:20:36,979 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 544.29 sec
2013-09-18 00:20:37,984 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 544.29 sec
2013-09-18 00:20:38,988 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 544.29 sec
2013-09-18 00:20:39,999 Stage-2 map = 100%,  reduce = 76%, Cumulative CPU 544.29 sec
2013-09-18 00:20:41,004 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 555.06 sec
2013-09-18 00:20:42,008 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 555.06 sec
MapReduce Total cumulative CPU time: 9 minutes 15 seconds 60 msec
Ended Job = job_201309172235_0093
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1786.03 sec   HDFS Read: 843455069 HDFS Write: 2238428518 SUCCESS
Job 1: Map: 9  Reduce: 1   Cumulative CPU: 555.06 sec   HDFS Read: 2238513761 HDFS Write: 278 SUCCESS
Total MapReduce CPU Time Spent: 39 minutes 1 seconds 90 msec
OK
Time taken: 234.271 seconds, Fetched: 10 row(s)
hive> quit;
-- ещё более сложная агрегация, не стоит выполнять на больших таблицах.;


times: 1
query: SELECT UserID FROM hits_100m WHERE UserID = 12345678901234567890;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_7682@mturlrep13_201309180020_537514801.txt
hive> SELECT UserID FROM hits_100m WHERE UserID = 12345678901234567890;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0094
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 0
2013-09-18 00:20:59,984 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:21:09,101 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:21:10,107 Stage-1 map = 16%,  reduce = 0%
2013-09-18 00:21:14,125 Stage-1 map = 32%,  reduce = 0%
2013-09-18 00:21:15,130 Stage-1 map = 34%,  reduce = 0%
2013-09-18 00:21:17,153 Stage-1 map = 35%,  reduce = 0%, Cumulative CPU 4.53 sec
2013-09-18 00:21:19,500 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 15.48 sec
2013-09-18 00:21:20,507 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 35.24 sec
2013-09-18 00:21:21,514 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 170.31 sec
2013-09-18 00:21:22,521 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 199.34 sec
2013-09-18 00:21:23,526 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 199.34 sec
MapReduce Total cumulative CPU time: 3 minutes 19 seconds 340 msec
Ended Job = job_201309172235_0094
MapReduce Jobs Launched: 
Job 0: Map: 37   Cumulative CPU: 199.34 sec   HDFS Read: 663537300 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 3 minutes 19 seconds 340 msec
OK
Time taken: 34.019 seconds
hive> quit;

times: 2
query: SELECT UserID FROM hits_100m WHERE UserID = 12345678901234567890;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_12482@mturlrep13_201309180021_1171483805.txt
hive> SELECT UserID FROM hits_100m WHERE UserID = 12345678901234567890;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0095
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 0
2013-09-18 00:21:32,957 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:21:43,193 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 9.02 sec
2013-09-18 00:21:44,200 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 148.9 sec
2013-09-18 00:21:45,208 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 193.53 sec
2013-09-18 00:21:46,214 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 193.53 sec
2013-09-18 00:21:47,220 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 193.53 sec
MapReduce Total cumulative CPU time: 3 minutes 13 seconds 530 msec
Ended Job = job_201309172235_0095
MapReduce Jobs Launched: 
Job 0: Map: 37   Cumulative CPU: 193.53 sec   HDFS Read: 663537300 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 3 minutes 13 seconds 530 msec
OK
Time taken: 21.834 seconds
hive> quit;

times: 3
query: SELECT UserID FROM hits_100m WHERE UserID = 12345678901234567890;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_17377@mturlrep13_201309180021_991857587.txt
hive> SELECT UserID FROM hits_100m WHERE UserID = 12345678901234567890;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0096
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 0
2013-09-18 00:21:56,800 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:22:07,146 Stage-1 map = 26%,  reduce = 0%, Cumulative CPU 4.76 sec
2013-09-18 00:22:08,183 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 190.17 sec
2013-09-18 00:22:09,190 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 196.69 sec
2013-09-18 00:22:10,195 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 196.69 sec
MapReduce Total cumulative CPU time: 3 minutes 16 seconds 690 msec
Ended Job = job_201309172235_0096
MapReduce Jobs Launched: 
Job 0: Map: 37   Cumulative CPU: 196.69 sec   HDFS Read: 663537300 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 3 minutes 16 seconds 690 msec
OK
Time taken: 21.08 seconds
hive> quit;
-- мощная фильтрация по столбцу типа UInt64.;


times: 1
query: SELECT count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%';
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_22377@mturlrep13_201309180022_1306519242.txt
hive> SELECT count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%';;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0097
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:22:28,083 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:22:37,160 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:22:38,170 Stage-1 map = 6%,  reduce = 0%
2013-09-18 00:22:42,194 Stage-1 map = 20%,  reduce = 0%
2013-09-18 00:22:46,226 Stage-1 map = 46%,  reduce = 0%
2013-09-18 00:22:47,241 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 10.63 sec
2013-09-18 00:22:48,259 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 28.62 sec
2013-09-18 00:22:49,271 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 38.64 sec
2013-09-18 00:22:50,278 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 61.43 sec
2013-09-18 00:22:51,542 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 244.69 sec
2013-09-18 00:22:52,548 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 321.75 sec
2013-09-18 00:22:53,555 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 417.5 sec
2013-09-18 00:22:54,560 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 417.5 sec
2013-09-18 00:22:55,566 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 417.5 sec
2013-09-18 00:22:56,572 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 417.5 sec
2013-09-18 00:22:57,581 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 420.38 sec
2013-09-18 00:22:58,586 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 420.38 sec
MapReduce Total cumulative CPU time: 7 minutes 0 seconds 380 msec
Ended Job = job_201309172235_0097
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 420.38 sec   HDFS Read: 882764316 HDFS Write: 5 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 0 seconds 380 msec
OK
5238
Time taken: 41.195 seconds, Fetched: 1 row(s)
hive> quit;

times: 2
query: SELECT count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%';
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_27348@mturlrep13_201309180023_143827785.txt
hive> SELECT count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%';;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0098
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:23:09,675 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:23:19,749 Stage-1 map = 10%,  reduce = 0%
2013-09-18 00:23:20,754 Stage-1 map = 12%,  reduce = 0%
2013-09-18 00:23:21,760 Stage-1 map = 13%,  reduce = 0%
2013-09-18 00:23:22,765 Stage-1 map = 30%,  reduce = 0%
2013-09-18 00:23:23,779 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 8.29 sec
2013-09-18 00:23:24,790 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 8.29 sec
2013-09-18 00:23:25,805 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 38.21 sec
2013-09-18 00:23:26,811 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 125.77 sec
2013-09-18 00:23:27,819 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 297.36 sec
2013-09-18 00:23:28,825 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 425.9 sec
2013-09-18 00:23:29,830 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 439.17 sec
2013-09-18 00:23:30,836 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 439.17 sec
2013-09-18 00:23:31,841 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 439.17 sec
2013-09-18 00:23:32,847 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 439.17 sec
2013-09-18 00:23:33,853 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 439.17 sec
2013-09-18 00:23:34,858 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 439.17 sec
2013-09-18 00:23:35,865 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 441.94 sec
2013-09-18 00:23:36,871 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 441.94 sec
MapReduce Total cumulative CPU time: 7 minutes 21 seconds 940 msec
Ended Job = job_201309172235_0098
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 441.94 sec   HDFS Read: 882764316 HDFS Write: 5 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 21 seconds 940 msec
OK
5238
Time taken: 36.443 seconds, Fetched: 1 row(s)
hive> quit;

times: 3
query: SELECT count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%';
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_32317@mturlrep13_201309180023_1607167986.txt
hive> SELECT count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%';;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0099
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:23:47,464 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:23:56,520 Stage-1 map = 3%,  reduce = 0%
2013-09-18 00:23:57,526 Stage-1 map = 11%,  reduce = 0%
2013-09-18 00:23:59,535 Stage-1 map = 15%,  reduce = 0%
2013-09-18 00:24:00,541 Stage-1 map = 38%,  reduce = 0%
2013-09-18 00:24:01,559 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 8.66 sec
2013-09-18 00:24:02,572 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 8.66 sec
2013-09-18 00:24:03,580 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 50.46 sec
2013-09-18 00:24:04,591 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 152.61 sec
2013-09-18 00:24:05,597 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 361.23 sec
2013-09-18 00:24:06,603 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 427.75 sec
2013-09-18 00:24:07,609 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 441.29 sec
2013-09-18 00:24:08,613 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 441.29 sec
2013-09-18 00:24:09,618 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 441.29 sec
2013-09-18 00:24:10,624 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 441.29 sec
2013-09-18 00:24:11,696 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 441.29 sec
2013-09-18 00:24:12,703 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 444.08 sec
2013-09-18 00:24:13,709 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 444.08 sec
2013-09-18 00:24:14,715 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 444.08 sec
MapReduce Total cumulative CPU time: 7 minutes 24 seconds 80 msec
Ended Job = job_201309172235_0099
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 444.08 sec   HDFS Read: 882764316 HDFS Write: 5 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 24 seconds 80 msec
OK
5238
Time taken: 35.979 seconds, Fetched: 1 row(s)
hive> quit;
-- фильтрация по поиску подстроки в строке.;


times: 1
query: SELECT SearchPhrase, MAX(URL), count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_4931@mturlrep13_201309180024_377998774.txt
hive> SELECT SearchPhrase, MAX(URL), count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0100
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:24:32,532 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:24:41,635 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:24:42,650 Stage-1 map = 4%,  reduce = 0%
2013-09-18 00:24:46,694 Stage-1 map = 15%,  reduce = 0%
2013-09-18 00:24:49,790 Stage-1 map = 39%,  reduce = 0%
2013-09-18 00:24:50,805 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 12.57 sec
2013-09-18 00:24:54,895 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 60.14 sec
2013-09-18 00:24:55,919 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 143.58 sec
2013-09-18 00:24:56,938 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 219.04 sec
2013-09-18 00:24:57,944 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 404.21 sec
2013-09-18 00:24:58,950 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 468.52 sec
2013-09-18 00:24:59,956 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 468.52 sec
2013-09-18 00:25:00,962 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 468.52 sec
2013-09-18 00:25:01,971 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 471.46 sec
2013-09-18 00:25:02,978 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 474.57 sec
2013-09-18 00:25:03,988 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 490.52 sec
2013-09-18 00:25:04,995 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 500.12 sec
2013-09-18 00:25:06,001 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 500.12 sec
2013-09-18 00:25:07,008 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 503.18 sec
2013-09-18 00:25:08,019 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 503.18 sec
MapReduce Total cumulative CPU time: 8 minutes 23 seconds 180 msec
Ended Job = job_201309172235_0100
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0101
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 00:25:11,520 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:25:13,528 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 00:25:14,534 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 00:25:15,540 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 00:25:16,545 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 00:25:17,549 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 00:25:18,554 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 00:25:19,559 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 00:25:20,564 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.73 sec
2013-09-18 00:25:21,569 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.17 sec
2013-09-18 00:25:22,574 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.17 sec
MapReduce Total cumulative CPU time: 2 seconds 170 msec
Ended Job = job_201309172235_0101
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 503.18 sec   HDFS Read: 1060099081 HDFS Write: 11615 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.17 sec   HDFS Read: 15192 HDFS Write: 1152 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 25 seconds 350 msec
OK
Time taken: 60.385 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT SearchPhrase, MAX(URL), count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_12874@mturlrep13_201309180025_385714226.txt
hive> SELECT SearchPhrase, MAX(URL), count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0102
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:25:33,396 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:25:43,564 Stage-1 map = 9%,  reduce = 0%
2013-09-18 00:25:45,601 Stage-1 map = 10%,  reduce = 0%
2013-09-18 00:25:46,613 Stage-1 map = 33%,  reduce = 0%
2013-09-18 00:25:47,620 Stage-1 map = 37%,  reduce = 0%
2013-09-18 00:25:48,634 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 9.38 sec
2013-09-18 00:25:49,642 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 9.38 sec
2013-09-18 00:25:50,658 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 9.38 sec
2013-09-18 00:25:51,664 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 21.89 sec
2013-09-18 00:25:52,694 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 218.15 sec
2013-09-18 00:25:53,701 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 479.2 sec
2013-09-18 00:25:54,708 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 493.05 sec
2013-09-18 00:25:55,715 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 493.05 sec
2013-09-18 00:25:56,721 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 493.05 sec
2013-09-18 00:25:57,727 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 493.05 sec
2013-09-18 00:25:58,733 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 493.05 sec
2013-09-18 00:25:59,740 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 493.05 sec
2013-09-18 00:26:00,748 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 493.05 sec
2013-09-18 00:26:01,758 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 510.23 sec
2013-09-18 00:26:02,768 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 524.01 sec
2013-09-18 00:26:03,775 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 530.35 sec
2013-09-18 00:26:04,781 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 530.35 sec
MapReduce Total cumulative CPU time: 8 minutes 50 seconds 350 msec
Ended Job = job_201309172235_0102
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0103
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 00:26:08,247 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:26:09,253 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:26:10,258 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:26:11,263 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:26:12,269 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:26:13,275 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:26:14,280 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:26:15,284 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:26:16,289 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:26:17,294 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.35 sec
2013-09-18 00:26:18,299 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.35 sec
2013-09-18 00:26:19,304 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.35 sec
MapReduce Total cumulative CPU time: 2 seconds 350 msec
Ended Job = job_201309172235_0103
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 530.35 sec   HDFS Read: 1060099081 HDFS Write: 11615 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.35 sec   HDFS Read: 15192 HDFS Write: 1152 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 52 seconds 700 msec
OK
Time taken: 54.888 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT SearchPhrase, MAX(URL), count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_19768@mturlrep13_201309180026_1068991002.txt
hive> SELECT SearchPhrase, MAX(URL), count(*) AS c FROM hits_100m WHERE URL LIKE '%metrika%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0104
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:26:30,060 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:26:39,192 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:26:40,200 Stage-1 map = 9%,  reduce = 0%
2013-09-18 00:26:41,206 Stage-1 map = 10%,  reduce = 0%
2013-09-18 00:26:42,212 Stage-1 map = 11%,  reduce = 0%
2013-09-18 00:26:43,218 Stage-1 map = 29%,  reduce = 0%
2013-09-18 00:26:44,224 Stage-1 map = 35%,  reduce = 0%
2013-09-18 00:26:45,239 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 9.36 sec
2013-09-18 00:26:46,268 Stage-1 map = 45%,  reduce = 0%, Cumulative CPU 9.36 sec
2013-09-18 00:26:47,275 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 9.36 sec
2013-09-18 00:26:48,316 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 21.78 sec
2013-09-18 00:26:49,334 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 217.46 sec
2013-09-18 00:26:50,341 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 406.39 sec
2013-09-18 00:26:51,348 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 477.62 sec
2013-09-18 00:26:52,356 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 491.94 sec
2013-09-18 00:26:53,362 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 491.94 sec
2013-09-18 00:26:54,368 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 491.94 sec
2013-09-18 00:26:55,374 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 491.94 sec
2013-09-18 00:26:56,380 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 491.94 sec
2013-09-18 00:26:57,388 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 491.94 sec
2013-09-18 00:26:58,397 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 504.87 sec
2013-09-18 00:26:59,404 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 518.69 sec
2013-09-18 00:27:00,410 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 528.53 sec
2013-09-18 00:27:01,416 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 528.53 sec
2013-09-18 00:27:02,423 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 528.53 sec
MapReduce Total cumulative CPU time: 8 minutes 48 seconds 530 msec
Ended Job = job_201309172235_0104
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0105
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 00:27:06,499 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:27:07,505 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 00:27:08,522 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 00:27:09,527 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 00:27:10,532 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 00:27:11,538 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 00:27:12,543 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 00:27:13,547 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 00:27:14,552 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 00:27:15,558 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.26 sec
2013-09-18 00:27:16,563 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.26 sec
2013-09-18 00:27:17,568 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.26 sec
MapReduce Total cumulative CPU time: 2 seconds 260 msec
Ended Job = job_201309172235_0105
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 528.53 sec   HDFS Read: 1060099081 HDFS Write: 11615 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.26 sec   HDFS Read: 15192 HDFS Write: 1152 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 50 seconds 790 msec
OK
Time taken: 56.541 seconds, Fetched: 10 row(s)
hive> quit;
-- вынимаем большие столбцы, фильтрация по строке.;


times: 1
query: SELECT SearchPhrase, MAX(URL), MAX(Title), count(*) AS c, count(DISTINCT UserID) FROM hits_100m WHERE Title LIKE '%Яндекс%' AND URL NOT LIKE '%.yandex.%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_26625@mturlrep13_201309180027_303480636.txt
hive> SELECT SearchPhrase, MAX(URL), MAX(Title), count(*) AS c, count(DISTINCT UserID) FROM hits_100m WHERE Title LIKE '%Яндекс%' AND URL NOT LIKE '%.yandex.%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0106
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:27:35,507 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:27:45,586 Stage-1 map = 2%,  reduce = 0%
2013-09-18 00:27:48,661 Stage-1 map = 10%,  reduce = 0%
2013-09-18 00:27:50,957 Stage-1 map = 14%,  reduce = 0%
2013-09-18 00:27:51,973 Stage-1 map = 20%,  reduce = 0%
2013-09-18 00:27:53,983 Stage-1 map = 22%,  reduce = 0%
2013-09-18 00:27:56,028 Stage-1 map = 35%,  reduce = 0%
2013-09-18 00:27:58,127 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 12.89 sec
2013-09-18 00:27:59,135 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 12.89 sec
2013-09-18 00:28:00,141 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 27.18 sec
2013-09-18 00:28:01,159 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 27.18 sec
2013-09-18 00:28:02,167 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 59.55 sec
2013-09-18 00:28:04,208 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 92.71 sec
2013-09-18 00:28:05,214 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 175.04 sec
2013-09-18 00:28:06,329 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 341.42 sec
2013-09-18 00:28:07,336 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 556.8 sec
2013-09-18 00:28:08,342 Stage-1 map = 100%,  reduce = 2%, Cumulative CPU 608.33 sec
2013-09-18 00:28:09,348 Stage-1 map = 100%,  reduce = 2%, Cumulative CPU 608.33 sec
2013-09-18 00:28:10,354 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 608.33 sec
2013-09-18 00:28:11,362 Stage-1 map = 100%,  reduce = 20%, Cumulative CPU 611.74 sec
2013-09-18 00:28:12,369 Stage-1 map = 100%,  reduce = 47%, Cumulative CPU 622.43 sec
2013-09-18 00:28:13,376 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 632.58 sec
2013-09-18 00:28:14,383 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 632.58 sec
2013-09-18 00:28:15,389 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 643.54 sec
2013-09-18 00:28:16,396 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 647.01 sec
2013-09-18 00:28:17,401 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 647.01 sec
MapReduce Total cumulative CPU time: 10 minutes 47 seconds 10 msec
Ended Job = job_201309172235_0106
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0107
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 00:28:19,874 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:28:21,883 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.61 sec
2013-09-18 00:28:22,887 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.61 sec
2013-09-18 00:28:23,892 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.61 sec
2013-09-18 00:28:24,896 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.61 sec
2013-09-18 00:28:25,900 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.61 sec
2013-09-18 00:28:26,904 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.61 sec
2013-09-18 00:28:27,908 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.61 sec
2013-09-18 00:28:28,912 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.61 sec
2013-09-18 00:28:29,917 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.61 sec
2013-09-18 00:28:30,923 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.4 sec
2013-09-18 00:28:31,928 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.4 sec
MapReduce Total cumulative CPU time: 3 seconds 400 msec
Ended Job = job_201309172235_0107
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 647.01 sec   HDFS Read: 2479854627 HDFS Write: 1092954 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.4 sec   HDFS Read: 1096531 HDFS Write: 1307 SUCCESS
Total MapReduce CPU Time Spent: 10 minutes 50 seconds 410 msec
OK
Time taken: 67.257 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT SearchPhrase, MAX(URL), MAX(Title), count(*) AS c, count(DISTINCT UserID) FROM hits_100m WHERE Title LIKE '%Яндекс%' AND URL NOT LIKE '%.yandex.%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_1658@mturlrep13_201309180028_324429197.txt
hive> SELECT SearchPhrase, MAX(URL), MAX(Title), count(*) AS c, count(DISTINCT UserID) FROM hits_100m WHERE Title LIKE '%Яндекс%' AND URL NOT LIKE '%.yandex.%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0108
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:28:42,812 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:28:52,951 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:28:54,963 Stage-1 map = 11%,  reduce = 0%
2013-09-18 00:28:55,970 Stage-1 map = 18%,  reduce = 0%
2013-09-18 00:28:56,983 Stage-1 map = 19%,  reduce = 0%
2013-09-18 00:28:57,990 Stage-1 map = 26%,  reduce = 0%
2013-09-18 00:28:59,003 Stage-1 map = 29%,  reduce = 0%
2013-09-18 00:29:00,009 Stage-1 map = 32%,  reduce = 0%
2013-09-18 00:29:01,027 Stage-1 map = 39%,  reduce = 0%
2013-09-18 00:29:02,034 Stage-1 map = 47%,  reduce = 0%
2013-09-18 00:29:03,043 Stage-1 map = 51%,  reduce = 0%
2013-09-18 00:29:04,091 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 30.86 sec
2013-09-18 00:29:05,114 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 63.18 sec
2013-09-18 00:29:06,133 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 233.25 sec
2013-09-18 00:29:07,141 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 514.05 sec
2013-09-18 00:29:08,148 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 641.77 sec
2013-09-18 00:29:09,154 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 641.77 sec
2013-09-18 00:29:10,160 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 641.77 sec
2013-09-18 00:29:11,166 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 641.77 sec
2013-09-18 00:29:12,173 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 641.77 sec
2013-09-18 00:29:13,179 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 641.77 sec
2013-09-18 00:29:14,186 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 641.77 sec
2013-09-18 00:29:15,211 Stage-1 map = 100%,  reduce = 45%, Cumulative CPU 648.85 sec
2013-09-18 00:29:16,218 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 679.72 sec
2013-09-18 00:29:17,224 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 683.49 sec
2013-09-18 00:29:18,230 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 683.49 sec
MapReduce Total cumulative CPU time: 11 minutes 23 seconds 490 msec
Ended Job = job_201309172235_0108
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0109
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 00:29:20,736 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:29:23,749 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.6 sec
2013-09-18 00:29:24,754 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.6 sec
2013-09-18 00:29:25,759 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.6 sec
2013-09-18 00:29:26,763 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.6 sec
2013-09-18 00:29:27,767 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.6 sec
2013-09-18 00:29:28,772 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.6 sec
2013-09-18 00:29:29,776 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.6 sec
2013-09-18 00:29:30,782 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.6 sec
2013-09-18 00:29:31,787 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.35 sec
2013-09-18 00:29:32,792 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.35 sec
MapReduce Total cumulative CPU time: 3 seconds 350 msec
Ended Job = job_201309172235_0109
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 683.49 sec   HDFS Read: 2479854627 HDFS Write: 1092954 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.35 sec   HDFS Read: 1096531 HDFS Write: 1307 SUCCESS
Total MapReduce CPU Time Spent: 11 minutes 26 seconds 840 msec
OK
Time taken: 58.995 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT SearchPhrase, MAX(URL), MAX(Title), count(*) AS c, count(DISTINCT UserID) FROM hits_100m WHERE Title LIKE '%Яндекс%' AND URL NOT LIKE '%.yandex.%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_9465@mturlrep13_201309180029_1752618536.txt
hive> SELECT SearchPhrase, MAX(URL), MAX(Title), count(*) AS c, count(DISTINCT UserID) FROM hits_100m WHERE Title LIKE '%Яндекс%' AND URL NOT LIKE '%.yandex.%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0110
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:29:42,551 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:29:53,652 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:29:56,667 Stage-1 map = 16%,  reduce = 0%
2013-09-18 00:29:57,673 Stage-1 map = 21%,  reduce = 0%
2013-09-18 00:29:58,678 Stage-1 map = 26%,  reduce = 0%
2013-09-18 00:29:59,684 Stage-1 map = 28%,  reduce = 0%
2013-09-18 00:30:00,690 Stage-1 map = 31%,  reduce = 0%
2013-09-18 00:30:01,707 Stage-1 map = 38%,  reduce = 0%
2013-09-18 00:30:02,717 Stage-1 map = 43%,  reduce = 0%
2013-09-18 00:30:03,733 Stage-1 map = 51%,  reduce = 0%
2013-09-18 00:30:04,774 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 14.24 sec
2013-09-18 00:30:05,800 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 45.73 sec
2013-09-18 00:30:06,808 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 128.94 sec
2013-09-18 00:30:07,818 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 335.92 sec
2013-09-18 00:30:08,826 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 549.46 sec
2013-09-18 00:30:09,832 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 641.18 sec
2013-09-18 00:30:10,838 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 641.18 sec
2013-09-18 00:30:11,844 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 641.18 sec
2013-09-18 00:30:12,850 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 641.18 sec
2013-09-18 00:30:13,856 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 641.18 sec
2013-09-18 00:30:14,863 Stage-1 map = 100%,  reduce = 15%, Cumulative CPU 641.18 sec
2013-09-18 00:30:15,874 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 644.72 sec
2013-09-18 00:30:16,882 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 667.58 sec
2013-09-18 00:30:17,888 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 682.62 sec
2013-09-18 00:30:18,895 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 682.62 sec
2013-09-18 00:30:19,901 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 682.62 sec
MapReduce Total cumulative CPU time: 11 minutes 22 seconds 620 msec
Ended Job = job_201309172235_0110
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0111
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 00:30:23,450 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:30:25,458 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.57 sec
2013-09-18 00:30:26,463 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.57 sec
2013-09-18 00:30:27,468 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.57 sec
2013-09-18 00:30:28,472 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.57 sec
2013-09-18 00:30:29,476 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.57 sec
2013-09-18 00:30:30,480 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.57 sec
2013-09-18 00:30:31,484 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 1.57 sec
2013-09-18 00:30:32,489 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 1.57 sec
2013-09-18 00:30:33,494 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.37 sec
2013-09-18 00:30:34,500 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 3.37 sec
MapReduce Total cumulative CPU time: 3 seconds 370 msec
Ended Job = job_201309172235_0111
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 682.62 sec   HDFS Read: 2479854627 HDFS Write: 1092954 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 3.37 sec   HDFS Read: 1096531 HDFS Write: 1307 SUCCESS
Total MapReduce CPU Time Spent: 11 minutes 25 seconds 990 msec
OK
Time taken: 59.828 seconds, Fetched: 10 row(s)
hive> quit;
-- чуть больше столбцы.;


times: 1
query: SELECT * FROM hits_100m WHERE URL LIKE '%metrika%' ORDER BY EventTime LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_17569@mturlrep13_201309180030_564255388.txt
hive> SELECT * FROM hits_100m WHERE URL LIKE '%metrika%' ORDER BY EventTime LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0112
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:30:52,920 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:31:12,034 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:31:15,535 Stage-1 map = 3%,  reduce = 0%
2013-09-18 00:31:16,540 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:31:17,545 Stage-1 map = 7%,  reduce = 0%
2013-09-18 00:31:18,551 Stage-1 map = 10%,  reduce = 0%
2013-09-18 00:31:19,555 Stage-1 map = 13%,  reduce = 0%
2013-09-18 00:31:21,568 Stage-1 map = 15%,  reduce = 0%
2013-09-18 00:31:22,572 Stage-1 map = 17%,  reduce = 0%
2013-09-18 00:31:23,581 Stage-1 map = 19%,  reduce = 0%
2013-09-18 00:31:24,586 Stage-1 map = 20%,  reduce = 0%
2013-09-18 00:31:25,609 Stage-1 map = 25%,  reduce = 0%
2013-09-18 00:31:26,614 Stage-1 map = 27%,  reduce = 0%
2013-09-18 00:31:27,633 Stage-1 map = 28%,  reduce = 0%
2013-09-18 00:31:28,641 Stage-1 map = 30%,  reduce = 0%
2013-09-18 00:31:29,675 Stage-1 map = 32%,  reduce = 0%
2013-09-18 00:31:30,698 Stage-1 map = 33%,  reduce = 0%
2013-09-18 00:31:32,058 Stage-1 map = 36%,  reduce = 0%
2013-09-18 00:31:33,079 Stage-1 map = 38%,  reduce = 0%
2013-09-18 00:31:34,102 Stage-1 map = 40%,  reduce = 0%
2013-09-18 00:31:35,117 Stage-1 map = 44%,  reduce = 0%
2013-09-18 00:31:36,122 Stage-1 map = 45%,  reduce = 0%
2013-09-18 00:31:37,152 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 1131.99 sec
2013-09-18 00:31:38,167 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 1131.99 sec
2013-09-18 00:31:39,186 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:40,202 Stage-1 map = 51%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:41,208 Stage-1 map = 54%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:42,221 Stage-1 map = 58%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:43,226 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:44,286 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:45,292 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:46,298 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:47,305 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:48,325 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:49,332 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:50,340 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 1135.59 sec
2013-09-18 00:31:51,346 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 1150.66 sec
2013-09-18 00:31:52,357 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 1150.66 sec
2013-09-18 00:31:53,363 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 1201.45 sec
2013-09-18 00:31:54,370 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 1286.34 sec
2013-09-18 00:31:55,375 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 1340.63 sec
2013-09-18 00:31:56,384 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 1412.26 sec
2013-09-18 00:31:57,396 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 1529.38 sec
2013-09-18 00:31:58,401 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 1690.25 sec
2013-09-18 00:31:59,406 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1753.24 sec
2013-09-18 00:32:00,412 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1777.14 sec
2013-09-18 00:32:01,416 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1828.2 sec
2013-09-18 00:32:02,421 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1828.2 sec
2013-09-18 00:32:03,426 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1828.2 sec
2013-09-18 00:32:04,430 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1828.2 sec
2013-09-18 00:32:05,435 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1828.2 sec
2013-09-18 00:32:06,439 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1828.2 sec
2013-09-18 00:32:07,444 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1828.2 sec
2013-09-18 00:32:08,451 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1831.77 sec
2013-09-18 00:32:09,457 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1831.77 sec
MapReduce Total cumulative CPU time: 30 minutes 31 seconds 770 msec
Ended Job = job_201309172235_0112
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 1831.77 sec   HDFS Read: 10081441390 HDFS Write: 5802 SUCCESS
Total MapReduce CPU Time Spent: 30 minutes 31 seconds 770 msec
OK
Time taken: 87.697 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT * FROM hits_100m WHERE URL LIKE '%metrika%' ORDER BY EventTime LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_22644@mturlrep13_201309180032_2112140807.txt
hive> SELECT * FROM hits_100m WHERE URL LIKE '%metrika%' ORDER BY EventTime LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0113
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:32:23,647 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:32:44,899 Stage-1 map = 1%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:45,910 Stage-1 map = 4%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:46,921 Stage-1 map = 5%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:47,927 Stage-1 map = 7%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:48,933 Stage-1 map = 9%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:49,944 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:50,950 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:51,968 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:52,974 Stage-1 map = 18%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:53,981 Stage-1 map = 19%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:54,996 Stage-1 map = 21%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:56,002 Stage-1 map = 26%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:57,007 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:58,206 Stage-1 map = 30%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:32:59,212 Stage-1 map = 32%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:00,218 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:01,225 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:02,242 Stage-1 map = 37%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:03,248 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:04,253 Stage-1 map = 41%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:05,260 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:06,266 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:07,271 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:08,282 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:09,287 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 144.44 sec
2013-09-18 00:33:10,300 Stage-1 map = 54%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:11,313 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:12,329 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:13,349 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:14,361 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:15,370 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:16,376 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:17,382 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:18,389 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:19,406 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:20,411 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:21,417 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:22,426 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 174.77 sec
2013-09-18 00:33:23,432 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 345.6 sec
2013-09-18 00:33:24,439 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 435.05 sec
2013-09-18 00:33:25,444 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 477.09 sec
2013-09-18 00:33:26,450 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 930.38 sec
2013-09-18 00:33:27,467 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 1352.12 sec
2013-09-18 00:33:28,477 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 1492.34 sec
2013-09-18 00:33:29,483 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 1684.33 sec
2013-09-18 00:33:30,488 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1684.33 sec
2013-09-18 00:33:31,494 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1784.31 sec
2013-09-18 00:33:32,499 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1839.12 sec
2013-09-18 00:33:33,504 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1839.12 sec
2013-09-18 00:33:34,509 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1839.12 sec
2013-09-18 00:33:35,515 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1839.12 sec
2013-09-18 00:33:36,519 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1839.12 sec
2013-09-18 00:33:37,525 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1839.93 sec
2013-09-18 00:33:38,530 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1839.93 sec
2013-09-18 00:33:39,537 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1842.69 sec
2013-09-18 00:33:40,543 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1842.69 sec
MapReduce Total cumulative CPU time: 30 minutes 42 seconds 690 msec
Ended Job = job_201309172235_0113
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 1842.69 sec   HDFS Read: 10081441390 HDFS Write: 5802 SUCCESS
Total MapReduce CPU Time Spent: 30 minutes 42 seconds 690 msec
OK
Time taken: 88.521 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT * FROM hits_100m WHERE URL LIKE '%metrika%' ORDER BY EventTime LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_27658@mturlrep13_201309180033_1797641182.txt
hive> SELECT * FROM hits_100m WHERE URL LIKE '%metrika%' ORDER BY EventTime LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0114
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:33:53,935 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:34:14,165 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:34:16,361 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:34:17,366 Stage-1 map = 6%,  reduce = 0%
2013-09-18 00:34:18,371 Stage-1 map = 7%,  reduce = 0%
2013-09-18 00:34:19,376 Stage-1 map = 10%,  reduce = 0%
2013-09-18 00:34:20,389 Stage-1 map = 14%,  reduce = 0%
2013-09-18 00:34:21,394 Stage-1 map = 15%,  reduce = 0%
2013-09-18 00:34:22,404 Stage-1 map = 18%,  reduce = 0%
2013-09-18 00:34:23,409 Stage-1 map = 19%,  reduce = 0%
2013-09-18 00:34:25,642 Stage-1 map = 22%,  reduce = 0%
2013-09-18 00:34:26,649 Stage-1 map = 27%,  reduce = 0%
2013-09-18 00:34:28,674 Stage-1 map = 28%,  reduce = 0%
2013-09-18 00:34:29,680 Stage-1 map = 31%,  reduce = 0%
2013-09-18 00:34:30,686 Stage-1 map = 32%,  reduce = 0%
2013-09-18 00:34:31,692 Stage-1 map = 34%,  reduce = 0%
2013-09-18 00:34:32,697 Stage-1 map = 36%,  reduce = 0%
2013-09-18 00:34:33,702 Stage-1 map = 39%,  reduce = 0%
2013-09-18 00:34:34,719 Stage-1 map = 40%,  reduce = 0%
2013-09-18 00:34:35,725 Stage-1 map = 42%,  reduce = 0%
2013-09-18 00:34:36,730 Stage-1 map = 44%,  reduce = 0%
2013-09-18 00:34:37,740 Stage-1 map = 45%,  reduce = 0%, Cumulative CPU 1134.9 sec
2013-09-18 00:34:38,752 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 1134.9 sec
2013-09-18 00:34:39,768 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 1134.9 sec
2013-09-18 00:34:40,779 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 1134.9 sec
2013-09-18 00:34:41,816 Stage-1 map = 54%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:42,825 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:43,831 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:44,837 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:45,843 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:46,856 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:47,862 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:48,880 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:49,889 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:50,902 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:51,908 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 1140.46 sec
2013-09-18 00:34:52,913 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 1171.26 sec
2013-09-18 00:34:53,952 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 1186.68 sec
2013-09-18 00:34:54,957 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 1255.59 sec
2013-09-18 00:34:55,970 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 1310.59 sec
2013-09-18 00:34:56,976 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 1386.73 sec
2013-09-18 00:34:57,981 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 1506.92 sec
2013-09-18 00:34:58,986 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 1683.99 sec
2013-09-18 00:34:59,991 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 1723.75 sec
2013-09-18 00:35:00,997 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 1772.53 sec
2013-09-18 00:35:02,002 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1772.53 sec
2013-09-18 00:35:03,006 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1826.62 sec
2013-09-18 00:35:04,011 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1855.25 sec
2013-09-18 00:35:05,015 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1855.25 sec
2013-09-18 00:35:06,019 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1855.25 sec
2013-09-18 00:35:07,023 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1855.25 sec
2013-09-18 00:35:08,028 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1855.25 sec
2013-09-18 00:35:09,033 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1855.25 sec
2013-09-18 00:35:10,038 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1855.25 sec
2013-09-18 00:35:11,045 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1858.98 sec
2013-09-18 00:35:12,051 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1858.98 sec
MapReduce Total cumulative CPU time: 30 minutes 58 seconds 980 msec
Ended Job = job_201309172235_0114
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 1858.98 sec   HDFS Read: 10081441390 HDFS Write: 5802 SUCCESS
Total MapReduce CPU Time Spent: 30 minutes 58 seconds 980 msec
OK
Time taken: 89.324 seconds, Fetched: 10 row(s)
hive> quit;
-- плохой запрос - вынимаем все столбцы.;


times: 1
query: SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_637@mturlrep13_201309180035_636558816.txt
hive> SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0115
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:35:29,995 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:35:41,126 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:35:42,132 Stage-1 map = 4%,  reduce = 0%
2013-09-18 00:35:43,139 Stage-1 map = 12%,  reduce = 0%
2013-09-18 00:35:44,146 Stage-1 map = 16%,  reduce = 0%
2013-09-18 00:35:45,160 Stage-1 map = 18%,  reduce = 0%, Cumulative CPU 8.89 sec
2013-09-18 00:35:46,172 Stage-1 map = 18%,  reduce = 0%, Cumulative CPU 8.89 sec
2013-09-18 00:35:47,181 Stage-1 map = 18%,  reduce = 0%, Cumulative CPU 8.89 sec
2013-09-18 00:35:48,823 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 19.53 sec
2013-09-18 00:35:50,089 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 92.18 sec
2013-09-18 00:35:51,103 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 137.0 sec
2013-09-18 00:35:52,111 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 217.4 sec
2013-09-18 00:35:53,117 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 317.92 sec
2013-09-18 00:35:54,123 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 435.87 sec
2013-09-18 00:35:55,129 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 435.87 sec
2013-09-18 00:35:56,135 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 435.87 sec
2013-09-18 00:35:57,141 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 435.87 sec
2013-09-18 00:35:58,148 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 435.87 sec
2013-09-18 00:35:59,154 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 435.87 sec
2013-09-18 00:36:00,159 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 435.87 sec
2013-09-18 00:36:01,165 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 435.87 sec
2013-09-18 00:36:02,170 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 435.87 sec
2013-09-18 00:36:03,175 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 435.87 sec
2013-09-18 00:36:04,180 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 435.87 sec
2013-09-18 00:36:05,186 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 435.87 sec
2013-09-18 00:36:06,191 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 435.87 sec
2013-09-18 00:36:07,197 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 435.87 sec
2013-09-18 00:36:08,202 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 435.87 sec
2013-09-18 00:36:09,208 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 435.87 sec
2013-09-18 00:36:10,212 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 435.87 sec
2013-09-18 00:36:11,217 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 435.87 sec
2013-09-18 00:36:12,224 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 457.03 sec
2013-09-18 00:36:13,230 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 457.03 sec
MapReduce Total cumulative CPU time: 7 minutes 37 seconds 30 msec
Ended Job = job_201309172235_0115
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 457.03 sec   HDFS Read: 184864723 HDFS Write: 747 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 37 seconds 30 msec
OK
Time taken: 53.796 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_6758@mturlrep13_201309180036_334325394.txt
hive> SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0116
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:36:23,502 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:36:35,575 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:36:36,583 Stage-1 map = 9%,  reduce = 0%
2013-09-18 00:36:37,595 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 232.24 sec
2013-09-18 00:36:38,607 Stage-1 map = 24%,  reduce = 0%, Cumulative CPU 232.24 sec
2013-09-18 00:36:39,625 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 232.24 sec
2013-09-18 00:36:40,671 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 246.94 sec
2013-09-18 00:36:41,688 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 294.67 sec
2013-09-18 00:36:42,699 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 362.37 sec
2013-09-18 00:36:43,705 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 448.5 sec
2013-09-18 00:36:44,711 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 448.5 sec
2013-09-18 00:36:45,717 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 448.5 sec
2013-09-18 00:36:46,723 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 448.5 sec
2013-09-18 00:36:47,729 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 448.5 sec
2013-09-18 00:36:48,735 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 448.5 sec
2013-09-18 00:36:49,741 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 448.5 sec
2013-09-18 00:36:50,747 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 448.5 sec
2013-09-18 00:36:51,753 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 448.5 sec
2013-09-18 00:36:52,759 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 448.5 sec
2013-09-18 00:36:53,764 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 448.5 sec
2013-09-18 00:36:54,770 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 448.5 sec
2013-09-18 00:36:55,776 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 448.5 sec
2013-09-18 00:36:56,782 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 448.5 sec
2013-09-18 00:36:57,789 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 448.5 sec
2013-09-18 00:36:58,796 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 448.5 sec
2013-09-18 00:36:59,802 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 448.5 sec
2013-09-18 00:37:00,808 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 448.5 sec
2013-09-18 00:37:01,814 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 448.5 sec
2013-09-18 00:37:02,820 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 448.5 sec
2013-09-18 00:37:03,828 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 469.6 sec
2013-09-18 00:37:04,834 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 469.6 sec
2013-09-18 00:37:05,840 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 469.6 sec
MapReduce Total cumulative CPU time: 7 minutes 49 seconds 600 msec
Ended Job = job_201309172235_0116
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 469.6 sec   HDFS Read: 184864723 HDFS Write: 747 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 49 seconds 600 msec
OK
Time taken: 50.592 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_12112@mturlrep13_201309180037_615192245.txt
hive> SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0117
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:37:16,411 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:37:27,531 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:37:28,541 Stage-1 map = 11%,  reduce = 0%
2013-09-18 00:37:29,559 Stage-1 map = 19%,  reduce = 0%
2013-09-18 00:37:30,564 Stage-1 map = 24%,  reduce = 0%
2013-09-18 00:37:31,580 Stage-1 map = 51%,  reduce = 0%, Cumulative CPU 9.16 sec
2013-09-18 00:37:32,592 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 47.94 sec
2013-09-18 00:37:33,600 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 167.83 sec
2013-09-18 00:37:34,607 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 256.71 sec
2013-09-18 00:37:35,612 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 445.74 sec
2013-09-18 00:37:36,617 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 445.74 sec
2013-09-18 00:37:37,622 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 445.74 sec
2013-09-18 00:37:38,627 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 445.74 sec
2013-09-18 00:37:39,632 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 445.74 sec
2013-09-18 00:37:40,637 Stage-1 map = 100%,  reduce = 11%, Cumulative CPU 445.74 sec
2013-09-18 00:37:41,643 Stage-1 map = 100%,  reduce = 11%, Cumulative CPU 445.74 sec
2013-09-18 00:37:42,648 Stage-1 map = 100%,  reduce = 11%, Cumulative CPU 445.74 sec
2013-09-18 00:37:43,654 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 445.74 sec
2013-09-18 00:37:44,659 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 445.74 sec
2013-09-18 00:37:45,664 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 445.74 sec
2013-09-18 00:37:46,669 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 445.74 sec
2013-09-18 00:37:47,675 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 445.74 sec
2013-09-18 00:37:48,680 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 445.74 sec
2013-09-18 00:37:49,685 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 445.74 sec
2013-09-18 00:37:50,691 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 445.74 sec
2013-09-18 00:37:51,696 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 445.74 sec
2013-09-18 00:37:52,701 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 445.74 sec
2013-09-18 00:37:53,706 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 445.74 sec
2013-09-18 00:37:54,711 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 445.74 sec
2013-09-18 00:37:55,718 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 467.25 sec
2013-09-18 00:37:56,723 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 467.25 sec
2013-09-18 00:37:57,729 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 467.25 sec
MapReduce Total cumulative CPU time: 7 minutes 47 seconds 250 msec
Ended Job = job_201309172235_0117
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 467.25 sec   HDFS Read: 184864723 HDFS Write: 747 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 47 seconds 250 msec
OK
Time taken: 50.118 seconds, Fetched: 10 row(s)
hive> quit;
-- большая сортировка.;


times: 1
query: SELECT SearchPhrase FROM hits_100m WHERE SearchPhrase != '' ORDER BY SearchPhrase LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_17453@mturlrep13_201309180038_1517091963.txt
hive> SELECT SearchPhrase FROM hits_100m WHERE SearchPhrase != '' ORDER BY SearchPhrase LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0118
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:38:15,283 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:38:25,343 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:38:27,409 Stage-1 map = 4%,  reduce = 0%
2013-09-18 00:38:28,422 Stage-1 map = 14%,  reduce = 0%
2013-09-18 00:38:29,428 Stage-1 map = 20%,  reduce = 0%
2013-09-18 00:38:31,442 Stage-1 map = 46%,  reduce = 0%
2013-09-18 00:38:32,447 Stage-1 map = 54%,  reduce = 0%
2013-09-18 00:38:34,502 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 56.43 sec
2013-09-18 00:38:35,526 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 160.97 sec
2013-09-18 00:38:36,540 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 264.56 sec
2013-09-18 00:38:37,546 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 421.84 sec
2013-09-18 00:38:38,552 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 433.68 sec
2013-09-18 00:38:39,557 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 442.48 sec
2013-09-18 00:38:40,562 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 442.48 sec
2013-09-18 00:38:41,567 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 442.48 sec
2013-09-18 00:38:42,573 Stage-1 map = 100%,  reduce = 5%, Cumulative CPU 442.48 sec
2013-09-18 00:38:43,579 Stage-1 map = 100%,  reduce = 5%, Cumulative CPU 442.48 sec
2013-09-18 00:38:44,585 Stage-1 map = 100%,  reduce = 5%, Cumulative CPU 442.48 sec
2013-09-18 00:38:45,590 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 442.48 sec
2013-09-18 00:38:46,596 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 442.48 sec
2013-09-18 00:38:47,602 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 442.48 sec
2013-09-18 00:38:48,607 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 442.48 sec
2013-09-18 00:38:49,613 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 442.48 sec
2013-09-18 00:38:50,618 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 442.48 sec
2013-09-18 00:38:51,624 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 442.48 sec
2013-09-18 00:38:52,629 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 442.48 sec
2013-09-18 00:38:53,635 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 442.48 sec
2013-09-18 00:38:54,640 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 442.48 sec
2013-09-18 00:38:55,645 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 442.48 sec
2013-09-18 00:38:56,649 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 442.48 sec
2013-09-18 00:38:57,654 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 442.48 sec
2013-09-18 00:38:58,658 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 442.48 sec
2013-09-18 00:38:59,663 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 442.48 sec
2013-09-18 00:39:00,668 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 442.48 sec
2013-09-18 00:39:01,673 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 442.48 sec
2013-09-18 00:39:02,677 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 442.48 sec
2013-09-18 00:39:03,682 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 442.48 sec
2013-09-18 00:39:04,689 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 477.8 sec
2013-09-18 00:39:08,034 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 477.8 sec
2013-09-18 00:39:09,045 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 477.8 sec
2013-09-18 00:39:10,050 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 477.8 sec
2013-09-18 00:39:11,056 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 477.8 sec
2013-09-18 00:39:12,062 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 477.8 sec
MapReduce Total cumulative CPU time: 7 minutes 57 seconds 800 msec
Ended Job = job_201309172235_0118
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 477.8 sec   HDFS Read: 182281719 HDFS Write: 509 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 57 seconds 800 msec
OK

 снип 2.07.01-89
 19105-79
! 20 самых ужасные пыток в истории человечества. видео
! bt
! гобилен желтый купить )
! жизнь в тюрьме россии -
! минимальное значение измеряемой величины?
! новый htc 603e
! о чем говорит огромное количество антител в крови к кишечной палочке
Time taken: 68.786 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT SearchPhrase FROM hits_100m WHERE SearchPhrase != '' ORDER BY SearchPhrase LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_22553@mturlrep13_201309180039_34144715.txt
hive> SELECT SearchPhrase FROM hits_100m WHERE SearchPhrase != '' ORDER BY SearchPhrase LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0119
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:39:23,346 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:39:35,450 Stage-1 map = 2%,  reduce = 0%
2013-09-18 00:39:36,456 Stage-1 map = 19%,  reduce = 0%
2013-09-18 00:39:37,472 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 259.71 sec
2013-09-18 00:39:38,480 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 259.71 sec
2013-09-18 00:39:39,499 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 267.32 sec
2013-09-18 00:39:40,516 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 282.8 sec
2013-09-18 00:39:41,528 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 308.59 sec
2013-09-18 00:39:42,536 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 377.66 sec
2013-09-18 00:39:43,542 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 452.74 sec
2013-09-18 00:39:44,548 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 452.74 sec
2013-09-18 00:39:45,553 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 452.74 sec
2013-09-18 00:39:46,559 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 452.74 sec
2013-09-18 00:39:47,565 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 452.74 sec
2013-09-18 00:39:48,572 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 452.74 sec
2013-09-18 00:39:49,578 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 452.74 sec
2013-09-18 00:39:50,584 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 452.74 sec
2013-09-18 00:39:51,590 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 452.74 sec
2013-09-18 00:39:52,596 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 452.74 sec
2013-09-18 00:39:53,602 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 452.74 sec
2013-09-18 00:39:54,607 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.74 sec
2013-09-18 00:39:55,613 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.74 sec
2013-09-18 00:39:56,619 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.74 sec
2013-09-18 00:39:57,625 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.74 sec
2013-09-18 00:39:58,630 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.74 sec
2013-09-18 00:39:59,636 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.74 sec
2013-09-18 00:40:00,642 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.74 sec
2013-09-18 00:40:01,648 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.74 sec
2013-09-18 00:40:02,655 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.74 sec
2013-09-18 00:40:03,660 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 452.74 sec
2013-09-18 00:40:04,665 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 452.74 sec
2013-09-18 00:40:05,670 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 452.74 sec
2013-09-18 00:40:06,675 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 452.74 sec
2013-09-18 00:40:07,680 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 452.74 sec
2013-09-18 00:40:08,686 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 452.74 sec
2013-09-18 00:40:09,691 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 452.74 sec
2013-09-18 00:40:10,699 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 487.91 sec
2013-09-18 00:40:11,704 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 487.91 sec
2013-09-18 00:40:12,710 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 487.91 sec
MapReduce Total cumulative CPU time: 8 minutes 7 seconds 910 msec
Ended Job = job_201309172235_0119
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 487.91 sec   HDFS Read: 182281719 HDFS Write: 509 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 7 seconds 910 msec
OK

 снип 2.07.01-89
 19105-79
! 20 самых ужасные пыток в истории человечества. видео
! bt
! гобилен желтый купить )
! жизнь в тюрьме россии -
! минимальное значение измеряемой величины?
! новый htc 603e
! о чем говорит огромное количество антител в крови к кишечной палочке
Time taken: 63.993 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT SearchPhrase FROM hits_100m WHERE SearchPhrase != '' ORDER BY SearchPhrase LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_27974@mturlrep13_201309180040_814853077.txt
hive> SELECT SearchPhrase FROM hits_100m WHERE SearchPhrase != '' ORDER BY SearchPhrase LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0120
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:40:30,016 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:40:42,091 Stage-1 map = 12%,  reduce = 0%
2013-09-18 00:40:43,103 Stage-1 map = 28%,  reduce = 0%
2013-09-18 00:40:44,113 Stage-1 map = 34%,  reduce = 0%
2013-09-18 00:40:45,135 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 18.32 sec
2013-09-18 00:40:46,146 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 77.03 sec
2013-09-18 00:40:47,154 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 151.48 sec
2013-09-18 00:40:48,161 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 201.55 sec
2013-09-18 00:40:49,167 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 395.4 sec
2013-09-18 00:40:50,173 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 452.5 sec
2013-09-18 00:40:51,178 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 452.5 sec
2013-09-18 00:40:52,184 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 452.5 sec
2013-09-18 00:40:53,259 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 452.5 sec
2013-09-18 00:40:54,265 Stage-1 map = 100%,  reduce = 7%, Cumulative CPU 452.5 sec
2013-09-18 00:40:55,271 Stage-1 map = 100%,  reduce = 7%, Cumulative CPU 452.5 sec
2013-09-18 00:40:56,281 Stage-1 map = 100%,  reduce = 7%, Cumulative CPU 452.5 sec
2013-09-18 00:40:57,288 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 452.5 sec
2013-09-18 00:40:58,293 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 452.5 sec
2013-09-18 00:40:59,298 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 452.5 sec
2013-09-18 00:41:00,303 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.5 sec
2013-09-18 00:41:01,307 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.5 sec
2013-09-18 00:41:02,312 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.5 sec
2013-09-18 00:41:03,317 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.5 sec
2013-09-18 00:41:04,322 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.5 sec
2013-09-18 00:41:05,326 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.5 sec
2013-09-18 00:41:06,331 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.5 sec
2013-09-18 00:41:07,336 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.5 sec
2013-09-18 00:41:08,341 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 452.5 sec
2013-09-18 00:41:09,346 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 452.5 sec
2013-09-18 00:41:10,351 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 452.5 sec
2013-09-18 00:41:11,356 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 452.5 sec
2013-09-18 00:41:12,360 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 452.5 sec
2013-09-18 00:41:13,365 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 452.5 sec
2013-09-18 00:41:14,369 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 452.5 sec
2013-09-18 00:41:15,374 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 452.5 sec
2013-09-18 00:41:16,381 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 487.48 sec
2013-09-18 00:41:17,387 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 487.48 sec
2013-09-18 00:41:18,392 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 487.48 sec
MapReduce Total cumulative CPU time: 8 minutes 7 seconds 480 msec
Ended Job = job_201309172235_0120
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 487.48 sec   HDFS Read: 182281719 HDFS Write: 509 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 7 seconds 480 msec
OK

 снип 2.07.01-89
 19105-79
! 20 самых ужасные пыток в истории человечества. видео
! bt
! гобилен желтый купить )
! жизнь в тюрьме россии -
! минимальное значение измеряемой величины?
! новый htc 603e
! о чем говорит огромное количество антител в крови к кишечной палочке
Time taken: 62.886 seconds, Fetched: 10 row(s)
hive> quit;
-- большая сортировка по строкам.;


times: 1
query: SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime, SearchPhrase LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_964@mturlrep13_201309180041_1295667471.txt
hive> SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime, SearchPhrase LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0121
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:41:41,880 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:41:53,981 Stage-1 map = 3%,  reduce = 0%
2013-09-18 00:41:54,989 Stage-1 map = 10%,  reduce = 0%
2013-09-18 00:41:55,998 Stage-1 map = 16%,  reduce = 0%
2013-09-18 00:41:57,005 Stage-1 map = 24%,  reduce = 0%
2013-09-18 00:41:58,010 Stage-1 map = 34%,  reduce = 0%
2013-09-18 00:41:59,015 Stage-1 map = 39%,  reduce = 0%
2013-09-18 00:42:00,845 Stage-1 map = 51%,  reduce = 0%
2013-09-18 00:42:02,853 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 116.0 sec
2013-09-18 00:42:03,921 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 228.22 sec
2013-09-18 00:42:04,929 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 296.35 sec
2013-09-18 00:42:05,935 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 412.74 sec
2013-09-18 00:42:06,941 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 470.11 sec
2013-09-18 00:42:07,947 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 470.11 sec
2013-09-18 00:42:08,953 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 470.11 sec
2013-09-18 00:42:09,962 Stage-1 map = 100%,  reduce = 5%, Cumulative CPU 470.11 sec
2013-09-18 00:42:10,968 Stage-1 map = 100%,  reduce = 5%, Cumulative CPU 470.11 sec
2013-09-18 00:42:11,974 Stage-1 map = 100%,  reduce = 5%, Cumulative CPU 470.11 sec
2013-09-18 00:42:12,980 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 470.11 sec
2013-09-18 00:42:13,985 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 470.11 sec
2013-09-18 00:42:14,991 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 470.11 sec
2013-09-18 00:42:15,996 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 470.11 sec
2013-09-18 00:42:17,001 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 470.11 sec
2013-09-18 00:42:18,007 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 470.11 sec
2013-09-18 00:42:19,012 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 470.11 sec
2013-09-18 00:42:20,018 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 470.11 sec
2013-09-18 00:42:21,024 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 470.11 sec
2013-09-18 00:42:22,030 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 470.11 sec
2013-09-18 00:42:23,036 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 470.11 sec
2013-09-18 00:42:24,040 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 470.11 sec
2013-09-18 00:42:25,044 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 470.11 sec
2013-09-18 00:42:26,049 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 470.11 sec
2013-09-18 00:42:27,054 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 470.11 sec
2013-09-18 00:42:28,059 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 470.11 sec
2013-09-18 00:42:29,064 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 470.11 sec
2013-09-18 00:42:30,071 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 503.72 sec
2013-09-18 00:42:31,077 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 503.72 sec
2013-09-18 00:42:32,083 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 503.72 sec
MapReduce Total cumulative CPU time: 8 minutes 23 seconds 720 msec
Ended Job = job_201309172235_0121
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 503.72 sec   HDFS Read: 184864723 HDFS Write: 758 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 23 seconds 720 msec
OK
Time taken: 60.715 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime, SearchPhrase LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_6757@mturlrep13_201309180042_1836094631.txt
hive> SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime, SearchPhrase LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0122
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:42:42,848 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:42:53,935 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:42:54,967 Stage-1 map = 7%,  reduce = 0%
2013-09-18 00:42:55,986 Stage-1 map = 15%,  reduce = 0%
2013-09-18 00:42:56,991 Stage-1 map = 19%,  reduce = 0%
2013-09-18 00:42:58,003 Stage-1 map = 37%,  reduce = 0%
2013-09-18 00:42:59,028 Stage-1 map = 58%,  reduce = 0%, Cumulative CPU 20.21 sec
2013-09-18 00:43:00,042 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 95.48 sec
2013-09-18 00:43:01,060 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 203.64 sec
2013-09-18 00:43:02,067 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 316.18 sec
2013-09-18 00:43:03,074 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 451.06 sec
2013-09-18 00:43:04,079 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 482.61 sec
2013-09-18 00:43:05,085 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 482.61 sec
2013-09-18 00:43:06,090 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 482.61 sec
2013-09-18 00:43:07,095 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 482.61 sec
2013-09-18 00:43:08,100 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 482.61 sec
2013-09-18 00:43:09,105 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 482.61 sec
2013-09-18 00:43:10,112 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 482.61 sec
2013-09-18 00:43:11,119 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 482.61 sec
2013-09-18 00:43:12,124 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 482.61 sec
2013-09-18 00:43:13,130 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 482.61 sec
2013-09-18 00:43:14,136 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 482.61 sec
2013-09-18 00:43:15,141 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 482.61 sec
2013-09-18 00:43:16,147 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 482.61 sec
2013-09-18 00:43:17,152 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 482.61 sec
2013-09-18 00:43:18,158 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 482.61 sec
2013-09-18 00:43:19,162 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 482.61 sec
2013-09-18 00:43:20,167 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 482.61 sec
2013-09-18 00:43:21,172 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 482.61 sec
2013-09-18 00:43:22,177 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 482.61 sec
2013-09-18 00:43:23,182 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 482.61 sec
2013-09-18 00:43:24,186 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 482.61 sec
2013-09-18 00:43:25,190 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 482.61 sec
2013-09-18 00:43:26,195 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 482.61 sec
2013-09-18 00:43:27,200 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 482.61 sec
2013-09-18 00:43:28,206 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 517.47 sec
2013-09-18 00:43:29,827 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 517.47 sec
MapReduce Total cumulative CPU time: 8 minutes 37 seconds 470 msec
Ended Job = job_201309172235_0122
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 517.47 sec   HDFS Read: 184864723 HDFS Write: 758 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 37 seconds 470 msec
OK
Time taken: 56.14 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime, SearchPhrase LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_12116@mturlrep13_201309180043_688174400.txt
hive> SELECT SearchPhrase, EventTime FROM hits_100m WHERE SearchPhrase != '' ORDER BY EventTime, SearchPhrase LIMIT 10;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0123
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:43:39,885 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:43:52,084 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:43:53,089 Stage-1 map = 9%,  reduce = 0%
2013-09-18 00:43:54,103 Stage-1 map = 21%,  reduce = 0%
2013-09-18 00:43:55,109 Stage-1 map = 22%,  reduce = 0%
2013-09-18 00:43:56,124 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 9.78 sec
2013-09-18 00:43:57,139 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 49.2 sec
2013-09-18 00:43:58,156 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 146.89 sec
2013-09-18 00:43:59,166 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 211.3 sec
2013-09-18 00:44:00,173 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 339.17 sec
2013-09-18 00:44:01,179 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 474.94 sec
2013-09-18 00:44:02,184 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 474.94 sec
2013-09-18 00:44:03,190 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 474.94 sec
2013-09-18 00:44:04,196 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 474.94 sec
2013-09-18 00:44:05,203 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 474.94 sec
2013-09-18 00:44:06,209 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 474.94 sec
2013-09-18 00:44:07,216 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 474.94 sec
2013-09-18 00:44:08,222 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 474.94 sec
2013-09-18 00:44:09,228 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 474.94 sec
2013-09-18 00:44:10,235 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 474.94 sec
2013-09-18 00:44:11,241 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 474.94 sec
2013-09-18 00:44:12,247 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 474.94 sec
2013-09-18 00:44:13,253 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 474.94 sec
2013-09-18 00:44:14,258 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 474.94 sec
2013-09-18 00:44:15,264 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 474.94 sec
2013-09-18 00:44:16,270 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 474.94 sec
2013-09-18 00:44:17,277 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 474.94 sec
2013-09-18 00:44:18,283 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 474.94 sec
2013-09-18 00:44:19,290 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 474.94 sec
2013-09-18 00:44:20,296 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 474.94 sec
2013-09-18 00:44:21,301 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 474.94 sec
2013-09-18 00:44:22,306 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 474.94 sec
2013-09-18 00:44:23,312 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 474.94 sec
2013-09-18 00:44:24,318 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 474.94 sec
2013-09-18 00:44:25,324 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 474.94 sec
2013-09-18 00:44:26,332 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 511.22 sec
2013-09-18 00:44:27,337 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 511.22 sec
MapReduce Total cumulative CPU time: 8 minutes 31 seconds 220 msec
Ended Job = job_201309172235_0123
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 511.22 sec   HDFS Read: 184864723 HDFS Write: 758 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 31 seconds 220 msec
OK
Time taken: 55.303 seconds, Fetched: 10 row(s)
hive> quit;
-- большая сортировка по кортежу.;


times: 1
query: SELECT CounterID, avg(length(URL)) AS l, count(*) AS c FROM hits_100m WHERE URL != '' GROUP BY CounterID HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_17460@mturlrep13_201309180044_876855052.txt
hive> SELECT CounterID, avg(length(URL)) AS l, count(*) AS c FROM hits_100m WHERE URL != '' GROUP BY CounterID HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0124
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:44:53,239 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:45:09,346 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:45:10,353 Stage-1 map = 7%,  reduce = 0%
2013-09-18 00:45:11,358 Stage-1 map = 8%,  reduce = 0%
2013-09-18 00:45:12,365 Stage-1 map = 11%,  reduce = 0%
2013-09-18 00:45:14,374 Stage-1 map = 14%,  reduce = 0%
2013-09-18 00:45:15,389 Stage-1 map = 17%,  reduce = 0%
2013-09-18 00:45:16,396 Stage-1 map = 21%,  reduce = 0%
2013-09-18 00:45:17,410 Stage-1 map = 25%,  reduce = 0%
2013-09-18 00:45:18,415 Stage-1 map = 28%,  reduce = 0%
2013-09-18 00:45:19,438 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:20,445 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:22,432 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:23,470 Stage-1 map = 45%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:24,485 Stage-1 map = 45%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:25,493 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:27,100 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:28,112 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:29,121 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:30,130 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:31,442 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:32,466 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 14.93 sec
2013-09-18 00:45:33,482 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 72.87 sec
2013-09-18 00:45:34,504 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 152.52 sec
2013-09-18 00:45:35,718 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 324.07 sec
2013-09-18 00:45:36,726 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 615.07 sec
2013-09-18 00:45:37,738 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 861.87 sec
2013-09-18 00:45:38,744 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1085.12 sec
2013-09-18 00:45:39,750 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:40,756 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:41,761 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:42,772 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:43,781 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:44,801 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:45,818 Stage-1 map = 100%,  reduce = 15%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:46,825 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:47,831 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:48,838 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:49,845 Stage-1 map = 100%,  reduce = 44%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:50,852 Stage-1 map = 100%,  reduce = 47%, Cumulative CPU 1087.12 sec
2013-09-18 00:45:51,861 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 1098.67 sec
2013-09-18 00:45:52,867 Stage-1 map = 100%,  reduce = 60%, Cumulative CPU 1098.67 sec
2013-09-18 00:45:53,875 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1098.67 sec
2013-09-18 00:45:54,882 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1098.67 sec
2013-09-18 00:45:55,889 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1114.66 sec
2013-09-18 00:45:56,895 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 1114.66 sec
2013-09-18 00:45:57,902 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1132.03 sec
2013-09-18 00:45:58,908 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 1132.03 sec
2013-09-18 00:45:59,915 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1132.03 sec
2013-09-18 00:46:00,922 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1175.97 sec
2013-09-18 00:46:01,929 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1199.97 sec
2013-09-18 00:46:02,936 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1199.97 sec
2013-09-18 00:46:03,943 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1199.97 sec
2013-09-18 00:46:04,949 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1199.97 sec
2013-09-18 00:46:05,955 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1199.97 sec
2013-09-18 00:46:10,612 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1224.95 sec
2013-09-18 00:46:11,618 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1224.95 sec
2013-09-18 00:46:13,271 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1250.63 sec
2013-09-18 00:46:14,276 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1250.63 sec
2013-09-18 00:46:15,344 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1277.48 sec
2013-09-18 00:46:16,350 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1333.41 sec
2013-09-18 00:46:17,356 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1333.41 sec
2013-09-18 00:46:20,396 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1333.41 sec
MapReduce Total cumulative CPU time: 22 minutes 13 seconds 410 msec
Ended Job = job_201309172235_0124
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0125
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 00:46:24,314 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:46:25,319 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:46:26,325 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:46:27,330 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:46:28,335 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:46:29,340 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:46:30,345 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:46:31,350 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:46:32,355 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 00:46:33,360 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.29 sec
2013-09-18 00:46:34,366 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.29 sec
2013-09-18 00:46:35,371 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.29 sec
MapReduce Total cumulative CPU time: 2 seconds 290 msec
Ended Job = job_201309172235_0125
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1333.41 sec   HDFS Read: 978257057 HDFS Write: 4324 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.29 sec   HDFS Read: 7901 HDFS Write: 790 SUCCESS
Total MapReduce CPU Time Spent: 22 minutes 15 seconds 700 msec
OK
Time taken: 112.58 seconds, Fetched: 25 row(s)
hive> quit;

times: 2
query: SELECT CounterID, avg(length(URL)) AS l, count(*) AS c FROM hits_100m WHERE URL != '' GROUP BY CounterID HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_25268@mturlrep13_201309180046_2058731462.txt
hive> SELECT CounterID, avg(length(URL)) AS l, count(*) AS c FROM hits_100m WHERE URL != '' GROUP BY CounterID HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0126
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:46:46,178 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:46:59,296 Stage-1 map = 4%,  reduce = 0%
2013-09-18 00:47:00,310 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:47:01,316 Stage-1 map = 6%,  reduce = 0%
2013-09-18 00:47:03,326 Stage-1 map = 8%,  reduce = 0%
2013-09-18 00:47:04,331 Stage-1 map = 13%,  reduce = 0%
2013-09-18 00:47:05,337 Stage-1 map = 16%,  reduce = 0%
2013-09-18 00:47:06,343 Stage-1 map = 19%,  reduce = 0%
2013-09-18 00:47:07,364 Stage-1 map = 25%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:08,406 Stage-1 map = 30%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:09,418 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:10,425 Stage-1 map = 39%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:11,459 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:12,470 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:13,476 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:14,517 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:15,536 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:16,546 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:17,560 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:18,584 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:19,602 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 14.43 sec
2013-09-18 00:47:20,610 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 40.87 sec
2013-09-18 00:47:21,619 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 69.23 sec
2013-09-18 00:47:22,628 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 237.25 sec
2013-09-18 00:47:23,638 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 408.41 sec
2013-09-18 00:47:24,670 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 645.37 sec
2013-09-18 00:47:25,677 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 955.16 sec
2013-09-18 00:47:26,685 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 987.34 sec
2013-09-18 00:47:27,691 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:28,696 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:29,702 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:30,722 Stage-1 map = 100%,  reduce = 4%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:31,729 Stage-1 map = 100%,  reduce = 15%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:32,735 Stage-1 map = 100%,  reduce = 16%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:33,741 Stage-1 map = 100%,  reduce = 17%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:34,747 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:35,753 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:36,759 Stage-1 map = 100%,  reduce = 40%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:37,765 Stage-1 map = 100%,  reduce = 45%, Cumulative CPU 1085.3 sec
2013-09-18 00:47:38,774 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 1196.28 sec
2013-09-18 00:47:39,781 Stage-1 map = 100%,  reduce = 51%, Cumulative CPU 1196.28 sec
2013-09-18 00:47:40,787 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1196.28 sec
2013-09-18 00:47:41,794 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1196.28 sec
2013-09-18 00:47:42,800 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1201.93 sec
2013-09-18 00:47:43,806 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1201.93 sec
2013-09-18 00:47:44,820 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 1209.07 sec
2013-09-18 00:47:45,826 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 1209.07 sec
2013-09-18 00:47:46,832 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 1209.07 sec
2013-09-18 00:47:47,838 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1220.62 sec
2013-09-18 00:47:48,844 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1220.62 sec
2013-09-18 00:47:49,850 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1246.48 sec
2013-09-18 00:47:50,856 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1260.92 sec
2013-09-18 00:47:51,862 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1276.65 sec
2013-09-18 00:47:52,868 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1293.47 sec
2013-09-18 00:47:53,873 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:47:56,015 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:47:57,021 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:47:58,026 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:47:59,031 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:48:00,036 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:48:01,041 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:48:02,046 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:48:03,366 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:48:04,371 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:48:05,376 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1293.47 sec
2013-09-18 00:48:06,382 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1328.31 sec
2013-09-18 00:48:09,722 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1328.31 sec
MapReduce Total cumulative CPU time: 22 minutes 8 seconds 310 msec
Ended Job = job_201309172235_0126
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0127
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 00:48:13,466 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:48:15,474 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.82 sec
2013-09-18 00:48:16,478 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.82 sec
2013-09-18 00:48:17,481 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.82 sec
2013-09-18 00:48:18,485 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.82 sec
2013-09-18 00:48:19,489 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.82 sec
2013-09-18 00:48:20,494 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.82 sec
2013-09-18 00:48:21,497 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.82 sec
2013-09-18 00:48:22,502 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.38 sec
2013-09-18 00:48:23,508 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.38 sec
2013-09-18 00:48:24,513 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.38 sec
MapReduce Total cumulative CPU time: 2 seconds 380 msec
Ended Job = job_201309172235_0127
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1328.31 sec   HDFS Read: 978257057 HDFS Write: 4324 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.38 sec   HDFS Read: 7901 HDFS Write: 790 SUCCESS
Total MapReduce CPU Time Spent: 22 minutes 10 seconds 690 msec
OK
Time taken: 107.297 seconds, Fetched: 25 row(s)
hive> quit;

times: 3
query: SELECT CounterID, avg(length(URL)) AS l, count(*) AS c FROM hits_100m WHERE URL != '' GROUP BY CounterID HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_32317@mturlrep13_201309180048_1314756439.txt
hive> SELECT CounterID, avg(length(URL)) AS l, count(*) AS c FROM hits_100m WHERE URL != '' GROUP BY CounterID HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0128
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:48:35,184 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:48:48,323 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:48:49,329 Stage-1 map = 7%,  reduce = 0%
2013-09-18 00:48:53,356 Stage-1 map = 11%,  reduce = 0%
2013-09-18 00:48:54,367 Stage-1 map = 15%,  reduce = 0%
2013-09-18 00:48:55,377 Stage-1 map = 18%,  reduce = 0%
2013-09-18 00:48:56,389 Stage-1 map = 23%,  reduce = 0%
2013-09-18 00:48:57,404 Stage-1 map = 29%,  reduce = 0%
2013-09-18 00:48:58,435 Stage-1 map = 32%,  reduce = 0%
2013-09-18 00:48:59,450 Stage-1 map = 37%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:00,470 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:01,482 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:02,500 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:03,514 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:04,524 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:05,530 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:06,542 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:07,548 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:08,560 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:09,568 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 15.22 sec
2013-09-18 00:49:10,574 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 69.2 sec
2013-09-18 00:49:11,591 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 154.6 sec
2013-09-18 00:49:12,600 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 386.29 sec
2013-09-18 00:49:13,616 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 684.84 sec
2013-09-18 00:49:14,628 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 904.62 sec
2013-09-18 00:49:15,634 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 999.79 sec
2013-09-18 00:49:16,640 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:17,645 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:18,651 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:19,657 Stage-1 map = 100%,  reduce = 2%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:20,664 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:21,693 Stage-1 map = 100%,  reduce = 17%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:22,703 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:23,709 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:24,715 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:25,721 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:26,728 Stage-1 map = 100%,  reduce = 41%, Cumulative CPU 1097.36 sec
2013-09-18 00:49:27,736 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 1108.08 sec
2013-09-18 00:49:28,742 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 1108.08 sec
2013-09-18 00:49:29,748 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1108.08 sec
2013-09-18 00:49:30,754 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1108.08 sec
2013-09-18 00:49:31,760 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 1124.02 sec
2013-09-18 00:49:32,766 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1139.64 sec
2013-09-18 00:49:33,772 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 1139.64 sec
2013-09-18 00:49:34,778 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 1139.64 sec
2013-09-18 00:49:35,783 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 1139.64 sec
2013-09-18 00:49:36,789 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1160.79 sec
2013-09-18 00:49:37,795 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1160.79 sec
2013-09-18 00:49:38,801 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1313.18 sec
2013-09-18 00:49:39,807 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1314.93 sec
2013-09-18 00:49:40,813 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1321.52 sec
2013-09-18 00:49:41,819 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1321.52 sec
2013-09-18 00:49:42,825 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1332.94 sec
2013-09-18 00:49:43,911 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1332.94 sec
2013-09-18 00:49:44,917 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1332.94 sec
2013-09-18 00:49:45,922 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1332.94 sec
2013-09-18 00:49:46,926 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1332.94 sec
2013-09-18 00:49:47,931 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1340.83 sec
2013-09-18 00:49:50,236 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1340.83 sec
2013-09-18 00:49:51,240 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1340.83 sec
2013-09-18 00:49:52,245 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1340.83 sec
2013-09-18 00:49:53,256 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1340.83 sec
MapReduce Total cumulative CPU time: 22 minutes 20 seconds 830 msec
Ended Job = job_201309172235_0128
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0129
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 00:50:01,798 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:50:03,805 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 00:50:04,810 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 00:50:05,814 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 00:50:06,819 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 00:50:07,823 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 00:50:08,828 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 00:50:09,832 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 00:50:10,837 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.79 sec
2013-09-18 00:50:11,843 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.4 sec
2013-09-18 00:50:12,847 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.4 sec
MapReduce Total cumulative CPU time: 2 seconds 400 msec
Ended Job = job_201309172235_0129
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1340.83 sec   HDFS Read: 978257057 HDFS Write: 4324 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.4 sec   HDFS Read: 7901 HDFS Write: 790 SUCCESS
Total MapReduce CPU Time Spent: 22 minutes 23 seconds 230 msec
OK
Time taken: 106.493 seconds, Fetched: 25 row(s)
hive> quit;
-- считаем средние длины URL для крупных счётчиков.;


times: 1
query: SELECT SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1, GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)),        avg(length(Referer)) AS l, count(*) AS c, MAX(Referer) FROM hits_100m WHERE Referer != '' GROUP BY SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1,     GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)) HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_8110@mturlrep13_201309180050_337201518.txt
hive> SELECT SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1, GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)),        avg(length(Referer)) AS l, count(*) AS c, MAX(Referer) FROM hits_100m WHERE Referer != '' GROUP BY SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1,     GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)) HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;;
FAILED: SemanticException [Error 10011]: Line 1:337 Invalid function 'GREATEST'
hive> quit;

times: 2
query: SELECT SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1, GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)),        avg(length(Referer)) AS l, count(*) AS c, MAX(Referer) FROM hits_100m WHERE Referer != '' GROUP BY SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1,     GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)) HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_8329@mturlrep13_201309180050_1201800197.txt
hive> SELECT SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1, GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)),        avg(length(Referer)) AS l, count(*) AS c, MAX(Referer) FROM hits_100m WHERE Referer != '' GROUP BY SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1,     GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)) HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;;
FAILED: SemanticException [Error 10011]: Line 1:337 Invalid function 'GREATEST'
hive> quit;

times: 3
query: SELECT SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1, GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)),        avg(length(Referer)) AS l, count(*) AS c, MAX(Referer) FROM hits_100m WHERE Referer != '' GROUP BY SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1,     GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)) HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_8542@mturlrep13_201309180050_468390903.txt
hive> SELECT SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1, GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)),        avg(length(Referer)) AS l, count(*) AS c, MAX(Referer) FROM hits_100m WHERE Referer != '' GROUP BY SUBSTRING(SUBSTRING(Referer, FIND_IN_SET('//',  Referer) + 2), 1,     GREATEST(0, FIND_IN_SET('/', SUBSTRING(Referer, FIND_IN_SET('//', Referer) + 2)) - 1)) HAVING count(*) > 100000 ORDER BY l DESC LIMIT 25;;
FAILED: SemanticException [Error 10011]: Line 1:337 Invalid function 'GREATEST'
hive> quit;
-- то же самое, но с разбивкой по доменам.;


times: 1
query: SELECT sum(ResolutionWidth), sum(ResolutionWidth + 1), sum(ResolutionWidth + 2), sum(ResolutionWidth + 3), sum(ResolutionWidth + 4), sum(ResolutionWidth + 5), sum(ResolutionWidth + 6), sum(ResolutionWidth + 7), sum(ResolutionWidth + 8), sum(ResolutionWidth + 9), sum(ResolutionWidth + 10), sum(ResolutionWidth + 11), sum(ResolutionWidth + 12), sum(ResolutionWidth + 13), sum(ResolutionWidth + 14), sum(ResolutionWidth + 15), sum(ResolutionWidth + 16), sum(ResolutionWidth + 17), sum(ResolutionWidth + 18), sum(ResolutionWidth + 19), sum(ResolutionWidth + 20), sum(ResolutionWidth + 21), sum(ResolutionWidth + 22), sum(ResolutionWidth + 23), sum(ResolutionWidth + 24), sum(ResolutionWidth + 25), sum(ResolutionWidth + 26), sum(ResolutionWidth + 27), sum(ResolutionWidth + 28), sum(ResolutionWidth + 29), sum(ResolutionWidth + 30), sum(ResolutionWidth + 31), sum(ResolutionWidth + 32), sum(ResolutionWidth + 33), sum(ResolutionWidth + 34), sum(ResolutionWidth + 35), sum(ResolutionWidth + 36), sum(ResolutionWidth + 37), sum(ResolutionWidth + 38), sum(ResolutionWidth + 39), sum(ResolutionWidth + 40), sum(ResolutionWidth + 41), sum(ResolutionWidth + 42), sum(ResolutionWidth + 43), sum(ResolutionWidth + 44), sum(ResolutionWidth + 45), sum(ResolutionWidth + 46), sum(ResolutionWidth + 47), sum(ResolutionWidth + 48), sum(ResolutionWidth + 49), sum(ResolutionWidth + 50), sum(ResolutionWidth + 51), sum(ResolutionWidth + 52), sum(ResolutionWidth + 53), sum(ResolutionWidth + 54), sum(ResolutionWidth + 55), sum(ResolutionWidth + 56), sum(ResolutionWidth + 57), sum(ResolutionWidth + 58), sum(ResolutionWidth + 59), sum(ResolutionWidth + 60), sum(ResolutionWidth + 61), sum(ResolutionWidth + 62), sum(ResolutionWidth + 63), sum(ResolutionWidth + 64), sum(ResolutionWidth + 65), sum(ResolutionWidth + 66), sum(ResolutionWidth + 67), sum(ResolutionWidth + 68), sum(ResolutionWidth + 69), sum(ResolutionWidth + 70), sum(ResolutionWidth + 71), sum(ResolutionWidth + 72), sum(ResolutionWidth + 73), sum(ResolutionWidth + 74), sum(ResolutionWidth + 75), sum(ResolutionWidth + 76), sum(ResolutionWidth + 77), sum(ResolutionWidth + 78), sum(ResolutionWidth + 79), sum(ResolutionWidth + 80), sum(ResolutionWidth + 81), sum(ResolutionWidth + 82), sum(ResolutionWidth + 83), sum(ResolutionWidth + 84), sum(ResolutionWidth + 85), sum(ResolutionWidth + 86), sum(ResolutionWidth + 87), sum(ResolutionWidth + 88), sum(ResolutionWidth + 89) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_8757@mturlrep13_201309180050_2105633958.txt
hive> SELECT sum(ResolutionWidth), sum(ResolutionWidth + 1), sum(ResolutionWidth + 2), sum(ResolutionWidth + 3), sum(ResolutionWidth + 4), sum(ResolutionWidth + 5), sum(ResolutionWidth + 6), sum(ResolutionWidth + 7), sum(ResolutionWidth + 8), sum(ResolutionWidth + 9), sum(ResolutionWidth + 10), sum(ResolutionWidth + 11), sum(ResolutionWidth + 12), sum(ResolutionWidth + 13), sum(ResolutionWidth + 14), sum(ResolutionWidth + 15), sum(ResolutionWidth + 16), sum(ResolutionWidth + 17), sum(ResolutionWidth + 18), sum(ResolutionWidth + 19), sum(ResolutionWidth + 20), sum(ResolutionWidth + 21), sum(ResolutionWidth + 22), sum(ResolutionWidth + 23), sum(ResolutionWidth + 24), sum(ResolutionWidth + 25), sum(ResolutionWidth + 26), sum(ResolutionWidth + 27), sum(ResolutionWidth + 28), sum(ResolutionWidth + 29), sum(ResolutionWidth + 30), sum(ResolutionWidth + 31), sum(ResolutionWidth + 32), sum(ResolutionWidth + 33), sum(ResolutionWidth + 34), sum(ResolutionWidth + 35), sum(ResolutionWidth + 36), sum(ResolutionWidth + 37), sum(ResolutionWidth + 38), sum(ResolutionWidth + 39), sum(ResolutionWidth + 40), sum(ResolutionWidth + 41), sum(ResolutionWidth + 42), sum(ResolutionWidth + 43), sum(ResolutionWidth + 44), sum(ResolutionWidth + 45), sum(ResolutionWidth + 46), sum(ResolutionWidth + 47), sum(ResolutionWidth + 48), sum(ResolutionWidth + 49), sum(ResolutionWidth + 50), sum(ResolutionWidth + 51), sum(ResolutionWidth + 52), sum(ResolutionWidth + 53), sum(ResolutionWidth + 54), sum(ResolutionWidth + 55), sum(ResolutionWidth + 56), sum(ResolutionWidth + 57), sum(ResolutionWidth + 58), sum(ResolutionWidth + 59), sum(ResolutionWidth + 60), sum(ResolutionWidth + 61), sum(ResolutionWidth + 62), sum(ResolutionWidth + 63), sum(ResolutionWidth + 64), sum(ResolutionWidth + 65), sum(ResolutionWidth + 66), sum(ResolutionWidth + 67), sum(ResolutionWidth + 68), sum(ResolutionWidth + 69), sum(ResolutionWidth + 70), sum(ResolutionWidth + 71), sum(ResolutionWidth + 72), sum(ResolutionWidth + 73), sum(ResolutionWidth + 74), sum(ResolutionWidth + 75), sum(ResolutionWidth + 76), sum(ResolutionWidth + 77), sum(ResolutionWidth + 78), sum(ResolutionWidth + 79), sum(ResolutionWidth + 80), sum(ResolutionWidth + 81), sum(ResolutionWidth + 82), sum(ResolutionWidth + 83), sum(ResolutionWidth + 84), sum(ResolutionWidth + 85), sum(ResolutionWidth + 86), sum(ResolutionWidth + 87), sum(ResolutionWidth + 88), sum(ResolutionWidth + 89) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0130
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:50:56,035 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:51:21,228 Stage-1 map = 1%,  reduce = 0%
2013-09-18 00:51:22,233 Stage-1 map = 4%,  reduce = 0%
2013-09-18 00:51:23,242 Stage-1 map = 5%,  reduce = 0%
2013-09-18 00:51:24,247 Stage-1 map = 6%,  reduce = 0%
2013-09-18 00:51:25,251 Stage-1 map = 11%,  reduce = 0%
2013-09-18 00:51:26,256 Stage-1 map = 13%,  reduce = 0%
2013-09-18 00:51:28,265 Stage-1 map = 14%,  reduce = 0%
2013-09-18 00:51:34,323 Stage-1 map = 17%,  reduce = 0%
2013-09-18 00:51:35,333 Stage-1 map = 19%,  reduce = 0%
2013-09-18 00:51:36,342 Stage-1 map = 20%,  reduce = 0%
2013-09-18 00:51:37,347 Stage-1 map = 21%,  reduce = 0%
2013-09-18 00:51:38,352 Stage-1 map = 25%,  reduce = 0%
2013-09-18 00:51:39,363 Stage-1 map = 26%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:40,382 Stage-1 map = 26%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:41,395 Stage-1 map = 27%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:42,410 Stage-1 map = 27%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:43,416 Stage-1 map = 29%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:44,421 Stage-1 map = 31%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:45,427 Stage-1 map = 32%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:46,437 Stage-1 map = 32%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:47,444 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:48,450 Stage-1 map = 33%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:49,455 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:50,461 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:51,466 Stage-1 map = 39%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:52,472 Stage-1 map = 39%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:53,478 Stage-1 map = 41%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:54,483 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:55,489 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:56,494 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:57,500 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:58,535 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:51:59,548 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:00,553 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:01,575 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:02,586 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:03,591 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:04,597 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:05,602 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:06,608 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:07,614 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:08,653 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:09,659 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:10,687 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:11,702 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:13,057 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:14,078 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:15,103 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:16,131 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:17,155 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:18,174 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:19,198 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 1178.1 sec
2013-09-18 00:52:20,210 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 1212.03 sec
2013-09-18 00:52:21,248 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 1212.03 sec
2013-09-18 00:52:22,262 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 1212.03 sec
2013-09-18 00:52:23,274 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 1212.03 sec
2013-09-18 00:52:24,280 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 1212.03 sec
2013-09-18 00:52:25,290 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 1212.03 sec
2013-09-18 00:52:26,311 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 1212.03 sec
2013-09-18 00:52:27,322 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 1257.13 sec
2013-09-18 00:52:28,327 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 1347.07 sec
2013-09-18 00:52:29,333 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 1347.07 sec
2013-09-18 00:52:30,346 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 1393.25 sec
2013-09-18 00:52:31,358 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 1483.18 sec
2013-09-18 00:52:32,373 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 1674.0 sec
2013-09-18 00:52:33,378 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 1768.61 sec
2013-09-18 00:52:34,383 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 1815.43 sec
2013-09-18 00:52:35,389 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 1865.46 sec
2013-09-18 00:52:36,395 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 2120.58 sec
2013-09-18 00:52:37,401 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 2333.45 sec
2013-09-18 00:52:38,406 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 2333.45 sec
2013-09-18 00:52:39,412 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 3023.48 sec
2013-09-18 00:52:40,418 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 3027.43 sec
2013-09-18 00:52:41,424 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 3039.12 sec
2013-09-18 00:52:42,429 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 3047.25 sec
2013-09-18 00:52:43,434 Stage-1 map = 98%,  reduce = 25%, Cumulative CPU 3052.91 sec
2013-09-18 00:52:44,440 Stage-1 map = 99%,  reduce = 25%, Cumulative CPU 3052.91 sec
2013-09-18 00:52:45,453 Stage-1 map = 99%,  reduce = 25%, Cumulative CPU 3052.91 sec
2013-09-18 00:52:46,459 Stage-1 map = 99%,  reduce = 25%, Cumulative CPU 3061.93 sec
2013-09-18 00:52:47,464 Stage-1 map = 99%,  reduce = 25%, Cumulative CPU 3061.93 sec
2013-09-18 00:52:48,469 Stage-1 map = 99%,  reduce = 25%, Cumulative CPU 3061.93 sec
2013-09-18 00:52:49,474 Stage-1 map = 99%,  reduce = 25%, Cumulative CPU 3061.93 sec
2013-09-18 00:52:50,480 Stage-1 map = 99%,  reduce = 25%, Cumulative CPU 3061.93 sec
2013-09-18 00:52:51,485 Stage-1 map = 99%,  reduce = 25%, Cumulative CPU 3061.93 sec
2013-09-18 00:52:52,490 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 3091.79 sec
2013-09-18 00:52:53,495 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 3091.79 sec
2013-09-18 00:52:54,500 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 3091.79 sec
2013-09-18 00:52:55,504 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 3091.79 sec
2013-09-18 00:52:56,509 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 3091.79 sec
2013-09-18 00:52:57,514 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 3091.79 sec
2013-09-18 00:52:58,521 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3095.82 sec
2013-09-18 00:52:59,529 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3095.82 sec
2013-09-18 00:53:00,534 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3095.82 sec
MapReduce Total cumulative CPU time: 51 minutes 35 seconds 820 msec
Ended Job = job_201309172235_0130
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 3095.82 sec   HDFS Read: 77419054 HDFS Write: 1170 SUCCESS
Total MapReduce CPU Time Spent: 51 minutes 35 seconds 820 msec
OK
Time taken: 136.36 seconds, Fetched: 1 row(s)
hive> quit;

times: 2
query: SELECT sum(ResolutionWidth), sum(ResolutionWidth + 1), sum(ResolutionWidth + 2), sum(ResolutionWidth + 3), sum(ResolutionWidth + 4), sum(ResolutionWidth + 5), sum(ResolutionWidth + 6), sum(ResolutionWidth + 7), sum(ResolutionWidth + 8), sum(ResolutionWidth + 9), sum(ResolutionWidth + 10), sum(ResolutionWidth + 11), sum(ResolutionWidth + 12), sum(ResolutionWidth + 13), sum(ResolutionWidth + 14), sum(ResolutionWidth + 15), sum(ResolutionWidth + 16), sum(ResolutionWidth + 17), sum(ResolutionWidth + 18), sum(ResolutionWidth + 19), sum(ResolutionWidth + 20), sum(ResolutionWidth + 21), sum(ResolutionWidth + 22), sum(ResolutionWidth + 23), sum(ResolutionWidth + 24), sum(ResolutionWidth + 25), sum(ResolutionWidth + 26), sum(ResolutionWidth + 27), sum(ResolutionWidth + 28), sum(ResolutionWidth + 29), sum(ResolutionWidth + 30), sum(ResolutionWidth + 31), sum(ResolutionWidth + 32), sum(ResolutionWidth + 33), sum(ResolutionWidth + 34), sum(ResolutionWidth + 35), sum(ResolutionWidth + 36), sum(ResolutionWidth + 37), sum(ResolutionWidth + 38), sum(ResolutionWidth + 39), sum(ResolutionWidth + 40), sum(ResolutionWidth + 41), sum(ResolutionWidth + 42), sum(ResolutionWidth + 43), sum(ResolutionWidth + 44), sum(ResolutionWidth + 45), sum(ResolutionWidth + 46), sum(ResolutionWidth + 47), sum(ResolutionWidth + 48), sum(ResolutionWidth + 49), sum(ResolutionWidth + 50), sum(ResolutionWidth + 51), sum(ResolutionWidth + 52), sum(ResolutionWidth + 53), sum(ResolutionWidth + 54), sum(ResolutionWidth + 55), sum(ResolutionWidth + 56), sum(ResolutionWidth + 57), sum(ResolutionWidth + 58), sum(ResolutionWidth + 59), sum(ResolutionWidth + 60), sum(ResolutionWidth + 61), sum(ResolutionWidth + 62), sum(ResolutionWidth + 63), sum(ResolutionWidth + 64), sum(ResolutionWidth + 65), sum(ResolutionWidth + 66), sum(ResolutionWidth + 67), sum(ResolutionWidth + 68), sum(ResolutionWidth + 69), sum(ResolutionWidth + 70), sum(ResolutionWidth + 71), sum(ResolutionWidth + 72), sum(ResolutionWidth + 73), sum(ResolutionWidth + 74), sum(ResolutionWidth + 75), sum(ResolutionWidth + 76), sum(ResolutionWidth + 77), sum(ResolutionWidth + 78), sum(ResolutionWidth + 79), sum(ResolutionWidth + 80), sum(ResolutionWidth + 81), sum(ResolutionWidth + 82), sum(ResolutionWidth + 83), sum(ResolutionWidth + 84), sum(ResolutionWidth + 85), sum(ResolutionWidth + 86), sum(ResolutionWidth + 87), sum(ResolutionWidth + 88), sum(ResolutionWidth + 89) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_13963@mturlrep13_201309180053_177355133.txt
hive> SELECT sum(ResolutionWidth), sum(ResolutionWidth + 1), sum(ResolutionWidth + 2), sum(ResolutionWidth + 3), sum(ResolutionWidth + 4), sum(ResolutionWidth + 5), sum(ResolutionWidth + 6), sum(ResolutionWidth + 7), sum(ResolutionWidth + 8), sum(ResolutionWidth + 9), sum(ResolutionWidth + 10), sum(ResolutionWidth + 11), sum(ResolutionWidth + 12), sum(ResolutionWidth + 13), sum(ResolutionWidth + 14), sum(ResolutionWidth + 15), sum(ResolutionWidth + 16), sum(ResolutionWidth + 17), sum(ResolutionWidth + 18), sum(ResolutionWidth + 19), sum(ResolutionWidth + 20), sum(ResolutionWidth + 21), sum(ResolutionWidth + 22), sum(ResolutionWidth + 23), sum(ResolutionWidth + 24), sum(ResolutionWidth + 25), sum(ResolutionWidth + 26), sum(ResolutionWidth + 27), sum(ResolutionWidth + 28), sum(ResolutionWidth + 29), sum(ResolutionWidth + 30), sum(ResolutionWidth + 31), sum(ResolutionWidth + 32), sum(ResolutionWidth + 33), sum(ResolutionWidth + 34), sum(ResolutionWidth + 35), sum(ResolutionWidth + 36), sum(ResolutionWidth + 37), sum(ResolutionWidth + 38), sum(ResolutionWidth + 39), sum(ResolutionWidth + 40), sum(ResolutionWidth + 41), sum(ResolutionWidth + 42), sum(ResolutionWidth + 43), sum(ResolutionWidth + 44), sum(ResolutionWidth + 45), sum(ResolutionWidth + 46), sum(ResolutionWidth + 47), sum(ResolutionWidth + 48), sum(ResolutionWidth + 49), sum(ResolutionWidth + 50), sum(ResolutionWidth + 51), sum(ResolutionWidth + 52), sum(ResolutionWidth + 53), sum(ResolutionWidth + 54), sum(ResolutionWidth + 55), sum(ResolutionWidth + 56), sum(ResolutionWidth + 57), sum(ResolutionWidth + 58), sum(ResolutionWidth + 59), sum(ResolutionWidth + 60), sum(ResolutionWidth + 61), sum(ResolutionWidth + 62), sum(ResolutionWidth + 63), sum(ResolutionWidth + 64), sum(ResolutionWidth + 65), sum(ResolutionWidth + 66), sum(ResolutionWidth + 67), sum(ResolutionWidth + 68), sum(ResolutionWidth + 69), sum(ResolutionWidth + 70), sum(ResolutionWidth + 71), sum(ResolutionWidth + 72), sum(ResolutionWidth + 73), sum(ResolutionWidth + 74), sum(ResolutionWidth + 75), sum(ResolutionWidth + 76), sum(ResolutionWidth + 77), sum(ResolutionWidth + 78), sum(ResolutionWidth + 79), sum(ResolutionWidth + 80), sum(ResolutionWidth + 81), sum(ResolutionWidth + 82), sum(ResolutionWidth + 83), sum(ResolutionWidth + 84), sum(ResolutionWidth + 85), sum(ResolutionWidth + 86), sum(ResolutionWidth + 87), sum(ResolutionWidth + 88), sum(ResolutionWidth + 89) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0131
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:53:11,219 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:53:36,459 Stage-1 map = 2%,  reduce = 0%
2013-09-18 00:53:38,478 Stage-1 map = 4%,  reduce = 0%
2013-09-18 00:53:39,508 Stage-1 map = 7%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:40,526 Stage-1 map = 10%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:41,534 Stage-1 map = 13%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:42,546 Stage-1 map = 13%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:43,564 Stage-1 map = 13%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:44,570 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:45,576 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:46,582 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:47,588 Stage-1 map = 15%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:48,594 Stage-1 map = 15%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:49,600 Stage-1 map = 17%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:50,610 Stage-1 map = 20%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:51,667 Stage-1 map = 22%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:52,674 Stage-1 map = 23%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:53,680 Stage-1 map = 25%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:54,685 Stage-1 map = 25%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:55,691 Stage-1 map = 25%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:56,703 Stage-1 map = 27%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:57,708 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:58,720 Stage-1 map = 29%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:53:59,732 Stage-1 map = 30%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:00,738 Stage-1 map = 31%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:01,743 Stage-1 map = 32%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:02,749 Stage-1 map = 32%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:03,754 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:04,759 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:05,766 Stage-1 map = 37%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:06,771 Stage-1 map = 38%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:07,784 Stage-1 map = 39%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:08,789 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:09,795 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:10,800 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:11,806 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:12,811 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:13,823 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:14,829 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:15,834 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:16,844 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:17,850 Stage-1 map = 51%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:18,858 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:19,871 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:20,878 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:21,883 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:22,888 Stage-1 map = 57%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:23,894 Stage-1 map = 58%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:24,901 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:25,930 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:27,189 Stage-1 map = 62%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:28,194 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:29,201 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:30,206 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:31,220 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 679.85 sec
2013-09-18 00:54:32,227 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 726.21 sec
2013-09-18 00:54:33,241 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 726.21 sec
2013-09-18 00:54:34,246 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 726.21 sec
2013-09-18 00:54:35,255 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 726.21 sec
2013-09-18 00:54:36,261 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 726.21 sec
2013-09-18 00:54:37,266 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 726.21 sec
2013-09-18 00:54:38,275 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 726.21 sec
2013-09-18 00:54:39,280 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 2591.6 sec
2013-09-18 00:54:40,290 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 2591.6 sec
2013-09-18 00:54:41,295 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 2591.6 sec
2013-09-18 00:54:42,301 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 2596.52 sec
2013-09-18 00:54:43,327 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 2606.89 sec
2013-09-18 00:54:44,338 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 2623.82 sec
2013-09-18 00:54:45,343 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 2631.82 sec
2013-09-18 00:54:46,349 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 2631.82 sec
2013-09-18 00:54:47,355 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 2647.97 sec
2013-09-18 00:54:48,360 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 2657.08 sec
2013-09-18 00:54:49,367 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 2679.32 sec
2013-09-18 00:54:50,372 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 2712.4 sec
2013-09-18 00:54:51,377 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 2738.4 sec
2013-09-18 00:54:52,383 Stage-1 map = 92%,  reduce = 14%, Cumulative CPU 2779.62 sec
2013-09-18 00:54:53,388 Stage-1 map = 93%,  reduce = 14%, Cumulative CPU 2809.67 sec
2013-09-18 00:54:54,399 Stage-1 map = 95%,  reduce = 14%, Cumulative CPU 2872.97 sec
2013-09-18 00:54:55,404 Stage-1 map = 96%,  reduce = 14%, Cumulative CPU 2923.72 sec
2013-09-18 00:54:56,410 Stage-1 map = 97%,  reduce = 14%, Cumulative CPU 2923.72 sec
2013-09-18 00:54:57,414 Stage-1 map = 97%,  reduce = 14%, Cumulative CPU 2942.34 sec
2013-09-18 00:54:58,419 Stage-1 map = 98%,  reduce = 14%, Cumulative CPU 2962.2 sec
2013-09-18 00:54:59,424 Stage-1 map = 98%,  reduce = 14%, Cumulative CPU 2983.52 sec
2013-09-18 00:55:00,429 Stage-1 map = 98%,  reduce = 14%, Cumulative CPU 3004.99 sec
2013-09-18 00:55:01,435 Stage-1 map = 98%,  reduce = 27%, Cumulative CPU 3004.99 sec
2013-09-18 00:55:02,440 Stage-1 map = 99%,  reduce = 27%, Cumulative CPU 3004.99 sec
2013-09-18 00:55:03,445 Stage-1 map = 99%,  reduce = 27%, Cumulative CPU 3029.77 sec
2013-09-18 00:55:04,450 Stage-1 map = 99%,  reduce = 27%, Cumulative CPU 3029.77 sec
2013-09-18 00:55:05,454 Stage-1 map = 99%,  reduce = 27%, Cumulative CPU 3029.77 sec
2013-09-18 00:55:06,459 Stage-1 map = 99%,  reduce = 27%, Cumulative CPU 3057.4 sec
2013-09-18 00:55:07,464 Stage-1 map = 99%,  reduce = 32%, Cumulative CPU 3057.4 sec
2013-09-18 00:55:08,469 Stage-1 map = 99%,  reduce = 32%, Cumulative CPU 3057.4 sec
2013-09-18 00:55:09,474 Stage-1 map = 99%,  reduce = 32%, Cumulative CPU 3057.4 sec
2013-09-18 00:55:10,479 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 3089.43 sec
2013-09-18 00:55:11,484 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 3089.43 sec
2013-09-18 00:55:12,489 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 3089.43 sec
2013-09-18 00:55:13,495 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3093.95 sec
2013-09-18 00:55:14,501 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3093.95 sec
2013-09-18 00:55:15,506 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3093.95 sec
MapReduce Total cumulative CPU time: 51 minutes 33 seconds 950 msec
Ended Job = job_201309172235_0131
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 3093.95 sec   HDFS Read: 77419054 HDFS Write: 1170 SUCCESS
Total MapReduce CPU Time Spent: 51 minutes 33 seconds 950 msec
OK
Time taken: 133.102 seconds, Fetched: 1 row(s)
hive> quit;

times: 3
query: SELECT sum(ResolutionWidth), sum(ResolutionWidth + 1), sum(ResolutionWidth + 2), sum(ResolutionWidth + 3), sum(ResolutionWidth + 4), sum(ResolutionWidth + 5), sum(ResolutionWidth + 6), sum(ResolutionWidth + 7), sum(ResolutionWidth + 8), sum(ResolutionWidth + 9), sum(ResolutionWidth + 10), sum(ResolutionWidth + 11), sum(ResolutionWidth + 12), sum(ResolutionWidth + 13), sum(ResolutionWidth + 14), sum(ResolutionWidth + 15), sum(ResolutionWidth + 16), sum(ResolutionWidth + 17), sum(ResolutionWidth + 18), sum(ResolutionWidth + 19), sum(ResolutionWidth + 20), sum(ResolutionWidth + 21), sum(ResolutionWidth + 22), sum(ResolutionWidth + 23), sum(ResolutionWidth + 24), sum(ResolutionWidth + 25), sum(ResolutionWidth + 26), sum(ResolutionWidth + 27), sum(ResolutionWidth + 28), sum(ResolutionWidth + 29), sum(ResolutionWidth + 30), sum(ResolutionWidth + 31), sum(ResolutionWidth + 32), sum(ResolutionWidth + 33), sum(ResolutionWidth + 34), sum(ResolutionWidth + 35), sum(ResolutionWidth + 36), sum(ResolutionWidth + 37), sum(ResolutionWidth + 38), sum(ResolutionWidth + 39), sum(ResolutionWidth + 40), sum(ResolutionWidth + 41), sum(ResolutionWidth + 42), sum(ResolutionWidth + 43), sum(ResolutionWidth + 44), sum(ResolutionWidth + 45), sum(ResolutionWidth + 46), sum(ResolutionWidth + 47), sum(ResolutionWidth + 48), sum(ResolutionWidth + 49), sum(ResolutionWidth + 50), sum(ResolutionWidth + 51), sum(ResolutionWidth + 52), sum(ResolutionWidth + 53), sum(ResolutionWidth + 54), sum(ResolutionWidth + 55), sum(ResolutionWidth + 56), sum(ResolutionWidth + 57), sum(ResolutionWidth + 58), sum(ResolutionWidth + 59), sum(ResolutionWidth + 60), sum(ResolutionWidth + 61), sum(ResolutionWidth + 62), sum(ResolutionWidth + 63), sum(ResolutionWidth + 64), sum(ResolutionWidth + 65), sum(ResolutionWidth + 66), sum(ResolutionWidth + 67), sum(ResolutionWidth + 68), sum(ResolutionWidth + 69), sum(ResolutionWidth + 70), sum(ResolutionWidth + 71), sum(ResolutionWidth + 72), sum(ResolutionWidth + 73), sum(ResolutionWidth + 74), sum(ResolutionWidth + 75), sum(ResolutionWidth + 76), sum(ResolutionWidth + 77), sum(ResolutionWidth + 78), sum(ResolutionWidth + 79), sum(ResolutionWidth + 80), sum(ResolutionWidth + 81), sum(ResolutionWidth + 82), sum(ResolutionWidth + 83), sum(ResolutionWidth + 84), sum(ResolutionWidth + 85), sum(ResolutionWidth + 86), sum(ResolutionWidth + 87), sum(ResolutionWidth + 88), sum(ResolutionWidth + 89) FROM hits_100m;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_19854@mturlrep13_201309180055_1882171438.txt
hive> SELECT sum(ResolutionWidth), sum(ResolutionWidth + 1), sum(ResolutionWidth + 2), sum(ResolutionWidth + 3), sum(ResolutionWidth + 4), sum(ResolutionWidth + 5), sum(ResolutionWidth + 6), sum(ResolutionWidth + 7), sum(ResolutionWidth + 8), sum(ResolutionWidth + 9), sum(ResolutionWidth + 10), sum(ResolutionWidth + 11), sum(ResolutionWidth + 12), sum(ResolutionWidth + 13), sum(ResolutionWidth + 14), sum(ResolutionWidth + 15), sum(ResolutionWidth + 16), sum(ResolutionWidth + 17), sum(ResolutionWidth + 18), sum(ResolutionWidth + 19), sum(ResolutionWidth + 20), sum(ResolutionWidth + 21), sum(ResolutionWidth + 22), sum(ResolutionWidth + 23), sum(ResolutionWidth + 24), sum(ResolutionWidth + 25), sum(ResolutionWidth + 26), sum(ResolutionWidth + 27), sum(ResolutionWidth + 28), sum(ResolutionWidth + 29), sum(ResolutionWidth + 30), sum(ResolutionWidth + 31), sum(ResolutionWidth + 32), sum(ResolutionWidth + 33), sum(ResolutionWidth + 34), sum(ResolutionWidth + 35), sum(ResolutionWidth + 36), sum(ResolutionWidth + 37), sum(ResolutionWidth + 38), sum(ResolutionWidth + 39), sum(ResolutionWidth + 40), sum(ResolutionWidth + 41), sum(ResolutionWidth + 42), sum(ResolutionWidth + 43), sum(ResolutionWidth + 44), sum(ResolutionWidth + 45), sum(ResolutionWidth + 46), sum(ResolutionWidth + 47), sum(ResolutionWidth + 48), sum(ResolutionWidth + 49), sum(ResolutionWidth + 50), sum(ResolutionWidth + 51), sum(ResolutionWidth + 52), sum(ResolutionWidth + 53), sum(ResolutionWidth + 54), sum(ResolutionWidth + 55), sum(ResolutionWidth + 56), sum(ResolutionWidth + 57), sum(ResolutionWidth + 58), sum(ResolutionWidth + 59), sum(ResolutionWidth + 60), sum(ResolutionWidth + 61), sum(ResolutionWidth + 62), sum(ResolutionWidth + 63), sum(ResolutionWidth + 64), sum(ResolutionWidth + 65), sum(ResolutionWidth + 66), sum(ResolutionWidth + 67), sum(ResolutionWidth + 68), sum(ResolutionWidth + 69), sum(ResolutionWidth + 70), sum(ResolutionWidth + 71), sum(ResolutionWidth + 72), sum(ResolutionWidth + 73), sum(ResolutionWidth + 74), sum(ResolutionWidth + 75), sum(ResolutionWidth + 76), sum(ResolutionWidth + 77), sum(ResolutionWidth + 78), sum(ResolutionWidth + 79), sum(ResolutionWidth + 80), sum(ResolutionWidth + 81), sum(ResolutionWidth + 82), sum(ResolutionWidth + 83), sum(ResolutionWidth + 84), sum(ResolutionWidth + 85), sum(ResolutionWidth + 86), sum(ResolutionWidth + 87), sum(ResolutionWidth + 88), sum(ResolutionWidth + 89) FROM hits_100m;;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0132
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 1
2013-09-18 00:55:27,376 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:55:51,539 Stage-1 map = 1%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:55:52,555 Stage-1 map = 2%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:55:53,565 Stage-1 map = 5%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:55:54,571 Stage-1 map = 7%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:55:55,576 Stage-1 map = 10%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:55:56,580 Stage-1 map = 12%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:55:57,586 Stage-1 map = 13%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:55:58,602 Stage-1 map = 13%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:55:59,608 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:00,614 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:01,619 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:02,625 Stage-1 map = 15%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:03,631 Stage-1 map = 15%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:04,639 Stage-1 map = 17%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:05,645 Stage-1 map = 19%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:06,650 Stage-1 map = 20%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:07,655 Stage-1 map = 22%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:08,661 Stage-1 map = 24%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:09,666 Stage-1 map = 25%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:10,671 Stage-1 map = 27%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:11,676 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:12,681 Stage-1 map = 28%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:13,687 Stage-1 map = 29%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:14,692 Stage-1 map = 29%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:15,697 Stage-1 map = 32%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:16,703 Stage-1 map = 32%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:17,708 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:18,797 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:19,802 Stage-1 map = 35%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:20,808 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:21,813 Stage-1 map = 37%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:22,818 Stage-1 map = 39%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:23,824 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:24,829 Stage-1 map = 42%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:25,834 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:26,840 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:27,845 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:28,850 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:29,855 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:30,861 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:31,866 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:32,871 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:33,876 Stage-1 map = 51%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:34,881 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:35,887 Stage-1 map = 54%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:36,899 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:38,073 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:39,078 Stage-1 map = 58%,  reduce = 0%, Cumulative CPU 28.25 sec
2013-09-18 00:56:40,093 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 2125.98 sec
2013-09-18 00:56:41,098 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 2125.98 sec
2013-09-18 00:56:42,104 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 2125.98 sec
2013-09-18 00:56:43,109 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 2125.98 sec
2013-09-18 00:56:44,116 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:45,121 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:46,127 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:47,136 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:48,141 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:49,147 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:50,152 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:51,158 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:52,163 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:53,168 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:54,174 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 2129.75 sec
2013-09-18 00:56:55,179 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 2144.3 sec
2013-09-18 00:56:56,184 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 2144.3 sec
2013-09-18 00:56:57,189 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 2160.94 sec
2013-09-18 00:56:58,195 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 2178.2 sec
2013-09-18 00:56:59,200 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 2178.2 sec
2013-09-18 00:57:00,205 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 2199.66 sec
2013-09-18 00:57:01,210 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 2218.05 sec
2013-09-18 00:57:02,216 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 2282.19 sec
2013-09-18 00:57:03,221 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 2304.47 sec
2013-09-18 00:57:04,227 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 2349.8 sec
2013-09-18 00:57:05,233 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 2372.02 sec
2013-09-18 00:57:06,242 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 2427.89 sec
2013-09-18 00:57:07,248 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 2503.41 sec
2013-09-18 00:57:08,254 Stage-1 map = 94%,  reduce = 11%, Cumulative CPU 2613.26 sec
2013-09-18 00:57:09,259 Stage-1 map = 95%,  reduce = 11%, Cumulative CPU 2701.51 sec
2013-09-18 00:57:10,265 Stage-1 map = 96%,  reduce = 11%, Cumulative CPU 2758.3 sec
2013-09-18 00:57:11,271 Stage-1 map = 97%,  reduce = 22%, Cumulative CPU 2875.86 sec
2013-09-18 00:57:12,278 Stage-1 map = 97%,  reduce = 22%, Cumulative CPU 2875.86 sec
2013-09-18 00:57:13,284 Stage-1 map = 98%,  reduce = 22%, Cumulative CPU 2941.83 sec
2013-09-18 00:57:14,290 Stage-1 map = 99%,  reduce = 22%, Cumulative CPU 3011.19 sec
2013-09-18 00:57:15,296 Stage-1 map = 99%,  reduce = 22%, Cumulative CPU 3011.19 sec
2013-09-18 00:57:16,302 Stage-1 map = 99%,  reduce = 22%, Cumulative CPU 3011.19 sec
2013-09-18 00:57:17,307 Stage-1 map = 99%,  reduce = 22%, Cumulative CPU 3011.19 sec
2013-09-18 00:57:18,313 Stage-1 map = 99%,  reduce = 22%, Cumulative CPU 3011.19 sec
2013-09-18 00:57:19,319 Stage-1 map = 99%,  reduce = 22%, Cumulative CPU 3011.19 sec
2013-09-18 00:57:20,325 Stage-1 map = 99%,  reduce = 32%, Cumulative CPU 3011.19 sec
2013-09-18 00:57:21,330 Stage-1 map = 99%,  reduce = 32%, Cumulative CPU 3011.19 sec
2013-09-18 00:57:22,335 Stage-1 map = 99%,  reduce = 32%, Cumulative CPU 3011.19 sec
2013-09-18 00:57:23,340 Stage-1 map = 99%,  reduce = 32%, Cumulative CPU 3053.74 sec
2013-09-18 00:57:24,344 Stage-1 map = 99%,  reduce = 32%, Cumulative CPU 3053.74 sec
2013-09-18 00:57:25,348 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 3099.46 sec
2013-09-18 00:57:26,352 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 3099.46 sec
2013-09-18 00:57:27,356 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 3099.46 sec
2013-09-18 00:57:28,359 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 3099.46 sec
2013-09-18 00:57:29,364 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 3099.46 sec
2013-09-18 00:57:30,368 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 3099.46 sec
2013-09-18 00:57:31,373 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 3099.46 sec
2013-09-18 00:57:32,380 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3104.28 sec
2013-09-18 00:57:33,385 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3104.28 sec
MapReduce Total cumulative CPU time: 51 minutes 44 seconds 280 msec
Ended Job = job_201309172235_0132
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 1   Cumulative CPU: 3104.28 sec   HDFS Read: 77419054 HDFS Write: 1170 SUCCESS
Total MapReduce CPU Time Spent: 51 minutes 44 seconds 280 msec
OK
Time taken: 135.896 seconds, Fetched: 1 row(s)
hive> quit;
-- много тупых агрегатных функций.;


times: 1
query: SELECT SearchEngineID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_25111@mturlrep13_201309180057_1190943875.txt
hive> SELECT SearchEngineID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0133
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:57:50,216 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:58:00,383 Stage-1 map = 2%,  reduce = 0%
2013-09-18 00:58:04,415 Stage-1 map = 8%,  reduce = 0%
2013-09-18 00:58:07,430 Stage-1 map = 24%,  reduce = 0%
2013-09-18 00:58:10,044 Stage-1 map = 33%,  reduce = 0%
2013-09-18 00:58:11,136 Stage-1 map = 37%,  reduce = 0%
2013-09-18 00:58:13,987 Stage-1 map = 45%,  reduce = 0%, Cumulative CPU 10.4 sec
2013-09-18 00:58:14,998 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 10.4 sec
2013-09-18 00:58:16,016 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 44.12 sec
2013-09-18 00:58:17,029 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 44.12 sec
2013-09-18 00:58:19,269 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 44.12 sec
2013-09-18 00:58:20,277 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 54.05 sec
2013-09-18 00:58:21,283 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 54.05 sec
2013-09-18 00:58:22,290 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 217.63 sec
2013-09-18 00:58:23,297 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 293.68 sec
2013-09-18 00:58:24,303 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 354.53 sec
2013-09-18 00:58:25,310 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 431.46 sec
2013-09-18 00:58:26,316 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 532.5 sec
2013-09-18 00:58:27,322 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 532.5 sec
2013-09-18 00:58:28,329 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 532.5 sec
2013-09-18 00:58:29,337 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 532.5 sec
2013-09-18 00:58:30,344 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 532.5 sec
2013-09-18 00:58:31,351 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 532.5 sec
2013-09-18 00:58:32,359 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 532.5 sec
2013-09-18 00:58:33,367 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 532.5 sec
2013-09-18 00:58:34,374 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 532.5 sec
2013-09-18 00:58:35,379 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 532.5 sec
2013-09-18 00:58:36,387 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 600.15 sec
2013-09-18 00:58:37,394 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 600.15 sec
2013-09-18 00:58:38,402 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 622.76 sec
2013-09-18 00:58:39,408 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 656.89 sec
2013-09-18 00:58:40,415 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 656.89 sec
MapReduce Total cumulative CPU time: 10 minutes 56 seconds 890 msec
Ended Job = job_201309172235_0133
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0134
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 00:58:43,889 Stage-2 map = 0%,  reduce = 0%
2013-09-18 00:58:50,910 Stage-2 map = 9%,  reduce = 0%
2013-09-18 00:58:53,919 Stage-2 map = 18%,  reduce = 0%
2013-09-18 00:58:56,928 Stage-2 map = 27%,  reduce = 0%
2013-09-18 00:58:59,938 Stage-2 map = 45%,  reduce = 0%
2013-09-18 00:59:02,948 Stage-2 map = 55%,  reduce = 0%
2013-09-18 00:59:05,958 Stage-2 map = 64%,  reduce = 0%
2013-09-18 00:59:08,967 Stage-2 map = 73%,  reduce = 0%
2013-09-18 00:59:11,976 Stage-2 map = 82%,  reduce = 0%
2013-09-18 00:59:14,991 Stage-2 map = 91%,  reduce = 0%
2013-09-18 00:59:20,012 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.91 sec
2013-09-18 00:59:21,017 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.91 sec
2013-09-18 00:59:22,021 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.91 sec
2013-09-18 00:59:23,025 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.91 sec
2013-09-18 00:59:24,029 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.91 sec
2013-09-18 00:59:25,033 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.91 sec
2013-09-18 00:59:26,038 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.91 sec
2013-09-18 00:59:27,043 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 43.91 sec
2013-09-18 00:59:28,047 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 43.91 sec
2013-09-18 00:59:29,052 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 43.91 sec
2013-09-18 00:59:30,056 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 43.91 sec
2013-09-18 00:59:31,065 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 43.91 sec
2013-09-18 00:59:32,070 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 50.31 sec
2013-09-18 00:59:33,075 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 50.31 sec
MapReduce Total cumulative CPU time: 50 seconds 310 msec
Ended Job = job_201309172235_0134
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 656.89 sec   HDFS Read: 613013229 HDFS Write: 191007594 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 50.31 sec   HDFS Read: 191011171 HDFS Write: 399 SUCCESS
Total MapReduce CPU Time Spent: 11 minutes 47 seconds 200 msec
OK
Time taken: 113.348 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT SearchEngineID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_32655@mturlrep13_201309180059_1219302235.txt
hive> SELECT SearchEngineID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0135
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 00:59:42,709 Stage-1 map = 0%,  reduce = 0%
2013-09-18 00:59:54,021 Stage-1 map = 2%,  reduce = 0%
2013-09-18 00:59:55,026 Stage-1 map = 6%,  reduce = 0%
2013-09-18 00:59:56,031 Stage-1 map = 13%,  reduce = 0%
2013-09-18 00:59:57,047 Stage-1 map = 19%,  reduce = 0%
2013-09-18 00:59:58,067 Stage-1 map = 26%,  reduce = 0%
2013-09-18 00:59:59,081 Stage-1 map = 39%,  reduce = 0%
2013-09-18 01:00:00,095 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 11.06 sec
2013-09-18 01:00:01,103 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 22.66 sec
2013-09-18 01:00:02,114 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 93.97 sec
2013-09-18 01:00:03,127 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 133.19 sec
2013-09-18 01:00:04,152 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 186.98 sec
2013-09-18 01:00:05,168 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 248.68 sec
2013-09-18 01:00:06,178 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 281.8 sec
2013-09-18 01:00:07,185 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 545.59 sec
2013-09-18 01:00:08,191 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 563.72 sec
2013-09-18 01:00:09,197 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 563.72 sec
2013-09-18 01:00:10,203 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 563.72 sec
2013-09-18 01:00:11,210 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 563.72 sec
2013-09-18 01:00:12,218 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 563.72 sec
2013-09-18 01:00:13,233 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 563.72 sec
2013-09-18 01:00:14,240 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 563.72 sec
2013-09-18 01:00:15,247 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 563.72 sec
2013-09-18 01:00:16,253 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 563.72 sec
2013-09-18 01:00:17,261 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 563.72 sec
2013-09-18 01:00:18,270 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 574.8 sec
2013-09-18 01:00:19,278 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 609.56 sec
2013-09-18 01:00:20,285 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 690.28 sec
2013-09-18 01:00:21,291 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 690.28 sec
2013-09-18 01:00:22,299 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 690.28 sec
MapReduce Total cumulative CPU time: 11 minutes 30 seconds 280 msec
Ended Job = job_201309172235_0135
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0136
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 01:00:25,765 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:00:32,789 Stage-2 map = 9%,  reduce = 0%
2013-09-18 01:00:35,800 Stage-2 map = 18%,  reduce = 0%
2013-09-18 01:00:38,811 Stage-2 map = 27%,  reduce = 0%
2013-09-18 01:00:41,823 Stage-2 map = 45%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:42,828 Stage-2 map = 45%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:43,834 Stage-2 map = 45%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:44,839 Stage-2 map = 55%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:45,844 Stage-2 map = 55%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:46,848 Stage-2 map = 55%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:47,853 Stage-2 map = 64%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:48,859 Stage-2 map = 64%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:49,864 Stage-2 map = 64%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:50,869 Stage-2 map = 73%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:51,874 Stage-2 map = 73%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:52,879 Stage-2 map = 73%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:53,885 Stage-2 map = 91%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:54,890 Stage-2 map = 91%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:55,895 Stage-2 map = 91%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:56,901 Stage-2 map = 91%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:57,906 Stage-2 map = 91%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:58,911 Stage-2 map = 91%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:00:59,916 Stage-2 map = 91%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:01:00,921 Stage-2 map = 91%,  reduce = 0%, Cumulative CPU 17.28 sec
2013-09-18 01:01:01,926 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.57 sec
2013-09-18 01:01:02,936 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.57 sec
2013-09-18 01:01:03,941 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.57 sec
2013-09-18 01:01:04,945 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.57 sec
2013-09-18 01:01:05,949 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.57 sec
2013-09-18 01:01:06,954 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.57 sec
2013-09-18 01:01:07,958 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 43.57 sec
2013-09-18 01:01:08,963 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 43.57 sec
2013-09-18 01:01:09,968 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 43.57 sec
2013-09-18 01:01:10,972 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 43.57 sec
2013-09-18 01:01:11,976 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 43.57 sec
2013-09-18 01:01:12,981 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 43.57 sec
2013-09-18 01:01:13,986 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 49.9 sec
2013-09-18 01:01:14,991 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 49.9 sec
MapReduce Total cumulative CPU time: 49 seconds 900 msec
Ended Job = job_201309172235_0136
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 690.28 sec   HDFS Read: 613013229 HDFS Write: 191007594 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 49.9 sec   HDFS Read: 191011171 HDFS Write: 399 SUCCESS
Total MapReduce CPU Time Spent: 12 minutes 20 seconds 180 msec
OK
Time taken: 100.032 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT SearchEngineID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_8933@mturlrep13_201309180101_1921637373.txt
hive> SELECT SearchEngineID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0137
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:01:24,685 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:01:35,760 Stage-1 map = 2%,  reduce = 0%
2013-09-18 01:01:36,769 Stage-1 map = 7%,  reduce = 0%
2013-09-18 01:01:37,774 Stage-1 map = 12%,  reduce = 0%
2013-09-18 01:01:38,803 Stage-1 map = 17%,  reduce = 0%
2013-09-18 01:01:39,847 Stage-1 map = 23%,  reduce = 0%
2013-09-18 01:01:40,868 Stage-1 map = 39%,  reduce = 0%, Cumulative CPU 286.36 sec
2013-09-18 01:01:41,883 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 286.36 sec
2013-09-18 01:01:42,895 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 289.4 sec
2013-09-18 01:01:43,902 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 301.74 sec
2013-09-18 01:01:44,920 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 341.22 sec
2013-09-18 01:01:45,934 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 365.65 sec
2013-09-18 01:01:46,942 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 373.98 sec
2013-09-18 01:01:47,949 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 460.32 sec
2013-09-18 01:01:48,957 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 557.15 sec
2013-09-18 01:01:49,963 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 557.15 sec
2013-09-18 01:01:50,969 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 557.15 sec
2013-09-18 01:01:51,974 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 557.15 sec
2013-09-18 01:01:52,986 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 557.15 sec
2013-09-18 01:01:53,993 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 557.15 sec
2013-09-18 01:01:55,000 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 557.15 sec
2013-09-18 01:01:56,006 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 557.15 sec
2013-09-18 01:01:57,013 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 557.15 sec
2013-09-18 01:01:58,020 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 557.15 sec
2013-09-18 01:01:59,026 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 557.15 sec
2013-09-18 01:02:00,035 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 568.49 sec
2013-09-18 01:02:01,042 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 603.17 sec
2013-09-18 01:02:02,048 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 671.6 sec
2013-09-18 01:02:03,056 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 682.9 sec
2013-09-18 01:02:04,062 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 682.9 sec
MapReduce Total cumulative CPU time: 11 minutes 22 seconds 900 msec
Ended Job = job_201309172235_0137
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0138
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 01:02:07,547 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:02:14,570 Stage-2 map = 9%,  reduce = 0%
2013-09-18 01:02:17,581 Stage-2 map = 18%,  reduce = 0%
2013-09-18 01:02:20,591 Stage-2 map = 27%,  reduce = 0%
2013-09-18 01:02:23,602 Stage-2 map = 45%,  reduce = 0%
2013-09-18 01:02:26,613 Stage-2 map = 55%,  reduce = 0%
2013-09-18 01:02:29,623 Stage-2 map = 64%,  reduce = 0%
2013-09-18 01:02:32,634 Stage-2 map = 73%,  reduce = 0%
2013-09-18 01:02:35,643 Stage-2 map = 91%,  reduce = 0%
2013-09-18 01:02:43,669 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 42.8 sec
2013-09-18 01:02:44,674 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 42.8 sec
2013-09-18 01:02:45,679 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 42.8 sec
2013-09-18 01:02:46,684 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 42.8 sec
2013-09-18 01:02:47,688 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 42.8 sec
2013-09-18 01:02:48,693 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 42.8 sec
2013-09-18 01:02:49,698 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 42.8 sec
2013-09-18 01:02:50,704 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 42.8 sec
2013-09-18 01:02:51,708 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 42.8 sec
2013-09-18 01:02:52,713 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 42.8 sec
2013-09-18 01:02:53,717 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 42.8 sec
2013-09-18 01:02:54,722 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 49.05 sec
2013-09-18 01:02:55,728 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 49.05 sec
2013-09-18 01:02:56,733 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 49.05 sec
MapReduce Total cumulative CPU time: 49 seconds 50 msec
Ended Job = job_201309172235_0138
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 682.9 sec   HDFS Read: 613013229 HDFS Write: 191007594 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 49.05 sec   HDFS Read: 191011171 HDFS Write: 399 SUCCESS
Total MapReduce CPU Time Spent: 12 minutes 11 seconds 950 msec
OK
Time taken: 99.905 seconds, Fetched: 10 row(s)
hive> quit;
-- сложная агрегация, для больших таблиц может не хватить оперативки.;


times: 1
query: SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_16458@mturlrep13_201309180103_1014086590.txt
hive> SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0139
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:03:15,821 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:03:25,881 Stage-1 map = 2%,  reduce = 0%
2013-09-18 01:03:28,896 Stage-1 map = 3%,  reduce = 0%
2013-09-18 01:03:30,685 Stage-1 map = 11%,  reduce = 0%
2013-09-18 01:03:31,692 Stage-1 map = 12%,  reduce = 0%
2013-09-18 01:03:33,713 Stage-1 map = 24%,  reduce = 0%
2013-09-18 01:03:34,719 Stage-1 map = 25%,  reduce = 0%
2013-09-18 01:03:41,834 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 361.72 sec
2013-09-18 01:03:42,856 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 361.72 sec
2013-09-18 01:03:43,882 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 363.16 sec
2013-09-18 01:03:44,896 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 378.03 sec
2013-09-18 01:03:45,904 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 403.06 sec
2013-09-18 01:03:46,911 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 418.77 sec
2013-09-18 01:03:48,058 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 434.02 sec
2013-09-18 01:03:49,066 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 460.07 sec
2013-09-18 01:03:50,076 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 509.72 sec
2013-09-18 01:03:51,083 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 547.13 sec
2013-09-18 01:03:52,089 Stage-1 map = 100%,  reduce = 2%, Cumulative CPU 547.13 sec
2013-09-18 01:03:53,096 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 547.13 sec
2013-09-18 01:03:54,105 Stage-1 map = 100%,  reduce = 20%, Cumulative CPU 547.13 sec
2013-09-18 01:03:55,112 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 547.13 sec
2013-09-18 01:03:56,118 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 547.13 sec
2013-09-18 01:03:57,125 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 547.13 sec
2013-09-18 01:03:58,132 Stage-1 map = 100%,  reduce = 65%, Cumulative CPU 547.13 sec
2013-09-18 01:03:59,138 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 547.13 sec
2013-09-18 01:04:00,145 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 547.13 sec
2013-09-18 01:04:01,154 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 560.31 sec
2013-09-18 01:04:02,162 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 573.82 sec
2013-09-18 01:04:03,169 Stage-1 map = 100%,  reduce = 87%, Cumulative CPU 600.97 sec
2013-09-18 01:04:04,176 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 683.89 sec
2013-09-18 01:04:05,182 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 683.89 sec
2013-09-18 01:04:06,188 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 697.41 sec
2013-09-18 01:04:07,193 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 697.41 sec
MapReduce Total cumulative CPU time: 11 minutes 37 seconds 410 msec
Ended Job = job_201309172235_0139
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0140
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-18 01:04:10,746 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:04:20,776 Stage-2 map = 10%,  reduce = 0%
2013-09-18 01:04:23,786 Stage-2 map = 18%,  reduce = 0%
2013-09-18 01:04:26,796 Stage-2 map = 28%,  reduce = 0%
2013-09-18 01:04:29,806 Stage-2 map = 37%,  reduce = 0%
2013-09-18 01:04:32,817 Stage-2 map = 55%,  reduce = 0%
2013-09-18 01:04:38,835 Stage-2 map = 73%,  reduce = 0%
2013-09-18 01:04:44,858 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 69.67 sec
2013-09-18 01:04:45,863 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 69.67 sec
2013-09-18 01:04:46,868 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 69.67 sec
2013-09-18 01:04:47,873 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 69.67 sec
2013-09-18 01:04:48,879 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 69.67 sec
2013-09-18 01:04:49,884 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 81.11 sec
2013-09-18 01:04:50,889 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 81.11 sec
2013-09-18 01:04:51,894 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 81.11 sec
2013-09-18 01:04:52,898 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 81.11 sec
2013-09-18 01:04:53,903 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 81.11 sec
2013-09-18 01:04:54,907 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 81.11 sec
2013-09-18 01:04:55,912 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 81.11 sec
2013-09-18 01:04:56,917 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 100.5 sec
2013-09-18 01:04:57,927 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 100.5 sec
2013-09-18 01:04:58,931 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 100.5 sec
2013-09-18 01:04:59,935 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 100.5 sec
2013-09-18 01:05:00,939 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 100.5 sec
2013-09-18 01:05:01,943 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 100.5 sec
2013-09-18 01:05:02,947 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 100.5 sec
2013-09-18 01:05:03,952 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 100.5 sec
2013-09-18 01:05:04,956 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 100.5 sec
2013-09-18 01:05:05,960 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 100.5 sec
2013-09-18 01:05:06,965 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 100.5 sec
2013-09-18 01:05:07,969 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 113.63 sec
2013-09-18 01:05:08,973 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 113.63 sec
MapReduce Total cumulative CPU time: 1 minutes 53 seconds 630 msec
Ended Job = job_201309172235_0140
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 697.41 sec   HDFS Read: 1046532092 HDFS Write: 545445520 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 113.63 sec   HDFS Read: 545449242 HDFS Write: 418 SUCCESS
Total MapReduce CPU Time Spent: 13 minutes 31 seconds 40 msec
OK
Time taken: 124.002 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_24725@mturlrep13_201309180105_1494771182.txt
hive> SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0141
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:05:19,764 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:05:29,107 Stage-1 map = 1%,  reduce = 0%
2013-09-18 01:05:30,113 Stage-1 map = 4%,  reduce = 0%
2013-09-18 01:05:31,124 Stage-1 map = 5%,  reduce = 0%
2013-09-18 01:05:32,129 Stage-1 map = 12%,  reduce = 0%
2013-09-18 01:05:33,137 Stage-1 map = 20%,  reduce = 0%
2013-09-18 01:05:34,151 Stage-1 map = 22%,  reduce = 0%
2013-09-18 01:05:35,158 Stage-1 map = 36%,  reduce = 0%
2013-09-18 01:05:36,163 Stage-1 map = 39%,  reduce = 0%
2013-09-18 01:05:37,173 Stage-1 map = 43%,  reduce = 0%
2013-09-18 01:05:38,187 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 34.53 sec
2013-09-18 01:05:39,204 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 108.67 sec
2013-09-18 01:05:40,224 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 150.31 sec
2013-09-18 01:05:41,238 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 497.43 sec
2013-09-18 01:05:42,253 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 506.05 sec
2013-09-18 01:05:43,260 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 533.06 sec
2013-09-18 01:05:44,266 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 577.12 sec
2013-09-18 01:05:45,271 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 577.12 sec
2013-09-18 01:05:46,277 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 577.12 sec
2013-09-18 01:05:47,282 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 577.12 sec
2013-09-18 01:05:48,288 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 577.12 sec
2013-09-18 01:05:49,309 Stage-1 map = 100%,  reduce = 17%, Cumulative CPU 577.12 sec
2013-09-18 01:05:50,315 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 577.12 sec
2013-09-18 01:05:51,322 Stage-1 map = 100%,  reduce = 43%, Cumulative CPU 577.12 sec
2013-09-18 01:05:52,329 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 577.12 sec
2013-09-18 01:05:53,335 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 577.12 sec
2013-09-18 01:05:54,342 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 577.12 sec
2013-09-18 01:05:55,349 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 577.12 sec
2013-09-18 01:05:56,355 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 577.12 sec
2013-09-18 01:05:57,364 Stage-1 map = 100%,  reduce = 87%, Cumulative CPU 590.71 sec
2013-09-18 01:05:58,370 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 646.07 sec
2013-09-18 01:05:59,388 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 729.12 sec
2013-09-18 01:06:00,394 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 729.12 sec
2013-09-18 01:06:01,400 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 729.12 sec
MapReduce Total cumulative CPU time: 12 minutes 9 seconds 120 msec
Ended Job = job_201309172235_0141
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0142
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-18 01:06:03,914 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:06:14,948 Stage-2 map = 8%,  reduce = 0%
2013-09-18 01:06:17,957 Stage-2 map = 18%,  reduce = 0%
2013-09-18 01:06:20,967 Stage-2 map = 37%,  reduce = 0%
2013-09-18 01:06:26,985 Stage-2 map = 55%,  reduce = 0%
2013-09-18 01:06:33,003 Stage-2 map = 73%,  reduce = 0%
2013-09-18 01:06:39,021 Stage-2 map = 82%,  reduce = 0%
2013-09-18 01:06:44,041 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 87.48 sec
2013-09-18 01:06:45,046 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 87.48 sec
2013-09-18 01:06:46,051 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 87.48 sec
2013-09-18 01:06:47,056 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 87.48 sec
2013-09-18 01:06:48,061 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 87.48 sec
2013-09-18 01:06:49,066 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 87.48 sec
2013-09-18 01:06:50,071 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 99.29 sec
2013-09-18 01:06:51,076 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 99.29 sec
2013-09-18 01:06:52,092 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 99.29 sec
2013-09-18 01:06:53,097 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 99.29 sec
2013-09-18 01:06:54,101 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 99.29 sec
2013-09-18 01:06:55,106 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 99.29 sec
2013-09-18 01:06:56,110 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 99.29 sec
2013-09-18 01:06:57,115 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 99.29 sec
2013-09-18 01:06:58,119 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 99.29 sec
2013-09-18 01:06:59,123 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 99.29 sec
2013-09-18 01:07:00,127 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 99.29 sec
2013-09-18 01:07:01,132 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 99.29 sec
2013-09-18 01:07:02,136 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 99.29 sec
2013-09-18 01:07:03,140 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 99.29 sec
2013-09-18 01:07:04,144 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 99.29 sec
2013-09-18 01:07:05,148 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 99.29 sec
2013-09-18 01:07:06,154 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 112.67 sec
2013-09-18 01:07:07,159 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 112.67 sec
2013-09-18 01:07:08,164 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 112.67 sec
MapReduce Total cumulative CPU time: 1 minutes 52 seconds 670 msec
Ended Job = job_201309172235_0142
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 729.12 sec   HDFS Read: 1046532092 HDFS Write: 545445520 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 112.67 sec   HDFS Read: 545449242 HDFS Write: 418 SUCCESS
Total MapReduce CPU Time Spent: 14 minutes 1 seconds 790 msec
OK
Time taken: 117.366 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_583@mturlrep13_201309180107_219916780.txt
hive> SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m WHERE SearchPhrase != '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0143
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:07:18,186 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:07:29,269 Stage-1 map = 4%,  reduce = 0%
2013-09-18 01:07:30,388 Stage-1 map = 9%,  reduce = 0%
2013-09-18 01:07:31,400 Stage-1 map = 11%,  reduce = 0%
2013-09-18 01:07:32,410 Stage-1 map = 19%,  reduce = 0%
2013-09-18 01:07:33,417 Stage-1 map = 20%,  reduce = 0%
2013-09-18 01:07:34,436 Stage-1 map = 36%,  reduce = 0%
2013-09-18 01:07:35,442 Stage-1 map = 39%,  reduce = 0%
2013-09-18 01:07:36,448 Stage-1 map = 42%,  reduce = 0%
2013-09-18 01:07:37,463 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 36.86 sec
2013-09-18 01:07:38,488 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 89.65 sec
2013-09-18 01:07:39,524 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 170.2 sec
2013-09-18 01:07:40,544 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 229.86 sec
2013-09-18 01:07:41,573 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 524.54 sec
2013-09-18 01:07:42,584 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 543.69 sec
2013-09-18 01:07:43,591 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 583.3 sec
2013-09-18 01:07:44,597 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 583.3 sec
2013-09-18 01:07:45,602 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 583.3 sec
2013-09-18 01:07:46,609 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 583.3 sec
2013-09-18 01:07:47,617 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 583.3 sec
2013-09-18 01:07:48,627 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 583.3 sec
2013-09-18 01:07:49,634 Stage-1 map = 100%,  reduce = 35%, Cumulative CPU 583.3 sec
2013-09-18 01:07:50,641 Stage-1 map = 100%,  reduce = 38%, Cumulative CPU 583.3 sec
2013-09-18 01:07:51,648 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 583.3 sec
2013-09-18 01:07:52,655 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 583.3 sec
2013-09-18 01:07:53,662 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 583.3 sec
2013-09-18 01:07:54,669 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 583.3 sec
2013-09-18 01:07:55,676 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 583.3 sec
2013-09-18 01:07:56,686 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 610.88 sec
2013-09-18 01:07:57,693 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 665.19 sec
2013-09-18 01:07:58,698 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 735.33 sec
2013-09-18 01:07:59,704 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 735.33 sec
MapReduce Total cumulative CPU time: 12 minutes 15 seconds 330 msec
Ended Job = job_201309172235_0143
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0144
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-18 01:08:02,196 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:08:13,229 Stage-2 map = 18%,  reduce = 0%
2013-09-18 01:08:19,247 Stage-2 map = 37%,  reduce = 0%
2013-09-18 01:08:26,358 Stage-2 map = 55%,  reduce = 0%
2013-09-18 01:08:31,375 Stage-2 map = 73%,  reduce = 0%
2013-09-18 01:08:34,385 Stage-2 map = 82%,  reduce = 0%
2013-09-18 01:08:40,407 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 45.23 sec
2013-09-18 01:08:41,412 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 90.45 sec
2013-09-18 01:08:42,416 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 90.45 sec
2013-09-18 01:08:43,421 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 90.45 sec
2013-09-18 01:08:44,425 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 90.45 sec
2013-09-18 01:08:45,430 Stage-2 map = 92%,  reduce = 0%, Cumulative CPU 90.45 sec
2013-09-18 01:08:46,435 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 96.92 sec
2013-09-18 01:08:47,441 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 96.92 sec
2013-09-18 01:08:48,446 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 96.92 sec
2013-09-18 01:08:49,451 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 96.92 sec
2013-09-18 01:08:50,457 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 96.92 sec
2013-09-18 01:08:51,461 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 96.92 sec
2013-09-18 01:08:52,467 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 96.92 sec
2013-09-18 01:08:53,472 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 96.92 sec
2013-09-18 01:08:54,477 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 96.92 sec
2013-09-18 01:08:55,482 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 96.92 sec
2013-09-18 01:08:56,488 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 96.92 sec
2013-09-18 01:08:57,499 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 96.92 sec
2013-09-18 01:08:58,504 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 109.55 sec
2013-09-18 01:08:59,510 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 109.55 sec
MapReduce Total cumulative CPU time: 1 minutes 49 seconds 550 msec
Ended Job = job_201309172235_0144
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 735.33 sec   HDFS Read: 1046532092 HDFS Write: 545445520 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 109.55 sec   HDFS Read: 545449242 HDFS Write: 418 SUCCESS
Total MapReduce CPU Time Spent: 14 minutes 4 seconds 880 msec
OK
Time taken: 109.524 seconds, Fetched: 10 row(s)
hive> quit;
-- агрегация по двум полям, которая ничего не агрегирует. Для больших таблиц выполнить не получится.;


times: 1
query: SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_9002@mturlrep13_201309180109_1127417909.txt
hive> SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0145
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:09:21,157 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:09:34,238 Stage-1 map = 1%,  reduce = 0%
2013-09-18 01:09:37,271 Stage-1 map = 2%,  reduce = 0%
2013-09-18 01:09:38,281 Stage-1 map = 5%,  reduce = 0%
2013-09-18 01:09:39,287 Stage-1 map = 6%,  reduce = 0%
2013-09-18 01:09:40,293 Stage-1 map = 11%,  reduce = 0%
2013-09-18 01:09:41,307 Stage-1 map = 15%,  reduce = 0%
2013-09-18 01:09:42,344 Stage-1 map = 16%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:43,369 Stage-1 map = 18%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:44,396 Stage-1 map = 21%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:45,410 Stage-1 map = 22%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:46,422 Stage-1 map = 26%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:47,429 Stage-1 map = 32%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:48,440 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:49,454 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:50,470 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:51,477 Stage-1 map = 45%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:52,487 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:53,494 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:54,510 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:55,539 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:56,564 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:57,574 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:58,586 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:09:59,593 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:10:00,605 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:10:01,613 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 424.15 sec
2013-09-18 01:10:02,631 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 461.38 sec
2013-09-18 01:10:03,659 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 461.38 sec
2013-09-18 01:10:04,670 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 504.24 sec
2013-09-18 01:10:05,681 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 504.24 sec
2013-09-18 01:10:06,688 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 526.72 sec
2013-09-18 01:10:07,709 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 618.33 sec
2013-09-18 01:10:08,722 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 708.66 sec
2013-09-18 01:10:09,737 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 906.45 sec
2013-09-18 01:10:10,743 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 956.72 sec
2013-09-18 01:10:11,750 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 1148.71 sec
2013-09-18 01:10:12,756 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1258.63 sec
2013-09-18 01:10:13,762 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1258.63 sec
2013-09-18 01:10:14,770 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1288.05 sec
2013-09-18 01:10:15,777 Stage-1 map = 100%,  reduce = 2%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:16,784 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:17,807 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:18,813 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:19,819 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:20,825 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:21,831 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:22,837 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:26,141 Stage-1 map = 100%,  reduce = 45%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:27,147 Stage-1 map = 100%,  reduce = 45%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:28,153 Stage-1 map = 100%,  reduce = 48%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:29,328 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:30,334 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:31,340 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:32,346 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:33,352 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:34,359 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:36,108 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:37,114 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:38,121 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:39,470 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:40,477 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:41,483 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1356.32 sec
2013-09-18 01:10:42,629 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:45,862 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:48,964 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:49,969 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:50,976 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:51,982 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:52,988 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:53,993 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:54,999 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:56,005 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:57,010 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:58,016 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1569.69 sec
2013-09-18 01:10:59,021 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:00,027 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:01,033 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:02,038 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:03,043 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:04,049 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:06,499 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:07,504 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:08,509 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:09,513 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:10,519 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:12,086 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:13,091 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:14,097 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:15,102 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:16,108 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:17,635 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:18,641 Stage-1 map = 100%,  reduce = 85%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:21,528 Stage-1 map = 100%,  reduce = 87%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:24,908 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:25,914 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:26,919 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:27,926 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:28,931 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:29,959 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:30,966 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:31,971 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:33,303 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:34,309 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:35,315 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1569.69 sec
2013-09-18 01:11:36,321 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1607.57 sec
2013-09-18 01:11:37,327 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1607.57 sec
2013-09-18 01:11:38,336 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1728.83 sec
2013-09-18 01:11:39,342 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1812.48 sec
2013-09-18 01:11:40,348 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1812.48 sec
2013-09-18 01:11:41,354 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1812.48 sec
2013-09-18 01:11:42,360 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 2013.45 sec
2013-09-18 01:11:43,365 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 2030.96 sec
2013-09-18 01:11:44,371 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 2030.96 sec
MapReduce Total cumulative CPU time: 33 minutes 50 seconds 960 msec
Ended Job = job_201309172235_0145
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0146
Hadoop job information for Stage-2: number of mappers: 14; number of reducers: 1
2013-09-18 01:11:47,866 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:12:01,932 Stage-2 map = 2%,  reduce = 0%
2013-09-18 01:12:04,943 Stage-2 map = 15%,  reduce = 0%
2013-09-18 01:12:07,952 Stage-2 map = 19%,  reduce = 0%
2013-09-18 01:12:09,062 Stage-2 map = 22%,  reduce = 0%
2013-09-18 01:12:11,068 Stage-2 map = 26%,  reduce = 0%
2013-09-18 01:12:15,081 Stage-2 map = 28%,  reduce = 0%
2013-09-18 01:12:17,087 Stage-2 map = 34%,  reduce = 0%
2013-09-18 01:12:18,091 Stage-2 map = 43%,  reduce = 0%
2013-09-18 01:12:21,102 Stage-2 map = 48%,  reduce = 0%
2013-09-18 01:12:23,108 Stage-2 map = 52%,  reduce = 0%
2013-09-18 01:12:24,112 Stage-2 map = 64%,  reduce = 0%
2013-09-18 01:12:30,131 Stage-2 map = 73%,  reduce = 0%
2013-09-18 01:12:33,143 Stage-2 map = 78%,  reduce = 0%
2013-09-18 01:12:35,159 Stage-2 map = 79%,  reduce = 0%, Cumulative CPU 54.68 sec
2013-09-18 01:12:36,935 Stage-2 map = 79%,  reduce = 0%, Cumulative CPU 54.68 sec
2013-09-18 01:12:37,940 Stage-2 map = 79%,  reduce = 0%, Cumulative CPU 54.68 sec
2013-09-18 01:12:38,945 Stage-2 map = 81%,  reduce = 0%, Cumulative CPU 111.92 sec
2013-09-18 01:12:39,950 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 231.05 sec
2013-09-18 01:12:40,956 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 231.05 sec
2013-09-18 01:12:41,961 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 231.05 sec
2013-09-18 01:12:42,966 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 842.33 sec
2013-09-18 01:12:43,971 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 842.33 sec
2013-09-18 01:12:45,166 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 842.33 sec
2013-09-18 01:12:46,171 Stage-2 map = 86%,  reduce = 10%, Cumulative CPU 845.72 sec
2013-09-18 01:12:47,176 Stage-2 map = 95%,  reduce = 10%, Cumulative CPU 870.78 sec
2013-09-18 01:12:48,180 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 887.76 sec
2013-09-18 01:12:49,184 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 887.76 sec
2013-09-18 01:12:50,188 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 887.76 sec
2013-09-18 01:12:53,214 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 887.76 sec
2013-09-18 01:12:55,469 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 887.76 sec
2013-09-18 01:12:56,473 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 887.76 sec
2013-09-18 01:12:57,477 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 887.76 sec
2013-09-18 01:12:58,480 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 887.76 sec
2013-09-18 01:12:59,485 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 887.76 sec
2013-09-18 01:13:00,557 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 887.76 sec
2013-09-18 01:13:01,561 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 887.76 sec
2013-09-18 01:13:02,566 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 887.76 sec
2013-09-18 01:13:03,571 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 887.76 sec
2013-09-18 01:13:04,575 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 887.76 sec
2013-09-18 01:13:05,579 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 887.76 sec
2013-09-18 01:13:06,583 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 887.76 sec
2013-09-18 01:13:10,271 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 887.76 sec
2013-09-18 01:13:11,275 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 887.76 sec
2013-09-18 01:13:12,279 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 887.76 sec
2013-09-18 01:13:13,283 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.76 sec
2013-09-18 01:13:14,287 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.76 sec
2013-09-18 01:13:15,291 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.76 sec
2013-09-18 01:13:16,295 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.76 sec
2013-09-18 01:13:17,544 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.76 sec
2013-09-18 01:13:18,548 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.76 sec
2013-09-18 01:13:19,552 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.76 sec
2013-09-18 01:13:21,226 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 887.76 sec
2013-09-18 01:13:22,398 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 887.76 sec
2013-09-18 01:13:26,427 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 887.76 sec
2013-09-18 01:13:27,431 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 887.76 sec
2013-09-18 01:13:28,435 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 887.76 sec
2013-09-18 01:13:29,440 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 887.76 sec
2013-09-18 01:13:30,447 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 887.76 sec
2013-09-18 01:13:31,451 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 887.76 sec
2013-09-18 01:13:32,455 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 887.76 sec
2013-09-18 01:13:33,459 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 887.76 sec
2013-09-18 01:13:34,463 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 887.76 sec
2013-09-18 01:13:35,467 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 887.76 sec
2013-09-18 01:13:36,471 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 887.76 sec
2013-09-18 01:13:37,475 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 887.76 sec
2013-09-18 01:13:38,520 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 887.76 sec
2013-09-18 01:13:39,524 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 887.76 sec
2013-09-18 01:13:40,528 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 887.76 sec
2013-09-18 01:13:41,533 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 887.76 sec
2013-09-18 01:13:42,538 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 947.26 sec
2013-09-18 01:13:43,543 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 947.26 sec
2013-09-18 01:13:44,548 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:45,553 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:46,557 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:47,561 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:48,566 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:49,570 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:50,574 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:51,662 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:52,666 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:57,873 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:58,877 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:13:59,880 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:14:00,884 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:14:01,888 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:14:02,892 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:14:03,895 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:14:04,899 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:14:05,903 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 947.26 sec
2013-09-18 01:14:10,178 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 976.07 sec
2013-09-18 01:14:11,182 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 976.07 sec
2013-09-18 01:14:12,187 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 976.07 sec
MapReduce Total cumulative CPU time: 16 minutes 16 seconds 70 msec
Ended Job = job_201309172235_0146
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 2030.96 sec   HDFS Read: 869197327 HDFS Write: 4140713341 SUCCESS
Job 1: Map: 14  Reduce: 1   Cumulative CPU: 976.07 sec   HDFS Read: 4140841812 HDFS Write: 415 SUCCESS
Total MapReduce CPU Time Spent: 50 minutes 7 seconds 30 msec
OK
Time taken: 301.77 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_19253@mturlrep13_201309180114_1319337284.txt
hive> SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0147
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:14:21,934 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:14:38,062 Stage-1 map = 3%,  reduce = 0%
2013-09-18 01:14:39,067 Stage-1 map = 4%,  reduce = 0%
2013-09-18 01:14:40,081 Stage-1 map = 5%,  reduce = 0%
2013-09-18 01:14:41,089 Stage-1 map = 9%,  reduce = 0%
2013-09-18 01:14:42,106 Stage-1 map = 14%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:43,120 Stage-1 map = 15%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:44,128 Stage-1 map = 19%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:45,134 Stage-1 map = 27%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:46,141 Stage-1 map = 27%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:47,152 Stage-1 map = 29%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:48,158 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:49,164 Stage-1 map = 36%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:50,177 Stage-1 map = 38%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:51,183 Stage-1 map = 45%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:52,193 Stage-1 map = 48%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:53,199 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:54,206 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:55,212 Stage-1 map = 58%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:56,218 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:57,225 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:58,236 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:14:59,256 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 456.6 sec
2013-09-18 01:15:00,265 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 472.96 sec
2013-09-18 01:15:01,273 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 472.96 sec
2013-09-18 01:15:02,279 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 472.96 sec
2013-09-18 01:15:03,289 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 472.96 sec
2013-09-18 01:15:04,295 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 472.96 sec
2013-09-18 01:15:05,301 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 535.78 sec
2013-09-18 01:15:06,312 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 600.59 sec
2013-09-18 01:15:07,321 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 709.5 sec
2013-09-18 01:15:08,347 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 776.98 sec
2013-09-18 01:15:09,362 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 945.32 sec
2013-09-18 01:15:10,368 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 1072.86 sec
2013-09-18 01:15:11,374 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 1125.81 sec
2013-09-18 01:15:12,400 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1285.77 sec
2013-09-18 01:15:13,406 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1285.77 sec
2013-09-18 01:15:14,411 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1285.77 sec
2013-09-18 01:15:15,418 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:16,425 Stage-1 map = 100%,  reduce = 7%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:17,452 Stage-1 map = 100%,  reduce = 8%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:18,458 Stage-1 map = 100%,  reduce = 17%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:19,463 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:20,469 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:21,475 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:23,349 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:24,354 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:25,360 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:26,368 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:27,374 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:29,096 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:30,102 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:31,107 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:32,113 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:33,119 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:34,124 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:35,356 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:36,362 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:37,368 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:38,374 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:39,379 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:40,385 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:41,390 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1349.28 sec
2013-09-18 01:15:43,287 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:44,293 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:45,299 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:46,306 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:47,312 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:48,318 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:49,324 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:50,329 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:51,335 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:52,341 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:53,347 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:54,353 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:57,177 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:58,183 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1558.8 sec
2013-09-18 01:15:59,188 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1558.8 sec
2013-09-18 01:16:00,499 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:01,505 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:02,511 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:03,517 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:04,524 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:05,551 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:06,557 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:07,562 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:08,567 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:09,573 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:10,583 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:11,588 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:12,594 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:13,600 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:14,606 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:16,677 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:17,682 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:20,073 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:21,079 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:23,600 Stage-1 map = 100%,  reduce = 87%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:24,606 Stage-1 map = 100%,  reduce = 87%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:25,614 Stage-1 map = 100%,  reduce = 87%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:26,620 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:27,625 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:28,631 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:29,637 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:30,642 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:31,648 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:32,654 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:33,660 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:34,666 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1536.68 sec
2013-09-18 01:16:35,672 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1613.31 sec
2013-09-18 01:16:37,443 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1656.62 sec
2013-09-18 01:16:38,449 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1742.64 sec
2013-09-18 01:16:39,455 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1879.43 sec
2013-09-18 01:16:40,460 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1970.65 sec
2013-09-18 01:16:41,465 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1970.65 sec
2013-09-18 01:16:43,070 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:44,076 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:45,081 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:46,086 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:47,092 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:50,460 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:51,564 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:52,760 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:53,765 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:54,771 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:55,776 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:16:56,781 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:17:00,159 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:17:01,164 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:17:02,169 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:17:03,175 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:17:04,180 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:17:05,185 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:17:06,189 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:17:07,194 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:17:08,199 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1997.98 sec
2013-09-18 01:17:09,934 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 2024.84 sec
2013-09-18 01:17:10,938 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 2024.84 sec
MapReduce Total cumulative CPU time: 33 minutes 44 seconds 840 msec
Ended Job = job_201309172235_0147
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0148
Hadoop job information for Stage-2: number of mappers: 14; number of reducers: 1
2013-09-18 01:17:14,417 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:17:27,528 Stage-2 map = 1%,  reduce = 0%
2013-09-18 01:17:28,533 Stage-2 map = 2%,  reduce = 0%
2013-09-18 01:17:31,543 Stage-2 map = 22%,  reduce = 0%
2013-09-18 01:17:37,560 Stage-2 map = 28%,  reduce = 0%
2013-09-18 01:17:40,569 Stage-2 map = 43%,  reduce = 0%
2013-09-18 01:17:46,588 Stage-2 map = 53%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:47,592 Stage-2 map = 53%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:48,597 Stage-2 map = 53%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:49,601 Stage-2 map = 64%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:50,606 Stage-2 map = 64%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:51,611 Stage-2 map = 64%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:52,616 Stage-2 map = 70%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:53,621 Stage-2 map = 70%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:55,731 Stage-2 map = 78%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:56,736 Stage-2 map = 78%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:57,740 Stage-2 map = 78%,  reduce = 0%, Cumulative CPU 446.67 sec
2013-09-18 01:17:58,745 Stage-2 map = 79%,  reduce = 0%, Cumulative CPU 467.91 sec
2013-09-18 01:17:59,749 Stage-2 map = 79%,  reduce = 0%, Cumulative CPU 467.91 sec
2013-09-18 01:18:00,757 Stage-2 map = 79%,  reduce = 0%, Cumulative CPU 467.91 sec
2013-09-18 01:18:01,762 Stage-2 map = 79%,  reduce = 0%, Cumulative CPU 467.91 sec
2013-09-18 01:18:02,767 Stage-2 map = 79%,  reduce = 0%, Cumulative CPU 467.91 sec
2013-09-18 01:18:03,771 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 547.4 sec
2013-09-18 01:18:04,776 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 547.4 sec
2013-09-18 01:18:05,781 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 547.4 sec
2013-09-18 01:18:06,786 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 547.4 sec
2013-09-18 01:18:07,791 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 547.4 sec
2013-09-18 01:18:09,462 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 547.4 sec
2013-09-18 01:18:10,467 Stage-2 map = 88%,  reduce = 10%, Cumulative CPU 613.28 sec
2013-09-18 01:18:11,472 Stage-2 map = 95%,  reduce = 10%, Cumulative CPU 782.49 sec
2013-09-18 01:18:12,482 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 886.7 sec
2013-09-18 01:18:13,487 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 886.7 sec
2013-09-18 01:18:14,491 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 886.7 sec
2013-09-18 01:18:15,496 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 886.7 sec
2013-09-18 01:18:16,501 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 886.7 sec
2013-09-18 01:18:17,505 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 886.7 sec
2013-09-18 01:18:18,510 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 886.7 sec
2013-09-18 01:18:19,566 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 886.7 sec
2013-09-18 01:18:20,571 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 886.7 sec
2013-09-18 01:18:21,574 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 886.7 sec
2013-09-18 01:18:22,578 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 886.7 sec
2013-09-18 01:18:23,583 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 886.7 sec
2013-09-18 01:18:24,587 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 886.7 sec
2013-09-18 01:18:25,592 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 886.7 sec
2013-09-18 01:18:26,597 Stage-2 map = 100%,  reduce = 14%, Cumulative CPU 886.7 sec
2013-09-18 01:18:27,680 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 886.7 sec
2013-09-18 01:18:28,684 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 886.7 sec
2013-09-18 01:18:29,689 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 886.7 sec
2013-09-18 01:18:30,693 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 886.7 sec
2013-09-18 01:18:31,698 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 886.7 sec
2013-09-18 01:18:32,702 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 886.7 sec
2013-09-18 01:18:34,644 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 886.7 sec
2013-09-18 01:18:35,648 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 886.7 sec
2013-09-18 01:18:36,653 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 886.7 sec
2013-09-18 01:18:37,657 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 886.7 sec
2013-09-18 01:18:38,662 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 886.7 sec
2013-09-18 01:18:39,666 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 886.7 sec
2013-09-18 01:18:40,670 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 886.7 sec
2013-09-18 01:18:41,674 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 886.7 sec
2013-09-18 01:18:42,679 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 924.39 sec
2013-09-18 01:18:43,683 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 924.39 sec
2013-09-18 01:18:44,687 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 924.39 sec
2013-09-18 01:18:49,434 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 924.39 sec
2013-09-18 01:18:50,438 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 924.39 sec
2013-09-18 01:18:51,443 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 924.39 sec
2013-09-18 01:18:52,448 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 924.39 sec
2013-09-18 01:18:53,452 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 924.39 sec
2013-09-18 01:18:54,457 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 924.39 sec
2013-09-18 01:18:55,461 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:18:56,465 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:18:57,469 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:18:58,473 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:18:59,477 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:19:00,481 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:19:05,293 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:19:06,297 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:19:07,301 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:19:08,306 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:19:09,310 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:19:10,314 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 924.39 sec
2013-09-18 01:19:11,319 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:12,323 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:13,331 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:14,336 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:16,921 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:17,927 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:18,931 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:19,935 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:20,939 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:21,943 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:22,947 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:23,951 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:24,955 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:25,959 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:26,963 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:27,966 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:28,970 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:29,974 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:30,977 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:31,981 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:32,985 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:33,989 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:34,992 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:35,996 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 924.39 sec
2013-09-18 01:19:37,001 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 976.38 sec
2013-09-18 01:19:38,006 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 976.38 sec
2013-09-18 01:19:39,010 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 976.38 sec
MapReduce Total cumulative CPU time: 16 minutes 16 seconds 380 msec
Ended Job = job_201309172235_0148
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 2024.84 sec   HDFS Read: 869197327 HDFS Write: 4140713341 SUCCESS
Job 1: Map: 14  Reduce: 1   Cumulative CPU: 976.38 sec   HDFS Read: 4140841812 HDFS Write: 417 SUCCESS
Total MapReduce CPU Time Spent: 50 minutes 1 seconds 220 msec
OK
Time taken: 324.867 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_29643@mturlrep13_201309180119_1843662809.txt
hive> SELECT WatchID, ClientIP, count(*) AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0149
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:19:49,884 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:20:04,981 Stage-1 map = 2%,  reduce = 0%
2013-09-18 01:20:05,998 Stage-1 map = 3%,  reduce = 0%
2013-09-18 01:20:08,008 Stage-1 map = 9%,  reduce = 0%
2013-09-18 01:20:09,017 Stage-1 map = 14%,  reduce = 0%
2013-09-18 01:20:11,035 Stage-1 map = 16%,  reduce = 0%
2013-09-18 01:20:12,040 Stage-1 map = 26%,  reduce = 0%
2013-09-18 01:20:13,052 Stage-1 map = 27%,  reduce = 0%
2013-09-18 01:20:14,066 Stage-1 map = 28%,  reduce = 0%
2013-09-18 01:20:15,083 Stage-1 map = 35%,  reduce = 0%
2013-09-18 01:20:16,111 Stage-1 map = 36%,  reduce = 0%
2013-09-18 01:20:17,123 Stage-1 map = 37%,  reduce = 0%
2013-09-18 01:20:18,128 Stage-1 map = 45%,  reduce = 0%
2013-09-18 01:20:19,164 Stage-1 map = 47%,  reduce = 0%
2013-09-18 01:20:21,208 Stage-1 map = 55%,  reduce = 0%
2013-09-18 01:20:22,215 Stage-1 map = 58%,  reduce = 0%
2013-09-18 01:20:24,224 Stage-1 map = 65%,  reduce = 0%
2013-09-18 01:20:25,234 Stage-1 map = 68%,  reduce = 0%
2013-09-18 01:20:27,288 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 28.16 sec
2013-09-18 01:20:28,302 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 28.16 sec
2013-09-18 01:20:29,308 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 28.16 sec
2013-09-18 01:20:30,314 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 92.89 sec
2013-09-18 01:20:31,331 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 92.89 sec
2013-09-18 01:20:32,346 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 193.14 sec
2013-09-18 01:20:33,367 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 259.68 sec
2013-09-18 01:20:34,384 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 362.61 sec
2013-09-18 01:20:35,420 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 431.76 sec
2013-09-18 01:20:36,426 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 683.06 sec
2013-09-18 01:20:37,432 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 832.82 sec
2013-09-18 01:20:38,438 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 1057.56 sec
2013-09-18 01:20:39,444 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1253.19 sec
2013-09-18 01:20:40,455 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 1253.19 sec
2013-09-18 01:20:41,462 Stage-1 map = 99%,  reduce = 1%, Cumulative CPU 1253.19 sec
2013-09-18 01:20:42,468 Stage-1 map = 99%,  reduce = 4%, Cumulative CPU 1297.77 sec
2013-09-18 01:20:43,475 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:44,481 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:45,487 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:46,493 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:47,499 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:48,505 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:49,534 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:50,540 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:51,546 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:52,552 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:53,962 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:54,968 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:55,973 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:56,979 Stage-1 map = 100%,  reduce = 48%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:57,988 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1357.14 sec
2013-09-18 01:20:59,065 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:00,071 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:01,077 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:02,083 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:03,089 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:04,198 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:05,204 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:06,211 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:07,217 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:08,223 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:09,720 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:10,726 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:11,732 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:12,738 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:13,744 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:16,687 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:17,693 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:18,698 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:19,707 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:20,715 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:21,721 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:25,701 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:26,707 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:28,607 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:29,613 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:30,619 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:31,625 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:32,630 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:33,636 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:34,641 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:35,646 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:37,031 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:38,037 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:39,043 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:40,049 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:41,055 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:42,060 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1357.14 sec
2013-09-18 01:21:43,068 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1852.58 sec
2013-09-18 01:21:44,074 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1852.58 sec
2013-09-18 01:21:45,080 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1852.58 sec
2013-09-18 01:21:46,963 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1852.58 sec
2013-09-18 01:21:47,969 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1852.58 sec
2013-09-18 01:21:48,975 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1852.58 sec
2013-09-18 01:21:52,859 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1852.58 sec
2013-09-18 01:21:54,438 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1852.58 sec
2013-09-18 01:21:55,444 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1852.58 sec
2013-09-18 01:21:57,237 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1852.58 sec
2013-09-18 01:21:58,259 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1873.66 sec
2013-09-18 01:21:59,265 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1873.66 sec
2013-09-18 01:22:00,270 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1873.66 sec
2013-09-18 01:22:01,276 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1886.44 sec
2013-09-18 01:22:02,282 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1886.44 sec
2013-09-18 01:22:03,288 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1961.16 sec
2013-09-18 01:22:04,293 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1978.53 sec
2013-09-18 01:22:05,299 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 2014.08 sec
2013-09-18 01:22:06,305 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 2014.08 sec
2013-09-18 01:22:07,310 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 2014.08 sec
MapReduce Total cumulative CPU time: 33 minutes 34 seconds 80 msec
Ended Job = job_201309172235_0149
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0150
Hadoop job information for Stage-2: number of mappers: 14; number of reducers: 1
2013-09-18 01:22:10,699 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:22:27,319 Stage-2 map = 5%,  reduce = 0%
2013-09-18 01:22:30,330 Stage-2 map = 22%,  reduce = 0%
2013-09-18 01:22:33,340 Stage-2 map = 25%,  reduce = 0%
2013-09-18 01:22:36,352 Stage-2 map = 31%,  reduce = 0%
2013-09-18 01:22:39,362 Stage-2 map = 43%,  reduce = 0%
2013-09-18 01:22:42,371 Stage-2 map = 52%,  reduce = 0%
2013-09-18 01:22:45,406 Stage-2 map = 62%,  reduce = 0%, Cumulative CPU 517.66 sec
2013-09-18 01:22:46,420 Stage-2 map = 62%,  reduce = 0%, Cumulative CPU 517.66 sec
2013-09-18 01:22:47,425 Stage-2 map = 62%,  reduce = 0%, Cumulative CPU 517.66 sec
2013-09-18 01:22:48,431 Stage-2 map = 64%,  reduce = 0%, Cumulative CPU 517.66 sec
2013-09-18 01:22:49,436 Stage-2 map = 64%,  reduce = 0%, Cumulative CPU 517.66 sec
2013-09-18 01:22:50,441 Stage-2 map = 64%,  reduce = 0%, Cumulative CPU 517.66 sec
2013-09-18 01:22:51,446 Stage-2 map = 78%,  reduce = 0%, Cumulative CPU 517.66 sec
2013-09-18 01:22:54,870 Stage-2 map = 78%,  reduce = 0%, Cumulative CPU 517.66 sec
2013-09-18 01:22:55,875 Stage-2 map = 78%,  reduce = 0%, Cumulative CPU 517.66 sec
2013-09-18 01:22:56,880 Stage-2 map = 78%,  reduce = 0%, Cumulative CPU 517.66 sec
2013-09-18 01:22:57,885 Stage-2 map = 79%,  reduce = 0%, Cumulative CPU 533.57 sec
2013-09-18 01:23:00,883 Stage-2 map = 79%,  reduce = 0%, Cumulative CPU 533.57 sec
2013-09-18 01:23:01,889 Stage-2 map = 81%,  reduce = 0%, Cumulative CPU 553.42 sec
2013-09-18 01:23:02,895 Stage-2 map = 81%,  reduce = 0%, Cumulative CPU 553.42 sec
2013-09-18 01:23:03,900 Stage-2 map = 81%,  reduce = 0%, Cumulative CPU 553.42 sec
2013-09-18 01:23:04,905 Stage-2 map = 81%,  reduce = 0%, Cumulative CPU 553.42 sec
2013-09-18 01:23:06,469 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 597.6 sec
2013-09-18 01:23:07,474 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 597.6 sec
2013-09-18 01:23:08,479 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 597.6 sec
2013-09-18 01:23:09,483 Stage-2 map = 88%,  reduce = 10%, Cumulative CPU 653.66 sec
2013-09-18 01:23:10,488 Stage-2 map = 92%,  reduce = 10%, Cumulative CPU 738.42 sec
2013-09-18 01:23:11,492 Stage-2 map = 98%,  reduce = 10%, Cumulative CPU 857.27 sec
2013-09-18 01:23:12,884 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 887.38 sec
2013-09-18 01:23:14,634 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 887.38 sec
2013-09-18 01:23:15,638 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 887.38 sec
2013-09-18 01:23:16,643 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 887.38 sec
2013-09-18 01:23:17,648 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 887.38 sec
2013-09-18 01:23:18,652 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 887.38 sec
2013-09-18 01:23:19,657 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 887.38 sec
2013-09-18 01:23:20,662 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 887.38 sec
2013-09-18 01:23:21,667 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 887.38 sec
2013-09-18 01:23:22,671 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 887.38 sec
2013-09-18 01:23:23,797 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 887.38 sec
2013-09-18 01:23:24,801 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 887.38 sec
2013-09-18 01:23:25,806 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 887.38 sec
2013-09-18 01:23:26,811 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 887.38 sec
2013-09-18 01:23:27,815 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 887.38 sec
2013-09-18 01:23:28,820 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 887.38 sec
2013-09-18 01:23:29,825 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 887.38 sec
2013-09-18 01:23:31,274 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 887.38 sec
2013-09-18 01:23:32,278 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 887.38 sec
2013-09-18 01:23:33,283 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 887.38 sec
2013-09-18 01:23:34,287 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 887.38 sec
2013-09-18 01:23:35,291 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 887.38 sec
2013-09-18 01:23:36,295 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 887.38 sec
2013-09-18 01:23:37,300 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.38 sec
2013-09-18 01:23:38,305 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.38 sec
2013-09-18 01:23:39,310 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.38 sec
2013-09-18 01:23:41,159 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.38 sec
2013-09-18 01:23:42,164 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.38 sec
2013-09-18 01:23:43,170 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 887.38 sec
2013-09-18 01:23:44,175 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 923.46 sec
2013-09-18 01:23:47,083 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 923.46 sec
2013-09-18 01:23:53,300 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 923.46 sec
2013-09-18 01:23:54,304 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 923.46 sec
2013-09-18 01:23:55,307 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 923.46 sec
2013-09-18 01:23:56,311 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:23:57,314 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:23:58,318 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:23:59,322 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:24:00,325 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:24:01,329 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:24:02,333 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:24:03,337 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:24:08,423 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:24:09,427 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:24:10,431 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 923.46 sec
2013-09-18 01:24:11,435 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:12,439 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:13,442 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:14,446 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:15,450 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:16,454 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:17,458 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:18,462 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:19,893 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:20,896 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:21,900 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:22,904 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:23,908 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:24,918 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:25,922 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:26,926 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:27,929 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:28,932 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:29,936 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:30,939 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:31,943 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:32,947 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 923.46 sec
2013-09-18 01:24:33,952 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 975.78 sec
2013-09-18 01:24:34,957 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 975.78 sec
MapReduce Total cumulative CPU time: 16 minutes 15 seconds 780 msec
Ended Job = job_201309172235_0150
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 2014.08 sec   HDFS Read: 869197327 HDFS Write: 4140713341 SUCCESS
Job 1: Map: 14  Reduce: 1   Cumulative CPU: 975.78 sec   HDFS Read: 4140841812 HDFS Write: 419 SUCCESS
Total MapReduce CPU Time Spent: 49 minutes 49 seconds 860 msec
OK
Time taken: 294.034 seconds, Fetched: 10 row(s)
hive> quit;
-- то же самое, но ещё и без фильтрации.;


times: 1
query: SELECT URL, count(*) AS c FROM hits_100m GROUP BY URL ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_7945@mturlrep13_201309180124_199584274.txt
hive> SELECT URL, count(*) AS c FROM hits_100m GROUP BY URL ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0151
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:24:53,603 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:25:03,695 Stage-1 map = 2%,  reduce = 0%
2013-09-18 01:25:04,700 Stage-1 map = 3%,  reduce = 0%
2013-09-18 01:25:09,065 Stage-1 map = 17%,  reduce = 0%
2013-09-18 01:25:10,073 Stage-1 map = 22%,  reduce = 0%
2013-09-18 01:25:11,078 Stage-1 map = 28%,  reduce = 0%
2013-09-18 01:25:12,084 Stage-1 map = 31%,  reduce = 0%
2013-09-18 01:25:13,091 Stage-1 map = 34%,  reduce = 0%
2013-09-18 01:25:14,104 Stage-1 map = 44%,  reduce = 0%
2013-09-18 01:25:15,116 Stage-1 map = 47%,  reduce = 0%
2013-09-18 01:25:16,125 Stage-1 map = 50%,  reduce = 0%
2013-09-18 01:25:17,131 Stage-1 map = 57%,  reduce = 0%
2013-09-18 01:25:18,139 Stage-1 map = 61%,  reduce = 0%
2013-09-18 01:25:20,170 Stage-1 map = 64%,  reduce = 0%
2013-09-18 01:25:21,175 Stage-1 map = 66%,  reduce = 0%
2013-09-18 01:25:22,187 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 17.87 sec
2013-09-18 01:25:23,203 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 38.23 sec
2013-09-18 01:25:24,213 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 80.85 sec
2013-09-18 01:25:25,223 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 123.43 sec
2013-09-18 01:25:26,229 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 191.82 sec
2013-09-18 01:25:27,236 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 236.59 sec
2013-09-18 01:25:28,248 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 236.59 sec
2013-09-18 01:25:29,255 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 284.0 sec
2013-09-18 01:25:30,261 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 284.0 sec
2013-09-18 01:25:31,273 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 284.0 sec
2013-09-18 01:25:32,278 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 284.0 sec
2013-09-18 01:25:33,284 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 284.0 sec
2013-09-18 01:25:34,290 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 428.41 sec
2013-09-18 01:25:35,759 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 428.41 sec
2013-09-18 01:25:36,772 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 428.41 sec
2013-09-18 01:25:37,778 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 428.41 sec
2013-09-18 01:25:38,785 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 565.42 sec
2013-09-18 01:25:41,597 Stage-1 map = 94%,  reduce = 15%, Cumulative CPU 565.42 sec
2013-09-18 01:25:45,549 Stage-1 map = 96%,  reduce = 17%, Cumulative CPU 1109.57 sec
2013-09-18 01:25:46,775 Stage-1 map = 99%,  reduce = 17%, Cumulative CPU 1121.14 sec
2013-09-18 01:25:47,781 Stage-1 map = 99%,  reduce = 17%, Cumulative CPU 1123.73 sec
2013-09-18 01:25:48,787 Stage-1 map = 99%,  reduce = 17%, Cumulative CPU 1123.73 sec
2013-09-18 01:25:49,792 Stage-1 map = 99%,  reduce = 17%, Cumulative CPU 1123.73 sec
2013-09-18 01:25:50,799 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 1123.73 sec
2013-09-18 01:25:51,805 Stage-1 map = 99%,  reduce = 19%, Cumulative CPU 1123.73 sec
2013-09-18 01:25:52,811 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 1130.32 sec
2013-09-18 01:25:53,816 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 1130.32 sec
2013-09-18 01:25:55,206 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 1130.32 sec
2013-09-18 01:25:56,212 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 1130.32 sec
2013-09-18 01:25:57,221 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1130.32 sec
2013-09-18 01:25:58,227 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1130.32 sec
2013-09-18 01:25:59,260 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:00,267 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:01,273 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:02,279 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:03,286 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:04,292 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:09,638 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:10,644 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:14,140 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:15,145 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:16,152 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:17,158 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:18,165 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:19,170 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:20,177 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:21,183 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:22,189 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:23,194 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:24,203 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:25,208 Stage-1 map = 100%,  reduce = 77%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:26,409 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:27,415 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:28,420 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:29,426 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:30,432 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:31,439 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:33,621 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:34,626 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:35,631 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:36,637 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:37,643 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:38,648 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:40,155 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:41,161 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:43,159 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1130.32 sec
2013-09-18 01:26:44,308 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1494.96 sec
2013-09-18 01:26:45,314 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1494.96 sec
2013-09-18 01:26:46,518 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1494.96 sec
2013-09-18 01:26:47,523 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1494.96 sec
2013-09-18 01:26:48,532 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1494.96 sec
2013-09-18 01:26:49,538 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1526.79 sec
2013-09-18 01:26:50,543 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1526.79 sec
MapReduce Total cumulative CPU time: 25 minutes 26 seconds 790 msec
Ended Job = job_201309172235_0151
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0152
Hadoop job information for Stage-2: number of mappers: 14; number of reducers: 1
2013-09-18 01:27:03,639 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:27:11,681 Stage-2 map = 9%,  reduce = 0%
2013-09-18 01:27:12,685 Stage-2 map = 12%,  reduce = 0%
2013-09-18 01:27:15,339 Stage-2 map = 32%,  reduce = 0%
2013-09-18 01:27:18,350 Stage-2 map = 59%,  reduce = 0%
2013-09-18 01:27:21,361 Stage-2 map = 78%,  reduce = 0%
2013-09-18 01:27:23,369 Stage-2 map = 81%,  reduce = 0%, Cumulative CPU 47.02 sec
2013-09-18 01:27:24,374 Stage-2 map = 87%,  reduce = 0%, Cumulative CPU 142.27 sec
2013-09-18 01:27:25,378 Stage-2 map = 90%,  reduce = 0%, Cumulative CPU 190.85 sec
2013-09-18 01:27:26,383 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.83 sec
2013-09-18 01:27:27,388 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.83 sec
2013-09-18 01:27:28,392 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.83 sec
2013-09-18 01:27:29,400 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.83 sec
2013-09-18 01:27:30,405 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.83 sec
2013-09-18 01:27:31,411 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.83 sec
2013-09-18 01:27:32,416 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.83 sec
2013-09-18 01:27:33,421 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 350.83 sec
2013-09-18 01:27:34,425 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 350.83 sec
2013-09-18 01:27:35,430 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 350.83 sec
2013-09-18 01:27:36,435 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 350.83 sec
2013-09-18 01:27:37,440 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 350.83 sec
2013-09-18 01:27:38,445 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 350.83 sec
2013-09-18 01:27:39,453 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 350.83 sec
2013-09-18 01:27:40,457 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 350.83 sec
2013-09-18 01:27:41,461 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 350.83 sec
2013-09-18 01:27:42,466 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 350.83 sec
2013-09-18 01:27:43,471 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 350.83 sec
2013-09-18 01:27:44,508 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 368.28 sec
2013-09-18 01:27:45,513 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 368.28 sec
2013-09-18 01:27:46,517 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 368.28 sec
2013-09-18 01:27:47,522 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 368.28 sec
2013-09-18 01:27:48,527 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 368.28 sec
2013-09-18 01:27:50,078 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 368.28 sec
2013-09-18 01:27:51,083 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 368.28 sec
2013-09-18 01:27:52,091 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 368.28 sec
2013-09-18 01:27:53,096 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 368.28 sec
2013-09-18 01:27:54,100 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 368.28 sec
2013-09-18 01:27:55,105 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 368.28 sec
2013-09-18 01:27:56,109 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 368.28 sec
2013-09-18 01:27:58,555 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 368.28 sec
2013-09-18 01:27:59,560 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 368.28 sec
2013-09-18 01:28:00,564 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 368.28 sec
2013-09-18 01:28:02,601 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 368.28 sec
2013-09-18 01:28:03,606 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 368.28 sec
2013-09-18 01:28:04,611 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 368.28 sec
2013-09-18 01:28:05,616 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 368.28 sec
2013-09-18 01:28:06,621 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 368.28 sec
2013-09-18 01:28:07,625 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 368.28 sec
2013-09-18 01:28:08,630 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 368.28 sec
2013-09-18 01:28:09,635 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:10,639 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:11,644 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:12,648 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:13,653 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:14,657 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:15,662 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:16,666 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:17,671 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:18,675 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:19,679 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:20,684 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:21,688 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:22,692 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:23,696 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:24,700 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:29,545 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:31,146 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:32,150 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:33,154 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:34,158 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 368.28 sec
2013-09-18 01:28:35,163 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 400.78 sec
2013-09-18 01:28:36,169 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 400.78 sec
2013-09-18 01:28:37,173 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 400.78 sec
MapReduce Total cumulative CPU time: 6 minutes 40 seconds 780 msec
Ended Job = job_201309172235_0152
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1526.79 sec   HDFS Read: 882764316 HDFS Write: 4155848594 SUCCESS
Job 1: Map: 14  Reduce: 1   Cumulative CPU: 400.78 sec   HDFS Read: 4156020610 HDFS Write: 399 SUCCESS
Total MapReduce CPU Time Spent: 32 minutes 7 seconds 570 msec
OK
Time taken: 233.937 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT URL, count(*) AS c FROM hits_100m GROUP BY URL ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_17325@mturlrep13_201309180128_811787741.txt
hive> SELECT URL, count(*) AS c FROM hits_100m GROUP BY URL ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0153
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:28:48,068 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:28:58,179 Stage-1 map = 7%,  reduce = 0%
2013-09-18 01:29:01,193 Stage-1 map = 12%,  reduce = 0%
2013-09-18 01:29:02,199 Stage-1 map = 16%,  reduce = 0%
2013-09-18 01:29:03,232 Stage-1 map = 18%,  reduce = 0%
2013-09-18 01:29:04,245 Stage-1 map = 25%,  reduce = 0%
2013-09-18 01:29:05,251 Stage-1 map = 34%,  reduce = 0%
2013-09-18 01:29:06,256 Stage-1 map = 38%,  reduce = 0%
2013-09-18 01:29:07,261 Stage-1 map = 44%,  reduce = 0%
2013-09-18 01:29:08,269 Stage-1 map = 50%,  reduce = 0%
2013-09-18 01:29:09,284 Stage-1 map = 56%,  reduce = 0%
2013-09-18 01:29:10,289 Stage-1 map = 57%,  reduce = 0%
2013-09-18 01:29:11,317 Stage-1 map = 60%,  reduce = 0%
2013-09-18 01:29:12,323 Stage-1 map = 63%,  reduce = 0%
2013-09-18 01:29:13,344 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 17.02 sec
2013-09-18 01:29:14,380 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 17.02 sec
2013-09-18 01:29:15,404 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 57.53 sec
2013-09-18 01:29:16,412 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 98.75 sec
2013-09-18 01:29:17,422 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 119.1 sec
2013-09-18 01:29:18,433 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 183.33 sec
2013-09-18 01:29:19,440 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 229.01 sec
2013-09-18 01:29:20,451 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 229.01 sec
2013-09-18 01:29:21,457 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 229.01 sec
2013-09-18 01:29:22,468 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 276.95 sec
2013-09-18 01:29:23,483 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 276.95 sec
2013-09-18 01:29:24,490 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 276.95 sec
2013-09-18 01:29:25,502 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 362.44 sec
2013-09-18 01:29:26,510 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 362.44 sec
2013-09-18 01:29:29,051 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 392.77 sec
2013-09-18 01:29:35,351 Stage-1 map = 94%,  reduce = 15%, Cumulative CPU 521.09 sec
2013-09-18 01:29:37,211 Stage-1 map = 98%,  reduce = 15%, Cumulative CPU 907.91 sec
2013-09-18 01:29:38,217 Stage-1 map = 99%,  reduce = 15%, Cumulative CPU 946.27 sec
2013-09-18 01:29:39,222 Stage-1 map = 99%,  reduce = 15%, Cumulative CPU 946.27 sec
2013-09-18 01:29:40,228 Stage-1 map = 99%,  reduce = 16%, Cumulative CPU 946.27 sec
2013-09-18 01:29:41,234 Stage-1 map = 99%,  reduce = 17%, Cumulative CPU 946.27 sec
2013-09-18 01:29:42,239 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 946.27 sec
2013-09-18 01:29:43,245 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 946.27 sec
2013-09-18 01:29:44,252 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 1126.33 sec
2013-09-18 01:29:45,257 Stage-1 map = 100%,  reduce = 19%, Cumulative CPU 1126.92 sec
2013-09-18 01:29:46,262 Stage-1 map = 100%,  reduce = 19%, Cumulative CPU 1126.92 sec
2013-09-18 01:29:47,268 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 1126.92 sec
2013-09-18 01:29:48,275 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 1126.92 sec
2013-09-18 01:29:49,281 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 1126.92 sec
2013-09-18 01:29:50,287 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1126.92 sec
2013-09-18 01:29:51,294 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1126.92 sec
2013-09-18 01:29:54,543 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1126.92 sec
2013-09-18 01:29:55,549 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1126.92 sec
2013-09-18 01:29:56,556 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:01,304 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:02,310 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:03,316 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:04,324 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:05,852 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:06,858 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:08,923 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:09,929 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:11,956 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:12,962 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:13,969 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:14,974 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:15,978 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:16,984 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:21,891 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:22,897 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:23,902 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:24,908 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:26,217 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:27,223 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:28,229 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:29,235 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:30,240 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:31,246 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:32,252 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:33,258 Stage-1 map = 100%,  reduce = 89%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:34,263 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:35,269 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:36,274 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:38,613 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:39,619 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:40,625 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:41,630 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:42,636 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:43,641 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:44,647 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:45,651 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:46,656 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:47,661 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:48,666 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:49,671 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:50,676 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:51,681 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:52,685 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:53,691 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1126.92 sec
2013-09-18 01:30:54,698 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:30:55,703 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:30:56,708 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:30:57,714 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:30:58,719 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:30:59,724 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:31:00,730 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:31:01,739 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:31:02,744 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:31:03,750 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:31:04,755 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:31:05,759 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1511.04 sec
2013-09-18 01:31:09,986 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1512.0 sec
2013-09-18 01:31:11,626 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1513.29 sec
2013-09-18 01:31:12,632 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1513.29 sec
2013-09-18 01:31:14,618 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1513.29 sec
2013-09-18 01:31:15,624 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1513.29 sec
2013-09-18 01:31:16,629 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1515.81 sec
2013-09-18 01:31:20,387 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1518.52 sec
2013-09-18 01:31:21,393 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1518.52 sec
MapReduce Total cumulative CPU time: 25 minutes 18 seconds 520 msec
Ended Job = job_201309172235_0153
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0154
Hadoop job information for Stage-2: number of mappers: 14; number of reducers: 1
2013-09-18 01:31:35,149 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:31:42,177 Stage-2 map = 13%,  reduce = 0%
2013-09-18 01:31:45,189 Stage-2 map = 19%,  reduce = 0%
2013-09-18 01:31:46,194 Stage-2 map = 36%,  reduce = 0%
2013-09-18 01:31:49,205 Stage-2 map = 73%,  reduce = 0%
2013-09-18 01:31:52,217 Stage-2 map = 82%,  reduce = 0%, Cumulative CPU 45.62 sec
2013-09-18 01:31:53,223 Stage-2 map = 89%,  reduce = 0%, Cumulative CPU 162.4 sec
2013-09-18 01:31:54,251 Stage-2 map = 94%,  reduce = 0%, Cumulative CPU 239.19 sec
2013-09-18 01:31:55,256 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 346.31 sec
2013-09-18 01:31:56,260 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 346.31 sec
2013-09-18 01:31:57,265 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 346.31 sec
2013-09-18 01:31:58,270 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 346.31 sec
2013-09-18 01:31:59,275 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 346.31 sec
2013-09-18 01:32:00,280 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 346.31 sec
2013-09-18 01:32:01,285 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 346.31 sec
2013-09-18 01:32:02,291 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 346.31 sec
2013-09-18 01:32:03,296 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 346.31 sec
2013-09-18 01:32:04,301 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 346.31 sec
2013-09-18 01:32:05,307 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 346.31 sec
2013-09-18 01:32:06,312 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 346.31 sec
2013-09-18 01:32:07,318 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 346.31 sec
2013-09-18 01:32:08,323 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 346.31 sec
2013-09-18 01:32:09,328 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 346.31 sec
2013-09-18 01:32:10,332 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 346.31 sec
2013-09-18 01:32:11,338 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 346.31 sec
2013-09-18 01:32:12,343 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 346.31 sec
2013-09-18 01:32:13,348 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 346.31 sec
2013-09-18 01:32:14,353 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 346.31 sec
2013-09-18 01:32:15,358 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 346.31 sec
2013-09-18 01:32:16,363 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 346.31 sec
2013-09-18 01:32:17,368 Stage-2 map = 100%,  reduce = 26%, Cumulative CPU 346.31 sec
2013-09-18 01:32:18,372 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 346.31 sec
2013-09-18 01:32:19,377 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 346.31 sec
2013-09-18 01:32:20,382 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 346.31 sec
2013-09-18 01:32:21,387 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 346.31 sec
2013-09-18 01:32:23,213 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 346.31 sec
2013-09-18 01:32:24,218 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:25,222 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:28,762 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:29,767 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:30,772 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:31,777 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:34,832 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:35,836 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:36,841 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:38,072 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:39,077 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 346.31 sec
2013-09-18 01:32:40,083 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 346.31 sec
2013-09-18 01:32:41,087 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 346.31 sec
2013-09-18 01:32:42,093 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 346.31 sec
2013-09-18 01:32:44,011 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:32:45,016 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:32:46,021 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:32:47,025 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:32:48,030 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:32:49,034 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:32:50,039 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:32:51,044 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:32:52,048 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:32:53,052 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:32:54,064 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 346.31 sec
2013-09-18 01:33:02,924 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 389.68 sec
2013-09-18 01:33:03,928 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 389.68 sec
2013-09-18 01:33:06,050 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 389.68 sec
2013-09-18 01:33:07,054 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 389.68 sec
2013-09-18 01:33:08,058 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 389.68 sec
2013-09-18 01:33:09,063 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 395.95 sec
2013-09-18 01:33:10,068 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 395.95 sec
2013-09-18 01:33:11,072 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 395.95 sec
MapReduce Total cumulative CPU time: 6 minutes 35 seconds 950 msec
Ended Job = job_201309172235_0154
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1518.52 sec   HDFS Read: 882764316 HDFS Write: 4155848594 SUCCESS
Job 1: Map: 14  Reduce: 1   Cumulative CPU: 395.95 sec   HDFS Read: 4156020676 HDFS Write: 399 SUCCESS
Total MapReduce CPU Time Spent: 31 minutes 54 seconds 470 msec
OK
Time taken: 272.053 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT URL, count(*) AS c FROM hits_100m GROUP BY URL ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_26732@mturlrep13_201309180133_998177234.txt
hive> SELECT URL, count(*) AS c FROM hits_100m GROUP BY URL ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0155
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:33:21,912 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:33:30,987 Stage-1 map = 2%,  reduce = 0%
2013-09-18 01:33:31,993 Stage-1 map = 5%,  reduce = 0%
2013-09-18 01:33:32,999 Stage-1 map = 6%,  reduce = 0%
2013-09-18 01:33:34,004 Stage-1 map = 7%,  reduce = 0%
2013-09-18 01:33:35,010 Stage-1 map = 12%,  reduce = 0%
2013-09-18 01:33:36,017 Stage-1 map = 15%,  reduce = 0%
2013-09-18 01:33:37,023 Stage-1 map = 19%,  reduce = 0%
2013-09-18 01:33:38,030 Stage-1 map = 27%,  reduce = 0%
2013-09-18 01:33:39,035 Stage-1 map = 32%,  reduce = 0%
2013-09-18 01:33:40,041 Stage-1 map = 36%,  reduce = 0%
2013-09-18 01:33:41,047 Stage-1 map = 45%,  reduce = 0%
2013-09-18 01:33:42,053 Stage-1 map = 49%,  reduce = 0%
2013-09-18 01:33:43,073 Stage-1 map = 55%,  reduce = 0%
2013-09-18 01:33:44,079 Stage-1 map = 58%,  reduce = 0%
2013-09-18 01:33:45,084 Stage-1 map = 61%,  reduce = 0%
2013-09-18 01:33:46,177 Stage-1 map = 63%,  reduce = 0%
2013-09-18 01:33:47,185 Stage-1 map = 65%,  reduce = 0%
2013-09-18 01:33:48,199 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 17.75 sec
2013-09-18 01:33:49,213 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 17.75 sec
2013-09-18 01:33:50,227 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 58.6 sec
2013-09-18 01:33:51,238 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 121.99 sec
2013-09-18 01:33:52,254 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 143.37 sec
2013-09-18 01:33:53,270 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 234.85 sec
2013-09-18 01:33:54,282 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 282.41 sec
2013-09-18 01:33:55,289 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 875.12 sec
2013-09-18 01:33:56,303 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 877.62 sec
2013-09-18 01:33:57,310 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 877.62 sec
2013-09-18 01:33:58,319 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 882.85 sec
2013-09-18 01:33:59,327 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 888.05 sec
2013-09-18 01:34:00,333 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 888.05 sec
2013-09-18 01:34:03,143 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 892.75 sec
2013-09-18 01:34:05,074 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 892.75 sec
2013-09-18 01:34:07,380 Stage-1 map = 94%,  reduce = 15%, Cumulative CPU 917.16 sec
2013-09-18 01:34:08,387 Stage-1 map = 94%,  reduce = 15%, Cumulative CPU 917.16 sec
2013-09-18 01:34:09,405 Stage-1 map = 94%,  reduce = 15%, Cumulative CPU 917.16 sec
2013-09-18 01:34:12,449 Stage-1 map = 97%,  reduce = 16%, Cumulative CPU 991.88 sec
2013-09-18 01:34:13,455 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 1042.76 sec
2013-09-18 01:34:14,461 Stage-1 map = 99%,  reduce = 19%, Cumulative CPU 1042.76 sec
2013-09-18 01:34:15,467 Stage-1 map = 99%,  reduce = 19%, Cumulative CPU 1042.76 sec
2013-09-18 01:34:16,473 Stage-1 map = 99%,  reduce = 19%, Cumulative CPU 1085.14 sec
2013-09-18 01:34:17,478 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:18,484 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:19,489 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:20,495 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:21,500 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:22,513 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:23,989 Stage-1 map = 100%,  reduce = 25%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:24,995 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:26,001 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:27,007 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:28,013 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:29,816 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:30,821 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:31,827 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:32,834 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:33,839 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:34,845 Stage-1 map = 100%,  reduce = 42%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:38,795 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:40,768 Stage-1 map = 100%,  reduce = 64%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:43,249 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:44,255 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:45,262 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:46,269 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:47,275 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:48,396 Stage-1 map = 100%,  reduce = 74%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:49,403 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:50,409 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:51,415 Stage-1 map = 100%,  reduce = 78%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:52,420 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:53,426 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:54,433 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:55,438 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1095.25 sec
2013-09-18 01:34:57,179 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 1415.82 sec
2013-09-18 01:34:58,185 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1415.82 sec
2013-09-18 01:34:59,192 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1415.82 sec
2013-09-18 01:35:00,198 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1415.82 sec
2013-09-18 01:35:01,205 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1415.82 sec
2013-09-18 01:35:02,210 Stage-1 map = 100%,  reduce = 93%, Cumulative CPU 1415.82 sec
2013-09-18 01:35:03,309 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1415.82 sec
2013-09-18 01:35:04,314 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1415.82 sec
2013-09-18 01:35:05,319 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1415.82 sec
2013-09-18 01:35:06,326 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1418.06 sec
2013-09-18 01:35:07,332 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1418.06 sec
2013-09-18 01:35:08,338 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1418.06 sec
2013-09-18 01:35:09,344 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1418.06 sec
2013-09-18 01:35:11,606 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1472.49 sec
2013-09-18 01:35:12,612 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1472.49 sec
2013-09-18 01:35:16,063 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1515.38 sec
2013-09-18 01:35:17,068 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1515.38 sec
MapReduce Total cumulative CPU time: 25 minutes 15 seconds 380 msec
Ended Job = job_201309172235_0155
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0156
Hadoop job information for Stage-2: number of mappers: 14; number of reducers: 1
2013-09-18 01:35:35,260 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:35:46,317 Stage-2 map = 17%,  reduce = 0%
2013-09-18 01:35:49,327 Stage-2 map = 53%,  reduce = 0%
2013-09-18 01:35:52,338 Stage-2 map = 78%,  reduce = 0%
2013-09-18 01:35:55,352 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 310.42 sec
2013-09-18 01:35:56,358 Stage-2 map = 89%,  reduce = 0%, Cumulative CPU 312.75 sec
2013-09-18 01:35:57,366 Stage-2 map = 97%,  reduce = 0%, Cumulative CPU 320.21 sec
2013-09-18 01:35:58,371 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 325.09 sec
2013-09-18 01:35:59,376 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 325.09 sec
2013-09-18 01:36:00,380 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 325.09 sec
2013-09-18 01:36:01,386 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 325.09 sec
2013-09-18 01:36:02,391 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 325.09 sec
2013-09-18 01:36:03,397 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 325.09 sec
2013-09-18 01:36:04,402 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 325.09 sec
2013-09-18 01:36:05,407 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 325.09 sec
2013-09-18 01:36:06,412 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 325.09 sec
2013-09-18 01:36:07,417 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 325.09 sec
2013-09-18 01:36:08,422 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 325.09 sec
2013-09-18 01:36:09,427 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 325.09 sec
2013-09-18 01:36:10,433 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 325.09 sec
2013-09-18 01:36:11,438 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 325.09 sec
2013-09-18 01:36:12,444 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 325.09 sec
2013-09-18 01:36:13,449 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 325.09 sec
2013-09-18 01:36:14,453 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 325.09 sec
2013-09-18 01:36:15,458 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 325.09 sec
2013-09-18 01:36:16,463 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 325.09 sec
2013-09-18 01:36:17,468 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 325.09 sec
2013-09-18 01:36:18,473 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 325.09 sec
2013-09-18 01:36:19,477 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 325.09 sec
2013-09-18 01:36:20,482 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 325.09 sec
2013-09-18 01:36:21,486 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 325.09 sec
2013-09-18 01:36:25,199 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 325.09 sec
2013-09-18 01:36:26,204 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 325.09 sec
2013-09-18 01:36:27,208 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 325.09 sec
2013-09-18 01:36:28,213 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 325.09 sec
2013-09-18 01:36:29,217 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 325.09 sec
2013-09-18 01:36:32,866 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 325.09 sec
2013-09-18 01:36:33,870 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:34,875 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:35,880 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:36,885 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:37,888 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:38,892 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:39,896 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:40,900 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:41,904 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:43,207 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:44,211 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:45,216 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:46,220 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 325.09 sec
2013-09-18 01:36:47,225 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 325.09 sec
2013-09-18 01:36:48,377 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 325.09 sec
2013-09-18 01:36:49,382 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 325.09 sec
2013-09-18 01:36:50,386 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 325.09 sec
2013-09-18 01:36:51,390 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 325.09 sec
2013-09-18 01:36:52,394 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 325.09 sec
2013-09-18 01:36:53,398 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 325.09 sec
2013-09-18 01:36:54,402 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 325.09 sec
2013-09-18 01:36:55,406 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 325.09 sec
2013-09-18 01:37:04,189 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 369.99 sec
2013-09-18 01:37:05,193 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 369.99 sec
2013-09-18 01:37:06,197 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 369.99 sec
2013-09-18 01:37:07,202 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 369.99 sec
2013-09-18 01:37:08,206 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 369.99 sec
2013-09-18 01:37:09,212 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 369.99 sec
2013-09-18 01:37:10,216 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 376.06 sec
2013-09-18 01:37:11,220 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 376.06 sec
MapReduce Total cumulative CPU time: 6 minutes 16 seconds 60 msec
Ended Job = job_201309172235_0156
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1515.38 sec   HDFS Read: 882764316 HDFS Write: 4155848594 SUCCESS
Job 1: Map: 14  Reduce: 1   Cumulative CPU: 376.06 sec   HDFS Read: 4156020676 HDFS Write: 399 SUCCESS
Total MapReduce CPU Time Spent: 31 minutes 31 seconds 440 msec
OK
Time taken: 238.167 seconds, Fetched: 10 row(s)
hive> quit;
-- агрегация по URL.;


times: 1
query: SELECT 1, URL, count(*) AS c FROM hits_100m GROUP BY 1, URL ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_3752@mturlrep13_201309180137_1035052608.txt
hive> SELECT 1, URL, count(*) AS c FROM hits_100m GROUP BY 1, URL ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0157
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:37:29,808 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:37:39,901 Stage-1 map = 5%,  reduce = 0%
2013-09-18 01:37:42,922 Stage-1 map = 8%,  reduce = 0%
2013-09-18 01:37:43,935 Stage-1 map = 11%,  reduce = 0%
2013-09-18 01:37:44,947 Stage-1 map = 13%,  reduce = 0%
2013-09-18 01:37:45,953 Stage-1 map = 18%,  reduce = 0%
2013-09-18 01:37:46,968 Stage-1 map = 25%,  reduce = 0%
2013-09-18 01:37:47,979 Stage-1 map = 32%,  reduce = 0%
2013-09-18 01:37:48,989 Stage-1 map = 36%,  reduce = 0%
2013-09-18 01:37:50,002 Stage-1 map = 43%,  reduce = 0%
2013-09-18 01:37:51,007 Stage-1 map = 48%,  reduce = 0%
2013-09-18 01:37:52,029 Stage-1 map = 49%,  reduce = 0%
2013-09-18 01:37:53,038 Stage-1 map = 54%,  reduce = 0%
2013-09-18 01:37:54,051 Stage-1 map = 58%,  reduce = 0%
2013-09-18 01:37:55,056 Stage-1 map = 59%,  reduce = 0%
2013-09-18 01:37:56,063 Stage-1 map = 61%,  reduce = 0%
2013-09-18 01:37:57,079 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 669.47 sec
2013-09-18 01:37:58,095 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 671.87 sec
2013-09-18 01:37:59,110 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 674.43 sec
2013-09-18 01:38:00,117 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 680.69 sec
2013-09-18 01:38:01,130 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 692.13 sec
2013-09-18 01:38:02,144 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 697.59 sec
2013-09-18 01:38:03,151 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 703.09 sec
2013-09-18 01:38:04,158 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 715.71 sec
2013-09-18 01:38:05,171 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 724.69 sec
2013-09-18 01:38:06,182 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 733.5 sec
2013-09-18 01:38:07,188 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 733.5 sec
2013-09-18 01:38:12,135 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 799.0 sec
2013-09-18 01:38:15,432 Stage-1 map = 94%,  reduce = 5%, Cumulative CPU 849.63 sec
2013-09-18 01:38:18,103 Stage-1 map = 94%,  reduce = 16%, Cumulative CPU 849.63 sec
2013-09-18 01:38:21,194 Stage-1 map = 94%,  reduce = 17%, Cumulative CPU 849.63 sec
2013-09-18 01:38:22,199 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 1126.64 sec
2013-09-18 01:38:23,206 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 1126.64 sec
2013-09-18 01:38:24,212 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 1149.35 sec
2013-09-18 01:38:25,218 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 1149.35 sec
2013-09-18 01:38:26,223 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 1149.35 sec
2013-09-18 01:38:27,229 Stage-1 map = 99%,  reduce = 19%, Cumulative CPU 1149.35 sec
2013-09-18 01:38:28,235 Stage-1 map = 99%,  reduce = 19%, Cumulative CPU 1149.35 sec
2013-09-18 01:38:29,242 Stage-1 map = 99%,  reduce = 20%, Cumulative CPU 1149.35 sec
2013-09-18 01:38:30,940 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:31,946 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:32,952 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:33,959 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:34,965 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:37,627 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:38,638 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:39,961 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:40,968 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:41,978 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:42,985 Stage-1 map = 100%,  reduce = 49%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:43,991 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:45,341 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:46,346 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:47,352 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:49,208 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:50,214 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:51,221 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:52,229 Stage-1 map = 100%,  reduce = 68%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:53,235 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:54,242 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:55,248 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:56,281 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:57,288 Stage-1 map = 100%,  reduce = 75%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:58,296 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1172.28 sec
2013-09-18 01:38:59,305 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:00,313 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:02,538 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:03,544 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:04,551 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:05,560 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:06,566 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:07,700 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:08,706 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:09,713 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:12,763 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:13,794 Stage-1 map = 100%,  reduce = 95%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:14,799 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:15,805 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:18,636 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:19,643 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1469.23 sec
2013-09-18 01:39:21,526 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1479.55 sec
2013-09-18 01:39:22,532 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1479.55 sec
2013-09-18 01:39:23,538 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1479.55 sec
2013-09-18 01:39:24,543 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1490.68 sec
2013-09-18 01:39:25,549 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1490.68 sec
2013-09-18 01:39:26,555 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1490.68 sec
2013-09-18 01:39:27,560 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1490.68 sec
2013-09-18 01:39:28,565 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1490.68 sec
2013-09-18 01:39:29,571 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1490.68 sec
2013-09-18 01:39:30,577 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1490.68 sec
2013-09-18 01:39:31,766 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1504.37 sec
2013-09-18 01:39:32,771 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1504.37 sec
2013-09-18 01:39:33,777 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1504.37 sec
2013-09-18 01:39:34,782 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1543.5 sec
2013-09-18 01:39:35,788 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1543.5 sec
2013-09-18 01:39:36,793 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1559.47 sec
2013-09-18 01:39:37,798 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1559.47 sec
2013-09-18 01:39:38,803 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1572.01 sec
2013-09-18 01:39:39,808 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1572.01 sec
2013-09-18 01:39:42,552 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1585.06 sec
2013-09-18 01:39:43,556 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1585.06 sec
2013-09-18 01:39:44,715 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1611.38 sec
2013-09-18 01:39:45,727 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1611.38 sec
2013-09-18 01:39:46,732 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1611.38 sec
2013-09-18 01:39:47,738 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1611.38 sec
2013-09-18 01:39:50,360 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1611.38 sec
2013-09-18 01:39:51,394 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1611.38 sec
2013-09-18 01:39:52,399 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1611.38 sec
MapReduce Total cumulative CPU time: 26 minutes 51 seconds 380 msec
Ended Job = job_201309172235_0157
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0158
Hadoop job information for Stage-2: number of mappers: 14; number of reducers: 1
2013-09-18 01:39:56,709 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:40:04,747 Stage-2 map = 15%,  reduce = 0%
2013-09-18 01:40:07,758 Stage-2 map = 38%,  reduce = 0%
2013-09-18 01:40:10,768 Stage-2 map = 73%,  reduce = 0%
2013-09-18 01:40:13,811 Stage-2 map = 78%,  reduce = 0%
2013-09-18 01:40:14,816 Stage-2 map = 86%,  reduce = 0%, Cumulative CPU 115.95 sec
2013-09-18 01:40:15,821 Stage-2 map = 91%,  reduce = 0%, Cumulative CPU 189.93 sec
2013-09-18 01:40:16,826 Stage-2 map = 95%,  reduce = 0%, Cumulative CPU 268.74 sec
2013-09-18 01:40:17,830 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.63 sec
2013-09-18 01:40:18,835 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.63 sec
2013-09-18 01:40:19,839 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.63 sec
2013-09-18 01:40:20,843 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.63 sec
2013-09-18 01:40:21,848 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.63 sec
2013-09-18 01:40:22,853 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.63 sec
2013-09-18 01:40:23,857 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 350.63 sec
2013-09-18 01:40:24,862 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 350.63 sec
2013-09-18 01:40:25,866 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 350.63 sec
2013-09-18 01:40:26,871 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 350.63 sec
2013-09-18 01:40:27,875 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 350.63 sec
2013-09-18 01:40:28,880 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 350.63 sec
2013-09-18 01:40:29,884 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 350.63 sec
2013-09-18 01:40:30,889 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 350.63 sec
2013-09-18 01:40:31,893 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 350.63 sec
2013-09-18 01:40:32,897 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 350.63 sec
2013-09-18 01:40:33,901 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 350.63 sec
2013-09-18 01:40:34,905 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 350.63 sec
2013-09-18 01:40:35,909 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 350.63 sec
2013-09-18 01:40:36,914 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 350.63 sec
2013-09-18 01:40:37,918 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 350.63 sec
2013-09-18 01:40:38,923 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 350.63 sec
2013-09-18 01:40:39,927 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 350.63 sec
2013-09-18 01:40:40,931 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 350.63 sec
2013-09-18 01:40:41,935 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 350.63 sec
2013-09-18 01:40:42,939 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 350.63 sec
2013-09-18 01:40:43,943 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 350.63 sec
2013-09-18 01:40:44,947 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 350.63 sec
2013-09-18 01:40:45,951 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 350.63 sec
2013-09-18 01:40:46,956 Stage-2 map = 100%,  reduce = 31%, Cumulative CPU 350.63 sec
2013-09-18 01:40:52,079 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 350.63 sec
2013-09-18 01:40:53,087 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 350.63 sec
2013-09-18 01:40:54,091 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 350.63 sec
2013-09-18 01:40:55,095 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 350.63 sec
2013-09-18 01:40:56,100 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 350.63 sec
2013-09-18 01:40:57,104 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 350.63 sec
2013-09-18 01:40:58,109 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 350.63 sec
2013-09-18 01:40:59,113 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:00,117 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:01,220 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:02,224 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:03,228 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:04,232 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:05,237 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:07,415 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:08,419 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:09,423 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:10,427 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:11,431 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:12,435 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:13,439 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:14,522 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:15,526 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:16,530 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:17,534 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:18,538 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 392.86 sec
2013-09-18 01:41:19,543 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 400.39 sec
2013-09-18 01:41:20,547 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 400.39 sec
MapReduce Total cumulative CPU time: 6 minutes 40 seconds 390 msec
Ended Job = job_201309172235_0158
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1611.38 sec   HDFS Read: 882764316 HDFS Write: 4177941697 SUCCESS
Job 1: Map: 14  Reduce: 1   Cumulative CPU: 400.39 sec   HDFS Read: 4178102019 HDFS Write: 419 SUCCESS
Total MapReduce CPU Time Spent: 33 minutes 31 seconds 770 msec
OK
Time taken: 241.283 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT 1, URL, count(*) AS c FROM hits_100m GROUP BY 1, URL ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_13514@mturlrep13_201309180141_1043268364.txt
hive> SELECT 1, URL, count(*) AS c FROM hits_100m GROUP BY 1, URL ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0159
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:41:31,551 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:41:40,616 Stage-1 map = 1%,  reduce = 0%
2013-09-18 01:41:41,622 Stage-1 map = 4%,  reduce = 0%
2013-09-18 01:41:42,627 Stage-1 map = 5%,  reduce = 0%
2013-09-18 01:41:44,636 Stage-1 map = 13%,  reduce = 0%
2013-09-18 01:41:45,642 Stage-1 map = 15%,  reduce = 0%
2013-09-18 01:41:46,663 Stage-1 map = 17%,  reduce = 0%
2013-09-18 01:41:47,669 Stage-1 map = 23%,  reduce = 0%
2013-09-18 01:41:48,691 Stage-1 map = 32%,  reduce = 0%
2013-09-18 01:41:49,697 Stage-1 map = 34%,  reduce = 0%
2013-09-18 01:41:50,703 Stage-1 map = 37%,  reduce = 0%
2013-09-18 01:41:51,708 Stage-1 map = 47%,  reduce = 0%
2013-09-18 01:41:52,719 Stage-1 map = 52%,  reduce = 0%
2013-09-18 01:41:53,724 Stage-1 map = 54%,  reduce = 0%
2013-09-18 01:41:54,734 Stage-1 map = 57%,  reduce = 0%
2013-09-18 01:41:55,745 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 18.08 sec
2013-09-18 01:41:56,770 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 18.08 sec
2013-09-18 01:41:57,780 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 18.08 sec
2013-09-18 01:41:58,786 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 18.08 sec
2013-09-18 01:41:59,801 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 706.43 sec
2013-09-18 01:42:00,824 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 708.46 sec
2013-09-18 01:42:01,830 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 716.88 sec
2013-09-18 01:42:02,842 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 720.42 sec
2013-09-18 01:42:03,848 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 736.35 sec
2013-09-18 01:42:04,853 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 752.31 sec
2013-09-18 01:42:05,863 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 752.31 sec
2013-09-18 01:42:06,870 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 759.4 sec
2013-09-18 01:42:07,876 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 759.4 sec
2013-09-18 01:42:08,882 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 759.4 sec
2013-09-18 01:42:12,723 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 801.07 sec
2013-09-18 01:42:13,730 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 825.56 sec
2013-09-18 01:42:18,187 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 825.56 sec
2013-09-18 01:42:19,197 Stage-1 map = 93%,  reduce = 13%, Cumulative CPU 825.56 sec
2013-09-18 01:42:20,202 Stage-1 map = 93%,  reduce = 13%, Cumulative CPU 825.56 sec
2013-09-18 01:42:21,215 Stage-1 map = 93%,  reduce = 13%, Cumulative CPU 825.56 sec
2013-09-18 01:42:22,230 Stage-1 map = 97%,  reduce = 17%, Cumulative CPU 976.34 sec
2013-09-18 01:42:23,237 Stage-1 map = 97%,  reduce = 17%, Cumulative CPU 994.69 sec
2013-09-18 01:42:25,056 Stage-1 map = 99%,  reduce = 18%, Cumulative CPU 1066.61 sec
2013-09-18 01:42:26,062 Stage-1 map = 99%,  reduce = 19%, Cumulative CPU 1066.61 sec
2013-09-18 01:42:27,068 Stage-1 map = 99%,  reduce = 19%, Cumulative CPU 1088.21 sec
2013-09-18 01:42:28,074 Stage-1 map = 99%,  reduce = 21%, Cumulative CPU 1088.21 sec
2013-09-18 01:42:30,283 Stage-1 map = 99%,  reduce = 26%, Cumulative CPU 1088.21 sec
2013-09-18 01:42:31,294 Stage-1 map = 99%,  reduce = 29%, Cumulative CPU 1131.05 sec
2013-09-18 01:42:32,300 Stage-1 map = 99%,  reduce = 29%, Cumulative CPU 1131.05 sec
2013-09-18 01:42:33,318 Stage-1 map = 99%,  reduce = 29%, Cumulative CPU 1131.05 sec
2013-09-18 01:42:34,323 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:35,330 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:36,336 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:37,343 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:38,350 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:39,356 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:40,361 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:42,047 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:43,052 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:44,058 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:45,064 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:46,070 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:48,082 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:51,512 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:52,518 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:53,525 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:56,648 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:57,653 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:58,659 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1155.52 sec
2013-09-18 01:42:59,667 Stage-1 map = 100%,  reduce = 56%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:00,674 Stage-1 map = 100%,  reduce = 56%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:01,680 Stage-1 map = 100%,  reduce = 56%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:02,686 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:03,695 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:04,701 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:05,707 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:06,713 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:08,485 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:09,490 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:10,495 Stage-1 map = 100%,  reduce = 80%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:11,501 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:12,507 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:13,772 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:14,777 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:15,782 Stage-1 map = 100%,  reduce = 81%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:16,788 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:17,794 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:18,800 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:19,806 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:21,139 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:22,145 Stage-1 map = 100%,  reduce = 90%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:23,151 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:26,220 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:30,331 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:31,337 Stage-1 map = 100%,  reduce = 96%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:32,343 Stage-1 map = 100%,  reduce = 98%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:33,349 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1351.1 sec
2013-09-18 01:43:34,355 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1392.71 sec
2013-09-18 01:43:35,367 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1597.17 sec
2013-09-18 01:43:36,372 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1597.17 sec
MapReduce Total cumulative CPU time: 26 minutes 37 seconds 170 msec
Ended Job = job_201309172235_0159
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0160
Hadoop job information for Stage-2: number of mappers: 14; number of reducers: 1
2013-09-18 01:43:38,868 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:43:49,917 Stage-2 map = 6%,  reduce = 0%
2013-09-18 01:43:55,242 Stage-2 map = 16%,  reduce = 0%
2013-09-18 01:43:58,254 Stage-2 map = 58%,  reduce = 0%
2013-09-18 01:44:01,267 Stage-2 map = 78%,  reduce = 0%, Cumulative CPU 232.96 sec
2013-09-18 01:44:02,272 Stage-2 map = 80%,  reduce = 0%, Cumulative CPU 236.94 sec
2013-09-18 01:44:03,277 Stage-2 map = 85%,  reduce = 0%, Cumulative CPU 254.07 sec
2013-09-18 01:44:04,282 Stage-2 map = 91%,  reduce = 0%, Cumulative CPU 282.18 sec
2013-09-18 01:44:05,288 Stage-2 map = 95%,  reduce = 0%, Cumulative CPU 324.72 sec
2013-09-18 01:44:06,294 Stage-2 map = 98%,  reduce = 0%, Cumulative CPU 324.72 sec
2013-09-18 01:44:07,299 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 336.02 sec
2013-09-18 01:44:08,304 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 336.02 sec
2013-09-18 01:44:09,309 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 336.02 sec
2013-09-18 01:44:10,314 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 336.02 sec
2013-09-18 01:44:11,319 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 336.02 sec
2013-09-18 01:44:12,324 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 336.02 sec
2013-09-18 01:44:13,329 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 336.02 sec
2013-09-18 01:44:14,334 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 336.02 sec
2013-09-18 01:44:15,338 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 336.02 sec
2013-09-18 01:44:16,343 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 336.02 sec
2013-09-18 01:44:17,347 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 336.02 sec
2013-09-18 01:44:18,351 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 336.02 sec
2013-09-18 01:44:19,356 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 336.02 sec
2013-09-18 01:44:20,365 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 336.02 sec
2013-09-18 01:44:21,370 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 336.02 sec
2013-09-18 01:44:22,374 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 336.02 sec
2013-09-18 01:44:23,379 Stage-2 map = 100%,  reduce = 19%, Cumulative CPU 336.02 sec
2013-09-18 01:44:24,383 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 336.02 sec
2013-09-18 01:44:25,388 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 336.02 sec
2013-09-18 01:44:26,392 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 336.02 sec
2013-09-18 01:44:27,396 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 336.02 sec
2013-09-18 01:44:28,401 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 336.02 sec
2013-09-18 01:44:29,405 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 336.02 sec
2013-09-18 01:44:30,409 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 336.02 sec
2013-09-18 01:44:31,413 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 336.02 sec
2013-09-18 01:44:32,417 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 336.02 sec
2013-09-18 01:44:33,421 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 336.02 sec
2013-09-18 01:44:34,425 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 336.02 sec
2013-09-18 01:44:37,987 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 336.02 sec
2013-09-18 01:44:38,993 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 336.02 sec
2013-09-18 01:44:39,997 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 336.02 sec
2013-09-18 01:44:41,001 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 336.02 sec
2013-09-18 01:44:42,006 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 336.02 sec
2013-09-18 01:44:43,010 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 336.02 sec
2013-09-18 01:44:44,083 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 336.02 sec
2013-09-18 01:44:45,087 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 336.02 sec
2013-09-18 01:44:46,091 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 336.02 sec
2013-09-18 01:44:47,095 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 336.02 sec
2013-09-18 01:44:48,099 Stage-2 map = 100%,  reduce = 67%, Cumulative CPU 336.02 sec
2013-09-18 01:44:49,103 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 336.02 sec
2013-09-18 01:44:50,106 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 336.02 sec
2013-09-18 01:44:51,478 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 336.02 sec
2013-09-18 01:44:52,482 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 336.02 sec
2013-09-18 01:44:53,485 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 336.02 sec
2013-09-18 01:44:54,489 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 336.02 sec
2013-09-18 01:44:55,493 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 336.02 sec
2013-09-18 01:44:56,497 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 336.02 sec
2013-09-18 01:44:57,500 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 336.02 sec
2013-09-18 01:44:58,503 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 336.02 sec
2013-09-18 01:44:59,507 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 381.91 sec
2013-09-18 01:45:00,511 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 381.91 sec
2013-09-18 01:45:01,514 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 381.91 sec
2013-09-18 01:45:02,518 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 381.91 sec
2013-09-18 01:45:03,522 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 381.91 sec
2013-09-18 01:45:04,526 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 381.91 sec
2013-09-18 01:45:05,530 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 381.91 sec
2013-09-18 01:45:06,534 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 381.91 sec
2013-09-18 01:45:07,538 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 381.91 sec
2013-09-18 01:45:08,543 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 386.71 sec
2013-09-18 01:45:09,548 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 386.71 sec
2013-09-18 01:45:10,553 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 386.71 sec
MapReduce Total cumulative CPU time: 6 minutes 26 seconds 710 msec
Ended Job = job_201309172235_0160
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1597.17 sec   HDFS Read: 882764316 HDFS Write: 4177941697 SUCCESS
Job 1: Map: 14  Reduce: 1   Cumulative CPU: 386.71 sec   HDFS Read: 4178102019 HDFS Write: 419 SUCCESS
Total MapReduce CPU Time Spent: 33 minutes 3 seconds 880 msec
OK
Time taken: 228.198 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT 1, URL, count(*) AS c FROM hits_100m GROUP BY 1, URL ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_22530@mturlrep13_201309180145_1699071486.txt
hive> SELECT 1, URL, count(*) AS c FROM hits_100m GROUP BY 1, URL ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0161
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:45:20,932 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:45:30,999 Stage-1 map = 2%,  reduce = 0%
2013-09-18 01:45:32,006 Stage-1 map = 4%,  reduce = 0%
2013-09-18 01:45:35,020 Stage-1 map = 9%,  reduce = 0%
2013-09-18 01:45:36,034 Stage-1 map = 16%,  reduce = 0%
2013-09-18 01:45:37,039 Stage-1 map = 17%,  reduce = 0%
2013-09-18 01:45:38,045 Stage-1 map = 20%,  reduce = 0%
2013-09-18 01:45:39,054 Stage-1 map = 33%,  reduce = 0%
2013-09-18 01:45:41,072 Stage-1 map = 36%,  reduce = 0%
2013-09-18 01:45:42,085 Stage-1 map = 50%,  reduce = 0%
2013-09-18 01:45:44,098 Stage-1 map = 54%,  reduce = 0%
2013-09-18 01:45:45,104 Stage-1 map = 55%,  reduce = 0%
2013-09-18 01:45:46,112 Stage-1 map = 60%,  reduce = 0%
2013-09-18 01:45:48,123 Stage-1 map = 62%,  reduce = 0%
2013-09-18 01:45:49,139 Stage-1 map = 65%,  reduce = 0%, Cumulative CPU 19.16 sec
2013-09-18 01:45:50,153 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 61.29 sec
2013-09-18 01:45:51,160 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 82.27 sec
2013-09-18 01:45:52,178 Stage-1 map = 73%,  reduce = 0%, Cumulative CPU 104.66 sec
2013-09-18 01:45:53,184 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 149.39 sec
2013-09-18 01:45:54,190 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 196.78 sec
2013-09-18 01:45:55,197 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 221.14 sec
2013-09-18 01:45:56,210 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 271.7 sec
2013-09-18 01:45:57,219 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 297.77 sec
2013-09-18 01:45:58,225 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 297.77 sec
2013-09-18 01:45:59,232 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 326.46 sec
2013-09-18 01:46:02,398 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 976.26 sec
2013-09-18 01:46:03,406 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 978.66 sec
2013-09-18 01:46:07,469 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 981.5 sec
2013-09-18 01:46:08,492 Stage-1 map = 93%,  reduce = 9%, Cumulative CPU 985.55 sec
2013-09-18 01:46:09,498 Stage-1 map = 93%,  reduce = 9%, Cumulative CPU 990.67 sec
2013-09-18 01:46:12,267 Stage-1 map = 94%,  reduce = 9%, Cumulative CPU 999.32 sec
2013-09-18 01:46:13,634 Stage-1 map = 96%,  reduce = 16%, Cumulative CPU 1050.69 sec
2013-09-18 01:46:14,648 Stage-1 map = 98%,  reduce = 16%, Cumulative CPU 1108.43 sec
2013-09-18 01:46:15,653 Stage-1 map = 99%,  reduce = 16%, Cumulative CPU 1125.98 sec
2013-09-18 01:46:16,659 Stage-1 map = 100%,  reduce = 16%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:17,666 Stage-1 map = 100%,  reduce = 16%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:18,672 Stage-1 map = 100%,  reduce = 16%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:20,622 Stage-1 map = 100%,  reduce = 19%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:21,629 Stage-1 map = 100%,  reduce = 22%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:22,635 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:23,643 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:24,982 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:25,989 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:26,996 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:28,002 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:30,536 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:31,543 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:32,549 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:35,134 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:36,140 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:39,099 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:40,105 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:41,113 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:42,118 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:43,185 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:44,192 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:45,198 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:46,205 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:47,211 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:48,217 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:49,748 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:50,753 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:51,759 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:52,765 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:53,772 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:54,777 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:56,274 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:57,278 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:58,286 Stage-1 map = 100%,  reduce = 83%, Cumulative CPU 1161.21 sec
2013-09-18 01:46:59,292 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1161.21 sec
2013-09-18 01:47:00,301 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1508.73 sec
2013-09-18 01:47:01,307 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 1508.73 sec
2013-09-18 01:47:02,711 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1508.73 sec
2013-09-18 01:47:03,717 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1508.73 sec
2013-09-18 01:47:04,723 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 1508.73 sec
2013-09-18 01:47:05,730 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1508.73 sec
2013-09-18 01:47:09,415 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 1508.73 sec
2013-09-18 01:47:13,308 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1508.73 sec
2013-09-18 01:47:14,314 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1516.27 sec
2013-09-18 01:47:15,320 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1516.27 sec
2013-09-18 01:47:16,326 Stage-1 map = 100%,  reduce = 97%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:17,332 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:18,337 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:19,342 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:20,348 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:21,353 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:22,358 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:23,363 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:24,368 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:25,373 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:26,379 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1524.18 sec
2013-09-18 01:47:27,385 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1541.34 sec
2013-09-18 01:47:28,390 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1541.34 sec
2013-09-18 01:47:29,396 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1541.34 sec
2013-09-18 01:47:30,401 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1541.34 sec
2013-09-18 01:47:31,406 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1555.39 sec
2013-09-18 01:47:32,411 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1555.39 sec
2013-09-18 01:47:33,417 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1555.39 sec
2013-09-18 01:47:34,423 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1555.39 sec
2013-09-18 01:47:35,428 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1555.39 sec
2013-09-18 01:47:36,433 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1555.39 sec
2013-09-18 01:47:37,855 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1555.39 sec
2013-09-18 01:47:38,866 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1555.39 sec
2013-09-18 01:47:39,872 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1555.39 sec
2013-09-18 01:47:40,877 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1555.39 sec
2013-09-18 01:47:41,881 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1565.2 sec
2013-09-18 01:47:42,886 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1565.2 sec
2013-09-18 01:47:44,926 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1565.2 sec
2013-09-18 01:47:45,931 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1565.2 sec
2013-09-18 01:47:46,936 Stage-1 map = 100%,  reduce = 99%, Cumulative CPU 1565.2 sec
2013-09-18 01:47:47,942 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1621.8 sec
2013-09-18 01:47:51,089 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1621.8 sec
MapReduce Total cumulative CPU time: 27 minutes 1 seconds 800 msec
Ended Job = job_201309172235_0161
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0162
Hadoop job information for Stage-2: number of mappers: 14; number of reducers: 1
2013-09-18 01:47:56,428 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:48:04,465 Stage-2 map = 7%,  reduce = 0%
2013-09-18 01:48:05,469 Stage-2 map = 10%,  reduce = 0%
2013-09-18 01:48:07,477 Stage-2 map = 21%,  reduce = 0%
2013-09-18 01:48:08,482 Stage-2 map = 34%,  reduce = 0%
2013-09-18 01:48:10,489 Stage-2 map = 43%,  reduce = 0%
2013-09-18 01:48:11,494 Stage-2 map = 70%,  reduce = 0%
2013-09-18 01:48:14,505 Stage-2 map = 78%,  reduce = 0%
2013-09-18 01:48:15,510 Stage-2 map = 89%,  reduce = 0%, Cumulative CPU 167.69 sec
2013-09-18 01:48:16,516 Stage-2 map = 89%,  reduce = 0%, Cumulative CPU 167.69 sec
2013-09-18 01:48:17,535 Stage-2 map = 94%,  reduce = 0%, Cumulative CPU 246.5 sec
2013-09-18 01:48:18,541 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 356.26 sec
2013-09-18 01:48:19,545 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 356.26 sec
2013-09-18 01:48:20,551 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 356.26 sec
2013-09-18 01:48:21,555 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 356.26 sec
2013-09-18 01:48:22,560 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 356.26 sec
2013-09-18 01:48:23,565 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 356.26 sec
2013-09-18 01:48:24,570 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 356.26 sec
2013-09-18 01:48:25,574 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 356.26 sec
2013-09-18 01:48:26,578 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 356.26 sec
2013-09-18 01:48:27,582 Stage-2 map = 100%,  reduce = 10%, Cumulative CPU 356.26 sec
2013-09-18 01:48:28,587 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 356.26 sec
2013-09-18 01:48:29,591 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 356.26 sec
2013-09-18 01:48:30,595 Stage-2 map = 100%,  reduce = 12%, Cumulative CPU 356.26 sec
2013-09-18 01:48:31,599 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 356.26 sec
2013-09-18 01:48:32,603 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 356.26 sec
2013-09-18 01:48:33,606 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 356.26 sec
2013-09-18 01:48:34,610 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 356.26 sec
2013-09-18 01:48:35,615 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 356.26 sec
2013-09-18 01:48:36,619 Stage-2 map = 100%,  reduce = 21%, Cumulative CPU 356.26 sec
2013-09-18 01:48:37,624 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 356.26 sec
2013-09-18 01:48:38,628 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 356.26 sec
2013-09-18 01:48:39,632 Stage-2 map = 100%,  reduce = 24%, Cumulative CPU 356.26 sec
2013-09-18 01:48:40,636 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 356.26 sec
2013-09-18 01:48:41,641 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 356.26 sec
2013-09-18 01:48:42,645 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 356.26 sec
2013-09-18 01:48:43,649 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 356.26 sec
2013-09-18 01:48:44,653 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 356.26 sec
2013-09-18 01:48:45,658 Stage-2 map = 100%,  reduce = 29%, Cumulative CPU 356.26 sec
2013-09-18 01:48:46,662 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 356.26 sec
2013-09-18 01:48:52,547 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 356.26 sec
2013-09-18 01:48:53,551 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 356.26 sec
2013-09-18 01:48:54,555 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 356.26 sec
2013-09-18 01:48:55,649 Stage-2 map = 100%,  reduce = 68%, Cumulative CPU 356.26 sec
2013-09-18 01:48:56,653 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 356.26 sec
2013-09-18 01:48:57,657 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 356.26 sec
2013-09-18 01:48:58,661 Stage-2 map = 100%,  reduce = 70%, Cumulative CPU 356.26 sec
2013-09-18 01:49:00,401 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:01,405 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:02,409 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:03,413 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:04,417 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:05,421 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:06,424 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:07,428 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:08,431 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:09,435 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:10,439 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:11,443 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:12,447 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:13,452 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:14,456 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:15,461 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:16,465 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:17,470 Stage-2 map = 100%,  reduce = 78%, Cumulative CPU 399.92 sec
2013-09-18 01:49:18,477 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 406.76 sec
2013-09-18 01:49:19,481 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 406.76 sec
MapReduce Total cumulative CPU time: 6 minutes 46 seconds 760 msec
Ended Job = job_201309172235_0162
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 1621.8 sec   HDFS Read: 882764316 HDFS Write: 4177941697 SUCCESS
Job 1: Map: 14  Reduce: 1   Cumulative CPU: 406.76 sec   HDFS Read: 4178102019 HDFS Write: 419 SUCCESS
Total MapReduce CPU Time Spent: 33 minutes 48 seconds 560 msec
OK
Time taken: 246.999 seconds, Fetched: 10 row(s)
hive> quit;
-- агрегация по URL и числу.;


times: 1
query: SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, count(*) AS c FROM hits_100m GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_31570@mturlrep13_201309180149_1050651506.txt
hive> SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, count(*) AS c FROM hits_100m GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0163
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:49:38,118 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:49:51,275 Stage-1 map = 2%,  reduce = 0%
2013-09-18 01:49:52,282 Stage-1 map = 6%,  reduce = 0%
2013-09-18 01:49:53,293 Stage-1 map = 8%,  reduce = 0%
2013-09-18 01:49:54,299 Stage-1 map = 25%,  reduce = 0%
2013-09-18 01:49:55,305 Stage-1 map = 31%,  reduce = 0%
2013-09-18 01:49:56,314 Stage-1 map = 33%,  reduce = 0%
2013-09-18 01:49:57,321 Stage-1 map = 49%,  reduce = 0%
2013-09-18 01:49:58,338 Stage-1 map = 55%,  reduce = 0%
2013-09-18 01:49:59,351 Stage-1 map = 60%,  reduce = 0%, Cumulative CPU 13.34 sec
2013-09-18 01:50:00,378 Stage-1 map = 70%,  reduce = 0%, Cumulative CPU 494.57 sec
2013-09-18 01:50:01,385 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 494.57 sec
2013-09-18 01:50:02,403 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 498.86 sec
2013-09-18 01:50:03,411 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 506.12 sec
2013-09-18 01:50:04,419 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 517.77 sec
2013-09-18 01:50:05,426 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 529.34 sec
2013-09-18 01:50:06,432 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 536.01 sec
2013-09-18 01:50:07,440 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 574.29 sec
2013-09-18 01:50:08,448 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 643.5 sec
2013-09-18 01:50:09,455 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 690.76 sec
2013-09-18 01:50:10,461 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 701.69 sec
2013-09-18 01:50:11,468 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 701.69 sec
2013-09-18 01:50:12,491 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 716.61 sec
2013-09-18 01:50:13,497 Stage-1 map = 99%,  reduce = 4%, Cumulative CPU 758.65 sec
2013-09-18 01:50:14,502 Stage-1 map = 99%,  reduce = 19%, Cumulative CPU 758.65 sec
2013-09-18 01:50:15,508 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 774.74 sec
2013-09-18 01:50:16,515 Stage-1 map = 100%,  reduce = 26%, Cumulative CPU 774.74 sec
2013-09-18 01:50:17,521 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 774.74 sec
2013-09-18 01:50:18,526 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 774.74 sec
2013-09-18 01:50:19,533 Stage-1 map = 100%,  reduce = 32%, Cumulative CPU 774.74 sec
2013-09-18 01:50:20,539 Stage-1 map = 100%,  reduce = 33%, Cumulative CPU 774.74 sec
2013-09-18 01:50:21,545 Stage-1 map = 100%,  reduce = 39%, Cumulative CPU 774.74 sec
2013-09-18 01:50:22,551 Stage-1 map = 100%,  reduce = 42%, Cumulative CPU 774.74 sec
2013-09-18 01:50:23,557 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 774.74 sec
2013-09-18 01:50:24,562 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 774.74 sec
2013-09-18 01:50:25,568 Stage-1 map = 100%,  reduce = 71%, Cumulative CPU 774.74 sec
2013-09-18 01:50:26,574 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 774.74 sec
2013-09-18 01:50:27,583 Stage-1 map = 100%,  reduce = 86%, Cumulative CPU 802.67 sec
2013-09-18 01:50:28,589 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 874.81 sec
2013-09-18 01:50:29,594 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 934.02 sec
2013-09-18 01:50:30,617 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 934.02 sec
MapReduce Total cumulative CPU time: 15 minutes 34 seconds 20 msec
Ended Job = job_201309172235_0163
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0164
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-18 01:50:34,091 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:50:44,120 Stage-2 map = 23%,  reduce = 0%
2013-09-18 01:50:47,129 Stage-2 map = 46%,  reduce = 0%
2013-09-18 01:50:51,142 Stage-2 map = 62%,  reduce = 0%, Cumulative CPU 23.2 sec
2013-09-18 01:50:52,147 Stage-2 map = 62%,  reduce = 0%, Cumulative CPU 23.2 sec
2013-09-18 01:50:53,152 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 23.2 sec
2013-09-18 01:50:54,157 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 23.2 sec
2013-09-18 01:50:55,162 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 23.2 sec
2013-09-18 01:50:56,167 Stage-2 map = 75%,  reduce = 0%, Cumulative CPU 23.2 sec
2013-09-18 01:50:58,116 Stage-2 map = 75%,  reduce = 0%, Cumulative CPU 23.2 sec
2013-09-18 01:50:59,121 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 23.2 sec
2013-09-18 01:51:00,129 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 23.2 sec
2013-09-18 01:51:01,134 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:02,139 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:03,144 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:04,149 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:05,154 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:06,158 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:07,163 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:08,172 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:09,177 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:10,182 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:11,186 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:12,191 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:13,195 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:14,200 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:15,205 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:16,209 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.42 sec
2013-09-18 01:51:17,214 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.78 sec
2013-09-18 01:51:18,218 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.78 sec
2013-09-18 01:51:19,223 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.78 sec
2013-09-18 01:51:20,227 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.78 sec
2013-09-18 01:51:21,232 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.78 sec
2013-09-18 01:51:22,236 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.78 sec
2013-09-18 01:51:23,241 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 77.78 sec
2013-09-18 01:51:24,245 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 77.78 sec
2013-09-18 01:51:25,250 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 77.78 sec
2013-09-18 01:51:26,254 Stage-2 map = 100%,  reduce = 89%, Cumulative CPU 77.78 sec
2013-09-18 01:51:27,259 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 87.04 sec
2013-09-18 01:51:28,264 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 87.04 sec
MapReduce Total cumulative CPU time: 1 minutes 27 seconds 40 msec
Ended Job = job_201309172235_0164
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 934.02 sec   HDFS Read: 339202346 HDFS Write: 374792539 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 87.04 sec   HDFS Read: 374796261 HDFS Write: 498 SUCCESS
Total MapReduce CPU Time Spent: 17 minutes 1 seconds 60 msec
OK
Time taken: 120.684 seconds, Fetched: 10 row(s)
hive> quit;

times: 2
query: SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, count(*) AS c FROM hits_100m GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_7222@mturlrep13_201309180151_945335118.txt
hive> SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, count(*) AS c FROM hits_100m GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0165
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:51:37,818 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:51:49,905 Stage-1 map = 2%,  reduce = 0%
2013-09-18 01:51:50,923 Stage-1 map = 6%,  reduce = 0%
2013-09-18 01:51:52,936 Stage-1 map = 15%,  reduce = 0%
2013-09-18 01:51:53,959 Stage-1 map = 33%,  reduce = 0%
2013-09-18 01:51:54,965 Stage-1 map = 36%,  reduce = 0%
2013-09-18 01:51:55,979 Stage-1 map = 41%,  reduce = 0%
2013-09-18 01:51:57,007 Stage-1 map = 62%,  reduce = 0%
2013-09-18 01:51:58,018 Stage-1 map = 63%,  reduce = 0%
2013-09-18 01:51:59,032 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 13.91 sec
2013-09-18 01:52:00,049 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 13.91 sec
2013-09-18 01:52:01,062 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 546.5 sec
2013-09-18 01:52:02,075 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 546.5 sec
2013-09-18 01:52:03,086 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 553.11 sec
2013-09-18 01:52:04,094 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 570.16 sec
2013-09-18 01:52:05,112 Stage-1 map = 86%,  reduce = 0%, Cumulative CPU 591.73 sec
2013-09-18 01:52:06,122 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 606.61 sec
2013-09-18 01:52:07,134 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 689.49 sec
2013-09-18 01:52:08,140 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 704.24 sec
2013-09-18 01:52:09,146 Stage-1 map = 95%,  reduce = 0%, Cumulative CPU 704.24 sec
2013-09-18 01:52:10,152 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 704.24 sec
2013-09-18 01:52:11,158 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 704.24 sec
2013-09-18 01:52:12,165 Stage-1 map = 98%,  reduce = 4%, Cumulative CPU 717.42 sec
2013-09-18 01:52:13,170 Stage-1 map = 99%,  reduce = 23%, Cumulative CPU 745.46 sec
2013-09-18 01:52:14,176 Stage-1 map = 100%,  reduce = 25%, Cumulative CPU 775.29 sec
2013-09-18 01:52:15,181 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 775.29 sec
2013-09-18 01:52:16,187 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 775.29 sec
2013-09-18 01:52:17,193 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 775.29 sec
2013-09-18 01:52:18,199 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 775.29 sec
2013-09-18 01:52:19,204 Stage-1 map = 100%,  reduce = 31%, Cumulative CPU 775.29 sec
2013-09-18 01:52:20,209 Stage-1 map = 100%,  reduce = 34%, Cumulative CPU 775.29 sec
2013-09-18 01:52:21,215 Stage-1 map = 100%,  reduce = 43%, Cumulative CPU 775.29 sec
2013-09-18 01:52:22,221 Stage-1 map = 100%,  reduce = 60%, Cumulative CPU 775.29 sec
2013-09-18 01:52:23,226 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 775.29 sec
2013-09-18 01:52:24,232 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 775.29 sec
2013-09-18 01:52:25,238 Stage-1 map = 100%,  reduce = 79%, Cumulative CPU 775.29 sec
2013-09-18 01:52:26,245 Stage-1 map = 100%,  reduce = 84%, Cumulative CPU 789.78 sec
2013-09-18 01:52:27,251 Stage-1 map = 100%,  reduce = 91%, Cumulative CPU 848.54 sec
2013-09-18 01:52:28,257 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 936.77 sec
2013-09-18 01:52:29,275 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 936.77 sec
2013-09-18 01:52:30,282 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 936.77 sec
MapReduce Total cumulative CPU time: 15 minutes 36 seconds 770 msec
Ended Job = job_201309172235_0165
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0166
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-18 01:52:33,818 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:52:43,850 Stage-2 map = 23%,  reduce = 0%
2013-09-18 01:52:46,861 Stage-2 map = 46%,  reduce = 0%
2013-09-18 01:52:49,871 Stage-2 map = 52%,  reduce = 0%
2013-09-18 01:52:51,880 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 24.11 sec
2013-09-18 01:52:52,885 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 24.11 sec
2013-09-18 01:52:53,890 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 24.11 sec
2013-09-18 01:52:54,896 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 24.11 sec
2013-09-18 01:52:55,901 Stage-2 map = 75%,  reduce = 0%, Cumulative CPU 24.11 sec
2013-09-18 01:52:56,907 Stage-2 map = 75%,  reduce = 0%, Cumulative CPU 24.11 sec
2013-09-18 01:52:57,912 Stage-2 map = 75%,  reduce = 0%, Cumulative CPU 24.11 sec
2013-09-18 01:52:58,918 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 24.11 sec
2013-09-18 01:52:59,923 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 24.11 sec
2013-09-18 01:53:00,928 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:01,933 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:02,938 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:03,943 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:04,948 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:05,959 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:06,963 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:07,968 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:08,973 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:09,979 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:10,984 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:11,989 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:12,995 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:14,000 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:15,006 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 58.35 sec
2013-09-18 01:53:16,010 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 78.21 sec
2013-09-18 01:53:17,014 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 78.21 sec
2013-09-18 01:53:18,018 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 78.21 sec
2013-09-18 01:53:19,023 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 78.21 sec
2013-09-18 01:53:20,027 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 78.21 sec
2013-09-18 01:53:21,031 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 78.21 sec
2013-09-18 01:53:22,035 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 78.21 sec
2013-09-18 01:53:23,039 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 78.21 sec
2013-09-18 01:53:24,044 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 78.21 sec
2013-09-18 01:53:25,048 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 78.21 sec
2013-09-18 01:53:26,053 Stage-2 map = 100%,  reduce = 83%, Cumulative CPU 78.21 sec
2013-09-18 01:53:27,058 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 87.43 sec
2013-09-18 01:53:28,063 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 87.43 sec
2013-09-18 01:53:29,068 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 87.43 sec
MapReduce Total cumulative CPU time: 1 minutes 27 seconds 430 msec
Ended Job = job_201309172235_0166
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 936.77 sec   HDFS Read: 339202346 HDFS Write: 374792539 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 87.43 sec   HDFS Read: 374796261 HDFS Write: 498 SUCCESS
Total MapReduce CPU Time Spent: 17 minutes 4 seconds 200 msec
OK
Time taken: 118.948 seconds, Fetched: 10 row(s)
hive> quit;

times: 3
query: SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, count(*) AS c FROM hits_100m GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_14110@mturlrep13_201309180153_1737492136.txt
hive> SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, count(*) AS c FROM hits_100m GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0167
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:53:39,704 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:53:50,787 Stage-1 map = 1%,  reduce = 0%
2013-09-18 01:53:51,799 Stage-1 map = 4%,  reduce = 0%
2013-09-18 01:53:52,813 Stage-1 map = 5%,  reduce = 0%
2013-09-18 01:53:53,831 Stage-1 map = 14%,  reduce = 0%
2013-09-18 01:53:54,840 Stage-1 map = 34%,  reduce = 0%
2013-09-18 01:53:55,846 Stage-1 map = 35%,  reduce = 0%
2013-09-18 01:53:56,851 Stage-1 map = 39%,  reduce = 0%
2013-09-18 01:53:57,860 Stage-1 map = 59%,  reduce = 0%
2013-09-18 01:53:58,883 Stage-1 map = 63%,  reduce = 0%
2013-09-18 01:53:59,905 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 13.8 sec
2013-09-18 01:54:00,943 Stage-1 map = 72%,  reduce = 0%, Cumulative CPU 13.8 sec
2013-09-18 01:54:01,952 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 535.34 sec
2013-09-18 01:54:02,959 Stage-1 map = 80%,  reduce = 0%, Cumulative CPU 537.7 sec
2013-09-18 01:54:03,966 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 540.16 sec
2013-09-18 01:54:04,973 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 551.46 sec
2013-09-18 01:54:05,979 Stage-1 map = 85%,  reduce = 0%, Cumulative CPU 573.16 sec
2013-09-18 01:54:07,008 Stage-1 map = 89%,  reduce = 0%, Cumulative CPU 605.0 sec
2013-09-18 01:54:08,027 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 654.51 sec
2013-09-18 01:54:09,034 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 691.64 sec
2013-09-18 01:54:10,040 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 700.45 sec
2013-09-18 01:54:11,047 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 700.45 sec
2013-09-18 01:54:12,053 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 700.45 sec
2013-09-18 01:54:13,060 Stage-1 map = 98%,  reduce = 3%, Cumulative CPU 700.45 sec
2013-09-18 01:54:14,066 Stage-1 map = 99%,  reduce = 21%, Cumulative CPU 745.67 sec
2013-09-18 01:54:15,072 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 776.9 sec
2013-09-18 01:54:16,078 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 776.9 sec
2013-09-18 01:54:17,084 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 776.9 sec
2013-09-18 01:54:18,094 Stage-1 map = 100%,  reduce = 29%, Cumulative CPU 776.9 sec
2013-09-18 01:54:19,103 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 776.9 sec
2013-09-18 01:54:20,109 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 776.9 sec
2013-09-18 01:54:21,115 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 776.9 sec
2013-09-18 01:54:22,125 Stage-1 map = 100%,  reduce = 43%, Cumulative CPU 776.9 sec
2013-09-18 01:54:23,131 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 776.9 sec
2013-09-18 01:54:24,137 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 776.9 sec
2013-09-18 01:54:25,144 Stage-1 map = 100%,  reduce = 72%, Cumulative CPU 776.9 sec
2013-09-18 01:54:26,149 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 776.9 sec
2013-09-18 01:54:27,155 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 776.9 sec
2013-09-18 01:54:28,164 Stage-1 map = 100%,  reduce = 92%, Cumulative CPU 862.86 sec
2013-09-18 01:54:29,177 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 934.72 sec
2013-09-18 01:54:30,183 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 934.72 sec
2013-09-18 01:54:31,188 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 934.72 sec
MapReduce Total cumulative CPU time: 15 minutes 34 seconds 720 msec
Ended Job = job_201309172235_0167
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0168
Hadoop job information for Stage-2: number of mappers: 2; number of reducers: 1
2013-09-18 01:54:34,764 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:54:44,797 Stage-2 map = 23%,  reduce = 0%
2013-09-18 01:54:47,807 Stage-2 map = 46%,  reduce = 0%
2013-09-18 01:54:50,817 Stage-2 map = 52%,  reduce = 0%
2013-09-18 01:54:52,826 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 23.57 sec
2013-09-18 01:54:53,831 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 23.57 sec
2013-09-18 01:54:54,836 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 23.57 sec
2013-09-18 01:54:55,841 Stage-2 map = 69%,  reduce = 0%, Cumulative CPU 23.57 sec
2013-09-18 01:54:56,847 Stage-2 map = 75%,  reduce = 0%, Cumulative CPU 23.57 sec
2013-09-18 01:54:57,852 Stage-2 map = 75%,  reduce = 0%, Cumulative CPU 23.57 sec
2013-09-18 01:54:58,857 Stage-2 map = 75%,  reduce = 0%, Cumulative CPU 23.57 sec
2013-09-18 01:54:59,863 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 23.57 sec
2013-09-18 01:55:00,868 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 23.57 sec
2013-09-18 01:55:01,874 Stage-2 map = 81%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:02,879 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:03,883 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:04,888 Stage-2 map = 87%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:05,898 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:06,903 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:07,908 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:08,913 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:09,918 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:10,923 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:11,928 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:12,933 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:13,938 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:14,942 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:15,947 Stage-2 map = 94%,  reduce = 17%, Cumulative CPU 57.88 sec
2013-09-18 01:55:16,952 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.06 sec
2013-09-18 01:55:17,957 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.06 sec
2013-09-18 01:55:18,962 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.06 sec
2013-09-18 01:55:19,967 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.06 sec
2013-09-18 01:55:20,972 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.06 sec
2013-09-18 01:55:21,977 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.06 sec
2013-09-18 01:55:22,982 Stage-2 map = 100%,  reduce = 17%, Cumulative CPU 77.06 sec
2013-09-18 01:55:23,987 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 77.06 sec
2013-09-18 01:55:24,992 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 77.06 sec
2013-09-18 01:55:25,997 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 77.06 sec
2013-09-18 01:55:27,002 Stage-2 map = 100%,  reduce = 89%, Cumulative CPU 77.06 sec
2013-09-18 01:55:28,008 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 86.19 sec
2013-09-18 01:55:29,013 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 86.19 sec
MapReduce Total cumulative CPU time: 1 minutes 26 seconds 190 msec
Ended Job = job_201309172235_0168
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 934.72 sec   HDFS Read: 339202346 HDFS Write: 374792539 SUCCESS
Job 1: Map: 2  Reduce: 1   Cumulative CPU: 86.19 sec   HDFS Read: 374796261 HDFS Write: 498 SUCCESS
Total MapReduce CPU Time Spent: 17 minutes 0 seconds 910 msec
OK
Time taken: 118.18 seconds, Fetched: 10 row(s)
hive> quit;

times: 1
query:  
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_21734@mturlrep13_201309180155_15887287.txt
hive>  ;
hive> quit;

times: 2
query:  
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_21941@mturlrep13_201309180155_1922409770.txt
hive>  ;
hive> quit;

times: 3
query:  
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_22141@mturlrep13_201309180155_818952348.txt
hive>  ;
hive> quit;

times: 1
query: SELECT URL,     count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND URL != '' GROUP BY URL ORDER BY PageViews DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_22349@mturlrep13_201309180155_129320467.txt
hive> SELECT URL,     count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND URL != '' GROUP BY URL ORDER BY PageViews DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0169
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:55:58,117 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:56:11,233 Stage-1 map = 2%,  reduce = 0%
2013-09-18 01:56:12,984 Stage-1 map = 3%,  reduce = 0%
2013-09-18 01:56:13,990 Stage-1 map = 8%,  reduce = 0%
2013-09-18 01:56:14,996 Stage-1 map = 14%,  reduce = 0%
2013-09-18 01:56:16,314 Stage-1 map = 16%,  reduce = 0%
2013-09-18 01:56:21,716 Stage-1 map = 36%,  reduce = 0%
2013-09-18 01:56:22,731 Stage-1 map = 38%,  reduce = 0%
2013-09-18 01:56:24,752 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 22.96 sec
2013-09-18 01:56:25,760 Stage-1 map = 58%,  reduce = 0%, Cumulative CPU 22.96 sec
2013-09-18 01:56:26,767 Stage-1 map = 58%,  reduce = 0%, Cumulative CPU 22.96 sec
2013-09-18 01:56:28,182 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 44.33 sec
2013-09-18 01:56:29,188 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 44.33 sec
2013-09-18 01:56:30,404 Stage-1 map = 75%,  reduce = 0%, Cumulative CPU 90.25 sec
2013-09-18 01:56:31,813 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 335.9 sec
2013-09-18 01:56:32,821 Stage-1 map = 98%,  reduce = 0%, Cumulative CPU 404.3 sec
2013-09-18 01:56:33,827 Stage-1 map = 100%,  reduce = 2%, Cumulative CPU 427.67 sec
2013-09-18 01:56:34,833 Stage-1 map = 100%,  reduce = 11%, Cumulative CPU 427.67 sec
2013-09-18 01:56:35,840 Stage-1 map = 100%,  reduce = 14%, Cumulative CPU 427.67 sec
2013-09-18 01:56:36,846 Stage-1 map = 100%,  reduce = 23%, Cumulative CPU 427.67 sec
2013-09-18 01:56:37,854 Stage-1 map = 100%,  reduce = 41%, Cumulative CPU 436.66 sec
2013-09-18 01:56:38,862 Stage-1 map = 100%,  reduce = 47%, Cumulative CPU 436.66 sec
2013-09-18 01:56:39,869 Stage-1 map = 100%,  reduce = 69%, Cumulative CPU 446.83 sec
2013-09-18 01:56:40,875 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 454.28 sec
2013-09-18 01:56:41,881 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 463.94 sec
2013-09-18 01:56:42,887 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 463.94 sec
MapReduce Total cumulative CPU time: 7 minutes 43 seconds 940 msec
Ended Job = job_201309172235_0169
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0170
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 01:56:46,407 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:56:48,416 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.68 sec
2013-09-18 01:56:49,421 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.68 sec
2013-09-18 01:56:50,426 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.68 sec
2013-09-18 01:56:51,430 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.68 sec
2013-09-18 01:56:52,435 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.68 sec
2013-09-18 01:56:53,439 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.68 sec
2013-09-18 01:56:54,444 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.68 sec
2013-09-18 01:56:55,450 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.14 sec
2013-09-18 01:56:56,455 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.14 sec
2013-09-18 01:56:57,460 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.14 sec
MapReduce Total cumulative CPU time: 2 seconds 140 msec
Ended Job = job_201309172235_0170
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 463.94 sec   HDFS Read: 993495605 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.14 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 46 seconds 80 msec
OK
Time taken: 70.292 seconds
hive> quit;

times: 2
query: SELECT URL,     count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND URL != '' GROUP BY URL ORDER BY PageViews DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_29743@mturlrep13_201309180156_208621189.txt
hive> SELECT URL,     count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND URL != '' GROUP BY URL ORDER BY PageViews DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0171
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:57:08,252 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:57:17,362 Stage-1 map = 1%,  reduce = 0%
2013-09-18 01:57:18,402 Stage-1 map = 8%,  reduce = 0%
2013-09-18 01:57:19,415 Stage-1 map = 9%,  reduce = 0%
2013-09-18 01:57:20,427 Stage-1 map = 13%,  reduce = 0%
2013-09-18 01:57:21,432 Stage-1 map = 36%,  reduce = 0%
2013-09-18 01:57:22,447 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 8.67 sec
2013-09-18 01:57:23,455 Stage-1 map = 46%,  reduce = 0%, Cumulative CPU 8.67 sec
2013-09-18 01:57:24,469 Stage-1 map = 52%,  reduce = 0%, Cumulative CPU 8.67 sec
2013-09-18 01:57:25,476 Stage-1 map = 64%,  reduce = 0%, Cumulative CPU 33.09 sec
2013-09-18 01:57:26,484 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 276.39 sec
2013-09-18 01:57:27,491 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 454.8 sec
2013-09-18 01:57:28,497 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 454.8 sec
2013-09-18 01:57:29,504 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 454.8 sec
2013-09-18 01:57:30,510 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 454.8 sec
2013-09-18 01:57:31,516 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 454.8 sec
2013-09-18 01:57:32,522 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 454.8 sec
2013-09-18 01:57:33,529 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 454.8 sec
2013-09-18 01:57:34,536 Stage-1 map = 100%,  reduce = 18%, Cumulative CPU 454.8 sec
2013-09-18 01:57:35,549 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 469.58 sec
2013-09-18 01:57:36,556 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 487.12 sec
2013-09-18 01:57:37,564 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 493.48 sec
2013-09-18 01:57:38,570 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 493.48 sec
2013-09-18 01:57:39,576 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 493.48 sec
MapReduce Total cumulative CPU time: 8 minutes 13 seconds 480 msec
Ended Job = job_201309172235_0171
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0172
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 01:57:43,495 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:57:44,501 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 01:57:45,506 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 01:57:46,511 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 01:57:47,516 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 01:57:48,520 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 01:57:49,524 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 01:57:50,529 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 01:57:51,533 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 01:57:52,538 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.25 sec
2013-09-18 01:57:53,544 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.25 sec
2013-09-18 01:57:54,549 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.25 sec
MapReduce Total cumulative CPU time: 2 seconds 250 msec
Ended Job = job_201309172235_0172
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 493.48 sec   HDFS Read: 993495605 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.25 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 15 seconds 730 msec
OK
Time taken: 55.211 seconds
hive> quit;

times: 3
query: SELECT URL,     count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND URL != '' GROUP BY URL ORDER BY PageViews DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_4778@mturlrep13_201309180157_2071772922.txt
hive> SELECT URL,     count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND URL != '' GROUP BY URL ORDER BY PageViews DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0173
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:58:05,573 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:58:14,649 Stage-1 map = 1%,  reduce = 0%
2013-09-18 01:58:15,664 Stage-1 map = 9%,  reduce = 0%
2013-09-18 01:58:17,723 Stage-1 map = 12%,  reduce = 0%
2013-09-18 01:58:18,734 Stage-1 map = 36%,  reduce = 0%
2013-09-18 01:58:19,756 Stage-1 map = 39%,  reduce = 0%, Cumulative CPU 8.38 sec
2013-09-18 01:58:20,765 Stage-1 map = 43%,  reduce = 0%, Cumulative CPU 8.38 sec
2013-09-18 01:58:21,773 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 8.38 sec
2013-09-18 01:58:22,797 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 32.26 sec
2013-09-18 01:58:23,804 Stage-1 map = 87%,  reduce = 0%, Cumulative CPU 254.83 sec
2013-09-18 01:58:24,812 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 460.38 sec
2013-09-18 01:58:25,818 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 460.38 sec
2013-09-18 01:58:26,825 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 460.38 sec
2013-09-18 01:58:27,831 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 460.38 sec
2013-09-18 01:58:28,836 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 460.38 sec
2013-09-18 01:58:29,842 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 460.38 sec
2013-09-18 01:58:30,847 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 460.38 sec
2013-09-18 01:58:31,854 Stage-1 map = 100%,  reduce = 15%, Cumulative CPU 460.38 sec
2013-09-18 01:58:32,864 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 474.21 sec
2013-09-18 01:58:33,870 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 487.03 sec
2013-09-18 01:58:34,876 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 496.96 sec
2013-09-18 01:58:35,882 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 496.96 sec
2013-09-18 01:58:36,888 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 496.96 sec
MapReduce Total cumulative CPU time: 8 minutes 16 seconds 960 msec
Ended Job = job_201309172235_0173
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0174
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 01:58:40,381 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:58:41,387 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 01:58:42,393 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 01:58:43,398 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 01:58:44,403 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 01:58:45,408 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 01:58:46,413 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 01:58:47,418 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 01:58:48,423 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 01:58:49,429 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.2 sec
2013-09-18 01:58:50,435 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.2 sec
2013-09-18 01:58:51,440 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.2 sec
MapReduce Total cumulative CPU time: 2 seconds 200 msec
Ended Job = job_201309172235_0174
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 496.96 sec   HDFS Read: 993495605 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.2 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 19 seconds 160 msec
OK
Time taken: 55.076 seconds
hive> quit;

times: 1
query: SELECT Title, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND Title != '' GROUP BY Title ORDER BY PageViews DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_12570@mturlrep13_201309180158_2097047961.txt
hive> SELECT Title, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND Title != '' GROUP BY Title ORDER BY PageViews DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0175
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 01:59:09,511 Stage-1 map = 0%,  reduce = 0%
2013-09-18 01:59:19,576 Stage-1 map = 5%,  reduce = 0%
2013-09-18 01:59:22,593 Stage-1 map = 15%,  reduce = 0%
2013-09-18 01:59:25,607 Stage-1 map = 29%,  reduce = 0%
2013-09-18 01:59:26,612 Stage-1 map = 30%,  reduce = 0%
2013-09-18 01:59:28,833 Stage-1 map = 42%,  reduce = 0%
2013-09-18 01:59:34,330 Stage-1 map = 67%,  reduce = 0%, Cumulative CPU 122.09 sec
2013-09-18 01:59:35,337 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 176.65 sec
2013-09-18 01:59:36,349 Stage-1 map = 82%,  reduce = 0%, Cumulative CPU 230.21 sec
2013-09-18 01:59:37,356 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 395.83 sec
2013-09-18 01:59:38,362 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 407.07 sec
2013-09-18 01:59:39,368 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 407.07 sec
2013-09-18 01:59:40,374 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 407.07 sec
2013-09-18 01:59:41,380 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 407.07 sec
2013-09-18 01:59:42,386 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 407.07 sec
2013-09-18 01:59:43,395 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 413.7 sec
2013-09-18 01:59:44,403 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 423.63 sec
2013-09-18 01:59:45,410 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 434.03 sec
2013-09-18 01:59:46,417 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 443.68 sec
2013-09-18 01:59:47,424 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 443.68 sec
MapReduce Total cumulative CPU time: 7 minutes 23 seconds 680 msec
Ended Job = job_201309172235_0175
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0176
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 01:59:50,937 Stage-2 map = 0%,  reduce = 0%
2013-09-18 01:59:52,947 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 01:59:53,953 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 01:59:54,958 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 01:59:55,963 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 01:59:56,969 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 01:59:57,974 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 01:59:58,981 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.72 sec
2013-09-18 01:59:59,987 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.72 sec
2013-09-18 02:00:00,993 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.25 sec
2013-09-18 02:00:01,999 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.25 sec
MapReduce Total cumulative CPU time: 2 seconds 250 msec
Ended Job = job_201309172235_0176
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 443.68 sec   HDFS Read: 876843443 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.25 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 25 seconds 930 msec
OK
Time taken: 63.4 seconds
hive> quit;

times: 2
query: SELECT Title, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND Title != '' GROUP BY Title ORDER BY PageViews DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_19761@mturlrep13_201309180200_1379051620.txt
hive> SELECT Title, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND Title != '' GROUP BY Title ORDER BY PageViews DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0177
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:00:13,526 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:00:23,659 Stage-1 map = 11%,  reduce = 0%
2013-09-18 02:00:26,730 Stage-1 map = 48%,  reduce = 0%
2013-09-18 02:00:28,741 Stage-1 map = 50%,  reduce = 0%
2013-09-18 02:00:29,755 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 48.44 sec
2013-09-18 02:00:30,772 Stage-1 map = 77%,  reduce = 0%, Cumulative CPU 100.32 sec
2013-09-18 02:00:31,784 Stage-1 map = 91%,  reduce = 0%, Cumulative CPU 281.43 sec
2013-09-18 02:00:32,792 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 435.17 sec
2013-09-18 02:00:33,799 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 435.17 sec
2013-09-18 02:00:34,806 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 435.17 sec
2013-09-18 02:00:35,812 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 435.17 sec
2013-09-18 02:00:36,818 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 435.17 sec
2013-09-18 02:00:37,824 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 435.17 sec
2013-09-18 02:00:38,831 Stage-1 map = 100%,  reduce = 15%, Cumulative CPU 435.17 sec
2013-09-18 02:00:39,843 Stage-1 map = 100%,  reduce = 58%, Cumulative CPU 448.52 sec
2013-09-18 02:00:40,850 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 470.07 sec
2013-09-18 02:00:41,857 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 473.43 sec
2013-09-18 02:00:42,864 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 473.43 sec
MapReduce Total cumulative CPU time: 7 minutes 53 seconds 430 msec
Ended Job = job_201309172235_0177
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0178
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:00:45,348 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:00:47,357 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:00:48,363 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:00:49,367 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:00:50,372 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:00:51,376 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:00:52,381 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:00:53,385 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:00:54,390 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.71 sec
2013-09-18 02:00:55,396 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.19 sec
2013-09-18 02:00:56,403 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.19 sec
MapReduce Total cumulative CPU time: 2 seconds 190 msec
Ended Job = job_201309172235_0178
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 473.43 sec   HDFS Read: 876843443 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.19 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 55 seconds 620 msec
OK
Time taken: 52.093 seconds
hive> quit;

times: 3
query: SELECT Title, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND Title != '' GROUP BY Title ORDER BY PageViews DESC LIMIT 10;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_27543@mturlrep13_201309180200_2073175542.txt
hive> SELECT Title, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT DontCountHits != 0 AND NOT Refresh != 0 AND Title != '' GROUP BY Title ORDER BY PageViews DESC LIMIT 10;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0179
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:01:07,075 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:01:16,231 Stage-1 map = 4%,  reduce = 0%
2013-09-18 02:01:17,250 Stage-1 map = 10%,  reduce = 0%
2013-09-18 02:01:19,265 Stage-1 map = 22%,  reduce = 0%
2013-09-18 02:01:20,281 Stage-1 map = 48%,  reduce = 0%
2013-09-18 02:01:21,287 Stage-1 map = 50%,  reduce = 0%
2013-09-18 02:01:22,302 Stage-1 map = 54%,  reduce = 0%, Cumulative CPU 28.59 sec
2013-09-18 02:01:23,344 Stage-1 map = 69%,  reduce = 0%, Cumulative CPU 58.45 sec
2013-09-18 02:01:24,364 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 92.94 sec
2013-09-18 02:01:25,375 Stage-1 map = 93%,  reduce = 0%, Cumulative CPU 322.31 sec
2013-09-18 02:01:26,382 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 437.99 sec
2013-09-18 02:01:27,387 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 437.99 sec
2013-09-18 02:01:28,392 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 437.99 sec
2013-09-18 02:01:29,398 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 437.99 sec
2013-09-18 02:01:30,404 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 437.99 sec
2013-09-18 02:01:31,410 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 437.99 sec
2013-09-18 02:01:32,417 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 437.99 sec
2013-09-18 02:01:33,427 Stage-1 map = 100%,  reduce = 52%, Cumulative CPU 448.33 sec
2013-09-18 02:01:34,433 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 477.42 sec
2013-09-18 02:01:35,440 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 477.42 sec
2013-09-18 02:01:36,447 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 477.42 sec
MapReduce Total cumulative CPU time: 7 minutes 57 seconds 420 msec
Ended Job = job_201309172235_0179
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0180
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:01:39,908 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:01:40,913 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:01:41,919 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:01:42,924 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:01:43,928 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:01:44,933 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:01:46,032 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:01:47,037 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:01:48,042 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.79 sec
2013-09-18 02:01:49,048 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.21 sec
2013-09-18 02:01:50,053 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.21 sec
2013-09-18 02:01:51,058 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.21 sec
MapReduce Total cumulative CPU time: 2 seconds 210 msec
Ended Job = job_201309172235_0180
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 477.42 sec   HDFS Read: 876843443 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.21 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 59 seconds 630 msec
OK
Time taken: 52.895 seconds
hive> quit;

times: 1
query: SELECT URL, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND IsLink != 0 AND NOT IsDownload != 0  GROUP BY URL ORDER BY PageViews DESC LIMIT 1000;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_2326@mturlrep13_201309180201_303775162.txt
hive> SELECT URL, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND IsLink != 0 AND NOT IsDownload != 0  GROUP BY URL ORDER BY PageViews DESC LIMIT 1000;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0181
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:02:08,645 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:02:18,769 Stage-1 map = 1%,  reduce = 0%
2013-09-18 02:02:22,796 Stage-1 map = 5%,  reduce = 0%
2013-09-18 02:02:23,803 Stage-1 map = 6%,  reduce = 0%
2013-09-18 02:02:27,959 Stage-1 map = 24%,  reduce = 0%
2013-09-18 02:02:32,561 Stage-1 map = 44%,  reduce = 0%
2013-09-18 02:02:33,569 Stage-1 map = 54%,  reduce = 0%
2013-09-18 02:02:34,585 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 56.79 sec
2013-09-18 02:02:35,594 Stage-1 map = 62%,  reduce = 0%, Cumulative CPU 80.27 sec
2013-09-18 02:02:38,969 Stage-1 map = 74%,  reduce = 0%, Cumulative CPU 148.88 sec
2013-09-18 02:02:39,974 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 219.04 sec
2013-09-18 02:02:40,981 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 315.44 sec
2013-09-18 02:02:41,987 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 427.42 sec
2013-09-18 02:02:42,993 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 438.91 sec
2013-09-18 02:02:44,000 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 438.91 sec
2013-09-18 02:02:45,006 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 438.91 sec
2013-09-18 02:02:46,014 Stage-1 map = 100%,  reduce = 45%, Cumulative CPU 454.78 sec
2013-09-18 02:02:47,020 Stage-1 map = 100%,  reduce = 55%, Cumulative CPU 454.78 sec
2013-09-18 02:02:48,027 Stage-1 map = 100%,  reduce = 82%, Cumulative CPU 465.13 sec
2013-09-18 02:02:49,033 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 474.77 sec
2013-09-18 02:02:50,040 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 474.77 sec
2013-09-18 02:02:51,046 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 474.77 sec
MapReduce Total cumulative CPU time: 7 minutes 54 seconds 770 msec
Ended Job = job_201309172235_0181
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0182
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:02:54,504 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:02:56,511 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:02:57,516 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:02:58,520 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:02:59,524 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:00,529 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:01,533 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:02,538 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:03,543 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.2 sec
2013-09-18 02:03:04,547 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.2 sec
2013-09-18 02:03:05,553 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.2 sec
MapReduce Total cumulative CPU time: 2 seconds 200 msec
Ended Job = job_201309172235_0182
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 474.77 sec   HDFS Read: 988828092 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.2 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 56 seconds 970 msec
OK
Time taken: 67.483 seconds
hive> quit;

times: 2
query: SELECT URL, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND IsLink != 0 AND NOT IsDownload != 0  GROUP BY URL ORDER BY PageViews DESC LIMIT 1000;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_10101@mturlrep13_201309180203_1414891537.txt
hive> SELECT URL, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND IsLink != 0 AND NOT IsDownload != 0  GROUP BY URL ORDER BY PageViews DESC LIMIT 1000;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0183
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:03:15,331 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:03:25,419 Stage-1 map = 2%,  reduce = 0%
2013-09-18 02:03:26,435 Stage-1 map = 10%,  reduce = 0%
2013-09-18 02:03:28,457 Stage-1 map = 14%,  reduce = 0%
2013-09-18 02:03:29,465 Stage-1 map = 37%,  reduce = 0%
2013-09-18 02:03:30,480 Stage-1 map = 41%,  reduce = 0%, Cumulative CPU 8.3 sec
2013-09-18 02:03:31,488 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 8.3 sec
2013-09-18 02:03:32,496 Stage-1 map = 56%,  reduce = 0%, Cumulative CPU 19.5 sec
2013-09-18 02:03:33,503 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 31.5 sec
2013-09-18 02:03:34,511 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 192.2 sec
2013-09-18 02:03:35,518 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 435.25 sec
2013-09-18 02:03:36,525 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 463.51 sec
2013-09-18 02:03:37,531 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 463.51 sec
2013-09-18 02:03:38,536 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 463.51 sec
2013-09-18 02:03:39,542 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 463.51 sec
2013-09-18 02:03:40,548 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 463.51 sec
2013-09-18 02:03:41,554 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 463.51 sec
2013-09-18 02:03:42,560 Stage-1 map = 100%,  reduce = 24%, Cumulative CPU 463.51 sec
2013-09-18 02:03:43,568 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 487.09 sec
2013-09-18 02:03:44,575 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 500.52 sec
2013-09-18 02:03:45,581 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 500.52 sec
2013-09-18 02:03:46,587 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 500.52 sec
MapReduce Total cumulative CPU time: 8 minutes 20 seconds 520 msec
Ended Job = job_201309172235_0183
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0184
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:03:49,037 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:03:51,046 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:52,051 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:53,056 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:54,061 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:55,066 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:56,071 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:57,076 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:58,080 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.71 sec
2013-09-18 02:03:59,086 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.34 sec
2013-09-18 02:04:00,091 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.34 sec
2013-09-18 02:04:01,096 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.34 sec
MapReduce Total cumulative CPU time: 2 seconds 340 msec
Ended Job = job_201309172235_0184
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 500.52 sec   HDFS Read: 988828092 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.34 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 22 seconds 860 msec
OK
Time taken: 53.704 seconds
hive> quit;

times: 3
query: SELECT URL, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND IsLink != 0 AND NOT IsDownload != 0  GROUP BY URL ORDER BY PageViews DESC LIMIT 1000;
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_17498@mturlrep13_201309180204_309695720.txt
hive> SELECT URL, count(*) AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND IsLink != 0 AND NOT IsDownload != 0  GROUP BY URL ORDER BY PageViews DESC LIMIT 1000;;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0185
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:04:11,776 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:04:20,938 Stage-1 map = 3%,  reduce = 0%
2013-09-18 02:04:21,952 Stage-1 map = 11%,  reduce = 0%
2013-09-18 02:04:23,979 Stage-1 map = 13%,  reduce = 0%
2013-09-18 02:04:24,984 Stage-1 map = 37%,  reduce = 0%
2013-09-18 02:04:25,990 Stage-1 map = 44%,  reduce = 0%
2013-09-18 02:04:28,019 Stage-1 map = 53%,  reduce = 0%, Cumulative CPU 8.73 sec
2013-09-18 02:04:29,048 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 43.13 sec
2013-09-18 02:04:30,071 Stage-1 map = 84%,  reduce = 0%, Cumulative CPU 227.84 sec
2013-09-18 02:04:31,079 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 459.59 sec
2013-09-18 02:04:32,089 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 459.59 sec
2013-09-18 02:04:33,095 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 459.59 sec
2013-09-18 02:04:34,101 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 459.59 sec
2013-09-18 02:04:35,106 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 459.59 sec
2013-09-18 02:04:36,112 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 459.59 sec
2013-09-18 02:04:37,119 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 459.59 sec
2013-09-18 02:04:38,128 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 462.59 sec
2013-09-18 02:04:39,135 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 476.43 sec
2013-09-18 02:04:40,142 Stage-1 map = 100%,  reduce = 88%, Cumulative CPU 490.11 sec
2013-09-18 02:04:41,149 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 496.41 sec
2013-09-18 02:04:42,156 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 496.41 sec
2013-09-18 02:04:43,162 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 496.41 sec
MapReduce Total cumulative CPU time: 8 minutes 16 seconds 410 msec
Ended Job = job_201309172235_0185
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0186
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:04:45,666 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:04:47,674 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:04:48,680 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:04:49,685 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:04:50,689 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:04:51,694 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:04:52,699 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:04:53,703 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:04:54,707 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.7 sec
2013-09-18 02:04:55,712 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.15 sec
2013-09-18 02:04:56,718 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.15 sec
2013-09-18 02:04:57,723 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.15 sec
MapReduce Total cumulative CPU time: 2 seconds 150 msec
Ended Job = job_201309172235_0186
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 496.41 sec   HDFS Read: 988828092 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.15 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 8 minutes 18 seconds 560 msec
OK
Time taken: 54.814 seconds
hive> quit;

times: 1
query: SELECT TraficSourceID, SearchEngineID, AdvEngineID, URL, count(*), if(SearchEngineID     = 0 AND AdvEngineID = 0 , Referer, '') AS SRC  FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND       EventDate <= TIMESTAMP('2013-07-31') AND          NOT Refresh != 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, if(SearchEngineID = 0 AND AdvEngineID = 0 , Referer, ''), URL ORDER BY count(*) DESC LIMIT 1000; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_24933@mturlrep13_201309180205_1377920242.txt
hive> SELECT TraficSourceID, SearchEngineID, AdvEngineID, URL, count(*), if(SearchEngineID     = 0 AND AdvEngineID = 0 , Referer, '') AS SRC  FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND       EventDate <= TIMESTAMP('2013-07-31') AND          NOT Refresh != 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, if(SearchEngineID = 0 AND AdvEngineID = 0 , Referer, ''), URL ORDER BY count(*) DESC LIMIT 1000; ;
FAILED: SemanticException [Error 10128]: Line 1:415 Not yet supported place for UDAF 'count'
hive> quit;

times: 2
query: SELECT TraficSourceID, SearchEngineID, AdvEngineID, URL, count(*), if(SearchEngineID     = 0 AND AdvEngineID = 0 , Referer, '') AS SRC  FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND       EventDate <= TIMESTAMP('2013-07-31') AND          NOT Refresh != 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, if(SearchEngineID = 0 AND AdvEngineID = 0 , Referer, ''), URL ORDER BY count(*) DESC LIMIT 1000; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_25460@mturlrep13_201309180205_1132498226.txt
hive> SELECT TraficSourceID, SearchEngineID, AdvEngineID, URL, count(*), if(SearchEngineID     = 0 AND AdvEngineID = 0 , Referer, '') AS SRC  FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND       EventDate <= TIMESTAMP('2013-07-31') AND          NOT Refresh != 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, if(SearchEngineID = 0 AND AdvEngineID = 0 , Referer, ''), URL ORDER BY count(*) DESC LIMIT 1000; ;
FAILED: SemanticException [Error 10128]: Line 1:415 Not yet supported place for UDAF 'count'
hive> quit;

times: 3
query: SELECT TraficSourceID, SearchEngineID, AdvEngineID, URL, count(*), if(SearchEngineID     = 0 AND AdvEngineID = 0 , Referer, '') AS SRC  FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND       EventDate <= TIMESTAMP('2013-07-31') AND          NOT Refresh != 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, if(SearchEngineID = 0 AND AdvEngineID = 0 , Referer, ''), URL ORDER BY count(*) DESC LIMIT 1000; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_26035@mturlrep13_201309180205_978107887.txt
hive> SELECT TraficSourceID, SearchEngineID, AdvEngineID, URL, count(*), if(SearchEngineID     = 0 AND AdvEngineID = 0 , Referer, '') AS SRC  FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND       EventDate <= TIMESTAMP('2013-07-31') AND          NOT Refresh != 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, if(SearchEngineID = 0 AND AdvEngineID = 0 , Referer, ''), URL ORDER BY count(*) DESC LIMIT 1000; ;
FAILED: SemanticException [Error 10128]: Line 1:415 Not yet supported place for UDAF 'count'
hive> quit;

times: 1
query: SELECT URLHash, EventDate, count(*) AS PageViews FROM hits_100m WHERE   CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND TraficSourceID IN (-1, 6) AND RefererHash =  6202628419148573758  GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 100000; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_26250@mturlrep13_201309180205_637154344.txt
hive> SELECT URLHash, EventDate, count(*) AS PageViews FROM hits_100m WHERE   CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND TraficSourceID IN (-1, 6) AND RefererHash =  6202628419148573758  GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 100000; ;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0187
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:05:41,227 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:05:51,363 Stage-1 map = 5%,  reduce = 0%
2013-09-18 02:05:54,380 Stage-1 map = 16%,  reduce = 0%
2013-09-18 02:05:57,395 Stage-1 map = 27%,  reduce = 0%
2013-09-18 02:05:58,400 Stage-1 map = 30%,  reduce = 0%
2013-09-18 02:06:00,410 Stage-1 map = 42%,  reduce = 0%
2013-09-18 02:06:01,424 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 17.15 sec
2013-09-18 02:06:02,445 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 262.15 sec
2013-09-18 02:06:04,340 Stage-1 map = 47%,  reduce = 0%, Cumulative CPU 262.15 sec
2013-09-18 02:06:05,365 Stage-1 map = 59%,  reduce = 0%, Cumulative CPU 262.15 sec
2013-09-18 02:06:06,379 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 285.59 sec
2013-09-18 02:06:07,390 Stage-1 map = 71%,  reduce = 0%, Cumulative CPU 291.15 sec
2013-09-18 02:06:08,397 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 320.44 sec
2013-09-18 02:06:09,404 Stage-1 map = 97%,  reduce = 0%, Cumulative CPU 364.61 sec
2013-09-18 02:06:10,410 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 373.66 sec
2013-09-18 02:06:11,417 Stage-1 map = 100%,  reduce = 3%, Cumulative CPU 373.66 sec
2013-09-18 02:06:12,424 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 373.66 sec
2013-09-18 02:06:13,433 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 379.8 sec
2013-09-18 02:06:14,439 Stage-1 map = 100%,  reduce = 73%, Cumulative CPU 396.64 sec
2013-09-18 02:06:15,446 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 407.2 sec
2013-09-18 02:06:16,453 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 407.2 sec
2013-09-18 02:06:17,460 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 410.44 sec
2013-09-18 02:06:18,468 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 410.44 sec
MapReduce Total cumulative CPU time: 6 minutes 50 seconds 440 msec
Ended Job = job_201309172235_0187
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0188
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:06:21,966 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:06:22,972 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:06:23,978 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:06:24,984 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:06:25,990 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:06:26,994 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:06:27,999 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:06:29,004 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:06:30,008 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.79 sec
2013-09-18 02:06:31,015 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.41 sec
2013-09-18 02:06:32,021 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.41 sec
2013-09-18 02:06:33,027 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.41 sec
MapReduce Total cumulative CPU time: 2 seconds 410 msec
Ended Job = job_201309172235_0188
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 410.44 sec   HDFS Read: 1326437326 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.41 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 6 minutes 52 seconds 850 msec
OK
Time taken: 62.644 seconds
hive> quit;

times: 2
query: SELECT URLHash, EventDate, count(*) AS PageViews FROM hits_100m WHERE   CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND TraficSourceID IN (-1, 6) AND RefererHash =  6202628419148573758  GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 100000; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_1255@mturlrep13_201309180206_866108725.txt
hive> SELECT URLHash, EventDate, count(*) AS PageViews FROM hits_100m WHERE   CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND TraficSourceID IN (-1, 6) AND RefererHash =  6202628419148573758  GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 100000; ;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0189
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:06:42,810 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:06:53,890 Stage-1 map = 1%,  reduce = 0%
2013-09-18 02:06:54,899 Stage-1 map = 3%,  reduce = 0%
2013-09-18 02:06:55,912 Stage-1 map = 12%,  reduce = 0%
2013-09-18 02:06:56,943 Stage-1 map = 19%,  reduce = 0%
2013-09-18 02:06:57,951 Stage-1 map = 20%,  reduce = 0%
2013-09-18 02:06:58,970 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 9.16 sec
2013-09-18 02:07:00,000 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 83.33 sec
2013-09-18 02:07:01,028 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 218.77 sec
2013-09-18 02:07:02,042 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 399.52 sec
2013-09-18 02:07:03,049 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 425.85 sec
2013-09-18 02:07:04,055 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 425.85 sec
2013-09-18 02:07:05,061 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 425.85 sec
2013-09-18 02:07:06,067 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 425.85 sec
2013-09-18 02:07:07,073 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 425.85 sec
2013-09-18 02:07:08,080 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 425.85 sec
2013-09-18 02:07:09,086 Stage-1 map = 100%,  reduce = 21%, Cumulative CPU 425.85 sec
2013-09-18 02:07:10,096 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 446.81 sec
2013-09-18 02:07:11,102 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 459.98 sec
2013-09-18 02:07:12,109 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 463.17 sec
2013-09-18 02:07:13,117 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 463.17 sec
MapReduce Total cumulative CPU time: 7 minutes 43 seconds 170 msec
Ended Job = job_201309172235_0189
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0190
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:07:15,611 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:07:17,620 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:07:18,626 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:07:19,631 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:07:20,636 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:07:21,641 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:07:22,650 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:07:23,655 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:07:24,660 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.73 sec
2013-09-18 02:07:25,666 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.32 sec
2013-09-18 02:07:26,675 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.32 sec
MapReduce Total cumulative CPU time: 2 seconds 320 msec
Ended Job = job_201309172235_0190
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 463.17 sec   HDFS Read: 1326437326 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.32 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 45 seconds 490 msec
OK
Time taken: 51.812 seconds
hive> quit;

times: 3
query: SELECT URLHash, EventDate, count(*) AS PageViews FROM hits_100m WHERE   CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND TraficSourceID IN (-1, 6) AND RefererHash =  6202628419148573758  GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 100000; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_9097@mturlrep13_201309180207_752390467.txt
hive> SELECT URLHash, EventDate, count(*) AS PageViews FROM hits_100m WHERE   CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND TraficSourceID IN (-1, 6) AND RefererHash =  6202628419148573758  GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 100000; ;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0191
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:07:36,443 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:07:48,547 Stage-1 map = 3%,  reduce = 0%
2013-09-18 02:07:49,583 Stage-1 map = 13%,  reduce = 0%
2013-09-18 02:07:50,619 Stage-1 map = 15%,  reduce = 0%
2013-09-18 02:07:51,643 Stage-1 map = 22%,  reduce = 0%
2013-09-18 02:07:52,690 Stage-1 map = 54%,  reduce = 0%, Cumulative CPU 9.35 sec
2013-09-18 02:07:53,721 Stage-1 map = 62%,  reduce = 0%, Cumulative CPU 50.99 sec
2013-09-18 02:07:54,737 Stage-1 map = 81%,  reduce = 0%, Cumulative CPU 242.49 sec
2013-09-18 02:07:55,745 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 412.77 sec
2013-09-18 02:07:56,751 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 426.23 sec
2013-09-18 02:07:57,758 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 426.23 sec
2013-09-18 02:07:58,763 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 426.23 sec
2013-09-18 02:07:59,770 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 426.23 sec
2013-09-18 02:08:00,776 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 426.23 sec
2013-09-18 02:08:01,781 Stage-1 map = 100%,  reduce = 9%, Cumulative CPU 426.23 sec
2013-09-18 02:08:02,791 Stage-1 map = 100%,  reduce = 27%, Cumulative CPU 433.68 sec
2013-09-18 02:08:03,799 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 444.3 sec
2013-09-18 02:08:04,806 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 460.63 sec
2013-09-18 02:08:05,812 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 464.02 sec
2013-09-18 02:08:06,819 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 464.02 sec
MapReduce Total cumulative CPU time: 7 minutes 44 seconds 20 msec
Ended Job = job_201309172235_0191
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0192
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:08:09,707 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:08:11,717 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.74 sec
2013-09-18 02:08:12,723 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.74 sec
2013-09-18 02:08:13,729 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.74 sec
2013-09-18 02:08:14,734 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.74 sec
2013-09-18 02:08:15,740 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.74 sec
2013-09-18 02:08:16,744 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.74 sec
2013-09-18 02:08:17,749 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.74 sec
2013-09-18 02:08:18,754 Stage-2 map = 100%,  reduce = 33%, Cumulative CPU 0.74 sec
2013-09-18 02:08:19,760 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.29 sec
2013-09-18 02:08:20,766 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.29 sec
2013-09-18 02:08:21,771 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.29 sec
MapReduce Total cumulative CPU time: 2 seconds 290 msec
Ended Job = job_201309172235_0192
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 464.02 sec   HDFS Read: 1326437326 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.29 sec   HDFS Read: 4622 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 46 seconds 310 msec
OK
Time taken: 53.23 seconds
hive> quit;

times: 1
query: SELECT WindowClientWidth, WindowClientHeight, count(*)  AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate       <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND NOT DontCountHits != 0 AND URLHash =  6202628419148573758 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY      PageViews  DESC LIMIT 10000; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_16494@mturlrep13_201309180208_1768463196.txt
hive> SELECT WindowClientWidth, WindowClientHeight, count(*)  AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate       <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND NOT DontCountHits != 0 AND URLHash =  6202628419148573758 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY      PageViews  DESC LIMIT 10000; ;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0193
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:08:39,476 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:08:52,587 Stage-1 map = 2%,  reduce = 0%
2013-09-18 02:08:53,593 Stage-1 map = 3%,  reduce = 0%
2013-09-18 02:08:58,633 Stage-1 map = 18%,  reduce = 0%
2013-09-18 02:08:59,640 Stage-1 map = 22%,  reduce = 0%
2013-09-18 02:09:01,649 Stage-1 map = 37%,  reduce = 0%
2013-09-18 02:09:02,654 Stage-1 map = 38%,  reduce = 0%
2013-09-18 02:09:04,909 Stage-1 map = 39%,  reduce = 0%, Cumulative CPU 242.14 sec
2013-09-18 02:09:05,923 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 243.18 sec
2013-09-18 02:09:06,929 Stage-1 map = 49%,  reduce = 0%, Cumulative CPU 243.18 sec
2013-09-18 02:09:07,941 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 244.77 sec
2013-09-18 02:09:08,962 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 244.77 sec
2013-09-18 02:09:10,398 Stage-1 map = 66%,  reduce = 0%, Cumulative CPU 250.0 sec
2013-09-18 02:09:11,404 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 278.26 sec
2013-09-18 02:09:12,413 Stage-1 map = 76%,  reduce = 0%, Cumulative CPU 278.26 sec
2013-09-18 02:09:13,428 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 289.83 sec
2013-09-18 02:09:14,435 Stage-1 map = 97%,  reduce = 1%, Cumulative CPU 362.43 sec
2013-09-18 02:09:15,442 Stage-1 map = 100%,  reduce = 1%, Cumulative CPU 399.93 sec
2013-09-18 02:09:16,449 Stage-1 map = 100%,  reduce = 4%, Cumulative CPU 399.93 sec
2013-09-18 02:09:17,455 Stage-1 map = 100%,  reduce = 15%, Cumulative CPU 399.93 sec
2013-09-18 02:09:18,463 Stage-1 map = 100%,  reduce = 45%, Cumulative CPU 412.86 sec
2013-09-18 02:09:19,469 Stage-1 map = 100%,  reduce = 63%, Cumulative CPU 419.48 sec
2013-09-18 02:09:20,476 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 419.48 sec
2013-09-18 02:09:21,483 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 434.29 sec
2013-09-18 02:09:22,490 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 437.36 sec
2013-09-18 02:09:23,496 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 437.36 sec
MapReduce Total cumulative CPU time: 7 minutes 17 seconds 360 msec
Ended Job = job_201309172235_0193
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0194
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:09:27,006 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:09:28,011 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:09:29,015 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:09:30,021 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:09:31,024 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:09:32,028 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:09:33,032 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:09:34,037 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:09:35,041 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.7 sec
2013-09-18 02:09:36,047 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.21 sec
2013-09-18 02:09:37,052 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.21 sec
2013-09-18 02:09:38,057 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.21 sec
MapReduce Total cumulative CPU time: 2 seconds 210 msec
Ended Job = job_201309172235_0194
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 437.36 sec   HDFS Read: 1004716597 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.21 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 19 seconds 570 msec
OK
Time taken: 69.068 seconds
hive> quit;

times: 2
query: SELECT WindowClientWidth, WindowClientHeight, count(*)  AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate       <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND NOT DontCountHits != 0 AND URLHash =  6202628419148573758 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY      PageViews  DESC LIMIT 10000; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_24195@mturlrep13_201309180209_485949400.txt
hive> SELECT WindowClientWidth, WindowClientHeight, count(*)  AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate       <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND NOT DontCountHits != 0 AND URLHash =  6202628419148573758 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY      PageViews  DESC LIMIT 10000; ;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0195
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:09:47,707 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:09:58,784 Stage-1 map = 1%,  reduce = 0%
2013-09-18 02:09:59,803 Stage-1 map = 8%,  reduce = 0%
2013-09-18 02:10:00,817 Stage-1 map = 13%,  reduce = 0%
2013-09-18 02:10:01,836 Stage-1 map = 18%,  reduce = 0%
2013-09-18 02:10:02,908 Stage-1 map = 32%,  reduce = 0%, Cumulative CPU 279.26 sec
2013-09-18 02:10:03,925 Stage-1 map = 55%,  reduce = 0%, Cumulative CPU 286.13 sec
2013-09-18 02:10:04,949 Stage-1 map = 62%,  reduce = 0%, Cumulative CPU 292.13 sec
2013-09-18 02:10:05,955 Stage-1 map = 83%,  reduce = 0%, Cumulative CPU 359.01 sec
2013-09-18 02:10:06,963 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 409.34 sec
2013-09-18 02:10:07,970 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 422.22 sec
2013-09-18 02:10:08,976 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 422.22 sec
2013-09-18 02:10:09,981 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 422.22 sec
2013-09-18 02:10:10,987 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 422.22 sec
2013-09-18 02:10:11,993 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 422.22 sec
2013-09-18 02:10:13,002 Stage-1 map = 100%,  reduce = 12%, Cumulative CPU 422.22 sec
2013-09-18 02:10:14,012 Stage-1 map = 100%,  reduce = 30%, Cumulative CPU 425.38 sec
2013-09-18 02:10:15,028 Stage-1 map = 100%,  reduce = 76%, Cumulative CPU 448.26 sec
2013-09-18 02:10:16,035 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 462.58 sec
2013-09-18 02:10:17,041 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 462.58 sec
2013-09-18 02:10:18,047 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 462.58 sec
MapReduce Total cumulative CPU time: 7 minutes 42 seconds 580 msec
Ended Job = job_201309172235_0195
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0196
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:10:22,087 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:10:24,096 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 02:10:25,101 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 02:10:26,106 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 02:10:27,111 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 02:10:28,116 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 02:10:29,121 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 02:10:30,126 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.78 sec
2013-09-18 02:10:31,131 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.25 sec
2013-09-18 02:10:32,137 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.25 sec
2013-09-18 02:10:33,142 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.25 sec
MapReduce Total cumulative CPU time: 2 seconds 250 msec
Ended Job = job_201309172235_0196
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 462.58 sec   HDFS Read: 1004716597 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.25 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 44 seconds 830 msec
OK
Time taken: 53.234 seconds
hive> quit;

times: 3
query: SELECT WindowClientWidth, WindowClientHeight, count(*)  AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate       <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND NOT DontCountHits != 0 AND URLHash =  6202628419148573758 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY      PageViews  DESC LIMIT 10000; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_32545@mturlrep13_201309180210_1142487279.txt
hive> SELECT WindowClientWidth, WindowClientHeight, count(*)  AS PageViews FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate       <= TIMESTAMP('2013-07-31') AND NOT Refresh != 0 AND NOT DontCountHits != 0 AND URLHash =  6202628419148573758 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY      PageViews  DESC LIMIT 10000; ;
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 11
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0197
Hadoop job information for Stage-1: number of mappers: 37; number of reducers: 11
2013-09-18 02:10:43,786 Stage-1 map = 0%,  reduce = 0%
2013-09-18 02:10:53,999 Stage-1 map = 1%,  reduce = 0%
2013-09-18 02:10:55,007 Stage-1 map = 4%,  reduce = 0%
2013-09-18 02:10:56,013 Stage-1 map = 11%,  reduce = 0%
2013-09-18 02:10:57,019 Stage-1 map = 17%,  reduce = 0%
2013-09-18 02:10:58,030 Stage-1 map = 28%,  reduce = 0%
2013-09-18 02:10:59,047 Stage-1 map = 48%,  reduce = 0%
2013-09-18 02:11:00,066 Stage-1 map = 61%,  reduce = 0%, Cumulative CPU 51.25 sec
2013-09-18 02:11:01,076 Stage-1 map = 78%,  reduce = 0%, Cumulative CPU 185.97 sec
2013-09-18 02:11:02,087 Stage-1 map = 99%,  reduce = 0%, Cumulative CPU 391.71 sec
2013-09-18 02:11:03,100 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 432.05 sec
2013-09-18 02:11:04,107 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 432.05 sec
2013-09-18 02:11:05,113 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 432.05 sec
2013-09-18 02:11:06,118 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 432.05 sec
2013-09-18 02:11:07,124 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 432.05 sec
2013-09-18 02:11:08,130 Stage-1 map = 100%,  reduce = 6%, Cumulative CPU 432.05 sec
2013-09-18 02:11:09,140 Stage-1 map = 100%,  reduce = 36%, Cumulative CPU 438.09 sec
2013-09-18 02:11:10,147 Stage-1 map = 100%,  reduce = 70%, Cumulative CPU 452.13 sec
2013-09-18 02:11:11,154 Stage-1 map = 100%,  reduce = 94%, Cumulative CPU 466.84 sec
2013-09-18 02:11:12,160 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 470.1 sec
2013-09-18 02:11:13,166 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 470.1 sec
MapReduce Total cumulative CPU time: 7 minutes 50 seconds 100 msec
Ended Job = job_201309172235_0197
Launching Job 2 out of 2
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Kill Command = /usr/libexec/../bin/hadoop job  -kill job_201309172235_0198
Hadoop job information for Stage-2: number of mappers: 1; number of reducers: 1
2013-09-18 02:11:16,669 Stage-2 map = 0%,  reduce = 0%
2013-09-18 02:11:17,675 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:11:18,680 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:11:19,686 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:11:20,691 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:11:21,696 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:11:22,700 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:11:23,705 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:11:24,711 Stage-2 map = 100%,  reduce = 0%, Cumulative CPU 0.73 sec
2013-09-18 02:11:25,716 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.26 sec
2013-09-18 02:11:26,721 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.26 sec
2013-09-18 02:11:27,727 Stage-2 map = 100%,  reduce = 100%, Cumulative CPU 2.26 sec
MapReduce Total cumulative CPU time: 2 seconds 260 msec
Ended Job = job_201309172235_0198
MapReduce Jobs Launched: 
Job 0: Map: 37  Reduce: 11   Cumulative CPU: 470.1 sec   HDFS Read: 1004716597 HDFS Write: 1056 SUCCESS
Job 1: Map: 1  Reduce: 1   Cumulative CPU: 2.26 sec   HDFS Read: 4633 HDFS Write: 0 SUCCESS
Total MapReduce CPU Time Spent: 7 minutes 52 seconds 360 msec
OK
Time taken: 52.853 seconds
hive> quit;

times: 1
query: SELECT unix_timestamp(EventTime) - SECOND(EventTime), count(*) FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-02') AND NOT Refresh != 0 AND NOT DontCountHits != 0 GROUP BY m ORDER BY m; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_8277@mturlrep13_201309180211_1471622.txt
hive> SELECT unix_timestamp(EventTime) - SECOND(EventTime), count(*) FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-02') AND NOT Refresh != 0 AND NOT DontCountHits != 0 GROUP BY m ORDER BY m; ;
hive> quit;

times: 2
query: SELECT unix_timestamp(EventTime) - SECOND(EventTime), count(*) FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-02') AND NOT Refresh != 0 AND NOT DontCountHits != 0 GROUP BY m ORDER BY m; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_8496@mturlrep13_201309180211_1815785181.txt
hive> SELECT unix_timestamp(EventTime) - SECOND(EventTime), count(*) FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-02') AND NOT Refresh != 0 AND NOT DontCountHits != 0 GROUP BY m ORDER BY m; ;
hive> quit;

times: 3
query: SELECT unix_timestamp(EventTime) - SECOND(EventTime), count(*) FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-02') AND NOT Refresh != 0 AND NOT DontCountHits != 0 GROUP BY m ORDER BY m; 
spawn hive

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j.properties
Hive history file=/tmp/kartavyy/hive_job_log_kartavyy_8703@mturlrep13_201309180211_1151158687.txt
hive> SELECT unix_timestamp(EventTime) - SECOND(EventTime), count(*) FROM hits_100m WHERE CounterID = 34 AND EventDate >= TIMESTAMP('2013-07-01') AND EventDate <= TIMESTAMP('2013-07-02') AND NOT Refresh != 0 AND NOT DontCountHits != 0 GROUP BY m ORDER BY m; ;
hive> quit;
stop time: Ср. сент. 18 02:11:52 MSK 2013