Commit Graph

9546 Commits

Author SHA1 Message Date
Amos Bird
a589230249 Faster path for varint read.
clickhouse-benchmark <<< 'select count() from s where not ignore(s);'

before this patch:

```

QPS: 0.732, RPS: 2346562049.608, MiB/s: 22378.560, result RPS: 0.732, result MiB/s: 0.000.
0.000%  1.310 sec.
10.000% 1.321 sec.
20.000% 1.327 sec.
30.000% 1.337 sec.
40.000% 1.343 sec.
50.000% 1.359 sec.
60.000% 1.366 sec.
70.000% 1.381 sec.
80.000% 1.400 sec.
90.000% 1.434 sec.
95.000% 1.448 sec.
99.000% 1.489 sec.
99.900% 1.499 sec.
99.990% 1.500 sec.

```

after this patch:

```
QPS: 0.787, RPS: 2524560389.064, MiB/s: 24076.084, result RPS: 0.787, result MiB/s: 0.000.
0.000%  1.228 sec.
10.000% 1.232 sec.
20.000% 1.235 sec.
30.000% 1.241 sec.
40.000% 1.246 sec.
50.000% 1.256 sec.
60.000% 1.265 sec.
70.000% 1.278 sec.
80.000% 1.296 sec.
90.000% 1.321 sec.
95.000% 1.354 sec.
99.000% 1.421 sec.
99.900% 1.453 sec.
99.990% 1.456 sec.
```

I also tried a SSE2 implementation and it's much slower (50%)
2018-08-26 02:23:26 +08:00
alexey-milovidov
732afd27cf
Merge pull request #2953 from yandex/max-block-size-in-system-tables-columns
Support for max_block_size in system.tables and system.columns tables
2018-08-25 20:36:54 +03:00
Amos Bird
9623e59870 Simplify read helpers.
Use similar APIs like https://en.cppreference.com/w/cpp/string/byte/strstr
2018-08-26 01:27:28 +08:00
Alexey Milovidov
aa936b09f9 Added check for max_block_size = 0 #2447 2018-08-25 19:32:44 +03:00
Alexey Milovidov
068e50329e Support for max_block_size in system.tables and system.columns tables #2447 2018-08-25 19:07:48 +03:00
Amos Bird
5f03082af9 Make sure BufferBase is fully swapped. 2018-08-25 19:58:16 +08:00
alexey-milovidov
046137f9c5
Merge pull request #2946 from amosbird/locking
Better locking for StorageBuffer
2018-08-25 13:47:29 +03:00
Amos Bird
2602177525 Better locking for StorageBuffer 2018-08-25 13:52:06 +08:00
Alexey Milovidov
8cc3501c04 Attempt to generalize ZooKeeper interface [#CLICKHOUSE-2] 2018-08-25 05:00:36 +03:00
Alexey Milovidov
3475ef5304 Attempt to generalize ZooKeeper interface [#CLICKHOUSE-2] 2018-08-25 04:58:14 +03:00
Alexey Milovidov
4d343a441c Attempt to generalize ZooKeeper interface [#CLICKHOUSE-2] 2018-08-25 04:14:02 +03:00
Alexey Milovidov
01082d03d2 Adopted logic in ReplicatedMergeTreeBlockOutputStream #2949 2018-08-25 01:55:02 +03:00
Alexey Milovidov
cc56235ad9 Merge remote-tracking branch 'origin/master' into distinguish-between-definitely-lost-and-unknown-requests-in-zookeeper 2018-08-25 01:53:58 +03:00
Alexey Milovidov
2530561086 ZooKeeperImpl: allow to distinguish cases when request was definitely not been processed #2939 2018-08-25 01:50:20 +03:00
alexey-milovidov
7ba62c2e25
Merge pull request #2939 from yandex/fix-potential-data-loss-in-case-of-zk-errors
Fixed potential data loss in case of ZK errors [#CLICKHOUSE-3916]
2018-08-25 01:08:31 +03:00
chertus
53ec40eeee some decimal field improvements CLICKHOUSE-3765 2018-08-24 19:35:00 +03:00
chertus
2e65f4922f extract overflow funcs to arithmeticOverflow.h CLICKHOUSE-3906 2018-08-24 17:20:23 +03:00
chertus
d61614a3d9 fix enums getTypeId() CLICKHOUSE-3765 2018-08-24 15:55:31 +03:00
alexey-milovidov
3466277b0a
Merge pull request #2940 from yandex/better-exception-message-in-case-of-wrong-number-of-arguments-for-function-if
Better exception message in case of wrong number of arguments for function "if" [#CLICKHOUSE-3932]
2018-08-24 15:31:22 +03:00
chertus
54a3c028c5 Merge branch 'master' into decimal 2018-08-24 14:19:41 +03:00
alexey-milovidov
eab95e43d1
Merge pull request #2938 from yandex/speed-up-fuzz-tests-2
Speed up fuzz tests, part 2 #2924
2018-08-24 14:07:35 +03:00
Alexey Milovidov
eee6644a22 Better exception message in case of wrong number of arguments for function "if" [#CLICKHOUSE-3932] 2018-08-24 12:57:42 +03:00
Alexey Milovidov
eec8af52f3 Fixed potential data loss [#CLICKHOUSE-3916] 2018-08-24 11:57:24 +03:00
Alexey Milovidov
86ede3d38c Removed "_dl_addr" from profile when high amount of queries are instantly rejected with exception as in case when "Too many simultaneous queries" [#CLICKHOUSE-3923] 2018-08-24 11:32:33 +03:00
Amos Bird
26903d6e8f Add schema argument to ODBC table func.
Useful for datbases like postgres and greenplum.
2018-08-24 16:18:17 +08:00
Alexey Milovidov
080b0f7f1f Speed up fuzz tests, part 2 #2924 2018-08-24 10:35:53 +03:00
Alexey Milovidov
a2fefa1617 Speed up fuzz tests, part 2 #2924 2018-08-24 10:30:53 +03:00
alexey-milovidov
5637b30556
Merge pull request #2929 from zhang2014/fix/support_asterisk_for_predcate_optimize
ISSUES-863 fix predicate not work for asterisk
2018-08-24 08:53:04 +03:00
Alexey Milovidov
91496574aa Fixed error #2913 2018-08-24 08:45:03 +03:00
Alexey Milovidov
761422da22 Miscellaneous #2913 2018-08-24 08:37:06 +03:00
zhang2014
21528697f8 ISSUES-863 turn off enable_optimize_predicate_expression 2018-08-24 13:30:49 +08:00
Alexey Milovidov
6043ea02de Style #2913 2018-08-24 08:25:00 +03:00
Alexey Milovidov
27d43fb7ff Fixed typo #2913 2018-08-24 08:21:53 +03:00
Alexey Milovidov
f4813a47f8 Whitespaces #2913 2018-08-24 08:20:18 +03:00
zhang2014
bf517d5523 ISSUES-863 fix failure tests 2018-08-24 12:11:32 +08:00
Alexey Milovidov
a55b9623c5 Fixed arrayDistinct function [#CLICKHOUSE-3928] 2018-08-24 05:54:02 +03:00
zhang2014
32cc611f5d ISSUES-863 fix predicate optimizer not work for asterisk 2018-08-24 10:05:43 +08:00
Alexey Milovidov
38d5d8eaad Reverted #2325 #2261 2018-08-24 03:53:09 +03:00
Alexey Milovidov
1437e5cbe3 Fix style [#CLICKHOUSE-2] 2018-08-24 03:07:25 +03:00
alexey-milovidov
e1e6e316d9
Merge pull request #2930 from yandex/fix-empty-parts-vertical-merge
Fix vertical merge of empty parts
2018-08-24 01:47:11 +03:00
alexey-milovidov
c538e78453
Merge pull request #2933 from zlobober/patch-1
More verbose access denied error message
2018-08-24 01:44:51 +03:00
chertus
1467ca5610 decimal field visitors CLICKHOUSE-3765 2018-08-23 22:11:31 +03:00
Max Akhmedov
d5ccd26348
More verbose access denied error message 2018-08-23 21:44:13 +03:00
proller
fe45a62c34 fix 2018-08-23 21:39:21 +03:00
Vadim
1c596db7c0
Update ReplicatedMergeTreeCleanupThread.cpp 2018-08-23 20:19:05 +03:00
proller
380a472a8d Merge remote-tracking branch 'upstream/master' into fix3 2018-08-23 20:16:14 +03:00
Vadim
0bc224ab46
Update StorageReplicatedMergeTree.cpp 2018-08-23 19:40:59 +03:00
Vadim
5a129f5383
Update ReplicatedMergeTreeCleanupThread.cpp 2018-08-23 19:37:28 +03:00
Vadim
380452c933
Update ReplicatedMergeTreeCleanupThread.cpp 2018-08-23 19:36:54 +03:00
Alexey Zatelepin
08f6305dba add missed logging message 2018-08-23 19:33:25 +03:00
Vadim
e7d6bc3dbc
Update ReplicatedMergeTreeCleanupThread.cpp 2018-08-23 19:02:19 +03:00
VadimPE
ce217af78a CLICKHOUSE-3847 fix 2018-08-23 18:58:29 +03:00
Vadim
9271ea7a5f
Update ReplicatedMergeTreeCleanupThread.cpp 2018-08-23 17:20:13 +03:00
Vadim
c1274f9776
Update StorageReplicatedMergeTree.cpp 2018-08-23 17:13:15 +03:00
Vadim
94699b8f6a
Update ReplicatedMergeTreeCleanupThread.cpp 2018-08-23 17:08:24 +03:00
Vadim
234242dd2d
Update ReplicatedMergeTreeCleanupThread.cpp 2018-08-23 17:04:53 +03:00
chertus
cbb80f52c2 own DecimalField for Decimal32/64/128, better field comparison CLICKHOUSE-3765 2018-08-23 17:03:37 +03:00
VadimPE
9beb841406 CLICKHOUSE-3847 fix throw and del log 2018-08-23 16:55:59 +03:00
proller
38347f120d fixes 2018-08-23 15:20:54 +03:00
Alexey Zatelepin
f592aa84d1 fix vertical merge for empty parts [#CLICKHOUSE-3913] 2018-08-23 15:04:30 +03:00
proller
426b8c6130 Merge remote-tracking branch 'upstream/master' into fix3 2018-08-23 14:27:05 +03:00
alexey-milovidov
9e652d8ccf
Merge pull request #2924 from yandex/speed-up-fuzz-tests
Speed up fuzz tests
2018-08-23 05:45:37 +03:00
Alexey Milovidov
451c6cd304 Removed a notice produced by gcc during build [#CLICKHOUSE-2] 2018-08-23 04:42:55 +03:00
Alexey Milovidov
b9778a3764 Speed up fuzz tests [#CLICKHOUSE-2] 2018-08-23 04:31:28 +03:00
Alexey Milovidov
246f1947c5 Improvement #2887 2018-08-23 04:06:21 +03:00
Alexey Milovidov
a2674d46fd Fixed wrong code #2887 2018-08-23 03:49:18 +03:00
Alexey Milovidov
61b8b60f53 Whitespaces [#CLICKHOUSE-3878] 2018-08-23 03:12:24 +03:00
alexey-milovidov
5c1925af58
Merge pull request #2920 from VadimPE/CLICKHOUSE-3819
CLICKHOUSE-3819 add CASE without ELSE
2018-08-22 22:52:24 +03:00
alexey-milovidov
aaf218dab8
Merge pull request #2919 from yandex/low-cardinality-fix-partial-granule-reading
Low cardinality fix partial granule reading
2018-08-22 22:48:42 +03:00
alexey-milovidov
0265cca018
Merge pull request #2921 from alesapin/master
Finally create test with odbc drivers
2018-08-22 22:19:07 +03:00
chertus
c61ccc4dfe toDecimal128() CLICKHOUSE-3905 2018-08-22 20:26:11 +03:00
proller
e40708a1a5 clean 2018-08-22 19:56:00 +03:00
proller
e7d7479ed0 fix 2018-08-22 19:53:11 +03:00
chertus
faafe1852b add some comments CLICKHOUSE-3906 2018-08-22 19:28:02 +03:00
chertus
fd6a8142b7 rename toDecimal functions CLICKHOUSE-3905 2018-08-22 19:16:39 +03:00
alesapin
ba0bf14175 CLICKHOUSE-3878: Fix message 2018-08-22 18:54:08 +03:00
alesapin
e2f18da1a9 CLICKHOUSE-3878: Add ODBC test 2018-08-22 18:42:27 +03:00
VadimPE
9e485dae1e CLICKHOUSE-3819 add CASE without ELSE 2018-08-22 18:17:40 +03:00
Vadim
1f7ec5d595
Update ErrorCodes.cpp 2018-08-22 17:27:13 +03:00
Vadim
22b7c436e5
Update StorageReplicatedMergeTree.cpp 2018-08-22 17:26:37 +03:00
Vadim
62b9b039cd
change exception 2018-08-22 17:25:30 +03:00
Vadim
623c6e0553
Update StorageReplicatedMergeTree.cpp 2018-08-22 17:19:48 +03:00
Vadim
b8dd1cbdf2
Update ReplicatedMergeTreeQueue.cpp 2018-08-22 17:18:05 +03:00
Vadim
58fa299b0a
Update ReplicatedMergeTreeCleanupThread.cpp 2018-08-22 17:14:30 +03:00
Vadim
9bce62719f
Update ReplicatedMergeTreeQueue.cpp 2018-08-22 17:10:00 +03:00
Vadim
984d6b5db8
Update ReplicatedMergeTreeCleanupThread.cpp 2018-08-22 17:06:34 +03:00
proller
f47d93c92f Merge remote-tracking branch 'upstream/master' into fix3 2018-08-22 17:06:19 +03:00
VadimPE
a6ef4632a6 CLICKHOUSE-3847 add ErrorCodes 2018-08-22 17:04:04 +03:00
VadimPE
d81d4dfc70 CLICKHOUSE-3847 change createReplica and change check in cloneReolica() 2018-08-22 17:01:54 +03:00
Alexey Zatelepin
66d49f0abc start and stop mutations_finalizing_task during replica restart [#CLICKHOUSE-3747] 2018-08-22 16:57:30 +03:00
Alexey Zatelepin
b6aa7542d0 add BackgroundSchedulePool::TaskInfo::activateAndSchedule() method
Almost always it (and not activate()) is precisely what is needed.
2018-08-22 16:43:27 +03:00
chertus
314dcc5e9e add two options to disable Decimal overflow checks CLICKHOUSE-3906 2018-08-22 16:22:56 +03:00
proller
d2729f33e6 apple fix 2018-08-22 16:12:20 +03:00
proller
496cb9c2b1 Merge remote-tracking branch 'upstream/master' into fix3 2018-08-22 16:02:23 +03:00
proller
ed5c5083c8 wip 2018-08-22 16:01:50 +03:00
Nikolai Kochetov
b165a53e30 Fix reading from partial granule for low cardinality type. #2896 2018-08-22 15:47:06 +03:00
Alexey Zatelepin
5c748bfdce fix bug when MATERIALIZED columns were not selected during mutation #2900 [#CLICKHOUSE-3911] 2018-08-22 15:45:24 +03:00
proller
b6dabaacb3 Merge remote-tracking branch 'upstream/master' into fix3 2018-08-22 15:22:58 +03:00
Alexey Milovidov
6497d3032e Dummy implementation for non-Linux platforms #2482 2018-08-22 09:00:08 +03:00
Alexey Milovidov
bc30d10b67 Dummy implementation for non-Linux platforms #2482 2018-08-22 08:59:27 +03:00