thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-10-01 14:10:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
18,046 Commits 8,962 Branches 1,767 Tags 1.8 GiB
8d6a26b8df
Commit Graph

631 Commits

Author SHA1 Message Date
Amos Bird
8d6a26b8df Enable sse2 for CSV parsing. 
Testing data

```
select 'aaaaaaaa,bbbbbbbb,cccccccc,dddddddd,eeeeeeee,ffffffff,gggg,hhh' from numbers(3000000) into outfile '/tmp/test.csv'
```

Testing command
```
echo "select count() from file('/tmp/test.csv', CSV, 'a String, b String, c String, d String, e String, f String, g String, h String') where not ignore(e)" | clickhouse-benchmark

```

Before
```
QPS: 1.317, RPS: 3949749.687, MiB/s: 478.380, result RPS: 1.317, result MiB/s: 0.000.
0.000%  0.704 sec.
10.000% 0.712 sec.
20.000% 0.718 sec.
30.000% 0.726 sec.
40.000% 0.739 sec.
50.000% 0.754 sec.
60.000% 0.770 sec.
70.000% 0.788 sec.
80.000% 0.798 sec.
90.000% 0.815 sec.
95.000% 0.826 sec.
99.000% 0.850 sec.
99.900% 0.857 sec.
99.990% 0.858 sec.
```

After
```
QPS: 1.533, RPS: 4598308.336, MiB/s: 556.932, result RPS: 1.533, result MiB/s: 0.000.
0.000%  0.626 sec.
10.000% 0.635 sec.
20.000% 0.639 sec.
30.000% 0.642 sec.
40.000% 0.643 sec.
50.000% 0.645 sec.
60.000% 0.649 sec.
70.000% 0.652 sec.
80.000% 0.658 sec.
90.000% 0.682 sec.
95.000% 0.710 sec.
99.000% 0.727 sec.
99.900% 0.733 sec.
99.990% 0.734 sec.
```
 2018-08-28 19:46:11 +08:00
Alexey Milovidov
24bd803c07 Whitespaces #2758 2018-08-27 22:06:32 +03:00
alexey-milovidov
98148d914c
Merge pull request #2958 from blahgeek/skip-json-object 
Skip object fields on json input
 2018-08-27 20:07:32 +03:00
chertus
ee873c8e9f Merge branch 'master' into decimal 2018-08-27 19:18:48 +03:00
alexey-milovidov
7de0549879
Update ReadHelpers.h 2018-08-27 18:00:08 +03:00
alexey-milovidov
01f01b1ee1
Update ReadHelpers.cpp 2018-08-27 17:54:58 +03:00
BlahGeek
739119d355 skip object fields on json input 2018-08-27 12:01:47 +08:00
Alexey Milovidov
2ce783e016 Merge branch 'read' of https://github.com/amosbird/ClickHouse into amosbird-read 2018-08-26 02:55:22 +03:00
alexey-milovidov
00404a911c
Update VarInt.h 2018-08-26 02:46:18 +03:00
Amos Bird
a589230249 Faster path for varint read. 
clickhouse-benchmark <<< 'select count() from s where not ignore(s);'

before this patch:

```

QPS: 0.732, RPS: 2346562049.608, MiB/s: 22378.560, result RPS: 0.732, result MiB/s: 0.000.
0.000%  1.310 sec.
10.000% 1.321 sec.
20.000% 1.327 sec.
30.000% 1.337 sec.
40.000% 1.343 sec.
50.000% 1.359 sec.
60.000% 1.366 sec.
70.000% 1.381 sec.
80.000% 1.400 sec.
90.000% 1.434 sec.
95.000% 1.448 sec.
99.000% 1.489 sec.
99.900% 1.499 sec.
99.990% 1.500 sec.

```

after this patch:

```
QPS: 0.787, RPS: 2524560389.064, MiB/s: 24076.084, result RPS: 0.787, result MiB/s: 0.000.
0.000%  1.228 sec.
10.000% 1.232 sec.
20.000% 1.235 sec.
30.000% 1.241 sec.
40.000% 1.246 sec.
50.000% 1.256 sec.
60.000% 1.265 sec.
70.000% 1.278 sec.
80.000% 1.296 sec.
90.000% 1.321 sec.
95.000% 1.354 sec.
99.000% 1.421 sec.
99.900% 1.453 sec.
99.990% 1.456 sec.
```

I also tried a SSE2 implementation and it's much slower (50%)
 2018-08-26 02:23:26 +08:00
Amos Bird
9623e59870 Simplify read helpers. 
Use similar APIs like https://en.cppreference.com/w/cpp/string/byte/strstr
 2018-08-26 01:27:28 +08:00
Amos Bird
5f03082af9 Make sure BufferBase is fully swapped. 2018-08-25 19:58:16 +08:00
chertus
54a3c028c5 Merge branch 'master' into decimal 2018-08-24 14:19:41 +03:00
Alexey Milovidov
080b0f7f1f Speed up fuzz tests, part 2 #2924 2018-08-24 10:35:53 +03:00
Alexey Milovidov
a2fefa1617 Speed up fuzz tests, part 2 #2924 2018-08-24 10:30:53 +03:00
Alexey Milovidov
1437e5cbe3 Fix style [#CLICKHOUSE-2] 2018-08-24 03:07:25 +03:00
chertus
1467ca5610 decimal field visitors CLICKHOUSE-3765 2018-08-23 22:11:31 +03:00
proller
b6dabaacb3 Merge remote-tracking branch 'upstream/master' into fix3 2018-08-22 15:22:58 +03:00
Alexey Milovidov
a1d745888d Dummy implementation for non-Linux platforms #2482 2018-08-22 08:56:06 +03:00
Alexey Milovidov
43f1f70c9b Code cleanups #2482 2018-08-22 03:24:55 +03:00
proller
300457da7a fix 2018-08-21 21:35:47 +03:00
proller
fc2925b567 Squashed commit of the following: 
commit 2722e540abfee4a527d716474c4ca582eceeee08
Merge: b4f3af824 aebddd550
Author: proller <proller@github.com>
Date:   Tue Aug 21 18:34:18 2018 +0300

    Merge remote-tracking branch 'upstream/master' into fix3

commit b4f3af824b88a8b6826583bb483730827574e8ad
Author: proller <proller@github.com>
Date:   Tue Aug 21 17:00:20 2018 +0300

    fix

commit 3a18fa7ded2a7d6b7a0883a1b9c2d6b99360837d
Author: proller <proller@github.com>
Date:   Tue Aug 21 16:57:17 2018 +0300

    fix

commit 5d42220c2cf47e1a86bdc73dbfc24b68f8626891
Author: proller <proller@github.com>
Date:   Tue Aug 21 16:50:21 2018 +0300

    freebsd fixes

commit 7baf4fb5ba4575f79b0d12a9ffaaabd34e1006da
Author: proller <proller@github.com>
Date:   Tue Aug 21 16:17:19 2018 +0300

    fix

commit e1fe707fd765b841a8d0952d91a980128cbf91d0
Author: proller <proller@github.com>
Date:   Tue Aug 21 15:35:21 2018 +0300

    fix

commit 027887c71b3ffa98d9473d50d8c3b79cbf3304ac
Author: proller <proller@github.com>
Date:   Tue Aug 21 15:25:57 2018 +0300

    fix

commit 81af41bfcfff7c02fe4060196cf03c2d2aab416e
Author: proller <proller@github.com>
Date:   Tue Aug 21 15:20:41 2018 +0300

    fix

commit 93d572c85d9c7e331254999a614c3b22b5573b02
Author: proller <proller@github.com>
Date:   Tue Aug 21 14:58:07 2018 +0300

    SPLIT_SHARED

commit dd5b8990d8527d59b1e890943d80c2bf27c613ce
Author: proller <proller@github.com>
Date:   Tue Aug 21 14:52:39 2018 +0300

    fix

commit 4840ca12ab752a38c1ef482e8ec59c5859bb48d7
Author: proller <proller@github.com>
Date:   Tue Aug 21 14:46:31 2018 +0300

    fix

commit abde633beb86f2a0d025d6fcf079965dbd827b92
Author: proller <proller@github.com>
Date:   Tue Aug 21 14:25:32 2018 +0300

    fix

commit 0d94a5476a5ba8ba5e88638d58f2cfbf2b4b662d
Author: proller <proller@github.com>
Date:   Tue Aug 21 14:45:18 2018 +0300

    Travis: try fail on ninja fail

commit c2686f90b68255c2beb0a708804aef404e80a6d2
Merge: 2c3427bbd 2aa7eb463
Author: proller <proller@github.com>
Date:   Tue Aug 21 14:25:15 2018 +0300

    Merge remote-tracking branch 'upstream/master' into fix3

commit 2c3427bbdb861edbb188ed4621e8a05cafaedafb
Author: proller <proller@github.com>
Date:   Mon Aug 20 23:57:24 2018 +0300

    fix

commit 530170c6a81d31dcfa81230e48520383234df4bc
Merge: 9abec162c f6e4ec970
Author: proller <proller@github.com>
Date:   Mon Aug 20 23:57:03 2018 +0300

    Merge remote-tracking branch 'upstream/master' into fix3

commit 9abec162cb2e09bbc2f33cbe80fe76791f6e5a77
Author: proller <proller@github.com>
Date:   Mon Aug 20 23:49:58 2018 +0300

    Apple fixes

commit 36d05e8217440fbc8ae21571b06d4eb6d679d538
Author: proller <proller@github.com>
Date:   Mon Aug 20 23:25:05 2018 +0300

    apple fix

commit aeec3e845e4456e89fbb1b1af6f9f36820a46e33
Author: proller <proller@github.com>
Date:   Mon Aug 20 23:20:06 2018 +0300

    fixes

commit 427961d916a5954981e47d94733996deb2a616ce
Author: proller <proller@github.com>
Date:   Mon Aug 20 23:11:11 2018 +0300

    fix

commit a7dd55ff8f653624c0f3dbcbc54defd3b3ae97af
Author: proller <proller@github.com>
Date:   Mon Aug 20 22:41:53 2018 +0300

    fix

commit 6200e0d315c7a62bae63a8de0fc32f7937770ad2
Merge: 8a541d7e6 21cedbe46
Author: proller <proller@github.com>
Date:   Mon Aug 20 22:35:49 2018 +0300

    Merge remote-tracking branch 'upstream/master' into fix3

commit 8a541d7e64c89e2c16af6c909e0353361153aaa3
Author: proller <proller@github.com>
Date:   Mon Aug 20 22:34:32 2018 +0300

    Do not use poco types

commit fd560f43d048b7e3307c6c6b9c9d9918230014d8
Author: proller <proller@github.com>
Date:   Mon Aug 20 22:20:42 2018 +0300

    Try fix apple build

commit cfb2eba07ac06f19e822d3474341d800b1f98cf1
Merge: 8d2e31c90 5b81fdfc0
Author: proller <proller@github.com>
Date:   Mon Aug 20 22:03:24 2018 +0300

    Merge remote-tracking branch 'upstream/master' into fix3

commit 8d2e31c908be2e99d09e2a9dde2414ab82a5e93c
Author: proller <proller@github.com>
Date:   Mon Aug 20 21:56:06 2018 +0300

    travis: TEST_SERVER_STARTUP_WAIT=10
 2018-08-21 18:56:50 +03:00
Alexey Milovidov
f1ba2f9a33 Try to fix "query in form data" feature while still tracking limits for external tables (try №2) #2482 2018-08-20 05:23:35 +03:00
Alexey Milovidov
1802b00c0b Fixed bad arithmetic; miscellaneous #2482 2018-08-19 07:25:53 +03:00
Alexey Milovidov
acb99b49df Merge branch 'master' into CLICKHOUSE-2910 2018-08-14 23:29:42 +03:00
Alexey Milovidov
979154122f Changed strange name #2846 2018-08-14 22:34:25 +03:00
chertus
21fe103831 fix clang build 2018-08-14 22:16:56 +03:00
chertus
15cdd7b2bd make clang happy 2018-08-14 17:19:58 +03:00
alexey-milovidov
f538bd2113
Merge branch 'master' into decimal 2018-08-13 11:57:42 +03:00
Alexey Milovidov
7e50c88538 Added test case [#CLICKHOUSE-3891] 2018-08-13 07:35:12 +03:00
Alexey Milovidov
36db216abf Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
Alex Bocharov
e8d4654419 Remove FormatSettings default value for writeJSONString and add tests for output_format_json_escape_forward_slashes. 2018-08-08 02:35:32 +03:00
alexey-milovidov
0a4d07b995 Update WriteHelpers.h 2018-08-08 02:35:32 +03:00
Alex Bocharov
a8818bb8af Add output_format_json_escape_slashes setting - controls escaping slashes for string outputs in JSON output format. 2018-08-08 02:35:32 +03:00
chertus
de68022fc5 rewrite Decimal with own data type 2018-08-07 16:57:28 +03:00
chertus
c590a74981 Merge branch 'master' into decimal 2018-08-06 13:38:18 +03:00
Alexey Milovidov
e523f9a2c8 Fixed names [#CLICKHOUSE-2] 2018-08-05 11:54:57 +03:00
Alexey Milovidov
1b1754e598 Fixed build #2794 2018-08-04 00:13:54 +03:00
chertus
582f76c83b string to decimal, decimal to string, check decimal bounds 2018-07-30 21:10:38 +03:00
chertus
b9fd87eff7 Merge branch 'master' into decimal 2018-07-27 20:31:33 +03:00
alesapin
6abbfbce87 CLICKHOUSE-3832: Add HTTP Basic authentification in replication protocol 2018-07-26 18:10:57 +03:00
chertus
c1c149d74b decimal comparisons 2018-07-25 22:38:21 +03:00
chertus
02f016579c decimal (de)serialization (in progress) 2018-07-23 23:19:26 +03:00
chertus
5ec47b4a1c decimal (in progress) 2018-07-20 22:05:07 +03:00
Alexey Milovidov
245b114363 Quote all identifiers when querying external database #2635 2018-07-20 03:04:48 +03:00
Alexey Milovidov
a32c2e14a6 Removed unused method that became incompatible [#CLICKHOUSE-2] 2018-07-18 20:16:04 +03:00
alexey-milovidov
4b31f389b7 Update BufferBase.h 2018-07-17 19:41:41 +03:00
Amos Bird
c3c762cde7 Fast path for stringify float points. 2018-07-17 19:41:41 +03:00
Alexey Milovidov
6ba5b3bd82 Fixed warning [#CLICKHOUSE-2] 2018-07-16 09:09:27 +03:00
Alexey Milovidov
317807f03d Fixed warning [#CLICKHOUSE-2] 2018-07-16 09:08:39 +03:00