thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 10:22:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
8d6a26b8df
ClickHouse/dbms/src/IO
History
Amos Bird 8d6a26b8df Enable sse2 for CSV parsing. 
Testing data

```
select 'aaaaaaaa,bbbbbbbb,cccccccc,dddddddd,eeeeeeee,ffffffff,gggg,hhh' from numbers(3000000) into outfile '/tmp/test.csv'
```

Testing command
```
echo "select count() from file('/tmp/test.csv', CSV, 'a String, b String, c String, d String, e String, f String, g String, h String') where not ignore(e)" | clickhouse-benchmark

```

Before
```
QPS: 1.317, RPS: 3949749.687, MiB/s: 478.380, result RPS: 1.317, result MiB/s: 0.000.
0.000%  0.704 sec.
10.000% 0.712 sec.
20.000% 0.718 sec.
30.000% 0.726 sec.
40.000% 0.739 sec.
50.000% 0.754 sec.
60.000% 0.770 sec.
70.000% 0.788 sec.
80.000% 0.798 sec.
90.000% 0.815 sec.
95.000% 0.826 sec.
99.000% 0.850 sec.
99.900% 0.857 sec.
99.990% 0.858 sec.
```

After
```
QPS: 1.533, RPS: 4598308.336, MiB/s: 556.932, result RPS: 1.533, result MiB/s: 0.000.
0.000%  0.626 sec.
10.000% 0.635 sec.
20.000% 0.639 sec.
30.000% 0.642 sec.
40.000% 0.643 sec.
50.000% 0.645 sec.
60.000% 0.649 sec.
70.000% 0.652 sec.
80.000% 0.658 sec.
90.000% 0.682 sec.
95.000% 0.710 sec.
99.000% 0.727 sec.
99.900% 0.733 sec.
99.990% 0.734 sec.
```
2018-08-28 19:46:11 +08:00
..
tests Fix style [#CLICKHOUSE-2] 2018-08-24 03:07:25 +03:00
AIO.cpp Dummy implementation for non-Linux platforms #2482 2018-08-22 08:56:06 +03:00
AIO.h Merge remote-tracking branch 'upstream/master' into fix3 2018-08-22 15:22:58 +03:00
AIOContextPool.cpp Dummy implementation for non-Linux platforms #2482 2018-08-22 08:56:06 +03:00
AIOContextPool.h Dummy implementation for non-Linux platforms #2482 2018-08-22 08:56:06 +03:00
AsynchronousWriteBuffer.h Make sure BufferBase is fully swapped. 2018-08-25 19:58:16 +08:00
BufferBase.h Make sure BufferBase is fully swapped. 2018-08-25 19:58:16 +08:00
BufferWithOwnMemory.h Allocator fix: nothing do when sizes equal, allow reallocate to smaller size (#1512) 2017-11-17 23:27:12 +03:00
CachedCompressedReadBuffer.cpp Fixed potential issue #1890 2018-06-12 06:25:54 +03:00
CachedCompressedReadBuffer.h CLICKHOUSE-3762 Build fixes (#2488) 2018-06-19 21:09:09 +03:00
CascadeWriteBuffer.cpp
CascadeWriteBuffer.h Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
CMakeLists.txt
CompressedReadBuffer.cpp Continued experiments [#CLICKHOUSE-2] 2018-01-16 04:59:51 +03:00
CompressedReadBuffer.h
CompressedReadBufferBase.cpp Update CompressedReadBufferBase.cpp 2018-07-13 06:41:18 +03:00
CompressedReadBufferBase.h Continued experiments [#CLICKHOUSE-2] 2018-01-16 04:59:51 +03:00
CompressedReadBufferFromFile.cpp Continued experiments [#CLICKHOUSE-2] 2018-01-16 04:59:51 +03:00
CompressedReadBufferFromFile.h CLICKHOUSE-3762 Build fixes (#2488) 2018-06-19 21:09:09 +03:00
CompressedStream.h Continued experiments [#CLICKHOUSE-2] 2018-01-16 04:59:51 +03:00
CompressedWriteBuffer.cpp Insignificant changes for PVS-Studio [#CLICKHOUSE-3] 2018-06-04 22:22:27 +03:00
CompressedWriteBuffer.h dbms: Added compression level for ZSTD. [#METR-26742] 2017-10-13 04:08:23 +03:00
CompressionSettings.cpp fix use of uninitialized variable [#CLICKHOUSE-2] 2017-11-03 19:39:06 +03:00
CompressionSettings.h fix use of uninitialized variable [#CLICKHOUSE-2] 2017-11-03 19:39:06 +03:00
ConcatReadBuffer.h
ConnectionTimeouts.h Fixed error: wrong timeouts passed to ConnectionPoolWithFailover [#CLICKHOUSE-3641] 2018-03-12 23:25:18 +03:00
copyData.cpp more honest constness in ActionBlocker 2018-05-25 23:06:36 +03:00
copyData.h more honest constness in ActionBlocker 2018-05-25 23:06:36 +03:00
createReadBufferFromFileBase.cpp Dummy implementation for non-Linux platforms #2482 2018-08-22 08:56:06 +03:00
createReadBufferFromFileBase.h
createWriteBufferFromFileBase.cpp Dummy implementation for non-Linux platforms #2482 2018-08-22 08:56:06 +03:00
createWriteBufferFromFileBase.h
DoubleConverter.h Non significant changes according to clang's -Weverything, part 5 [#CLICKHOUSE-2] 2018-06-04 01:11:50 +03:00
HashingReadBuffer.h Better [#CLICKHOUSE-2]. 2017-12-01 21:36:55 +03:00
HashingWriteBuffer.cpp Unification [#CLICKHOUSE-2]. 2017-09-15 15:16:12 +03:00
HashingWriteBuffer.h Unification [#CLICKHOUSE-2]. 2017-09-15 15:16:12 +03:00
HexWriteBuffer.cpp Attempt to improve performance [#CLICKHOUSE-3]. 2017-07-23 09:53:28 +03:00
HexWriteBuffer.h
HTTPCommon.cpp Fixed error with timeouts [#CLICKHOUSE-2837] 2018-07-10 20:40:17 +03:00
HTTPCommon.h Fixed strange code #2501 2018-06-16 08:54:06 +03:00
InterserverWriteBuffer.cpp Fix on an compiling error 2018-02-09 11:30:08 +08:00
InterserverWriteBuffer.h Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
IReadableWriteBuffer.h
LimitReadBuffer.cpp Try to fix "query in form data" feature while still tracking limits for external tables (try №2) #2482 2018-08-20 05:23:35 +03:00
LimitReadBuffer.h Squashed commit of the following: 2018-08-21 18:56:50 +03:00
LZ4_decompress_faster.cpp Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
LZ4_decompress_faster.h Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
MemoryReadWriteBuffer.cpp Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
MemoryReadWriteBuffer.h Moved code to cpp [#CLICKHOUSE-2]. 2017-04-08 04:32:05 +03:00
MMapReadBufferFromFile.cpp Updated performance testing tool #1890 2018-06-13 05:52:03 +03:00
MMapReadBufferFromFile.h Updated performance testing tool #1890 2018-06-13 05:52:03 +03:00
MMapReadBufferFromFileDescriptor.cpp Updated performance testing tool #1890 2018-06-13 05:52:03 +03:00
MMapReadBufferFromFileDescriptor.h Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
Operators.h Better [#CLICKHOUSE-2]. 2017-12-01 20:49:12 +03:00
parseDateTimeBestEffort.cpp Suppressions for PVS-Studio [#CLICKHOUSE-3] 2018-06-04 18:28:25 +03:00
parseDateTimeBestEffort.h Misc build and test fixes (#1905) 2018-02-14 20:27:14 +03:00
Progress.cpp Removed the possibility to use queries with empty query_id; better semantics #245 2018-03-10 02:04:26 +03:00
Progress.h Removed the possibility to use queries with empty query_id; better semantics #245 2018-03-10 02:04:26 +03:00
ReadBuffer.h More information in 'Cannot read all data' errors 2018-07-13 22:06:26 +03:00
ReadBufferAIO.cpp Make sure BufferBase is fully swapped. 2018-08-25 19:58:16 +08:00
ReadBufferAIO.h Dummy implementation for non-Linux platforms #2482 2018-08-22 08:56:06 +03:00
ReadBufferFromFile.cpp Miscellaneous (no effect) [#CLICKHOUSE-2] 2018-05-09 07:22:30 +03:00
ReadBufferFromFile.h Better [#CLICKHOUSE-2]. 2017-12-01 20:49:12 +03:00
ReadBufferFromFileBase.cpp
ReadBufferFromFileBase.h CLICKHOUSE-3762 Build fixes (#2488) 2018-06-19 21:09:09 +03:00
ReadBufferFromFileDescriptor.cpp Code cleanups #2482 2018-08-22 03:24:55 +03:00
ReadBufferFromFileDescriptor.h libs: compiling with msvc2017 (#1996) 2018-03-23 19:05:14 +03:00
ReadBufferFromIStream.cpp Fixed build #2627 2018-07-11 15:50:23 +03:00
ReadBufferFromIStream.h Update ReadBufferFromIStream.h 2018-07-10 23:46:10 +03:00
ReadBufferFromMemory.h
ReadBufferFromPocoSocket.cpp Merge branch 'master' into CLICKHOUSE-2910 2018-08-14 23:29:42 +03:00
ReadBufferFromPocoSocket.h
ReadBufferFromString.h
readFloatText.cpp Addition to prev. revision #1665 2018-01-13 08:20:18 +03:00
readFloatText.h rewrite Decimal with own data type 2018-08-07 16:57:28 +03:00
ReadHelpers.cpp Enable sse2 for CSV parsing. 2018-08-28 19:46:11 +08:00
ReadHelpers.h Update ReadHelpers.h 2018-08-27 18:00:08 +03:00
ReadWriteBufferFromHTTP.cpp CLICKHOUSE-3832: Add HTTP Basic authentification in replication protocol 2018-07-26 18:10:57 +03:00
ReadWriteBufferFromHTTP.h CLICKHOUSE-3832: Add HTTP Basic authentification in replication protocol 2018-07-26 18:10:57 +03:00
UncompressedCache.h Fixed non-harmful data race [#CLICKHOUSE-2] 2018-03-09 02:07:39 +03:00
VarInt.h Update VarInt.h 2018-08-26 02:46:18 +03:00
WriteBuffer.h Non significant changes according to clang's -Weverything, part 1 [#CLICKHOUSE-2] 2018-06-03 19:51:31 +03:00
WriteBufferAIO.cpp Make sure BufferBase is fully swapped. 2018-08-25 19:58:16 +08:00
WriteBufferAIO.h Dummy implementation for non-Linux platforms #2482 2018-08-22 08:56:06 +03:00
WriteBufferFromArena.h Allow to use values of AggregateFunction data type in groupArray aggregate function [#CLICKHOUSE-3485]. 2017-12-06 06:10:32 +03:00
WriteBufferFromFile.cpp Miscellaneous (no effect) [#CLICKHOUSE-2] 2018-05-09 07:22:30 +03:00
WriteBufferFromFile.h
WriteBufferFromFileBase.cpp
WriteBufferFromFileBase.h
WriteBufferFromFileDescriptor.cpp [WIP] Introduced query_thread_log. [#CLICKHOUSE-2910] 2018-05-31 18:54:08 +03:00
WriteBufferFromFileDescriptor.h
WriteBufferFromHTTP.cpp Fixed strange code #2501 2018-06-16 08:54:06 +03:00
WriteBufferFromHTTP.h Returned back "explicit" #2501 2018-06-16 09:00:51 +03:00
WriteBufferFromHTTPServerResponse.cpp Core/FieldVisitors.h -> Common/FieldVisitors.h 2017-11-24 16:55:31 +03:00
WriteBufferFromHTTPServerResponse.h Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
WriteBufferFromOStream.cpp Update WriteBufferFromOStream.cpp 2018-07-10 23:46:10 +03:00
WriteBufferFromOStream.h Update WriteBufferFromOStream.h 2018-07-10 23:46:10 +03:00
WriteBufferFromPocoSocket.cpp Merge branch 'master' into CLICKHOUSE-2910 2018-08-14 23:29:42 +03:00
WriteBufferFromPocoSocket.h Non significant changes according to clang's -Weverything, part 3 [#CLICKHOUSE-2] 2018-06-03 23:39:06 +03:00
WriteBufferFromString.h Miscellaneous [#CLICKHOUSE-2] 2018-05-16 03:34:56 +03:00
WriteBufferFromTemporaryFile.cpp
WriteBufferFromTemporaryFile.h
WriteBufferFromVector.h fixed several UB-abusing places 2017-11-15 22:22:32 +03:00
WriteBufferValidUTF8.cpp Allow compile on 32bit systems (#1175) 2017-09-01 20:21:03 +03:00
WriteBufferValidUTF8.h
WriteHelpers.cpp Speed up fuzz tests, part 2 #2924 2018-08-24 10:30:53 +03:00
WriteHelpers.h Merge branch 'master' into decimal 2018-08-27 19:18:48 +03:00
WriteIntText.h Changed strange name #2846 2018-08-14 22:34:25 +03:00
ZlibCompressionMethod.h
ZlibDeflatingWriteBuffer.cpp Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
ZlibDeflatingWriteBuffer.h
ZlibInflatingReadBuffer.cpp Preparation for extra warnings [#CLICKHOUSE-2] 2018-08-10 07:02:56 +03:00
ZlibInflatingReadBuffer.h