thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-19 06:01:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
18,046 Commits 9,136 Branches 1,778 Tags 2 GiB
8d6a26b8df
Commit Graph

84 Commits

Author SHA1 Message Date
Amos Bird
8d6a26b8df Enable sse2 for CSV parsing. 
Testing data

```
select 'aaaaaaaa,bbbbbbbb,cccccccc,dddddddd,eeeeeeee,ffffffff,gggg,hhh' from numbers(3000000) into outfile '/tmp/test.csv'
```

Testing command
```
echo "select count() from file('/tmp/test.csv', CSV, 'a String, b String, c String, d String, e String, f String, g String, h String') where not ignore(e)" | clickhouse-benchmark

```

Before
```
QPS: 1.317, RPS: 3949749.687, MiB/s: 478.380, result RPS: 1.317, result MiB/s: 0.000.
0.000%  0.704 sec.
10.000% 0.712 sec.
20.000% 0.718 sec.
30.000% 0.726 sec.
40.000% 0.739 sec.
50.000% 0.754 sec.
60.000% 0.770 sec.
70.000% 0.788 sec.
80.000% 0.798 sec.
90.000% 0.815 sec.
95.000% 0.826 sec.
99.000% 0.850 sec.
99.900% 0.857 sec.
99.990% 0.858 sec.
```

After
```
QPS: 1.533, RPS: 4598308.336, MiB/s: 556.932, result RPS: 1.533, result MiB/s: 0.000.
0.000%  0.626 sec.
10.000% 0.635 sec.
20.000% 0.639 sec.
30.000% 0.642 sec.
40.000% 0.643 sec.
50.000% 0.645 sec.
60.000% 0.649 sec.
70.000% 0.652 sec.
80.000% 0.658 sec.
90.000% 0.682 sec.
95.000% 0.710 sec.
99.000% 0.727 sec.
99.900% 0.733 sec.
99.990% 0.734 sec.
```
 2018-08-28 19:46:11 +08:00
Alexey Milovidov
24bd803c07 Whitespaces #2758 2018-08-27 22:06:32 +03:00
alexey-milovidov
98148d914c
Merge pull request #2958 from blahgeek/skip-json-object 
Skip object fields on json input
 2018-08-27 20:07:32 +03:00
alexey-milovidov
01f01b1ee1
Update ReadHelpers.cpp 2018-08-27 17:54:58 +03:00
BlahGeek
739119d355 skip object fields on json input 2018-08-27 12:01:47 +08:00
Amos Bird
9623e59870 Simplify read helpers. 
Use similar APIs like https://en.cppreference.com/w/cpp/string/byte/strstr
 2018-08-26 01:27:28 +08:00
Alexey Milovidov
080b0f7f1f Speed up fuzz tests, part 2 #2924 2018-08-24 10:35:53 +03:00
Alexey Milovidov
6ba5b3bd82 Fixed warning [#CLICKHOUSE-2] 2018-07-16 09:09:27 +03:00
Alexey Milovidov
317807f03d Fixed warning [#CLICKHOUSE-2] 2018-07-16 09:08:39 +03:00
Alexey Milovidov
eee02143a8 Implemented more accurate CAST from String to Nullable of Date and DateTime #1322 2018-07-16 03:30:13 +03:00
Alexey Milovidov
297805f719 Implemented toOrZero, toOrNull functions for Date and DateTime arguments (incomplete) #1322 2018-07-16 03:30:13 +03:00
Alexey Milovidov
7209d8b76f Adaptations #2574 2018-07-05 00:09:58 +03:00
Alexey Milovidov
84fd35da41 Adaptations #2574 2018-07-05 00:00:50 +03:00
Amos Bird
900b046943 Add options to enable parsing CSV with single/double quote. 2018-06-29 15:34:55 +08:00
proller
d97b2edc7c Make separate libs Common/ZooKeeper and Common/ConfigProcessor (#1749) 
* Make separate libs Common/ZooKeeper and Common/ConfigProcessor
renamed:    dbms/src/Common/StringUtils.h -> dbms/src/Common/StringUtils/StringUtils.h

* Fix space

* Fix data in system.build_options

* merge

* merge
 2018-01-15 22:07:47 +03:00
Alexey Milovidov
ce5de79263 Fixed tests #1665 2018-01-14 03:12:23 +03:00
Alexey Milovidov
e300ecd157 Addition to prev. revision #1665 2018-01-13 07:45:13 +03:00
Alexey Milovidov
2bf79be52f Added all methods #1665 2018-01-13 07:43:10 +03:00
Alexey Milovidov
8ff93487e3 Miscellaneous [#CLICKHOUSE-2]. 2017-12-25 07:01:46 +03:00
alexey-milovidov
19e0cd879d
Extended date parsing (#1495) 
* Allowed to parse Date in YYYY-mM-dD format [#CLICKHOUSE-3].

* Allowed to parse Date in YYYY-mM-dD format [#CLICKHOUSE-3].
 2017-11-15 05:08:55 +03:00
Alexey Milovidov
86c46ad1bd Unification [#CLICKHOUSE-2]. 2017-09-15 15:16:12 +03:00
Alexey Milovidov
8b313ab99e Fixed warnings found by cppcheck [#CLICKHOUSE-3300]. 2017-09-08 00:04:48 +03:00
Alexey Milovidov
46b5c9ce03 Miscellaneous [#CLICKHOUSE-2]. 2017-08-10 02:57:54 +03:00
Alexey Milovidov
4f633f316c Removed TODO [#CLICKHOUSE-2]. 2017-08-09 04:38:56 +03:00
Alexey Milovidov
6edb14ba6b Fixed garbage [#CLICKHOUSE-2]. 2017-08-09 04:35:12 +03:00
Alexey Milovidov
1fc8548018 Fixing garbage (incomplete) [#CLICKHOUSE-2]. 2017-08-09 04:35:12 +03:00
Alexey Milovidov
6578fb2645 Fixed error [#CLICKHOUSE-3191]. 2017-08-01 00:39:24 +03:00
Alexey Milovidov
e1cc8289b2 Fixed error after merge [#CLICKHOUSE-3150]. 2017-07-21 09:46:50 +03:00
Guillaume Tassery
57b4066da8 Improve performance for deserialization/serialization of UUID type 2017-07-19 19:26:37 +03:00
Vitaliy Lyudvichenko
b843ea5b3f Resolves #964. Fixed input_format_skip_unknown_fields for negative numbers. [#CLICKHOUSE-3] 2017-07-11 21:36:51 +03:00
Guillaume Tassery
2ec96fe822 Make UUID as a strong type 2017-07-06 16:50:29 +02:00
Guillaume Tassery
aeeb81fadf Rename field in UInt128 2017-07-04 18:10:36 +02:00
Guillaume Tassery
cba9815981 Change Uuid to UUID datatype and make UInt128 a generic type 2017-07-04 12:42:53 +02:00
Guillaume Tassery
d6ab06437a Remove mention of UUID type for dictonnary 2017-07-03 18:08:32 +02:00
Guillaume Tassery
525851ec68 sync master 2017-06-30 22:21:42 +02:00
Guillaume Tassery
1231952fca Implement UUID type and toUuid function 2017-06-30 22:01:27 +02:00
Alexey Milovidov
dbc61a6a43 Added support for SQL-style quoting [#CLICKHOUSE-20]. 2017-06-25 06:43:37 +03:00
Alexey Milovidov
10f5543a49 Revert "Add support for skipping whitespaces" 
This reverts commit 704a79ee6c.
 2017-06-13 09:09:59 +03:00
alexey-milovidov
3f6cccaaa1 Merge pull request #834 from luc1ph3r/Add-performance-test 
Add performance test
 2017-06-13 07:20:01 +03:00
Alexey Milovidov
862a304db2 Small refinements [#CLICKHOUSE-2]. 2017-06-01 16:41:58 +03:00
ivanzhukov
704a79ee6c Add support for skipping whitespaces 2017-05-09 21:53:47 +03:00
Alexey Milovidov
41e97787e3 Less dependencies [#CLICKHOUSE-2]. 2017-04-16 08:40:17 +03:00
Alexey Milovidov
310736cd7e Moved headers and sources to same place [#CLICKHOUSE-3]. 2017-04-01 12:22:42 +03:00
Alexey Milovidov
137ad95929 Changed tabs to spaces in code [#CLICKHOUSE-3]. 2017-04-01 11:35:09 +03:00
f1yegor
bc0d9a2cae translate comments 2017-03-25 21:12:56 +01:00
proller
8b4b1534e3 Cmake: Test on having sse (#405) 
* Cmake: Test on having sse

* fix

* wip

* wip

* wip

* wip

* wip

* wip

* fix

* fix

* fix

* fix

* fix

* fix
 2017-01-27 23:55:33 +04:00
alexey-milovidov
de9bc1dc75 Text formats allow to skip errors (#407) 
* Allow to skip errors in text formats: added settings 'input_format_allow_errors_num' and 'input_format_allow_errors_ratio' [#CLICKHOUSE-2778]. https://github.com/yandex/ClickHouse/issues/134

* Allow to skip errors in text formats: added settings 'input_format_allow_errors_num' and 'input_format_allow_errors_ratio' [#CLICKHOUSE-2778]. https://github.com/yandex/ClickHouse/issues/134

* Allow to skip errors in text formats: added settings 'input_format_allow_errors_num' and 'input_format_allow_errors_ratio' [#CLICKHOUSE-2778]. https://github.com/yandex/ClickHouse/issues/134

* Allow to skip errors in text formats: added settings 'input_format_allow_errors_num' and 'input_format_allow_errors_ratio' [#CLICKHOUSE-2778]. https://github.com/yandex/ClickHouse/issues/134

* Allow to skip errors in text formats: added settings 'input_format_allow_errors_num' and 'input_format_allow_errors_ratio' [#CLICKHOUSE-2778]. https://github.com/yandex/ClickHouse/issues/134

* Allow to skip errors in text formats: added settings 'input_format_allow_errors_num' and 'input_format_allow_errors_ratio' [#CLICKHOUSE-2778]. https://github.com/yandex/ClickHouse/issues/134

* Allow to skip errors in text formats: added settings 'input_format_allow_errors_num' and 'input_format_allow_errors_ratio' [#CLICKHOUSE-2778]. https://github.com/yandex/ClickHouse/issues/134

* Allow to skip errors in text formats: added settings 'input_format_allow_errors_num' and 'input_format_allow_errors_ratio' [#CLICKHOUSE-2778]. https://github.com/yandex/ClickHouse/issues/134

* Allow to skip errors in text formats: added settings 'input_format_allow_errors_num' and 'input_format_allow_errors_ratio' [#CLICKHOUSE-2778]. https://github.com/yandex/ClickHouse/issues/134

* Allow to skip errors in text formats: added settings 'input_format_allow_errors_num' and 'input_format_allow_errors_ratio' [#CLICKHOUSE-2778]. https://github.com/yandex/ClickHouse/issues/134

* Added test [#CLICKHOUSE-2778].
 2017-01-27 08:29:47 +04:00
Alexey Milovidov
641faac912 More correct time zones support; removed support for non-constant time zone argument [#CLICKHOUSE-2765]. 2017-01-22 11:57:34 +03:00
Alexey Milovidov
baf29f5c48 NULLs support: fixed totally wrong code [#METR-19266]. 2016-12-30 08:13:14 +03:00
Vitaliy Lyudvichenko
f3d70eb87c Added test for JSONEachRow format. [#METR-22801] 2016-09-28 21:45:28 +03:00