Commit Graph

96 Commits

Author SHA1 Message Date
alesapin
1909c87885 Fix column description parse. Now creation of replicated MT table with codecs and comments is possible without default 2019-02-27 19:41:51 +03:00
Alexey Milovidov
e150e62040 Grouped includes (40 of 182) 2019-02-10 20:40:52 +03:00
maiha
6c83beafdd dbms: Fixed misspells in comments 2019-01-23 04:56:53 +09:00
Alexey Milovidov
1d4701b0bc Added some warnings from clang's -Weverything 2019-01-04 15:10:00 +03:00
Alexey Milovidov
c77930fe30 Whitespaces [#CLICKHOUSE-2] 2018-11-26 03:56:50 +03:00
Alexey Milovidov
141e9799e4 Fixed error with "extractURLParameter" function (read after buffer); improved performance; added support for zero bytes in URLs; renamed Chars_t type [#CLICKHOUSE-2] 2018-11-25 03:08:50 +03:00
Alexey Milovidov
681929cbda Attempt to fix performance regression while parsing JSONEachRow #3223 2018-10-10 00:32:15 +03:00
Veloman Yunkan
47eb0e28b3 Nested JSON data is mapped to nested table data 2018-09-14 12:15:32 +00:00
Alexey Milovidov
5df5df8971 utils/check-style/check-style -n 2018-08-31 00:13:34 +03:00
alexey-milovidov
e0b1b5ff78
Merge pull request #2980 from amosbird/memcpySmall
Padding for IO buffers.
2018-08-30 22:53:47 +03:00
chertus
7bac9698f6 fix build 2018-08-29 16:24:33 +03:00
Amos Bird
8851fbcab2 Padding for IO buffers.
Testing data

```
select 'aaaaaaaa','bbbbbbbb','cccccccc','dddddddd','eeeeeeee','ffffffff','gggg','hhh' from numbers(3000000) into outfile '/tmp/test.tsv'
```

Testing command
```
echo "select count() from file('/tmp/test.tsv', CSV, 'a String, b String, c String, d String, e String, f String, g String, h String') where not ignore(e)" | clickhouse-benchmark
```

TSV parser has less overhead than CSV, using it would better unveil the benefits of memcpySmall.

Before
```
QPS: 1.662, RPS: 4985463.906, MiB/s: 603.823, result RPS: 1.662, result MiB/s: 0.000.
0.000%  0.559 sec.
10.000% 0.564 sec.
20.000% 0.568 sec.
30.000% 0.572 sec.
40.000% 0.575 sec.
50.000% 0.581 sec.
60.000% 0.592 sec.
70.000% 0.624 sec.
80.000% 0.639 sec.
90.000% 0.664 sec.
95.000% 0.686 sec.
99.000% 0.711 sec.
99.900% 0.715 sec.
99.990% 0.716 sec.
```

After
```
QPS: 1.861, RPS: 5582303.107, MiB/s: 676.110, result RPS: 1.861, result MiB/s: 0.000.
0.000%  0.510 sec.
10.000% 0.514 sec.
20.000% 0.517 sec.
30.000% 0.521 sec.
40.000% 0.523 sec.
50.000% 0.527 sec.
60.000% 0.530 sec.
70.000% 0.539 sec.
80.000% 0.558 sec.
90.000% 0.584 sec.
95.000% 0.589 sec.
99.000% 0.608 sec.
99.900% 0.655 sec.
99.990% 0.663 sec.
```
2018-08-29 09:39:44 +08:00
Amos Bird
8d6a26b8df Enable sse2 for CSV parsing.
Testing data

```
select 'aaaaaaaa,bbbbbbbb,cccccccc,dddddddd,eeeeeeee,ffffffff,gggg,hhh' from numbers(3000000) into outfile '/tmp/test.csv'
```

Testing command
```
echo "select count() from file('/tmp/test.csv', CSV, 'a String, b String, c String, d String, e String, f String, g String, h String') where not ignore(e)" | clickhouse-benchmark

```

Before
```
QPS: 1.317, RPS: 3949749.687, MiB/s: 478.380, result RPS: 1.317, result MiB/s: 0.000.
0.000%  0.704 sec.
10.000% 0.712 sec.
20.000% 0.718 sec.
30.000% 0.726 sec.
40.000% 0.739 sec.
50.000% 0.754 sec.
60.000% 0.770 sec.
70.000% 0.788 sec.
80.000% 0.798 sec.
90.000% 0.815 sec.
95.000% 0.826 sec.
99.000% 0.850 sec.
99.900% 0.857 sec.
99.990% 0.858 sec.
```

After
```
QPS: 1.533, RPS: 4598308.336, MiB/s: 556.932, result RPS: 1.533, result MiB/s: 0.000.
0.000%  0.626 sec.
10.000% 0.635 sec.
20.000% 0.639 sec.
30.000% 0.642 sec.
40.000% 0.643 sec.
50.000% 0.645 sec.
60.000% 0.649 sec.
70.000% 0.652 sec.
80.000% 0.658 sec.
90.000% 0.682 sec.
95.000% 0.710 sec.
99.000% 0.727 sec.
99.900% 0.733 sec.
99.990% 0.734 sec.
```
2018-08-28 19:46:11 +08:00
Alexey Milovidov
24bd803c07 Whitespaces #2758 2018-08-27 22:06:32 +03:00
alexey-milovidov
98148d914c
Merge pull request #2958 from blahgeek/skip-json-object
Skip object fields on json input
2018-08-27 20:07:32 +03:00
alexey-milovidov
01f01b1ee1
Update ReadHelpers.cpp 2018-08-27 17:54:58 +03:00
BlahGeek
739119d355 skip object fields on json input 2018-08-27 12:01:47 +08:00
Amos Bird
9623e59870 Simplify read helpers.
Use similar APIs like https://en.cppreference.com/w/cpp/string/byte/strstr
2018-08-26 01:27:28 +08:00
Alexey Milovidov
080b0f7f1f Speed up fuzz tests, part 2 #2924 2018-08-24 10:35:53 +03:00
Alexey Milovidov
6ba5b3bd82 Fixed warning [#CLICKHOUSE-2] 2018-07-16 09:09:27 +03:00
Alexey Milovidov
317807f03d Fixed warning [#CLICKHOUSE-2] 2018-07-16 09:08:39 +03:00
Alexey Milovidov
eee02143a8 Implemented more accurate CAST from String to Nullable of Date and DateTime #1322 2018-07-16 03:30:13 +03:00
Alexey Milovidov
297805f719 Implemented toOrZero, toOrNull functions for Date and DateTime arguments (incomplete) #1322 2018-07-16 03:30:13 +03:00
Alexey Milovidov
7209d8b76f Adaptations #2574 2018-07-05 00:09:58 +03:00
Alexey Milovidov
84fd35da41 Adaptations #2574 2018-07-05 00:00:50 +03:00
Amos Bird
900b046943 Add options to enable parsing CSV with single/double quote. 2018-06-29 15:34:55 +08:00
proller
d97b2edc7c Make separate libs Common/ZooKeeper and Common/ConfigProcessor (#1749)
* Make separate libs Common/ZooKeeper and Common/ConfigProcessor
renamed:    dbms/src/Common/StringUtils.h -> dbms/src/Common/StringUtils/StringUtils.h

* Fix space

* Fix data in system.build_options

* merge

* merge
2018-01-15 22:07:47 +03:00
Alexey Milovidov
ce5de79263 Fixed tests #1665 2018-01-14 03:12:23 +03:00
Alexey Milovidov
e300ecd157 Addition to prev. revision #1665 2018-01-13 07:45:13 +03:00
Alexey Milovidov
2bf79be52f Added all methods #1665 2018-01-13 07:43:10 +03:00
Alexey Milovidov
8ff93487e3 Miscellaneous [#CLICKHOUSE-2]. 2017-12-25 07:01:46 +03:00
alexey-milovidov
19e0cd879d
Extended date parsing (#1495)
* Allowed to parse Date in YYYY-mM-dD format [#CLICKHOUSE-3].

* Allowed to parse Date in YYYY-mM-dD format [#CLICKHOUSE-3].
2017-11-15 05:08:55 +03:00
Alexey Milovidov
86c46ad1bd Unification [#CLICKHOUSE-2]. 2017-09-15 15:16:12 +03:00
Alexey Milovidov
8b313ab99e Fixed warnings found by cppcheck [#CLICKHOUSE-3300]. 2017-09-08 00:04:48 +03:00
Alexey Milovidov
46b5c9ce03 Miscellaneous [#CLICKHOUSE-2]. 2017-08-10 02:57:54 +03:00
Alexey Milovidov
4f633f316c Removed TODO [#CLICKHOUSE-2]. 2017-08-09 04:38:56 +03:00
Alexey Milovidov
6edb14ba6b Fixed garbage [#CLICKHOUSE-2]. 2017-08-09 04:35:12 +03:00
Alexey Milovidov
1fc8548018 Fixing garbage (incomplete) [#CLICKHOUSE-2]. 2017-08-09 04:35:12 +03:00
Alexey Milovidov
6578fb2645 Fixed error [#CLICKHOUSE-3191]. 2017-08-01 00:39:24 +03:00
Alexey Milovidov
e1cc8289b2 Fixed error after merge [#CLICKHOUSE-3150]. 2017-07-21 09:46:50 +03:00
Guillaume Tassery
57b4066da8 Improve performance for deserialization/serialization of UUID type 2017-07-19 19:26:37 +03:00
Vitaliy Lyudvichenko
b843ea5b3f Resolves #964. Fixed input_format_skip_unknown_fields for negative numbers. [#CLICKHOUSE-3] 2017-07-11 21:36:51 +03:00
Guillaume Tassery
2ec96fe822 Make UUID as a strong type 2017-07-06 16:50:29 +02:00
Guillaume Tassery
aeeb81fadf Rename field in UInt128 2017-07-04 18:10:36 +02:00
Guillaume Tassery
cba9815981 Change Uuid to UUID datatype and make UInt128 a generic type 2017-07-04 12:42:53 +02:00
Guillaume Tassery
d6ab06437a Remove mention of UUID type for dictonnary 2017-07-03 18:08:32 +02:00
Guillaume Tassery
525851ec68 sync master 2017-06-30 22:21:42 +02:00
Guillaume Tassery
1231952fca Implement UUID type and toUuid function 2017-06-30 22:01:27 +02:00
Alexey Milovidov
dbc61a6a43 Added support for SQL-style quoting [#CLICKHOUSE-20]. 2017-06-25 06:43:37 +03:00
Alexey Milovidov
10f5543a49 Revert "Add support for skipping whitespaces"
This reverts commit 704a79ee6c.
2017-06-13 09:09:59 +03:00