Commit Graph

624 Commits

Author SHA1 Message Date
Ivan Lezhankin
d1ead22073 Rename As<…> → as<…> 2019-03-11 16:22:51 +03:00
Ivan Lezhankin
1e9f7847dc Merge remote-tracking branch 'upstream/master' into iast-as 2019-03-11 16:11:22 +03:00
Ivan Lezhankin
a7e838e328 Replace all other casts to method IAST::As<Derived>
Also remove some redundant methods like IsIdentifier() et. al.
2019-03-11 15:49:39 +03:00
alexey-milovidov
edacebac21
Merge pull request #4634 from kirillDanshin/feature/topK-loadFactor
topK and topKWeighted now supports custom loadFactor
2019-03-11 03:41:55 +03:00
alexey-milovidov
86e7889622
Update AggregateFunctionTopK.h 2019-03-11 00:33:57 +03:00
alexey-milovidov
a16c16152d
Update AggregateFunctionTopK.cpp 2019-03-11 00:32:42 +03:00
Alexey Milovidov
ac4045c55d Fixed UB in topKWeighted aggregate function #4634 2019-03-11 00:31:10 +03:00
alexey-milovidov
6db73152d2
Hardening debug build (experimental) (#4632)
* Hardening debug build: more granular memory mappings and ASLR; add memory protection for mark cache and index

* Addition to prev. revision

* Addition to prev. revision

* Addition to prev. revision
2019-03-10 06:16:51 +03:00
Kirill Danshin
698c022a52
topK and topKWeighted now supports custom loadFactor
Signed-off-by: Kirill Danshin <kirill@danshin.pro>
2019-03-09 08:27:40 +03:00
Ivan Lezhankin
0e325c7a4a Replace const typeid_casts with As<Derived>() method invocations 2019-03-08 12:48:20 +03:00
Alexey Milovidov
9d89cc0fc1 Debugging PODArray 2019-03-08 06:23:02 +03:00
proller
89014b5480
Build fixes (#4591) 2019-03-05 13:15:47 +03:00
Vitaly Baranov
6edec1c63f Remove unnecessary 'this->'. 2019-03-04 22:46:05 +03:00
Vitaly Baranov
405a747ddc Fix compilation after changing SmallTable. 2019-03-04 22:45:51 +03:00
Vitaly Baranov
61e21d50e1
Merge pull request #4207 from andyyzh/bitmap_feature
Added bitmap function feature with roaring bitmap
2019-03-04 22:42:47 +03:00
Alexander Kozhikhov
6371b4021d changes in gradient_computer class 2019-03-03 13:21:12 +03:00
quid
3fa972e6b6 changes in weights updater 2019-03-03 11:46:36 +03:00
Alexey Milovidov
51ca51aa9e Merge branch 'master' into quantile-timing-ubsan-fix 2019-03-01 17:53:04 +03:00
Amos Bird
26ab5dd7a7 A Proper lookup table that uses HashTable's API
This is the first step of allowing heterogeneous cells in hash tables.

performance test results are

```

1. HashMap<UInt16, UInt8, TrivialHash, HashTableFixedGrower<16>>;
2. NewLookupMap<UInt16, UInt8>

ResolutionWidth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................223550276.46
ResolutionWidth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................248772721.24
Best: 2 - 24877272124

ResolutionWidth 100000 1 ..........................................................................................................................................................................................................................................................238498413.99
ResolutionWidth 100000 2 ..........................................................................................................................................................................................................................................................261808889.98
Best: 2 - 26180888998

ResolutionWidth 300000 1 ...................................................................................239307348.81
ResolutionWidth 300000 2 ...................................................................................257592761.30
Best: 2 - 25759276130

ResolutionWidth 1000000 1 .........................240144759.26
ResolutionWidth 1000000 2 .........................257093531.91
Best: 2 - 25709353191

ResolutionWidth 5000000 1 .....241573260.35
ResolutionWidth 5000000 2 .....259314162.79
Best: 2 - 25931416279

ResolutionDepth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................217108119.84
ResolutionDepth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................249459504.41
Best: 2 - 24945950441

ResolutionDepth 100000 1 ..........................................................................................................................................................................................................................................................229065162.17
ResolutionDepth 100000 2 ..........................................................................................................................................................................................................................................................253769105.64
Best: 2 - 25376910564

ResolutionDepth 300000 1 ...................................................................................233079225.18
ResolutionDepth 300000 2 ...................................................................................256316273.78
Best: 2 - 25631627378

ResolutionDepth 1000000 1 .........................234184633.51
ResolutionDepth 1000000 2 .........................261100491.57
Best: 2 - 26110049157

ResolutionDepth 5000000 1 .....233118795.66
ResolutionDepth 5000000 2 .....252436160.41
Best: 2 - 25243616041

```
2019-03-01 16:47:13 +08:00
Alexey Milovidov
610f9523b0 Addition to prev. revision 2019-02-26 22:34:32 +03:00
Masha
67b28c2240 Nesterov and Adam + tests 2019-02-26 08:12:16 +00:00
Alexey Milovidov
f02a834615 Fixed fuzz test in UBSan 2019-02-25 21:03:52 +03:00
Masha
e708983e9b Changed Linear Regression and test 2019-02-15 23:38:13 +00:00
Masha
d5753636bd Logistic Regression and new tests 2019-02-15 22:47:56 +00:00
Andy Yang
4d42a297ae Adjust to the latest IAggregateFunctionDataHelper 2019-02-14 19:51:13 +08:00
Andy Yang
4a4dbcd2c1 Merge remote-tracking branch 'origin' into bitmap_feature 2019-02-14 15:14:18 +08:00
Alexey Milovidov
a0b91077e7 Fixed -Wshadow-field-in-constructor (clang 8 warning) 2019-02-13 15:53:36 +03:00
alexander kozhikhov
b23cd21ce1 predict correctly using scalar aggr state 2019-02-13 01:33:37 +03:00
alexander kozhikhov
6a8542c8f6 constructor changes change 2019-02-13 00:18:27 +03:00
alexander kozhikhov
aedf4925dc Merge branch 'master' into ml_methods 2019-02-12 23:04:25 +03:00
alexander kozhikhov
f20ad204e7 Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-02-12 23:00:47 +03:00
Nikolai Kochetov
ec6a5590fd Fix build. 2019-02-12 12:31:20 +03:00
Nikolai Kochetov
2b8b342ccd Update IAggregateFunction interface. 2019-02-11 22:26:32 +03:00
Nikolai Kochetov
c35439a5f0 Updated FieldVisitor. 2019-02-11 18:46:19 +03:00
alexander kozhikhov
fc4c721fa5 some review fixes 2019-02-11 01:07:47 +03:00
alexander kozhikhov
a948f223bc Merge branch 'master' of https://github.com/Quid37/ClickHouse 2019-02-11 00:17:14 +03:00
alexander kozhikhov
bfccafef49 small test code 2019-02-11 00:16:16 +03:00
Alexey Milovidov
7ddede66db Added support for generic case for aggregate function topKWeighted; fixed errors #4245 2019-02-11 00:01:26 +03:00
Alexey Milovidov
ec5fbce404 Merge branch 'add_topkweighed' of https://github.com/andrewgolman/ClickHouse into andrewgolman-add_topkweighed 2019-02-10 23:31:17 +03:00
Alexey Milovidov
e150e62040 Grouped includes (40 of 182) 2019-02-10 20:40:52 +03:00
Alexey Milovidov
ba3cb83a84 Removed duplicate includes 2019-02-10 20:12:22 +03:00
alexander kozhikhov
7cea77b8c0 Merge branch 'master' of https://github.com/Quid37/ClickHouse into ml_methods 2019-02-10 00:19:08 +03:00
alexander kozhikhov
be17c6fab6 Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-02-10 00:10:33 +03:00
Alexey Milovidov
7d86f2debe Added comments 2019-02-09 23:35:55 +03:00
Alexey Milovidov
bf95f19655 Fixed entropy aggregate function #4238 2019-02-09 23:17:20 +03:00
Andy Yang
1a28ba01c1 Added bitmap function feature, fixed test errors and code styles 2019-02-09 14:33:09 +08:00
Alexey Milovidov
c60db4b978 Better code 2019-02-07 00:47:31 +03:00
alexey-milovidov
2216250a84
Merge pull request #4238 from Quid37/yandex_open_code_competition
Implement Shannon entropy aggregate function
2019-02-04 18:43:29 +03:00
andrewsg
31f2f9ce2f Add topKWeighed aggregate function 2019-02-03 13:59:27 +03:00
alexander kozhikhov
c5924d8bfb review fixes 2019-02-02 18:19:59 +03:00
Danila Kutenin
5e62a0825a produce hints for typo functions and types 2019-02-02 17:54:50 +03:00
alexander kozhikhov
8c2726b77c Aggregate function for entropy 2019-02-02 17:27:43 +03:00
Andy Yang
7e7a864fd4 Added bitmap function feature with roaring bitmap 2019-01-31 21:26:11 +08:00
Masha
39f1d9756e fixed name Logistic 2019-01-28 11:54:55 +00:00
Masha
286fa25ad9 logreg and momentum 2019-01-28 10:39:57 +00:00
alexander kozhikhov
57cd47a194 LinearModelData completed 2019-01-26 15:38:42 +03:00
Léo Ercolanelli
038a48bb38 sumMap: implement sumMapWithOverflow 2019-01-25 20:35:53 +01:00
Léo Ercolanelli
57104f81da rename widen to promote for IDataType 2019-01-25 15:16:23 +01:00
Léo Ercolanelli
0d6094a3ea sumMap: return types less prone to oveflows
It used to be that sumMap would return the same type as the values
columns. If columns of Array(UInt8) were to be given, that would really
easily cause oveflow. It now uses `getWidenDataType` (and ultimately
`NearestFieldType`) in order to define the result type.
2019-01-25 14:29:45 +01:00
alexey-milovidov
2626a2614c
Merge pull request #4145 from yandex/fix-fuzz-test-in-ubsan-quantile-weighted
Fix fuzz test in ubsan: added parameter type check for quantile*Weighted.
2019-01-25 02:45:07 +03:00
Alexey Milovidov
ef50601b5c Removed non-breaking spaces: find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l $'\xc2\xa0' | xargs sed -i -r -e 's/\xc2\xa0/ /g' 2019-01-25 02:01:56 +03:00
Alexey Milovidov
5ccc42fd96 Revert "Removed non-breaking spaces: find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l $'\xc2\xa0' | xargs sed -i -r -e 's/\xc2\xa0/ /'"
This reverts commit a28f0acf09.
2019-01-25 01:59:45 +03:00
Alexey Milovidov
a28f0acf09 Removed non-breaking spaces: find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l $'\xc2\xa0' | xargs sed -i -r -e 's/\xc2\xa0/ /' 2019-01-24 22:49:36 +03:00
Alexey Milovidov
ffd18163de Fixed fuzz test in UBSan: added parameter check for quantile*Weighted function 2019-01-24 21:33:57 +03:00
alexey-milovidov
fbefc99fb1
Merge pull request #4129 from ercolanelli-leo/sumMapFiltered
implement sumMapFiltered
2019-01-24 21:27:03 +03:00
Maxim Kuznetsov
73d83d0ab6 Added LinearModelData 2019-01-24 17:22:35 +03:00
Léo Ercolanelli
ba1f1c7460 sumMapFiltered: use an unordered_set 2019-01-24 13:32:30 +01:00
Léo Ercolanelli
90afad8c00 sumMapFiltered: improve error reporting on parameter type mismatch 2019-01-24 13:32:30 +01:00
Léo Ercolanelli
a8647761ce sumMapFiltered: fix dangling reference 2019-01-24 13:32:30 +01:00
Léo Ercolanelli
22f7bd03b9 sumMapFiltered: use CRTP instead of virtual for keepKey 2019-01-24 09:02:17 +01:00
Léo Ercolanelli
efb1cb0528 sumMapFiltered: remove this-> in constructor 2019-01-24 08:58:57 +01:00
alexander kozhikhov
8472048328 Merge branch 'master' into alexkoja_ML 2019-01-23 21:07:05 +03:00
alexander kozhikhov
b229498203 mini-batches 2019-01-23 21:03:26 +03:00
Masha
b9972f8e67 code style AggregateFunctionMLMethod.cpp 2019-01-23 14:53:50 +00:00
Léo Ercolanelli
8ad1a55f3b implement sumMapFiltered 2019-01-23 15:23:04 +01:00
Maxim Kuznetsov
fd1beddb6d code style 2019-01-23 15:11:17 +03:00
Maxim Kuznetsov
03deb677b9 Code style 2019-01-23 14:58:05 +03:00
alexander kozhikhov
fd8f9c4fba changes after review 2019-01-23 04:29:53 +03:00
alexander kozhikhov
ffd0d9af26 Merge branch 'alexkoja_ML' 2019-01-23 00:31:53 +03:00
alexander kozhikhov
61bb3b8ade simple linear regression 2019-01-23 00:07:05 +03:00
maiha
6c83beafdd dbms: Fixed misspells in comments 2019-01-23 04:56:53 +09:00
Alexey Milovidov
c7b95b5175 Miscellaneous changes for PVS-Studio 2019-01-20 02:27:52 +03:00
Alexey Milovidov
61a59956b3 Indentation #4004 2019-01-16 20:25:06 +03:00
Alexey Milovidov
23fbf18555 Merge branch 'sequence-max-iterations' of https://github.com/ercolanelli-leo/ClickHouse into ercolanelli-leo-sequence-max-iterations 2019-01-16 19:37:51 +03:00
Léo Ercolanelli
043ef82468 fix style in AggregateFunctionSequenceMatch.h 2019-01-16 17:20:29 +01:00
Alexey Milovidov
bf2ca149c9 Merge branch 'sequence-max-iterations' of https://github.com/ercolanelli-leo/ClickHouse into ercolanelli-leo-sequence-max-iterations 2019-01-16 18:22:35 +03:00
Léo Ercolanelli
6048a83462 add comment to sequenceMatch's DFA implementation 2019-01-16 10:39:35 +01:00
Léo Ercolanelli
04651503d6 sequence: implement DFA matching algorithm 2019-01-16 10:39:35 +01:00
Alexey Milovidov
ad586e9bd2 Miscellaneous #4000 2019-01-08 18:57:15 +03:00
Alexey Milovidov
a594293b50 Addition to Amos Bird changes #3920 2019-01-08 17:55:37 +03:00
alexey-milovidov
2884b870b3
Merge pull request #3993 from yandex/weverything-2
Added more warnings from clang's -Weverything, that are available in clang 8
2019-01-08 17:54:43 +03:00
Alexey Milovidov
0630e67a21 Addition to Amos Bird changes #3920 2019-01-08 13:07:33 +03:00
Alexey Milovidov
0dfa553fca Added more warnings from clang's -Weverything, that are available in clang 8 2019-01-07 13:40:58 +03:00
Alexey Milovidov
4622f5dc7c Added some warnings from clang's -Weverything 2019-01-04 16:32:08 +03:00
Alexey Milovidov
1d4701b0bc Added some warnings from clang's -Weverything 2019-01-04 15:10:00 +03:00
Alexey Milovidov
654e530bf1 Merge branch 'master' into fix-error-in-quantile-t-digest-implementation 2018-12-28 03:08:04 +03:00
Alexey Milovidov
353c226893 Fixed UB #3926 2018-12-27 04:56:16 +03:00
Alexey Milovidov
4c35a73a74 Fixed UB #3926 2018-12-27 04:03:16 +03:00
Alexey Milovidov
fec30acf81 Allowed some UB #3926 2018-12-27 03:51:14 +03:00
Alexey Milovidov
5570ddada5 Fixed error in internal implementation of quantileTDigest [#CLICKHOUSE-2] 2018-12-27 00:02:39 +03:00
Alexey Milovidov
4a45db2b9b Fixed UB #3926 2018-12-26 07:40:16 +03:00
Alexey Milovidov
fc39ce8673 Fixed UB #3926 2018-12-26 07:12:14 +03:00
Alexey Milovidov
0ba7c1dd3a Fixed UB #3926 2018-12-26 06:32:18 +03:00
Alexey Milovidov
665fc23f57 Fixed UB #3569 2018-12-26 06:27:26 +03:00
Alexey Milovidov
b1b1c676c9 Fixed error #3926 2018-12-26 03:30:11 +03:00
Alexey Milovidov
cfc1e207ab Fixed UB #3926 2018-12-26 02:27:52 +03:00
Amos Bird
8736a3908f left pad 2018-12-24 23:51:06 +08:00
Alexey Milovidov
1841634ae2 Removed accidential change [#CLICKHOUSE-2] 2018-12-20 20:49:37 +03:00
Alexey Milovidov
edc415857c Removed accidential change [#CLICKHOUSE-2] 2018-12-20 20:46:31 +03:00
Alexey Milovidov
731d76821d Merging abandoned pull request with "boundingRatio" function #3139 2018-12-20 18:14:32 +03:00
Alexey Milovidov
113ff56384 Merge branch 'master' into twosixlabs-add_rate_function 2018-12-20 17:19:06 +03:00
alexey-milovidov
6d27777b19
Merge branch 'master' into ubsan-fixes-3 2018-12-14 20:52:54 +03:00
Alexey Milovidov
f3c72daf50 Fixed name [#CLICKHOUSE-2] 2018-12-07 06:21:17 +03:00
Alexey Milovidov
c77930fe30 Whitespaces [#CLICKHOUSE-2] 2018-11-26 03:56:50 +03:00
alexey-milovidov
1002ed7487
Merge branch 'master' into 3958/many_templates 2018-11-23 01:48:42 +03:00
alexey-milovidov
2c2d75cceb
Don't allow exceptions without code (#3645)
* Don't allow exceptions without code #3633

* Don't allow exceptions without code #3633
2018-11-23 00:19:58 +03:00
proller
1d40c2c871 Better link and deps (#3631)
* try fix 1

* Better deps

* fix

* fix

* fix

* Update CMakeLists.txt

* Update CMakeLists.txt

* fix

* fix

* fix

* disable old

* Update find_odbc.cmake
2018-11-21 22:20:27 +03:00
Alexey Milovidov
78e72a1962 Little better #3624 2018-11-20 23:09:20 +03:00
alexey-milovidov
de5d4b6256
Update AggregateFunctionQuantile.h 2018-11-19 21:25:57 +03:00
Ivan Lezhankin
b63f0a5a9f Minor refactoring 2018-11-19 17:25:42 +03:00
Ivan Lezhankin
fc272ecded Some fixes to quantile*() functions related to DateTime argument type 2018-11-14 18:04:23 +03:00
Alexey Milovidov
accda8bed2 Fixed some code (incomplete) #3139 2018-11-01 20:55:11 +03:00
Alexey Milovidov
dd3f57dd5d Fixed some code (incomplete) #3139 2018-11-01 20:54:53 +03:00
Alexey Milovidov
4a5fd44da2 Merge branch 'add_rate_function' of https://github.com/twosixlabs/ClickHouse into twosixlabs-add_rate_function 2018-11-01 20:41:32 +03:00
Ivan Lezhankin
0592081f2f Reduce maximum allowable size of |HashSet|. 2018-10-25 16:17:29 +03:00
Ivan Lezhankin
aa6a69b635 Add TODO comment 2018-10-24 17:28:23 +03:00
Ivan Lezhankin
69ebaaeed6 Merge branch '3958/many_templates' of github.com:abyss7/ClickHouse into 3958/many_templates 2018-10-24 17:24:52 +03:00
Ivan Lezhankin
9296ef5008 Temporarily disable precalculated bias estimations for precisions other than 17. 2018-10-24 17:23:57 +03:00
Amos Bird
3cf2177268 Flexible fields assignment
Field f;

before:

f = Int64(-42);

after:

f = -42;
2018-10-24 08:47:33 +08:00
alexey-milovidov
fd86a6bea5
Update AggregateFunctionUniqCombined.cpp 2018-10-22 22:38:46 +03:00
alexey-milovidov
bd148bb082
Update AggregateFunctionUniqCombined.cpp 2018-10-22 22:37:55 +03:00
alexey-milovidov
f90783d885
Update AggregateFunctionUniqCombined.cpp 2018-10-22 22:36:32 +03:00
alexey-milovidov
c8868cffaa
Update AggregateFunctionUniqCombined.cpp 2018-10-22 22:34:39 +03:00
Ivan Lezhankin
2a53716043 Move the precision template argument one level up. 2018-10-22 20:18:08 +03:00
Ivan Lezhankin
19ed7b6ae9 Minor optimizations 2018-10-22 13:00:37 +03:00
proller
abfe675097
Build fixes (#3419) 2018-10-18 22:28:05 +03:00
Ivan Lezhankin
28ea773b9e Use a single "place" for a default precision constant.
Also update stateless tests.
2018-10-18 21:38:45 +03:00
Ivan Lezhankin
45e6fd8cd2 Fix compilation with gcc and code style 2018-10-18 18:25:39 +03:00
Ivan Lezhankin
e16081d4d4 Fix initialization of precision. 2018-10-18 14:15:47 +03:00
Ivan Lezhankin
6d783ef013 Move-away "uniqCombined" as a separate aggregated function with HLL precision 2018-10-17 17:31:06 +03:00
Alexey Milovidov
f720a5bf95 Fixed typos in code #3373 2018-10-13 17:33:43 +03:00
chertus
5896e45dac SummingMergeTree for Decimal128 [issue-3282] 2018-10-11 17:33:01 +03:00
chertus
5b987f02d7 sumMap for number-based types [issue-3277] 2018-10-03 17:22:28 +03:00
alexey-milovidov
a149233d99
Merge pull request #3246 from yandex/low-cardinality-fixes
Low cardinality fixes
2018-09-28 19:50:53 +03:00
chertus
292a885533 fix typo in quantiles func registration 2018-09-28 18:08:37 +03:00
Nikolai Kochetov
97add69825 Rename WithDictionary to LowCardinality. 2018-09-28 13:44:59 +03:00
chertus
3ff2cfb4af fix div by zero in decimal var/stddev 2018-09-26 15:38:36 +03:00
alexey-milovidov
babc145f5f
Merge pull request #3188 from yandex/fix_segfault_3151
Add isState for array function combinator
2018-09-21 22:30:15 +03:00
alesapin
4d29141957 Add isState for array function combinator 2018-09-21 12:37:09 +03:00
Alexey Milovidov
16dacff488 Fixed build [#CLICKHOUSE-2] 2018-09-20 23:50:58 +03:00