Commit Graph

684 Commits

Author SHA1 Message Date
sundy-li
4c95b9d007 modify createAggregateFunctionWindowFunnel 2019-05-17 19:31:29 +08:00
sundy-li
d3c4c1c255 modify createAggregateFunctionWindowFunnel 2019-05-17 19:30:09 +08:00
sundy-li
760bc5708d support unsigned integer type in windowFunnel Aggregate function 2019-05-17 19:17:52 +08:00
chertus
fdcbf509bb support quantile(s)ExactWeighted for Decimal 2019-05-16 17:33:13 +03:00
chertus
50c35a70db support quanitle, quantiles, median for Decimal 2019-05-16 15:53:33 +03:00
proller
0dd29e7c5b clang-format of AggregateFunctions/AggregateFunctionTSGroupSum.* 2019-05-15 19:27:44 +03:00
Yangkuan Liu
0760a3436f add AggregateFunction TSgroup{Rate}Sum (#4542) 2019-05-15 19:16:25 +03:00
proller
7cb8d46338
Fix build (move code from AggregateFunctionMLMethod.h) (#5266) 2019-05-14 22:52:29 +03:00
Alexander Kozhikhov
332a8a97ab style 2019-05-13 20:08:58 +03:00
Alexander Kozhikhov
9b4f322fd8 Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-05-13 16:50:56 +03:00
Alexander Kozhikhov
79010b7c3c some review fixes 2019-05-13 16:49:53 +03:00
svladykin
22389d4eca Support for bitmapHasAny and bitmapHasAll functions. 2019-05-12 17:47:31 +03:00
hcz
cf95327cba Remove unused variables and add comment 2019-05-11 02:06:58 +08:00
hcz
4f3e164c73 Fix and add tests 2019-05-06 12:08:54 +08:00
hcz
d86d60297a Add skewness and kurtosis functions 2019-05-06 11:49:14 +08:00
alexey-milovidov
1d69a73424
Merge pull request #5119 from nicelulu/update_24
modification retention function
2019-04-29 04:51:18 +03:00
Evgenii Pravda
91ccf2ab20 Fixes 2019-04-27 20:52:43 +03:00
Evgenii Pravda
568ef78803 Add MSD radix sort 2019-04-27 01:07:32 +03:00
821008736@qq.com
02e9cc8f2d fix retention function 2019-04-26 17:36:35 +08:00
alexey-milovidov
d2a4aa964d
Merge pull request #5026 from PerformanceVision/uniq_array
Add an optional max_size parameter to groupUniqArray
2019-04-24 14:34:39 +03:00
proller
ba4687e110
Cmake: dont use LINK_MODE (but use BUILD_SHARED_LIBS) (#5091) 2019-04-24 12:06:28 +03:00
alexey-milovidov
3db650861d
Update AggregateFunctionGroupUniqArray.h 2019-04-24 01:40:16 +03:00
hcz
3bbe57adac
Rename AggregateFunctionsStatisticsSimple.cpp to AggregateFunctionStatisticsSimple.cpp 2019-04-22 17:31:01 +08:00
Alexander Kozhikhov
6c7f2cbc37 asan 2019-04-21 19:08:55 +03:00
Alexander Kozhikhov
fd07c97097 Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-04-21 18:43:29 +03:00
Alexander Kozhikhov
0bedabb5e4 review fixes 2019-04-21 17:44:39 +03:00
Alexander Kozhikhov
ef98afb6a6 review fixes 2019-04-21 17:42:37 +03:00
Alexander Kozhikhov
daf4690d37 review fixes 2019-04-21 17:32:42 +03:00
Alexander Kozhikhov
b6d2c9f4d2 some review fixes 2019-04-21 02:22:42 +03:00
Guillaume Tassery
a96e3c470e Reformate code for optional parameter for groupUniqArray 2019-04-17 17:24:02 +07:00
Guillaume Tassery
2a6ff2f240 Code refactoring for the new size parameter on the GroupUniqArray aggregation function 2019-04-15 18:32:43 +07:00
Guillaume Tassery
803675892b Handle merge operation for size parameter on groupUniqArray aggregation function 2019-04-15 16:09:52 +07:00
Alexander Kozhikhov
a2a9d4b513 style 2019-04-15 10:59:56 +03:00
Guillaume Tassery
857bccc981 Add a way to set a size parameter for groupUniqArray 2019-04-15 13:36:24 +07:00
Alexander Kozhikhov
9ed740e3ea style 2019-04-15 04:10:39 +03:00
Alexander Kozhikhov
19021e76bb style 2019-04-15 03:16:13 +03:00
Alexey Milovidov
985936995f Fixed Java programmer's error (found by PVS-Studio) 2019-04-10 23:15:44 +03:00
Alexander Kozhikhov
c93aae6741 linear regression 2019-04-09 01:40:37 +03:00
Alexander Kozhikhov
a9ae6d0681 Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-04-09 00:01:28 +03:00
Alexander Kozhikhov
35a266c96a linear regression tests 2019-04-09 00:01:10 +03:00
Alexey Milovidov
571bc7065a Removed unused code related to MemoryTracker 2019-04-06 01:35:56 +03:00
Nikolai Kochetov
085c758dbd Avoid divizion by zero in leastSqr function. 2019-04-05 17:52:27 +03:00
Nikolai Kochetov
bb9958b0d7 Minor fixes for leastSqr. 2019-04-05 16:42:33 +03:00
Nikolai Kochetov
feb16eedd2 Merge branch 'master' of https://github.com/hczhcz/ClickHouse into hczhcz-master 2019-04-05 16:10:29 +03:00
Alexander Kozhikhov
f550304fa4 Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-04-04 03:17:37 +03:00
Alexander Kozhikhov
b5f0414200 new test 2019-04-04 03:17:27 +03:00
Alexey Milovidov
67b6a8966b Attempt to make compositions with COWPtr more convenient 2019-03-25 20:56:18 +03:00
hcz
b74d3501e4
Fix style error 2019-03-23 20:20:40 +08:00
hcz
38ea75748c
Update AggregateFunctionLeastSqr.h 2019-03-22 15:57:33 +08:00
hcz
06bb9ad833
Update AggregateFunctionLeastSqr.cpp 2019-03-22 15:57:17 +08:00
olegkv
a4739c23fb
Update AggregateFunctionHistogram.h 2019-03-20 15:33:01 -04:00
Nikolai Kochetov
c166614173 Fix rare ubsan false positive error while deserializing empty array from StringRef with length = 0 and trash in data. 2019-03-19 14:11:05 +03:00
alexey-milovidov
93356b5190
Merge pull request #4702 from hczhcz/patch-3
Fix #4690
2019-03-16 20:55:24 +03:00
Ivan
8ac3e89628
Merge pull request #4649 from abyss7/iast-as
Implement and use method IAST::as<Derived>()
2019-03-15 23:22:46 +03:00
Ivan Lezhankin
61954d855d More reverts 2019-03-15 20:09:14 +03:00
hcz
c8dd9d2df9 Fix #4690 2019-03-15 16:33:59 +08:00
Alexey Milovidov
a459b2c298 Fixed insufficient type checking in hasAll, hasAny functions 2019-03-15 02:10:51 +03:00
Ivan Lezhankin
f64118030a Merge remote-tracking branch 'upstream/master' into iast-as 2019-03-13 17:23:02 +03:00
proller
cf8001b6a9
Fix includes (#4673) 2019-03-13 16:58:04 +03:00
hcz
72bcbc76b1 Add aggregate function leastSqr 2019-03-13 15:22:57 +08:00
Ivan Lezhankin
d1ead22073 Rename As<…> → as<…> 2019-03-11 16:22:51 +03:00
Ivan Lezhankin
1e9f7847dc Merge remote-tracking branch 'upstream/master' into iast-as 2019-03-11 16:11:22 +03:00
Ivan Lezhankin
a7e838e328 Replace all other casts to method IAST::As<Derived>
Also remove some redundant methods like IsIdentifier() et. al.
2019-03-11 15:49:39 +03:00
alexey-milovidov
edacebac21
Merge pull request #4634 from kirillDanshin/feature/topK-loadFactor
topK and topKWeighted now supports custom loadFactor
2019-03-11 03:41:55 +03:00
alexey-milovidov
86e7889622
Update AggregateFunctionTopK.h 2019-03-11 00:33:57 +03:00
alexey-milovidov
a16c16152d
Update AggregateFunctionTopK.cpp 2019-03-11 00:32:42 +03:00
Alexey Milovidov
ac4045c55d Fixed UB in topKWeighted aggregate function #4634 2019-03-11 00:31:10 +03:00
alexey-milovidov
6db73152d2
Hardening debug build (experimental) (#4632)
* Hardening debug build: more granular memory mappings and ASLR; add memory protection for mark cache and index

* Addition to prev. revision

* Addition to prev. revision

* Addition to prev. revision
2019-03-10 06:16:51 +03:00
Kirill Danshin
698c022a52
topK and topKWeighted now supports custom loadFactor
Signed-off-by: Kirill Danshin <kirill@danshin.pro>
2019-03-09 08:27:40 +03:00
Ivan Lezhankin
0e325c7a4a Replace const typeid_casts with As<Derived>() method invocations 2019-03-08 12:48:20 +03:00
Alexey Milovidov
9d89cc0fc1 Debugging PODArray 2019-03-08 06:23:02 +03:00
proller
89014b5480
Build fixes (#4591) 2019-03-05 13:15:47 +03:00
Vitaly Baranov
6edec1c63f Remove unnecessary 'this->'. 2019-03-04 22:46:05 +03:00
Vitaly Baranov
405a747ddc Fix compilation after changing SmallTable. 2019-03-04 22:45:51 +03:00
Vitaly Baranov
61e21d50e1
Merge pull request #4207 from andyyzh/bitmap_feature
Added bitmap function feature with roaring bitmap
2019-03-04 22:42:47 +03:00
Alexander Kozhikhov
6371b4021d changes in gradient_computer class 2019-03-03 13:21:12 +03:00
quid
3fa972e6b6 changes in weights updater 2019-03-03 11:46:36 +03:00
Alexey Milovidov
51ca51aa9e Merge branch 'master' into quantile-timing-ubsan-fix 2019-03-01 17:53:04 +03:00
Amos Bird
26ab5dd7a7 A Proper lookup table that uses HashTable's API
This is the first step of allowing heterogeneous cells in hash tables.

performance test results are

```

1. HashMap<UInt16, UInt8, TrivialHash, HashTableFixedGrower<16>>;
2. NewLookupMap<UInt16, UInt8>

ResolutionWidth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................223550276.46
ResolutionWidth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................248772721.24
Best: 2 - 24877272124

ResolutionWidth 100000 1 ..........................................................................................................................................................................................................................................................238498413.99
ResolutionWidth 100000 2 ..........................................................................................................................................................................................................................................................261808889.98
Best: 2 - 26180888998

ResolutionWidth 300000 1 ...................................................................................239307348.81
ResolutionWidth 300000 2 ...................................................................................257592761.30
Best: 2 - 25759276130

ResolutionWidth 1000000 1 .........................240144759.26
ResolutionWidth 1000000 2 .........................257093531.91
Best: 2 - 25709353191

ResolutionWidth 5000000 1 .....241573260.35
ResolutionWidth 5000000 2 .....259314162.79
Best: 2 - 25931416279

ResolutionDepth 30000 1 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................217108119.84
ResolutionDepth 30000 2 .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................249459504.41
Best: 2 - 24945950441

ResolutionDepth 100000 1 ..........................................................................................................................................................................................................................................................229065162.17
ResolutionDepth 100000 2 ..........................................................................................................................................................................................................................................................253769105.64
Best: 2 - 25376910564

ResolutionDepth 300000 1 ...................................................................................233079225.18
ResolutionDepth 300000 2 ...................................................................................256316273.78
Best: 2 - 25631627378

ResolutionDepth 1000000 1 .........................234184633.51
ResolutionDepth 1000000 2 .........................261100491.57
Best: 2 - 26110049157

ResolutionDepth 5000000 1 .....233118795.66
ResolutionDepth 5000000 2 .....252436160.41
Best: 2 - 25243616041

```
2019-03-01 16:47:13 +08:00
Alexey Milovidov
610f9523b0 Addition to prev. revision 2019-02-26 22:34:32 +03:00
Masha
67b28c2240 Nesterov and Adam + tests 2019-02-26 08:12:16 +00:00
Alexey Milovidov
f02a834615 Fixed fuzz test in UBSan 2019-02-25 21:03:52 +03:00
Masha
e708983e9b Changed Linear Regression and test 2019-02-15 23:38:13 +00:00
Masha
d5753636bd Logistic Regression and new tests 2019-02-15 22:47:56 +00:00
Andy Yang
4d42a297ae Adjust to the latest IAggregateFunctionDataHelper 2019-02-14 19:51:13 +08:00
Andy Yang
4a4dbcd2c1 Merge remote-tracking branch 'origin' into bitmap_feature 2019-02-14 15:14:18 +08:00
Alexey Milovidov
a0b91077e7 Fixed -Wshadow-field-in-constructor (clang 8 warning) 2019-02-13 15:53:36 +03:00
alexander kozhikhov
b23cd21ce1 predict correctly using scalar aggr state 2019-02-13 01:33:37 +03:00
alexander kozhikhov
6a8542c8f6 constructor changes change 2019-02-13 00:18:27 +03:00
alexander kozhikhov
aedf4925dc Merge branch 'master' into ml_methods 2019-02-12 23:04:25 +03:00
alexander kozhikhov
f20ad204e7 Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-02-12 23:00:47 +03:00
Nikolai Kochetov
ec6a5590fd Fix build. 2019-02-12 12:31:20 +03:00
Nikolai Kochetov
2b8b342ccd Update IAggregateFunction interface. 2019-02-11 22:26:32 +03:00
Nikolai Kochetov
c35439a5f0 Updated FieldVisitor. 2019-02-11 18:46:19 +03:00
alexander kozhikhov
fc4c721fa5 some review fixes 2019-02-11 01:07:47 +03:00
alexander kozhikhov
a948f223bc Merge branch 'master' of https://github.com/Quid37/ClickHouse 2019-02-11 00:17:14 +03:00
alexander kozhikhov
bfccafef49 small test code 2019-02-11 00:16:16 +03:00
Alexey Milovidov
7ddede66db Added support for generic case for aggregate function topKWeighted; fixed errors #4245 2019-02-11 00:01:26 +03:00
Alexey Milovidov
ec5fbce404 Merge branch 'add_topkweighed' of https://github.com/andrewgolman/ClickHouse into andrewgolman-add_topkweighed 2019-02-10 23:31:17 +03:00
Alexey Milovidov
e150e62040 Grouped includes (40 of 182) 2019-02-10 20:40:52 +03:00
Alexey Milovidov
ba3cb83a84 Removed duplicate includes 2019-02-10 20:12:22 +03:00
alexander kozhikhov
7cea77b8c0 Merge branch 'master' of https://github.com/Quid37/ClickHouse into ml_methods 2019-02-10 00:19:08 +03:00
alexander kozhikhov
be17c6fab6 Merge branch 'master' of https://github.com/yandex/ClickHouse 2019-02-10 00:10:33 +03:00
Alexey Milovidov
7d86f2debe Added comments 2019-02-09 23:35:55 +03:00
Alexey Milovidov
bf95f19655 Fixed entropy aggregate function #4238 2019-02-09 23:17:20 +03:00
Andy Yang
1a28ba01c1 Added bitmap function feature, fixed test errors and code styles 2019-02-09 14:33:09 +08:00
Alexey Milovidov
c60db4b978 Better code 2019-02-07 00:47:31 +03:00
alexey-milovidov
2216250a84
Merge pull request #4238 from Quid37/yandex_open_code_competition
Implement Shannon entropy aggregate function
2019-02-04 18:43:29 +03:00
andrewsg
31f2f9ce2f Add topKWeighed aggregate function 2019-02-03 13:59:27 +03:00
alexander kozhikhov
c5924d8bfb review fixes 2019-02-02 18:19:59 +03:00
Danila Kutenin
5e62a0825a produce hints for typo functions and types 2019-02-02 17:54:50 +03:00
alexander kozhikhov
8c2726b77c Aggregate function for entropy 2019-02-02 17:27:43 +03:00
Andy Yang
7e7a864fd4 Added bitmap function feature with roaring bitmap 2019-01-31 21:26:11 +08:00
Masha
39f1d9756e fixed name Logistic 2019-01-28 11:54:55 +00:00
Masha
286fa25ad9 logreg and momentum 2019-01-28 10:39:57 +00:00
alexander kozhikhov
57cd47a194 LinearModelData completed 2019-01-26 15:38:42 +03:00
Léo Ercolanelli
038a48bb38 sumMap: implement sumMapWithOverflow 2019-01-25 20:35:53 +01:00
Léo Ercolanelli
57104f81da rename widen to promote for IDataType 2019-01-25 15:16:23 +01:00
Léo Ercolanelli
0d6094a3ea sumMap: return types less prone to oveflows
It used to be that sumMap would return the same type as the values
columns. If columns of Array(UInt8) were to be given, that would really
easily cause oveflow. It now uses `getWidenDataType` (and ultimately
`NearestFieldType`) in order to define the result type.
2019-01-25 14:29:45 +01:00
alexey-milovidov
2626a2614c
Merge pull request #4145 from yandex/fix-fuzz-test-in-ubsan-quantile-weighted
Fix fuzz test in ubsan: added parameter type check for quantile*Weighted.
2019-01-25 02:45:07 +03:00
Alexey Milovidov
ef50601b5c Removed non-breaking spaces: find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l $'\xc2\xa0' | xargs sed -i -r -e 's/\xc2\xa0/ /g' 2019-01-25 02:01:56 +03:00
Alexey Milovidov
5ccc42fd96 Revert "Removed non-breaking spaces: find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l $'\xc2\xa0' | xargs sed -i -r -e 's/\xc2\xa0/ /'"
This reverts commit a28f0acf09.
2019-01-25 01:59:45 +03:00
Alexey Milovidov
a28f0acf09 Removed non-breaking spaces: find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l $'\xc2\xa0' | xargs sed -i -r -e 's/\xc2\xa0/ /' 2019-01-24 22:49:36 +03:00
Alexey Milovidov
ffd18163de Fixed fuzz test in UBSan: added parameter check for quantile*Weighted function 2019-01-24 21:33:57 +03:00
alexey-milovidov
fbefc99fb1
Merge pull request #4129 from ercolanelli-leo/sumMapFiltered
implement sumMapFiltered
2019-01-24 21:27:03 +03:00
Maxim Kuznetsov
73d83d0ab6 Added LinearModelData 2019-01-24 17:22:35 +03:00
Léo Ercolanelli
ba1f1c7460 sumMapFiltered: use an unordered_set 2019-01-24 13:32:30 +01:00
Léo Ercolanelli
90afad8c00 sumMapFiltered: improve error reporting on parameter type mismatch 2019-01-24 13:32:30 +01:00
Léo Ercolanelli
a8647761ce sumMapFiltered: fix dangling reference 2019-01-24 13:32:30 +01:00
Léo Ercolanelli
22f7bd03b9 sumMapFiltered: use CRTP instead of virtual for keepKey 2019-01-24 09:02:17 +01:00
Léo Ercolanelli
efb1cb0528 sumMapFiltered: remove this-> in constructor 2019-01-24 08:58:57 +01:00
alexander kozhikhov
8472048328 Merge branch 'master' into alexkoja_ML 2019-01-23 21:07:05 +03:00
alexander kozhikhov
b229498203 mini-batches 2019-01-23 21:03:26 +03:00
Masha
b9972f8e67 code style AggregateFunctionMLMethod.cpp 2019-01-23 14:53:50 +00:00
Léo Ercolanelli
8ad1a55f3b implement sumMapFiltered 2019-01-23 15:23:04 +01:00
Maxim Kuznetsov
fd1beddb6d code style 2019-01-23 15:11:17 +03:00
Maxim Kuznetsov
03deb677b9 Code style 2019-01-23 14:58:05 +03:00
alexander kozhikhov
fd8f9c4fba changes after review 2019-01-23 04:29:53 +03:00
alexander kozhikhov
ffd0d9af26 Merge branch 'alexkoja_ML' 2019-01-23 00:31:53 +03:00
alexander kozhikhov
61bb3b8ade simple linear regression 2019-01-23 00:07:05 +03:00
maiha
6c83beafdd dbms: Fixed misspells in comments 2019-01-23 04:56:53 +09:00
Alexey Milovidov
c7b95b5175 Miscellaneous changes for PVS-Studio 2019-01-20 02:27:52 +03:00
Alexey Milovidov
61a59956b3 Indentation #4004 2019-01-16 20:25:06 +03:00
Alexey Milovidov
23fbf18555 Merge branch 'sequence-max-iterations' of https://github.com/ercolanelli-leo/ClickHouse into ercolanelli-leo-sequence-max-iterations 2019-01-16 19:37:51 +03:00
Léo Ercolanelli
043ef82468 fix style in AggregateFunctionSequenceMatch.h 2019-01-16 17:20:29 +01:00
Alexey Milovidov
bf2ca149c9 Merge branch 'sequence-max-iterations' of https://github.com/ercolanelli-leo/ClickHouse into ercolanelli-leo-sequence-max-iterations 2019-01-16 18:22:35 +03:00
Léo Ercolanelli
6048a83462 add comment to sequenceMatch's DFA implementation 2019-01-16 10:39:35 +01:00
Léo Ercolanelli
04651503d6 sequence: implement DFA matching algorithm 2019-01-16 10:39:35 +01:00
Alexey Milovidov
ad586e9bd2 Miscellaneous #4000 2019-01-08 18:57:15 +03:00
Alexey Milovidov
a594293b50 Addition to Amos Bird changes #3920 2019-01-08 17:55:37 +03:00