thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-30 13:40:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
33,627 Commits 8,956 Branches 1,767 Tags 1.8 GiB
0c6b5ebd48
Commit Graph

763 Commits

Author SHA1 Message Date
VDimir
99f531c58f Fix spelling require -> requires 2019-10-13 18:37:01 +00:00
VDimir
853322503f Allow count function arity at most one 2019-10-12 15:15:16 +00:00
Nikolai Kochetov
6f758babf6
Merge pull request #7289 from infinivision/fix_pull_7109 
Fixed #7109 groupBitmapOr(31) segfault
 2019-10-12 16:42:11 +03:00
Zhichang Yu
679db35ed3 Fixed #7109 groupBitmapOr(31) segfault 2019-10-12 10:36:52 +08:00
Alexander Kuzmenkov
57b5592236 Work around garbage in roaring.h to fix Mac OS build. 2019-10-11 14:28:16 +03:00
alexey-milovidov
689411abe0
Merge pull request #7222 from azat/uniqCombined64 
[RFC] Introduce uniqCombined64() to get sane results for cardinality > UINT_MAX
 2019-10-09 20:29:56 +03:00
alexey-milovidov
6cd3e9a458
Update AggregateFunctionUniqCombined.h 2019-10-09 20:26:01 +03:00
alexey-milovidov
81a9c81ba7
Merge pull request #7236 from azat/uniqCombined-memory-usage 
Do not use more then 98K of memory for uniqCombined*
 2019-10-09 20:19:30 +03:00
Azat Khuzhin
e373862c83 Do not use more then 98K of memory for uniqCombined* 
uniqCombined() uses hashtable for medium cardinality, and since
HashTable resize by the power of 2 (well actually HashTableGrower grows
double by the power of 2, hence HashTableGrower::increaseSize() should
be overwritten to change this), with 1<<13 (default for uniqCombined)
and UInt64 HashValueType, the HashTable will takes:

  getBufferSizeInBytes() == 131072

While it should be not greater then sizeof(HLL) ~= 98K, so reduce the
maximum cardinality for hashtable to 1<<12 with UInt64 HashValueType and
to 1<13 with UInt32, overwrite HashTableGrower::increaseSize() and cover
this using max_memory_usage.

Refs: https://github.com/ClickHouse/ClickHouse/pull/7221#issuecomment-539672742

v2: cover uniqCombined() with non-default K
 2019-10-09 02:39:23 +03:00
alexey-milovidov
4c1f0177b6
Merge pull request #7109 from infinivision/aggBitmapAnd 
added groupBitmapAnd, groupBitmapOr, groupBitmapXor
 2019-10-08 22:51:46 +03:00
Azat Khuzhin
f46c5a47c0 Introduce uniqCombined64() to get sane results for cardinality > UINT_MAX 
By default uniqCombined() uses 32-bit hash for all types except String,
so this means that you cannot use uniqCombined() with i.e UInt64 and
cardinality > UINT_MAX, although you can use uniqCombined(toString())
and this will lead to 64-bit hash, but will not have good performance.

So uniqCombined64() had been introduced to fix this.

Requires: #7213
 2019-10-08 21:59:35 +03:00
Alexander Kuzmenkov
2b30f2696c Include Field.h in less files. 2019-10-08 16:23:24 +03:00
Alexander Kuzmenkov
8e245d4e8c
Merge pull request #7135 from ClickHouse/aku/msan-field 
Fix some MemorySanitizer warnings for Fields.
 2019-10-03 10:33:44 +03:00
chertus
5df8f870c4 try remove some PVS errors 2019-09-30 18:45:53 +03:00
Alexander Kuzmenkov
ec2d9a083c Remove excessive includes. 2019-09-30 15:31:14 +03:00
Zhichang Yu
8945714c27 added groupBitmapAnd, groupBitmapOr, groupBitmapXor 2019-09-27 10:21:07 +08:00
Alexander Kuzmenkov
f3bde19b74 Do not use iterators in find() and emplace() methods of hash tables. 
Instead, these methods return a pointer to the required data as they are
stored inside the hash table. The caller uses overloaded functions to
get the key and "mapped" values from this pointer. Such an interface
avoids the need for constructing iterator-like wrapper objects, which is
especially important for compound hash tables such as the future
StringHashMap.
 2019-09-24 17:44:35 +03:00
alexey-milovidov
6020ba187e
Update AggregateFunctionGroupBitmapData.h 2019-09-22 04:58:21 +03:00
Zhichang Yu
0dc3866c36 added bitmapSubsetLimit 2019-09-19 10:15:36 +08:00
alexey-milovidov
1327e71d80
Changed C-style cast to constructor-style cast. 2019-09-19 02:01:06 +03:00
Zhichang Yu
460ccb04a2 added bitmapMin and bitmapMax 2019-09-18 16:56:12 +08:00
Nikolai Kochetov
1319168a3a Merge branch 'master' into processors-2 2019-09-12 16:28:51 +03:00
alexey-milovidov
a14b3c7fcb
Merge pull request #6908 from infinivision/fix_6880 
use croaring serialization functions
 2019-09-12 03:21:05 +03:00
Nikolai Kochetov
d913e415b8 Fix Adam::merge if right state wasn't initialized. 2019-09-11 20:48:42 +03:00
Zhichang Yu
6c5019ca39 use croaring serialization functions 2019-09-11 12:35:43 +00:00
Amos Bird
a1310ffbe6 Split libdbms.so using object library 
Now the linking time of incremental builds are around 1-2 seconds
 2019-09-11 01:56:01 +08:00
Alexander Kuzmenkov
cacf6d3282 Key memory management for compound hash tables. 
Some aggregation methods initially emplace a temporary StringRef key
into a hash table. Then, if the key was not seen before, they make a
persistent copy of the key and update the hash table with it. This
approach is not suitable for compound hash tables, because the logic of
when the persistent key is needed is more complex, and is contained
within the hash table itself.

In this commit, we switch to managing key memory with callbacks passed
to the hash table, that allow it to request a persistent copy of the key
if it is needed. This should be more appropriate for compound hash
tables.

This commit prepares for StringHashMap PR #5417.
 2019-09-06 18:05:28 +03:00
Alexander Kuzmenkov
24a8755b82 Formatting changes for StringHashMap PR #5417. 2019-08-28 13:52:14 +03:00
Alexey Milovidov
b42f85e16b Added a check for double whitespaces 2019-08-23 21:30:04 +03:00
Alexey Milovidov
8966520801 Added "assert_cast" function as a replacement for static_cast for columns 2019-08-21 05:28:04 +03:00
alexey-milovidov
c870cbce73
Merge pull request #6548 from achimbab/master 
Added 'strict' parameter in windowFunnel() and added testcases.
 2019-08-20 05:05:49 +03:00
alexey-milovidov
a116fd7a69
Merge pull request #6477 from dimarub2000/quantileExact_improvement 
QuantileExactExclusive & Inclusive function added.
 2019-08-19 20:39:37 +03:00
dimarub2000
bb4136f7b0 virtual functions deleted 2019-08-19 19:03:25 +03:00
philip.han
61769865d8 Added 'strict' parameter in windowFunnel() and added testcases. 2019-08-19 23:52:42 +09:00
dimarub2000
73e208917c Fixed comment 2019-08-14 14:18:46 +03:00
dimarub2000
6b6e477c76 Added QuantileExactInclusive function. Deleted redundant virtuals. 2019-08-14 14:13:04 +03:00
dimarub2000
43ee50e512 QuantileExactExclusive function added. 2019-08-13 22:12:31 +03:00
Alexey Milovidov
f00fa640ba Removed old optimization that is now dominated by the new optimization 2019-08-11 02:40:15 +03:00
Alexey Milovidov
3885cc20b8 Improved performance of aggregation without key 2019-08-11 02:22:23 +03:00
Alexey Milovidov
c98d2fe6e1 Addition to prev. revision 2019-08-11 01:54:33 +03:00
Alexey Milovidov
efa51a6cd9 Batch aggregation (experimental) 2019-08-11 01:36:55 +03:00
kreuzerkrieg
112fc71276 adding -Wshadow for GCC 2019-08-09 23:58:16 +03:00
Zhichang Yu
09edfaf49c let Style Check happy 2019-08-04 22:48:48 +08:00
Zhichang Yu
4f146eaa7e added bitmapRange function 2019-08-03 23:07:46 +08:00
alexey-milovidov
149ed67462
Merge pull request #6281 from yandex/fix-low-cardinality-aggregate-arguments 
Fix LowCardinality arguments conversion in AggregateFunctionFactory
 2019-08-03 01:50:21 +03:00
Nikolai Kochetov
a8378e8ef0 Fix LowCardinality arguments conversion for AggregateFunctionFactory. 2019-08-01 21:22:38 +03:00
Alexander Kuzmenkov
011150fa8a Make PairNoInit a simple struct. 2019-08-01 19:54:33 +03:00
alexey-milovidov
ad95d4e3e0
Merge pull request #6000 from Quid37/master 
regression methods adam optimizer
 2019-07-20 00:45:51 +03:00
Alexey Milovidov
a8caf269ca Better rehash loop 2019-07-18 23:28:47 +03:00
Alexey Milovidov
268854d14e Fix non-deterministic result of "uniq" aggregate function in extreme rare cases 2019-07-18 23:21:24 +03:00
Alexey Milovidov
71eed6507e Removed manual memory tracking when appropriate 2019-07-18 23:10:31 +03:00
Alexander Kozhikhov
52007c96d9 adam is default now 2019-07-17 00:11:10 +03:00
Alexander Kozhikhov
860be14761 adam optimizer 2019-07-14 23:35:34 +03:00
Alexander Kozhikhov
c5b0871c9b small fix 2019-07-12 00:02:22 +03:00
Alexander Kozhikhov
e046cdd62d some change on regression method 2019-07-11 23:56:58 +03:00
Alexey Milovidov
e8b6ac3fad Returned back support for floating point argument in function quantileTiming 2019-07-07 02:15:14 +03:00
Alexander Kuzmenkov
bd33bf8f96 Don't copy aggregate function states with memcpy. 
They may contain pointers to themselves, such as when using
PODArrayWithStackMemory. Instead, initialize empty new structures
with `create`, and merge the old one to it with `merge`.
 2019-07-02 20:48:00 +03:00
Alexander Kuzmenkov
a719933c58 Fix initial size of some inline PODArray's. 
A template parameter of PODArray named INITIAL_SIZE didn't make its
units clear, which made some callers to erroneously assume that it
specifies the number of elements and not the number of bytes.
Rename it, fix the wrong usages and, where possible, use the
PODArrayWithStackMemory typedef for arrays with inline memory.
 2019-06-28 16:03:14 +03:00
alexey-milovidov
b9cbd36487
Merge pull request #5750 from hczhcz/patch-7 
Refactor: Improve function registering
 2019-06-28 14:50:38 +03:00
Alexander Kuzmenkov
cb718c194f Fix a hard-to-spot typo: aggreAGte -> aggregate. 2019-06-26 15:35:27 +03:00
hcz
97b762ad06 Improve function registering 2019-06-26 15:53:43 +08:00
alesapin
57ed08f832
Merge pull request #5738 from yandex/hczhcz-patch-6 
Slightly better style
 2019-06-25 15:14:28 +03:00
alesapin
2fcb786093 Slightly more ClickHouse style 2019-06-25 13:17:11 +03:00
alesapin
b14a6ab8ab Merge branch 'patch-6' of https://github.com/hczhcz/ClickHouse into hczhcz-patch-6 2019-06-25 12:40:13 +03:00
hcz
3368ed933f Improve error codes 2019-06-25 12:39:00 +08:00
hcz
8a2c6985a4 Use size_t step 2019-06-25 12:37:40 +08:00
hcz
f1e146d208 Fix parameter handling 2019-06-25 12:28:20 +08:00
alexey-milovidov
b43e75200e
Merge pull request #5595 from inv2004/moving-sum 
movingSum/Avg window functions for numeric and decimals
 2019-06-24 19:13:26 +03:00
alesapin
88b9bc6254 Merge branch 'patch-6' of https://github.com/hczhcz/ClickHouse into hczhcz-patch-6 2019-06-24 14:29:35 +03:00
proller
fa809de91f
Move all url functions to clickhouse_functions_url lib (#5696) 2019-06-20 18:49:54 +03:00
unknown
238c0e3b9b movingSum: uses template for groupArrayMovingSum and groupArrayMovingAvg 2019-06-18 10:18:33 -04:00
unknown
414bb21238 Merge remote-tracking branch 'upstream/master' into moving-sum 2019-06-17 19:09:58 -04:00
Alexey Milovidov
413b780b65 Various fixes for PVS-Studio 2019-06-16 21:13:39 +03:00
unknown
7dc95e17a4 movingSum : macros name + indent 2019-06-14 11:14:58 -04:00
hcz
0385e0923a Fix style 2019-06-14 21:20:21 +08:00
Alexey Milovidov
95c7537656 Merge branch 'lin_ref_perf' of https://github.com/Quid37/ClickHouse into Quid37-lin_ref_perf 2019-06-14 15:33:29 +03:00
unknown
0dab40404e movingSum format fix: trailing space 2019-06-13 12:18:32 -04:00
Alexey Milovidov
c846d0f6af Namespace comments are unneeded according to the code style. 
find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l -P '}\s*//+\s*namespace\s*' | xargs sed -i -r -e 's/}\s*\/\/+\s*namespace\s*.*$/}/'
 2019-06-13 13:43:37 +03:00
Alexey Milovidov
8b87511e25 Namespace comments are unneeded according to the code style. 
find dbms -name '*.h' -or -name '*.cpp' | xargs grep -l -P '}\s*//+\s*namespace\s*' | xargs sed -i -r -e 's/}\s*\/\/+\s*namespace\s*\w+/}/'
 2019-06-13 13:37:13 +03:00
unknown
1e56a5b7ac movingSum format 2019-06-12 20:03:55 -04:00
unknown
8028495f78 movingSum fix : deserialize restores sum field 2019-06-12 12:10:47 -04:00
hcz
45529d8489 Fix bugs 2019-06-12 16:25:29 +08:00
hcz
6b6f2293fc Change style 2019-06-12 15:48:40 +08:00
hcz
57db1fac59 Add aggregate function combinator Resample 2019-06-12 15:46:36 +08:00
unknown
d75c73ece2 movingSum with/or without window_size parameter for numeric and decimal types 2019-06-12 01:56:37 -04:00
alexey-milovidov
4ca476ec80
Merge pull request #5516 from yandex/fix-regression-models 
Fix regression models
 2019-06-05 08:07:18 +03:00
Георгий Кондратьев
2499cefdb9 DEVTOOLS-5507 Include roaring.h as roaring/roaring.h 
This is needed to unbundle roaring. Its original cmake build system installs:

     /usr/include/roaring/roaring.h
     /usr/include/roaring/roaring.hh

and declares:

     set_target_properties(roaring::roaring PROPERTIES
       INTERFACE_INCLUDE_DIRECTORIES "/usr/include"
     )
 2019-06-03 17:03:41 +00:00
Nikolai Kochetov
7cb3e3b251 Set default learnng_rate to 0.01. 2019-06-03 08:50:07 +03:00
Nikolai Kochetov
47f82dac72 Fix build. 2019-06-03 08:31:04 +03:00
Nikolai Kochetov
d224d3dcd2 Fix build. 2019-06-03 08:25:48 +03:00
Nikolai Kochetov
1034295f04 Fixed UInt32 overflow bug in linear models. Allow eval ML model for non-const model argument. 2019-06-03 08:11:15 +03:00
Alexander Kozhikhov
ba3d6c4b6c unique 2019-06-02 01:46:25 +03:00
Alexander Kozhikhov
dfea2dcd8f unique ptr 2019-06-02 01:44:25 +03:00
Alexander Kozhikhov
da8d49a67a shared to unique 2019-05-31 18:53:31 +03:00
Alexander Kozhikhov
73c07cb9d5 fix on lin_ref_perf 2019-05-31 18:39:17 +03:00
Alexander Kozhikhov
42f3ee2e2b conflict 2019-05-31 01:55:18 +03:00
Alexander Kozhikhov
0f222640d5 style 2019-05-31 01:53:56 +03:00
Alexander Kozhikhov
15fe367f83 few fixes 2019-05-31 01:48:44 +03:00
Alexander Kozhikhov
ef23528452 some changes and fixes on regression methods 2019-05-31 00:59:40 +03:00
Nikolai Kochetov
713ccd1f60 Added comment for IAggregateFunction::getReturnTypeToPredict(). Small code fixes. 2019-05-29 06:42:33 +03:00