mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-09-19 16:20:50 +00:00
Merge aadccdedcd
into b94a7167a8
This commit is contained in:
commit
e8d10d0f34
@ -0,0 +1,77 @@
|
|||||||
|
---
|
||||||
|
slug: /en/sql-reference/aggregate-functions/reference/quantileExactWeightedInterpolated
|
||||||
|
sidebar_position: 176
|
||||||
|
---
|
||||||
|
|
||||||
|
# quantileExactWeightedInterpolated
|
||||||
|
|
||||||
|
Computes [quantile](https://en.wikipedia.org/wiki/Quantile) of a numeric data sequence using linear interpolation, taking into account the weight of each element.
|
||||||
|
|
||||||
|
To get the interpolated value, all the passed values are combined into an array, which are then sorted by their corresponding weights. Quantile interpolation is then performed using the [weighted percentile method](https://en.wikipedia.org/wiki/Percentile#The_weighted_percentile_method) by building a cumulative distribution based on weights and then a linear interpolation is performed using the weights and the values to compute the quantiles.
|
||||||
|
|
||||||
|
When using multiple `quantile*` functions with different levels in a query, the internal states are not combined (that is, the query works less efficiently than it could). In this case, use the [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) function.
|
||||||
|
|
||||||
|
We strongly recommend using `quantileExactWeightedInterpolated` instead of `quantileInterpolatedWeighted` because `quantileExactWeightedInterpolated` is more accurate than `quantileInterpolatedWeighted`. Here is an example:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT
|
||||||
|
quantileExactWeightedInterpolated(0.99)(number, 1),
|
||||||
|
quantile(0.99)(number),
|
||||||
|
quantileInterpolatedWeighted(0.99)(number, 1)
|
||||||
|
FROM numbers(9)
|
||||||
|
|
||||||
|
|
||||||
|
┌─quantileExactWeightedInterpolated(0.99)(number, 1)─┬─quantile(0.99)(number)─┬─quantileInterpolatedWeighted(0.99)(number, 1)─┐
|
||||||
|
│ 7.92 │ 7.92 │ 8 │
|
||||||
|
└────────────────────────────────────────────────────┴────────────────────────┴───────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
quantileExactWeightedInterpolated(level)(expr, weight)
|
||||||
|
```
|
||||||
|
|
||||||
|
Alias: `medianExactWeightedInterpolated`.
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `level` — Level of quantile. Optional parameter. Constant floating-point number from 0 to 1. We recommend using a `level` value in the range of `[0.01, 0.99]`. Default value: 0.5. At `level=0.5` the function calculates [median](https://en.wikipedia.org/wiki/Median).
|
||||||
|
- `expr` — Expression over the column values resulting in numeric [data types](../../../sql-reference/data-types/index.md#data_types), [Date](../../../sql-reference/data-types/date.md) or [DateTime](../../../sql-reference/data-types/datetime.md).
|
||||||
|
- `weight` — Column with weights of sequence members. Weight is a number of value occurrences.
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Quantile of the specified level.
|
||||||
|
|
||||||
|
Type:
|
||||||
|
|
||||||
|
- [Float64](../../../sql-reference/data-types/float.md) for numeric data type input.
|
||||||
|
- [Date](../../../sql-reference/data-types/date.md) if input values have the `Date` type.
|
||||||
|
- [DateTime](../../../sql-reference/data-types/datetime.md) if input values have the `DateTime` type.
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Input table:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌─n─┬─val─┐
|
||||||
|
│ 0 │ 3 │
|
||||||
|
│ 1 │ 2 │
|
||||||
|
│ 2 │ 1 │
|
||||||
|
│ 5 │ 4 │
|
||||||
|
└───┴─────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌─quantileExactWeightedInterpolated(n, val)─┐
|
||||||
|
│ 1.5 │
|
||||||
|
└───────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
**See Also**
|
||||||
|
|
||||||
|
- [median](../../../sql-reference/aggregate-functions/reference/median.md#median)
|
||||||
|
- [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles)
|
@ -9,7 +9,7 @@ sidebar_position: 177
|
|||||||
|
|
||||||
Syntax: `quantiles(level1, level2, ...)(x)`
|
Syntax: `quantiles(level1, level2, ...)(x)`
|
||||||
|
|
||||||
All the quantile functions also have corresponding quantiles functions: `quantiles`, `quantilesDeterministic`, `quantilesTiming`, `quantilesTimingWeighted`, `quantilesExact`, `quantilesExactWeighted`, `quantileInterpolatedWeighted`, `quantilesTDigest`, `quantilesBFloat16`, `quantilesDD`. These functions calculate all the quantiles of the listed levels in one pass, and return an array of the resulting values.
|
All the quantile functions also have corresponding quantiles functions: `quantiles`, `quantilesDeterministic`, `quantilesTiming`, `quantilesTimingWeighted`, `quantilesExact`, `quantilesExactWeighted`, `quantileExactWeightedInterpolated`, `quantileInterpolatedWeighted`, `quantilesTDigest`, `quantilesBFloat16`, `quantilesDD`. These functions calculate all the quantiles of the listed levels in one pass, and return an array of the resulting values.
|
||||||
|
|
||||||
## quantilesExactExclusive
|
## quantilesExactExclusive
|
||||||
|
|
||||||
|
@ -312,6 +312,9 @@ struct NameQuantilesExactInclusive { static constexpr auto name = "quantilesExac
|
|||||||
struct NameQuantileExactWeighted { static constexpr auto name = "quantileExactWeighted"; };
|
struct NameQuantileExactWeighted { static constexpr auto name = "quantileExactWeighted"; };
|
||||||
struct NameQuantilesExactWeighted { static constexpr auto name = "quantilesExactWeighted"; };
|
struct NameQuantilesExactWeighted { static constexpr auto name = "quantilesExactWeighted"; };
|
||||||
|
|
||||||
|
struct NameQuantileExactWeightedInterpolated { static constexpr auto name = "quantileExactWeightedInterpolated"; };
|
||||||
|
struct NameQuantilesExactWeightedInterpolated { static constexpr auto name = "quantilesExactWeightedInterpolated"; };
|
||||||
|
|
||||||
struct NameQuantileInterpolatedWeighted { static constexpr auto name = "quantileInterpolatedWeighted"; };
|
struct NameQuantileInterpolatedWeighted { static constexpr auto name = "quantileInterpolatedWeighted"; };
|
||||||
struct NameQuantilesInterpolatedWeighted { static constexpr auto name = "quantilesInterpolatedWeighted"; };
|
struct NameQuantilesInterpolatedWeighted { static constexpr auto name = "quantilesInterpolatedWeighted"; };
|
||||||
|
|
||||||
|
@ -29,7 +29,7 @@ namespace
|
|||||||
* It uses O(distinct(N)) memory. Can be naturally applied for values with weight.
|
* It uses O(distinct(N)) memory. Can be naturally applied for values with weight.
|
||||||
* In case of many identical values, it can be more efficient than QuantileExact even when weight is not used.
|
* In case of many identical values, it can be more efficient than QuantileExact even when weight is not used.
|
||||||
*/
|
*/
|
||||||
template <typename Value>
|
template <typename Value, bool interpolated>
|
||||||
struct QuantileExactWeighted
|
struct QuantileExactWeighted
|
||||||
{
|
{
|
||||||
struct Int128Hash
|
struct Int128Hash
|
||||||
@ -46,6 +46,7 @@ struct QuantileExactWeighted
|
|||||||
|
|
||||||
/// When creating, the hash table must be small.
|
/// When creating, the hash table must be small.
|
||||||
using Map = HashMapWithStackMemory<UnderlyingType, Weight, Hasher, 4>;
|
using Map = HashMapWithStackMemory<UnderlyingType, Weight, Hasher, 4>;
|
||||||
|
using Pair = typename Map::value_type;
|
||||||
|
|
||||||
Map map;
|
Map map;
|
||||||
|
|
||||||
@ -85,6 +86,42 @@ struct QuantileExactWeighted
|
|||||||
|
|
||||||
/// Get the value of the `level` quantile. The level must be between 0 and 1.
|
/// Get the value of the `level` quantile. The level must be between 0 and 1.
|
||||||
Value get(Float64 level) const
|
Value get(Float64 level) const
|
||||||
|
{
|
||||||
|
if constexpr (interpolated)
|
||||||
|
return getInterpolatedImpl(level);
|
||||||
|
else
|
||||||
|
return getImpl(level);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Get the `size` values of `levels` quantiles. Write `size` results starting with `result` address.
|
||||||
|
/// indices - an array of index levels such that the corresponding elements will go in ascending order.
|
||||||
|
void getMany(const Float64 * levels, const size_t * indices, size_t num_levels, Value * result) const
|
||||||
|
{
|
||||||
|
if constexpr (interpolated)
|
||||||
|
getManyInterpolatedImpl(levels, indices, num_levels, result);
|
||||||
|
else
|
||||||
|
getManyImpl(levels, indices, num_levels, result);
|
||||||
|
}
|
||||||
|
|
||||||
|
Float64 getFloat(Float64 level) const
|
||||||
|
{
|
||||||
|
if constexpr (interpolated)
|
||||||
|
return getFloatInterpolatedImpl(level);
|
||||||
|
else
|
||||||
|
return getFloatImpl(level);
|
||||||
|
}
|
||||||
|
|
||||||
|
void getManyFloat(const Float64 * levels, const size_t * indices, size_t num_levels, Float64 * result) const
|
||||||
|
{
|
||||||
|
if constexpr (interpolated)
|
||||||
|
getManyFloatInterpolatedImpl(levels, indices, num_levels, result);
|
||||||
|
else
|
||||||
|
getManyFloatImpl(levels, indices, num_levels, result);
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
/// get implementation without interpolation
|
||||||
|
Value getImpl(Float64 level) const
|
||||||
{
|
{
|
||||||
size_t size = map.size();
|
size_t size = map.size();
|
||||||
|
|
||||||
@ -92,7 +129,6 @@ struct QuantileExactWeighted
|
|||||||
return std::numeric_limits<Value>::quiet_NaN();
|
return std::numeric_limits<Value>::quiet_NaN();
|
||||||
|
|
||||||
/// Copy the data to a temporary array to get the element you need in order.
|
/// Copy the data to a temporary array to get the element you need in order.
|
||||||
using Pair = typename Map::value_type;
|
|
||||||
std::unique_ptr<Pair[]> array_holder(new Pair[size]);
|
std::unique_ptr<Pair[]> array_holder(new Pair[size]);
|
||||||
Pair * array = array_holder.get();
|
Pair * array = array_holder.get();
|
||||||
|
|
||||||
@ -135,9 +171,8 @@ struct QuantileExactWeighted
|
|||||||
return it->first;
|
return it->first;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Get the `size` values of `levels` quantiles. Write `size` results starting with `result` address.
|
/// getMany implementation without interpolation
|
||||||
/// indices - an array of index levels such that the corresponding elements will go in ascending order.
|
void getManyImpl(const Float64 * levels, const size_t * indices, size_t num_levels, Value * result) const
|
||||||
void getMany(const Float64 * levels, const size_t * indices, size_t num_levels, Value * result) const
|
|
||||||
{
|
{
|
||||||
size_t size = map.size();
|
size_t size = map.size();
|
||||||
|
|
||||||
@ -149,7 +184,6 @@ struct QuantileExactWeighted
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Copy the data to a temporary array to get the element you need in order.
|
/// Copy the data to a temporary array to get the element you need in order.
|
||||||
using Pair = typename Map::value_type;
|
|
||||||
std::unique_ptr<Pair[]> array_holder(new Pair[size]);
|
std::unique_ptr<Pair[]> array_holder(new Pair[size]);
|
||||||
Pair * array = array_holder.get();
|
Pair * array = array_holder.get();
|
||||||
|
|
||||||
@ -197,23 +231,167 @@ struct QuantileExactWeighted
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The same, but in the case of an empty state, NaN is returned.
|
/// getFloat implementation without interpolation
|
||||||
Float64 getFloat(Float64) const
|
Float64 getFloatImpl(Float64) const
|
||||||
{
|
{
|
||||||
throw Exception(ErrorCodes::NOT_IMPLEMENTED, "Method getFloat is not implemented for QuantileExact");
|
throw Exception(ErrorCodes::NOT_IMPLEMENTED, "Method getFloat is not implemented for QuantileExact");
|
||||||
}
|
}
|
||||||
|
|
||||||
void getManyFloat(const Float64 *, const size_t *, size_t, Float64 *) const
|
/// getManyFloat implementation without interpolation
|
||||||
|
void getManyFloatImpl(const Float64 *, const size_t *, size_t, Float64 *) const
|
||||||
{
|
{
|
||||||
throw Exception(ErrorCodes::NOT_IMPLEMENTED, "Method getManyFloat is not implemented for QuantileExact");
|
throw Exception(ErrorCodes::NOT_IMPLEMENTED, "Method getManyFloat is not implemented for QuantileExact");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// get implementation with interpolation
|
||||||
|
Value getInterpolatedImpl(Float64 level) const
|
||||||
|
{
|
||||||
|
size_t size = map.size();
|
||||||
|
if (0 == size)
|
||||||
|
return std::numeric_limits<Value>::quiet_NaN();
|
||||||
|
|
||||||
|
Float64 res = getFloatInterpolatedImpl(level);
|
||||||
|
if constexpr (is_decimal<Value>)
|
||||||
|
return Value(static_cast<typename Value::NativeType>(res));
|
||||||
|
else
|
||||||
|
return static_cast<Value>(res);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// getMany implementation with interpolation
|
||||||
|
void getManyInterpolatedImpl(const Float64 * levels, const size_t * indices, size_t num_levels, Value * result) const
|
||||||
|
{
|
||||||
|
size_t size = map.size();
|
||||||
|
if (0 == size)
|
||||||
|
{
|
||||||
|
for (size_t i = 0; i < num_levels; ++i)
|
||||||
|
result[i] = Value();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::unique_ptr<Float64 []> res_holder(new Float64[num_levels]);
|
||||||
|
Float64 * res = res_holder.get();
|
||||||
|
getManyFloatInterpolatedImpl(levels, indices, num_levels, res);
|
||||||
|
for (size_t i = 0; i < num_levels; ++i)
|
||||||
|
{
|
||||||
|
if constexpr (is_decimal<Value>)
|
||||||
|
result[i] = Value(static_cast<typename Value::NativeType>(res[i]));
|
||||||
|
else
|
||||||
|
result[i] = Value(res[i]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// getFloat implementation with interpolation
|
||||||
|
Float64 getFloatInterpolatedImpl(Float64 level) const
|
||||||
|
{
|
||||||
|
size_t size = map.size();
|
||||||
|
|
||||||
|
if (0 == size)
|
||||||
|
return std::numeric_limits<Float64>::quiet_NaN();
|
||||||
|
|
||||||
|
/// Copy the data to a temporary array to get the element you need in order.
|
||||||
|
std::unique_ptr<Pair[]> array_holder(new Pair[size]);
|
||||||
|
Pair * array = array_holder.get();
|
||||||
|
|
||||||
|
size_t i = 0;
|
||||||
|
for (const auto & pair : map)
|
||||||
|
{
|
||||||
|
array[i] = pair.getValue();
|
||||||
|
++i;
|
||||||
|
}
|
||||||
|
|
||||||
|
::sort(array, array + size, [](const Pair & a, const Pair & b) { return a.first < b.first; });
|
||||||
|
std::partial_sum(array, array + size, array, [](const Pair & acc, const Pair & p) { return Pair(p.first, acc.second + p.second); });
|
||||||
|
Weight max_position = array[size - 1].second - 1;
|
||||||
|
Float64 position = max_position * level;
|
||||||
|
return quantileInterpolated(array, size, position);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// getManyFloat implementation with interpolation
|
||||||
|
void getManyFloatInterpolatedImpl(const Float64 * levels, const size_t * indices, size_t num_levels, Float64 * result) const
|
||||||
|
{
|
||||||
|
size_t size = map.size();
|
||||||
|
if (0 == size)
|
||||||
|
{
|
||||||
|
for (size_t i = 0; i < num_levels; ++i)
|
||||||
|
result[i] = std::numeric_limits<Float64>::quiet_NaN();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Copy the data to a temporary array to get the element you need in order.
|
||||||
|
std::unique_ptr<Pair[]> array_holder(new Pair[size]);
|
||||||
|
Pair * array = array_holder.get();
|
||||||
|
|
||||||
|
size_t i = 0;
|
||||||
|
for (const auto & pair : map)
|
||||||
|
{
|
||||||
|
array[i] = pair.getValue();
|
||||||
|
++i;
|
||||||
|
}
|
||||||
|
|
||||||
|
::sort(array, array + size, [](const Pair & a, const Pair & b) { return a.first < b.first; });
|
||||||
|
std::partial_sum(array, array + size, array, [](Pair acc, Pair & p) { return Pair(p.first, acc.second + p.second); });
|
||||||
|
Weight max_position = array[size - 1].second - 1;
|
||||||
|
|
||||||
|
for (size_t j = 0; j < num_levels; ++j)
|
||||||
|
{
|
||||||
|
Float64 position = max_position * levels[indices[j]];
|
||||||
|
result[indices[j]] = quantileInterpolated(array, size, position);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Calculate quantile, using linear interpolation between two closest values
|
||||||
|
Float64 NO_SANITIZE_UNDEFINED quantileInterpolated(const Pair * array, size_t size, Float64 position) const
|
||||||
|
{
|
||||||
|
/*
|
||||||
|
for (size_t i = 0; i < size; ++i)
|
||||||
|
std::cout << "array[" << i << "]: " << toString(Field(array[i].first)) << ", " << array[i].second << std::endl;
|
||||||
|
std::cout << "position: " << position << std::endl;
|
||||||
|
*/
|
||||||
|
size_t lower = static_cast<size_t>(std::floor(position));
|
||||||
|
size_t higher = static_cast<size_t>(std::ceil(position));
|
||||||
|
// std::cout << "lower: " << lower << ", higher: " << higher << std::endl;
|
||||||
|
|
||||||
|
const auto * lower_it = std::lower_bound(array, array + size, lower + 1, [](const Pair & a, size_t b) { return a.second < b; });
|
||||||
|
const auto * higher_it = std::lower_bound(array, array + size, higher + 1, [](const Pair & a, size_t b) { return a.second < b; });
|
||||||
|
if (lower_it == array + size)
|
||||||
|
lower_it = array + size - 1;
|
||||||
|
if (higher_it == array + size)
|
||||||
|
higher_it = array + size - 1;
|
||||||
|
// std::cout << "lower_index:" << lower_it - array << ", higher_index:" << higher_it - array << std::endl;
|
||||||
|
|
||||||
|
UnderlyingType lower_key = lower_it->first;
|
||||||
|
UnderlyingType higher_key = higher_it->first;
|
||||||
|
|
||||||
|
if (lower == higher)
|
||||||
|
return static_cast<Float64>(lower_key);
|
||||||
|
if (lower_key == higher_key)
|
||||||
|
return static_cast<Float64>(lower_key);
|
||||||
|
|
||||||
|
return (static_cast<Float64>(higher) - position) * lower_key + (position - static_cast<Float64>(lower)) * higher_key;
|
||||||
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
template <typename Value, bool _> using FuncQuantileExactWeighted = AggregateFunctionQuantile<Value, QuantileExactWeighted<Value>, NameQuantileExactWeighted, true, void, false, false>;
|
template <typename Value, bool return_float, bool interpolated>
|
||||||
template <typename Value, bool _> using FuncQuantilesExactWeighted = AggregateFunctionQuantile<Value, QuantileExactWeighted<Value>, NameQuantilesExactWeighted, true, void, true, false>;
|
using FuncQuantileExactWeighted = AggregateFunctionQuantile<
|
||||||
|
Value,
|
||||||
|
QuantileExactWeighted<Value, interpolated>,
|
||||||
|
NameQuantileExactWeighted,
|
||||||
|
true,
|
||||||
|
std::conditional_t<return_float, Float64, void>,
|
||||||
|
false,
|
||||||
|
false>;
|
||||||
|
template <typename Value, bool return_float, bool interpolated>
|
||||||
|
using FuncQuantilesExactWeighted = AggregateFunctionQuantile<
|
||||||
|
Value,
|
||||||
|
QuantileExactWeighted<Value, interpolated>,
|
||||||
|
NameQuantilesExactWeighted,
|
||||||
|
true,
|
||||||
|
std::conditional_t<return_float, Float64, void>,
|
||||||
|
true,
|
||||||
|
false>;
|
||||||
|
|
||||||
template <template <typename, bool> class Function>
|
template <template <typename, bool, bool> class Function, bool interpolated>
|
||||||
AggregateFunctionPtr createAggregateFunctionQuantile(
|
AggregateFunctionPtr createAggregateFunctionQuantile(
|
||||||
const std::string & name, const DataTypes & argument_types, const Array & params, const Settings *)
|
const std::string & name, const DataTypes & argument_types, const Array & params, const Settings *)
|
||||||
{
|
{
|
||||||
@ -224,22 +402,23 @@ AggregateFunctionPtr createAggregateFunctionQuantile(
|
|||||||
WhichDataType which(argument_type);
|
WhichDataType which(argument_type);
|
||||||
|
|
||||||
#define DISPATCH(TYPE) \
|
#define DISPATCH(TYPE) \
|
||||||
if (which.idx == TypeIndex::TYPE) return std::make_shared<Function<TYPE, true>>(argument_types, params);
|
if (which.idx == TypeIndex::TYPE) \
|
||||||
|
return std::make_shared<Function<TYPE, interpolated, interpolated>>(argument_types, params);
|
||||||
FOR_BASIC_NUMERIC_TYPES(DISPATCH)
|
FOR_BASIC_NUMERIC_TYPES(DISPATCH)
|
||||||
#undef DISPATCH
|
#undef DISPATCH
|
||||||
if (which.idx == TypeIndex::Date) return std::make_shared<Function<DataTypeDate::FieldType, false>>(argument_types, params);
|
if (which.idx == TypeIndex::Date) return std::make_shared<Function<DataTypeDate::FieldType, false, interpolated>>(argument_types, params);
|
||||||
if (which.idx == TypeIndex::DateTime) return std::make_shared<Function<DataTypeDateTime::FieldType, false>>(argument_types, params);
|
if (which.idx == TypeIndex::DateTime) return std::make_shared<Function<DataTypeDateTime::FieldType, false, interpolated>>(argument_types, params);
|
||||||
|
|
||||||
if (which.idx == TypeIndex::Decimal32) return std::make_shared<Function<Decimal32, false>>(argument_types, params);
|
if (which.idx == TypeIndex::Decimal32) return std::make_shared<Function<Decimal32, false, interpolated>>(argument_types, params);
|
||||||
if (which.idx == TypeIndex::Decimal64) return std::make_shared<Function<Decimal64, false>>(argument_types, params);
|
if (which.idx == TypeIndex::Decimal64) return std::make_shared<Function<Decimal64, false, interpolated>>(argument_types, params);
|
||||||
if (which.idx == TypeIndex::Decimal128) return std::make_shared<Function<Decimal128, false>>(argument_types, params);
|
if (which.idx == TypeIndex::Decimal128) return std::make_shared<Function<Decimal128, false, interpolated>>(argument_types, params);
|
||||||
if (which.idx == TypeIndex::Decimal256) return std::make_shared<Function<Decimal256, false>>(argument_types, params);
|
if (which.idx == TypeIndex::Decimal256) return std::make_shared<Function<Decimal256, false, interpolated>>(argument_types, params);
|
||||||
if (which.idx == TypeIndex::DateTime64) return std::make_shared<Function<DateTime64, false>>(argument_types, params);
|
if (which.idx == TypeIndex::DateTime64) return std::make_shared<Function<DateTime64, false, interpolated>>(argument_types, params);
|
||||||
|
|
||||||
if (which.idx == TypeIndex::Int128) return std::make_shared<Function<Int128, true>>(argument_types, params);
|
if (which.idx == TypeIndex::Int128) return std::make_shared<Function<Int128, interpolated, interpolated>>(argument_types, params);
|
||||||
if (which.idx == TypeIndex::UInt128) return std::make_shared<Function<UInt128, true>>(argument_types, params);
|
if (which.idx == TypeIndex::UInt128) return std::make_shared<Function<UInt128, interpolated, interpolated>>(argument_types, params);
|
||||||
if (which.idx == TypeIndex::Int256) return std::make_shared<Function<Int256, true>>(argument_types, params);
|
if (which.idx == TypeIndex::Int256) return std::make_shared<Function<Int256, interpolated, interpolated>>(argument_types, params);
|
||||||
if (which.idx == TypeIndex::UInt256) return std::make_shared<Function<UInt256, true>>(argument_types, params);
|
if (which.idx == TypeIndex::UInt256) return std::make_shared<Function<UInt256, interpolated, interpolated>>(argument_types, params);
|
||||||
|
|
||||||
throw Exception(ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT, "Illegal type {} of argument for aggregate function {}",
|
throw Exception(ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT, "Illegal type {} of argument for aggregate function {}",
|
||||||
argument_type->getName(), name);
|
argument_type->getName(), name);
|
||||||
@ -252,11 +431,17 @@ void registerAggregateFunctionsQuantileExactWeighted(AggregateFunctionFactory &
|
|||||||
/// For aggregate functions returning array we cannot return NULL on empty set.
|
/// For aggregate functions returning array we cannot return NULL on empty set.
|
||||||
AggregateFunctionProperties properties = { .returns_default_when_only_null = true };
|
AggregateFunctionProperties properties = { .returns_default_when_only_null = true };
|
||||||
|
|
||||||
factory.registerFunction(NameQuantileExactWeighted::name, createAggregateFunctionQuantile<FuncQuantileExactWeighted>);
|
factory.registerFunction(NameQuantileExactWeighted::name, createAggregateFunctionQuantile<FuncQuantileExactWeighted, false>);
|
||||||
factory.registerFunction(NameQuantilesExactWeighted::name, { createAggregateFunctionQuantile<FuncQuantilesExactWeighted>, properties });
|
factory.registerFunction(
|
||||||
|
NameQuantilesExactWeighted::name, {createAggregateFunctionQuantile<FuncQuantilesExactWeighted, false>, properties});
|
||||||
|
|
||||||
|
factory.registerFunction(NameQuantileExactWeightedInterpolated::name, createAggregateFunctionQuantile<FuncQuantileExactWeighted, true>);
|
||||||
|
factory.registerFunction(
|
||||||
|
NameQuantilesExactWeightedInterpolated::name, {createAggregateFunctionQuantile<FuncQuantilesExactWeighted, true>, properties});
|
||||||
|
|
||||||
/// 'median' is an alias for 'quantile'
|
/// 'median' is an alias for 'quantile'
|
||||||
factory.registerAlias("medianExactWeighted", NameQuantileExactWeighted::name);
|
factory.registerAlias("medianExactWeighted", NameQuantileExactWeighted::name);
|
||||||
|
factory.registerAlias("medianExactWeightedInterpolated", NameQuantileExactWeightedInterpolated::name);
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
@ -1,4 +1,6 @@
|
|||||||
10 [1,1,1,1,10,10,10,10,100,100,100]
|
10 [1,1,1,1,10,10,10,10,100,100,100]
|
||||||
10 [1,1,2,4,7,10,35,61,87,100,100]
|
10 [1,1,2,4,7,10,35,61,87,100,100]
|
||||||
|
10 [1,1,1,7.299999999999997,10,10,10,36.999999999999986,100,100,100]
|
||||||
|
10 [1,1,1,7.299999999999997,10,10,10,36.999999999999986,100,100,100]
|
||||||
100 100
|
100 100
|
||||||
61 61
|
61 61
|
||||||
|
@ -1,4 +1,6 @@
|
|||||||
SELECT quantileExactWeighted(0.5)(x, 1) AS q5, quantilesExactWeighted(0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1)(x, 1) AS qs FROM (SELECT arrayJoin([1, 1, 1, 10, 10, 10, 10, 100, 100, 100]) AS x);
|
SELECT quantileExactWeighted(0.5)(x, 1) AS q5, quantilesExactWeighted(0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1)(x, 1) AS qs FROM (SELECT arrayJoin([1, 1, 1, 10, 10, 10, 10, 100, 100, 100]) AS x);
|
||||||
SELECT quantileInterpolatedWeighted(0.5)(x, 1) AS q5, quantilesInterpolatedWeighted(0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1)(x, 1) AS qs FROM (SELECT arrayJoin([1, 1, 1, 10, 10, 10, 10, 100, 100, 100]) AS x);
|
SELECT quantileInterpolatedWeighted(0.5)(x, 1) AS q5, quantilesInterpolatedWeighted(0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1)(x, 1) AS qs FROM (SELECT arrayJoin([1, 1, 1, 10, 10, 10, 10, 100, 100, 100]) AS x);
|
||||||
|
SELECT quantileExactWeightedInterpolated(0.5)(x, 1) AS q5, quantilesExactWeightedInterpolated(0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1)(x, 1) AS qs FROM (SELECT arrayJoin([1, 1, 1, 10, 10, 10, 10, 100, 100, 100]) AS x);
|
||||||
|
SELECT quantile(0.5)(x) AS q5, quantiles(0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1)(x) AS qs FROM (SELECT arrayJoin([1, 1, 1, 10, 10, 10, 10, 100, 100, 100]) AS x);
|
||||||
SELECT quantileExact(0)(x), quantileTiming(0)(x) FROM (SELECT number + 100 AS x FROM system.numbers LIMIT 10000);
|
SELECT quantileExact(0)(x), quantileTiming(0)(x) FROM (SELECT number + 100 AS x FROM system.numbers LIMIT 10000);
|
||||||
SELECT quantileExact(x), quantileTiming(x) FROM (SELECT number % 123 AS x FROM system.numbers LIMIT 10000);
|
SELECT quantileExact(x), quantileTiming(x) FROM (SELECT number % 123 AS x FROM system.numbers LIMIT 10000);
|
||||||
|
@ -8,6 +8,8 @@
|
|||||||
['2016-06-15 23:00:00']
|
['2016-06-15 23:00:00']
|
||||||
2016-06-15 23:00:00
|
2016-06-15 23:00:00
|
||||||
['2016-06-15 23:00:00']
|
['2016-06-15 23:00:00']
|
||||||
|
2016-06-15 23:00:00
|
||||||
|
['2016-06-15 23:00:00']
|
||||||
30000
|
30000
|
||||||
[30000]
|
[30000]
|
||||||
30000
|
30000
|
||||||
|
@ -18,6 +18,9 @@ SELECT quantilesExactWeighted(0.2)(d, 1) FROM datetime;
|
|||||||
SELECT quantileInterpolatedWeighted(0.2)(d, 1) FROM datetime;
|
SELECT quantileInterpolatedWeighted(0.2)(d, 1) FROM datetime;
|
||||||
SELECT quantilesInterpolatedWeighted(0.2)(d, 1) FROM datetime;
|
SELECT quantilesInterpolatedWeighted(0.2)(d, 1) FROM datetime;
|
||||||
|
|
||||||
|
SELECT quantileExactWeightedInterpolated(0.2)(d, 1) FROM datetime;
|
||||||
|
SELECT quantilesExactWeightedInterpolated(0.2)(d, 1) FROM datetime;
|
||||||
|
|
||||||
SELECT quantileTiming(0.2)(d) FROM datetime;
|
SELECT quantileTiming(0.2)(d) FROM datetime;
|
||||||
SELECT quantilesTiming(0.2)(d) FROM datetime;
|
SELECT quantilesTiming(0.2)(d) FROM datetime;
|
||||||
|
|
||||||
|
@ -10,3 +10,15 @@ quantileInterpolatedWeighted
|
|||||||
[-50,-40.4,-30.3,-20.2,-10.1,0,10.1,20.2,30.3,40.4,50]
|
[-50,-40.4,-30.3,-20.2,-10.1,0,10.1,20.2,30.3,40.4,50]
|
||||||
[-16.66666666,-13.46666666,-10.09999999,-6.73333332,-3.36666666,0,3.36666666,6.73333332,10.09999999,13.46666666,16.66666666]
|
[-16.66666666,-13.46666666,-10.09999999,-6.73333332,-3.36666666,0,3.36666666,6.73333332,10.09999999,13.46666666,16.66666666]
|
||||||
[-10,-8.08,-6.06,-4.04,-2.02,0,2.02,4.04,6.06,8.08,10]
|
[-10,-8.08,-6.06,-4.04,-2.02,0,2.02,4.04,6.06,8.08,10]
|
||||||
|
quantileExactWeightedInterpolated
|
||||||
|
0 0 0 Decimal(38, 8)
|
||||||
|
-25.5 -8.49999999 -5.1 Decimal(38, 8)
|
||||||
|
0 0 0
|
||||||
|
10 3.33333333 2
|
||||||
|
20 6.66666666 4
|
||||||
|
30 10 6
|
||||||
|
40 13.33333333 8
|
||||||
|
50 16.66666666 10
|
||||||
|
[-50,-40,-30,-20,-10,0,10,20,30,40,50]
|
||||||
|
[-16.66666666,-13.33333333,-10,-6.66666666,-3.33333333,0,3.33333333,6.66666666,10,13.33333333,16.66666666]
|
||||||
|
[-10,-8,-6,-4,-2,0,2,4,6,8,10]
|
||||||
|
@ -24,4 +24,17 @@ SELECT quantilesInterpolatedWeighted(0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8
|
|||||||
SELECT quantilesInterpolatedWeighted(0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)(b, 2) FROM decimal;
|
SELECT quantilesInterpolatedWeighted(0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)(b, 2) FROM decimal;
|
||||||
SELECT quantilesInterpolatedWeighted(0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)(c, 3) FROM decimal;
|
SELECT quantilesInterpolatedWeighted(0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)(c, 3) FROM decimal;
|
||||||
|
|
||||||
|
SELECT 'quantileExactWeightedInterpolated';
|
||||||
|
SELECT medianExactWeightedInterpolated(a, 1), medianExactWeightedInterpolated(b, 2), medianExactWeightedInterpolated(c, 3) as x, toTypeName(x) FROM decimal;
|
||||||
|
SELECT quantileExactWeightedInterpolated(a, 1), quantileExactWeightedInterpolated(b, 2), quantileExactWeightedInterpolated(c, 3) as x, toTypeName(x) FROM decimal WHERE a < 0;
|
||||||
|
SELECT quantileExactWeightedInterpolated(0.0)(a, 1), quantileExactWeightedInterpolated(0.0)(b, 2), quantileExactWeightedInterpolated(0.0)(c, 3) FROM decimal WHERE a >= 0;
|
||||||
|
SELECT quantileExactWeightedInterpolated(0.2)(a, 1), quantileExactWeightedInterpolated(0.2)(b, 2), quantileExactWeightedInterpolated(0.2)(c, 3) FROM decimal WHERE a >= 0;
|
||||||
|
SELECT quantileExactWeightedInterpolated(0.4)(a, 1), quantileExactWeightedInterpolated(0.4)(b, 2), quantileExactWeightedInterpolated(0.4)(c, 3) FROM decimal WHERE a >= 0;
|
||||||
|
SELECT quantileExactWeightedInterpolated(0.6)(a, 1), quantileExactWeightedInterpolated(0.6)(b, 2), quantileExactWeightedInterpolated(0.6)(c, 3) FROM decimal WHERE a >= 0;
|
||||||
|
SELECT quantileExactWeightedInterpolated(0.8)(a, 1), quantileExactWeightedInterpolated(0.8)(b, 2), quantileExactWeightedInterpolated(0.8)(c, 3) FROM decimal WHERE a >= 0;
|
||||||
|
SELECT quantileExactWeightedInterpolated(1.0)(a, 1), quantileExactWeightedInterpolated(1.0)(b, 2), quantileExactWeightedInterpolated(1.0)(c, 3) FROM decimal WHERE a >= 0;
|
||||||
|
SELECT quantilesExactWeightedInterpolated(0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)(a, 1) FROM decimal;
|
||||||
|
SELECT quantilesExactWeightedInterpolated(0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)(b, 2) FROM decimal;
|
||||||
|
SELECT quantilesExactWeightedInterpolated(0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)(c, 3) FROM decimal;
|
||||||
|
|
||||||
DROP TABLE IF EXISTS decimal;
|
DROP TABLE IF EXISTS decimal;
|
||||||
|
@ -1,4 +1,4 @@
|
|||||||
personal_ws-1.1 en 2983
|
personal_ws-1.1 en 3036
|
||||||
AArch
|
AArch
|
||||||
ACLs
|
ACLs
|
||||||
ALTERs
|
ALTERs
|
||||||
@ -24,7 +24,6 @@ Aggregatefunction
|
|||||||
AggregatingMergeTree
|
AggregatingMergeTree
|
||||||
AggregatorThreads
|
AggregatorThreads
|
||||||
AggregatorThreadsActive
|
AggregatorThreadsActive
|
||||||
AzureQueue
|
|
||||||
Akka
|
Akka
|
||||||
AlertManager
|
AlertManager
|
||||||
Alexey
|
Alexey
|
||||||
@ -48,6 +47,7 @@ AutoFDO
|
|||||||
AutoML
|
AutoML
|
||||||
Autocompletion
|
Autocompletion
|
||||||
AvroConfluent
|
AvroConfluent
|
||||||
|
AzureQueue
|
||||||
BIGINT
|
BIGINT
|
||||||
BIGSERIAL
|
BIGSERIAL
|
||||||
BORO
|
BORO
|
||||||
@ -115,13 +115,13 @@ CESU
|
|||||||
CIDR
|
CIDR
|
||||||
CIDRToRange
|
CIDRToRange
|
||||||
CKMAN
|
CKMAN
|
||||||
|
CKibana
|
||||||
CLOB
|
CLOB
|
||||||
CLion
|
CLion
|
||||||
CMPLNT
|
CMPLNT
|
||||||
CMake
|
CMake
|
||||||
CMakeLists
|
CMakeLists
|
||||||
CODECS
|
CODECS
|
||||||
CountMin
|
|
||||||
COVID
|
COVID
|
||||||
CPUFrequencyMHz
|
CPUFrequencyMHz
|
||||||
CPUs
|
CPUs
|
||||||
@ -153,7 +153,6 @@ ChannelID
|
|||||||
Cidr
|
Cidr
|
||||||
Ciphertext
|
Ciphertext
|
||||||
CityHash
|
CityHash
|
||||||
CKibana
|
|
||||||
Clangd
|
Clangd
|
||||||
ClickBench
|
ClickBench
|
||||||
ClickCat
|
ClickCat
|
||||||
@ -186,6 +185,7 @@ ConnectionDetails
|
|||||||
Const
|
Const
|
||||||
ContextLockWait
|
ContextLockWait
|
||||||
Contrib
|
Contrib
|
||||||
|
CountMin
|
||||||
Covid
|
Covid
|
||||||
Cramer's
|
Cramer's
|
||||||
Criteo
|
Criteo
|
||||||
@ -250,12 +250,12 @@ DoubleDelta
|
|||||||
Doxygen
|
Doxygen
|
||||||
Durre
|
Durre
|
||||||
ECMA
|
ECMA
|
||||||
ElasticSearch
|
|
||||||
ETag
|
ETag
|
||||||
Ecto
|
Ecto
|
||||||
EdgeAngle
|
EdgeAngle
|
||||||
EdgeLengthKm
|
EdgeLengthKm
|
||||||
EdgeLengthM
|
EdgeLengthM
|
||||||
|
ElasticSearch
|
||||||
EmbeddedRocksDB
|
EmbeddedRocksDB
|
||||||
Embeddings
|
Embeddings
|
||||||
Encodings
|
Encodings
|
||||||
@ -423,9 +423,9 @@ JSONCompactStrings
|
|||||||
JSONCompactStringsEachRow
|
JSONCompactStringsEachRow
|
||||||
JSONCompactStringsEachRowWithNames
|
JSONCompactStringsEachRowWithNames
|
||||||
JSONCompactStringsEachRowWithNamesAndTypes
|
JSONCompactStringsEachRowWithNamesAndTypes
|
||||||
|
JSONCompactWithProgress
|
||||||
JSONDynamicPaths
|
JSONDynamicPaths
|
||||||
JSONDynamicPathsWithTypes
|
JSONDynamicPathsWithTypes
|
||||||
JSONCompactWithProgress
|
|
||||||
JSONEachRow
|
JSONEachRow
|
||||||
JSONEachRowWithProgress
|
JSONEachRowWithProgress
|
||||||
JSONExtract
|
JSONExtract
|
||||||
@ -442,11 +442,11 @@ JSONExtractUInt
|
|||||||
JSONHas
|
JSONHas
|
||||||
JSONLength
|
JSONLength
|
||||||
JSONObjectEachRow
|
JSONObjectEachRow
|
||||||
|
JSONSharedDataPaths
|
||||||
|
JSONSharedDataPathsWithTypes
|
||||||
JSONStrings
|
JSONStrings
|
||||||
JSONStringsEachRow
|
JSONStringsEachRow
|
||||||
JSONStringsEachRowWithProgress
|
JSONStringsEachRowWithProgress
|
||||||
JSONSharedDataPaths
|
|
||||||
JSONSharedDataPathsWithTypes
|
|
||||||
JSONType
|
JSONType
|
||||||
JSONs
|
JSONs
|
||||||
Jaeger
|
Jaeger
|
||||||
@ -981,8 +981,8 @@ ThreadPoolRemoteFSReaderThreads
|
|||||||
ThreadPoolRemoteFSReaderThreadsActive
|
ThreadPoolRemoteFSReaderThreadsActive
|
||||||
ThreadsActive
|
ThreadsActive
|
||||||
ThreadsInOvercommitTracker
|
ThreadsInOvercommitTracker
|
||||||
TimescaleDB's
|
|
||||||
TimeSeries
|
TimeSeries
|
||||||
|
TimescaleDB's
|
||||||
Timeunit
|
Timeunit
|
||||||
TinyLog
|
TinyLog
|
||||||
Tkachenko
|
Tkachenko
|
||||||
@ -1547,6 +1547,7 @@ dequeues
|
|||||||
deserialization
|
deserialization
|
||||||
deserialized
|
deserialized
|
||||||
deserializing
|
deserializing
|
||||||
|
dest
|
||||||
destructor
|
destructor
|
||||||
destructors
|
destructors
|
||||||
detectCharset
|
detectCharset
|
||||||
@ -1572,12 +1573,12 @@ disjunction
|
|||||||
disjunctions
|
disjunctions
|
||||||
displayName
|
displayName
|
||||||
displaySecretsInShowAndSelect
|
displaySecretsInShowAndSelect
|
||||||
distro
|
|
||||||
distinctdynamictypes
|
|
||||||
distinctDynamicTypes
|
distinctDynamicTypes
|
||||||
distinctjsonpaths
|
|
||||||
distinctJSONPaths
|
distinctJSONPaths
|
||||||
distinctJSONPathsAndTypes
|
distinctJSONPathsAndTypes
|
||||||
|
distinctdynamictypes
|
||||||
|
distinctjsonpaths
|
||||||
|
distro
|
||||||
divideDecimal
|
divideDecimal
|
||||||
dmesg
|
dmesg
|
||||||
doesnt
|
doesnt
|
||||||
@ -2361,6 +2362,7 @@ quantileExactHigh
|
|||||||
quantileExactInclusive
|
quantileExactInclusive
|
||||||
quantileExactLow
|
quantileExactLow
|
||||||
quantileExactWeighted
|
quantileExactWeighted
|
||||||
|
quantileExactWeightedInterpolated
|
||||||
quantileGK
|
quantileGK
|
||||||
quantileInterpolatedWeighted
|
quantileInterpolatedWeighted
|
||||||
quantileTDigest
|
quantileTDigest
|
||||||
@ -2372,6 +2374,7 @@ quantileddsketch
|
|||||||
quantiledeterministic
|
quantiledeterministic
|
||||||
quantileexact
|
quantileexact
|
||||||
quantileexactweighted
|
quantileexactweighted
|
||||||
|
quantileexactweightedInterpolated
|
||||||
quantiles
|
quantiles
|
||||||
quantilesExactExclusive
|
quantilesExactExclusive
|
||||||
quantilesExactInclusive
|
quantilesExactInclusive
|
||||||
@ -2592,7 +2595,6 @@ sqlinsert
|
|||||||
sqlite
|
sqlite
|
||||||
sqrt
|
sqrt
|
||||||
src
|
src
|
||||||
dest
|
|
||||||
srcReplicas
|
srcReplicas
|
||||||
sshkey
|
sshkey
|
||||||
stackoverflow
|
stackoverflow
|
||||||
|
Loading…
Reference in New Issue
Block a user