mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-09-20 16:50:48 +00:00
Merge branch 'master' into standalone_keeper
This commit is contained in:
commit
62004cf0d9
2
contrib/simdjson
vendored
2
contrib/simdjson
vendored
@ -1 +1 @@
|
||||
Subproject commit 95b4870e20be5f97d9dcf63b23b1c6f520c366c1
|
||||
Subproject commit 8df32cea3359cb30120795da6020b3b73da01d38
|
@ -1,6 +1,6 @@
|
||||
# How to add test queries to ClickHouse CI
|
||||
|
||||
ClickHouse has hundreds (or even thousands) of features. Every commit get checked by a complex set of tests containing many thousands of test cases.
|
||||
ClickHouse has hundreds (or even thousands) of features. Every commit gets checked by a complex set of tests containing many thousands of test cases.
|
||||
|
||||
The core functionality is very well tested, but some corner-cases and different combinations of features can be uncovered with ClickHouse CI.
|
||||
|
||||
@ -105,13 +105,13 @@ clickhouse-client -nmT < tests/queries/0_stateless/01521_dummy_test.sql | tee te
|
||||
|
||||
5) ensure everything is correct, if the test output is incorrect (due to some bug for example), adjust the reference file using text editor.
|
||||
|
||||
#### How create good test
|
||||
#### How to create good test
|
||||
|
||||
- test should be
|
||||
- minimal - create only tables related to tested functionality, remove unrelated columns and parts of query
|
||||
- fast - should not take longer than few seconds (better subseconds)
|
||||
- correct - fails then feature is not working
|
||||
- deteministic
|
||||
- deterministic
|
||||
- isolated / stateless
|
||||
- don't rely on some environment things
|
||||
- don't rely on timing when possible
|
||||
@ -124,7 +124,7 @@ clickhouse-client -nmT < tests/queries/0_stateless/01521_dummy_test.sql | tee te
|
||||
- clean up the created objects after test and before the test (DROP IF EXISTS) - in case of some dirty state
|
||||
- prefer sync mode of operations (mutations, merges, etc.)
|
||||
- use other SQL files in the `0_stateless` folder as an example
|
||||
- ensure the feature / feature combination you want to tests is not covered yet with existsing tests
|
||||
- ensure the feature / feature combination you want to tests is not covered yet with existing tests
|
||||
|
||||
#### Commit / push / create PR.
|
||||
|
||||
|
@ -135,6 +135,39 @@ Default value: 604800 (1 week).
|
||||
|
||||
Similar to [replicated_deduplication_window](#replicated-deduplication-window), `replicated_deduplication_window_seconds` specifies how long to store hash sums of blocks for insert deduplication. Hash sums older than `replicated_deduplication_window_seconds` are removed from Zookeeper, even if they are less than ` replicated_deduplication_window`.
|
||||
|
||||
## replicated_fetches_http_connection_timeout {#replicated_fetches_http_connection_timeout}
|
||||
|
||||
HTTP connection timeout (in seconds) for part fetch requests. Inherited from default profile [http_connection_timeout](./settings.md#http_connection_timeout) if not set explicitly.
|
||||
|
||||
Possible values:
|
||||
|
||||
- Any positive integer.
|
||||
- 0 - Use value of `http_connection_timeout`.
|
||||
|
||||
Default value: 0.
|
||||
|
||||
## replicated_fetches_http_send_timeout {#replicated_fetches_http_send_timeout}
|
||||
|
||||
HTTP send timeout (in seconds) for part fetch requests. Inherited from default profile [http_send_timeout](./settings.md#http_send_timeout) if not set explicitly.
|
||||
|
||||
Possible values:
|
||||
|
||||
- Any positive integer.
|
||||
- 0 - Use value of `http_send_timeout`.
|
||||
|
||||
Default value: 0.
|
||||
|
||||
## replicated_fetches_http_receive_timeout {#replicated_fetches_http_receive_timeout}
|
||||
|
||||
HTTP receive timeout (in seconds) for fetch part requests. Inherited from default profile [http_receive_timeout](./settings.md#http_receive_timeout) if not set explicitly.
|
||||
|
||||
Possible values:
|
||||
|
||||
- Any positive integer.
|
||||
- 0 - Use value of `http_receive_timeout`.
|
||||
|
||||
Default value: 0.
|
||||
|
||||
## old_parts_lifetime {#old-parts-lifetime}
|
||||
|
||||
The time (in seconds) of storing inactive parts to protect against data loss during spontaneous server reboots.
|
||||
|
@ -2863,6 +2863,39 @@ Sets the interval in seconds after which periodically refreshed [live view](../.
|
||||
|
||||
Default value: `60`.
|
||||
|
||||
## http_connection_timeout {#http_connection_timeout}
|
||||
|
||||
HTTP connection timeout (in seconds).
|
||||
|
||||
Possible values:
|
||||
|
||||
- Any positive integer.
|
||||
- 0 - Disabled (infinite timeout).
|
||||
|
||||
Default value: 1.
|
||||
|
||||
## http_send_timeout {#http_send_timeout}
|
||||
|
||||
HTTP send timeout (in seconds).
|
||||
|
||||
Possible values:
|
||||
|
||||
- Any positive integer.
|
||||
- 0 - Disabled (infinite timeout).
|
||||
|
||||
Default value: 1800.
|
||||
|
||||
## http_receive_timeout {#http_receive_timeout}
|
||||
|
||||
HTTP receive timeout (in seconds).
|
||||
|
||||
Possible values:
|
||||
|
||||
- Any positive integer.
|
||||
- 0 - Disabled (infinite timeout).
|
||||
|
||||
Default value: 1800.
|
||||
|
||||
## check_query_single_value_result {#check_query_single_value_result}
|
||||
|
||||
Defines the level of detail for the [CHECK TABLE](../../sql-reference/statements/check-table.md#checking-mergetree-tables) query result for `MergeTree` family engines .
|
||||
|
@ -31,7 +31,7 @@ For example, Decimal32(4) can contain numbers from -99999.9999 to 99999.9999 wit
|
||||
|
||||
Internally data is represented as normal signed integers with respective bit width. Real value ranges that can be stored in memory are a bit larger than specified above, which are checked only on conversion from a string.
|
||||
|
||||
Because modern CPU’s do not support 128-bit integers natively, operations on Decimal128 are emulated. Because of this Decimal128 works significantly slower than Decimal32/Decimal64.
|
||||
Because modern CPUs do not support 128-bit integers natively, operations on Decimal128 are emulated. Because of this Decimal128 works significantly slower than Decimal32/Decimal64.
|
||||
|
||||
## Operations and Result Type {#operations-and-result-type}
|
||||
|
||||
|
@ -149,6 +149,39 @@ Eсли суммарное число активных кусков во все
|
||||
|
||||
Стандартное значение Linux dirty_expire_centisecs - 30 секунд (максимальное время, которое записанные данные хранятся только в оперативной памяти), но при больших нагрузках на дисковую систему, данные могут быть записаны намного позже. Экспериментально было найдено время - 480 секунд, за которое гарантированно новый кусок будет записан на диск.
|
||||
|
||||
## replicated_fetches_http_connection_timeout {#replicated_fetches_http_connection_timeout}
|
||||
|
||||
Тайм-аут HTTP-соединения (в секундах) для запросов на скачивание кусков. Наследуется из профиля по умолчанию [http_connection_timeout](./settings.md#http_connection_timeout), если не задан явно.
|
||||
|
||||
Возможные значения:
|
||||
|
||||
- 0 - используется значение `http_connection_timeout`.
|
||||
- Любое положительное целое число.
|
||||
|
||||
Значение по умолчанию: `0`.
|
||||
|
||||
## replicated_fetches_http_send_timeout {#replicated_fetches_http_send_timeout}
|
||||
|
||||
Тайм-аут (в секундах) для отправки HTTP-запросов на скачивание кусков. Наследуется из профиля по умолчанию [http_send_timeout](./settings.md#http_send_timeout), если не задан явно.
|
||||
|
||||
Возможные значения:
|
||||
|
||||
- 0 - используется значение `http_send_timeout`.
|
||||
- Любое положительное целое число.
|
||||
|
||||
Значение по умолчанию: `0`.
|
||||
|
||||
## replicated_fetches_http_receive_timeout {#replicated_fetches_http_receive_timeout}
|
||||
|
||||
Тайм-аут (в секундах) для получения HTTP-запросов на скачивание кусков. Наследуется из профиля по умолчанию [http_receive_timeout](./settings.md#http_receive_timeout), если не задан явно.
|
||||
|
||||
Возможные значения:
|
||||
|
||||
- 0 - используется значение `http_receive_timeout`.
|
||||
- Любое положительное целое число.
|
||||
|
||||
Значение по умолчанию: `0`.
|
||||
|
||||
## max_bytes_to_merge_at_max_space_in_pool {#max-bytes-to-merge-at-max-space-in-pool}
|
||||
|
||||
Максимальный суммарный размер кусков (в байтах) в одном слиянии, при наличии свободных ресурсов в фоновом пуле.
|
||||
|
@ -2857,5 +2857,37 @@ SELECT * FROM test LIMIT 10 OFFSET 100;
|
||||
│ 109 │
|
||||
└─────┘
|
||||
```
|
||||
## http_connection_timeout {#http_connection_timeout}
|
||||
|
||||
Тайм-аут для HTTP-соединения (в секундах).
|
||||
|
||||
Возможные значения:
|
||||
|
||||
- 0 - бесконечный тайм-аут.
|
||||
- Любое положительное целое число.
|
||||
|
||||
Значение по умолчанию: `1`.
|
||||
|
||||
## http_send_timeout {#http_send_timeout}
|
||||
|
||||
Тайм-аут для отправки данных через HTTP-интерфейс (в секундах).
|
||||
|
||||
Возможные значения:
|
||||
|
||||
- 0 - бесконечный тайм-аут.
|
||||
- Любое положительное целое число.
|
||||
|
||||
Значение по умолчанию: `1800`.
|
||||
|
||||
## http_receive_timeout {#http_receive_timeout}
|
||||
|
||||
Тайм-аут для получения данных через HTTP-интерфейс (в секундах).
|
||||
|
||||
Возможные значения:
|
||||
|
||||
- 0 - бесконечный тайм-аут.
|
||||
- Любое положительное целое число.
|
||||
|
||||
Значение по умолчанию: `1800`.
|
||||
|
||||
[Оригинальная статья](https://clickhouse.tech/docs/ru/operations/settings/settings/) <!--hide-->
|
||||
|
@ -2278,7 +2278,7 @@ private:
|
||||
size_t elapsed_ns = watch.elapsed();
|
||||
if (elapsed_ns)
|
||||
std::cout << " (" << formatReadableQuantity(progress.read_rows * 1000000000.0 / elapsed_ns) << " rows/s., "
|
||||
<< formatReadableSizeWithDecimalSuffix(progress.read_bytes * 1000000000.0 / elapsed_ns) << "/s.) ";
|
||||
<< formatReadableSizeWithDecimalSuffix(progress.read_bytes * 1000000000.0 / elapsed_ns) << "/s.)";
|
||||
else
|
||||
std::cout << ". ";
|
||||
}
|
||||
|
@ -106,8 +106,8 @@ endif()
|
||||
list (APPEND clickhouse_common_io_sources ${CONFIG_BUILD})
|
||||
list (APPEND clickhouse_common_io_headers ${CONFIG_VERSION} ${CONFIG_COMMON})
|
||||
|
||||
list (APPEND dbms_sources Functions/IFunction.cpp Functions/FunctionFactory.cpp Functions/FunctionHelpers.cpp Functions/extractTimeZoneFromFunctionArguments.cpp Functions/replicate.cpp Functions/FunctionsLogical.cpp)
|
||||
list (APPEND dbms_headers Functions/IFunctionImpl.h Functions/FunctionFactory.h Functions/FunctionHelpers.h Functions/extractTimeZoneFromFunctionArguments.h Functions/replicate.h Functions/FunctionsLogical.h)
|
||||
list (APPEND dbms_sources Functions/IFunction.cpp Functions/IFunctionOld.cpp Functions/FunctionFactory.cpp Functions/FunctionHelpers.cpp Functions/extractTimeZoneFromFunctionArguments.cpp Functions/replicate.cpp Functions/FunctionsLogical.cpp)
|
||||
list (APPEND dbms_headers Functions/IFunctionOld.h Functions/FunctionFactory.h Functions/FunctionHelpers.h Functions/extractTimeZoneFromFunctionArguments.h Functions/replicate.h Functions/FunctionsLogical.h)
|
||||
|
||||
list (APPEND dbms_sources
|
||||
AggregateFunctions/AggregateFunctionFactory.cpp
|
||||
|
@ -18,7 +18,6 @@ PEERDIR(
|
||||
contrib/libs/openssl
|
||||
contrib/libs/poco/NetSSL_OpenSSL
|
||||
contrib/libs/re2
|
||||
contrib/libs/cxxsupp/libcxxabi-parts
|
||||
contrib/restricted/dragonbox
|
||||
)
|
||||
|
||||
|
@ -246,9 +246,19 @@ void DatabaseConnectionMySQL::fetchLatestTablesStructureIntoCache(
|
||||
local_tables_cache.erase(iterator);
|
||||
}
|
||||
|
||||
local_tables_cache[table_name] = std::make_pair(table_modification_time, StorageMySQL::create(
|
||||
StorageID(database_name, table_name), std::move(mysql_pool), database_name_in_mysql, table_name,
|
||||
false, "", ColumnsDescription{columns_name_and_type}, ConstraintsDescription{}, getContext()));
|
||||
local_tables_cache[table_name] = std::make_pair(
|
||||
table_modification_time,
|
||||
StorageMySQL::create(
|
||||
StorageID(database_name, table_name),
|
||||
std::move(mysql_pool),
|
||||
database_name_in_mysql,
|
||||
table_name,
|
||||
false,
|
||||
"",
|
||||
ColumnsDescription{columns_name_and_type},
|
||||
ConstraintsDescription{},
|
||||
String{},
|
||||
getContext()));
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -169,8 +169,13 @@ StoragePtr DatabasePostgreSQL::fetchTable(const String & table_name, ContextPtr
|
||||
return StoragePtr{};
|
||||
|
||||
auto storage = StoragePostgreSQL::create(
|
||||
StorageID(database_name, table_name), connection_pool, table_name,
|
||||
ColumnsDescription{*columns}, ConstraintsDescription{}, local_context);
|
||||
StorageID(database_name, table_name),
|
||||
connection_pool,
|
||||
table_name,
|
||||
ColumnsDescription{*columns},
|
||||
ConstraintsDescription{},
|
||||
String{},
|
||||
local_context);
|
||||
|
||||
if (cache_tables)
|
||||
cached_tables[table_name] = storage;
|
||||
|
@ -5,8 +5,8 @@ add_subdirectory(divide)
|
||||
include("${ClickHouse_SOURCE_DIR}/cmake/dbms_glob_sources.cmake")
|
||||
add_headers_and_sources(clickhouse_functions .)
|
||||
|
||||
list(REMOVE_ITEM clickhouse_functions_sources IFunctionImpl.cpp FunctionFactory.cpp FunctionHelpers.cpp)
|
||||
list(REMOVE_ITEM clickhouse_functions_headers IFunctionImpl.h FunctionFactory.h FunctionHelpers.h)
|
||||
list(REMOVE_ITEM clickhouse_functions_sources IFunctionOld.cpp FunctionFactory.cpp FunctionHelpers.cpp)
|
||||
list(REMOVE_ITEM clickhouse_functions_headers IFunctionOld.h FunctionFactory.h FunctionHelpers.h)
|
||||
|
||||
add_library(clickhouse_functions ${clickhouse_functions_sources})
|
||||
|
||||
|
@ -6,7 +6,7 @@
|
||||
#include <Core/DecimalFunctions.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/extractTimeZoneFromFunctionArguments.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Common/Exception.h>
|
||||
#include <common/DateLUTImpl.h>
|
||||
|
||||
|
@ -7,7 +7,7 @@
|
||||
#include <Columns/ColumnVector.h>
|
||||
#include <Columns/ColumnDecimal.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/extractTimeZoneFromFunctionArguments.h>
|
||||
#include <DataTypes/DataTypeDateTime.h>
|
||||
#include <DataTypes/DataTypeDateTime64.h>
|
||||
|
@ -23,7 +23,7 @@
|
||||
#include <Columns/ColumnConst.h>
|
||||
#include <Columns/ColumnAggregateFunction.h>
|
||||
#include "Core/DecimalFunctions.h"
|
||||
#include "IFunctionImpl.h"
|
||||
#include "IFunctionOld.h"
|
||||
#include "FunctionHelpers.h"
|
||||
#include "IsOperation.h"
|
||||
#include "DivisionUtils.h"
|
||||
@ -1532,11 +1532,11 @@ private:
|
||||
};
|
||||
|
||||
template <template <typename, typename> class Op, typename Name, bool valid_on_default_arguments = true, bool valid_on_float_arguments = true>
|
||||
class BinaryArithmeticOverloadResolver : public IFunctionOverloadResolverImpl
|
||||
class BinaryArithmeticOverloadResolver : public IFunctionOverloadResolver
|
||||
{
|
||||
public:
|
||||
static constexpr auto name = Name::name;
|
||||
static FunctionOverloadResolverImplPtr create(ContextPtr context)
|
||||
static FunctionOverloadResolverPtr create(ContextPtr context)
|
||||
{
|
||||
return std::make_unique<BinaryArithmeticOverloadResolver>(context);
|
||||
}
|
||||
@ -1547,27 +1547,27 @@ public:
|
||||
size_t getNumberOfArguments() const override { return 2; }
|
||||
bool isVariadic() const override { return false; }
|
||||
|
||||
FunctionBaseImplPtr build(const ColumnsWithTypeAndName & arguments, const DataTypePtr & return_type) const override
|
||||
FunctionBasePtr buildImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & return_type) const override
|
||||
{
|
||||
/// More efficient specialization for two numeric arguments.
|
||||
if (arguments.size() == 2
|
||||
&& ((arguments[0].column && isColumnConst(*arguments[0].column))
|
||||
|| (arguments[1].column && isColumnConst(*arguments[1].column))))
|
||||
{
|
||||
return std::make_unique<DefaultFunction>(
|
||||
return std::make_unique<FunctionToFunctionBaseAdaptor>(
|
||||
FunctionBinaryArithmeticWithConstants<Op, Name, valid_on_default_arguments, valid_on_float_arguments>::create(
|
||||
arguments[0], arguments[1], return_type, context),
|
||||
ext::map<DataTypes>(arguments, [](const auto & elem) { return elem.type; }),
|
||||
return_type);
|
||||
}
|
||||
|
||||
return std::make_unique<DefaultFunction>(
|
||||
return std::make_unique<FunctionToFunctionBaseAdaptor>(
|
||||
FunctionBinaryArithmetic<Op, Name, valid_on_default_arguments, valid_on_float_arguments>::create(context),
|
||||
ext::map<DataTypes>(arguments, [](const auto & elem) { return elem.type; }),
|
||||
return_type);
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType(const DataTypes & arguments) const override
|
||||
DataTypePtr getReturnTypeImpl(const DataTypes & arguments) const override
|
||||
{
|
||||
if (arguments.size() != 2)
|
||||
throw Exception(
|
||||
|
@ -2,7 +2,7 @@
|
||||
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Columns/ColumnVector.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include <ext/range.h>
|
||||
|
@ -3,7 +3,7 @@
|
||||
#include <DataTypes/DataTypeDateTime.h>
|
||||
#include <DataTypes/DataTypeDateTime64.h>
|
||||
#include <Functions/CustomWeekTransforms.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/TransformDateTime64.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
|
||||
|
@ -7,7 +7,7 @@
|
||||
|
||||
#include <Columns/ColumnsNumber.h>
|
||||
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/castTypeToEither.h>
|
||||
#include <Functions/extractTimeZoneFromFunctionArguments.h>
|
||||
|
@ -1,7 +1,7 @@
|
||||
#pragma once
|
||||
#include <DataTypes/DataTypeDate.h>
|
||||
#include <DataTypes/DataTypeDateTime.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <DataTypes/DataTypeDateTime64.h>
|
||||
#include <Functions/extractTimeZoneFromFunctionArguments.h>
|
||||
#include <Functions/DateTimeTransforms.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <common/getFQDNOrHostName.h>
|
||||
|
@ -50,7 +50,7 @@ void FunctionFactory::registerFunction(const
|
||||
}
|
||||
|
||||
|
||||
FunctionOverloadResolverImplPtr FunctionFactory::getImpl(
|
||||
FunctionOverloadResolverPtr FunctionFactory::getImpl(
|
||||
const std::string & name,
|
||||
ContextPtr context) const
|
||||
{
|
||||
@ -84,15 +84,15 @@ FunctionOverloadResolverPtr FunctionFactory::get(
|
||||
const std::string & name,
|
||||
ContextPtr context) const
|
||||
{
|
||||
return std::make_shared<FunctionOverloadResolverAdaptor>(getImpl(name, context));
|
||||
return getImpl(name, context);
|
||||
}
|
||||
|
||||
FunctionOverloadResolverImplPtr FunctionFactory::tryGetImpl(
|
||||
FunctionOverloadResolverPtr FunctionFactory::tryGetImpl(
|
||||
const std::string & name_param,
|
||||
ContextPtr context) const
|
||||
{
|
||||
String name = getAliasToOrName(name_param);
|
||||
FunctionOverloadResolverImplPtr res;
|
||||
FunctionOverloadResolverPtr res;
|
||||
|
||||
auto it = functions.find(name);
|
||||
if (functions.end() != it)
|
||||
@ -123,8 +123,7 @@ FunctionOverloadResolverPtr FunctionFactory::tryGet(
|
||||
ContextPtr context) const
|
||||
{
|
||||
auto impl = tryGetImpl(name, context);
|
||||
return impl ? std::make_shared<FunctionOverloadResolverAdaptor>(std::move(impl))
|
||||
: nullptr;
|
||||
return impl ? std::move(impl) : nullptr;
|
||||
}
|
||||
|
||||
FunctionFactory & FunctionFactory::instance()
|
||||
|
@ -1,8 +1,10 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/IFunctionAdaptors.h>
|
||||
#include <Interpreters/Context_fwd.h>
|
||||
#include <Common/IFactoryWithAliases.h>
|
||||
#include <Functions/IFunction.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/IFunctionAdaptors.h>
|
||||
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
@ -18,7 +20,7 @@ namespace DB
|
||||
* some dictionaries from Context.
|
||||
*/
|
||||
class FunctionFactory : private boost::noncopyable,
|
||||
public IFactoryWithAliases<std::function<FunctionOverloadResolverImplPtr(ContextPtr)>>
|
||||
public IFactoryWithAliases<std::function<FunctionOverloadResolverPtr(ContextPtr)>>
|
||||
{
|
||||
public:
|
||||
static FunctionFactory & instance();
|
||||
@ -32,8 +34,9 @@ public:
|
||||
template <typename Function>
|
||||
void registerFunction(const std::string & name, CaseSensitiveness case_sensitiveness = CaseSensitive)
|
||||
{
|
||||
if constexpr (std::is_base_of<IFunction, Function>::value)
|
||||
registerFunction(name, &createDefaultFunction<Function>, case_sensitiveness);
|
||||
|
||||
if constexpr (std::is_base_of_v<IFunction, Function>)
|
||||
registerFunction(name, &adaptFunctionToOverloadResolver<Function>, case_sensitiveness);
|
||||
else
|
||||
registerFunction(name, &Function::create, case_sensitiveness);
|
||||
}
|
||||
@ -48,8 +51,8 @@ public:
|
||||
FunctionOverloadResolverPtr tryGet(const std::string & name, ContextPtr context) const;
|
||||
|
||||
/// The same methods to get developer interface implementation.
|
||||
FunctionOverloadResolverImplPtr getImpl(const std::string & name, ContextPtr context) const;
|
||||
FunctionOverloadResolverImplPtr tryGetImpl(const std::string & name, ContextPtr context) const;
|
||||
FunctionOverloadResolverPtr getImpl(const std::string & name, ContextPtr context) const;
|
||||
FunctionOverloadResolverPtr tryGetImpl(const std::string & name, ContextPtr context) const;
|
||||
|
||||
/// Register a function by its name.
|
||||
/// No locking, you must register all functions before usage of get.
|
||||
@ -65,9 +68,9 @@ private:
|
||||
Functions case_insensitive_functions;
|
||||
|
||||
template <typename Function>
|
||||
static FunctionOverloadResolverImplPtr createDefaultFunction(ContextPtr context)
|
||||
static FunctionOverloadResolverPtr adaptFunctionToOverloadResolver(ContextPtr context)
|
||||
{
|
||||
return std::make_unique<DefaultOverloadResolver>(Function::create(context));
|
||||
return std::make_unique<FunctionToOverloadResolverAdaptor>(Function::create(context));
|
||||
}
|
||||
|
||||
const Functions & getMap() const override { return functions; }
|
||||
|
@ -1,5 +1,5 @@
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Columns/ColumnTuple.h>
|
||||
#include <Columns/ColumnString.h>
|
||||
#include <Columns/ColumnFixedString.h>
|
||||
@ -231,4 +231,75 @@ bool areTypesEqual(const DataTypePtr & lhs, const DataTypePtr & rhs)
|
||||
return lhs_name == rhs_name;
|
||||
}
|
||||
|
||||
ColumnPtr wrapInNullable(const ColumnPtr & src, const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count)
|
||||
{
|
||||
ColumnPtr result_null_map_column;
|
||||
|
||||
/// If result is already nullable.
|
||||
ColumnPtr src_not_nullable = src;
|
||||
|
||||
if (src->onlyNull())
|
||||
return src;
|
||||
else if (const auto * nullable = checkAndGetColumn<ColumnNullable>(*src))
|
||||
{
|
||||
src_not_nullable = nullable->getNestedColumnPtr();
|
||||
result_null_map_column = nullable->getNullMapColumnPtr();
|
||||
}
|
||||
|
||||
for (const auto & elem : args)
|
||||
{
|
||||
if (!elem.type->isNullable())
|
||||
continue;
|
||||
|
||||
/// Const Nullable that are NULL.
|
||||
if (elem.column->onlyNull())
|
||||
{
|
||||
assert(result_type->isNullable());
|
||||
return result_type->createColumnConstWithDefaultValue(input_rows_count);
|
||||
}
|
||||
|
||||
if (isColumnConst(*elem.column))
|
||||
continue;
|
||||
|
||||
if (const auto * nullable = checkAndGetColumn<ColumnNullable>(*elem.column))
|
||||
{
|
||||
const ColumnPtr & null_map_column = nullable->getNullMapColumnPtr();
|
||||
if (!result_null_map_column) //-V1051
|
||||
{
|
||||
result_null_map_column = null_map_column;
|
||||
}
|
||||
else
|
||||
{
|
||||
MutableColumnPtr mutable_result_null_map_column = IColumn::mutate(std::move(result_null_map_column));
|
||||
|
||||
NullMap & result_null_map = assert_cast<ColumnUInt8 &>(*mutable_result_null_map_column).getData();
|
||||
const NullMap & src_null_map = assert_cast<const ColumnUInt8 &>(*null_map_column).getData();
|
||||
|
||||
for (size_t i = 0, size = result_null_map.size(); i < size; ++i)
|
||||
result_null_map[i] |= src_null_map[i];
|
||||
|
||||
result_null_map_column = std::move(mutable_result_null_map_column);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!result_null_map_column)
|
||||
return makeNullable(src);
|
||||
|
||||
return ColumnNullable::create(src_not_nullable->convertToFullColumnIfConst(), result_null_map_column);
|
||||
}
|
||||
|
||||
NullPresence getNullPresense(const ColumnsWithTypeAndName & args)
|
||||
{
|
||||
NullPresence res;
|
||||
|
||||
for (const auto & elem : args)
|
||||
{
|
||||
res.has_nullable |= elem.type->isNullable();
|
||||
res.has_null_constant |= elem.type->onlyNull();
|
||||
}
|
||||
|
||||
return res;
|
||||
}
|
||||
|
||||
}
|
||||
|
@ -156,4 +156,17 @@ checkAndGetNestedArrayOffset(const IColumn ** columns, size_t num_arguments);
|
||||
/// Check if two types are equal
|
||||
bool areTypesEqual(const DataTypePtr & lhs, const DataTypePtr & rhs);
|
||||
|
||||
/** Return ColumnNullable of src, with null map as OR-ed null maps of args columns.
|
||||
* Or ColumnConst(ColumnNullable) if the result is always NULL or if the result is constant and always not NULL.
|
||||
*/
|
||||
ColumnPtr wrapInNullable(const ColumnPtr & src, const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count);
|
||||
|
||||
struct NullPresence
|
||||
{
|
||||
bool has_nullable = false;
|
||||
bool has_null_constant = false;
|
||||
};
|
||||
|
||||
NullPresence getNullPresense(const ColumnsWithTypeAndName & args);
|
||||
|
||||
}
|
||||
|
@ -1,6 +1,6 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <DataTypes/Native.h>
|
||||
|
||||
#if !defined(ARCADIA_BUILD)
|
||||
|
@ -17,7 +17,7 @@ namespace ErrorCodes
|
||||
}
|
||||
|
||||
template <bool or_null>
|
||||
ColumnPtr ExecutableFunctionJoinGet<or_null>::execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t) const
|
||||
ColumnPtr ExecutableFunctionJoinGet<or_null>::executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t) const
|
||||
{
|
||||
ColumnsWithTypeAndName keys;
|
||||
for (size_t i = 2; i < arguments.size(); ++i)
|
||||
@ -29,7 +29,7 @@ ColumnPtr ExecutableFunctionJoinGet<or_null>::execute(const ColumnsWithTypeAndNa
|
||||
}
|
||||
|
||||
template <bool or_null>
|
||||
ExecutableFunctionImplPtr FunctionJoinGet<or_null>::prepare(const ColumnsWithTypeAndName &) const
|
||||
ExecutableFunctionPtr FunctionJoinGet<or_null>::prepare(const ColumnsWithTypeAndName &) const
|
||||
{
|
||||
Block result_columns {{return_type->createColumn(), return_type, attr_name}};
|
||||
return std::make_unique<ExecutableFunctionJoinGet<or_null>>(table_lock, storage_join, result_columns);
|
||||
@ -81,7 +81,7 @@ getJoin(const ColumnsWithTypeAndName & arguments, ContextPtr context)
|
||||
}
|
||||
|
||||
template <bool or_null>
|
||||
FunctionBaseImplPtr JoinGetOverloadResolver<or_null>::build(const ColumnsWithTypeAndName & arguments, const DataTypePtr &) const
|
||||
FunctionBasePtr JoinGetOverloadResolver<or_null>::buildImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &) const
|
||||
{
|
||||
if (arguments.size() < 3)
|
||||
throw Exception(
|
||||
|
@ -1,6 +1,6 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunction.h>
|
||||
#include <Interpreters/Context_fwd.h>
|
||||
#include <Storages/IStorage_fwd.h>
|
||||
#include <Storages/TableLockHolder.h>
|
||||
@ -14,7 +14,7 @@ class StorageJoin;
|
||||
using StorageJoinPtr = std::shared_ptr<StorageJoin>;
|
||||
|
||||
template <bool or_null>
|
||||
class ExecutableFunctionJoinGet final : public IExecutableFunctionImpl
|
||||
class ExecutableFunctionJoinGet final : public IExecutableFunction
|
||||
{
|
||||
public:
|
||||
ExecutableFunctionJoinGet(TableLockHolder table_lock_,
|
||||
@ -31,7 +31,7 @@ public:
|
||||
bool useDefaultImplementationForLowCardinalityColumns() const override { return true; }
|
||||
bool useDefaultImplementationForConstants() const override { return true; }
|
||||
|
||||
ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override;
|
||||
ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override;
|
||||
|
||||
String getName() const override { return name; }
|
||||
|
||||
@ -42,7 +42,7 @@ private:
|
||||
};
|
||||
|
||||
template <bool or_null>
|
||||
class FunctionJoinGet final : public IFunctionBaseImpl
|
||||
class FunctionJoinGet final : public IFunctionBase
|
||||
{
|
||||
public:
|
||||
static constexpr auto name = or_null ? "joinGetOrNull" : "joinGet";
|
||||
@ -63,7 +63,7 @@ public:
|
||||
const DataTypes & getArgumentTypes() const override { return argument_types; }
|
||||
const DataTypePtr & getResultType() const override { return return_type; }
|
||||
|
||||
ExecutableFunctionImplPtr prepare(const ColumnsWithTypeAndName &) const override;
|
||||
ExecutableFunctionPtr prepare(const ColumnsWithTypeAndName &) const override;
|
||||
|
||||
private:
|
||||
TableLockHolder table_lock;
|
||||
@ -74,18 +74,18 @@ private:
|
||||
};
|
||||
|
||||
template <bool or_null>
|
||||
class JoinGetOverloadResolver final : public IFunctionOverloadResolverImpl, WithContext
|
||||
class JoinGetOverloadResolver final : public IFunctionOverloadResolver, WithContext
|
||||
{
|
||||
public:
|
||||
static constexpr auto name = or_null ? "joinGetOrNull" : "joinGet";
|
||||
static FunctionOverloadResolverImplPtr create(ContextPtr context_) { return std::make_unique<JoinGetOverloadResolver>(context_); }
|
||||
static FunctionOverloadResolverPtr create(ContextPtr context_) { return std::make_unique<JoinGetOverloadResolver>(context_); }
|
||||
|
||||
explicit JoinGetOverloadResolver(ContextPtr context_) : WithContext(context_) {}
|
||||
|
||||
String getName() const override { return name; }
|
||||
|
||||
FunctionBaseImplPtr build(const ColumnsWithTypeAndName & arguments, const DataTypePtr &) const override;
|
||||
DataTypePtr getReturnType(const ColumnsWithTypeAndName &) const override { return {}; } // Not used
|
||||
FunctionBasePtr buildImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &) const override;
|
||||
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName &) const override { return {}; } // Not used
|
||||
|
||||
bool useDefaultImplementationForNulls() const override { return false; }
|
||||
bool useDefaultImplementationForLowCardinalityColumns() const override { return false; }
|
||||
|
@ -6,7 +6,7 @@
|
||||
#include <Columns/ColumnsNumber.h>
|
||||
#include <Columns/ColumnDecimal.h>
|
||||
#include <Columns/ColumnConst.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
|
||||
#if !defined(ARCADIA_BUILD)
|
||||
|
@ -2,7 +2,7 @@
|
||||
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Columns/ColumnsNumber.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
|
||||
|
||||
namespace DB
|
||||
|
@ -5,7 +5,7 @@
|
||||
#include <DataTypes/DataTypesDecimal.h>
|
||||
#include <Columns/ColumnsNumber.h>
|
||||
#include <Columns/ColumnDecimal.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
|
||||
#if !defined(ARCADIA_BUILD)
|
||||
|
@ -1,6 +1,6 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Columns/ColumnsNumber.h>
|
||||
|
@ -2,7 +2,7 @@
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/GatherUtils/GatherUtils.h>
|
||||
#include <Functions/GatherUtils/Sources.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/PerformanceAdaptors.h>
|
||||
#include <Functions/TargetSpecific.h>
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
|
@ -1,7 +1,7 @@
|
||||
#pragma once
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Columns/ColumnVector.h>
|
||||
#include <Columns/ColumnString.h>
|
||||
|
@ -4,7 +4,7 @@
|
||||
#include <Columns/ColumnString.h>
|
||||
#include <Columns/ColumnFixedString.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Interpreters/Context_fwd.h>
|
||||
|
||||
|
||||
|
@ -7,7 +7,7 @@
|
||||
#include <Columns/ColumnVector.h>
|
||||
#include <Columns/ColumnDecimal.h>
|
||||
#include <Columns/ColumnFixedString.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IsOperation.h>
|
||||
#include <Functions/castTypeToEither.h>
|
||||
|
@ -1,7 +1,7 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/extractTimeZoneFromFunctionArguments.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeDateTime64.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
|
@ -11,7 +11,7 @@
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Common/typeid_cast.h>
|
||||
#include <Common/assert_cast.h>
|
||||
|
||||
|
@ -17,7 +17,7 @@
|
||||
#include <DataTypes/DataTypeUUID.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Interpreters/Context_fwd.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include <Common/IPv6ToBinary.h>
|
||||
|
@ -1105,8 +1105,7 @@ public:
|
||||
|
||||
if (left_tuple && right_tuple)
|
||||
{
|
||||
auto adaptor = FunctionOverloadResolverAdaptor(std::make_unique<DefaultOverloadResolver>(
|
||||
FunctionComparison<Op, Name>::create(context)));
|
||||
auto func = FunctionToOverloadResolverAdaptor(FunctionComparison<Op, Name>::create(context));
|
||||
|
||||
bool has_nullable = false;
|
||||
bool has_null = false;
|
||||
@ -1116,7 +1115,7 @@ public:
|
||||
{
|
||||
ColumnsWithTypeAndName args = {{nullptr, left_tuple->getElements()[i], ""},
|
||||
{nullptr, right_tuple->getElements()[i], ""}};
|
||||
auto element_type = adaptor.build(args)->getResultType();
|
||||
auto element_type = func.build(args)->getResultType();
|
||||
has_nullable = has_nullable || element_type->isNullable();
|
||||
has_null = has_null || element_type->onlyNull();
|
||||
}
|
||||
|
@ -4,7 +4,7 @@
|
||||
#include <Columns/ColumnsNumber.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Common/typeid_cast.h>
|
||||
#include <Interpreters/Context_fwd.h>
|
||||
|
||||
|
@ -1289,10 +1289,29 @@ public:
|
||||
size_t getNumberOfArguments() const override { return 0; }
|
||||
bool isInjective(const ColumnsWithTypeAndName &) const override { return std::is_same_v<Name, NameToString>; }
|
||||
|
||||
using DefaultReturnTypeGetter = std::function<DataTypePtr(const ColumnsWithTypeAndName &)>;
|
||||
static DataTypePtr getReturnTypeDefaultImplementationForNulls(const ColumnsWithTypeAndName & arguments, const DefaultReturnTypeGetter & getter)
|
||||
{
|
||||
NullPresence null_presence = getNullPresense(arguments);
|
||||
|
||||
if (null_presence.has_null_constant)
|
||||
{
|
||||
return makeNullable(std::make_shared<DataTypeNothing>());
|
||||
}
|
||||
if (null_presence.has_nullable)
|
||||
{
|
||||
auto nested_columns = Block(createBlockWithNestedColumns(arguments));
|
||||
auto return_type = getter(ColumnsWithTypeAndName(nested_columns.begin(), nested_columns.end()));
|
||||
return makeNullable(return_type);
|
||||
}
|
||||
|
||||
return getter(arguments);
|
||||
}
|
||||
|
||||
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
|
||||
{
|
||||
auto getter = [&] (const auto & args) { return getReturnTypeImplRemovedNullable(args); };
|
||||
auto res = FunctionOverloadResolverAdaptor::getReturnTypeDefaultImplementationForNulls(arguments, getter);
|
||||
auto res = getReturnTypeDefaultImplementationForNulls(arguments, getter);
|
||||
to_nullable = res->isNullable();
|
||||
checked_return_type = true;
|
||||
return res;
|
||||
@ -2150,7 +2169,7 @@ using FunctionParseDateTime64BestEffortOrZero = FunctionConvertFromString<
|
||||
using FunctionParseDateTime64BestEffortOrNull = FunctionConvertFromString<
|
||||
DataTypeDateTime64, NameParseDateTime64BestEffortOrNull, ConvertFromStringExceptionMode::Null, ConvertFromStringParsingMode::BestEffort>;
|
||||
|
||||
class ExecutableFunctionCast : public IExecutableFunctionImpl
|
||||
class ExecutableFunctionCast : public IExecutableFunction
|
||||
{
|
||||
public:
|
||||
using WrapperType = std::function<ColumnPtr(ColumnsWithTypeAndName &, const DataTypePtr &, const ColumnNullable *, size_t)>;
|
||||
@ -2168,7 +2187,7 @@ public:
|
||||
String getName() const override { return name; }
|
||||
|
||||
protected:
|
||||
ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const override
|
||||
ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const override
|
||||
{
|
||||
/// drop second argument, pass others
|
||||
ColumnsWithTypeAndName new_arguments{arguments.front()};
|
||||
@ -2208,7 +2227,7 @@ enum class CastType
|
||||
accurateOrNull
|
||||
};
|
||||
|
||||
class FunctionCast final : public IFunctionBaseImpl
|
||||
class FunctionCast final : public IFunctionBase
|
||||
{
|
||||
public:
|
||||
using WrapperType = std::function<ColumnPtr(ColumnsWithTypeAndName &, const DataTypePtr &, const ColumnNullable *, size_t)>;
|
||||
@ -2227,7 +2246,7 @@ public:
|
||||
const DataTypes & getArgumentTypes() const override { return argument_types; }
|
||||
const DataTypePtr & getResultType() const override { return return_type; }
|
||||
|
||||
ExecutableFunctionImplPtr prepare(const ColumnsWithTypeAndName & /*sample_columns*/) const override
|
||||
ExecutableFunctionPtr prepare(const ColumnsWithTypeAndName & /*sample_columns*/) const override
|
||||
{
|
||||
try
|
||||
{
|
||||
@ -2271,8 +2290,7 @@ private:
|
||||
|
||||
static WrapperType createFunctionAdaptor(FunctionPtr function, const DataTypePtr & from_type)
|
||||
{
|
||||
auto function_adaptor = FunctionOverloadResolverAdaptor(std::make_unique<DefaultOverloadResolver>(function))
|
||||
.build({ColumnWithTypeAndName{nullptr, from_type, ""}});
|
||||
auto function_adaptor = std::make_unique<FunctionToOverloadResolverAdaptor>(function)->build({ColumnWithTypeAndName{nullptr, from_type, ""}});
|
||||
|
||||
return [function_adaptor]
|
||||
(ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, const ColumnNullable *, size_t input_rows_count)
|
||||
@ -3164,7 +3182,7 @@ public:
|
||||
};
|
||||
|
||||
template<CastType cast_type>
|
||||
class CastOverloadResolver : public IFunctionOverloadResolverImpl
|
||||
class CastOverloadResolver : public IFunctionOverloadResolver
|
||||
{
|
||||
public:
|
||||
using MonotonicityForRange = FunctionCast::MonotonicityForRange;
|
||||
@ -3178,12 +3196,12 @@ public:
|
||||
? accurate_cast_name
|
||||
: (cast_type == CastType::accurateOrNull ? accurate_cast_or_null_name : cast_name);
|
||||
|
||||
static FunctionOverloadResolverImplPtr create(ContextPtr context)
|
||||
static FunctionOverloadResolverPtr create(ContextPtr context)
|
||||
{
|
||||
return createImpl(context->getSettingsRef().cast_keep_nullable);
|
||||
}
|
||||
|
||||
static FunctionOverloadResolverImplPtr createImpl(bool keep_nullable, std::optional<Diagnostic> diagnostic = {})
|
||||
static FunctionOverloadResolverPtr createImpl(bool keep_nullable, std::optional<Diagnostic> diagnostic = {})
|
||||
{
|
||||
return std::make_unique<CastOverloadResolver>(keep_nullable, std::move(diagnostic));
|
||||
}
|
||||
@ -3201,7 +3219,7 @@ public:
|
||||
|
||||
protected:
|
||||
|
||||
FunctionBaseImplPtr build(const ColumnsWithTypeAndName & arguments, const DataTypePtr & return_type) const override
|
||||
FunctionBasePtr buildImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & return_type) const override
|
||||
{
|
||||
DataTypes data_types(arguments.size());
|
||||
|
||||
@ -3212,7 +3230,7 @@ protected:
|
||||
return std::make_unique<FunctionCast>(name, std::move(monotonicity), data_types, return_type, diagnostic, cast_type);
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType(const ColumnsWithTypeAndName & arguments) const override
|
||||
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
|
||||
{
|
||||
const auto & column = arguments.back().column;
|
||||
if (!column)
|
||||
|
@ -9,7 +9,7 @@
|
||||
#include <Columns/ColumnString.h>
|
||||
#include <Interpreters/Context.h>
|
||||
#include <Interpreters/EmbeddedDictionaries.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Dictionaries/Embedded/RegionsHierarchy.h>
|
||||
#include <Dictionaries/Embedded/RegionsHierarchies.h>
|
||||
|
@ -27,7 +27,7 @@
|
||||
#include <Interpreters/ExternalDictionariesLoader.h>
|
||||
#include <Interpreters/castColumn.h>
|
||||
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <ext/range.h>
|
||||
|
||||
|
@ -1,6 +1,6 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Interpreters/Context_fwd.h>
|
||||
|
||||
namespace DB
|
||||
|
@ -39,7 +39,7 @@
|
||||
#include <Columns/ColumnFixedString.h>
|
||||
#include <Columns/ColumnArray.h>
|
||||
#include <Columns/ColumnTuple.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/TargetSpecific.h>
|
||||
#include <Functions/PerformanceAdaptors.h>
|
||||
|
@ -1,6 +1,6 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Core/AccurateComparison.h>
|
||||
#include <Functions/DummyJSONParser.h>
|
||||
#include <Functions/SimdJSONParser.h>
|
||||
|
@ -4,7 +4,7 @@
|
||||
#include <Core/Defines.h>
|
||||
#include <DataTypes/IDataType.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include <type_traits>
|
||||
|
||||
|
@ -17,7 +17,7 @@ namespace ErrorCodes
|
||||
extern const int BAD_ARGUMENTS;
|
||||
}
|
||||
|
||||
class ExecutableFunctionExpression : public IExecutableFunctionImpl
|
||||
class ExecutableFunctionExpression : public IExecutableFunction
|
||||
{
|
||||
public:
|
||||
struct Signature
|
||||
@ -35,7 +35,7 @@ public:
|
||||
|
||||
String getName() const override { return "FunctionExpression"; }
|
||||
|
||||
ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t /*input_rows_count*/) const override
|
||||
ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t /*input_rows_count*/) const override
|
||||
{
|
||||
DB::Block expr_columns;
|
||||
for (size_t i = 0; i < arguments.size(); ++i)
|
||||
@ -58,7 +58,7 @@ private:
|
||||
};
|
||||
|
||||
/// Executes expression. Uses for lambda functions implementation. Can't be created from factory.
|
||||
class FunctionExpression : public IFunctionBaseImpl
|
||||
class FunctionExpression : public IFunctionBase
|
||||
{
|
||||
public:
|
||||
using Signature = ExecutableFunctionExpression::Signature;
|
||||
@ -81,7 +81,7 @@ public:
|
||||
const DataTypes & getArgumentTypes() const override { return argument_types; }
|
||||
const DataTypePtr & getResultType() const override { return return_type; }
|
||||
|
||||
ExecutableFunctionImplPtr prepare(const ColumnsWithTypeAndName &) const override
|
||||
ExecutableFunctionPtr prepare(const ColumnsWithTypeAndName &) const override
|
||||
{
|
||||
return std::make_unique<ExecutableFunctionExpression>(expression_actions, signature);
|
||||
}
|
||||
@ -97,7 +97,7 @@ private:
|
||||
/// Returns ColumnFunction with captured columns.
|
||||
/// For lambda(x, x + y) x is in lambda_arguments, y is in captured arguments, expression_actions is 'x + y'.
|
||||
/// execute(y) returns ColumnFunction(FunctionExpression(x + y), y) with type Function(x) -> function_return_type.
|
||||
class ExecutableFunctionCapture : public IExecutableFunctionImpl
|
||||
class ExecutableFunctionCapture : public IExecutableFunction
|
||||
{
|
||||
public:
|
||||
struct Capture
|
||||
@ -119,7 +119,7 @@ public:
|
||||
bool useDefaultImplementationForNulls() const override { return false; }
|
||||
bool useDefaultImplementationForLowCardinalityColumns() const override { return false; }
|
||||
|
||||
ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
|
||||
ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
|
||||
{
|
||||
Names names;
|
||||
DataTypes types;
|
||||
@ -138,8 +138,8 @@ public:
|
||||
|
||||
auto function = std::make_unique<FunctionExpression>(expression_actions, types, names,
|
||||
capture->return_type, capture->return_name);
|
||||
auto function_adaptor = std::make_shared<FunctionBaseAdaptor>(std::move(function));
|
||||
return ColumnFunction::create(input_rows_count, std::move(function_adaptor), arguments);
|
||||
|
||||
return ColumnFunction::create(input_rows_count, std::move(function), arguments);
|
||||
}
|
||||
|
||||
private:
|
||||
@ -147,7 +147,7 @@ private:
|
||||
CapturePtr capture;
|
||||
};
|
||||
|
||||
class FunctionCapture : public IFunctionBaseImpl
|
||||
class FunctionCapture : public IFunctionBase
|
||||
{
|
||||
public:
|
||||
using Capture = ExecutableFunctionCapture::Capture;
|
||||
@ -173,7 +173,7 @@ public:
|
||||
const DataTypes & getArgumentTypes() const override { return capture->captured_types; }
|
||||
const DataTypePtr & getResultType() const override { return return_type; }
|
||||
|
||||
ExecutableFunctionImplPtr prepare(const ColumnsWithTypeAndName &) const override
|
||||
ExecutableFunctionPtr prepare(const ColumnsWithTypeAndName &) const override
|
||||
{
|
||||
return std::make_unique<ExecutableFunctionCapture>(expression_actions, capture);
|
||||
}
|
||||
@ -185,7 +185,7 @@ private:
|
||||
String name;
|
||||
};
|
||||
|
||||
class FunctionCaptureOverloadResolver : public IFunctionOverloadResolverImpl
|
||||
class FunctionCaptureOverloadResolver : public IFunctionOverloadResolver
|
||||
{
|
||||
public:
|
||||
using Capture = ExecutableFunctionCapture::Capture;
|
||||
@ -246,10 +246,10 @@ public:
|
||||
String getName() const override { return name; }
|
||||
bool useDefaultImplementationForNulls() const override { return false; }
|
||||
bool useDefaultImplementationForLowCardinalityColumns() const override { return false; }
|
||||
DataTypePtr getReturnType(const ColumnsWithTypeAndName &) const override { return return_type; }
|
||||
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName &) const override { return return_type; }
|
||||
size_t getNumberOfArguments() const override { return capture->captured_types.size(); }
|
||||
|
||||
FunctionBaseImplPtr build(const ColumnsWithTypeAndName &, const DataTypePtr &) const override
|
||||
FunctionBasePtr buildImpl(const ColumnsWithTypeAndName &, const DataTypePtr &) const override
|
||||
{
|
||||
return std::make_unique<FunctionCapture>(expression_actions, capture, return_type, name);
|
||||
}
|
||||
|
@ -9,7 +9,7 @@
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include <Interpreters/Context.h>
|
||||
#include <common/StringRef.h>
|
||||
|
@ -9,7 +9,7 @@
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include <Interpreters/Context.h>
|
||||
#include <common/StringRef.h>
|
||||
|
@ -9,7 +9,7 @@
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include <Interpreters/Context.h>
|
||||
#include <common/StringRef.h>
|
||||
|
@ -2,7 +2,7 @@
|
||||
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Columns/ColumnVector.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/TargetSpecific.h>
|
||||
#include <Functions/PerformanceAdaptors.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
|
@ -9,7 +9,7 @@
|
||||
#include <DataTypes/DataTypeDateTime64.h>
|
||||
#include <Columns/ColumnVector.h>
|
||||
#include <Interpreters/castColumn.h>
|
||||
#include "IFunctionImpl.h"
|
||||
#include "IFunctionOld.h"
|
||||
#include <Common/intExp.h>
|
||||
#include <Common/assert_cast.h>
|
||||
#include <Core/Defines.h>
|
||||
|
@ -9,7 +9,7 @@
|
||||
#include <Common/StringUtils/StringUtils.h>
|
||||
#include <Common/typeid_cast.h>
|
||||
#include <Common/assert_cast.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/Regexps.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
|
@ -9,7 +9,7 @@
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Interpreters/Context_fwd.h>
|
||||
|
||||
namespace DB
|
||||
|
@ -8,7 +8,7 @@
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Interpreters/Context.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
|
||||
|
@ -9,7 +9,7 @@
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include <Interpreters/Context.h>
|
||||
#include <common/StringRef.h>
|
||||
|
@ -5,7 +5,7 @@
|
||||
#include <Columns/ColumnVector.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Interpreters/Context_fwd.h>
|
||||
|
||||
namespace DB
|
||||
|
@ -5,7 +5,7 @@
|
||||
#include <DataTypes/DataTypeFixedString.h>
|
||||
#include <Columns/ColumnString.h>
|
||||
#include <Common/Volnitsky.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <IO/ReadBufferFromMemory.h>
|
||||
#include <IO/ReadHelpers.h>
|
||||
|
@ -13,25 +13,10 @@
|
||||
#include <DataTypes/Native.h>
|
||||
#include <DataTypes/DataTypeLowCardinality.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Interpreters/ExpressionActions.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include <ext/collection_cast.h>
|
||||
#include <cstdlib>
|
||||
#include <memory>
|
||||
#include <optional>
|
||||
|
||||
#if !defined(ARCADIA_BUILD)
|
||||
# include <Common/config.h>
|
||||
#endif
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
# pragma GCC diagnostic push
|
||||
# pragma GCC diagnostic ignored "-Wunused-parameter"
|
||||
# include <llvm/IR/IRBuilder.h>
|
||||
# pragma GCC diagnostic pop
|
||||
#endif
|
||||
|
||||
|
||||
namespace DB
|
||||
{
|
||||
|
||||
@ -42,148 +27,9 @@ namespace ErrorCodes
|
||||
extern const int ILLEGAL_COLUMN;
|
||||
}
|
||||
|
||||
|
||||
/// Cache for functions result if it was executed on low cardinality column.
|
||||
/// It's LRUCache which stores function result executed on dictionary and index mapping.
|
||||
/// It's expected that cache_size is a number of reading streams (so, will store single cached value per thread).
|
||||
class ExecutableFunctionLowCardinalityResultCache
|
||||
{
|
||||
public:
|
||||
/// Will assume that dictionaries with same hash has the same keys.
|
||||
/// Just in case, check that they have also the same size.
|
||||
struct DictionaryKey
|
||||
{
|
||||
UInt128 hash;
|
||||
UInt64 size;
|
||||
|
||||
bool operator== (const DictionaryKey & other) const { return hash == other.hash && size == other.size; }
|
||||
};
|
||||
|
||||
struct DictionaryKeyHash
|
||||
{
|
||||
size_t operator()(const DictionaryKey & key) const
|
||||
{
|
||||
SipHash hash;
|
||||
hash.update(key.hash);
|
||||
hash.update(key.size);
|
||||
return hash.get64();
|
||||
}
|
||||
};
|
||||
|
||||
struct CachedValues
|
||||
{
|
||||
/// Store ptr to dictionary to be sure it won't be deleted.
|
||||
ColumnPtr dictionary_holder;
|
||||
ColumnUniquePtr function_result;
|
||||
/// Remap positions. new_pos = index_mapping->index(old_pos);
|
||||
ColumnPtr index_mapping;
|
||||
};
|
||||
|
||||
using CachedValuesPtr = std::shared_ptr<CachedValues>;
|
||||
|
||||
explicit ExecutableFunctionLowCardinalityResultCache(size_t cache_size) : cache(cache_size) {}
|
||||
|
||||
CachedValuesPtr get(const DictionaryKey & key) { return cache.get(key); }
|
||||
void set(const DictionaryKey & key, const CachedValuesPtr & mapped) { cache.set(key, mapped); }
|
||||
CachedValuesPtr getOrSet(const DictionaryKey & key, const CachedValuesPtr & mapped)
|
||||
{
|
||||
return cache.getOrSet(key, [&]() { return mapped; }).first;
|
||||
}
|
||||
|
||||
private:
|
||||
using Cache = LRUCache<DictionaryKey, CachedValues, DictionaryKeyHash>;
|
||||
Cache cache;
|
||||
};
|
||||
|
||||
|
||||
void ExecutableFunctionAdaptor::createLowCardinalityResultCache(size_t cache_size)
|
||||
{
|
||||
if (!low_cardinality_result_cache)
|
||||
low_cardinality_result_cache = std::make_shared<ExecutableFunctionLowCardinalityResultCache>(cache_size);
|
||||
}
|
||||
|
||||
|
||||
ColumnPtr wrapInNullable(const ColumnPtr & src, const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count)
|
||||
{
|
||||
ColumnPtr result_null_map_column;
|
||||
|
||||
/// If result is already nullable.
|
||||
ColumnPtr src_not_nullable = src;
|
||||
|
||||
if (src->onlyNull())
|
||||
return src;
|
||||
else if (const auto * nullable = checkAndGetColumn<ColumnNullable>(*src))
|
||||
{
|
||||
src_not_nullable = nullable->getNestedColumnPtr();
|
||||
result_null_map_column = nullable->getNullMapColumnPtr();
|
||||
}
|
||||
|
||||
for (const auto & elem : args)
|
||||
{
|
||||
if (!elem.type->isNullable())
|
||||
continue;
|
||||
|
||||
/// Const Nullable that are NULL.
|
||||
if (elem.column->onlyNull())
|
||||
{
|
||||
assert(result_type->isNullable());
|
||||
return result_type->createColumnConstWithDefaultValue(input_rows_count);
|
||||
}
|
||||
|
||||
if (isColumnConst(*elem.column))
|
||||
continue;
|
||||
|
||||
if (const auto * nullable = checkAndGetColumn<ColumnNullable>(*elem.column))
|
||||
{
|
||||
const ColumnPtr & null_map_column = nullable->getNullMapColumnPtr();
|
||||
if (!result_null_map_column) //-V1051
|
||||
{
|
||||
result_null_map_column = null_map_column;
|
||||
}
|
||||
else
|
||||
{
|
||||
MutableColumnPtr mutable_result_null_map_column = IColumn::mutate(std::move(result_null_map_column));
|
||||
|
||||
NullMap & result_null_map = assert_cast<ColumnUInt8 &>(*mutable_result_null_map_column).getData();
|
||||
const NullMap & src_null_map = assert_cast<const ColumnUInt8 &>(*null_map_column).getData();
|
||||
|
||||
for (size_t i = 0, size = result_null_map.size(); i < size; ++i)
|
||||
result_null_map[i] |= src_null_map[i];
|
||||
|
||||
result_null_map_column = std::move(mutable_result_null_map_column);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!result_null_map_column)
|
||||
return makeNullable(src);
|
||||
|
||||
return ColumnNullable::create(src_not_nullable->convertToFullColumnIfConst(), result_null_map_column);
|
||||
}
|
||||
|
||||
|
||||
namespace
|
||||
{
|
||||
|
||||
struct NullPresence
|
||||
{
|
||||
bool has_nullable = false;
|
||||
bool has_null_constant = false;
|
||||
};
|
||||
|
||||
NullPresence getNullPresense(const ColumnsWithTypeAndName & args)
|
||||
{
|
||||
NullPresence res;
|
||||
|
||||
for (const auto & elem : args)
|
||||
{
|
||||
res.has_nullable |= elem.type->isNullable();
|
||||
res.has_null_constant |= elem.type->onlyNull();
|
||||
}
|
||||
|
||||
return res;
|
||||
}
|
||||
|
||||
bool allArgumentsAreConstants(const ColumnsWithTypeAndName & args)
|
||||
{
|
||||
for (const auto & arg : args)
|
||||
@ -191,128 +37,8 @@ bool allArgumentsAreConstants(const ColumnsWithTypeAndName & args)
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
ColumnPtr ExecutableFunctionAdaptor::defaultImplementationForConstantArguments(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const
|
||||
{
|
||||
ColumnNumbers arguments_to_remain_constants = impl->getArgumentsThatAreAlwaysConstant();
|
||||
|
||||
/// Check that these arguments are really constant.
|
||||
for (auto arg_num : arguments_to_remain_constants)
|
||||
if (arg_num < args.size() && !isColumnConst(*args[arg_num].column))
|
||||
throw Exception("Argument at index " + toString(arg_num) + " for function " + getName() + " must be constant", ErrorCodes::ILLEGAL_COLUMN);
|
||||
|
||||
if (args.empty() || !impl->useDefaultImplementationForConstants() || !allArgumentsAreConstants(args))
|
||||
return nullptr;
|
||||
|
||||
ColumnsWithTypeAndName temporary_columns;
|
||||
bool have_converted_columns = false;
|
||||
|
||||
size_t arguments_size = args.size();
|
||||
temporary_columns.reserve(arguments_size);
|
||||
for (size_t arg_num = 0; arg_num < arguments_size; ++arg_num)
|
||||
{
|
||||
const ColumnWithTypeAndName & column = args[arg_num];
|
||||
|
||||
if (arguments_to_remain_constants.end() != std::find(arguments_to_remain_constants.begin(), arguments_to_remain_constants.end(), arg_num))
|
||||
{
|
||||
temporary_columns.emplace_back(ColumnWithTypeAndName{column.column->cloneResized(1), column.type, column.name});
|
||||
}
|
||||
else
|
||||
{
|
||||
have_converted_columns = true;
|
||||
temporary_columns.emplace_back(ColumnWithTypeAndName{ assert_cast<const ColumnConst *>(column.column.get())->getDataColumnPtr(), column.type, column.name });
|
||||
}
|
||||
}
|
||||
|
||||
/** When using default implementation for constants, the function requires at least one argument
|
||||
* not in "arguments_to_remain_constants" set. Otherwise we get infinite recursion.
|
||||
*/
|
||||
if (!have_converted_columns)
|
||||
throw Exception("Number of arguments for function " + getName() + " doesn't match: the function requires more arguments",
|
||||
ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH);
|
||||
|
||||
ColumnPtr result_column = executeWithoutLowCardinalityColumns(temporary_columns, result_type, 1, dry_run);
|
||||
|
||||
/// extremely rare case, when we have function with completely const arguments
|
||||
/// but some of them produced by non isDeterministic function
|
||||
if (result_column->size() > 1)
|
||||
result_column = result_column->cloneResized(1);
|
||||
|
||||
return ColumnConst::create(result_column, input_rows_count);
|
||||
}
|
||||
|
||||
|
||||
ColumnPtr ExecutableFunctionAdaptor::defaultImplementationForNulls(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const
|
||||
{
|
||||
if (args.empty() || !impl->useDefaultImplementationForNulls())
|
||||
return nullptr;
|
||||
|
||||
NullPresence null_presence = getNullPresense(args);
|
||||
|
||||
if (null_presence.has_null_constant)
|
||||
{
|
||||
// Default implementation for nulls returns null result for null arguments,
|
||||
// so the result type must be nullable.
|
||||
assert(result_type->isNullable());
|
||||
|
||||
return result_type->createColumnConstWithDefaultValue(input_rows_count);
|
||||
}
|
||||
|
||||
if (null_presence.has_nullable)
|
||||
{
|
||||
ColumnsWithTypeAndName temporary_columns = createBlockWithNestedColumns(args);
|
||||
auto temporary_result_type = removeNullable(result_type);
|
||||
|
||||
auto res = executeWithoutLowCardinalityColumns(temporary_columns, temporary_result_type, input_rows_count, dry_run);
|
||||
return wrapInNullable(res, args, result_type, input_rows_count);
|
||||
}
|
||||
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
ColumnPtr ExecutableFunctionAdaptor::executeWithoutLowCardinalityColumns(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const
|
||||
{
|
||||
if (auto res = defaultImplementationForConstantArguments(args, result_type, input_rows_count, dry_run))
|
||||
return res;
|
||||
|
||||
if (auto res = defaultImplementationForNulls(args, result_type, input_rows_count, dry_run))
|
||||
return res;
|
||||
|
||||
ColumnPtr res;
|
||||
if (dry_run)
|
||||
res = impl->executeDryRun(args, result_type, input_rows_count);
|
||||
else
|
||||
res = impl->execute(args, result_type, input_rows_count);
|
||||
|
||||
if (!res)
|
||||
throw Exception(ErrorCodes::LOGICAL_ERROR, "Empty column was returned by function {}", getName());
|
||||
|
||||
return res;
|
||||
}
|
||||
|
||||
static const ColumnLowCardinality * findLowCardinalityArgument(const ColumnsWithTypeAndName & arguments)
|
||||
{
|
||||
const ColumnLowCardinality * result_column = nullptr;
|
||||
|
||||
for (const auto & column : arguments)
|
||||
{
|
||||
if (const auto * low_cardinality_column = checkAndGetColumn<ColumnLowCardinality>(column.column.get()))
|
||||
{
|
||||
if (result_column)
|
||||
throw Exception("Expected single dictionary argument for function.", ErrorCodes::LOGICAL_ERROR);
|
||||
|
||||
result_column = low_cardinality_column;
|
||||
}
|
||||
}
|
||||
|
||||
return result_column;
|
||||
}
|
||||
|
||||
static ColumnPtr replaceLowCardinalityColumnsByNestedAndGetDictionaryIndexes(
|
||||
ColumnPtr replaceLowCardinalityColumnsByNestedAndGetDictionaryIndexes(
|
||||
ColumnsWithTypeAndName & args, bool can_be_executed_on_default_arguments, size_t input_rows_count)
|
||||
{
|
||||
size_t num_rows = input_rows_count;
|
||||
@ -366,7 +92,7 @@ static ColumnPtr replaceLowCardinalityColumnsByNestedAndGetDictionaryIndexes(
|
||||
return indexes;
|
||||
}
|
||||
|
||||
static void convertLowCardinalityColumnsToFull(ColumnsWithTypeAndName & args)
|
||||
void convertLowCardinalityColumnsToFull(ColumnsWithTypeAndName & args)
|
||||
{
|
||||
for (auto & column : args)
|
||||
{
|
||||
@ -375,32 +101,120 @@ static void convertLowCardinalityColumnsToFull(ColumnsWithTypeAndName & args)
|
||||
}
|
||||
}
|
||||
|
||||
ColumnPtr ExecutableFunctionAdaptor::execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const
|
||||
}
|
||||
|
||||
ColumnPtr IExecutableFunction::defaultImplementationForConstantArguments(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const
|
||||
{
|
||||
if (impl->useDefaultImplementationForLowCardinalityColumns())
|
||||
ColumnNumbers arguments_to_remain_constants = getArgumentsThatAreAlwaysConstant();
|
||||
|
||||
/// Check that these arguments are really constant.
|
||||
for (auto arg_num : arguments_to_remain_constants)
|
||||
if (arg_num < args.size() && !isColumnConst(*args[arg_num].column))
|
||||
throw Exception("Argument at index " + toString(arg_num) + " for function " + getName() + " must be constant", ErrorCodes::ILLEGAL_COLUMN);
|
||||
|
||||
if (args.empty() || !useDefaultImplementationForConstants() || !allArgumentsAreConstants(args))
|
||||
return nullptr;
|
||||
|
||||
ColumnsWithTypeAndName temporary_columns;
|
||||
bool have_converted_columns = false;
|
||||
|
||||
size_t arguments_size = args.size();
|
||||
temporary_columns.reserve(arguments_size);
|
||||
for (size_t arg_num = 0; arg_num < arguments_size; ++arg_num)
|
||||
{
|
||||
const ColumnWithTypeAndName & column = args[arg_num];
|
||||
|
||||
if (arguments_to_remain_constants.end() != std::find(arguments_to_remain_constants.begin(), arguments_to_remain_constants.end(), arg_num))
|
||||
{
|
||||
temporary_columns.emplace_back(ColumnWithTypeAndName{column.column->cloneResized(1), column.type, column.name});
|
||||
}
|
||||
else
|
||||
{
|
||||
have_converted_columns = true;
|
||||
temporary_columns.emplace_back(ColumnWithTypeAndName{ assert_cast<const ColumnConst *>(column.column.get())->getDataColumnPtr(), column.type, column.name });
|
||||
}
|
||||
}
|
||||
|
||||
/** When using default implementation for constants, the function requires at least one argument
|
||||
* not in "arguments_to_remain_constants" set. Otherwise we get infinite recursion.
|
||||
*/
|
||||
if (!have_converted_columns)
|
||||
throw Exception("Number of arguments for function " + getName() + " doesn't match: the function requires more arguments",
|
||||
ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH);
|
||||
|
||||
ColumnPtr result_column = executeWithoutLowCardinalityColumns(temporary_columns, result_type, 1, dry_run);
|
||||
|
||||
/// extremely rare case, when we have function with completely const arguments
|
||||
/// but some of them produced by non isDeterministic function
|
||||
if (result_column->size() > 1)
|
||||
result_column = result_column->cloneResized(1);
|
||||
|
||||
return ColumnConst::create(result_column, input_rows_count);
|
||||
}
|
||||
|
||||
|
||||
ColumnPtr IExecutableFunction::defaultImplementationForNulls(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const
|
||||
{
|
||||
if (args.empty() || !useDefaultImplementationForNulls())
|
||||
return nullptr;
|
||||
|
||||
NullPresence null_presence = getNullPresense(args);
|
||||
|
||||
if (null_presence.has_null_constant)
|
||||
{
|
||||
// Default implementation for nulls returns null result for null arguments,
|
||||
// so the result type must be nullable.
|
||||
assert(result_type->isNullable());
|
||||
|
||||
return result_type->createColumnConstWithDefaultValue(input_rows_count);
|
||||
}
|
||||
|
||||
if (null_presence.has_nullable)
|
||||
{
|
||||
ColumnsWithTypeAndName temporary_columns = createBlockWithNestedColumns(args);
|
||||
auto temporary_result_type = removeNullable(result_type);
|
||||
|
||||
auto res = executeWithoutLowCardinalityColumns(temporary_columns, temporary_result_type, input_rows_count, dry_run);
|
||||
return wrapInNullable(res, args, result_type, input_rows_count);
|
||||
}
|
||||
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
ColumnPtr IExecutableFunction::executeWithoutLowCardinalityColumns(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const
|
||||
{
|
||||
if (auto res = defaultImplementationForConstantArguments(args, result_type, input_rows_count, dry_run))
|
||||
return res;
|
||||
|
||||
if (auto res = defaultImplementationForNulls(args, result_type, input_rows_count, dry_run))
|
||||
return res;
|
||||
|
||||
ColumnPtr res;
|
||||
if (dry_run)
|
||||
res = executeDryRunImpl(args, result_type, input_rows_count);
|
||||
else
|
||||
res = executeImpl(args, result_type, input_rows_count);
|
||||
|
||||
if (!res)
|
||||
throw Exception(ErrorCodes::LOGICAL_ERROR, "Empty column was returned by function {}", getName());
|
||||
|
||||
return res;
|
||||
}
|
||||
|
||||
ColumnPtr IExecutableFunction::execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const
|
||||
{
|
||||
ColumnPtr result;
|
||||
|
||||
if (useDefaultImplementationForLowCardinalityColumns())
|
||||
{
|
||||
ColumnsWithTypeAndName columns_without_low_cardinality = arguments;
|
||||
|
||||
if (const auto * res_low_cardinality_type = typeid_cast<const DataTypeLowCardinality *>(result_type.get()))
|
||||
{
|
||||
const auto * low_cardinality_column = findLowCardinalityArgument(arguments);
|
||||
bool can_be_executed_on_default_arguments = impl->canBeExecutedOnDefaultArguments();
|
||||
bool use_cache = low_cardinality_result_cache && can_be_executed_on_default_arguments
|
||||
&& low_cardinality_column && low_cardinality_column->isSharedDictionary();
|
||||
ExecutableFunctionLowCardinalityResultCache::DictionaryKey key;
|
||||
|
||||
if (use_cache)
|
||||
{
|
||||
const auto & dictionary = low_cardinality_column->getDictionary();
|
||||
key = {dictionary.getHash(), dictionary.size()};
|
||||
|
||||
auto cached_values = low_cardinality_result_cache->get(key);
|
||||
if (cached_values)
|
||||
{
|
||||
auto indexes = cached_values->index_mapping->index(low_cardinality_column->getIndexes(), 0);
|
||||
return ColumnLowCardinality::create(cached_values->function_result, indexes, true);
|
||||
}
|
||||
}
|
||||
bool can_be_executed_on_default_arguments = canBeExecutedOnDefaultArguments();
|
||||
|
||||
const auto & dictionary_type = res_low_cardinality_type->getDictionaryType();
|
||||
ColumnPtr indexes = replaceLowCardinalityColumnsByNestedAndGetDictionaryIndexes(
|
||||
@ -418,37 +232,23 @@ ColumnPtr ExecutableFunctionAdaptor::execute(const ColumnsWithTypeAndName & argu
|
||||
ColumnUniquePtr res_dictionary = std::move(res_mut_dictionary);
|
||||
|
||||
if (indexes)
|
||||
{
|
||||
if (use_cache)
|
||||
{
|
||||
auto cache_values = std::make_shared<ExecutableFunctionLowCardinalityResultCache::CachedValues>();
|
||||
cache_values->dictionary_holder = low_cardinality_column->getDictionaryPtr();
|
||||
cache_values->function_result = res_dictionary;
|
||||
cache_values->index_mapping = res_indexes;
|
||||
|
||||
cache_values = low_cardinality_result_cache->getOrSet(key, cache_values);
|
||||
res_dictionary = cache_values->function_result;
|
||||
res_indexes = cache_values->index_mapping;
|
||||
}
|
||||
|
||||
return ColumnLowCardinality::create(res_dictionary, res_indexes->index(*indexes, 0), use_cache);
|
||||
}
|
||||
result = ColumnLowCardinality::create(res_dictionary, res_indexes->index(*indexes, 0));
|
||||
else
|
||||
{
|
||||
return ColumnLowCardinality::create(res_dictionary, res_indexes);
|
||||
}
|
||||
result = ColumnLowCardinality::create(res_dictionary, res_indexes);
|
||||
}
|
||||
else
|
||||
{
|
||||
convertLowCardinalityColumnsToFull(columns_without_low_cardinality);
|
||||
return executeWithoutLowCardinalityColumns(columns_without_low_cardinality, result_type, input_rows_count, dry_run);
|
||||
result = executeWithoutLowCardinalityColumns(columns_without_low_cardinality, result_type, input_rows_count, dry_run);
|
||||
}
|
||||
}
|
||||
else
|
||||
return executeWithoutLowCardinalityColumns(arguments, result_type, input_rows_count, dry_run);
|
||||
result = executeWithoutLowCardinalityColumns(arguments, result_type, input_rows_count, dry_run);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
void FunctionOverloadResolverAdaptor::checkNumberOfArguments(size_t number_of_arguments) const
|
||||
void IFunctionOverloadResolver::checkNumberOfArguments(size_t number_of_arguments) const
|
||||
{
|
||||
if (isVariadic())
|
||||
return;
|
||||
@ -461,111 +261,9 @@ void FunctionOverloadResolverAdaptor::checkNumberOfArguments(size_t number_of_ar
|
||||
ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH);
|
||||
}
|
||||
|
||||
|
||||
DataTypePtr FunctionOverloadResolverAdaptor::getReturnTypeDefaultImplementationForNulls(const ColumnsWithTypeAndName & arguments,
|
||||
const DefaultReturnTypeGetter & getter)
|
||||
DataTypePtr IFunctionOverloadResolver::getReturnType(const ColumnsWithTypeAndName & arguments) const
|
||||
{
|
||||
NullPresence null_presence = getNullPresense(arguments);
|
||||
|
||||
if (null_presence.has_null_constant)
|
||||
{
|
||||
return makeNullable(std::make_shared<DataTypeNothing>());
|
||||
}
|
||||
if (null_presence.has_nullable)
|
||||
{
|
||||
auto nested_columns = Block(createBlockWithNestedColumns(arguments));
|
||||
auto return_type = getter(ColumnsWithTypeAndName(nested_columns.begin(), nested_columns.end()));
|
||||
return makeNullable(return_type);
|
||||
}
|
||||
|
||||
return getter(arguments);
|
||||
}
|
||||
|
||||
DataTypePtr FunctionOverloadResolverAdaptor::getReturnTypeWithoutLowCardinality(const ColumnsWithTypeAndName & arguments) const
|
||||
{
|
||||
checkNumberOfArguments(arguments.size());
|
||||
|
||||
if (!arguments.empty() && impl->useDefaultImplementationForNulls())
|
||||
return getReturnTypeDefaultImplementationForNulls(arguments, [&](const auto & args) { return impl->getReturnType(args); });
|
||||
|
||||
return impl->getReturnType(arguments);
|
||||
}
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
|
||||
static std::optional<DataTypes> removeNullables(const DataTypes & types)
|
||||
{
|
||||
for (const auto & type : types)
|
||||
{
|
||||
if (!typeid_cast<const DataTypeNullable *>(type.get()))
|
||||
continue;
|
||||
DataTypes filtered;
|
||||
for (const auto & sub_type : types)
|
||||
filtered.emplace_back(removeNullable(sub_type));
|
||||
return filtered;
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
bool IFunction::isCompilable(const DataTypes & arguments) const
|
||||
{
|
||||
if (useDefaultImplementationForNulls())
|
||||
if (auto denulled = removeNullables(arguments))
|
||||
return isCompilableImpl(*denulled);
|
||||
return isCompilableImpl(arguments);
|
||||
}
|
||||
|
||||
llvm::Value * IFunction::compile(llvm::IRBuilderBase & builder, const DataTypes & arguments, Values values) const
|
||||
{
|
||||
auto denulled_arguments = removeNullables(arguments);
|
||||
if (useDefaultImplementationForNulls() && denulled_arguments)
|
||||
{
|
||||
auto & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
std::vector<llvm::Value*> unwrapped_values;
|
||||
std::vector<llvm::Value*> is_null_values;
|
||||
|
||||
unwrapped_values.reserve(arguments.size());
|
||||
is_null_values.reserve(arguments.size());
|
||||
|
||||
for (size_t i = 0; i < arguments.size(); ++i)
|
||||
{
|
||||
auto * value = values[i];
|
||||
|
||||
WhichDataType data_type(arguments[i]);
|
||||
if (data_type.isNullable())
|
||||
{
|
||||
unwrapped_values.emplace_back(b.CreateExtractValue(value, {0}));
|
||||
is_null_values.emplace_back(b.CreateExtractValue(value, {1}));
|
||||
}
|
||||
else
|
||||
{
|
||||
unwrapped_values.emplace_back(value);
|
||||
}
|
||||
}
|
||||
|
||||
auto * result = compileImpl(builder, *denulled_arguments, unwrapped_values);
|
||||
|
||||
auto * nullable_structure_type = toNativeType(b, makeNullable(getReturnTypeImpl(*denulled_arguments)));
|
||||
auto * nullable_structure_value = llvm::Constant::getNullValue(nullable_structure_type);
|
||||
|
||||
auto * nullable_structure_with_result_value = b.CreateInsertValue(nullable_structure_value, result, {0});
|
||||
auto * nullable_structure_result_null = b.CreateExtractValue(nullable_structure_with_result_value, {1});
|
||||
|
||||
for (auto * is_null_value : is_null_values)
|
||||
nullable_structure_result_null = b.CreateOr(nullable_structure_result_null, is_null_value);
|
||||
|
||||
return b.CreateInsertValue(nullable_structure_with_result_value, nullable_structure_result_null, {1});
|
||||
}
|
||||
|
||||
return compileImpl(builder, arguments, std::move(values));
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
DataTypePtr FunctionOverloadResolverAdaptor::getReturnType(const ColumnsWithTypeAndName & arguments) const
|
||||
{
|
||||
if (impl->useDefaultImplementationForLowCardinalityColumns())
|
||||
if (useDefaultImplementationForLowCardinalityColumns())
|
||||
{
|
||||
bool has_low_cardinality = false;
|
||||
size_t num_full_low_cardinality_columns = 0;
|
||||
@ -599,7 +297,7 @@ DataTypePtr FunctionOverloadResolverAdaptor::getReturnType(const ColumnsWithType
|
||||
|
||||
auto type_without_low_cardinality = getReturnTypeWithoutLowCardinality(args_without_low_cardinality);
|
||||
|
||||
if (impl->canBeExecutedOnLowCardinalityDictionary() && has_low_cardinality
|
||||
if (canBeExecutedOnLowCardinalityDictionary() && has_low_cardinality
|
||||
&& num_full_low_cardinality_columns <= 1 && num_full_ordinary_columns == 0
|
||||
&& type_without_low_cardinality->canBeInsideLowCardinality())
|
||||
return std::make_shared<DataTypeLowCardinality>(type_without_low_cardinality);
|
||||
@ -609,4 +307,40 @@ DataTypePtr FunctionOverloadResolverAdaptor::getReturnType(const ColumnsWithType
|
||||
|
||||
return getReturnTypeWithoutLowCardinality(arguments);
|
||||
}
|
||||
|
||||
FunctionBasePtr IFunctionOverloadResolver::build(const ColumnsWithTypeAndName & arguments) const
|
||||
{
|
||||
auto return_type = getReturnType(arguments);
|
||||
return buildImpl(arguments, return_type);
|
||||
}
|
||||
|
||||
void IFunctionOverloadResolver::getLambdaArgumentTypes(DataTypes & arguments [[maybe_unused]]) const
|
||||
{
|
||||
checkNumberOfArguments(arguments.size());
|
||||
return getLambdaArgumentTypesImpl(arguments);
|
||||
}
|
||||
|
||||
DataTypePtr IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(const ColumnsWithTypeAndName & arguments) const
|
||||
{
|
||||
checkNumberOfArguments(arguments.size());
|
||||
|
||||
if (!arguments.empty() && useDefaultImplementationForNulls())
|
||||
{
|
||||
NullPresence null_presence = getNullPresense(arguments);
|
||||
|
||||
if (null_presence.has_null_constant)
|
||||
{
|
||||
return makeNullable(std::make_shared<DataTypeNothing>());
|
||||
}
|
||||
if (null_presence.has_nullable)
|
||||
{
|
||||
Block nested_columns = createBlockWithNestedColumns(arguments);
|
||||
auto return_type = getReturnTypeImpl(ColumnsWithTypeAndName(nested_columns.begin(), nested_columns.end()));
|
||||
return makeNullable(return_type);
|
||||
}
|
||||
}
|
||||
|
||||
return getReturnTypeImpl(arguments);
|
||||
}
|
||||
|
||||
}
|
||||
|
@ -12,7 +12,6 @@
|
||||
#include <memory>
|
||||
|
||||
/// This file contains user interface for functions.
|
||||
/// For developer interface (in case you need to implement a new function) see IFunctionImpl.h
|
||||
|
||||
namespace llvm
|
||||
{
|
||||
@ -28,6 +27,7 @@ namespace DB
|
||||
namespace ErrorCodes
|
||||
{
|
||||
extern const int NOT_IMPLEMENTED;
|
||||
extern const int ILLEGAL_TYPE_OF_ARGUMENT;
|
||||
}
|
||||
|
||||
class Field;
|
||||
@ -46,23 +46,78 @@ public:
|
||||
/// Get the main function name.
|
||||
virtual String getName() const = 0;
|
||||
|
||||
virtual ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const = 0;
|
||||
ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const;
|
||||
|
||||
protected:
|
||||
|
||||
virtual ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const = 0;
|
||||
|
||||
virtual ColumnPtr executeDryRunImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const
|
||||
{
|
||||
return executeImpl(arguments, result_type, input_rows_count);
|
||||
}
|
||||
|
||||
/** Default implementation in presence of Nullable arguments or NULL constants as arguments is the following:
|
||||
* if some of arguments are NULL constants then return NULL constant,
|
||||
* if some of arguments are Nullable, then execute function as usual for columns,
|
||||
* where Nullable columns are substituted with nested columns (they have arbitrary values in rows corresponding to NULL value)
|
||||
* and wrap result in Nullable column where NULLs are in all rows where any of arguments are NULL.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForNulls() const { return true; }
|
||||
|
||||
/** If the function have non-zero number of arguments,
|
||||
* and if all arguments are constant, that we could automatically provide default implementation:
|
||||
* arguments are converted to ordinary columns with single value, then function is executed as usual,
|
||||
* and then the result is converted to constant column.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForConstants() const { return false; }
|
||||
|
||||
/** If function arguments has single low cardinality column and all other arguments are constants, call function on nested column.
|
||||
* Otherwise, convert all low cardinality columns to ordinary columns.
|
||||
* Returns ColumnLowCardinality if at least one argument is ColumnLowCardinality.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForLowCardinalityColumns() const { return true; }
|
||||
|
||||
/** Some arguments could remain constant during this implementation.
|
||||
*/
|
||||
virtual ColumnNumbers getArgumentsThatAreAlwaysConstant() const { return {}; }
|
||||
|
||||
/** True if function can be called on default arguments (include Nullable's) and won't throw.
|
||||
* Counterexample: modulo(0, 0)
|
||||
*/
|
||||
virtual bool canBeExecutedOnDefaultArguments() const { return true; }
|
||||
|
||||
private:
|
||||
|
||||
ColumnPtr defaultImplementationForConstantArguments(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const;
|
||||
|
||||
ColumnPtr defaultImplementationForNulls(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const;
|
||||
|
||||
ColumnPtr executeWithoutLowCardinalityColumns(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const;
|
||||
|
||||
virtual void createLowCardinalityResultCache(size_t cache_size) = 0;
|
||||
};
|
||||
|
||||
using ExecutableFunctionPtr = std::shared_ptr<IExecutableFunction>;
|
||||
|
||||
|
||||
using Values = std::vector<llvm::Value *>;
|
||||
|
||||
/// Function with known arguments and return type (when the specific overload was chosen).
|
||||
/// It is also the point where all function-specific properties are known.
|
||||
/** Function with known arguments and return type (when the specific overload was chosen).
|
||||
* It is also the point where all function-specific properties are known.
|
||||
*/
|
||||
class IFunctionBase
|
||||
{
|
||||
public:
|
||||
|
||||
virtual ~IFunctionBase() = default;
|
||||
|
||||
virtual ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run = false) const
|
||||
{
|
||||
return prepare(arguments)->execute(arguments, result_type, input_rows_count, dry_run);
|
||||
}
|
||||
|
||||
/// Get the main function name.
|
||||
virtual String getName() const = 0;
|
||||
|
||||
@ -73,11 +128,6 @@ public:
|
||||
/// sample_columns should contain data types of arguments and values of constants, if relevant.
|
||||
virtual ExecutableFunctionPtr prepare(const ColumnsWithTypeAndName & arguments) const = 0;
|
||||
|
||||
virtual ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run = false) const
|
||||
{
|
||||
return prepare(arguments)->execute(arguments, result_type, input_rows_count, dry_run);
|
||||
}
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
|
||||
virtual bool isCompilable() const { return false; }
|
||||
@ -147,9 +197,9 @@ public:
|
||||
* Example: now(). Another example: functions that work with periodically updated dictionaries.
|
||||
*/
|
||||
|
||||
virtual bool isDeterministic() const = 0;
|
||||
virtual bool isDeterministic() const { return true; }
|
||||
|
||||
virtual bool isDeterministicInScopeOfQuery() const = 0;
|
||||
virtual bool isDeterministicInScopeOfQuery() const { return true; }
|
||||
|
||||
/** Lets you know if the function is monotonic in a range of values.
|
||||
* This is used to work with the index in a sorted chunk of data.
|
||||
@ -176,58 +226,104 @@ public:
|
||||
{
|
||||
throw Exception("Function " + getName() + " has no information about its monotonicity.", ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
|
||||
};
|
||||
|
||||
using FunctionBasePtr = std::shared_ptr<IFunctionBase>;
|
||||
|
||||
|
||||
/// Creates IFunctionBase from argument types list (chooses one function overload).
|
||||
/** Creates IFunctionBase from argument types list (chooses one function overload).
|
||||
*/
|
||||
class IFunctionOverloadResolver
|
||||
{
|
||||
public:
|
||||
virtual ~IFunctionOverloadResolver() = default;
|
||||
|
||||
FunctionBasePtr build(const ColumnsWithTypeAndName & arguments) const;
|
||||
|
||||
DataTypePtr getReturnType(const ColumnsWithTypeAndName & arguments) const;
|
||||
|
||||
void getLambdaArgumentTypes(DataTypes & arguments) const;
|
||||
|
||||
/// Get the main function name.
|
||||
virtual String getName() const = 0;
|
||||
|
||||
/// See the comment for the same method in IFunctionBase
|
||||
virtual bool isDeterministic() const = 0;
|
||||
virtual bool isDeterministicInScopeOfQuery() const = 0;
|
||||
virtual bool isInjective(const ColumnsWithTypeAndName &) const = 0;
|
||||
|
||||
/// Override and return true if function needs to depend on the state of the data.
|
||||
virtual bool isStateful() const = 0;
|
||||
|
||||
/// Override and return true if function could take different number of arguments.
|
||||
virtual bool isVariadic() const = 0;
|
||||
|
||||
/// For non-variadic functions, return number of arguments; otherwise return zero (that should be ignored).
|
||||
virtual size_t getNumberOfArguments() const = 0;
|
||||
|
||||
/// Throw if number of arguments is incorrect.
|
||||
virtual void checkNumberOfArguments(size_t number_of_arguments) const = 0;
|
||||
/// TODO: This method should not be duplicated here and in IFunctionBase
|
||||
/// See the comment for the same method in IFunctionBase
|
||||
virtual bool isDeterministic() const { return true; }
|
||||
virtual bool isDeterministicInScopeOfQuery() const { return true; }
|
||||
virtual bool isInjective(const ColumnsWithTypeAndName &) const { return false; }
|
||||
|
||||
/// Check if arguments are correct and returns IFunctionBase.
|
||||
virtual FunctionBasePtr build(const ColumnsWithTypeAndName & arguments) const = 0;
|
||||
/// Override and return true if function needs to depend on the state of the data.
|
||||
virtual bool isStateful() const { return false; }
|
||||
|
||||
/// Override and return true if function could take different number of arguments.
|
||||
virtual bool isVariadic() const { return false; }
|
||||
|
||||
/// For non-variadic functions, return number of arguments; otherwise return zero (that should be ignored).
|
||||
/// For higher-order functions (functions, that have lambda expression as at least one argument).
|
||||
/// You pass data types with empty DataTypeFunction for lambda arguments.
|
||||
/// This function will replace it with DataTypeFunction containing actual types.
|
||||
virtual void getLambdaArgumentTypes(DataTypes & arguments) const = 0;
|
||||
virtual void getLambdaArgumentTypesImpl(DataTypes & arguments [[maybe_unused]]) const
|
||||
{
|
||||
throw Exception("Function " + getName() + " can't have lambda-expressions as arguments", ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
|
||||
}
|
||||
|
||||
/// Returns indexes of arguments, that must be ColumnConst
|
||||
virtual ColumnNumbers getArgumentsThatAreAlwaysConstant() const = 0;
|
||||
virtual ColumnNumbers getArgumentsThatAreAlwaysConstant() const { return {}; }
|
||||
|
||||
/// Returns indexes if arguments, that can be Nullable without making result of function Nullable
|
||||
/// (for functions like isNull(x))
|
||||
virtual ColumnNumbers getArgumentsThatDontImplyNullableReturnType(size_t number_of_arguments) const = 0;
|
||||
virtual ColumnNumbers getArgumentsThatDontImplyNullableReturnType(size_t number_of_arguments [[maybe_unused]]) const { return {}; }
|
||||
|
||||
protected:
|
||||
|
||||
virtual FunctionBasePtr buildImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type) const = 0;
|
||||
|
||||
virtual DataTypePtr getReturnTypeImpl(const DataTypes & /*arguments*/) const
|
||||
{
|
||||
throw Exception("getReturnType is not implemented for " + getName(), ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
|
||||
/// This function will be called in default implementation. You can overload it or the previous one.
|
||||
virtual DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const
|
||||
{
|
||||
DataTypes data_types(arguments.size());
|
||||
for (size_t i = 0; i < arguments.size(); ++i)
|
||||
data_types[i] = arguments[i].type;
|
||||
|
||||
return getReturnTypeImpl(data_types);
|
||||
}
|
||||
|
||||
/** If useDefaultImplementationForNulls() is true, than change arguments for getReturnType() and build():
|
||||
* if some of arguments are Nullable(Nothing) then don't call getReturnType(), call build() with return_type = Nullable(Nothing),
|
||||
* if some of arguments are Nullable, then:
|
||||
* - Nullable types are substituted with nested types for getReturnType() function
|
||||
* - wrap getReturnType() result in Nullable type and pass to build
|
||||
*
|
||||
* Otherwise build returns build(arguments, getReturnType(arguments));
|
||||
*/
|
||||
virtual bool useDefaultImplementationForNulls() const { return true; }
|
||||
|
||||
/** If useDefaultImplementationForNulls() is true, than change arguments for getReturnType() and build().
|
||||
* If function arguments has low cardinality types, convert them to ordinary types.
|
||||
* getReturnType returns ColumnLowCardinality if at least one argument type is ColumnLowCardinality.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForLowCardinalityColumns() const { return true; }
|
||||
|
||||
// /// If it isn't, will convert all ColumnLowCardinality arguments to full columns.
|
||||
virtual bool canBeExecutedOnLowCardinalityDictionary() const { return true; }
|
||||
|
||||
private:
|
||||
|
||||
void checkNumberOfArguments(size_t number_of_arguments) const;
|
||||
|
||||
DataTypePtr getReturnTypeWithoutLowCardinality(const ColumnsWithTypeAndName & arguments) const;
|
||||
};
|
||||
|
||||
using FunctionOverloadResolverPtr = std::shared_ptr<IFunctionOverloadResolver>;
|
||||
|
||||
|
||||
/** Return ColumnNullable of src, with null map as OR-ed null maps of args columns.
|
||||
* Or ColumnConst(ColumnNullable) if the result is always NULL or if the result is constant and always not NULL.
|
||||
*/
|
||||
ColumnPtr wrapInNullable(const ColumnPtr & src, const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count);
|
||||
|
||||
}
|
||||
|
@ -1,163 +1,31 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
|
||||
namespace DB
|
||||
{
|
||||
|
||||
/// Adaptors are implement user interfaces from IFunction.h via developer interfaces from IFunctionImpl.h
|
||||
/// Typically, you don't need to change this classes.
|
||||
/// Following class implement IExecutableFunction via IFunction.
|
||||
|
||||
class ExecutableFunctionAdaptor final : public IExecutableFunction
|
||||
class FunctionToExecutableFunctionAdaptor final : public IExecutableFunction
|
||||
{
|
||||
public:
|
||||
explicit ExecutableFunctionAdaptor(ExecutableFunctionImplPtr impl_) : impl(std::move(impl_)) {}
|
||||
|
||||
String getName() const final { return impl->getName(); }
|
||||
|
||||
ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const final;
|
||||
|
||||
void createLowCardinalityResultCache(size_t cache_size) override;
|
||||
|
||||
private:
|
||||
ExecutableFunctionImplPtr impl;
|
||||
|
||||
/// Cache is created by function createLowCardinalityResultCache()
|
||||
ExecutableFunctionLowCardinalityResultCachePtr low_cardinality_result_cache;
|
||||
|
||||
ColumnPtr defaultImplementationForConstantArguments(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const;
|
||||
|
||||
ColumnPtr defaultImplementationForNulls(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const;
|
||||
|
||||
ColumnPtr executeWithoutLowCardinalityColumns(
|
||||
const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count, bool dry_run) const;
|
||||
};
|
||||
|
||||
class FunctionBaseAdaptor final : public IFunctionBase
|
||||
{
|
||||
public:
|
||||
explicit FunctionBaseAdaptor(FunctionBaseImplPtr impl_) : impl(std::move(impl_)) {}
|
||||
|
||||
String getName() const final { return impl->getName(); }
|
||||
|
||||
const DataTypes & getArgumentTypes() const final { return impl->getArgumentTypes(); }
|
||||
const DataTypePtr & getResultType() const final { return impl->getResultType(); }
|
||||
|
||||
ExecutableFunctionPtr prepare(const ColumnsWithTypeAndName & arguments) const final
|
||||
{
|
||||
return std::make_shared<ExecutableFunctionAdaptor>(impl->prepare(arguments));
|
||||
}
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
|
||||
bool isCompilable() const final { return impl->isCompilable(); }
|
||||
|
||||
llvm::Value * compile(llvm::IRBuilderBase & builder, Values values) const override
|
||||
{
|
||||
return impl->compile(builder, std::move(values));
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
bool isStateful() const final { return impl->isStateful(); }
|
||||
bool isSuitableForConstantFolding() const final { return impl->isSuitableForConstantFolding(); }
|
||||
|
||||
ColumnPtr getResultIfAlwaysReturnsConstantAndHasArguments(const ColumnsWithTypeAndName & arguments) const final
|
||||
{
|
||||
return impl->getResultIfAlwaysReturnsConstantAndHasArguments(arguments);
|
||||
}
|
||||
|
||||
bool isInjective(const ColumnsWithTypeAndName & sample_columns) const final { return impl->isInjective(sample_columns); }
|
||||
bool isDeterministic() const final { return impl->isDeterministic(); }
|
||||
bool isDeterministicInScopeOfQuery() const final { return impl->isDeterministicInScopeOfQuery(); }
|
||||
bool hasInformationAboutMonotonicity() const final { return impl->hasInformationAboutMonotonicity(); }
|
||||
|
||||
Monotonicity getMonotonicityForRange(const IDataType & type, const Field & left, const Field & right) const final
|
||||
{
|
||||
return impl->getMonotonicityForRange(type, left, right);
|
||||
}
|
||||
|
||||
const IFunctionBaseImpl * getImpl() const { return impl.get(); }
|
||||
|
||||
private:
|
||||
FunctionBaseImplPtr impl;
|
||||
};
|
||||
|
||||
|
||||
class FunctionOverloadResolverAdaptor final : public IFunctionOverloadResolver
|
||||
{
|
||||
public:
|
||||
explicit FunctionOverloadResolverAdaptor(FunctionOverloadResolverImplPtr impl_) : impl(std::move(impl_)) {}
|
||||
|
||||
String getName() const final { return impl->getName(); }
|
||||
|
||||
bool isDeterministic() const final { return impl->isDeterministic(); }
|
||||
|
||||
bool isDeterministicInScopeOfQuery() const final { return impl->isDeterministicInScopeOfQuery(); }
|
||||
|
||||
bool isInjective(const ColumnsWithTypeAndName & columns) const final { return impl->isInjective(columns); }
|
||||
|
||||
bool isStateful() const final { return impl->isStateful(); }
|
||||
|
||||
bool isVariadic() const final { return impl->isVariadic(); }
|
||||
|
||||
size_t getNumberOfArguments() const final { return impl->getNumberOfArguments(); }
|
||||
|
||||
void checkNumberOfArguments(size_t number_of_arguments) const final;
|
||||
|
||||
FunctionBaseImplPtr buildImpl(const ColumnsWithTypeAndName & arguments) const
|
||||
{
|
||||
return impl->build(arguments, getReturnType(arguments));
|
||||
}
|
||||
|
||||
FunctionBasePtr build(const ColumnsWithTypeAndName & arguments) const final
|
||||
{
|
||||
return std::make_shared<FunctionBaseAdaptor>(buildImpl(arguments));
|
||||
}
|
||||
|
||||
void getLambdaArgumentTypes(DataTypes & arguments) const final
|
||||
{
|
||||
checkNumberOfArguments(arguments.size());
|
||||
impl->getLambdaArgumentTypes(arguments);
|
||||
}
|
||||
|
||||
ColumnNumbers getArgumentsThatAreAlwaysConstant() const final { return impl->getArgumentsThatAreAlwaysConstant(); }
|
||||
|
||||
ColumnNumbers getArgumentsThatDontImplyNullableReturnType(size_t number_of_arguments) const final
|
||||
{
|
||||
return impl->getArgumentsThatDontImplyNullableReturnType(number_of_arguments);
|
||||
}
|
||||
|
||||
using DefaultReturnTypeGetter = std::function<DataTypePtr(const ColumnsWithTypeAndName &)>;
|
||||
static DataTypePtr getReturnTypeDefaultImplementationForNulls(const ColumnsWithTypeAndName & arguments, const DefaultReturnTypeGetter & getter);
|
||||
private:
|
||||
FunctionOverloadResolverImplPtr impl;
|
||||
|
||||
DataTypePtr getReturnTypeWithoutLowCardinality(const ColumnsWithTypeAndName & arguments) const;
|
||||
DataTypePtr getReturnType(const ColumnsWithTypeAndName & arguments) const;
|
||||
};
|
||||
|
||||
|
||||
/// Following classes are implement IExecutableFunctionImpl, IFunctionBaseImpl and IFunctionOverloadResolverImpl via IFunction.
|
||||
|
||||
class DefaultExecutable final : public IExecutableFunctionImpl
|
||||
{
|
||||
public:
|
||||
explicit DefaultExecutable(std::shared_ptr<IFunction> function_) : function(std::move(function_)) {}
|
||||
explicit FunctionToExecutableFunctionAdaptor(std::shared_ptr<IFunction> function_) : function(std::move(function_)) {}
|
||||
|
||||
String getName() const override { return function->getName(); }
|
||||
|
||||
protected:
|
||||
ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const final
|
||||
|
||||
ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const final
|
||||
{
|
||||
return function->executeImpl(arguments, result_type, input_rows_count);
|
||||
}
|
||||
ColumnPtr executeDryRun(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const final
|
||||
|
||||
ColumnPtr executeDryRunImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const final
|
||||
{
|
||||
return function->executeImplDryRun(arguments, result_type, input_rows_count);
|
||||
}
|
||||
|
||||
bool useDefaultImplementationForNulls() const final { return function->useDefaultImplementationForNulls(); }
|
||||
bool useDefaultImplementationForConstants() const final { return function->useDefaultImplementationForConstants(); }
|
||||
bool useDefaultImplementationForLowCardinalityColumns() const final { return function->useDefaultImplementationForLowCardinalityColumns(); }
|
||||
@ -168,10 +36,12 @@ private:
|
||||
std::shared_ptr<IFunction> function;
|
||||
};
|
||||
|
||||
class DefaultFunction final : public IFunctionBaseImpl
|
||||
/// Following class implement IFunctionBase via IFunction.
|
||||
|
||||
class FunctionToFunctionBaseAdaptor final : public IFunctionBase
|
||||
{
|
||||
public:
|
||||
DefaultFunction(std::shared_ptr<IFunction> function_, DataTypes arguments_, DataTypePtr result_type_)
|
||||
FunctionToFunctionBaseAdaptor(std::shared_ptr<IFunction> function_, DataTypes arguments_, DataTypePtr result_type_)
|
||||
: function(std::move(function_)), arguments(std::move(arguments_)), result_type(std::move(result_type_)) {}
|
||||
|
||||
String getName() const override { return function->getName(); }
|
||||
@ -190,9 +60,9 @@ public:
|
||||
|
||||
#endif
|
||||
|
||||
ExecutableFunctionImplPtr prepare(const ColumnsWithTypeAndName & /*arguments*/) const override
|
||||
ExecutableFunctionPtr prepare(const ColumnsWithTypeAndName & /*arguments*/) const override
|
||||
{
|
||||
return std::make_unique<DefaultExecutable>(function);
|
||||
return std::make_unique<FunctionToExecutableFunctionAdaptor>(function);
|
||||
}
|
||||
|
||||
bool isSuitableForConstantFolding() const override { return function->isSuitableForConstantFolding(); }
|
||||
@ -211,7 +81,6 @@ public:
|
||||
|
||||
bool hasInformationAboutMonotonicity() const override { return function->hasInformationAboutMonotonicity(); }
|
||||
|
||||
using Monotonicity = IFunctionBase::Monotonicity;
|
||||
Monotonicity getMonotonicityForRange(const IDataType & type, const Field & left, const Field & right) const override
|
||||
{
|
||||
return function->getMonotonicityForRange(type, left, right);
|
||||
@ -222,10 +91,13 @@ private:
|
||||
DataTypePtr result_type;
|
||||
};
|
||||
|
||||
class DefaultOverloadResolver : public IFunctionOverloadResolverImpl
|
||||
|
||||
/// Following class implement IFunctionOverloadResolver via IFunction.
|
||||
|
||||
class FunctionToOverloadResolverAdaptor : public IFunctionOverloadResolver
|
||||
{
|
||||
public:
|
||||
explicit DefaultOverloadResolver(std::shared_ptr<IFunction> function_) : function(std::move(function_)) {}
|
||||
explicit FunctionToOverloadResolverAdaptor(std::shared_ptr<IFunction> function_) : function(std::move(function_)) {}
|
||||
|
||||
bool isDeterministic() const override { return function->isDeterministic(); }
|
||||
bool isDeterministicInScopeOfQuery() const override { return function->isDeterministicInScopeOfQuery(); }
|
||||
@ -242,22 +114,23 @@ public:
|
||||
return function->getArgumentsThatDontImplyNullableReturnType(number_of_arguments);
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType(const DataTypes & arguments) const override { return function->getReturnTypeImpl(arguments); }
|
||||
DataTypePtr getReturnType(const ColumnsWithTypeAndName & arguments) const override { return function->getReturnTypeImpl(arguments); }
|
||||
DataTypePtr getReturnTypeImpl(const DataTypes & arguments) const override { return function->getReturnTypeImpl(arguments); }
|
||||
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override { return function->getReturnTypeImpl(arguments); }
|
||||
|
||||
bool useDefaultImplementationForNulls() const override { return function->useDefaultImplementationForNulls(); }
|
||||
bool useDefaultImplementationForLowCardinalityColumns() const override { return function->useDefaultImplementationForLowCardinalityColumns(); }
|
||||
bool canBeExecutedOnLowCardinalityDictionary() const override { return function->canBeExecutedOnLowCardinalityDictionary(); }
|
||||
|
||||
FunctionBaseImplPtr build(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type) const override
|
||||
FunctionBasePtr buildImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type) const override
|
||||
{
|
||||
DataTypes data_types(arguments.size());
|
||||
for (size_t i = 0; i < arguments.size(); ++i)
|
||||
data_types[i] = arguments[i].type;
|
||||
return std::make_unique<DefaultFunction>(function, data_types, result_type);
|
||||
|
||||
return std::make_unique<FunctionToFunctionBaseAdaptor>(function, data_types, result_type);
|
||||
}
|
||||
|
||||
void getLambdaArgumentTypes(DataTypes & arguments) const override { function->getLambdaArgumentTypes(arguments); }
|
||||
void getLambdaArgumentTypesImpl(DataTypes & arguments) const override { function->getLambdaArgumentTypes(arguments); }
|
||||
|
||||
private:
|
||||
std::shared_ptr<IFunction> function;
|
||||
|
@ -1,307 +0,0 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/IFunction.h>
|
||||
|
||||
/// This file contains developer interface for functions.
|
||||
/// In order to implement a new function you can choose one of two options:
|
||||
/// * Implement interface for IFunction (old function interface, which is planned to be removed sometimes)
|
||||
/// * Implement three interfaces for IExecutableFunctionImpl, IFunctionBaseImpl and IFunctionOverloadResolverImpl
|
||||
/// Generally saying, IFunction represents a union of three new interfaces. However, it can't be used for all cases.
|
||||
/// Examples:
|
||||
/// * Function properties may depend on arguments type (e.g. toUInt32(UInt8) is globally monotonic, toUInt32(UInt64) - only on intervals)
|
||||
/// * In implementation of lambda functions DataTypeFunction needs an functional object with known arguments and return type
|
||||
/// * Function CAST prepares specific implementation based on argument types
|
||||
///
|
||||
/// Interfaces for IFunction, IExecutableFunctionImpl, IFunctionBaseImpl and IFunctionOverloadResolverImpl are pure.
|
||||
/// Default implementations are in adaptors classes (IFunctionAdaptors.h), which are implement user interfaces via developer ones.
|
||||
/// Interfaces IExecutableFunctionImpl, IFunctionBaseImpl and IFunctionOverloadResolverImpl are implemented via IFunction
|
||||
/// in DefaultExecutable, DefaultFunction and DefaultOverloadResolver classes (IFunctionAdaptors.h).
|
||||
|
||||
namespace DB
|
||||
{
|
||||
namespace ErrorCodes
|
||||
{
|
||||
extern const int ILLEGAL_TYPE_OF_ARGUMENT;
|
||||
extern const int NOT_IMPLEMENTED;
|
||||
}
|
||||
|
||||
/// Cache for functions result if it was executed on low cardinality column.
|
||||
class ExecutableFunctionLowCardinalityResultCache;
|
||||
using ExecutableFunctionLowCardinalityResultCachePtr = std::shared_ptr<ExecutableFunctionLowCardinalityResultCache>;
|
||||
|
||||
class IExecutableFunctionImpl
|
||||
{
|
||||
public:
|
||||
virtual ~IExecutableFunctionImpl() = default;
|
||||
|
||||
virtual String getName() const = 0;
|
||||
|
||||
virtual ColumnPtr execute(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const = 0;
|
||||
virtual ColumnPtr executeDryRun(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const
|
||||
{
|
||||
return execute(arguments, result_type, input_rows_count);
|
||||
}
|
||||
|
||||
/** Default implementation in presence of Nullable arguments or NULL constants as arguments is the following:
|
||||
* if some of arguments are NULL constants then return NULL constant,
|
||||
* if some of arguments are Nullable, then execute function as usual for columns,
|
||||
* where Nullable columns are substituted with nested columns (they have arbitrary values in rows corresponding to NULL value)
|
||||
* and wrap result in Nullable column where NULLs are in all rows where any of arguments are NULL.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForNulls() const { return true; }
|
||||
|
||||
/** If the function have non-zero number of arguments,
|
||||
* and if all arguments are constant, that we could automatically provide default implementation:
|
||||
* arguments are converted to ordinary columns with single value, then function is executed as usual,
|
||||
* and then the result is converted to constant column.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForConstants() const { return false; }
|
||||
|
||||
/** If function arguments has single low cardinality column and all other arguments are constants, call function on nested column.
|
||||
* Otherwise, convert all low cardinality columns to ordinary columns.
|
||||
* Returns ColumnLowCardinality if at least one argument is ColumnLowCardinality.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForLowCardinalityColumns() const { return true; }
|
||||
|
||||
/** Some arguments could remain constant during this implementation.
|
||||
*/
|
||||
virtual ColumnNumbers getArgumentsThatAreAlwaysConstant() const { return {}; }
|
||||
|
||||
/** True if function can be called on default arguments (include Nullable's) and won't throw.
|
||||
* Counterexample: modulo(0, 0)
|
||||
*/
|
||||
virtual bool canBeExecutedOnDefaultArguments() const { return true; }
|
||||
};
|
||||
|
||||
using ExecutableFunctionImplPtr = std::unique_ptr<IExecutableFunctionImpl>;
|
||||
|
||||
|
||||
/// This class generally has the same methods as in IFunctionBase.
|
||||
/// See comments for IFunctionBase in IFunction.h
|
||||
/// The main purpose is to implement `prepare` which returns IExecutableFunctionImpl, not IExecutableFunction
|
||||
/// Inheritance is not used for better readability.
|
||||
class IFunctionBaseImpl
|
||||
{
|
||||
public:
|
||||
virtual ~IFunctionBaseImpl() = default;
|
||||
|
||||
virtual String getName() const = 0;
|
||||
|
||||
virtual const DataTypes & getArgumentTypes() const = 0;
|
||||
virtual const DataTypePtr & getResultType() const = 0;
|
||||
|
||||
virtual ExecutableFunctionImplPtr prepare(const ColumnsWithTypeAndName & arguments) const = 0;
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
|
||||
virtual bool isCompilable() const { return false; }
|
||||
|
||||
virtual llvm::Value * compile(llvm::IRBuilderBase & /*builder*/, Values /*values*/) const
|
||||
{
|
||||
throw Exception(getName() + " is not JIT-compilable", ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
virtual bool isStateful() const { return false; }
|
||||
|
||||
virtual bool isSuitableForConstantFolding() const { return true; }
|
||||
virtual ColumnPtr getResultIfAlwaysReturnsConstantAndHasArguments(const ColumnsWithTypeAndName & /*arguments*/) const { return nullptr; }
|
||||
|
||||
virtual bool isInjective(const ColumnsWithTypeAndName & /*sample_columns*/) const { return false; }
|
||||
virtual bool isDeterministic() const { return true; }
|
||||
virtual bool isDeterministicInScopeOfQuery() const { return true; }
|
||||
virtual bool hasInformationAboutMonotonicity() const { return false; }
|
||||
|
||||
using Monotonicity = IFunctionBase::Monotonicity;
|
||||
virtual Monotonicity getMonotonicityForRange(const IDataType & /*type*/, const Field & /*left*/, const Field & /*right*/) const
|
||||
{
|
||||
throw Exception("Function " + getName() + " has no information about its monotonicity.", ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
};
|
||||
|
||||
using FunctionBaseImplPtr = std::unique_ptr<IFunctionBaseImpl>;
|
||||
|
||||
|
||||
class IFunctionOverloadResolverImpl
|
||||
{
|
||||
public:
|
||||
|
||||
virtual ~IFunctionOverloadResolverImpl() = default;
|
||||
|
||||
virtual String getName() const = 0;
|
||||
|
||||
virtual FunctionBaseImplPtr build(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type) const = 0;
|
||||
|
||||
virtual DataTypePtr getReturnType(const DataTypes & /*arguments*/) const
|
||||
{
|
||||
throw Exception("getReturnType is not implemented for " + getName(), ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
|
||||
/// This function will be called in default implementation. You can overload it or the previous one.
|
||||
virtual DataTypePtr getReturnType(const ColumnsWithTypeAndName & arguments) const
|
||||
{
|
||||
DataTypes data_types(arguments.size());
|
||||
for (size_t i = 0; i < arguments.size(); ++i)
|
||||
data_types[i] = arguments[i].type;
|
||||
|
||||
return getReturnType(data_types);
|
||||
}
|
||||
|
||||
/// For non-variadic functions, return number of arguments; otherwise return zero (that should be ignored).
|
||||
virtual size_t getNumberOfArguments() const = 0;
|
||||
|
||||
/// Properties from IFunctionOverloadResolver. See comments in IFunction.h
|
||||
virtual bool isDeterministic() const { return true; }
|
||||
virtual bool isDeterministicInScopeOfQuery() const { return true; }
|
||||
virtual bool isInjective(const ColumnsWithTypeAndName &) const { return false; }
|
||||
virtual bool isStateful() const { return false; }
|
||||
virtual bool isVariadic() const { return false; }
|
||||
|
||||
virtual void getLambdaArgumentTypes(DataTypes & /*arguments*/) const
|
||||
{
|
||||
throw Exception("Function " + getName() + " can't have lambda-expressions as arguments", ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
|
||||
}
|
||||
|
||||
virtual ColumnNumbers getArgumentsThatAreAlwaysConstant() const { return {}; }
|
||||
virtual ColumnNumbers getArgumentsThatDontImplyNullableReturnType(size_t /*number_of_arguments*/) const { return {}; }
|
||||
|
||||
/** If useDefaultImplementationForNulls() is true, than change arguments for getReturnType() and build():
|
||||
* if some of arguments are Nullable(Nothing) then don't call getReturnType(), call build() with return_type = Nullable(Nothing),
|
||||
* if some of arguments are Nullable, then:
|
||||
* - Nullable types are substituted with nested types for getReturnType() function
|
||||
* - wrap getReturnType() result in Nullable type and pass to build
|
||||
*
|
||||
* Otherwise build returns build(arguments, getReturnType(arguments));
|
||||
*/
|
||||
virtual bool useDefaultImplementationForNulls() const { return true; }
|
||||
|
||||
/** If useDefaultImplementationForNulls() is true, than change arguments for getReturnType() and build().
|
||||
* If function arguments has low cardinality types, convert them to ordinary types.
|
||||
* getReturnType returns ColumnLowCardinality if at least one argument type is ColumnLowCardinality.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForLowCardinalityColumns() const { return true; }
|
||||
|
||||
/// If it isn't, will convert all ColumnLowCardinality arguments to full columns.
|
||||
virtual bool canBeExecutedOnLowCardinalityDictionary() const { return true; }
|
||||
};
|
||||
|
||||
using FunctionOverloadResolverImplPtr = std::unique_ptr<IFunctionOverloadResolverImpl>;
|
||||
|
||||
|
||||
/// Previous function interface.
|
||||
class IFunction
|
||||
{
|
||||
public:
|
||||
|
||||
virtual ~IFunction() = default;
|
||||
|
||||
virtual String getName() const = 0;
|
||||
|
||||
virtual ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const = 0;
|
||||
virtual ColumnPtr executeImplDryRun(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const
|
||||
{
|
||||
return executeImpl(arguments, result_type, input_rows_count);
|
||||
}
|
||||
|
||||
/** Default implementation in presence of Nullable arguments or NULL constants as arguments is the following:
|
||||
* if some of arguments are NULL constants then return NULL constant,
|
||||
* if some of arguments are Nullable, then execute function as usual for columns,
|
||||
* where Nullable columns are substituted with nested columns (they have arbitrary values in rows corresponding to NULL value)
|
||||
* and wrap result in Nullable column where NULLs are in all rows where any of arguments are NULL.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForNulls() const { return true; }
|
||||
|
||||
/** If the function have non-zero number of arguments,
|
||||
* and if all arguments are constant, that we could automatically provide default implementation:
|
||||
* arguments are converted to ordinary columns with single value, then function is executed as usual,
|
||||
* and then the result is converted to constant column.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForConstants() const { return false; }
|
||||
|
||||
/** If function arguments has single low cardinality column and all other arguments are constants, call function on nested column.
|
||||
* Otherwise, convert all low cardinality columns to ordinary columns.
|
||||
* Returns ColumnLowCardinality if at least one argument is ColumnLowCardinality.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForLowCardinalityColumns() const { return true; }
|
||||
|
||||
/// If it isn't, will convert all ColumnLowCardinality arguments to full columns.
|
||||
virtual bool canBeExecutedOnLowCardinalityDictionary() const { return true; }
|
||||
|
||||
/** Some arguments could remain constant during this implementation.
|
||||
*/
|
||||
virtual ColumnNumbers getArgumentsThatAreAlwaysConstant() const { return {}; }
|
||||
|
||||
/** True if function can be called on default arguments (include Nullable's) and won't throw.
|
||||
* Counterexample: modulo(0, 0)
|
||||
*/
|
||||
virtual bool canBeExecutedOnDefaultArguments() const { return true; }
|
||||
|
||||
/// Properties from IFunctionBase (see IFunction.h)
|
||||
virtual bool isSuitableForConstantFolding() const { return true; }
|
||||
virtual ColumnPtr getResultIfAlwaysReturnsConstantAndHasArguments(const ColumnsWithTypeAndName & /*arguments*/) const { return nullptr; }
|
||||
virtual bool isInjective(const ColumnsWithTypeAndName & /*sample_columns*/) const { return false; }
|
||||
virtual bool isDeterministic() const { return true; }
|
||||
virtual bool isDeterministicInScopeOfQuery() const { return true; }
|
||||
virtual bool isStateful() const { return false; }
|
||||
virtual bool hasInformationAboutMonotonicity() const { return false; }
|
||||
|
||||
using Monotonicity = IFunctionBase::Monotonicity;
|
||||
virtual Monotonicity getMonotonicityForRange(const IDataType & /*type*/, const Field & /*left*/, const Field & /*right*/) const
|
||||
{
|
||||
throw Exception("Function " + getName() + " has no information about its monotonicity.", ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
|
||||
/// For non-variadic functions, return number of arguments; otherwise return zero (that should be ignored).
|
||||
virtual size_t getNumberOfArguments() const = 0;
|
||||
|
||||
virtual DataTypePtr getReturnTypeImpl(const DataTypes & /*arguments*/) const
|
||||
{
|
||||
throw Exception("getReturnType is not implemented for " + getName(), ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
|
||||
/// Get the result type by argument type. If the function does not apply to these arguments, throw an exception.
|
||||
virtual DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const
|
||||
{
|
||||
DataTypes data_types(arguments.size());
|
||||
for (size_t i = 0; i < arguments.size(); ++i)
|
||||
data_types[i] = arguments[i].type;
|
||||
|
||||
return getReturnTypeImpl(data_types);
|
||||
}
|
||||
|
||||
virtual bool isVariadic() const { return false; }
|
||||
|
||||
virtual void getLambdaArgumentTypes(DataTypes & /*arguments*/) const
|
||||
{
|
||||
throw Exception("Function " + getName() + " can't have lambda-expressions as arguments", ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
|
||||
}
|
||||
|
||||
virtual ColumnNumbers getArgumentsThatDontImplyNullableReturnType(size_t /*number_of_arguments*/) const { return {}; }
|
||||
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
|
||||
bool isCompilable(const DataTypes & arguments) const;
|
||||
|
||||
llvm::Value * compile(llvm::IRBuilderBase &, const DataTypes & arguments, Values values) const;
|
||||
|
||||
#endif
|
||||
|
||||
protected:
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
|
||||
virtual bool isCompilableImpl(const DataTypes &) const { return false; }
|
||||
|
||||
virtual llvm::Value * compileImpl(llvm::IRBuilderBase &, const DataTypes &, Values) const
|
||||
{
|
||||
throw Exception(getName() + " is not JIT-compilable", ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
|
||||
#endif
|
||||
};
|
||||
|
||||
using FunctionPtr = std::shared_ptr<IFunction>;
|
||||
|
||||
}
|
91
src/Functions/IFunctionOld.cpp
Normal file
91
src/Functions/IFunctionOld.cpp
Normal file
@ -0,0 +1,91 @@
|
||||
#include "IFunctionOld.h"
|
||||
|
||||
#include <DataTypes/Native.h>
|
||||
|
||||
#if !defined(ARCADIA_BUILD)
|
||||
# include <Common/config.h>
|
||||
#endif
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
# pragma GCC diagnostic push
|
||||
# pragma GCC diagnostic ignored "-Wunused-parameter"
|
||||
# include <llvm/IR/IRBuilder.h>
|
||||
# pragma GCC diagnostic pop
|
||||
#endif
|
||||
|
||||
namespace DB
|
||||
{
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
|
||||
static std::optional<DataTypes> removeNullables(const DataTypes & types)
|
||||
{
|
||||
for (const auto & type : types)
|
||||
{
|
||||
if (!typeid_cast<const DataTypeNullable *>(type.get()))
|
||||
continue;
|
||||
DataTypes filtered;
|
||||
for (const auto & sub_type : types)
|
||||
filtered.emplace_back(removeNullable(sub_type));
|
||||
return filtered;
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
bool IFunction::isCompilable(const DataTypes & arguments) const
|
||||
{
|
||||
if (useDefaultImplementationForNulls())
|
||||
if (auto denulled = removeNullables(arguments))
|
||||
return isCompilableImpl(*denulled);
|
||||
return isCompilableImpl(arguments);
|
||||
}
|
||||
|
||||
llvm::Value * IFunction::compile(llvm::IRBuilderBase & builder, const DataTypes & arguments, Values values) const
|
||||
{
|
||||
auto denulled_arguments = removeNullables(arguments);
|
||||
if (useDefaultImplementationForNulls() && denulled_arguments)
|
||||
{
|
||||
auto & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
std::vector<llvm::Value*> unwrapped_values;
|
||||
std::vector<llvm::Value*> is_null_values;
|
||||
|
||||
unwrapped_values.reserve(arguments.size());
|
||||
is_null_values.reserve(arguments.size());
|
||||
|
||||
for (size_t i = 0; i < arguments.size(); ++i)
|
||||
{
|
||||
auto * value = values[i];
|
||||
|
||||
WhichDataType data_type(arguments[i]);
|
||||
if (data_type.isNullable())
|
||||
{
|
||||
unwrapped_values.emplace_back(b.CreateExtractValue(value, {0}));
|
||||
is_null_values.emplace_back(b.CreateExtractValue(value, {1}));
|
||||
}
|
||||
else
|
||||
{
|
||||
unwrapped_values.emplace_back(value);
|
||||
}
|
||||
}
|
||||
|
||||
auto * result = compileImpl(builder, *denulled_arguments, unwrapped_values);
|
||||
|
||||
auto * nullable_structure_type = toNativeType(b, makeNullable(getReturnTypeImpl(*denulled_arguments)));
|
||||
auto * nullable_structure_value = llvm::Constant::getNullValue(nullable_structure_type);
|
||||
|
||||
auto * nullable_structure_with_result_value = b.CreateInsertValue(nullable_structure_value, result, {0});
|
||||
auto * nullable_structure_result_null = b.CreateExtractValue(nullable_structure_with_result_value, {1});
|
||||
|
||||
for (auto * is_null_value : is_null_values)
|
||||
nullable_structure_result_null = b.CreateOr(nullable_structure_result_null, is_null_value);
|
||||
|
||||
return b.CreateInsertValue(nullable_structure_with_result_value, nullable_structure_result_null, {1});
|
||||
}
|
||||
|
||||
return compileImpl(builder, arguments, std::move(values));
|
||||
}
|
||||
|
||||
#endif
|
||||
|
||||
}
|
129
src/Functions/IFunctionOld.h
Normal file
129
src/Functions/IFunctionOld.h
Normal file
@ -0,0 +1,129 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/IFunction.h>
|
||||
|
||||
namespace DB
|
||||
{
|
||||
|
||||
namespace ErrorCodes
|
||||
{
|
||||
extern const int ILLEGAL_TYPE_OF_ARGUMENT;
|
||||
extern const int NOT_IMPLEMENTED;
|
||||
}
|
||||
|
||||
/// Old function interface. Check documentation in IFunction.h
|
||||
class IFunction
|
||||
{
|
||||
public:
|
||||
|
||||
virtual ~IFunction() = default;
|
||||
|
||||
virtual String getName() const = 0;
|
||||
|
||||
virtual ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const = 0;
|
||||
virtual ColumnPtr executeImplDryRun(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_count) const
|
||||
{
|
||||
return executeImpl(arguments, result_type, input_rows_count);
|
||||
}
|
||||
|
||||
/** Default implementation in presence of Nullable arguments or NULL constants as arguments is the following:
|
||||
* if some of arguments are NULL constants then return NULL constant,
|
||||
* if some of arguments are Nullable, then execute function as usual for columns,
|
||||
* where Nullable columns are substituted with nested columns (they have arbitrary values in rows corresponding to NULL value)
|
||||
* and wrap result in Nullable column where NULLs are in all rows where any of arguments are NULL.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForNulls() const { return true; }
|
||||
|
||||
/** If the function have non-zero number of arguments,
|
||||
* and if all arguments are constant, that we could automatically provide default implementation:
|
||||
* arguments are converted to ordinary columns with single value, then function is executed as usual,
|
||||
* and then the result is converted to constant column.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForConstants() const { return false; }
|
||||
|
||||
/** If function arguments has single low cardinality column and all other arguments are constants, call function on nested column.
|
||||
* Otherwise, convert all low cardinality columns to ordinary columns.
|
||||
* Returns ColumnLowCardinality if at least one argument is ColumnLowCardinality.
|
||||
*/
|
||||
virtual bool useDefaultImplementationForLowCardinalityColumns() const { return true; }
|
||||
|
||||
/// If it isn't, will convert all ColumnLowCardinality arguments to full columns.
|
||||
virtual bool canBeExecutedOnLowCardinalityDictionary() const { return true; }
|
||||
|
||||
/** Some arguments could remain constant during this implementation.
|
||||
*/
|
||||
virtual ColumnNumbers getArgumentsThatAreAlwaysConstant() const { return {}; }
|
||||
|
||||
/** True if function can be called on default arguments (include Nullable's) and won't throw.
|
||||
* Counterexample: modulo(0, 0)
|
||||
*/
|
||||
virtual bool canBeExecutedOnDefaultArguments() const { return true; }
|
||||
|
||||
/// Properties from IFunctionBase (see IFunction.h)
|
||||
virtual bool isSuitableForConstantFolding() const { return true; }
|
||||
virtual ColumnPtr getResultIfAlwaysReturnsConstantAndHasArguments(const ColumnsWithTypeAndName & /*arguments*/) const { return nullptr; }
|
||||
virtual bool isInjective(const ColumnsWithTypeAndName & /*sample_columns*/) const { return false; }
|
||||
virtual bool isDeterministic() const { return true; }
|
||||
virtual bool isDeterministicInScopeOfQuery() const { return true; }
|
||||
virtual bool isStateful() const { return false; }
|
||||
virtual bool hasInformationAboutMonotonicity() const { return false; }
|
||||
|
||||
using Monotonicity = IFunctionBase::Monotonicity;
|
||||
virtual Monotonicity getMonotonicityForRange(const IDataType & /*type*/, const Field & /*left*/, const Field & /*right*/) const
|
||||
{
|
||||
throw Exception("Function " + getName() + " has no information about its monotonicity.", ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
|
||||
/// For non-variadic functions, return number of arguments; otherwise return zero (that should be ignored).
|
||||
virtual size_t getNumberOfArguments() const = 0;
|
||||
|
||||
virtual DataTypePtr getReturnTypeImpl(const DataTypes & /*arguments*/) const
|
||||
{
|
||||
throw Exception("getReturnType is not implemented for " + getName(), ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
|
||||
/// Get the result type by argument type. If the function does not apply to these arguments, throw an exception.
|
||||
virtual DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const
|
||||
{
|
||||
DataTypes data_types(arguments.size());
|
||||
for (size_t i = 0; i < arguments.size(); ++i)
|
||||
data_types[i] = arguments[i].type;
|
||||
|
||||
return getReturnTypeImpl(data_types);
|
||||
}
|
||||
|
||||
virtual bool isVariadic() const { return false; }
|
||||
|
||||
virtual void getLambdaArgumentTypes(DataTypes & /*arguments*/) const
|
||||
{
|
||||
throw Exception("Function " + getName() + " can't have lambda-expressions as arguments", ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
|
||||
}
|
||||
|
||||
virtual ColumnNumbers getArgumentsThatDontImplyNullableReturnType(size_t /*number_of_arguments*/) const { return {}; }
|
||||
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
|
||||
bool isCompilable(const DataTypes & arguments) const;
|
||||
|
||||
llvm::Value * compile(llvm::IRBuilderBase &, const DataTypes & arguments, Values values) const;
|
||||
|
||||
#endif
|
||||
|
||||
protected:
|
||||
|
||||
#if USE_EMBEDDED_COMPILER
|
||||
|
||||
virtual bool isCompilableImpl(const DataTypes &) const { return false; }
|
||||
|
||||
virtual llvm::Value * compileImpl(llvm::IRBuilderBase &, const DataTypes &, Values) const
|
||||
{
|
||||
throw Exception(getName() + " is not JIT-compilable", ErrorCodes::NOT_IMPLEMENTED);
|
||||
}
|
||||
|
||||
#endif
|
||||
};
|
||||
|
||||
using FunctionPtr = std::shared_ptr<IFunction>;
|
||||
|
||||
}
|
@ -4,7 +4,7 @@
|
||||
#include <DataTypes/NumberTraits.h>
|
||||
#include <Interpreters/castColumn.h>
|
||||
#include <Columns/ColumnsNumber.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <ext/map.h>
|
||||
|
||||
@ -87,12 +87,12 @@ private:
|
||||
|
||||
|
||||
template <LeastGreatest kind, typename SpecializedFunction>
|
||||
class LeastGreatestOverloadResolver : public IFunctionOverloadResolverImpl
|
||||
class LeastGreatestOverloadResolver : public IFunctionOverloadResolver
|
||||
{
|
||||
public:
|
||||
static constexpr auto name = kind == LeastGreatest::Least ? "least" : "greatest";
|
||||
|
||||
static FunctionOverloadResolverImplPtr create(ContextPtr context)
|
||||
static FunctionOverloadResolverPtr create(ContextPtr context)
|
||||
{
|
||||
return std::make_unique<LeastGreatestOverloadResolver<kind, SpecializedFunction>>(context);
|
||||
}
|
||||
@ -103,19 +103,19 @@ public:
|
||||
size_t getNumberOfArguments() const override { return 0; }
|
||||
bool isVariadic() const override { return true; }
|
||||
|
||||
FunctionBaseImplPtr build(const ColumnsWithTypeAndName & arguments, const DataTypePtr & return_type) const override
|
||||
FunctionBasePtr buildImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & return_type) const override
|
||||
{
|
||||
DataTypes argument_types;
|
||||
|
||||
/// More efficient specialization for two numeric arguments.
|
||||
if (arguments.size() == 2 && isNumber(arguments[0].type) && isNumber(arguments[1].type))
|
||||
return std::make_unique<DefaultFunction>(SpecializedFunction::create(context), argument_types, return_type);
|
||||
return std::make_unique<FunctionToFunctionBaseAdaptor>(SpecializedFunction::create(context), argument_types, return_type);
|
||||
|
||||
return std::make_unique<DefaultFunction>(
|
||||
return std::make_unique<FunctionToFunctionBaseAdaptor>(
|
||||
FunctionLeastGreatestGeneric<kind>::create(context), argument_types, return_type);
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType(const DataTypes & types) const override
|
||||
DataTypePtr getReturnTypeImpl(const DataTypes & types) const override
|
||||
{
|
||||
if (types.empty())
|
||||
throw Exception("Function " + getName() + " cannot be called without arguments", ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH);
|
||||
|
@ -1,7 +1,7 @@
|
||||
#pragma once
|
||||
|
||||
#include <Functions/TargetSpecific.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
|
||||
#include <Common/Stopwatch.h>
|
||||
#include <Interpreters/Context.h>
|
||||
|
@ -1,5 +1,5 @@
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Common/StringUtils/StringUtils.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <Columns/ColumnsNumber.h>
|
||||
|
@ -7,7 +7,7 @@
|
||||
#include <Columns/ColumnString.h>
|
||||
#include <Columns/ColumnsNumber.h>
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <IO/WriteBufferFromArena.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
|
@ -4,7 +4,7 @@
|
||||
#include <Columns/ColumnString.h>
|
||||
#include <Columns/ColumnsNumber.h>
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Access/AccessFlags.h>
|
||||
#include <Interpreters/Context.h>
|
||||
|
@ -3,7 +3,7 @@
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <ext/range.h>
|
||||
|
||||
|
||||
|
@ -8,7 +8,7 @@
|
||||
#include <Columns/ColumnFunction.h>
|
||||
#include <Common/typeid_cast.h>
|
||||
#include <Common/assert_cast.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include <Interpreters/Context_fwd.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunction.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
#include <DataTypes/getLeastSupertype.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/GatherUtils/GatherUtils.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,5 +1,5 @@
|
||||
#pragma once
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
|
@ -7,7 +7,7 @@
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
#include <DataTypes/getLeastSupertype.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Interpreters/AggregationCommon.h>
|
||||
#include <Common/ColumnsHashing.h>
|
||||
#include <Common/HashTable/ClearableHashMap.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,5 +1,5 @@
|
||||
#pragma once
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,5 +1,5 @@
|
||||
#pragma once
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/GatherUtils/GatherUtils.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
#include <Columns/ColumnArray.h>
|
||||
|
@ -1,5 +1,5 @@
|
||||
#pragma once
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/GatherUtils/GatherUtils.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
#include <DataTypes/getLeastSupertype.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/GatherUtils/GatherUtils.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/GatherUtils/GatherUtils.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,5 +1,5 @@
|
||||
#pragma once
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/GatherUtils/GatherUtils.h>
|
||||
#include <DataTypes/DataTypeArray.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <DataTypes/DataTypesNumber.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <DataTypes/DataTypeNullable.h>
|
||||
#include <Core/ColumnNumbers.h>
|
||||
|
@ -1,4 +1,4 @@
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
#include <Columns/ColumnString.h>
|
||||
|
@ -1,5 +1,5 @@
|
||||
#include <Functions/FunctionFactory.h>
|
||||
#include <Functions/IFunctionImpl.h>
|
||||
#include <Functions/IFunctionOld.h>
|
||||
#include <Columns/ColumnString.h>
|
||||
#include <Columns/ColumnVector.h>
|
||||
#include <DataTypes/DataTypeString.h>
|
||||
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user