Merge pull request #23028 from ClickHouse/cast_to_value_or_default

Merging #21330
This commit is contained in:
alexey-milovidov 2021-10-03 11:00:48 +03:00 committed by GitHub
commit ac57e057a1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 706 additions and 30 deletions

View File

@ -322,7 +322,7 @@ Truncates sub-seconds.
**Syntax** **Syntax**
``` sql ``` sql
toStartOfSecond(value[, timezone]) toStartOfSecond(value, [timezone])
``` ```
**Arguments** **Arguments**

View File

@ -90,6 +90,27 @@ Result:
└─────────────────────────┴───────────────────────────┘ └─────────────────────────┴───────────────────────────┘
``` ```
## toInt(8\|16\|32\|64\|128\|256)OrDefault {#toint8163264128256orDefault}
It takes an argument of type String and tries to parse it into Int (8 \| 16 \| 32 \| 64 \| 128 \| 256). If failed, returns the default type value.
**Example**
Query:
``` sql
SELECT toInt64OrDefault('123123', cast('-1' as Int64)), toInt8OrDefault('123qwe123', cast('-1' as Int8));
```
Result:
``` text
┌─toInt64OrDefault('123123', CAST('-1', 'Int64'))─┬─toInt8OrDefault('123qwe123', CAST('-1', 'Int8'))─┐
│ 123123 │ -1 │
└─────────────────────────────────────────────────┴──────────────────────────────────────────────────┘
```
## toUInt(8\|16\|32\|64\|256) {#touint8163264256} ## toUInt(8\|16\|32\|64\|256) {#touint8163264256}
Converts an input value to the [UInt](../../sql-reference/data-types/int-uint.md) data type. This function family includes: Converts an input value to the [UInt](../../sql-reference/data-types/int-uint.md) data type. This function family includes:
@ -132,12 +153,16 @@ Result:
## toUInt(8\|16\|32\|64\|256)OrNull {#touint8163264256ornull} ## toUInt(8\|16\|32\|64\|256)OrNull {#touint8163264256ornull}
## toUInt(8\|16\|32\|64\|256)OrDefault {#touint8163264256ordefault}
## toFloat(32\|64) {#tofloat3264} ## toFloat(32\|64) {#tofloat3264}
## toFloat(32\|64)OrZero {#tofloat3264orzero} ## toFloat(32\|64)OrZero {#tofloat3264orzero}
## toFloat(32\|64)OrNull {#tofloat3264ornull} ## toFloat(32\|64)OrNull {#tofloat3264ornull}
## toFloat(32\|64)OrDefault {#tofloat3264ordefault}
## toDate {#todate} ## toDate {#todate}
Alias: `DATE`. Alias: `DATE`.
@ -146,23 +171,27 @@ Alias: `DATE`.
## toDateOrNull {#todateornull} ## toDateOrNull {#todateornull}
## toDateOrDefault {#todateordefault}
## toDateTime {#todatetime} ## toDateTime {#todatetime}
## toDateTimeOrZero {#todatetimeorzero} ## toDateTimeOrZero {#todatetimeorzero}
## toDateTimeOrNull {#todatetimeornull} ## toDateTimeOrNull {#todatetimeornull}
## toDateTimeOrDefault {#todatetimeordefault}
## toDate32 {#todate32} ## toDate32 {#todate32}
Converts the argument to the [Date32](../../sql-reference/data-types/date32.md) data type. If the value is outside the range returns the border values supported by `Date32`. If the argument has [Date](../../sql-reference/data-types/date.md) type, borders of `Date` are taken into account. Converts the argument to the [Date32](../../sql-reference/data-types/date32.md) data type. If the value is outside the range returns the border values supported by `Date32`. If the argument has [Date](../../sql-reference/data-types/date.md) type, borders of `Date` are taken into account.
**Syntax** **Syntax**
``` sql ``` sql
toDate32(expr) toDate32(expr)
``` ```
**Arguments** **Arguments**
- `expr` — The value. [String](../../sql-reference/data-types/string.md), [UInt32](../../sql-reference/data-types/int-uint.md) or [Date](../../sql-reference/data-types/date.md). - `expr` — The value. [String](../../sql-reference/data-types/string.md), [UInt32](../../sql-reference/data-types/int-uint.md) or [Date](../../sql-reference/data-types/date.md).
@ -250,6 +279,26 @@ Result:
└──────────────────────────────┴────────────────────┘ └──────────────────────────────┴────────────────────┘
``` ```
## toDate32OrDefault {#todate32-or-null}
The same as [toDate32](#todate32) but returns default value if invalid argument is received.
**Example**
Query:
``` sql
SELECT toDate32OrDefault('1955-01-01'), toDate32OrDefault('');
```
Result:
``` text
┌─toDate32OrDefault('1955-01-01')─┬─toDate32OrDefault('')─┐
│ 1955-01-01 │ 1970-01-01 │
└─────────────────────────────────┴───────────────────────┘
```
## toDecimal(32\|64\|128\|256) {#todecimal3264128256} ## toDecimal(32\|64\|128\|256) {#todecimal3264128256}
Converts `value` to the [Decimal](../../sql-reference/data-types/decimal.md) data type with precision of `S`. The `value` can be a number or a string. The `S` (scale) parameter specifies the number of decimal places. Converts `value` to the [Decimal](../../sql-reference/data-types/decimal.md) data type with precision of `S`. The `value` can be a number or a string. The `S` (scale) parameter specifies the number of decimal places.
@ -312,6 +361,60 @@ Result:
└──────┴────────────────────────────────────────────────────┘ └──────┴────────────────────────────────────────────────────┘
``` ```
## toDecimal(32\|64\|128\|256)OrDefault {#todecimal3264128256ornull}
Converts an input string to a [Decimal(P,S)](../../sql-reference/data-types/decimal.md) data type value. This family of functions include:
- `toDecimal32OrDefault(expr, S)` — Results in `Decimal32(S)` data type.
- `toDecimal64OrDefault(expr, S)` — Results in `Decimal64(S)` data type.
- `toDecimal128OrDefault(expr, S)` — Results in `Decimal128(S)` data type.
- `toDecimal256OrDefault(expr, S)` — Results in `Decimal256(S)` data type.
These functions should be used instead of `toDecimal*()` functions, if you prefer to get a default value instead of an exception in the event of an input value parsing error.
**Arguments**
- `expr` — [Expression](../../sql-reference/syntax.md#syntax-expressions), returns a value in the [String](../../sql-reference/data-types/string.md) data type. ClickHouse expects the textual representation of the decimal number. For example, `'1.111'`.
- `S` — Scale, the number of decimal places in the resulting value.
**Returned value**
A value in the `Decimal(P,S)` data type. The value contains:
- Number with `S` decimal places, if ClickHouse interprets the input string as a number.
- Default `Decimal(P,S)` data type value, if ClickHouse cant interpret the input string as a number or if the input number contains more than `S` decimal places.
**Examples**
Query:
``` sql
SELECT toDecimal32OrDefault(toString(-1.111), 5) AS val, toTypeName(val);
```
Result:
``` text
┌────val─┬─toTypeName(toDecimal32OrDefault(toString(-1.111), 5))─┐
│ -1.111 │ Decimal(9, 5) │
└────────┴───────────────────────────────────────────────────────┘
```
Query:
``` sql
SELECT toDecimal32OrDefault(toString(-1.111), 2) AS val, toTypeName(val);
```
Result:
``` text
┌─val─┬─toTypeName(toDecimal32OrDefault(toString(-1.111), 2))─┐
│ 0 │ Decimal(9, 2) │
└─────┴───────────────────────────────────────────────────────┘
```
## toDecimal(32\|64\|128\|256)OrZero {#todecimal3264128256orzero} ## toDecimal(32\|64\|128\|256)OrZero {#todecimal3264128256orzero}
Converts an input value to the [Decimal(P,S)](../../sql-reference/data-types/decimal.md) data type. This family of functions include: Converts an input value to the [Decimal(P,S)](../../sql-reference/data-types/decimal.md) data type. This family of functions include:
@ -751,6 +854,63 @@ Result:
└───────┴──────┴──────────────┘ └───────┴──────┴──────────────┘
``` ```
## accurateCastOrDefault(x, T[, default_value]) {#type_conversion_function-accurate-cast_or_null}
Converts input value `x` to the specified data type `T`. Returns default type value or `default_value` if specified if the casted value is not representable in the target type.
**Syntax**
```sql
accurateCastOrNull(x, T)
```
**Parameters**
- `x` — Input value.
- `T` — The name of the returned data type.
- `default_value` - Default value of returned data type.
**Returned value**
- The value, converted to the specified data type `T`.
**Example**
Query:
``` sql
SELECT toTypeName(accurateCastOrDefault(5, 'UInt8'));
```
Result:
``` text
┌─toTypeName(accurateCastOrDefault(5, 'UInt8'))─┐
│ UInt8 │
└───────────────────────────────────────────────┘
```
Query:
``` sql
SELECT
accurateCastOrDefault(-1, 'UInt8') as uint8,
accurateCastOrDefault(-1, 'UInt8', 5) as uint8_default,
accurateCastOrDefault(128, 'Int8') as int8,
accurateCastOrDefault(128, 'Int8', 5) as int8_default,
accurateCastOrDefault('Test', 'FixedString(2)') as fixed_string,
accurateCastOrDefault('Test', 'FixedString(2)', 'Te') as fixed_string_default;
```
Result:
``` text
┌─uint8─┬─uint8_default─┬─int8─┬─int8_default─┬─fixed_string─┬─fixed_string_default─┐
│ 0 │ 5 │ 0 │ 5 │ │ Te │
└───────┴───────────────┴──────┴──────────────┴──────────────┴──────────────────────┘
```
## toInterval(Year\|Quarter\|Month\|Week\|Day\|Hour\|Minute\|Second) {#function-tointerval} ## toInterval(Year\|Quarter\|Month\|Week\|Day\|Hour\|Minute\|Second) {#function-tointerval}
Converts a Number type argument to an [Interval](../../sql-reference/data-types/special-data-types/interval.md) data type. Converts a Number type argument to an [Interval](../../sql-reference/data-types/special-data-types/interval.md) data type.

View File

@ -224,14 +224,19 @@ checkAndGetNestedArrayOffset(const IColumn ** columns, size_t num_arguments)
return {nested_columns, offsets->data()}; return {nested_columns, offsets->data()};
} }
bool areTypesEqual(const DataTypePtr & lhs, const DataTypePtr & rhs) bool areTypesEqual(const IDataType & lhs, const IDataType & rhs)
{ {
const auto & lhs_name = lhs->getName(); const auto & lhs_name = lhs.getName();
const auto & rhs_name = rhs->getName(); const auto & rhs_name = rhs.getName();
return lhs_name == rhs_name; return lhs_name == rhs_name;
} }
bool areTypesEqual(const DataTypePtr & lhs, const DataTypePtr & rhs)
{
return areTypesEqual(*lhs, *rhs);
}
ColumnPtr wrapInNullable(const ColumnPtr & src, const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count) ColumnPtr wrapInNullable(const ColumnPtr & src, const ColumnsWithTypeAndName & args, const DataTypePtr & result_type, size_t input_rows_count)
{ {
ColumnPtr result_null_map_column; ColumnPtr result_null_map_column;

View File

@ -108,8 +108,8 @@ struct FunctionArgumentDescriptor
{ {
const char * argument_name; const char * argument_name;
bool (* type_validator_func)(const IDataType &); std::function<bool (const IDataType &)> type_validator_func;
bool (* column_validator_func)(const IColumn &); std::function<bool (const IColumn &)> column_validator_func;
const char * expected_type_description; const char * expected_type_description;
@ -155,8 +155,9 @@ void validateFunctionArgumentTypes(const IFunction & func, const ColumnsWithType
std::pair<std::vector<const IColumn *>, const ColumnArray::Offset *> std::pair<std::vector<const IColumn *>, const ColumnArray::Offset *>
checkAndGetNestedArrayOffset(const IColumn ** columns, size_t num_arguments); checkAndGetNestedArrayOffset(const IColumn ** columns, size_t num_arguments);
/// Check if two types are equal /// Check if two types are equal
bool areTypesEqual(const IDataType & lhs, const IDataType & rhs);
bool areTypesEqual(const DataTypePtr & lhs, const DataTypePtr & rhs); bool areTypesEqual(const DataTypePtr & lhs, const DataTypePtr & rhs);
/** Return ColumnNullable of src, with null map as OR-ed null maps of args columns. /** Return ColumnNullable of src, with null map as OR-ed null maps of args columns.

View File

@ -156,21 +156,21 @@ private:
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
{ {
auto optional_args = FunctionArgumentDescriptors{ auto optional_args = FunctionArgumentDescriptors{
{"IV", isStringOrFixedString, nullptr, "Initialization vector binary string"}, {"IV", &isStringOrFixedString<IDataType>, nullptr, "Initialization vector binary string"},
}; };
if constexpr (compatibility_mode == OpenSSLDetails::CompatibilityMode::OpenSSL) if constexpr (compatibility_mode == OpenSSLDetails::CompatibilityMode::OpenSSL)
{ {
optional_args.emplace_back(FunctionArgumentDescriptor{ optional_args.emplace_back(FunctionArgumentDescriptor{
"AAD", isStringOrFixedString, nullptr, "Additional authenticated data binary string for GCM mode" "AAD", &isStringOrFixedString<IDataType>, nullptr, "Additional authenticated data binary string for GCM mode"
}); });
} }
validateFunctionArgumentTypes(*this, arguments, validateFunctionArgumentTypes(*this, arguments,
FunctionArgumentDescriptors{ FunctionArgumentDescriptors{
{"mode", isStringOrFixedString, isColumnConst, "encryption mode string"}, {"mode", &isStringOrFixedString<IDataType>, isColumnConst, "encryption mode string"},
{"input", isStringOrFixedString, nullptr, "plaintext"}, {"input", &isStringOrFixedString<IDataType>, nullptr, "plaintext"},
{"key", isStringOrFixedString, nullptr, "encryption key binary string"}, {"key", &isStringOrFixedString<IDataType>, nullptr, "encryption key binary string"},
}, },
optional_args optional_args
); );
@ -432,21 +432,21 @@ private:
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
{ {
auto optional_args = FunctionArgumentDescriptors{ auto optional_args = FunctionArgumentDescriptors{
{"IV", isStringOrFixedString, nullptr, "Initialization vector binary string"}, {"IV", &isStringOrFixedString<IDataType>, nullptr, "Initialization vector binary string"},
}; };
if constexpr (compatibility_mode == OpenSSLDetails::CompatibilityMode::OpenSSL) if constexpr (compatibility_mode == OpenSSLDetails::CompatibilityMode::OpenSSL)
{ {
optional_args.emplace_back(FunctionArgumentDescriptor{ optional_args.emplace_back(FunctionArgumentDescriptor{
"AAD", isStringOrFixedString, nullptr, "Additional authenticated data binary string for GCM mode" "AAD", &isStringOrFixedString<IDataType>, nullptr, "Additional authenticated data binary string for GCM mode"
}); });
} }
validateFunctionArgumentTypes(*this, arguments, validateFunctionArgumentTypes(*this, arguments,
FunctionArgumentDescriptors{ FunctionArgumentDescriptors{
{"mode", isStringOrFixedString, isColumnConst, "decryption mode string"}, {"mode", &isStringOrFixedString<IDataType>, isColumnConst, "decryption mode string"},
{"input", nullptr, nullptr, "ciphertext"}, {"input", nullptr, nullptr, "ciphertext"},
{"key", isStringOrFixedString, nullptr, "decryption key binary string"}, {"key", &isStringOrFixedString<IDataType>, nullptr, "decryption key binary string"},
}, },
optional_args optional_args
); );

View File

@ -1510,12 +1510,12 @@ public:
if constexpr (to_decimal) if constexpr (to_decimal)
{ {
mandatory_args.push_back({"scale", &isNativeInteger, &isColumnConst, "const Integer"}); mandatory_args.push_back({"scale", &isNativeInteger<IDataType>, &isColumnConst, "const Integer"});
} }
if (!to_decimal && isDateTime64<Name, ToDataType>(arguments)) if (!to_decimal && isDateTime64<Name, ToDataType>(arguments))
{ {
mandatory_args.push_back({"scale", &isNativeInteger, &isColumnConst, "const Integer"}); mandatory_args.push_back({"scale", &isNativeInteger<IDataType>, &isColumnConst, "const Integer"});
} }
// toString(DateTime or DateTime64, [timezone: String]) // toString(DateTime or DateTime64, [timezone: String])
@ -1531,7 +1531,7 @@ public:
// toDateTime64(value, scale : Integer[, timezone: String]) // toDateTime64(value, scale : Integer[, timezone: String])
|| std::is_same_v<ToDataType, DataTypeDateTime64>) || std::is_same_v<ToDataType, DataTypeDateTime64>)
{ {
optional_args.push_back({"timezone", &isString, &isColumnConst, "const String"}); optional_args.push_back({"timezone", &isString<IDataType>, &isColumnConst, "const String"});
} }
validateFunctionArgumentTypes(*this, arguments, mandatory_args, optional_args); validateFunctionArgumentTypes(*this, arguments, mandatory_args, optional_args);
@ -1811,11 +1811,11 @@ public:
if (isDateTime64<Name, ToDataType>(arguments)) if (isDateTime64<Name, ToDataType>(arguments))
{ {
validateFunctionArgumentTypes(*this, arguments, validateFunctionArgumentTypes(*this, arguments,
FunctionArgumentDescriptors{{"string", isStringOrFixedString, nullptr, "String or FixedString"}}, FunctionArgumentDescriptors{{"string", &isStringOrFixedString<IDataType>, nullptr, "String or FixedString"}},
// optional // optional
FunctionArgumentDescriptors{ FunctionArgumentDescriptors{
{"precision", isUInt8, isColumnConst, "const UInt8"}, {"precision", &isUInt8<IDataType>, isColumnConst, "const UInt8"},
{"timezone", isStringOrFixedString, isColumnConst, "const String or FixedString"}, {"timezone", &isStringOrFixedString<IDataType>, isColumnConst, "const String or FixedString"},
}); });
UInt64 scale = to_datetime64 ? DataTypeDateTime64::default_scale : 0; UInt64 scale = to_datetime64 ? DataTypeDateTime64::default_scale : 0;

View File

@ -0,0 +1,384 @@
#include <DataTypes/DataTypeFactory.h>
#include <DataTypes/DataTypeNullable.h>
#include <DataTypes/DataTypeString.h>
#include <DataTypes/DataTypesNumber.h>
#include <DataTypes/DataTypesDecimal.h>
#include <DataTypes/DataTypeDate.h>
#include <DataTypes/DataTypeDate32.h>
#include <DataTypes/DataTypeDateTime.h>
#include <DataTypes/DataTypeDateTime64.h>
#include <DataTypes/DataTypeUUID.h>
#include <Columns/ColumnString.h>
#include <Columns/ColumnConst.h>
#include <Columns/ColumnNullable.h>
#include <Interpreters/Context.h>
#include <Interpreters/castColumn.h>
#include <Functions/IFunction.h>
#include <Functions/FunctionHelpers.h>
#include <Functions/FunctionFactory.h>
#include <Functions/extractTimeZoneFromFunctionArguments.h>
namespace DB
{
namespace ErrorCodes
{
extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
extern const int BAD_ARGUMENTS;
extern const int ILLEGAL_TYPE_OF_ARGUMENT;
}
class FunctionCastOrDefault final : public IFunction
{
public:
static constexpr auto name = "accurateCastOrDefault";
static FunctionPtr create(ContextPtr context)
{
return std::make_shared<FunctionCastOrDefault>(context);
}
explicit FunctionCastOrDefault(ContextPtr context_)
: keep_nullable(context_->getSettingsRef().cast_keep_nullable)
{
}
String getName() const override { return name; }
size_t getNumberOfArguments() const override { return 0; }
bool isVariadic() const override { return true; }
ColumnNumbers getArgumentsThatAreAlwaysConstant() const override { return {1}; }
bool useDefaultImplementationForNulls() const override { return false; }
bool useDefaultImplementationForConstants() const override { return false; }
bool useDefaultImplementationForLowCardinalityColumns() const override { return true; }
bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return false; }
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
{
size_t arguments_size = arguments.size();
if (arguments_size != 2 && arguments_size != 3)
throw Exception(ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
"Function {} expected 2 or 3 arguments. Actual {}",
getName(),
arguments_size);
const auto & type_column = arguments[1].column;
const auto * type_column_typed = checkAndGetColumnConst<ColumnString>(type_column.get());
if (!type_column_typed)
throw Exception(ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
"Second argument to {} must be a constant string describing type. Actual {}",
getName(),
arguments[1].type->getName());
DataTypePtr result_type = DataTypeFactory::instance().get(type_column_typed->getValue<String>());
if (keep_nullable && arguments.front().type->isNullable())
result_type = makeNullable(result_type);
if (arguments.size() == 3)
{
auto default_value_type = arguments[2].type;
if (!areTypesEqual(result_type, default_value_type))
{
throw Exception(ErrorCodes::BAD_ARGUMENTS,
"Default value type should be same as cast type. Expected {}. Actual {}",
result_type->getName(),
default_value_type->getName());
}
}
return result_type;
}
ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & return_type, size_t) const override
{
const ColumnWithTypeAndName & column_to_cast = arguments[0];
auto non_const_column_to_cast = column_to_cast.column->convertToFullColumnIfConst();
ColumnWithTypeAndName column_to_cast_non_const { std::move(non_const_column_to_cast), column_to_cast.type, column_to_cast.name };
auto cast_result = castColumnAccurateOrNull(column_to_cast_non_const, return_type);
const auto & cast_result_nullable = assert_cast<const ColumnNullable &>(*cast_result);
const auto & null_map_data = cast_result_nullable.getNullMapData();
size_t null_map_data_size = null_map_data.size();
const auto & nested_column = cast_result_nullable.getNestedColumn();
IColumn::MutablePtr result = return_type->createColumn();
result->reserve(null_map_data_size);
size_t start_insert_index = 0;
/// Created separate branch because cast and inserting field from other column is slower
if (arguments.size() == 3)
{
const auto & default_column_with_type = arguments[2];
auto default_column = default_column_with_type.column->convertToFullColumnIfConst();
for (size_t i = 0; i < null_map_data_size; ++i)
{
bool is_current_index_null = null_map_data[i];
if (!is_current_index_null)
continue;
if (i != start_insert_index)
result->insertRangeFrom(nested_column, start_insert_index, i - start_insert_index);
result->insertFrom(*default_column, i);
start_insert_index = i + 1;
}
}
else
{
for (size_t i = 0; i < null_map_data_size; ++i)
{
bool is_current_index_null = null_map_data[i];
if (!is_current_index_null)
continue;
if (i != start_insert_index)
result->insertRangeFrom(nested_column, start_insert_index, i - start_insert_index);
result->insertDefault();
start_insert_index = i + 1;
}
}
if (null_map_data_size != start_insert_index)
result->insertRangeFrom(nested_column, start_insert_index, null_map_data_size - start_insert_index);
return result;
}
private:
bool keep_nullable;
};
template <typename Type, typename Name>
class FunctionCastOrDefaultTyped final : public IFunction
{
public:
static constexpr auto name = Name::name;
static FunctionPtr create(ContextPtr context)
{
return std::make_shared<FunctionCastOrDefaultTyped>(context);
}
explicit FunctionCastOrDefaultTyped(ContextPtr context_)
: impl(context_)
{
}
String getName() const override { return name; }
private:
size_t getNumberOfArguments() const override { return 0; }
bool isVariadic() const override { return true; }
bool useDefaultImplementationForNulls() const override { return impl.useDefaultImplementationForNulls(); }
bool useDefaultImplementationForLowCardinalityColumns() const override { return impl.useDefaultImplementationForLowCardinalityColumns();}
bool useDefaultImplementationForConstants() const override { return impl.useDefaultImplementationForConstants();}
bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & arguments) const override
{
return impl.isSuitableForShortCircuitArgumentsExecution(arguments);
}
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
{
FunctionArgumentDescriptors mandatory_args = {{"Value", nullptr, nullptr, nullptr}};
FunctionArgumentDescriptors optional_args;
if constexpr (IsDataTypeDecimal<Type>)
mandatory_args.push_back({"scale", &isNativeInteger<IDataType>, &isColumnConst, "const Integer"});
if (std::is_same_v<Type, DataTypeDateTime> || std::is_same_v<Type, DataTypeDateTime64>)
optional_args.push_back({"timezone", &isString<IDataType>, isColumnConst, "const String"});
optional_args.push_back({"default_value", nullptr, nullptr, nullptr});
validateFunctionArgumentTypes(*this, arguments, mandatory_args, optional_args);
size_t additional_argument_index = 1;
size_t scale = 0;
std::string time_zone;
if constexpr (IsDataTypeDecimal<Type>)
{
const auto & scale_argument = arguments[additional_argument_index];
WhichDataType scale_argument_type(scale_argument.type);
if (!scale_argument_type.isNativeUInt())
{
throw Exception(ErrorCodes::BAD_ARGUMENTS,
"Function {} decimal scale should have native UInt type. Actual {}",
scale_argument.type->getName());
}
scale = arguments[additional_argument_index].column->getUInt(0);
++additional_argument_index;
}
if constexpr (std::is_same_v<Type, DataTypeDateTime> || std::is_same_v<Type, DataTypeDateTime64>)
{
if (additional_argument_index < arguments.size())
{
time_zone = extractTimeZoneNameFromColumn(*arguments[additional_argument_index].column);
++additional_argument_index;
}
}
std::shared_ptr<Type> cast_type;
if constexpr (std::is_same_v<Type, DataTypeDateTime64>)
cast_type = std::make_shared<Type>(scale, time_zone);
else if constexpr (IsDataTypeDecimal<Type>)
cast_type = std::make_shared<Type>(Type::maxPrecision(), scale);
else if constexpr (std::is_same_v<Type, DataTypeDateTime> || std::is_same_v<Type, DataTypeDateTime64>)
cast_type = std::make_shared<Type>(time_zone);
else
cast_type = std::make_shared<Type>();
ColumnWithTypeAndName type_argument =
{
DataTypeString().createColumnConst(1, cast_type->getName()),
std::make_shared<DataTypeString>(),
""
};
ColumnsWithTypeAndName arguments_with_cast_type;
arguments_with_cast_type.reserve(arguments.size());
arguments_with_cast_type.emplace_back(arguments[0]);
arguments_with_cast_type.emplace_back(type_argument);
if (additional_argument_index < arguments.size())
{
arguments_with_cast_type.emplace_back(arguments[additional_argument_index]);
++additional_argument_index;
}
if (additional_argument_index < arguments.size())
throw Exception(ErrorCodes::BAD_ARGUMENTS, "{} wrong arguments size", getName());
return impl.getReturnTypeImpl(arguments_with_cast_type);
}
ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & result_type, size_t input_rows_size) const override
{
size_t additional_arguments_size = IsDataTypeDecimal<Type> + (std::is_same_v<Type, DataTypeDateTime> || std::is_same_v<Type, DataTypeDateTime64>);
ColumnWithTypeAndName second_argument =
{
DataTypeString().createColumnConst(arguments.begin()->column->size(), result_type->getName()),
std::make_shared<DataTypeString>(),
""
};
ColumnsWithTypeAndName arguments_with_cast_type;
arguments_with_cast_type.reserve(arguments.size());
arguments_with_cast_type.emplace_back(arguments[0]);
arguments_with_cast_type.emplace_back(second_argument);
size_t default_column_argument = 1 + additional_arguments_size;
if (default_column_argument < arguments.size())
arguments_with_cast_type.emplace_back(arguments[default_column_argument]);
return impl.executeImpl(arguments_with_cast_type, result_type, input_rows_size);
}
FunctionCastOrDefault impl;
};
struct NameToUInt8OrDefault { static constexpr auto name = "toUInt8OrDefault"; };
struct NameToUInt16OrDefault { static constexpr auto name = "toUInt16OrDefault"; };
struct NameToUInt32OrDefault { static constexpr auto name = "toUInt32OrDefault"; };
struct NameToUInt64OrDefault { static constexpr auto name = "toUInt64OrDefault"; };
struct NameToUInt256OrDefault { static constexpr auto name = "toUInt256OrDefault"; };
struct NameToInt8OrDefault { static constexpr auto name = "toInt8OrDefault"; };
struct NameToInt16OrDefault { static constexpr auto name = "toInt16OrDefault"; };
struct NameToInt32OrDefault { static constexpr auto name = "toInt32OrDefault"; };
struct NameToInt64OrDefault { static constexpr auto name = "toInt64OrDefault"; };
struct NameToInt128OrDefault { static constexpr auto name = "toInt128OrDefault"; };
struct NameToInt256OrDefault { static constexpr auto name = "toInt256OrDefault"; };
struct NameToFloat32OrDefault { static constexpr auto name = "toFloat32OrDefault"; };
struct NameToFloat64OrDefault { static constexpr auto name = "toFloat64OrDefault"; };
struct NameToDateOrDefault { static constexpr auto name = "toDateOrDefault"; };
struct NameToDate32OrDefault { static constexpr auto name = "toDate32OrDefault"; };
struct NameToDateTimeOrDefault { static constexpr auto name = "toDateTimeOrDefault"; };
struct NameToDateTime64OrDefault { static constexpr auto name = "toDateTime64OrDefault"; };
struct NameToDecimal32OrDefault { static constexpr auto name = "toDecimal32OrDefault"; };
struct NameToDecimal64OrDefault { static constexpr auto name = "toDecimal64OrDefault"; };
struct NameToDecimal128OrDefault { static constexpr auto name = "toDecimal128OrDefault"; };
struct NameToDecimal256OrDefault { static constexpr auto name = "toDecimal256OrDefault"; };
struct NameToUUIDOrDefault { static constexpr auto name = "toUUIDOrDefault"; };
using FunctionToUInt8OrDefault = FunctionCastOrDefaultTyped<DataTypeUInt8, NameToUInt8OrDefault>;
using FunctionToUInt16OrDefault = FunctionCastOrDefaultTyped<DataTypeUInt16, NameToUInt16OrDefault>;
using FunctionToUInt32OrDefault = FunctionCastOrDefaultTyped<DataTypeUInt32, NameToUInt32OrDefault>;
using FunctionToUInt64OrDefault = FunctionCastOrDefaultTyped<DataTypeUInt64, NameToUInt64OrDefault>;
using FunctionToUInt256OrDefault = FunctionCastOrDefaultTyped<DataTypeUInt256, NameToUInt256OrDefault>;
using FunctionToInt8OrDefault = FunctionCastOrDefaultTyped<DataTypeInt8, NameToInt8OrDefault>;
using FunctionToInt16OrDefault = FunctionCastOrDefaultTyped<DataTypeInt16, NameToInt16OrDefault>;
using FunctionToInt32OrDefault = FunctionCastOrDefaultTyped<DataTypeInt32, NameToInt32OrDefault>;
using FunctionToInt64OrDefault = FunctionCastOrDefaultTyped<DataTypeInt64, NameToInt64OrDefault>;
using FunctionToInt128OrDefault = FunctionCastOrDefaultTyped<DataTypeInt128, NameToInt128OrDefault>;
using FunctionToInt256OrDefault = FunctionCastOrDefaultTyped<DataTypeInt256, NameToInt256OrDefault>;
using FunctionToFloat32OrDefault = FunctionCastOrDefaultTyped<DataTypeFloat32, NameToFloat32OrDefault>;
using FunctionToFloat64OrDefault = FunctionCastOrDefaultTyped<DataTypeFloat64, NameToFloat64OrDefault>;
using FunctionToDateOrDefault = FunctionCastOrDefaultTyped<DataTypeDate, NameToDateOrDefault>;
using FunctionToDate32OrDefault = FunctionCastOrDefaultTyped<DataTypeDate32, NameToDate32OrDefault>;
using FunctionToDateTimeOrDefault = FunctionCastOrDefaultTyped<DataTypeDateTime, NameToDateTimeOrDefault>;
using FunctionToDateTime64OrDefault = FunctionCastOrDefaultTyped<DataTypeDateTime64, NameToDateTime64OrDefault>;
using FunctionToDecimal32OrDefault = FunctionCastOrDefaultTyped<DataTypeDecimal<Decimal32>, NameToDecimal32OrDefault>;
using FunctionToDecimal64OrDefault = FunctionCastOrDefaultTyped<DataTypeDecimal<Decimal64>, NameToDecimal64OrDefault>;
using FunctionToDecimal128OrDefault = FunctionCastOrDefaultTyped<DataTypeDecimal<Decimal128>, NameToDecimal128OrDefault>;
using FunctionToDecimal256OrDefault = FunctionCastOrDefaultTyped<DataTypeDecimal<Decimal256>, NameToDecimal256OrDefault>;
using FunctionToUUIDOrDefault = FunctionCastOrDefaultTyped<DataTypeUUID, NameToUUIDOrDefault>;
void registerFunctionCastOrDefault(FunctionFactory & factory)
{
factory.registerFunction<FunctionCastOrDefault>();
factory.registerFunction<FunctionToUInt8OrDefault>();
factory.registerFunction<FunctionToUInt16OrDefault>();
factory.registerFunction<FunctionToUInt32OrDefault>();
factory.registerFunction<FunctionToUInt64OrDefault>();
factory.registerFunction<FunctionToUInt256OrDefault>();
factory.registerFunction<FunctionToInt8OrDefault>();
factory.registerFunction<FunctionToInt16OrDefault>();
factory.registerFunction<FunctionToInt32OrDefault>();
factory.registerFunction<FunctionToInt64OrDefault>();
factory.registerFunction<FunctionToInt128OrDefault>();
factory.registerFunction<FunctionToInt256OrDefault>();
factory.registerFunction<FunctionToFloat32OrDefault>();
factory.registerFunction<FunctionToFloat64OrDefault>();
factory.registerFunction<FunctionToDateOrDefault>();
factory.registerFunction<FunctionToDate32OrDefault>();
factory.registerFunction<FunctionToDateTimeOrDefault>();
factory.registerFunction<FunctionToDateTime64OrDefault>();
factory.registerFunction<FunctionToDecimal32OrDefault>();
factory.registerFunction<FunctionToDecimal64OrDefault>();
factory.registerFunction<FunctionToDecimal128OrDefault>();
factory.registerFunction<FunctionToDecimal256OrDefault>();
factory.registerFunction<FunctionToUUIDOrDefault>();
}
}

View File

@ -73,8 +73,8 @@ public:
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
{ {
FunctionArgumentDescriptors args{ FunctionArgumentDescriptors args{
{"haystack", isStringOrFixedString, nullptr, "const String or const FixedString"}, {"haystack", &isStringOrFixedString<IDataType>, nullptr, "const String or const FixedString"},
{"needle", isStringOrFixedString, isColumnConst, "const String or const FixedString"}, {"needle", &isStringOrFixedString<IDataType>, isColumnConst, "const String or const FixedString"},
}; };
validateFunctionArgumentTypes(*this, arguments, args); validateFunctionArgumentTypes(*this, arguments, args);

View File

@ -45,8 +45,8 @@ public:
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
{ {
FunctionArgumentDescriptors args{ FunctionArgumentDescriptors args{
{"haystack", isStringOrFixedString, nullptr, "const String or const FixedString"}, {"haystack", &isStringOrFixedString<IDataType>, nullptr, "const String or const FixedString"},
{"needle", isStringOrFixedString, isColumnConst, "const String or const FixedString"}, {"needle", &isStringOrFixedString<IDataType>, isColumnConst, "const String or const FixedString"},
}; };
validateFunctionArgumentTypes(*this, arguments, args); validateFunctionArgumentTypes(*this, arguments, args);

View File

@ -17,7 +17,7 @@ namespace ErrorCodes
} }
static std::string extractTimeZoneNameFromColumn(const IColumn & column) std::string extractTimeZoneNameFromColumn(const IColumn & column)
{ {
const ColumnConst * time_zone_column = checkAndGetColumnConst<ColumnString>(&column); const ColumnConst * time_zone_column = checkAndGetColumnConst<ColumnString>(&column);

View File

@ -12,6 +12,8 @@ namespace DB
class Block; class Block;
std::string extractTimeZoneNameFromColumn(const IColumn & column);
/// Determine working timezone either from optional argument with time zone name or from time zone in DateTime type of argument. /// Determine working timezone either from optional argument with time zone name or from time zone in DateTime type of argument.
/// Returns empty string if default time zone should be used. /// Returns empty string if default time zone should be used.
std::string extractTimeZoneNameFromFunctionArguments( std::string extractTimeZoneNameFromFunctionArguments(

View File

@ -19,6 +19,7 @@ void registerFunctionChar(FunctionFactory &);
void registerFunctionsComparison(FunctionFactory &); void registerFunctionsComparison(FunctionFactory &);
void registerFunctionsConditional(FunctionFactory &); void registerFunctionsConditional(FunctionFactory &);
void registerFunctionsConversion(FunctionFactory &); void registerFunctionsConversion(FunctionFactory &);
void registerFunctionCastOrDefault(FunctionFactory &);
void registerFunctionsDateTime(FunctionFactory &); void registerFunctionsDateTime(FunctionFactory &);
void registerFunctionsEmbeddedDictionaries(FunctionFactory &); void registerFunctionsEmbeddedDictionaries(FunctionFactory &);
void registerFunctionsExternalDictionaries(FunctionFactory &); void registerFunctionsExternalDictionaries(FunctionFactory &);
@ -84,6 +85,7 @@ void registerFunctions()
registerFunctionsComparison(factory); registerFunctionsComparison(factory);
registerFunctionsConditional(factory); registerFunctionsConditional(factory);
registerFunctionsConversion(factory); registerFunctionsConversion(factory);
registerFunctionCastOrDefault(factory);
registerFunctionsDateTime(factory); registerFunctionsDateTime(factory);
registerFunctionsEmbeddedDictionaries(factory); registerFunctionsEmbeddedDictionaries(factory);
registerFunctionsExternalDictionaries(factory); registerFunctionsExternalDictionaries(factory);

View File

@ -9,7 +9,7 @@ namespace DB
template <CastType cast_type = CastType::nonAccurate> template <CastType cast_type = CastType::nonAccurate>
static ColumnPtr castColumn(const ColumnWithTypeAndName & arg, const DataTypePtr & type) static ColumnPtr castColumn(const ColumnWithTypeAndName & arg, const DataTypePtr & type)
{ {
if (arg.type->equals(*type)) if (arg.type->equals(*type) && cast_type != CastType::accurateOrNull)
return arg.column; return arg.column;
ColumnsWithTypeAndName arguments ColumnsWithTypeAndName arguments

View File

@ -0,0 +1,24 @@
1
2
-1
-2
1
2
-1
-2
1
2
-1
-2
1
2
-1
-2
-1
-2
1
2
-1
-2
61f0c404-5cb3-11e7-907b-a6006ad3dba0
59f0c404-5cb3-11e7-907b-a6006ad3dba0

View File

@ -0,0 +1,30 @@
select toUInt8OrDefault('1', cast(2 as UInt8));
select toUInt8OrDefault('1xx', cast(2 as UInt8));
select toInt8OrDefault('-1', cast(-2 as Int8));
select toInt8OrDefault('-1xx', cast(-2 as Int8));
select toUInt16OrDefault('1', cast(2 as UInt16));
select toUInt16OrDefault('1xx', cast(2 as UInt16));
select toInt16OrDefault('-1', cast(-2 as Int16));
select toInt16OrDefault('-1xx', cast(-2 as Int16));
select toUInt32OrDefault('1', cast(2 as UInt32));
select toUInt32OrDefault('1xx', cast(2 as UInt32));
select toInt32OrDefault('-1', cast(-2 as Int32));
select toInt32OrDefault('-1xx', cast(-2 as Int32));
select toUInt64OrDefault('1', cast(2 as UInt64));
select toUInt64OrDefault('1xx', cast(2 as UInt64));
select toInt64OrDefault('-1', cast(-2 as Int64));
select toInt64OrDefault('-1xx', cast(-2 as Int64));
select toInt128OrDefault('-1', cast(-2 as Int128));
select toInt128OrDefault('-1xx', cast(-2 as Int128));
select toUInt256OrDefault('1', cast(2 as UInt256));
select toUInt256OrDefault('1xx', cast(2 as UInt256));
select toInt256OrDefault('-1', cast(-2 as Int256));
select toInt256OrDefault('-1xx', cast(-2 as Int256));
SELECT toUUIDOrDefault('61f0c404-5cb3-11e7-907b-a6006ad3dba0', cast('59f0c404-5cb3-11e7-907b-a6006ad3dba0' as UUID));
SELECT toUUIDOrDefault('-----61f0c404-5cb3-11e7-907b-a6006ad3dba0', cast('59f0c404-5cb3-11e7-907b-a6006ad3dba0' as UUID));

View File

@ -0,0 +1,32 @@
0 5
5
0 5
0 5
5
0 5
0 5
5
0 5
0 5
5
0 5
5
0 5
5
0 5
0 2
1
0 2
\0\0 12
0 5
0 5
0 5
0 5
0 5
0 5
0 5
0 5
0 5
0 5
127 127
0 5

View File

@ -0,0 +1,36 @@
SELECT accurateCastOrDefault(-1, 'UInt8'), accurateCastOrDefault(5, 'UInt8');
SELECT accurateCastOrDefault(5, 'UInt8');
SELECT accurateCastOrDefault(257, 'UInt8'), accurateCastOrDefault(257, 'UInt8', 5);
SELECT accurateCastOrDefault(-1, 'UInt16'), accurateCastOrDefault(-1, 'UInt16', toUInt16(5));
SELECT accurateCastOrDefault(5, 'UInt16');
SELECT accurateCastOrDefault(65536, 'UInt16'), accurateCastOrDefault(65536, 'UInt16', toUInt16(5));
SELECT accurateCastOrDefault(-1, 'UInt32'), accurateCastOrDefault(-1, 'UInt32', toUInt32(5));
SELECT accurateCastOrDefault(5, 'UInt32');
SELECT accurateCastOrDefault(4294967296, 'UInt32'), accurateCastOrDefault(4294967296, 'UInt32', toUInt32(5));
SELECT accurateCastOrDefault(-1, 'UInt64'), accurateCastOrDefault(-1, 'UInt64', toUInt64(5));
SELECT accurateCastOrDefault(5, 'UInt64');
SELECT accurateCastOrDefault(-1, 'UInt256'), accurateCastOrDefault(-1, 'UInt256', toUInt256(5));
SELECT accurateCastOrDefault(5, 'UInt256');
SELECT accurateCastOrDefault(-129, 'Int8'), accurateCastOrDefault(-129, 'Int8', toInt8(5));
SELECT accurateCastOrDefault(5, 'Int8');
SELECT accurateCastOrDefault(128, 'Int8'), accurateCastOrDefault(128, 'Int8', toInt8(5));
SELECT accurateCastOrDefault(10, 'Decimal32(9)'), accurateCastOrDefault(10, 'Decimal32(9)', toDecimal32(2, 9));
SELECT accurateCastOrDefault(1, 'Decimal32(9)');
SELECT accurateCastOrDefault(-10, 'Decimal32(9)'), accurateCastOrDefault(-10, 'Decimal32(9)', toDecimal32(2, 9));
SELECT accurateCastOrDefault('123', 'FixedString(2)'), accurateCastOrDefault('123', 'FixedString(2)', cast('12', 'FixedString(2)'));
SELECT accurateCastOrDefault(inf, 'Int64'), accurateCastOrDefault(inf, 'Int64', toInt64(5));
SELECT accurateCastOrDefault(inf, 'Int128'), accurateCastOrDefault(inf, 'Int128', toInt128(5));
SELECT accurateCastOrDefault(inf, 'Int256'), accurateCastOrDefault(inf, 'Int256', toInt256(5));
SELECT accurateCastOrDefault(nan, 'Int64'), accurateCastOrDefault(nan, 'Int64', toInt64(5));
SELECT accurateCastOrDefault(nan, 'Int128'), accurateCastOrDefault(nan, 'Int128', toInt128(5));
SELECT accurateCastOrDefault(nan, 'Int256'), accurateCastOrDefault(nan, 'Int256', toInt256(5));
SELECT accurateCastOrDefault(inf, 'UInt64'), accurateCastOrDefault(inf, 'UInt64', toUInt64(5));
SELECT accurateCastOrDefault(inf, 'UInt256'), accurateCastOrDefault(inf, 'UInt256', toUInt256(5));
SELECT accurateCastOrDefault(nan, 'UInt64'), accurateCastOrDefault(nan, 'UInt64', toUInt64(5));
SELECT accurateCastOrDefault(nan, 'UInt256'), accurateCastOrDefault(nan, 'UInt256', toUInt256(5));
SELECT accurateCastOrDefault(number + 127, 'Int8') AS x, accurateCastOrDefault(number + 127, 'Int8', toInt8(5)) AS x_with_default FROM numbers (2) ORDER BY number;