Merge pull request #40217 from zvonand/zvonand-minmax

Fix conversion Date32 / DateTime64 / Date to narrow types
This commit is contained in:
Robert Schulze 2022-08-28 09:42:39 +02:00 committed by GitHub
commit df934d8762
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
14 changed files with 296 additions and 111 deletions

View File

@ -266,8 +266,14 @@ Result:
└────────────────┘
```
:::note
The return type `toStartOf*` functions described below is `Date` or `DateTime`. Though these functions can take `DateTime64` as an argument, passing them a `DateTime64` that is out of the normal range (years 1900 - 2299) will give an incorrect result.
:::Attention
The return type of `toStartOf*`, `toLastDayOfMonth`, `toMonday` functions described below is `Date` or `DateTime`.
Though these functions can take values of the extended types `Date32` and `DateTime64` as an argument, passing them a time outside the normal range (year 1970 to 2149 for `Date` / 2106 for `DateTime`) will produce wrong results.
In case argument is out of normal range:
* If the argument is smaller than 1970, the result will be calculated from the argument `1970-01-01 (00:00:00)` instead.
* If the return type is `DateTime` and the argument is larger than `2106-02-07 08:28:15`, the result will be calculated from the argument `2106-02-07 08:28:15` instead.
* If the return type is `Date` and the argument is larger than `2149-06-06`, the result will be calculated from the argument `2149-06-06` instead.
* If `toLastDayOfMonth` is called with an argument greater then `2149-05-31`, the result will be calculated from the argument `2149-05-31` instead.
:::
## toStartOfYear
@ -291,20 +297,23 @@ Returns the date.
Rounds down a date or date with time to the first day of the month.
Returns the date.
:::note
The behavior of parsing incorrect dates is implementation specific. ClickHouse may return zero date, throw an exception or do “natural” overflow.
:::
## toLastDayOfMonth
Rounds up a date or date with time to the last day of the month.
Returns the date.
## toMonday
Rounds down a date or date with time to the nearest Monday.
As a special case, date arguments `1970-01-01`, `1970-01-02`, `1970-01-03` and `1970-01-04` return date `1970-01-01`.
Returns the date.
## toStartOfWeek(t\[,mode\])
Rounds down a date or date with time to the nearest Sunday or Monday by mode.
Returns the date.
The mode argument works exactly like the mode argument to toWeek(). For the single-argument syntax, a mode value of 0 is used.
As a special case, date arguments `1970-01-01`, `1970-01-02`, `1970-01-03` and `1970-01-04` (and `1970-01-05` if `mode` is `1`) return date `1970-01-01`.
The `mode` argument works exactly like the mode argument to toWeek(). For the single-argument syntax, a mode value of 0 is used.
## toStartOfDay
@ -1039,7 +1048,7 @@ Example:
SELECT timeSlots(toDateTime('2012-01-01 12:20:00'), toUInt32(600));
SELECT timeSlots(toDateTime('1980-12-12 21:01:02', 'UTC'), toUInt32(600), 299);
SELECT timeSlots(toDateTime64('1980-12-12 21:01:02.1234', 4, 'UTC'), toDecimal64(600.1, 1), toDecimal64(299, 0));
```
```
``` text
┌─timeSlots(toDateTime('2012-01-01 12:20:00'), toUInt32(600))─┐
│ ['2012-01-01 12:00:00','2012-01-01 12:30:00'] │

View File

@ -6,7 +6,7 @@ sidebar_label: Date
# Date {#data-type-date}
Дата. Хранится в двух байтах в виде (беззнакового) числа дней, прошедших от 1970-01-01. Позволяет хранить значения от чуть больше, чем начала unix-эпохи до верхнего порога, определяющегося константой на этапе компиляции (сейчас - до 2106 года, последний полностью поддерживаемый год - 2105).
Дата. Хранится в двух байтах в виде (беззнакового) числа дней, прошедших от 1970-01-01. Позволяет хранить значения от чуть больше, чем начала unix-эпохи до верхнего порога, определяющегося константой на этапе компиляции (сейчас - до 2149 года, последний полностью поддерживаемый год - 2148).
Диапазон значений: \[1970-01-01, 2149-06-06\].

View File

@ -267,10 +267,25 @@ SELECT toUnixTimestamp('2017-11-05 08:07:47', 'Asia/Tokyo') AS unix_timestamp;
└────────────────┘
```
:::note "Attention"
`Date` или `DateTime` это возвращаемый тип функций `toStartOf*`, который описан ниже. Несмотря на то, что эти функции могут принимать `DateTime64` в качестве аргумента, если переданное значение типа `DateTime64` выходит за пределы нормального диапазона (с 1900 по 2299 год), то это даст неверный результат.
:::Attention
Тип возвращаемого описанными далее функциями `toStartOf*`, `toMonday` значения - `Date` или `DateTime`.
Хотя эти функции могут принимать значения типа `Date32` или `DateTime64` в качестве аргумента, при обработке аргумента вне нормального диапазона значений (`1970` - `2148` для `Date` и `1970-01-01 00:00:00`-`2106-02-07 08:28:15` для `DateTime`) будет получен некорректный результат.
Возвращаемые значения для значений вне нормального диапазона:
* `1970-01-01 (00:00:00)` будет возвращён для моментов времени до 1970 года,
* `2106-02-07 08:28:15` будет взят в качестве аргумента, если полученный аргумент превосходит данное значение и возвращаемый тип - `DateTime`,
* `2149-06-06` будет взят в качестве аргумента, если полученный аргумент превосходит данное значение и возвращаемый тип - `Date`,
* `2149-05-31` будет результатом функции `toLastDayOfMonth` при обработке аргумента больше `2149-05-31`.
:::
:::Attention
Тип возвращаемого описанными далее функциями `toStartOf*`, `toLastDayOfMonth`, `toMonday` значения - `Date` или `DateTime`.
Хотя эти функции могут принимать значения типа `Date32` или `DateTime64` в качестве аргумента, при обработке аргумента вне нормального диапазона значений (`1970` - `2148` для `Date` и `1970-01-01 00:00:00`-`2106-02-07 08:28:15` для `DateTime`) будет получен некорректный результат.
Возвращаемые значения для значений вне нормального диапазона:
* `1970-01-01 (00:00:00)` будет возвращён для моментов времени до 1970 года,
* `2106-02-07 08:28:15` будет взят в качестве аргумента, если полученный аргумент превосходит данное значение и возвращаемый тип - `DateTime`,
* `2149-06-06` будет взят в качестве аргумента, если полученный аргумент превосходит данное значение и возвращаемый тип - `Date`.
:::
*
## toStartOfYear {#tostartofyear}
Округляет дату или дату-с-временем вниз до первого дня года.
@ -304,20 +319,23 @@ SELECT toStartOfISOYear(toDate('2017-01-01')) AS ISOYear20170101;
Округляет дату или дату-с-временем вниз до первого дня месяца.
Возвращается дата.
:::note "Attention"
Возвращаемое значение для некорректных дат зависит от реализации. ClickHouse может вернуть нулевую дату, выбросить исключение, или выполнить «естественное» перетекание дат между месяцами.
:::
## toLastDayOfMonth
Округляет дату или дату-с-временем до последнего числа месяца.
Возвращается дата.
## toMonday {#tomonday}
Округляет дату или дату-с-временем вниз до ближайшего понедельника.
Частный случай: для дат `1970-01-01`, `1970-01-02`, `1970-01-03` и `1970-01-04` результатом будет `1970-01-01`.
Возвращается дата.
## toStartOfWeek(t[,mode]) {#tostartofweek}
Округляет дату или дату со временем до ближайшего воскресенья или понедельника в соответствии с mode.
Возвращается дата.
Аргумент mode работает точно так же, как аргумент mode [toWeek()](#toweek). Если аргумент mode опущен, то используется режим 0.
Частный случай: для дат `1970-01-01`, `1970-01-02`, `1970-01-03` и `1970-01-04``1970-01-05`, если `mode` равен `1`) результатом будет `1970-01-01`.
Аргумент `mode` работает точно так же, как аргумент mode [toWeek()](#toweek). Если аргумент mode опущен, то используется режим 0.
## toStartOfDay {#tostartofday}
@ -958,7 +976,7 @@ SELECT now('Europe/Moscow');
## timeSlots(StartTime, Duration,\[, Size\]) {#timeslotsstarttime-duration-size}
Для интервала, начинающегося в `StartTime` и длящегося `Duration` секунд, возвращает массив моментов времени, кратных `Size`. Параметр `Size` указывать необязательно, по умолчанию он равен 1800 секундам (30 минутам) - необязательный параметр.
Данная функция может использоваться, например, для анализа количества просмотров страницы за соответствующую сессию.
Данная функция может использоваться, например, для анализа количества просмотров страницы за соответствующую сессию.
Аргумент `StartTime` может иметь тип `DateTime` или `DateTime64`. В случае, если используется `DateTime`, аргументы `Duration` и `Size` должны иметь тип `UInt32`; Для DateTime64 они должны быть типа `Decimal64`.
Возвращает массив DateTime/DateTime64 (тип будет совпадать с типом параметра StartTime). Для DateTime64 масштаб(scale) возвращаемой величины может отличаться от масштаба фргумента StartTime --- результат будет иметь наибольший масштаб среди всех данных аргументов.

View File

@ -3,7 +3,7 @@ slug: /zh/sql-reference/data-types/date
---
# 日期 {#date}
日期类型,用两个字节存储,表示从 1970-01-01 (无符号) 到当前的日期值。允许存储从 Unix 纪元开始到编译阶段定义的上限阈值常量目前上限是2106年但最终完全支持的年份为2105。最小值输出为1970-01-01。
日期类型,用两个字节存储,表示从 1970-01-01 (无符号) 到当前的日期值。允许存储从 Unix 纪元开始到编译阶段定义的上限阈值常量目前上限是2149年但最终完全支持的年份为2148。最小值输出为1970-01-01。
值的范围: \[1970-01-01, 2149-06-06\]。

View File

@ -62,7 +62,10 @@ struct ToStartOfWeekImpl
static inline UInt16 execute(Int64 t, UInt8 week_mode, const DateLUTImpl & time_zone)
{
return time_zone.toFirstDayNumOfWeek(time_zone.toDayNum(t), week_mode);
if (t < 0)
return 0;
return time_zone.toFirstDayNumOfWeek(DayNum(std::min(Int32(time_zone.toDayNum(t)), Int32(DATE_LUT_MAX_DAY_NUM))), week_mode);
}
static inline UInt16 execute(UInt32 t, UInt8 week_mode, const DateLUTImpl & time_zone)
{
@ -70,7 +73,10 @@ struct ToStartOfWeekImpl
}
static inline UInt16 execute(Int32 d, UInt8 week_mode, const DateLUTImpl & time_zone)
{
return time_zone.toFirstDayNumOfWeek(ExtendedDayNum(d), week_mode);
if (d < 0)
return 0;
return time_zone.toFirstDayNumOfWeek(DayNum(std::min(d, Int32(DATE_LUT_MAX_DAY_NUM))), week_mode);
}
static inline UInt16 execute(UInt16 d, UInt8 week_mode, const DateLUTImpl & time_zone)
{

View File

@ -61,15 +61,15 @@ struct ToDateImpl
static inline UInt16 execute(Int64 t, const DateLUTImpl & time_zone)
{
return UInt16(time_zone.toDayNum(t));
return t < 0 ? 0 : std::min(Int32(time_zone.toDayNum(t)), Int32(DATE_LUT_MAX_DAY_NUM));
}
static inline UInt16 execute(UInt32 t, const DateLUTImpl & time_zone)
{
return UInt16(time_zone.toDayNum(t));
return time_zone.toDayNum(t);
}
static inline UInt16 execute(Int32, const DateLUTImpl &)
static inline UInt16 execute(Int32 t, const DateLUTImpl &)
{
return dateIsNotSupported(name);
return t < 0 ? 0 : std::min(t, Int32(DATE_LUT_MAX_DAY_NUM));
}
static inline UInt16 execute(UInt16 d, const DateLUTImpl &)
{
@ -111,7 +111,10 @@ struct ToStartOfDayImpl
//TODO: right now it is hardcoded to produce DateTime only, needs fixing later. See date_and_time_type_details::ResultDataTypeMap for deduction of result type example.
static inline UInt32 execute(const DecimalUtils::DecimalComponents<DateTime64> & t, const DateLUTImpl & time_zone)
{
return time_zone.toDate(static_cast<time_t>(t.whole));
if (t.whole < 0 || (t.whole >= 0 && t.fractional < 0))
return 0;
return time_zone.toDate(std::min<Int64>(t.whole, Int64(0xffffffff)));
}
static inline UInt32 execute(UInt32 t, const DateLUTImpl & time_zone)
{
@ -119,11 +122,19 @@ struct ToStartOfDayImpl
}
static inline UInt32 execute(Int32 d, const DateLUTImpl & time_zone)
{
return time_zone.toDate(ExtendedDayNum(d));
if (d < 0)
return 0;
auto date_time = time_zone.fromDayNum(ExtendedDayNum(d));
if (date_time <= 0xffffffff)
return date_time;
else
return time_zone.toDate(0xffffffff);
}
static inline UInt32 execute(UInt16 d, const DateLUTImpl & time_zone)
{
return time_zone.toDate(DayNum(d));
auto date_time = time_zone.fromDayNum(ExtendedDayNum(d));
return date_time < 0xffffffff ? date_time : time_zone.toDate(0xffffffff);
}
using FactorTransform = ZeroTransform;
@ -135,17 +146,16 @@ struct ToMondayImpl
static inline UInt16 execute(Int64 t, const DateLUTImpl & time_zone)
{
//return time_zone.toFirstDayNumOfWeek(time_zone.toDayNum(t));
return time_zone.toFirstDayNumOfWeek(t);
return t < 0 ? 0 : time_zone.toFirstDayNumOfWeek(ExtendedDayNum(
std::min(Int32(time_zone.toDayNum(t)), Int32(DATE_LUT_MAX_DAY_NUM))));
}
static inline UInt16 execute(UInt32 t, const DateLUTImpl & time_zone)
{
//return time_zone.toFirstDayNumOfWeek(time_zone.toDayNum(t));
return time_zone.toFirstDayNumOfWeek(t);
}
static inline UInt16 execute(Int32 d, const DateLUTImpl & time_zone)
{
return time_zone.toFirstDayNumOfWeek(ExtendedDayNum(d));
return d < 0 ? 0 : time_zone.toFirstDayNumOfWeek(ExtendedDayNum(std::min(d, Int32(DATE_LUT_MAX_DAY_NUM))));
}
static inline UInt16 execute(UInt16 d, const DateLUTImpl & time_zone)
{
@ -161,15 +171,15 @@ struct ToStartOfMonthImpl
static inline UInt16 execute(Int64 t, const DateLUTImpl & time_zone)
{
return time_zone.toFirstDayNumOfMonth(time_zone.toDayNum(t));
return t < 0 ? 0 : time_zone.toFirstDayNumOfMonth(ExtendedDayNum(std::min(Int32(time_zone.toDayNum(t)), Int32(DATE_LUT_MAX_DAY_NUM))));
}
static inline UInt16 execute(UInt32 t, const DateLUTImpl & time_zone)
{
return time_zone.toFirstDayNumOfMonth(time_zone.toDayNum(t));
return time_zone.toFirstDayNumOfMonth(ExtendedDayNum(time_zone.toDayNum(t)));
}
static inline UInt16 execute(Int32 d, const DateLUTImpl & time_zone)
{
return time_zone.toFirstDayNumOfMonth(ExtendedDayNum(d));
return d < 0 ? 0 : time_zone.toFirstDayNumOfMonth(ExtendedDayNum(std::min(d, Int32(DATE_LUT_MAX_DAY_NUM))));
}
static inline UInt16 execute(UInt16 d, const DateLUTImpl & time_zone)
{
@ -185,7 +195,11 @@ struct ToLastDayOfMonthImpl
static inline UInt16 execute(Int64 t, const DateLUTImpl & time_zone)
{
return time_zone.toLastDayNumOfMonth(time_zone.toDayNum(t));
if (t < 0)
return 0;
/// 0xFFF9 is Int value for 2149-05-31 -- the last day where we can actually find LastDayOfMonth. This will also be the return value.
return time_zone.toLastDayNumOfMonth(ExtendedDayNum(std::min(Int32(time_zone.toDayNum(t)), Int32(0xFFF9))));
}
static inline UInt16 execute(UInt32 t, const DateLUTImpl & time_zone)
{
@ -193,11 +207,16 @@ struct ToLastDayOfMonthImpl
}
static inline UInt16 execute(Int32 d, const DateLUTImpl & time_zone)
{
return time_zone.toLastDayNumOfMonth(ExtendedDayNum(d));
if (d < 0)
return 0;
/// 0xFFF9 is Int value for 2149-05-31 -- the last day where we can actually find LastDayOfMonth. This will also be the return value.
return time_zone.toLastDayNumOfMonth(ExtendedDayNum(std::min(d, Int32(0xFFF9))));
}
static inline UInt16 execute(UInt16 d, const DateLUTImpl & time_zone)
{
return time_zone.toLastDayNumOfMonth(DayNum(d));
/// 0xFFF9 is Int value for 2149-05-31 -- the last day where we can actually find LastDayOfMonth. This will also be the return value.
return time_zone.toLastDayNumOfMonth(DayNum(std::min(d, UInt16(0xFFF9))));
}
using FactorTransform = ZeroTransform;
@ -209,7 +228,7 @@ struct ToStartOfQuarterImpl
static inline UInt16 execute(Int64 t, const DateLUTImpl & time_zone)
{
return time_zone.toFirstDayNumOfQuarter(time_zone.toDayNum(t));
return t < 0 ? 0 : time_zone.toFirstDayNumOfQuarter(ExtendedDayNum(std::min<Int64>(Int64(time_zone.toDayNum(t)), Int64(DATE_LUT_MAX_DAY_NUM))));
}
static inline UInt16 execute(UInt32 t, const DateLUTImpl & time_zone)
{
@ -217,7 +236,7 @@ struct ToStartOfQuarterImpl
}
static inline UInt16 execute(Int32 d, const DateLUTImpl & time_zone)
{
return time_zone.toFirstDayNumOfQuarter(ExtendedDayNum(d));
return d < 0 ? 0 : time_zone.toFirstDayNumOfQuarter(ExtendedDayNum(std::min<Int32>(d, Int32(DATE_LUT_MAX_DAY_NUM))));
}
static inline UInt16 execute(UInt16 d, const DateLUTImpl & time_zone)
{
@ -233,7 +252,7 @@ struct ToStartOfYearImpl
static inline UInt16 execute(Int64 t, const DateLUTImpl & time_zone)
{
return time_zone.toFirstDayNumOfYear(time_zone.toDayNum(t));
return t < 0 ? 0 : time_zone.toFirstDayNumOfYear(ExtendedDayNum(std::min(Int32(time_zone.toDayNum(t)), Int32(DATE_LUT_MAX_DAY_NUM))));
}
static inline UInt16 execute(UInt32 t, const DateLUTImpl & time_zone)
{
@ -241,7 +260,7 @@ struct ToStartOfYearImpl
}
static inline UInt16 execute(Int32 d, const DateLUTImpl & time_zone)
{
return time_zone.toFirstDayNumOfYear(ExtendedDayNum(d));
return d < 0 ? 0 : time_zone.toFirstDayNumOfYear(ExtendedDayNum(std::min(d, Int32(DATE_LUT_MAX_DAY_NUM))));
}
static inline UInt16 execute(UInt16 d, const DateLUTImpl & time_zone)
{
@ -283,7 +302,10 @@ struct ToStartOfMinuteImpl
static inline UInt32 execute(const DecimalUtils::DecimalComponents<DateTime64> & t, const DateLUTImpl & time_zone)
{
return time_zone.toStartOfMinute(t.whole);
if (t.whole < 0 || (t.whole >= 0 && t.fractional < 0))
return 0;
return time_zone.toStartOfMinute(std::min<Int64>(t.whole, Int64(0xffffffff)));
}
static inline UInt32 execute(UInt32 t, const DateLUTImpl & time_zone)
{
@ -574,7 +596,10 @@ struct ToStartOfHourImpl
static inline UInt32 execute(const DecimalUtils::DecimalComponents<DateTime64> & t, const DateLUTImpl & time_zone)
{
return time_zone.toStartOfHour(t.whole);
if (t.whole < 0 || (t.whole >= 0 && t.fractional < 0))
return 0;
return time_zone.toStartOfHour(std::min<Int64>(t.whole, Int64(0xffffffff)));
}
static inline UInt32 execute(UInt32 t, const DateLUTImpl & time_zone)

View File

@ -41,23 +41,20 @@ public:
if (!isDate(arguments[0].type) && !isDate32(arguments[0].type) && !isDateTime(arguments[0].type) && !isDateTime64(arguments[0].type))
throw Exception(
"Illegal type " + arguments[0].type->getName() + " of argument of function " + getName()
+ ". Should be a date or a date with time",
+ ". Must be Date, Date32, DateTime or DateTime64.",
ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
}
else if (arguments.size() == 2)
{
if (!isDate(arguments[0].type) && !isDate32(arguments[0].type) && !isDateTime(arguments[0].type) && !isDateTime64(arguments[0].type))
throw Exception(
"Illegal type " + arguments[0].type->getName() + " of argument of function " + getName()
+ ". Should be a date or a date with time",
"Illegal type " + arguments[0].type->getName() + " of 1st argument of function " + getName()
+ ". Must be Date, Date32, DateTime or DateTime64.",
ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
if (!isUInt8(arguments[1].type))
throw Exception(
"Function " + getName()
+ " supports 1 or 2 or 3 arguments. The 1st argument "
"must be of type Date or DateTime. The 2nd argument (optional) must be "
"a constant UInt8 with week mode. The 3rd argument (optional) must be "
"a constant string with timezone name",
"Illegal type of 2nd (optional) argument of function " + getName()
+ ". Must be constant UInt8 (week mode).",
ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
}
else if (arguments.size() == 3)
@ -65,33 +62,28 @@ public:
if (!isDate(arguments[0].type) && !isDate32(arguments[0].type) && !isDateTime(arguments[0].type) && !isDateTime64(arguments[0].type))
throw Exception(
"Illegal type " + arguments[0].type->getName() + " of argument of function " + getName()
+ ". Should be a date or a date with time",
+ ". Must be Date, Date32, DateTime or DateTime64",
ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
if (!isUInt8(arguments[1].type))
throw Exception(
"Function " + getName()
+ " supports 1 or 2 or 3 arguments. The 1st argument "
"must be of type Date or DateTime. The 2nd argument (optional) must be "
"a constant UInt8 with week mode. The 3rd argument (optional) must be "
"a constant string with timezone name",
"Illegal type of 2nd (optional) argument of function " + getName()
+ ". Must be constant UInt8 (week mode).",
ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
if (!isString(arguments[2].type))
throw Exception(
"Function " + getName()
+ " supports 1 or 2 or 3 arguments. The 1st argument "
"must be of type Date or DateTime. The 2nd argument (optional) must be "
"a constant UInt8 with week mode. The 3rd argument (optional) must be "
"a constant string with timezone name",
"Illegal type of 3rd (optional) argument of function " + getName()
+ ". Must be constant string (timezone name).",
ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
if (isDate(arguments[0].type) && std::is_same_v<ToDataType, DataTypeDate>)
if ((isDate(arguments[0].type) || isDate32(arguments[0].type))
&& (std::is_same_v<ToDataType, DataTypeDate> || std::is_same_v<ToDataType, DataTypeDate32>))
throw Exception(
"The timezone argument of function " + getName() + " is allowed only when the 1st argument has the type DateTime",
"The timezone argument of function " + getName() + " is allowed only when the 1st argument is DateTime or DateTime64.",
ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT);
}
else
throw Exception(
"Number of arguments for function " + getName() + " doesn't match: passed " + toString(arguments.size())
+ ", should be 1 or 2 or 3",
+ ", expected 1, 2 or 3.",
ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH);
return std::make_shared<ToDataType>();

View File

@ -301,6 +301,11 @@ struct ConvertImpl
}
};
/** Conversion of Date32 to Date: check bounds.
*/
template <typename Name> struct ConvertImpl<DataTypeDate32, DataTypeDate, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeDate32, DataTypeDate, ToDateImpl> {};
/** Conversion of DateTime to Date: throw off time component.
*/
template <typename Name> struct ConvertImpl<DataTypeDateTime, DataTypeDate, Name, ConvertDefaultBehaviorTag>
@ -319,12 +324,17 @@ struct ToDateTimeImpl
static inline UInt32 execute(UInt16 d, const DateLUTImpl & time_zone)
{
return time_zone.fromDayNum(DayNum(d));
auto date_time = time_zone.fromDayNum(ExtendedDayNum(d));
return date_time <= 0xffffffff ? UInt32(date_time) : UInt32(0xffffffff);
}
static inline Int64 execute(Int32 d, const DateLUTImpl & time_zone)
static inline UInt32 execute(Int32 d, const DateLUTImpl & time_zone)
{
return time_zone.fromDayNum(ExtendedDayNum(d));
if (d < 0)
return 0;
auto date_time = time_zone.fromDayNum(ExtendedDayNum(d));
return date_time <= 0xffffffff ? date_time : 0xffffffff;
}
static inline UInt32 execute(UInt32 dt, const DateLUTImpl & /*time_zone*/)
@ -332,10 +342,21 @@ struct ToDateTimeImpl
return dt;
}
// TODO: return UInt32 ???
static inline Int64 execute(Int64 dt64, const DateLUTImpl & /*time_zone*/)
static inline UInt32 execute(Int64 d, const DateLUTImpl & time_zone)
{
return dt64;
if (d < 0)
return 0;
auto date_time = time_zone.toDate(d);
return date_time <= 0xffffffff ? date_time : 0xffffffff;
}
static inline UInt32 execute(const DecimalUtils::DecimalComponents<DateTime64> & t, const DateLUTImpl & /*time_zone*/)
{
if (t.whole < 0 || (t.whole >= 0 && t.fractional < 0))
return 0;
return std::min<Int64>(t.whole, Int64(0xFFFFFFFF));
}
};
@ -355,9 +376,12 @@ struct ToDateTransform32Or64
static inline NO_SANITIZE_UNDEFINED ToType execute(const FromType & from, const DateLUTImpl & time_zone)
{
// since converting to Date, no need in values outside of default LUT range.
if (from < 0)
return 0;
return (from < DATE_LUT_MAX_DAY_NUM)
? from
: time_zone.toDayNum(std::min(time_t(from), time_t(0xFFFFFFFF)));
: std::min<Int32>(Int32(time_zone.toDayNum(from)), Int32(DATE_LUT_MAX_DAY_NUM));
}
};
@ -372,9 +396,14 @@ struct ToDateTransform32Or64Signed
/// The function should be monotonic (better for query optimizations), so we saturate instead of overflow.
if (from < 0)
return 0;
auto day_num = time_zone.toDayNum(ExtendedDayNum(from));
return day_num < DATE_LUT_MAX_DAY_NUM ? day_num : DATE_LUT_MAX_DAY_NUM;
return (from < DATE_LUT_MAX_DAY_NUM)
? from
: time_zone.toDayNum(std::min(time_t(from), time_t(0xFFFFFFFF)));
: std::min<Int32>(Int32(time_zone.toDayNum(from)), Int32(0xFFFFFFFF));
}
};
@ -405,7 +434,7 @@ struct ToDate32Transform32Or64
{
return (from < DATE_LUT_MAX_EXTEND_DAY_NUM)
? from
: time_zone.toDayNum(std::min(time_t(from), time_t(0xFFFFFFFF)));
: std::min<Int32>(Int32(time_zone.toDayNum(from)), Int32(DATE_LUT_MAX_EXTEND_DAY_NUM));
}
};
@ -421,7 +450,7 @@ struct ToDate32Transform32Or64Signed
return daynum_min_offset;
return (from < DATE_LUT_MAX_EXTEND_DAY_NUM)
? from
: time_zone.toDayNum(std::min(time_t(from), time_t(0xFFFFFFFF)));
: time_zone.toDayNum(std::min<Int64>(Int64(from), Int64(0xFFFFFFFF)));
}
};
@ -447,35 +476,49 @@ struct ToDate32Transform8Or16Signed
*/
template <typename Name> struct ConvertImpl<DataTypeUInt32, DataTypeDate, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeUInt32, DataTypeDate, ToDateTransform32Or64<UInt32, UInt16>> {};
template <typename Name> struct ConvertImpl<DataTypeUInt64, DataTypeDate, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeUInt64, DataTypeDate, ToDateTransform32Or64<UInt64, UInt16>> {};
template <typename Name> struct ConvertImpl<DataTypeInt8, DataTypeDate, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeInt8, DataTypeDate, ToDateTransform8Or16Signed<Int8, UInt16>> {};
template <typename Name> struct ConvertImpl<DataTypeInt16, DataTypeDate, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeInt16, DataTypeDate, ToDateTransform8Or16Signed<Int16, UInt16>> {};
template <typename Name> struct ConvertImpl<DataTypeInt32, DataTypeDate, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeInt32, DataTypeDate, ToDateTransform32Or64Signed<Int32, UInt16>> {};
template <typename Name> struct ConvertImpl<DataTypeInt64, DataTypeDate, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeInt64, DataTypeDate, ToDateTransform32Or64Signed<Int64, UInt16>> {};
template <typename Name> struct ConvertImpl<DataTypeFloat32, DataTypeDate, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeFloat32, DataTypeDate, ToDateTransform32Or64Signed<Float32, UInt16>> {};
template <typename Name> struct ConvertImpl<DataTypeFloat64, DataTypeDate, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeFloat64, DataTypeDate, ToDateTransform32Or64Signed<Float64, UInt16>> {};
template <typename Name> struct ConvertImpl<DataTypeUInt32, DataTypeDate32, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeUInt32, DataTypeDate32, ToDate32Transform32Or64<UInt32, Int32>> {};
template <typename Name> struct ConvertImpl<DataTypeUInt64, DataTypeDate32, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeUInt64, DataTypeDate32, ToDate32Transform32Or64<UInt64, Int32>> {};
template <typename Name> struct ConvertImpl<DataTypeInt8, DataTypeDate32, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeInt8, DataTypeDate32, ToDate32Transform8Or16Signed<Int8, Int32>> {};
template <typename Name> struct ConvertImpl<DataTypeInt16, DataTypeDate32, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeInt16, DataTypeDate32, ToDate32Transform8Or16Signed<Int16, Int32>> {};
template <typename Name> struct ConvertImpl<DataTypeInt32, DataTypeDate32, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeInt32, DataTypeDate32, ToDate32Transform32Or64Signed<Int32, Int32>> {};
template <typename Name> struct ConvertImpl<DataTypeInt64, DataTypeDate32, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeInt64, DataTypeDate32, ToDate32Transform32Or64Signed<Int64, Int32>> {};
template <typename Name> struct ConvertImpl<DataTypeFloat32, DataTypeDate32, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeFloat32, DataTypeDate32, ToDate32Transform32Or64Signed<Float32, Int32>> {};
template <typename Name> struct ConvertImpl<DataTypeFloat64, DataTypeDate32, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeFloat64, DataTypeDate32, ToDate32Transform32Or64Signed<Float64, Int32>> {};
@ -487,7 +530,7 @@ struct ToDateTimeTransform64
static inline NO_SANITIZE_UNDEFINED ToType execute(const FromType & from, const DateLUTImpl &)
{
return std::min(time_t(from), time_t(0xFFFFFFFF));
return std::min<Int64>(Int64(from), Int64(0xFFFFFFFF));
}
};
@ -509,11 +552,12 @@ struct ToDateTimeTransform64Signed
{
static constexpr auto name = "toDateTime";
static inline NO_SANITIZE_UNDEFINED ToType execute(const FromType & from, const DateLUTImpl &)
static inline NO_SANITIZE_UNDEFINED ToType execute(const FromType & from, const DateLUTImpl & /* time_zone */)
{
if (from < 0)
return 0;
return std::min(time_t(from), time_t(0xFFFFFFFF));
return std::min<Int64>(Int64(from), Int64(0xFFFFFFFF));
}
};
@ -634,8 +678,6 @@ struct FromDateTime64Transform
}
};
/** Conversion of DateTime64 to Date or DateTime: discards fractional part.
*/
template <typename Name> struct ConvertImpl<DataTypeDateTime64, DataTypeDate, Name, ConvertDefaultBehaviorTag>
: DateTimeTransformImpl<DataTypeDateTime64, DataTypeDate, TransformDateTime64<ToDateImpl>> {};
template <typename Name> struct ConvertImpl<DataTypeDateTime64, DataTypeDateTime, Name, ConvertDefaultBehaviorTag>
@ -659,7 +701,7 @@ struct ToDateTime64Transform
inline DateTime64::NativeType execute(Int32 d, const DateLUTImpl & time_zone) const
{
const auto dt = ToDateTimeImpl::execute(d, time_zone);
const auto dt = time_zone.fromDayNum(ExtendedDayNum(d));
return DecimalUtils::decimalFromComponentsWithMultiplier<DateTime64>(dt, 0, scale_multiplier);
}
@ -1806,7 +1848,7 @@ private:
{
/// Account for optional timezone argument.
if (arguments.size() != 2 && arguments.size() != 3)
throw Exception{"Function " + getName() + " expects 2 or 3 arguments for DataTypeDateTime64.",
throw Exception{"Function " + getName() + " expects 2 or 3 arguments for DateTime64.",
ErrorCodes::TOO_FEW_ARGUMENTS_FOR_FUNCTION};
}
else if (arguments.size() != 2)

View File

@ -44,12 +44,12 @@ converted:
diff:
dest:
79 81 82 83 84 85 86 87 88 89 str01\0\0\0\0\0\0\0\0\0\0 fstr1\0\0\0\0\0\0\0\0\0\0 2003-03-04 2004-05-06 00:00:00 2004-05-06 07:08:09.012000000
80 81 82 83 84 85 86 87 88 89 str02 fstr2\0\0\0\0\0\0\0\0\0\0 2005-03-04 2006-08-09 10:11:12 2006-08-09 10:11:12.345000000
80 81 82 83 84 85 86 87 88 89 str02 fstr2\0\0\0\0\0\0\0\0\0\0 2149-06-06 2006-08-09 10:11:12 2006-08-09 10:11:12.345000000
min:
-128 0 0 0 0 0 0 0 -1 -1 string-1\0\0\0\0\0\0\0 fixedstring-1\0\0 2003-04-05 2003-02-03 2003-02-03 04:05:06.789000000
-108 108 8 92 -8 108 -40 -116 -1 -1 string-0\0\0\0\0\0\0\0 fixedstring\0\0\0\0 2001-02-03 2002-02-03 2002-02-03 04:05:06.789000000
-128 0 0 0 0 0 0 0 -1 -1 string-1\0\0\0\0\0\0\0 fixedstring-1\0\0 2003-04-05 2149-06-06 2003-02-03 04:05:06.789000000
-108 108 8 92 -8 108 -40 -116 -1 -1 string-0\0\0\0\0\0\0\0 fixedstring\0\0\0\0 2001-02-03 2149-06-06 2002-02-03 04:05:06.789000000
79 81 82 83 84 85 86 87 88 89 str01\0\0\0\0\0\0\0\0\0\0 fstr1\0\0\0\0\0\0\0\0\0\0 2003-03-04 2004-05-06 2004-05-06 07:08:09.012000000
127 -1 -1 -1 -1 -1 -1 -1 -1 -1 string-2\0\0\0\0\0\0\0 fixedstring-2\0\0 2004-06-07 2004-02-03 2004-02-03 04:05:06.789000000
127 -1 -1 -1 -1 -1 -1 -1 -1 -1 string-2\0\0\0\0\0\0\0 fixedstring-2\0\0 2004-06-07 2149-06-06 2004-02-03 04:05:06.789000000
max:
-128 0 -32768 0 -2147483648 0 -9223372036854775808 0 -1 -1 string-1 fixedstring-1\0\0 2003-04-05 00:00:00 2003-02-03 04:05:06 2003-02-03 04:05:06.789000000
-108 108 -1016 1116 -1032 1132 -1064 1164 -1 -1 string-0 fixedstring\0\0\0\0 2001-02-03 00:00:00 2002-02-03 04:05:06 2002-02-03 04:05:06.789000000

View File

@ -49,4 +49,3 @@ SELECT
toStartOfWeek(x, 3) AS w3,
toStartOfWeek(x_t, 3) AS wt3
FROM numbers(10);

View File

@ -1,4 +1,4 @@
0
1970-01-01 2106-02-07 1970-04-11 1970-01-01 2149-06-06
1970-01-01 2120-07-26 1970-04-11 1970-01-01 2149-06-06
1970-01-01 02:00:00 2106-02-07 09:28:15 1970-01-01 02:16:40
2000-01-01 13:12:12

View File

@ -43,16 +43,16 @@
-------toMinute---------
-------toSecond---------
-------toStartOfDay---------
2036-02-07 07:31:20
2036-02-07 07:31:20
2027-10-01 11:03:28
2027-10-17 11:03:28
1970-01-01 02:00:00
1970-01-01 02:00:00
2106-02-07 00:00:00
2106-02-07 00:00:00
2021-06-22 00:00:00
-------toMonday---------
2079-06-07
2079-06-07
2120-07-06
2120-07-20
1970-01-01
1970-01-01
2149-06-02
2149-06-02
2021-06-21
-------toISOWeek---------
1
@ -79,28 +79,28 @@
229953
202125
-------toStartOfWeek---------
2079-06-06
2079-06-06
2120-07-05
2120-07-26
1970-01-01
1970-01-01
2149-06-01
2149-06-01
2021-06-20
-------toStartOfMonth---------
2079-06-07
2079-06-07
2120-06-26
2120-06-26
1970-01-01
1970-01-01
2149-06-01
2149-06-01
2021-06-01
-------toStartOfQuarter---------
2079-06-07
2079-06-07
2120-04-26
2120-04-26
1970-01-01
1970-01-01
2149-04-01
2149-04-01
2021-04-01
-------toStartOfYear---------
2079-06-07
2079-06-07
2119-07-28
2119-07-28
1970-01-01
1970-01-01
2149-01-01
2149-01-01
2021-01-01
-------toStartOfSecond---------
-------toStartOfMinute---------

View File

@ -0,0 +1,20 @@
1970-01-01 2149-06-06 1970-01-01 2149-06-06 1900-01-01 1970-01-02 1970-01-01 00:00:00 2106-02-07 06:28:15
1970-01-01 2149-06-06
1970-01-01 2149-06-06
1970-01-01 00:00:00 2106-02-07 06:28:15
1970-01-01 00:00:00 2106-02-07 06:28:15
2106-02-07 06:28:15
toStartOfDay
2106-02-07 00:00:00 1970-01-01 00:00:00 2106-02-07 00:00:00 1970-01-01 00:00:00 2106-02-07 00:00:00
toStartOfWeek
1970-01-01 1970-01-01 1970-01-01 1970-01-01 1970-01-01 2149-06-01 1970-01-01 2149-06-02
toMonday
1970-01-01 1970-01-01 2149-06-02 1970-01-01 2149-06-02
toStartOfMonth
1970-01-01 2149-06-01 1970-01-01 2149-06-01
toLastDayOfMonth
2149-05-31 1970-01-01 2149-05-31 1970-01-01 2149-05-31
toStartOfQuarter
1970-01-01 2149-04-01 1970-01-01 2149-04-01
toStartOfYear
1970-01-01 2149-01-01 1970-01-01 2149-01-01

View File

@ -0,0 +1,74 @@
-- check conversion of numbers to date/time --
SELECT toDate(toInt32(toDate32('1930-01-01', 'UTC')), 'UTC'),
toDate(toInt32(toDate32('2151-01-01', 'UTC')), 'UTC'),
toDate(toInt64(toDateTime64('1930-01-01 12:12:12.123', 3, 'UTC')), 'UTC'),
toDate(toInt64(toDateTime64('2151-01-01 12:12:12.123', 3, 'UTC')), 'UTC'),
toDate32(toInt32(toDate32('1900-01-01', 'UTC')) - 1, 'UTC'),
toDate32(toInt32(toDate32('2299-12-31', 'UTC')) + 1, 'UTC'),
toDateTime(toInt64(toDateTime64('1930-01-01 12:12:12.123', 3, 'UTC')), 'UTC'),
toDateTime(toInt64(toDateTime64('2151-01-01 12:12:12.123', 3, 'UTC')), 'UTC');
-- check conversion of extended range type to normal range type --
SELECT toDate(toDate32('1930-01-01', 'UTC'), 'UTC'),
toDate(toDate32('2151-01-01', 'UTC'), 'UTC');
SELECT toDate(toDateTime64('1930-01-01 12:12:12.12', 3, 'UTC'), 'UTC'),
toDate(toDateTime64('2151-01-01 12:12:12.12', 3, 'UTC'), 'UTC');
SELECT toDateTime(toDateTime64('1930-01-01 12:12:12.12', 3, 'UTC'), 'UTC'),
toDateTime(toDateTime64('2151-01-01 12:12:12.12', 3, 'UTC'), 'UTC');
SELECT toDateTime(toDate32('1930-01-01', 'UTC'), 'UTC'),
toDateTime(toDate32('2151-01-01', 'UTC'), 'UTC');
SELECT toDateTime(toDate('2141-01-01', 'UTC'), 'UTC');
-- test DateTimeTransforms --
SELECT 'toStartOfDay';
SELECT toStartOfDay(toDate('2141-01-01', 'UTC'), 'UTC'),
toStartOfDay(toDate32('1930-01-01', 'UTC'), 'UTC'),
toStartOfDay(toDate32('2141-01-01', 'UTC'), 'UTC'),
toStartOfDay(toDateTime64('1930-01-01 12:12:12.123', 3, 'UTC'), 'UTC'),
toStartOfDay(toDateTime64('2141-01-01 12:12:12.123', 3, 'UTC'), 'UTC');
SELECT 'toStartOfWeek';
SELECT toStartOfWeek(toDate('1970-01-01', 'UTC')),
toStartOfWeek(toDate32('1970-01-01', 'UTC')),
toStartOfWeek(toDateTime('1970-01-01 10:10:10', 'UTC'), 0, 'UTC'),
toStartOfWeek(toDateTime64('1970-01-01 10:10:10.123', 3, 'UTC'), 1, 'UTC'),
toStartOfWeek(toDate32('1930-01-01', 'UTC')),
toStartOfWeek(toDate32('2151-01-01', 'UTC')),
toStartOfWeek(toDateTime64('1930-01-01 12:12:12.123', 3, 'UTC'), 2, 'UTC'),
toStartOfWeek(toDateTime64('2151-01-01 12:12:12.123', 3, 'UTC'), 3, 'UTC');
SELECT 'toMonday';
SELECT toMonday(toDate('1970-01-02', 'UTC')),
toMonday(toDate32('1930-01-01', 'UTC')),
toMonday(toDate32('2151-01-01', 'UTC')),
toMonday(toDateTime64('1930-01-01 12:12:12.123', 3, 'UTC'), 'UTC'),
toMonday(toDateTime64('2151-01-01 12:12:12.123', 3, 'UTC'), 'UTC');
SELECT 'toStartOfMonth';
SELECT toStartOfMonth(toDate32('1930-01-01', 'UTC')),
toStartOfMonth(toDate32('2151-01-01', 'UTC')),
toStartOfMonth(toDateTime64('1930-01-01 12:12:12.123', 3, 'UTC'), 'UTC'),
toStartOfMonth(toDateTime64('2151-01-01 12:12:12.123', 3, 'UTC'), 'UTC');
SELECT 'toLastDayOfMonth';
SELECT toLastDayOfMonth(toDate('2149-06-03', 'UTC')),
toLastDayOfMonth(toDate32('1930-01-01', 'UTC')),
toLastDayOfMonth(toDate32('2151-01-01', 'UTC')),
toLastDayOfMonth(toDateTime64('1930-01-01 12:12:12.123', 3, 'UTC'), 'UTC'),
toLastDayOfMonth(toDateTime64('2151-01-01 12:12:12.123', 3, 'UTC'), 'UTC');
SELECT 'toStartOfQuarter';
SELECT toStartOfQuarter(toDate32('1930-01-01', 'UTC')),
toStartOfQuarter(toDate32('2151-01-01', 'UTC')),
toStartOfQuarter(toDateTime64('1930-01-01 12:12:12.123', 3, 'UTC'), 'UTC'),
toStartOfQuarter(toDateTime64('2151-01-01 12:12:12.123', 3, 'UTC'), 'UTC');
SELECT 'toStartOfYear';
SELECT toStartOfYear(toDate32('1930-01-01', 'UTC')),
toStartOfYear(toDate32('2151-01-01', 'UTC')),
toStartOfYear(toDateTime64('1930-01-01 12:12:12.123', 3, 'UTC'), 'UTC'),
toStartOfYear(toDateTime64('2151-01-01 12:12:12.123', 3, 'UTC'), 'UTC');