Merge pull request #64616 from ClickHouse/revert-64386-fjurado-61548/feat-add-fromreadablesize-function

Revert "Add `fromReadableSize` function"
This commit is contained in:
Robert Schulze 2024-05-30 08:24:38 +00:00 committed by GitHub
commit 6af3b9ac12
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
9 changed files with 0 additions and 1050 deletions

View File

@ -735,8 +735,6 @@ LIMIT 10
Given a size (number of bytes), this function returns a readable, rounded size with suffix (KB, MB, etc.) as string.
The opposite operations of this function are [fromReadableDecimalSize](#fromReadableDecimalSize), [fromReadableDecimalSizeOrZero](#fromReadableDecimalSizeOrZero), and [fromReadableDecimalSizeOrNull](#fromReadableDecimalSizeOrNull).
**Syntax**
```sql
@ -768,8 +766,6 @@ Result:
Given a size (number of bytes), this function returns a readable, rounded size with suffix (KiB, MiB, etc.) as string.
The opposite operations of this function are [fromReadableSize](#fromReadableSize), [fromReadableSizeOrZero](#fromReadableSizeOrZero), and [fromReadableSizeOrNull](#fromReadableSizeOrNull).
**Syntax**
```sql
@ -894,238 +890,6 @@ SELECT
└────────────────────┴────────────────────────────────────────────────┘
```
## fromReadableSize
Given a string containing a byte size and `B`, `KiB`, `MiB`, etc. as a unit (i.e. [ISO/IEC 80000-13](https://en.wikipedia.org/wiki/ISO/IEC_80000) unit), this function returns the corresponding number of bytes.
If the function is unable to parse the input value, it throws an exception.
The opposite operation of this function is [formatReadableSize](#fromReadableSize).
**Syntax**
```sql
fromReadableSize(x)
```
**Arguments**
- `x` : Readable size with ISO/IEC 80000-13 units ([String](../../sql-reference/data-types/string.md)).
**Returned value**
- Number of bytes, rounded up to the nearest integer ([UInt64](../../sql-reference/data-types/int-uint.md)).
**Example**
```sql
SELECT
arrayJoin(['1 B', '1 KiB', '3 MiB', '5.314 KiB']) AS readable_sizes,
fromReadableSize(readable_sizes) AS sizes
```
```text
┌─readable_sizes─┬───sizes─┐
│ 1 B │ 1 │
│ 1 KiB │ 1024 │
│ 3 MiB │ 3145728 │
│ 5.314 KiB │ 5442 │
└────────────────┴─────────┘
```
## fromReadableSizeOrNull
Given a string containing a byte size and `B`, `KiB`, `MiB`, etc. as a unit (i.e. [ISO/IEC 80000-13](https://en.wikipedia.org/wiki/ISO/IEC_80000) unit), this function returns the corresponding number of bytes.
If the function is unable to parse the input value, it returns `NULL`.
The opposite operation of this function is [formatReadableSize](#fromReadableSize).
**Syntax**
```sql
fromReadableSizeOrNull(x)
```
**Arguments**
- `x` : Readable size with ISO/IEC 80000-13 units ([String](../../sql-reference/data-types/string.md)).
**Returned value**
- Number of bytes, rounded up to the nearest integer, or NULL if unable to parse the input (Nullable([UInt64](../../sql-reference/data-types/int-uint.md))).
**Example**
```sql
SELECT
arrayJoin(['1 B', '1 KiB', '3 MiB', '5.314 KiB', 'invalid']) AS readable_sizes,
fromReadableSizeOrNull(readable_sizes) AS sizes
```
```text
┌─readable_sizes─┬───sizes─┐
│ 1 B │ 1 │
│ 1 KiB │ 1024 │
│ 3 MiB │ 3145728 │
│ 5.314 KiB │ 5442 │
│ invalid │ ᴺᵁᴸᴸ │
└────────────────┴─────────┘
```
## fromReadableSizeOrZero
Given a string containing a byte size and `B`, `KiB`, `MiB`, etc. as a unit (i.e. [ISO/IEC 80000-13](https://en.wikipedia.org/wiki/ISO/IEC_80000) unit), this function returns the corresponding number of bytes.
If the function is unable to parse the input value, it returns `0`.
The opposite operation of this function is [formatReadableSize](#fromReadableSize).
**Syntax**
```sql
fromReadableSizeOrZero(x)
```
**Arguments**
- `x` : Readable size with ISO/IEC 80000-13 units ([String](../../sql-reference/data-types/string.md)).
**Returned value**
- Number of bytes, rounded up to the nearest integer, or 0 if unable to parse the input ([UInt64](../../sql-reference/data-types/int-uint.md)).
**Example**
```sql
SELECT
arrayJoin(['1 B', '1 KiB', '3 MiB', '5.314 KiB', 'invalid']) AS readable_sizes,
fromReadableSizeOrZero(readable_sizes) AS sizes
```
```text
┌─readable_sizes─┬───sizes─┐
│ 1 B │ 1 │
│ 1 KiB │ 1024 │
│ 3 MiB │ 3145728 │
│ 5.314 KiB │ 5442 │
│ invalid │ 0 │
└────────────────┴─────────┘
```
## fromReadableDecimalSize
Given a string containing a byte size and `B`, `KB`, `MB`, etc. as a unit, this function returns the corresponding number of bytes.
If the function is unable to parse the input value, it throws an exception.
The opposite operation of this function is [formatReadableDecimalSize](#formatReadableDecimalSize).
**Syntax**
```sql
fromReadableDecimalSize(x)
```
**Arguments**
- `x` : Readable size with decimal units ([String](../../sql-reference/data-types/string.md)).
**Returned value**
- Number of bytes, rounded up to the nearest integer ([UInt64](../../sql-reference/data-types/int-uint.md)).
**Example**
```sql
SELECT
arrayJoin(['1 B', '1 KB', '3 MB', '5.314 KB']) AS readable_sizes,
fromReadableDecimalSize(readable_sizes) AS sizes
```
```text
┌─readable_sizes─┬───sizes─┐
│ 1 B │ 1 │
│ 1 KB │ 1000 │
│ 3 MB │ 3000000 │
│ 5.314 KB │ 5314 │
└────────────────┴─────────┘
```
## fromReadableDecimalSizeOrNull
Given a string containing a byte size and `B`, `KB`, `MB`, etc. as a unit, this function returns the corresponding number of bytes.
If the function is unable to parse the input value, it returns `NULL`.
The opposite operation of this function is [formatReadableDecimalSize](#formatReadableDecimalSize).
**Syntax**
```sql
fromReadableDecimalSizeOrNull(x)
```
**Arguments**
- `x` : Readable size with decimal units ([String](../../sql-reference/data-types/string.md)).
**Returned value**
- Number of bytes, rounded up to the nearest integer, or NULL if unable to parse the input (Nullable([UInt64](../../sql-reference/data-types/int-uint.md))).
**Example**
```sql
SELECT
arrayJoin(['1 B', '1 KB', '3 MB', '5.314 KB', 'invalid']) AS readable_sizes,
fromReadableDecimalSizeOrNull(readable_sizes) AS sizes
```
```text
┌─readable_sizes─┬───sizes─┐
│ 1 B │ 1 │
│ 1 KB │ 1000 │
│ 3 MB │ 3000000 │
│ 5.314 KB │ 5314 │
│ invalid │ ᴺᵁᴸᴸ │
└────────────────┴─────────┘
```
## fromReadableDecimalSizeOrZero
Given a string containing a byte size and `B`, `KB`, `MB`, etc. as a unit, this function returns the corresponding number of bytes.
If the function is unable to parse the input value, it returns `0`.
The opposite operation of this function is [formatReadableDecimalSize](#formatReadableDecimalSize).
**Syntax**
```sql
fromReadableDecimalSizeOrZero(x)
```
**Arguments**
- `x` : Readable size with decimal units ([String](../../sql-reference/data-types/string.md)).
**Returned value**
- Number of bytes, rounded up to the nearest integer, or 0 if unable to parse the input ([UInt64](../../sql-reference/data-types/int-uint.md)).
**Example**
```sql
SELECT
arrayJoin(['1 B', '1 KB', '3 MB', '5.314 KB', 'invalid']) AS readable_sizes,
fromReadableSizeOrZero(readable_sizes) AS sizes
```
```text
┌─readable_sizes─┬───sizes─┐
│ 1 B │ 1 │
│ 1 KB │ 1000 │
│ 3 MB │ 3000000 │
│ 5.314 KB │ 5000 │
│ invalid │ 0 │
└────────────────┴─────────┘
```
## parseTimeDelta
Parse a sequence of numbers followed by something resembling a time unit.

View File

@ -1,221 +0,0 @@
#pragma once
#include <base/types.h>
#include <boost/algorithm/string/case_conv.hpp>
#include <Columns/ColumnNullable.h>
#include <Columns/ColumnsNumber.h>
#include <Columns/ColumnString.h>
#include <Common/Exception.h>
#include <DataTypes/DataTypeNullable.h>
#include <DataTypes/DataTypesNumber.h>
#include <Functions/FunctionHelpers.h>
#include <Functions/IFunction.h>
#include <IO/ReadBufferFromString.h>
#include <IO/ReadHelpers.h>
#include <cmath>
#include <string_view>
namespace DB
{
namespace ErrorCodes
{
extern const int BAD_ARGUMENTS;
extern const int CANNOT_PARSE_INPUT_ASSERTION_FAILED;
extern const int CANNOT_PARSE_NUMBER;
extern const int CANNOT_PARSE_TEXT;
extern const int ILLEGAL_COLUMN;
extern const int UNEXPECTED_DATA_AFTER_PARSED_VALUE;
}
enum class ErrorHandling : uint8_t
{
Exception,
Zero,
Null
};
using ScaleFactors = std::unordered_map<std::string_view, size_t>;
/** fromReadble*Size - Returns the number of bytes corresponding to a given readable binary or decimal size.
* Examples:
* - `fromReadableSize('123 MiB')`
* - `fromReadableDecimalSize('123 MB')`
* Meant to be the inverse of `formatReadable*Size` with the following exceptions:
* - Number of bytes is returned as an unsigned integer amount instead of a float. Decimal points are rounded up to the nearest integer.
* - Negative numbers are not allowed as negative sizes don't make sense.
* Flavours:
* - fromReadableSize
* - fromReadableSizeOrNull
* - fromReadableSizeOrZero
* - fromReadableDecimalSize
* - fromReadableDecimalSizeOrNull
* - fromReadableDecimalSizeOrZero
*/
template <typename Name, typename Impl, ErrorHandling error_handling>
class FunctionFromReadable : public IFunction
{
public:
static constexpr auto name = Name::name;
static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionFromReadable<Name, Impl, error_handling>>(); }
String getName() const override { return name; }
bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
bool useDefaultImplementationForConstants() const override { return true; }
size_t getNumberOfArguments() const override { return 1; }
DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
{
FunctionArgumentDescriptors args
{
{"readable_size", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isString), nullptr, "String"},
};
validateFunctionArgumentTypes(*this, arguments, args);
DataTypePtr return_type = std::make_shared<DataTypeUInt64>();
if (error_handling == ErrorHandling::Null)
return std::make_shared<DataTypeNullable>(return_type);
else
return return_type;
}
ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
{
const auto * col_str = checkAndGetColumn<ColumnString>(arguments[0].column.get());
if (!col_str)
{
throw Exception(
ErrorCodes::ILLEGAL_COLUMN,
"Illegal column {} of first ('str') argument of function {}. Must be string.",
arguments[0].column->getName(),
getName()
);
}
const ScaleFactors & scale_factors = Impl::getScaleFactors();
auto col_res = ColumnUInt64::create(input_rows_count);
ColumnUInt8::MutablePtr col_null_map;
if constexpr (error_handling == ErrorHandling::Null)
col_null_map = ColumnUInt8::create(input_rows_count, 0);
auto & res_data = col_res->getData();
for (size_t i = 0; i < input_rows_count; ++i)
{
std::string_view value = col_str->getDataAt(i).toView();
try
{
UInt64 num_bytes = parseReadableFormat(scale_factors, value);
res_data[i] = num_bytes;
}
catch (const Exception &)
{
if constexpr (error_handling == ErrorHandling::Exception)
{
throw;
}
else
{
res_data[i] = 0;
if constexpr (error_handling == ErrorHandling::Null)
col_null_map->getData()[i] = 1;
}
}
}
if constexpr (error_handling == ErrorHandling::Null)
return ColumnNullable::create(std::move(col_res), std::move(col_null_map));
else
return col_res;
}
private:
UInt64 parseReadableFormat(const ScaleFactors & scale_factors, const std::string_view & value) const
{
ReadBufferFromString buf(value);
// tryReadFloatText does seem to not raise any error when there is leading whitespace so we check it explicitly
skipWhitespaceIfAny(buf);
if (buf.getPosition() > 0)
{
throw Exception(
ErrorCodes::CANNOT_PARSE_INPUT_ASSERTION_FAILED,
"Invalid expression for function {} - Leading whitespace is not allowed (\"{}\")",
getName(),
value
);
}
Float64 base = 0;
if (!tryReadFloatTextPrecise(base, buf)) // If we use the default (fast) tryReadFloatText this returns True on garbage input so we use the Precise version
{
throw Exception(
ErrorCodes::CANNOT_PARSE_NUMBER,
"Invalid expression for function {} - Unable to parse readable size numeric component (\"{}\")",
getName(),
value
);
}
else if (std::isnan(base) || !std::isfinite(base))
{
throw Exception(
ErrorCodes::BAD_ARGUMENTS,
"Invalid expression for function {} - Invalid numeric component: {}",
getName(),
base
);
}
else if (base < 0)
{
throw Exception(
ErrorCodes::BAD_ARGUMENTS,
"Invalid expression for function {} - Negative sizes are not allowed ({})",
getName(),
base
);
}
skipWhitespaceIfAny(buf);
String unit;
readStringUntilWhitespace(unit, buf);
boost::algorithm::to_lower(unit);
auto iter = scale_factors.find(unit);
if (iter == scale_factors.end())
{
throw Exception(
ErrorCodes::CANNOT_PARSE_TEXT,
"Invalid expression for function {} - Unknown readable size unit (\"{}\")",
getName(),
unit
);
}
else if (!buf.eof())
{
throw Exception(
ErrorCodes::UNEXPECTED_DATA_AFTER_PARSED_VALUE,
"Invalid expression for function {} - Found trailing characters after readable size string (\"{}\")",
getName(),
value
);
}
Float64 num_bytes_with_decimals = base * iter->second;
if (num_bytes_with_decimals > std::numeric_limits<UInt64>::max())
{
throw Exception(
ErrorCodes::BAD_ARGUMENTS,
"Invalid expression for function {} - Result is too big for output type (\"{}\")",
getName(),
num_bytes_with_decimals
);
}
// As the input might be an arbitrary decimal number we might end up with a non-integer amount of bytes when parsing binary (eg MiB) units.
// This doesn't make sense so we round up to indicate the byte size that can fit the passed size.
return static_cast<UInt64>(std::ceil(num_bytes_with_decimals));
}
};
}

View File

@ -1,122 +0,0 @@
#include <base/types.h>
#include <Functions/FunctionFactory.h>
#include <Functions/fromReadable.h>
namespace DB
{
namespace
{
struct Impl
{
static const ScaleFactors & getScaleFactors()
{
static const ScaleFactors scale_factors =
{
{"b", 1ull},
{"kb", 1000ull},
{"mb", 1000ull * 1000ull},
{"gb", 1000ull * 1000ull * 1000ull},
{"tb", 1000ull * 1000ull * 1000ull * 1000ull},
{"pb", 1000ull * 1000ull * 1000ull * 1000ull * 1000ull},
{"eb", 1000ull * 1000ull * 1000ull * 1000ull * 1000ull * 1000ull},
};
return scale_factors;
}
};
struct NameFromReadableDecimalSize
{
static constexpr auto name = "fromReadableDecimalSize";
};
struct NameFromReadableDecimalSizeOrNull
{
static constexpr auto name = "fromReadableDecimalSizeOrNull";
};
struct NameFromReadableDecimalSizeOrZero
{
static constexpr auto name = "fromReadableDecimalSizeOrZero";
};
using FunctionFromReadableDecimalSize = FunctionFromReadable<NameFromReadableDecimalSize, Impl, ErrorHandling::Exception>;
using FunctionFromReadableDecimalSizeOrNull = FunctionFromReadable<NameFromReadableDecimalSizeOrNull, Impl, ErrorHandling::Null>;
using FunctionFromReadableDecimalSizeOrZero = FunctionFromReadable<NameFromReadableDecimalSizeOrZero, Impl, ErrorHandling::Zero>;
FunctionDocumentation fromReadableDecimalSize_documentation {
.description = "Given a string containing a byte size and `B`, `KB`, `MB`, etc. as a unit, this function returns the corresponding number of bytes. If the function is unable to parse the input value, it throws an exception.",
.syntax = "fromReadableDecimalSize(x)",
.arguments = {{"x", "Readable size with decimal units ([String](../../sql-reference/data-types/string.md))"}},
.returned_value = "Number of bytes, rounded up to the nearest integer ([UInt64](../../sql-reference/data-types/int-uint.md))",
.examples = {
{
"basic",
"SELECT arrayJoin(['1 B', '1 KB', '3 MB', '5.314 KB']) AS readable_sizes, fromReadableDecimalSize(readable_sizes) AS sizes;",
R"(
readable_sizessizes
1 B 1
1 KB 1000
3 MB 3000000
5.314 KB 5314
)"
},
},
.categories = {"OtherFunctions"},
};
FunctionDocumentation fromReadableDecimalSizeOrNull_documentation {
.description = "Given a string containing a byte size and `B`, `KiB`, `MiB`, etc. as a unit, this function returns the corresponding number of bytes. If the function is unable to parse the input value, it returns `NULL`",
.syntax = "fromReadableDecimalSizeOrNull(x)",
.arguments = {{"x", "Readable size with decimal units ([String](../../sql-reference/data-types/string.md))"}},
.returned_value = "Number of bytes, rounded up to the nearest integer, or NULL if unable to parse the input (Nullable([UInt64](../../sql-reference/data-types/int-uint.md)))",
.examples = {
{
"basic",
"SELECT arrayJoin(['1 B', '1 KB', '3 MB', '5.314 KB', 'invalid']) AS readable_sizes, fromReadableSizeOrNull(readable_sizes) AS sizes;",
R"(
readable_sizessizes
1 B 1
1 KB 1000
3 MB 3000000
5.314 KB 5314
invalid
)"
},
},
.categories = {"OtherFunctions"},
};
FunctionDocumentation fromReadableDecimalSizeOrZero_documentation {
.description = "Given a string containing a byte size and `B`, `KiB`, `MiB`, etc. as a unit, this function returns the corresponding number of bytes. If the function is unable to parse the input value, it returns `0`",
.syntax = "formatReadableSizeOrZero(x)",
.arguments = {{"x", "Readable size with decimal units ([String](../../sql-reference/data-types/string.md))"}},
.returned_value = "Number of bytes, rounded up to the nearest integer, or 0 if unable to parse the input ([UInt64](../../sql-reference/data-types/int-uint.md))",
.examples = {
{
"basic",
"SELECT arrayJoin(['1 B', '1 KB', '3 MB', '5.314 KB', 'invalid']) AS readable_sizes, fromReadableSizeOrZero(readable_sizes) AS sizes;",
R"(
readable_sizessizes
1 B 1
1 KB 1000
3 MB 3000000
5.314 KB 5000
invalid 0
)"
},
},
.categories = {"OtherFunctions"},
};
}
REGISTER_FUNCTION(FromReadableDecimalSize)
{
factory.registerFunction<FunctionFromReadableDecimalSize>(fromReadableDecimalSize_documentation);
factory.registerFunction<FunctionFromReadableDecimalSizeOrNull>(fromReadableDecimalSizeOrNull_documentation);
factory.registerFunction<FunctionFromReadableDecimalSizeOrZero>(fromReadableDecimalSizeOrZero_documentation);
}
}

View File

@ -1,124 +0,0 @@
#include <base/types.h>
#include <Functions/FunctionFactory.h>
#include <Functions/fromReadable.h>
#include "Common/FunctionDocumentation.h"
namespace DB
{
namespace
{
struct Impl
{
static const ScaleFactors & getScaleFactors()
{
// ISO/IEC 80000-13 binary units
static const ScaleFactors scale_factors =
{
{"b", 1ull},
{"kib", 1024ull},
{"mib", 1024ull * 1024ull},
{"gib", 1024ull * 1024ull * 1024ull},
{"tib", 1024ull * 1024ull * 1024ull * 1024ull},
{"pib", 1024ull * 1024ull * 1024ull * 1024ull * 1024ull},
{"eib", 1024ull * 1024ull * 1024ull * 1024ull * 1024ull * 1024ull},
};
return scale_factors;
}
};
struct NameFromReadableSize
{
static constexpr auto name = "fromReadableSize";
};
struct NameFromReadableSizeOrNull
{
static constexpr auto name = "fromReadableSizeOrNull";
};
struct NameFromReadableSizeOrZero
{
static constexpr auto name = "fromReadableSizeOrZero";
};
using FunctionFromReadableSize = FunctionFromReadable<NameFromReadableSize, Impl, ErrorHandling::Exception>;
using FunctionFromReadableSizeOrNull = FunctionFromReadable<NameFromReadableSizeOrNull, Impl, ErrorHandling::Null>;
using FunctionFromReadableSizeOrZero = FunctionFromReadable<NameFromReadableSizeOrZero, Impl, ErrorHandling::Zero>;
FunctionDocumentation fromReadableSize_documentation {
.description = "Given a string containing a byte size and `B`, `KiB`, `MiB`, etc. as a unit (i.e. [ISO/IEC 80000-13](https://en.wikipedia.org/wiki/ISO/IEC_80000) unit), this function returns the corresponding number of bytes. If the function is unable to parse the input value, it throws an exception.",
.syntax = "fromReadableSize(x)",
.arguments = {{"x", "Readable size with ISO/IEC 80000-13 units ([String](../../sql-reference/data-types/string.md))"}},
.returned_value = "Number of bytes, rounded up to the nearest integer ([UInt64](../../sql-reference/data-types/int-uint.md))",
.examples = {
{
"basic",
"SELECT arrayJoin(['1 B', '1 KiB', '3 MiB', '5.314 KiB']) AS readable_sizes, fromReadableSize(readable_sizes) AS sizes;",
R"(
readable_sizessizes
1 B 1
1 KiB 1024
3 MiB 3145728
5.314 KiB 5442
)"
},
},
.categories = {"OtherFunctions"},
};
FunctionDocumentation fromReadableSizeOrNull_documentation {
.description = "Given a string containing a byte size and `B`, `KiB`, `MiB`, etc. as a unit (i.e. [ISO/IEC 80000-13](https://en.wikipedia.org/wiki/ISO/IEC_80000) unit), this function returns the corresponding number of bytes. If the function is unable to parse the input value, it returns `NULL`",
.syntax = "fromReadableSizeOrNull(x)",
.arguments = {{"x", "Readable size with ISO/IEC 80000-13 units ([String](../../sql-reference/data-types/string.md))"}},
.returned_value = "Number of bytes, rounded up to the nearest integer, or NULL if unable to parse the input (Nullable([UInt64](../../sql-reference/data-types/int-uint.md)))",
.examples = {
{
"basic",
"SELECT arrayJoin(['1 B', '1 KiB', '3 MiB', '5.314 KiB', 'invalid']) AS readable_sizes, fromReadableSize(readable_sizes) AS sizes;",
R"(
readable_sizessizes
1 B 1
1 KiB 1024
3 MiB 3145728
5.314 KiB 5442
invalid
)"
},
},
.categories = {"OtherFunctions"},
};
FunctionDocumentation fromReadableSizeOrZero_documentation {
.description = "Given a string containing a byte size and `B`, `KiB`, `MiB`, etc. as a unit (i.e. [ISO/IEC 80000-13](https://en.wikipedia.org/wiki/ISO/IEC_80000) unit), this function returns the corresponding number of bytes. If the function is unable to parse the input value, it returns `0`",
.syntax = "fromReadableSizeOrZero(x)",
.arguments = {{"x", "Readable size with ISO/IEC 80000-13 units ([String](../../sql-reference/data-types/string.md))"}},
.returned_value = "Number of bytes, rounded up to the nearest integer, or 0 if unable to parse the input ([UInt64](../../sql-reference/data-types/int-uint.md))",
.examples = {
{
"basic",
"SELECT arrayJoin(['1 B', '1 KiB', '3 MiB', '5.314 KiB', 'invalid']) AS readable_sizes, fromReadableSize(readable_sizes) AS sizes;",
R"(
readable_sizessizes
1 B 1
1 KiB 1024
3 MiB 3145728
5.314 KiB 5442
invalid 0
)",
},
},
.categories = {"OtherFunctions"},
};
}
REGISTER_FUNCTION(FromReadableSize)
{
factory.registerFunction<FunctionFromReadableSize>(fromReadableSize_documentation);
factory.registerFunction<FunctionFromReadableSizeOrNull>(fromReadableSizeOrNull_documentation);
factory.registerFunction<FunctionFromReadableSizeOrZero>(fromReadableSizeOrZero_documentation);
}
}

View File

@ -1,53 +0,0 @@
1.00 B
1.00 KiB
1.00 MiB
1.00 GiB
1.00 TiB
1.00 PiB
1.00 EiB
1.00 MiB
1024
3072
1024
1024
1024
1024
1024
\N
3217
3217
1000
5
2048
8192
0 0 0
1 B 1
1 KiB 1024
1 MiB 1048576
1 GiB 1073741824
1 TiB 1099511627776
1 PiB 1125899906842624
1 EiB 1152921504606846976
invalid \N
1 Joe \N
1KB \N
1 GiB \N
1 TiB with fries \N
NaN KiB \N
Inf KiB \N
0xa123 KiB \N
1 B 1
1 KiB 1024
1 MiB 1048576
1 GiB 1073741824
1 TiB 1099511627776
1 PiB 1125899906842624
1 EiB 1152921504606846976
invalid 0
1 Joe 0
1KB 0
1 GiB 0
1 TiB with fries 0
NaN KiB 0
Inf KiB 0
0xa123 KiB 0

View File

@ -1,118 +0,0 @@
-- Should be kept in sync with 03167_fromReadableDecimalSize.sql
-- Should be the inverse of formatReadableSize
SELECT formatReadableSize(fromReadableSize('1 B'));
SELECT formatReadableSize(fromReadableSize('1 KiB'));
SELECT formatReadableSize(fromReadableSize('1 MiB'));
SELECT formatReadableSize(fromReadableSize('1 GiB'));
SELECT formatReadableSize(fromReadableSize('1 TiB'));
SELECT formatReadableSize(fromReadableSize('1 PiB'));
SELECT formatReadableSize(fromReadableSize('1 EiB'));
-- Is case-insensitive
SELECT formatReadableSize(fromReadableSize('1 mIb'));
-- Should be able to parse decimals
SELECT fromReadableSize('1.00 KiB'); -- 1024
SELECT fromReadableSize('3.00 KiB'); -- 3072
-- Infix whitespace is ignored
SELECT fromReadableSize('1 KiB');
SELECT fromReadableSize('1KiB');
-- Can parse LowCardinality
SELECT fromReadableSize(toLowCardinality('1 KiB'));
-- Can parse nullable fields
SELECT fromReadableSize(toNullable('1 KiB'));
-- Can parse non-const columns fields
SELECT fromReadableSize(materialize('1 KiB'));
-- Output is NULL if NULL arg is passed
SELECT fromReadableSize(NULL);
-- Can parse more decimal places than Float64's precision
SELECT fromReadableSize('3.14159265358979323846264338327950288419716939937510 KiB');
-- Can parse sizes prefixed with a plus sign
SELECT fromReadableSize('+3.1415 KiB');
-- Can parse amounts in scientific notation
SELECT fromReadableSize('10e2 B');
-- Can parse floats with no decimal points
SELECT fromReadableSize('5. B');
-- Can parse numbers with leading zeroes
SELECT fromReadableSize('002 KiB');
-- Can parse octal-like
SELECT fromReadableSize('08 KiB');
-- Can parse various flavours of zero
SELECT fromReadableSize('0 KiB'), fromReadableSize('+0 KiB'), fromReadableSize('-0 KiB');
-- ERRORS
-- No arguments
SELECT fromReadableSize(); -- { serverError NUMBER_OF_ARGUMENTS_DOESNT_MATCH }
-- Too many arguments
SELECT fromReadableSize('1 B', '2 B'); -- { serverError NUMBER_OF_ARGUMENTS_DOESNT_MATCH }
-- Wrong Type
SELECT fromReadableSize(12); -- { serverError ILLEGAL_TYPE_OF_ARGUMENT }
-- Invalid input - overall garbage
SELECT fromReadableSize('oh no'); -- { serverError CANNOT_PARSE_NUMBER }
-- Invalid input - unknown unit
SELECT fromReadableSize('12.3 rb'); -- { serverError CANNOT_PARSE_TEXT }
-- Invalid input - Leading whitespace
SELECT fromReadableSize(' 1 B'); -- { serverError CANNOT_PARSE_INPUT_ASSERTION_FAILED }
-- Invalid input - Trailing characters
SELECT fromReadableSize('1 B leftovers'); -- { serverError UNEXPECTED_DATA_AFTER_PARSED_VALUE }
-- Invalid input - Decimal size unit is not accepted
SELECT fromReadableSize('1 KB'); -- { serverError CANNOT_PARSE_TEXT }
-- Invalid input - Negative sizes are not allowed
SELECT fromReadableSize('-1 KiB'); -- { serverError BAD_ARGUMENTS }
-- Invalid input - Input too large to fit in UInt64
SELECT fromReadableSize('1000 EiB'); -- { serverError BAD_ARGUMENTS }
-- Invalid input - Hexadecimal is not supported
SELECT fromReadableSize('0xa123 KiB'); -- { serverError CANNOT_PARSE_TEXT }
-- Invalid input - NaN is not supported, with or without sign and with different capitalizations
SELECT fromReadableSize('nan KiB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableSize('+nan KiB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableSize('-nan KiB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableSize('NaN KiB'); -- { serverError BAD_ARGUMENTS }
-- Invalid input - Infinite is not supported, with or without sign, in all its forms
SELECT fromReadableSize('inf KiB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableSize('+inf KiB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableSize('-inf KiB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableSize('infinite KiB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableSize('+infinite KiB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableSize('-infinite KiB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableSize('Inf KiB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableSize('Infinite KiB'); -- { serverError BAD_ARGUMENTS }
-- OR NULL
-- Works as the regular version when inputs are correct
SELECT
arrayJoin(['1 B', '1 KiB', '1 MiB', '1 GiB', '1 TiB', '1 PiB', '1 EiB']) AS readable_sizes,
fromReadableSizeOrNull(readable_sizes) AS filesize;
-- Returns NULL on invalid values
SELECT
arrayJoin(['invalid', '1 Joe', '1KB', ' 1 GiB', '1 TiB with fries', 'NaN KiB', 'Inf KiB', '0xa123 KiB']) AS readable_sizes,
fromReadableSizeOrNull(readable_sizes) AS filesize;
-- OR ZERO
-- Works as the regular version when inputs are correct
SELECT
arrayJoin(['1 B', '1 KiB', '1 MiB', '1 GiB', '1 TiB', '1 PiB', '1 EiB']) AS readable_sizes,
fromReadableSizeOrZero(readable_sizes) AS filesize;
-- Returns NULL on invalid values
SELECT
arrayJoin(['invalid', '1 Joe', '1KB', ' 1 GiB', '1 TiB with fries', 'NaN KiB', 'Inf KiB', '0xa123 KiB']) AS readable_sizes,
fromReadableSizeOrZero(readable_sizes) AS filesize;

View File

@ -1,53 +0,0 @@
1.00 B
1.00 KB
1.00 MB
1.00 GB
1.00 TB
1.00 PB
1.00 EB
1.00 MB
1000
3000
1000
1000
1000
1000
1000
\N
3142
3142
1000
5
2000
8000
0 0 0
1 B 1
1 KB 1000
1 MB 1000000
1 GB 1000000000
1 TB 1000000000000
1 PB 1000000000000000
1 EB 1000000000000000000
invalid \N
1 Joe \N
1 KiB \N
1 GB \N
1 TB with fries \N
NaN KB \N
Inf KB \N
0xa123 KB \N
1 B 1
1 KB 1000
1 MB 1000000
1 GB 1000000000
1 TB 1000000000000
1 PB 1000000000000000
1 EB 1000000000000000000
invalid 0
1 Joe 0
1 KiB 0
1 GiB 0
1 TiB with fries 0
NaN KB 0
Inf KB 0
0xa123 KB 0

View File

@ -1,117 +0,0 @@
-- Should be kept in sync with 03166_fromReadableSize.sql
-- Should be the inverse of formatReadableDecimalSize
SELECT formatReadableDecimalSize(fromReadableDecimalSize('1 B'));
SELECT formatReadableDecimalSize(fromReadableDecimalSize('1 KB'));
SELECT formatReadableDecimalSize(fromReadableDecimalSize('1 MB'));
SELECT formatReadableDecimalSize(fromReadableDecimalSize('1 GB'));
SELECT formatReadableDecimalSize(fromReadableDecimalSize('1 TB'));
SELECT formatReadableDecimalSize(fromReadableDecimalSize('1 PB'));
SELECT formatReadableDecimalSize(fromReadableDecimalSize('1 EB'));
-- Is case-insensitive
SELECT formatReadableDecimalSize(fromReadableDecimalSize('1 mb'));
-- Should be able to parse decimals
SELECT fromReadableDecimalSize('1.00 KB'); -- 1024
SELECT fromReadableDecimalSize('3.00 KB'); -- 3072
-- Infix whitespace is ignored
SELECT fromReadableDecimalSize('1 KB');
SELECT fromReadableDecimalSize('1KB');
-- Can parse LowCardinality
SELECT fromReadableDecimalSize(toLowCardinality('1 KB'));
-- Can parse nullable fields
SELECT fromReadableDecimalSize(toNullable('1 KB'));
-- Can parse non-const columns fields
SELECT fromReadableDecimalSize(materialize('1 KB'));
-- Output is NULL if NULL arg is passed
SELECT fromReadableDecimalSize(NULL);
-- Can parse more decimal places than Float64's precision
SELECT fromReadableDecimalSize('3.14159265358979323846264338327950288419716939937510 KB');
-- Can parse sizes prefixed with a plus sign
SELECT fromReadableDecimalSize('+3.1415 KB');
-- Can parse amounts in scientific notation
SELECT fromReadableDecimalSize('10e2 B');
-- Can parse floats with no decimal points
SELECT fromReadableDecimalSize('5. B');
-- Can parse numbers with leading zeroes
SELECT fromReadableDecimalSize('002 KB');
-- Can parse octal-like
SELECT fromReadableDecimalSize('08 KB');
-- Can parse various flavours of zero
SELECT fromReadableDecimalSize('0 KB'), fromReadableDecimalSize('+0 KB'), fromReadableDecimalSize('-0 KB');
-- ERRORS
-- No arguments
SELECT fromReadableDecimalSize(); -- { serverError NUMBER_OF_ARGUMENTS_DOESNT_MATCH }
-- Too many arguments
SELECT fromReadableDecimalSize('1 B', '2 B'); -- { serverError NUMBER_OF_ARGUMENTS_DOESNT_MATCH }
-- Wrong Type
SELECT fromReadableDecimalSize(12); -- { serverError ILLEGAL_TYPE_OF_ARGUMENT }
-- Invalid input - overall garbage
SELECT fromReadableDecimalSize('oh no'); -- { serverError CANNOT_PARSE_NUMBER }
-- Invalid input - unknown unit
SELECT fromReadableDecimalSize('12.3 rb'); -- { serverError CANNOT_PARSE_TEXT }
-- Invalid input - Leading whitespace
SELECT fromReadableDecimalSize(' 1 B'); -- { serverError CANNOT_PARSE_INPUT_ASSERTION_FAILED }
-- Invalid input - Trailing characters
SELECT fromReadableDecimalSize('1 B leftovers'); -- { serverError UNEXPECTED_DATA_AFTER_PARSED_VALUE }
-- Invalid input - Binary size unit is not accepted
SELECT fromReadableDecimalSize('1 KiB'); -- { serverError CANNOT_PARSE_TEXT }
-- Invalid input - Negative sizes are not allowed
SELECT fromReadableDecimalSize('-1 KB'); -- { serverError BAD_ARGUMENTS }
-- Invalid input - Input too large to fit in UInt64
SELECT fromReadableDecimalSize('1000 EB'); -- { serverError BAD_ARGUMENTS }
-- Invalid input - Hexadecimal is not supported
SELECT fromReadableDecimalSize('0xa123 KB'); -- { serverError CANNOT_PARSE_TEXT }
-- Invalid input - NaN is not supported, with or without sign and with different capitalizations
SELECT fromReadableDecimalSize('nan KB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableDecimalSize('+nan KB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableDecimalSize('-nan KB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableDecimalSize('NaN KB'); -- { serverError BAD_ARGUMENTS }
-- Invalid input - Infinite is not supported, with or without sign, in all its forms
SELECT fromReadableDecimalSize('inf KB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableDecimalSize('+inf KB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableDecimalSize('-inf KB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableDecimalSize('infinite KB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableDecimalSize('+infinite KB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableDecimalSize('-infinite KB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableDecimalSize('Inf KB'); -- { serverError BAD_ARGUMENTS }
SELECT fromReadableDecimalSize('Infinite KB'); -- { serverError BAD_ARGUMENTS }
-- OR NULL
-- Works as the regular version when inputs are correct
SELECT
arrayJoin(['1 B', '1 KB', '1 MB', '1 GB', '1 TB', '1 PB', '1 EB']) AS readable_sizes,
fromReadableDecimalSizeOrNull(readable_sizes) AS filesize;
-- Returns NULL on invalid values
SELECT
arrayJoin(['invalid', '1 Joe', '1 KiB', ' 1 GB', '1 TB with fries', 'NaN KB', 'Inf KB', '0xa123 KB']) AS readable_sizes,
fromReadableDecimalSizeOrNull(readable_sizes) AS filesize;
-- OR ZERO
-- Works as the regular version when inputs are correct
SELECT
arrayJoin(['1 B', '1 KB', '1 MB', '1 GB', '1 TB', '1 PB', '1 EB']) AS readable_sizes,
fromReadableDecimalSizeOrZero(readable_sizes) AS filesize;
-- Returns NULL on invalid values
SELECT
arrayJoin(['invalid', '1 Joe', '1 KiB', ' 1 GiB', '1 TiB with fries', 'NaN KB', 'Inf KB', '0xa123 KB']) AS readable_sizes,
fromReadableDecimalSizeOrZero(readable_sizes) AS filesize;

View File

@ -1613,12 +1613,6 @@ freezed
fromDaysSinceYearZero
fromModifiedJulianDay
fromModifiedJulianDayOrNull
fromReadableSize
fromReadableSizeOrNull
fromReadableSizeOrZero
fromReadableDecimalSize
fromReadableDecimalSizeOrNull
fromReadableDecimalSizeOrZero
fromUTCTimestamp
fromUnixTimestamp
fromUnixTimestampInJodaSyntax