ClickHouse/docs/en/sql-reference/functions/string-functions.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

1311 lines
36 KiB
Markdown
Raw Normal View History

2020-04-03 13:23:32 +00:00
---
2022-08-28 14:53:34 +00:00
slug: /en/sql-reference/functions/string-functions
2023-04-19 17:05:55 +00:00
sidebar_position: 170
sidebar_label: Strings
2020-04-03 13:23:32 +00:00
---
2022-06-02 10:55:18 +00:00
# Functions for Working with Strings
2023-04-20 10:08:49 +00:00
Functions for [searching](string-search-functions.md) in strings and for [replacing](string-replace-functions.md) in strings are described separately.
2020-06-19 10:09:45 +00:00
2022-06-02 10:55:18 +00:00
## empty
2017-04-03 19:49:50 +00:00
2021-08-03 13:07:46 +00:00
Checks whether the input string is empty.
2023-04-19 20:04:59 +00:00
A string is considered non-empty if it contains at least one byte, even if this byte is a space or the null byte.
The function is also available for [arrays](array-functions.md#function-empty) and [UUIDs](uuid-functions.md#empty).
2021-08-03 13:07:46 +00:00
**Syntax**
``` sql
empty(x)
```
**Arguments**
- `x` — Input value. [String](../data-types/string.md).
2021-08-03 13:07:46 +00:00
**Returned value**
- Returns `1` for an empty string or `0` for a non-empty string.
2021-08-03 13:07:46 +00:00
Type: [UInt8](../data-types/int-uint.md).
**Example**
```sql
SELECT empty('');
```
Result:
2023-04-19 20:04:59 +00:00
```result
2021-08-03 13:07:46 +00:00
┌─empty('')─┐
│ 1 │
└───────────┘
```
2017-04-03 19:49:50 +00:00
2022-06-02 10:55:18 +00:00
## notEmpty
2021-08-10 10:53:35 +00:00
Checks whether the input string is non-empty.
2021-08-06 09:04:55 +00:00
2023-04-19 20:04:59 +00:00
A string is considered non-empty if it contains at least one byte, even if this byte is a space or the null byte.
The function is also available for [arrays](array-functions.md#function-notempty) and [UUIDs](uuid-functions.md#notempty).
2021-08-06 09:04:55 +00:00
**Syntax**
``` sql
2021-08-10 10:53:35 +00:00
notEmpty(x)
2021-08-06 09:04:55 +00:00
```
**Arguments**
- `x` — Input value. [String](../data-types/string.md).
2021-08-06 09:04:55 +00:00
**Returned value**
- Returns `1` for a non-empty string or `0` for an empty string string.
2021-08-06 09:04:55 +00:00
Type: [UInt8](../data-types/int-uint.md).
**Example**
```sql
2021-08-10 10:53:35 +00:00
SELECT notEmpty('text');
2021-08-06 09:04:55 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
2021-08-10 10:53:35 +00:00
┌─notEmpty('text')─┐
│ 1 │
└──────────────────┘
2021-08-06 09:04:55 +00:00
```
2017-04-03 19:49:50 +00:00
2022-06-02 10:55:18 +00:00
## length
2023-04-19 20:04:59 +00:00
Returns the length of a string in bytes (not: in characters or Unicode code points).
2017-04-26 19:16:38 +00:00
The function also works for arrays.
2017-04-03 19:49:50 +00:00
2022-06-02 10:55:18 +00:00
## lengthUTF8
2023-04-19 20:04:59 +00:00
Returns the length of a string in Unicode code points (not: in bytes or characters). It assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
2023-04-19 20:04:59 +00:00
Alias:
- `CHAR_LENGTH``
- `CHARACTER_LENGTH`
2022-06-02 10:55:18 +00:00
## leftPad
2021-08-03 18:58:32 +00:00
2023-04-19 20:04:59 +00:00
Pads a string from the left with spaces or with a specified string (multiple times, if needed) until the resulting string reaches the specified `length`.
2021-08-03 18:58:32 +00:00
**Syntax**
``` sql
2023-04-19 20:04:59 +00:00
leftPad(string, length[, pad_string])
2021-08-03 18:58:32 +00:00
```
2023-04-19 20:04:59 +00:00
Alias: `LPAD`
2021-08-03 18:58:32 +00:00
**Arguments**
2023-04-19 20:04:59 +00:00
- `string` — Input string that should be padded. [String](../data-types/string.md).
- `length` — The length of the resulting string. [UInt or Int](../data-types/int-uint.md). If the value is smaller than the input string length, then the input string is shortened to `length` characters.
- `pad_string` — The string to pad the input string with. [String](../data-types/string.md). Optional. If not specified, then the input string is padded with spaces.
2021-08-03 18:58:32 +00:00
2021-08-09 14:29:26 +00:00
**Returned value**
2021-08-03 18:58:32 +00:00
2023-04-19 20:04:59 +00:00
- A left-padded string of the given length.
2021-08-03 18:58:32 +00:00
Type: [String](../data-types/string.md).
**Example**
``` sql
SELECT leftPad('abc', 7, '*'), leftPad('def', 7);
2021-08-03 18:58:32 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─leftPad('abc', 7, '*')─┬─leftPad('def', 7)─┐
****abc │ def │
└────────────────────────┴───────────────────┘
2021-08-03 18:58:32 +00:00
```
2022-06-02 10:55:18 +00:00
## leftPadUTF8
2021-08-06 15:03:31 +00:00
2023-04-19 20:04:59 +00:00
Pads the string from the left with spaces or a specified string (multiple times, if needed) until the resulting string reaches the given length. Unlike [leftPad](#leftpad) which measures the string length in bytes, the string length is measured in code points.
2021-08-06 15:03:31 +00:00
**Syntax**
``` sql
2023-04-19 20:04:59 +00:00
leftPadUTF8(string, length[, pad_string])
2021-08-06 15:03:31 +00:00
```
**Arguments**
2023-04-19 20:04:59 +00:00
- `string` — Input string that should be padded. [String](../data-types/string.md).
- `length` — The length of the resulting string. [UInt or Int](../data-types/int-uint.md). If the value is smaller than the input string length, then the input string is shortened to `length` characters.
- `pad_string` — The string to pad the input string with. [String](../data-types/string.md). Optional. If not specified, then the input string is padded with spaces.
2021-08-06 15:03:31 +00:00
2021-08-09 14:29:26 +00:00
**Returned value**
2021-08-06 15:03:31 +00:00
2023-04-19 20:04:59 +00:00
- A left-padded string of the given length.
2021-08-06 15:03:31 +00:00
Type: [String](../data-types/string.md).
**Example**
``` sql
SELECT leftPadUTF8('абвг', 7, '*'), leftPadUTF8('дежз', 7);
2021-08-06 15:03:31 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─leftPadUTF8('абвг', 7, '*')─┬─leftPadUTF8('дежз', 7)─┐
***абвг │ дежз │
└─────────────────────────────┴────────────────────────┘
2021-08-06 15:03:31 +00:00
```
2022-06-02 10:55:18 +00:00
## rightPad
2021-08-03 18:58:32 +00:00
2023-04-19 20:04:59 +00:00
Pads a string from the right with spaces or with a specified string (multiple times, if needed) until the resulting string reaches the specified `length`.
2021-08-03 18:58:32 +00:00
**Syntax**
``` sql
2023-04-19 20:04:59 +00:00
rightPad(string, length[, pad_string])
2021-08-03 18:58:32 +00:00
```
2023-04-19 20:04:59 +00:00
Alias: `RPAD`
2021-08-03 18:58:32 +00:00
**Arguments**
2023-04-19 20:04:59 +00:00
- `string` — Input string that should be padded. [String](../data-types/string.md).
- `length` — The length of the resulting string. [UInt or Int](../data-types/int-uint.md). If the value is smaller than the input string length, then the input string is shortened to `length` characters.
- `pad_string` — The string to pad the input string with. [String](../data-types/string.md). Optional. If not specified, then the input string is padded with spaces.
2021-08-03 18:58:32 +00:00
2021-08-09 14:29:26 +00:00
**Returned value**
2021-08-03 18:58:32 +00:00
2023-04-19 20:04:59 +00:00
- A left-padded string of the given length.
2021-08-03 18:58:32 +00:00
Type: [String](../data-types/string.md).
**Example**
``` sql
2021-08-09 13:43:17 +00:00
SELECT rightPad('abc', 7, '*'), rightPad('abc', 7);
2021-08-03 18:58:32 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
2021-08-09 13:43:17 +00:00
┌─rightPad('abc', 7, '*')─┬─rightPad('abc', 7)─┐
│ abc**** │ abc │
└─────────────────────────┴────────────────────┘
2021-08-03 18:58:32 +00:00
```
2022-06-02 10:55:18 +00:00
## rightPadUTF8
2021-08-06 15:03:31 +00:00
2023-04-19 20:04:59 +00:00
Pads the string from the right with spaces or a specified string (multiple times, if needed) until the resulting string reaches the given length. Unlike [rightPad](#rightpad) which measures the string length in bytes, the string length is measured in code points.
2021-08-06 15:03:31 +00:00
**Syntax**
``` sql
2023-04-19 20:04:59 +00:00
rightPadUTF8(string, length[, pad_string])
2021-08-06 15:03:31 +00:00
```
**Arguments**
2023-04-19 20:04:59 +00:00
- `string` — Input string that should be padded. [String](../data-types/string.md).
- `length` — The length of the resulting string. [UInt or Int](../data-types/int-uint.md). If the value is smaller than the input string length, then the input string is shortened to `length` characters.
- `pad_string` — The string to pad the input string with. [String](../data-types/string.md). Optional. If not specified, then the input string is padded with spaces.
2021-08-06 15:03:31 +00:00
2021-08-09 14:29:26 +00:00
**Returned value**
2021-08-06 15:03:31 +00:00
2023-04-19 20:04:59 +00:00
- A right-padded string of the given length.
2021-08-06 15:03:31 +00:00
Type: [String](../data-types/string.md).
**Example**
``` sql
2021-08-09 13:43:17 +00:00
SELECT rightPadUTF8('абвг', 7, '*'), rightPadUTF8('абвг', 7);
2021-08-06 15:03:31 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
2021-08-09 13:43:17 +00:00
┌─rightPadUTF8('абвг', 7, '*')─┬─rightPadUTF8('абвг', 7)─┐
│ абвг*** │ абвг │
└──────────────────────────────┴─────────────────────────┘
2021-08-06 15:03:31 +00:00
```
2023-04-19 20:04:59 +00:00
## lower
Converts the ASCII Latin symbols in a string to lowercase.
2023-04-19 20:04:59 +00:00
Alias: `lcase`
2017-04-03 19:49:50 +00:00
2023-04-19 20:04:59 +00:00
## upper
2023-04-19 20:04:59 +00:00
Converts the ASCII Latin symbols in a string to uppercase.
Alias: `ucase`
2017-04-03 19:49:50 +00:00
2022-06-02 10:55:18 +00:00
## lowerUTF8
2023-04-19 20:04:59 +00:00
Converts a string to lowercase, assuming that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Does not detect the language, e.g. for Turkish the result might not be exactly correct (i/İ vs. i/I).
If the length of the UTF-8 byte sequence is different for upper and lower case of a code point, the result may be incorrect for this code point.
2022-06-02 10:55:18 +00:00
## upperUTF8
2017-04-03 19:49:50 +00:00
2023-04-19 20:04:59 +00:00
Converts a string to uppercase, assuming that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Does not detect the language, e.g. for Turkish the result might not be exactly correct (i/İ vs. i/I).
If the length of the UTF-8 byte sequence is different for upper and lower case of a code point, the result may be incorrect for this code point.
2022-06-02 10:55:18 +00:00
## isValidUTF8
2019-04-07 18:58:13 +00:00
2023-04-19 20:04:59 +00:00
Returns 1, if the set of bytes constitutes valid UTF-8-encoded text, otherwise 0.
2019-04-07 18:58:13 +00:00
2022-06-02 10:55:18 +00:00
## toValidUTF8
2019-05-17 12:55:21 +00:00
Replaces invalid UTF-8 characters by the `<60>` (U+FFFD) character. All running in a row invalid characters are collapsed into the one replacement character.
2023-04-19 20:04:59 +00:00
**Syntax**
2020-03-20 10:10:48 +00:00
``` sql
toValidUTF8(input_string)
```
2021-02-16 11:21:23 +00:00
**Arguments**
- `input_string` — Any set of bytes represented as the [String](../../sql-reference/data-types/string.md) data type object.
2023-04-19 20:04:59 +00:00
**Returned value**
- A valid UTF-8 string.
**Example**
2020-03-20 10:10:48 +00:00
``` sql
SELECT toValidUTF8('\x61\xF0\x80\x80\x80b');
```
2020-03-20 10:10:48 +00:00
2023-04-19 20:04:59 +00:00
```result
2019-05-23 11:37:05 +00:00
┌─toValidUTF8('a<><61><EFBFBD><EFBFBD>b')─┐
│ a<>b │
└───────────────────────┘
2021-10-14 10:15:45 +00:00
```
2022-06-02 10:55:18 +00:00
## repeat
2023-04-19 20:04:59 +00:00
Conatenates a string as many times with itself as specified.
2021-02-16 11:31:24 +00:00
**Syntax**
2020-03-20 10:10:48 +00:00
``` sql
repeat(s, n)
```
2023-04-19 20:04:59 +00:00
Alias: `REPEAT`
**Arguments**
- `s` — The string to repeat. [String](../../sql-reference/data-types/string.md).
- `n` — The number of times to repeat the string. [UInt or Int](../../sql-reference/data-types/int-uint.md).
**Returned value**
2023-04-19 20:04:59 +00:00
The single string containing string `s` repeated `n` times. If `n` \< 1, the function returns empty string.
Type: `String`.
**Example**
2020-03-20 10:10:48 +00:00
``` sql
SELECT repeat('abc', 10);
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─repeat('abc', 10)──────────────┐
│ abcabcabcabcabcabcabcabcabcabc │
└────────────────────────────────┘
```
2022-06-02 10:55:18 +00:00
## reverse
2017-04-03 19:49:50 +00:00
2023-04-19 20:04:59 +00:00
Reverses the sequence of bytes in a string.
2017-04-03 19:49:50 +00:00
2022-06-02 10:55:18 +00:00
## reverseUTF8
2023-04-19 20:04:59 +00:00
Reverses a sequence of Unicode code points in a string. Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
## format
2017-04-03 19:49:50 +00:00
2023-04-19 20:04:59 +00:00
Format the `pattern` string with the strings listed in the arguments, similar to formatting in Python. The pattern string can contain replacement fields surrounded by curly braces `{}`. Anything not contained in braces is considered literal text and copied verbatim into the output. Literal brace character can be escaped by two braces: `{{ '{{' }}` and `{{ '}}' }}`. Field names can be numbers (starting from zero) or empty (then they are implicitely given monotonically increasing numbers).
**Syntax**
```sql
format(pattern, s0, s1, …)
```
2019-05-18 11:30:36 +00:00
2023-04-19 20:04:59 +00:00
**Example**
2019-05-18 11:30:36 +00:00
2020-03-20 10:10:48 +00:00
``` sql
2019-05-18 11:30:36 +00:00
SELECT format('{1} {0} {1}', 'World', 'Hello')
DOCAPI-8530: Code blocks markup fix (#7060) * Typo fix. * Links fix. * Fixed links in docs. * More fixes. * docs/en: cleaning some files * docs/en: cleaning data_types * docs/en: cleaning database_engines * docs/en: cleaning development * docs/en: cleaning getting_started * docs/en: cleaning interfaces * docs/en: cleaning operations * docs/en: cleaning query_lamguage * docs/en: cleaning en * docs/ru: cleaning data_types * docs/ru: cleaning index * docs/ru: cleaning database_engines * docs/ru: cleaning development * docs/ru: cleaning general * docs/ru: cleaning getting_started * docs/ru: cleaning interfaces * docs/ru: cleaning operations * docs/ru: cleaning query_language * docs: cleaning interfaces/http * Update docs/en/data_types/array.md decorated ``` Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/getting_started/example_datasets/nyc_taxi.md fixed typo Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/getting_started/example_datasets/ontime.md fixed typo Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/interfaces/formats.md fixed error Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/table_engines/custom_partitioning_key.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/utils/clickhouse-local.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/dicts/external_dicts_dict_sources.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/utils/clickhouse-local.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/json_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/json_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/other_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/other_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/date_time_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/table_engines/jdbc.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * docs: fixed error * docs: fixed error
2019-09-23 15:31:46 +00:00
```
2020-03-20 10:10:48 +00:00
2023-04-19 20:04:59 +00:00
```result
2019-05-18 11:30:36 +00:00
┌─format('{1} {0} {1}', 'World', 'Hello')─┐
│ Hello World Hello │
└─────────────────────────────────────────┘
DOCAPI-8530: Code blocks markup fix (#7060) * Typo fix. * Links fix. * Fixed links in docs. * More fixes. * docs/en: cleaning some files * docs/en: cleaning data_types * docs/en: cleaning database_engines * docs/en: cleaning development * docs/en: cleaning getting_started * docs/en: cleaning interfaces * docs/en: cleaning operations * docs/en: cleaning query_lamguage * docs/en: cleaning en * docs/ru: cleaning data_types * docs/ru: cleaning index * docs/ru: cleaning database_engines * docs/ru: cleaning development * docs/ru: cleaning general * docs/ru: cleaning getting_started * docs/ru: cleaning interfaces * docs/ru: cleaning operations * docs/ru: cleaning query_language * docs: cleaning interfaces/http * Update docs/en/data_types/array.md decorated ``` Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/getting_started/example_datasets/nyc_taxi.md fixed typo Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/getting_started/example_datasets/ontime.md fixed typo Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/interfaces/formats.md fixed error Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/table_engines/custom_partitioning_key.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/utils/clickhouse-local.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/dicts/external_dicts_dict_sources.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/utils/clickhouse-local.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/json_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/json_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/other_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/other_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/date_time_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/table_engines/jdbc.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * docs: fixed error * docs: fixed error
2019-09-23 15:31:46 +00:00
```
2020-03-20 10:10:48 +00:00
2023-04-19 20:04:59 +00:00
With implicit numbers:
2020-03-20 10:10:48 +00:00
``` sql
2019-05-18 11:30:36 +00:00
SELECT format('{} {}', 'Hello', 'World')
DOCAPI-8530: Code blocks markup fix (#7060) * Typo fix. * Links fix. * Fixed links in docs. * More fixes. * docs/en: cleaning some files * docs/en: cleaning data_types * docs/en: cleaning database_engines * docs/en: cleaning development * docs/en: cleaning getting_started * docs/en: cleaning interfaces * docs/en: cleaning operations * docs/en: cleaning query_lamguage * docs/en: cleaning en * docs/ru: cleaning data_types * docs/ru: cleaning index * docs/ru: cleaning database_engines * docs/ru: cleaning development * docs/ru: cleaning general * docs/ru: cleaning getting_started * docs/ru: cleaning interfaces * docs/ru: cleaning operations * docs/ru: cleaning query_language * docs: cleaning interfaces/http * Update docs/en/data_types/array.md decorated ``` Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/getting_started/example_datasets/nyc_taxi.md fixed typo Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/getting_started/example_datasets/ontime.md fixed typo Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/interfaces/formats.md fixed error Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/table_engines/custom_partitioning_key.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/utils/clickhouse-local.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/dicts/external_dicts_dict_sources.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/utils/clickhouse-local.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/json_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/json_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/other_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/other_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/date_time_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/table_engines/jdbc.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * docs: fixed error * docs: fixed error
2019-09-23 15:31:46 +00:00
```
2020-03-20 10:10:48 +00:00
2023-04-19 20:04:59 +00:00
```result
2019-05-18 11:30:36 +00:00
┌─format('{} {}', 'Hello', 'World')─┐
│ Hello World │
└───────────────────────────────────┘
```
2022-06-02 10:55:18 +00:00
## concat
2023-04-19 20:04:59 +00:00
Concatenates the strings listed in the arguments without separator.
2020-03-20 10:10:48 +00:00
**Syntax**
2020-03-20 10:10:48 +00:00
``` sql
concat(s1, s2, ...)
```
**Arguments**
2020-02-02 21:59:23 +00:00
Values of type String or FixedString.
**Returned values**
2023-04-19 20:04:59 +00:00
The String created by concatenating the arguments.
2023-04-19 20:04:59 +00:00
If any of arguments is `NULL`, the function returns `NULL`.
**Example**
2020-03-20 10:10:48 +00:00
``` sql
SELECT concat('Hello, ', 'World!');
```
Result:
2023-04-19 20:04:59 +00:00
```result
2019-12-26 12:51:48 +00:00
┌─concat('Hello, ', 'World!')─┐
│ Hello, World! │
└─────────────────────────────┘
```
2022-06-02 10:55:18 +00:00
## concatAssumeInjective
2023-04-19 20:04:59 +00:00
Like [concat](#concat) but assumes that `concat(s1, s2, ...) → sn` is injective. Can be used for optimization of GROUP BY.
2023-04-19 20:04:59 +00:00
A function is called injective if it returns for different arguments different results. In other words: different arguments never produce identical result.
2020-03-20 10:10:48 +00:00
**Syntax**
2020-03-20 10:10:48 +00:00
``` sql
concatAssumeInjective(s1, s2, ...)
```
**Arguments**
Values of type String or FixedString.
**Returned values**
2023-04-19 20:04:59 +00:00
The String created by concatenating the arguments.
2023-04-19 20:04:59 +00:00
If any of argument values is `NULL`, the function returns `NULL`.
**Example**
2019-12-26 12:51:48 +00:00
Input table:
2020-03-20 10:10:48 +00:00
``` sql
2020-02-02 21:59:23 +00:00
CREATE TABLE key_val(`key1` String, `key2` String, `value` UInt32) ENGINE = TinyLog;
INSERT INTO key_val VALUES ('Hello, ','World',1), ('Hello, ','World',2), ('Hello, ','World!',3), ('Hello',', World!',2);
SELECT * from key_val;
2019-12-26 12:51:48 +00:00
```
2023-04-19 20:04:59 +00:00
```result
2019-12-26 12:51:48 +00:00
┌─key1────┬─key2─────┬─value─┐
│ Hello, │ World │ 1 │
│ Hello, │ World │ 2 │
│ Hello, │ World! │ 3 │
│ Hello │ , World! │ 2 │
└─────────┴──────────┴───────┘
```
2020-03-20 10:10:48 +00:00
``` sql
SELECT concat(key1, key2), sum(value) FROM key_val GROUP BY concatAssumeInjective(key1, key2);
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─concat(key1, key2)─┬─sum(value)─┐
│ Hello, World! │ 3 │
│ Hello, World! │ 2 │
│ Hello, World │ 3 │
└────────────────────┴────────────┘
```
2023-04-19 20:04:59 +00:00
## concatWithSeparator
Concatenates the given strings with a given separator.
2017-04-03 19:49:50 +00:00
2023-04-19 20:04:59 +00:00
**Syntax**
``` sql
concatWithSeparator(sep, expr1, expr2, expr3...)
```
2017-04-03 19:49:50 +00:00
2023-04-19 20:04:59 +00:00
**Arguments**
2023-04-19 20:04:59 +00:00
- sep — separator. Const [String](../../sql-reference/data-types/string.md) or [FixedString](../../sql-reference/data-types/fixedstring.md).
- exprN — expression to be concatenated. [String](../../sql-reference/data-types/string.md) or [FixedString](../../sql-reference/data-types/fixedstring.md).
2017-04-03 19:49:50 +00:00
2023-04-19 20:04:59 +00:00
**Returned values**
2023-04-19 20:04:59 +00:00
The String created by concatenating the arguments.
2023-04-19 20:04:59 +00:00
If any of the argument values is `NULL`, the function returns `NULL`.
2023-04-19 20:04:59 +00:00
**Example**
2017-04-03 19:49:50 +00:00
2023-04-19 20:04:59 +00:00
``` sql
SELECT concatWithSeparator('a', '1', '2', '3', '4')
```
2022-06-22 09:56:37 +00:00
2023-04-19 20:04:59 +00:00
Result:
```result
┌─concatWithSeparator('a', '1', '2', '3', '4')─┐
│ 1a2a3a4 │
└───────────────────────────────────┘
```
## concatWithSeparatorAssumeInjective
Like `concatWithSeparator` but assumes that `concatWithSeparator(sep, expr1, expr2, expr3...) → result` is injective. Can be used for optimization of GROUP BY.
A function is called injective if it returns for different arguments different results. In other words: different arguments never produce identical result.
## substring(s, offset, length)
Returns a substring with `length` many bytes, starting at the byte at index `offset`. Character indexing starts from 1.
**Syntax**
```sql
substring(s, offset, length)
```
Alias:
- `substr`
- `mid`
## substringUTF8
Like `substring` but for Unicode code points. Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
## appendTrailingCharIfAbsent
Appends character `c` to string `s` if `s` is non-empty and does not end with character `c`.
**Syntax**
```sql
appendTrailingCharIfAbsent(s, c)
```
## convertCharset
Returns string `s` converted from the encoding `from` to encoding `to`.
**Syntax**
```sql
convertCharset(s, from, to)
```
## base58Encode
Encodes a String using [Base58](https://tools.ietf.org/id/draft-msporny-base58-01.html) in the "Bitcoin" alphabet.
2022-06-22 09:56:37 +00:00
**Syntax**
```sql
2022-08-30 14:08:23 +00:00
base58Encode(plaintext)
2022-06-22 09:56:37 +00:00
```
**Arguments**
2022-08-30 14:08:23 +00:00
- `plaintext` — [String](../../sql-reference/data-types/string.md) column or constant.
2022-06-22 09:56:37 +00:00
**Returned value**
2023-04-19 20:04:59 +00:00
- A string containing the encoded value of the argument.
2022-06-22 09:56:37 +00:00
Type: [String](../../sql-reference/data-types/string.md).
**Example**
``` sql
2022-07-16 17:45:33 +00:00
SELECT base58Encode('Encoded');
2022-08-30 14:08:23 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
2022-08-30 14:08:23 +00:00
┌─base58Encode('Encoded')─┐
│ 3dc8KtHrwM │
└─────────────────────────┘
```
2023-04-19 20:04:59 +00:00
## base58Decode
2022-08-30 14:08:23 +00:00
Accepts a String and decodes it using [Base58](https://tools.ietf.org/id/draft-msporny-base58-01.html) encoding scheme using "Bitcoin" alphabet.
**Syntax**
```sql
2023-04-19 20:04:59 +00:00
base58Decode(encoded)
2022-08-30 14:08:23 +00:00
```
**Arguments**
2023-04-19 20:04:59 +00:00
- `encoded` — [String](../../sql-reference/data-types/string.md) column or constant. If the string is not a valid Base58-encoded value, an exception is thrown.
2022-08-30 14:08:23 +00:00
**Returned value**
2023-04-19 20:04:59 +00:00
- A string containing the decoded value of the argument.
2022-08-30 14:08:23 +00:00
Type: [String](../../sql-reference/data-types/string.md).
**Example**
``` sql
2022-08-30 11:40:26 +00:00
SELECT base58Decode('3dc8KtHrwM');
2022-06-22 09:56:37 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
2022-08-30 14:08:23 +00:00
┌─base58Decode('3dc8KtHrwM')─┐
│ Encoded │
└────────────────────────────┘
2022-06-22 09:56:37 +00:00
```
2023-04-19 20:04:59 +00:00
## tryBase58Decode
2023-04-19 20:04:59 +00:00
Like `base58Decode` but returns an empty string in case of error.
2023-04-19 20:04:59 +00:00
## base64Encode
2023-04-19 20:04:59 +00:00
Encodes a String or FixedString as base64.
2018-11-02 19:06:05 +00:00
2021-02-16 11:13:01 +00:00
Alias: `TO_BASE64`.
2023-04-19 20:04:59 +00:00
## base64Decode
2023-04-19 20:04:59 +00:00
Decodes a base64-encoded String or FixedString. Throws an exception in case of error.
2018-11-02 19:06:05 +00:00
2021-02-16 11:13:01 +00:00
Alias: `FROM_BASE64`.
2023-04-19 20:04:59 +00:00
## tryBase64Decode
2023-04-19 20:04:59 +00:00
Like `base64Decode` but returns an empty string in case of error.
2023-04-19 20:04:59 +00:00
## endsWith
2023-04-19 20:04:59 +00:00
Returns whether string `str` ends with `suffix`.
2023-04-19 20:04:59 +00:00
**Syntax**
2019-09-26 11:39:06 +00:00
2023-04-19 20:04:59 +00:00
```sql
endsWith(str, suffix)
2019-09-26 11:39:06 +00:00
```
2023-04-19 20:04:59 +00:00
## startsWith
2019-09-26 11:39:06 +00:00
2023-04-19 20:04:59 +00:00
Returns whether string `str` starts with `prefix`.
2019-09-26 11:39:06 +00:00
2023-04-19 20:04:59 +00:00
**Syntax**
2019-09-26 11:39:06 +00:00
2023-04-19 20:04:59 +00:00
```sql
startsWith(str, prefix)
2019-09-26 11:39:06 +00:00
```
2023-04-19 20:04:59 +00:00
**Example**
2019-09-26 11:39:06 +00:00
2023-04-19 20:04:59 +00:00
``` sql
SELECT startsWith('Spider-Man', 'Spi');
2019-09-26 11:39:06 +00:00
```
2022-06-02 10:55:18 +00:00
## trim
2023-04-19 20:04:59 +00:00
Removes the specified characters from the start or end of a string. If not specified otherwise, the function removes whitespace (ASCII-character 32).
**Syntax**
2020-03-20 10:10:48 +00:00
``` sql
trim([[LEADING|TRAILING|BOTH] trim_character FROM] input_string)
```
**Arguments**
- `trim_character` — Specified characters for trim. [String](../../sql-reference/data-types/string.md).
- `input_string` — String for trim. [String](../../sql-reference/data-types/string.md).
**Returned value**
2023-04-19 20:04:59 +00:00
A string without leading and/or trailing specified characters.
Type: `String`.
**Example**
2020-03-20 10:10:48 +00:00
``` sql
SELECT trim(BOTH ' ()' FROM '( Hello, world! )');
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─trim(BOTH ' ()' FROM '( Hello, world! )')─┐
│ Hello, world! │
└───────────────────────────────────────────────┘
```
2022-06-02 10:55:18 +00:00
## trimLeft
2023-04-19 20:04:59 +00:00
Removes the consecutive occurrences of whitespace (ASCII-character 32) from the start of a string.
2020-03-20 10:10:48 +00:00
**Syntax**
2020-03-20 10:10:48 +00:00
``` sql
trimLeft(input_string)
```
Alias: `ltrim(input_string)`.
**Arguments**
- `input_string` — string to trim. [String](../../sql-reference/data-types/string.md).
**Returned value**
A string without leading common whitespaces.
Type: `String`.
**Example**
2020-03-20 10:10:48 +00:00
``` sql
SELECT trimLeft(' Hello, world! ');
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─trimLeft(' Hello, world! ')─┐
│ Hello, world! │
└─────────────────────────────────────┘
```
2022-06-02 10:55:18 +00:00
## trimRight
2023-04-19 20:04:59 +00:00
Removes the consecutive occurrences of whitespace (ASCII-character 32) from the end of a string.
2020-03-20 10:10:48 +00:00
**Syntax**
2020-03-20 10:10:48 +00:00
``` sql
trimRight(input_string)
```
Alias: `rtrim(input_string)`.
**Arguments**
- `input_string` — string to trim. [String](../../sql-reference/data-types/string.md).
**Returned value**
A string without trailing common whitespaces.
Type: `String`.
**Example**
2020-03-20 10:10:48 +00:00
``` sql
SELECT trimRight(' Hello, world! ');
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─trimRight(' Hello, world! ')─┐
│ Hello, world! │
└──────────────────────────────────────┘
```
2022-06-02 10:55:18 +00:00
## trimBoth
2023-04-19 20:04:59 +00:00
Removes the consecutive occurrences of whitespace (ASCII-character 32) from both ends of a string.
2020-03-20 10:10:48 +00:00
**Syntax**
2020-03-20 10:10:48 +00:00
``` sql
trimBoth(input_string)
```
Alias: `trim(input_string)`.
**Arguments**
- `input_string` — string to trim. [String](../../sql-reference/data-types/string.md).
**Returned value**
A string without leading and trailing common whitespaces.
Type: `String`.
**Example**
2020-03-20 10:10:48 +00:00
``` sql
SELECT trimBoth(' Hello, world! ');
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─trimBoth(' Hello, world! ')─┐
│ Hello, world! │
└─────────────────────────────────────┘
```
2023-04-19 20:04:59 +00:00
## CRC32
2019-06-17 21:49:37 +00:00
2023-04-19 20:04:59 +00:00
Returns the CRC32 checksum of a string using CRC-32-IEEE 802.3 polynomial and initial value `0xffffffff` (zlib implementation).
The result type is UInt32.
2023-04-19 20:04:59 +00:00
## CRC32IEEE
Returns the CRC32 checksum of a string, using CRC-32-IEEE 802.3 polynomial.
2019-06-17 21:49:37 +00:00
The result type is UInt32.
2023-04-19 20:04:59 +00:00
## CRC64
Returns the CRC64 checksum of a string, using CRC-64-ECMA polynomial.
The result type is UInt64.
2022-06-02 10:55:18 +00:00
## normalizeQuery
Replaces literals, sequences of literals and complex aliases with placeholders.
**Syntax**
``` sql
normalizeQuery(x)
```
2021-07-29 15:20:55 +00:00
**Arguments**
- `x` — Sequence of characters. [String](../../sql-reference/data-types/string.md).
2020-09-28 20:58:08 +00:00
**Returned value**
- Sequence of characters with placeholders.
Type: [String](../../sql-reference/data-types/string.md).
**Example**
``` sql
SELECT normalizeQuery('[1, 2, 3, x]') AS query;
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─query────┐
│ [?.., x] │
└──────────┘
```
2022-06-02 10:55:18 +00:00
## normalizedQueryHash
2023-04-19 20:04:59 +00:00
Returns identical 64bit hash values without the values of literals for similar queries. Can be helpful to analyze query log.
2021-07-29 15:20:55 +00:00
**Syntax**
``` sql
normalizedQueryHash(x)
```
2021-07-29 15:20:55 +00:00
**Arguments**
- `x` — Sequence of characters. [String](../../sql-reference/data-types/string.md).
**Returned value**
- Hash value.
Type: [UInt64](../../sql-reference/data-types/int-uint.md#uint-ranges).
**Example**
``` sql
SELECT normalizedQueryHash('SELECT 1 AS `xyz`') != normalizedQueryHash('SELECT 1 AS `abc`') AS res;
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─res─┐
│ 1 │
└─────┘
```
2022-06-02 10:55:18 +00:00
## normalizeUTF8NFC
2021-10-19 22:31:39 +00:00
2023-04-19 20:04:59 +00:00
Converts a string to [NFC normalized form](https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms), assuming the string is valid UTF8-encoded text.
2021-10-19 22:31:39 +00:00
**Syntax**
``` sql
normalizeUTF8NFC(words)
2021-10-19 22:31:39 +00:00
```
**Arguments**
2023-04-19 20:04:59 +00:00
- `words` — UTF8-encoded input string. [String](../../sql-reference/data-types/string.md).
2021-10-19 22:31:39 +00:00
**Returned value**
- String transformed to NFC normalization form.
2021-10-19 22:31:39 +00:00
Type: [String](../../sql-reference/data-types/string.md).
2021-10-22 20:10:02 +00:00
**Example**
``` sql
SELECT length('â'), normalizeUTF8NFC('â') AS nfc, length(nfc) AS nfc_len;
2021-10-22 20:10:02 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─length('â')─┬─nfc─┬─nfc_len─┐
│ 2 │ â │ 2 │
2021-10-22 20:10:02 +00:00
└─────────────┴─────┴─────────┘
```
2022-06-02 10:55:18 +00:00
## normalizeUTF8NFD
2021-10-22 20:10:02 +00:00
2023-04-19 20:04:59 +00:00
Converts a string to [NFD normalized form](https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms), assuming the string is valid UTF8-encoded text.
2021-10-22 20:10:02 +00:00
**Syntax**
``` sql
normalizeUTF8NFD(words)
```
**Arguments**
2023-04-19 20:04:59 +00:00
- `words` — UTF8-encoded input string. [String](../../sql-reference/data-types/string.md).
2021-10-22 20:10:02 +00:00
**Returned value**
- String transformed to NFD normalization form.
2021-10-19 22:31:39 +00:00
2021-10-22 20:10:02 +00:00
Type: [String](../../sql-reference/data-types/string.md).
**Example**
``` sql
SELECT length('â'), normalizeUTF8NFD('â') AS nfd, length(nfd) AS nfd_len;
2021-10-22 20:10:02 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─length('â')─┬─nfd─┬─nfd_len─┐
│ 2 │ â │ 3 │
2021-10-22 20:10:02 +00:00
└─────────────┴─────┴─────────┘
```
2022-06-02 10:55:18 +00:00
## normalizeUTF8NFKC
2021-10-22 20:10:02 +00:00
2023-04-19 20:04:59 +00:00
Converts a string to [NFKC normalized form](https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms), assuming the string is valid UTF8-encoded text.
2021-10-22 20:10:02 +00:00
**Syntax**
``` sql
normalizeUTF8NFKC(words)
```
**Arguments**
2023-04-19 20:04:59 +00:00
- `words` — UTF8-encoded input string. [String](../../sql-reference/data-types/string.md).
2021-10-22 20:10:02 +00:00
**Returned value**
- String transformed to NFKC normalization form.
2021-10-22 20:10:02 +00:00
Type: [String](../../sql-reference/data-types/string.md).
**Example**
``` sql
SELECT length('â'), normalizeUTF8NFKC('â') AS nfkc, length(nfkc) AS nfkc_len;
2021-10-22 20:10:02 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─length('â')─┬─nfkc─┬─nfkc_len─┐
│ 2 │ â │ 2 │
2021-10-22 20:10:02 +00:00
└─────────────┴──────┴──────────┘
```
2022-06-02 10:55:18 +00:00
## normalizeUTF8NFKD
2021-10-22 20:10:02 +00:00
2023-04-19 20:04:59 +00:00
Converts a string to [NFKD normalized form](https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms), assuming the string is valid UTF8-encoded text.
2021-10-22 20:10:02 +00:00
**Syntax**
``` sql
normalizeUTF8NFKD(words)
```
**Arguments**
2023-04-19 20:04:59 +00:00
- `words` — UTF8-encoded input string. [String](../../sql-reference/data-types/string.md).
2021-10-22 20:10:02 +00:00
**Returned value**
- String transformed to NFKD normalization form.
2021-10-22 20:10:02 +00:00
Type: [String](../../sql-reference/data-types/string.md).
**Example**
``` sql
SELECT length('â'), normalizeUTF8NFKD('â') AS nfkd, length(nfkd) AS nfkd_len;
2021-10-22 20:10:02 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
┌─length('â')─┬─nfkd─┬─nfkd_len─┐
│ 2 │ â │ 3 │
2021-10-22 20:10:02 +00:00
└─────────────┴──────┴──────────┘
```
2021-10-19 22:31:39 +00:00
2022-06-02 10:55:18 +00:00
## encodeXMLComponent
2020-12-27 11:21:58 +00:00
2023-04-19 20:04:59 +00:00
Escapes characters with special meaning in XML such that they can afterwards be place into a XML text node or attribute.
2020-12-27 11:21:58 +00:00
2023-04-19 20:04:59 +00:00
The following characters are replaced: `<`, `&`, `>`, `"`, `'`.
Also see the [list of XML and HTML character entity references](https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references).
2020-12-27 20:49:05 +00:00
2021-07-29 15:20:55 +00:00
**Syntax**
2020-12-27 11:21:58 +00:00
``` sql
encodeXMLComponent(x)
```
2021-07-29 15:20:55 +00:00
**Arguments**
2020-12-27 11:21:58 +00:00
2023-04-19 20:04:59 +00:00
- `x` — An input string. [String](../../sql-reference/data-types/string.md).
2020-12-27 11:21:58 +00:00
**Returned value**
2020-12-27 11:21:58 +00:00
2023-04-19 20:04:59 +00:00
- The escaped string.
2020-12-27 11:21:58 +00:00
Type: [String](../../sql-reference/data-types/string.md).
**Example**
``` sql
2020-12-27 20:49:05 +00:00
SELECT encodeXMLComponent('Hello, "world"!');
SELECT encodeXMLComponent('<123>');
SELECT encodeXMLComponent('&clickhouse');
SELECT encodeXMLComponent('\'foo\'');
2020-12-27 11:21:58 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
2020-12-27 20:49:05 +00:00
Hello, &quot;world&quot;!
&lt;123&gt;
&amp;clickhouse
&apos;foo&apos;
2020-12-27 11:21:58 +00:00
```
2022-06-02 10:55:18 +00:00
## decodeXMLComponent
2021-02-12 19:28:03 +00:00
2023-04-19 20:04:59 +00:00
Un-escapes substrings with special meaning in XML. These substrings are: `&quot;` `&amp;` `&apos;` `&gt;` `&lt;`
2021-02-15 18:25:32 +00:00
This function also replaces numeric character references with Unicode characters. Both decimal (like `&#10003;`) and hexadecimal (`&#x2713;`) forms are supported.
2021-02-12 19:28:03 +00:00
**Syntax**
``` sql
decodeXMLComponent(x)
```
**Arguments**
2021-02-12 19:28:03 +00:00
2023-04-19 20:04:59 +00:00
- `x` — An input string. [String](../../sql-reference/data-types/string.md).
2021-02-12 19:28:03 +00:00
**Returned value**
2023-04-19 20:04:59 +00:00
- The un-escaped string.
2021-02-12 19:28:03 +00:00
Type: [String](../../sql-reference/data-types/string.md).
**Example**
``` sql
SELECT decodeXMLComponent('&apos;foo&apos;');
SELECT decodeXMLComponent('&lt; &#x3A3; &gt;');
```
Result:
2023-04-19 20:04:59 +00:00
```result
2021-07-29 15:20:55 +00:00
'foo'
2021-02-12 19:28:03 +00:00
< Σ >
```
2022-06-02 10:55:18 +00:00
## extractTextFromHTML
2021-03-28 18:42:04 +00:00
2023-04-19 20:04:59 +00:00
This function extracts plain text from HTML or XHTML.
2023-04-19 20:04:59 +00:00
It does not conform 100% to the HTML, XML or XHTML specification but the implementation is reasonably accurate and fast. The rules are the following:
1. Comments are skipped. Example: `<!-- test -->`. Comment must end with `-->`. Nested comments are disallowed.
2021-04-07 18:35:11 +00:00
Note: constructions like `<!-->` and `<!--->` are not valid comments in HTML but they are skipped by other rules.
2023-04-19 20:04:59 +00:00
2. CDATA is pasted verbatim. Note: CDATA is XML/XHTML-specific and processed on a "best-effort" basis.
2021-04-07 18:35:11 +00:00
3. `script` and `style` elements are removed with all their content. Note: it is assumed that closing tag cannot appear inside content. For example, in JS string literal has to be escaped like `"<\/script>"`.
2021-04-07 19:23:53 +00:00
Note: comments and CDATA are possible inside `script` or `style` - then closing tags are not searched inside CDATA. Example: `<script><![CDATA[</script>]]></script>`. But they are still searched inside comments. Sometimes it becomes complicated: `<script>var x = "<!--"; </script> var y = "-->"; alert(x + y);</script>`
2021-04-06 19:10:22 +00:00
Note: `script` and `style` can be the names of XML namespaces - then they are not treated like usual `script` or `style` elements. Example: `<script:a>Hello</script:a>`.
Note: whitespaces are possible after closing tag name: `</script >` but not before: `< / script>`.
4. Other tags or tag-like elements are skipped without inner content. Example: `<a>.</a>`
2021-04-07 18:35:11 +00:00
Note: it is expected that this HTML is illegal: `<a test=">"></a>`
Note: it also skips something like tags: `<>`, `<!>`, etc.
Note: tag without end is skipped to the end of input: `<hello `
2021-04-06 19:10:22 +00:00
5. HTML and XML entities are not decoded. They must be processed by separate function.
6. Whitespaces in the text are collapsed or inserted by specific rules.
- Whitespaces at the beginning and at the end are removed.
- Consecutive whitespaces are collapsed.
- But if the text is separated by other elements and there is no whitespace, it is inserted.
2021-04-07 18:35:11 +00:00
- It may cause unnatural examples: `Hello<b>world</b>`, `Hello<!-- -->world` - there is no whitespace in HTML, but the function inserts it. Also consider: `Hello<p>world</p>`, `Hello<br>world`. This behavior is reasonable for data analysis, e.g. to convert HTML to a bag of words.
7. Also note that correct handling of whitespaces requires the support of `<pre></pre>` and CSS `display` and `white-space` properties.
2021-03-28 18:42:04 +00:00
**Syntax**
``` sql
extractTextFromHTML(x)
```
2021-04-03 16:16:56 +00:00
**Arguments**
2021-03-28 18:42:04 +00:00
- `x` — input text. [String](../../sql-reference/data-types/string.md).
2021-03-28 18:42:04 +00:00
**Returned value**
- Extracted text.
2021-03-28 18:42:04 +00:00
Type: [String](../../sql-reference/data-types/string.md).
**Example**
2021-04-03 19:31:24 +00:00
The first example contains several tags and a comment and also shows whitespace processing.
The second example shows `CDATA` and `script` tag processing.
2021-04-07 18:35:11 +00:00
In the third example text is extracted from the full HTML response received by the [url](../../sql-reference/table-functions/url.md) function.
2021-04-03 16:16:56 +00:00
2021-03-28 18:42:04 +00:00
``` sql
SELECT extractTextFromHTML(' <p> A text <i>with</i><b>tags</b>. <!-- comments --> </p> ');
2021-04-03 16:16:56 +00:00
SELECT extractTextFromHTML('<![CDATA[The content within <b>CDATA</b>]]> <script>alert("Script");</script>');
2021-04-05 19:37:01 +00:00
SELECT extractTextFromHTML(html) FROM url('http://www.donothingfor2minutes.com/', RawBLOB, 'html String');
2021-03-28 18:42:04 +00:00
```
Result:
2023-04-19 20:04:59 +00:00
```result
A text with tags .
2021-04-03 16:16:56 +00:00
The content within <b>CDATA</b>
2021-04-05 19:37:01 +00:00
Do Nothing for 2 Minutes 2:00 &nbsp;
2021-04-03 16:16:56 +00:00
```
2022-10-25 04:07:07 +00:00
2023-04-19 20:04:59 +00:00
## ascii {#ascii}
2022-10-25 04:07:07 +00:00
2023-04-19 20:04:59 +00:00
Returns the ASCII code point (as Int32) of the first character of string `s`.
2022-10-25 04:07:07 +00:00
2023-04-19 20:04:59 +00:00
If `s` is empty, the result is 0. If the first character is not an ASCII character or not part of the Latin-1 supplement range of UTF-16, the result is undefined.
2022-12-01 02:56:53 +00:00
**Syntax**
2023-04-19 20:04:59 +00:00
```sql
ascii(s)
2022-12-01 02:56:53 +00:00
```
2023-04-09 09:00:20 +00:00
## soundex
Returns the [Soundex code](https://en.wikipedia.org/wiki/Soundex) of a string.
2023-04-09 09:00:20 +00:00
**Syntax**
``` sql
soundex(val)
2023-04-09 09:00:20 +00:00
```
**Arguments**
- `val` - Input value. [String](../data-types/string.md)
2023-04-09 09:00:20 +00:00
**Returned value**
- The Soundex code of the input value. [String](../data-types/string.md)
2023-04-09 09:00:20 +00:00
**Example**
``` sql
select soundex('aksel');
```
Result:
2023-04-19 20:04:59 +00:00
```result
2023-04-09 09:00:20 +00:00
┌─soundex('aksel')─┐
│ A240 │
└──────────────────┘
```
## extractKeyValuePairs
2023-04-05 13:34:39 +00:00
Extracts key-value pairs from any string. The string does not need to be 100% structured in a key value pair format;
2023-04-05 13:27:42 +00:00
It can contain noise (e.g. log files). The key-value pair format to be interpreted should be specified via function arguments.
2022-12-14 18:15:06 +00:00
2023-04-05 13:27:42 +00:00
A key-value pair consists of a key followed by a `key_value_delimiter` and a value. Quoted keys and values are also supported. Key value pairs must be separated by pair delimiters.
2022-12-14 18:15:06 +00:00
**Syntax**
``` sql
extractKeyValuePairs(data, [key_value_delimiter], [pair_delimiter], [quoting_character])
```
**Arguments**
2023-04-05 13:27:42 +00:00
- `data` - String to extract key-value pairs from. [String](../../sql-reference/data-types/string.md) or [FixedString](../../sql-reference/data-types/fixedstring.md).
- `key_value_delimiter` - Character to be used as delimiter between the key and the value. Defaults to `:`. [String](../../sql-reference/data-types/string.md) or [FixedString](../../sql-reference/data-types/fixedstring.md).
- `pair_delimiters` - Set of character to be used as delimiters between pairs. Defaults to `\space`, `,` and `;`. [String](../../sql-reference/data-types/string.md) or [FixedString](../../sql-reference/data-types/fixedstring.md).
- `quoting_character` - Character to be used as quoting character. Defaults to `"`. [String](../../sql-reference/data-types/string.md) or [FixedString](../../sql-reference/data-types/fixedstring.md).
**Returned values**
2023-04-05 13:27:42 +00:00
- The extracted key-value pairs in a Map(String, String).
2023-04-05 13:27:42 +00:00
**Examples**
Query:
2023-04-05 13:27:42 +00:00
**Simple case**
``` sql
2023-04-05 13:27:42 +00:00
arthur :) select extractKeyValuePairs('name:neymar, age:31 team:psg,nationality:brazil') as kv
SELECT extractKeyValuePairs('name:neymar, age:31 team:psg,nationality:brazil') as kv
Query id: f9e0ca6f-3178-4ee2-aa2c-a5517abb9cee
┌─kv──────────────────────────────────────────────────────────────────────┐
│ {'name':'neymar','age':'31','team':'psg','nationality':'brazil'} │
└─────────────────────────────────────────────────────────────────────────┘
```
2023-04-05 13:27:42 +00:00
**Single quote as quoting character**
``` sql
arthur :) select extractKeyValuePairs('name:\'neymar\';\'age\':31;team:psg;nationality:brazil,last_key:last_value', ':', ';,', '\'') as kv
SELECT extractKeyValuePairs('name:\'neymar\';\'age\':31;team:psg;nationality:brazil,last_key:last_value', ':', ';,', '\'') as kv
2023-04-05 13:27:42 +00:00
Query id: 0e22bf6b-9844-414a-99dc-32bf647abd5e
┌─kv───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ {'name':'neymar','age':'31','team':'psg','nationality':'brazil','last_key':'last_value'} │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```
**Escape sequences without escape sequences support**
``` sql
arthur :) select extractKeyValuePairs('age:a\\x0A\\n\\0') as kv
2023-04-05 13:27:42 +00:00
SELECT extractKeyValuePairs('age:a\\x0A\\n\\0') AS kv
2023-04-05 13:27:42 +00:00
Query id: e9fd26ee-b41f-4a11-b17f-25af6fd5d356
2023-04-05 13:27:42 +00:00
┌─kv─────────────────────┐
│ {'age':'a\\x0A\\n\\0'} │
└────────────────────────┘
2023-04-05 13:27:42 +00:00
```
## extractKeyValuePairsWithEscaping
Same as `extractKeyValuePairs` but with escaping support.
Escape sequences supported: `\x`, `\N`, `\a`, `\b`, `\e`, `\f`, `\n`, `\r`, `\t`, `\v` and `\0`.
Non standard escape sequences are returned as it is (including the backslash) unless they are one of the following:
2023-04-10 13:31:26 +00:00
`\\`, `'`, `"`, `backtick`, `/`, `=` or ASCII control characters (c <= 31).
2023-04-21 14:43:12 +00:00
This function will satisfy the use case where pre-escaping and post-escaping are not suitable. For instance, consider the following
input string: `a: "aaaa\"bbb"`. The expected output is: `a: aaaa\"bbbb`.
- Pre-escaping: Pre-escaping it will output: `a: "aaaa"bbb"` and `extractKeyValuePairs` will then output: `a: aaaa`
- Post-escaping: `extractKeyValuePairs` will output `a: aaaa\` and post-escaping will keep it as it is.
2023-04-17 14:20:03 +00:00
Leading escape sequences will be skipped in keys and will be considered invalid for values.
2023-04-05 13:27:42 +00:00
**Escape sequences with escape sequence support turned on**
``` sql
arthur :) select extractKeyValuePairsWithEscaping('age:a\\x0A\\n\\0') as kv
2023-04-05 13:27:42 +00:00
SELECT extractKeyValuePairsWithEscaping('age:a\\x0A\\n\\0') AS kv
2023-04-05 13:27:42 +00:00
Query id: 44c114f0-5658-4c75-ab87-4574de3a1645
2023-04-05 13:27:42 +00:00
┌─kv────────────────┐
│ {'age':'a\n\n\0'} │
└───────────────────┘
```