Add function substring_index, as in spark or mysql
33 KiB
slug | sidebar_position | sidebar_label |
---|---|---|
/en/sql-reference/functions/string-functions | 170 | Strings |
Functions for Working with Strings
Functions for searching in strings and for replacing in strings are described separately.
empty
Checks whether the input string is empty.
A string is considered non-empty if it contains at least one byte, even if this byte is a space or the null byte.
The function is also available for arrays and UUIDs.
Syntax
empty(x)
Arguments
x
— Input value. String.
Returned value
- Returns
1
for an empty string or0
for a non-empty string.
Type: UInt8.
Example
SELECT empty('');
Result:
┌─empty('')─┐
│ 1 │
└───────────┘
notEmpty
Checks whether the input string is non-empty.
A string is considered non-empty if it contains at least one byte, even if this byte is a space or the null byte.
The function is also available for arrays and UUIDs.
Syntax
notEmpty(x)
Arguments
x
— Input value. String.
Returned value
- Returns
1
for a non-empty string or0
for an empty string string.
Type: UInt8.
Example
SELECT notEmpty('text');
Result:
┌─notEmpty('text')─┐
│ 1 │
└──────────────────┘
length
Returns the length of a string in bytes (not: in characters or Unicode code points).
The function also works for arrays.
Alias: OCTET_LENGTH
lengthUTF8
Returns the length of a string in Unicode code points (not: in bytes or characters). It assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Alias:
- `CHAR_LENGTH``
CHARACTER_LENGTH
leftPad
Pads a string from the left with spaces or with a specified string (multiple times, if needed) until the resulting string reaches the specified length
.
Syntax
leftPad(string, length[, pad_string])
Alias: LPAD
Arguments
string
— Input string that should be padded. String.length
— The length of the resulting string. UInt or Int. If the value is smaller than the input string length, then the input string is shortened tolength
characters.pad_string
— The string to pad the input string with. String. Optional. If not specified, then the input string is padded with spaces.
Returned value
- A left-padded string of the given length.
Type: String.
Example
SELECT leftPad('abc', 7, '*'), leftPad('def', 7);
Result:
┌─leftPad('abc', 7, '*')─┬─leftPad('def', 7)─┐
│ ****abc │ def │
└────────────────────────┴───────────────────┘
leftPadUTF8
Pads the string from the left with spaces or a specified string (multiple times, if needed) until the resulting string reaches the given length. Unlike leftPad which measures the string length in bytes, the string length is measured in code points.
Syntax
leftPadUTF8(string, length[, pad_string])
Arguments
string
— Input string that should be padded. String.length
— The length of the resulting string. UInt or Int. If the value is smaller than the input string length, then the input string is shortened tolength
characters.pad_string
— The string to pad the input string with. String. Optional. If not specified, then the input string is padded with spaces.
Returned value
- A left-padded string of the given length.
Type: String.
Example
SELECT leftPadUTF8('абвг', 7, '*'), leftPadUTF8('дежз', 7);
Result:
┌─leftPadUTF8('абвг', 7, '*')─┬─leftPadUTF8('дежз', 7)─┐
│ ***абвг │ дежз │
└─────────────────────────────┴────────────────────────┘
rightPad
Pads a string from the right with spaces or with a specified string (multiple times, if needed) until the resulting string reaches the specified length
.
Syntax
rightPad(string, length[, pad_string])
Alias: RPAD
Arguments
string
— Input string that should be padded. String.length
— The length of the resulting string. UInt or Int. If the value is smaller than the input string length, then the input string is shortened tolength
characters.pad_string
— The string to pad the input string with. String. Optional. If not specified, then the input string is padded with spaces.
Returned value
- A left-padded string of the given length.
Type: String.
Example
SELECT rightPad('abc', 7, '*'), rightPad('abc', 7);
Result:
┌─rightPad('abc', 7, '*')─┬─rightPad('abc', 7)─┐
│ abc**** │ abc │
└─────────────────────────┴────────────────────┘
rightPadUTF8
Pads the string from the right with spaces or a specified string (multiple times, if needed) until the resulting string reaches the given length. Unlike rightPad which measures the string length in bytes, the string length is measured in code points.
Syntax
rightPadUTF8(string, length[, pad_string])
Arguments
string
— Input string that should be padded. String.length
— The length of the resulting string. UInt or Int. If the value is smaller than the input string length, then the input string is shortened tolength
characters.pad_string
— The string to pad the input string with. String. Optional. If not specified, then the input string is padded with spaces.
Returned value
- A right-padded string of the given length.
Type: String.
Example
SELECT rightPadUTF8('абвг', 7, '*'), rightPadUTF8('абвг', 7);
Result:
┌─rightPadUTF8('абвг', 7, '*')─┬─rightPadUTF8('абвг', 7)─┐
│ абвг*** │ абвг │
└──────────────────────────────┴─────────────────────────┘
lower
Converts the ASCII Latin symbols in a string to lowercase.
Alias: lcase
upper
Converts the ASCII Latin symbols in a string to uppercase.
Alias: ucase
lowerUTF8
Converts a string to lowercase, assuming that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Does not detect the language, e.g. for Turkish the result might not be exactly correct (i/İ vs. i/I).
If the length of the UTF-8 byte sequence is different for upper and lower case of a code point, the result may be incorrect for this code point.
upperUTF8
Converts a string to uppercase, assuming that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Does not detect the language, e.g. for Turkish the result might not be exactly correct (i/İ vs. i/I).
If the length of the UTF-8 byte sequence is different for upper and lower case of a code point, the result may be incorrect for this code point.
isValidUTF8
Returns 1, if the set of bytes constitutes valid UTF-8-encoded text, otherwise 0.
toValidUTF8
Replaces invalid UTF-8 characters by the <EFBFBD>
(U+FFFD) character. All running in a row invalid characters are collapsed into the one replacement character.
Syntax
toValidUTF8(input_string)
Arguments
input_string
— Any set of bytes represented as the String data type object.
Returned value
- A valid UTF-8 string.
Example
SELECT toValidUTF8('\x61\xF0\x80\x80\x80b');
┌─toValidUTF8('a<><61><EFBFBD><EFBFBD>b')─┐
│ a<>b │
└───────────────────────┘
repeat
Concatenates a string as many times with itself as specified.
Syntax
repeat(s, n)
Alias: REPEAT
Arguments
s
— The string to repeat. String.n
— The number of times to repeat the string. UInt* or Int*.
Returned value
A string containing string s
repeated n
times. If n
<= 0, the function returns the empty string.
Type: String
.
Example
SELECT repeat('abc', 10);
Result:
┌─repeat('abc', 10)──────────────┐
│ abcabcabcabcabcabcabcabcabcabc │
└────────────────────────────────┘
space
Concatenates a space (
) as many times with itself as specified.
Syntax
space(n)
Alias: SPACE
.
Arguments
n
— The number of times to repeat the space. UInt* or Int*.
Returned value
The string containing string
repeated n
times. If n
<= 0, the function returns the empty string.
Type: String
.
Example
Query:
SELECT space(3);
Result:
┌─space(3) ────┐
│ │
└──────────────┘
reverse
Reverses the sequence of bytes in a string.
reverseUTF8
Reverses a sequence of Unicode code points in a string. Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
format
Format the pattern
string with the strings listed in the arguments, similar to formatting in Python. The pattern string can contain replacement fields surrounded by curly braces {}
. Anything not contained in braces is considered literal text and copied verbatim into the output. Literal brace character can be escaped by two braces: {{ '{{' }}
and {{ '}}' }}
. Field names can be numbers (starting from zero) or empty (then they are implicitly given monotonically increasing numbers).
Syntax
format(pattern, s0, s1, …)
Example
SELECT format('{1} {0} {1}', 'World', 'Hello')
┌─format('{1} {0} {1}', 'World', 'Hello')─┐
│ Hello World Hello │
└─────────────────────────────────────────┘
With implicit numbers:
SELECT format('{} {}', 'Hello', 'World')
┌─format('{} {}', 'Hello', 'World')─┐
│ Hello World │
└───────────────────────────────────┘
concat
Concatenates the strings listed in the arguments without separator.
Syntax
concat(s1, s2, ...)
Arguments
Values of type String or FixedString.
Returned values
The String created by concatenating the arguments.
If any of arguments is NULL
, the function returns NULL
.
Example
SELECT concat('Hello, ', 'World!');
Result:
┌─concat('Hello, ', 'World!')─┐
│ Hello, World! │
└─────────────────────────────┘
concatAssumeInjective
Like concat but assumes that concat(s1, s2, ...) → sn
is injective. Can be used for optimization of GROUP BY.
A function is called injective if it returns for different arguments different results. In other words: different arguments never produce identical result.
Syntax
concatAssumeInjective(s1, s2, ...)
Arguments
Values of type String or FixedString.
Returned values
The String created by concatenating the arguments.
If any of argument values is NULL
, the function returns NULL
.
Example
Input table:
CREATE TABLE key_val(`key1` String, `key2` String, `value` UInt32) ENGINE = TinyLog;
INSERT INTO key_val VALUES ('Hello, ','World',1), ('Hello, ','World',2), ('Hello, ','World!',3), ('Hello',', World!',2);
SELECT * from key_val;
┌─key1────┬─key2─────┬─value─┐
│ Hello, │ World │ 1 │
│ Hello, │ World │ 2 │
│ Hello, │ World! │ 3 │
│ Hello │ , World! │ 2 │
└─────────┴──────────┴───────┘
SELECT concat(key1, key2), sum(value) FROM key_val GROUP BY concatAssumeInjective(key1, key2);
Result:
┌─concat(key1, key2)─┬─sum(value)─┐
│ Hello, World! │ 3 │
│ Hello, World! │ 2 │
│ Hello, World │ 3 │
└────────────────────┴────────────┘
concatWithSeparator
Concatenates the given strings with a given separator.
Syntax
concatWithSeparator(sep, expr1, expr2, expr3...)
Arguments
- sep — separator. Const String or FixedString.
- exprN — expression to be concatenated. String or FixedString.
Returned values
The String created by concatenating the arguments.
If any of the argument values is NULL
, the function returns NULL
.
Example
SELECT concatWithSeparator('a', '1', '2', '3', '4')
Result:
┌─concatWithSeparator('a', '1', '2', '3', '4')─┐
│ 1a2a3a4 │
└───────────────────────────────────┘
concatWithSeparatorAssumeInjective
Like concatWithSeparator
but assumes that concatWithSeparator(sep, expr1, expr2, expr3...) → result
is injective. Can be used for optimization of GROUP BY.
A function is called injective if it returns for different arguments different results. In other words: different arguments never produce identical result.
substring(s, offset, length)
Returns a substring with length
many bytes, starting at the byte at index offset
. Character indexing starts from 1.
Syntax
substring(s, offset, length)
Alias:
substr
mid
substringUTF8
Like substring
but for Unicode code points. Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
substringIndex(s, delim, count)
Returns the substring of s
before count
occurrences of the delimiter delim
, as in Spark or MySQL.
Syntax
substringIndex(s, delim, count)
Alias: SUBSTRING_INDEX
Arguments
- s: The string to extract substring from. String.
- delim: The character to split. String.
- count: The number of occurrences of the delimiter to count before extracting the substring. If count is positive, everything to the left of the final delimiter (counting from the left) is returned. If count is negative, everything to the right of the final delimiter (counting from the right) is returned. UInt or Int
Example
SELECT substringIndex('www.clickhouse.com', '.', 2)
Result:
┌─substringIndex('www.clickhouse.com', '.', 2)─┐
│ www.clickhouse │
└──────────────────────────────────────────────┘
substringIndexUTF8(s, delim, count)
Like substringIndex
but for Unicode code points. Assumes that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
appendTrailingCharIfAbsent
Appends character c
to string s
if s
is non-empty and does not end with character c
.
Syntax
appendTrailingCharIfAbsent(s, c)
convertCharset
Returns string s
converted from the encoding from
to encoding to
.
Syntax
convertCharset(s, from, to)
base58Encode
Encodes a String using Base58 in the "Bitcoin" alphabet.
Syntax
base58Encode(plaintext)
Arguments
plaintext
— String column or constant.
Returned value
- A string containing the encoded value of the argument.
Type: String.
Example
SELECT base58Encode('Encoded');
Result:
┌─base58Encode('Encoded')─┐
│ 3dc8KtHrwM │
└─────────────────────────┘
base58Decode
Accepts a String and decodes it using Base58 encoding scheme using "Bitcoin" alphabet.
Syntax
base58Decode(encoded)
Arguments
encoded
— String column or constant. If the string is not a valid Base58-encoded value, an exception is thrown.
Returned value
- A string containing the decoded value of the argument.
Type: String.
Example
SELECT base58Decode('3dc8KtHrwM');
Result:
┌─base58Decode('3dc8KtHrwM')─┐
│ Encoded │
└────────────────────────────┘
tryBase58Decode
Like base58Decode
but returns an empty string in case of error.
base64Encode
Encodes a String or FixedString as base64.
Alias: TO_BASE64
.
base64Decode
Decodes a base64-encoded String or FixedString. Throws an exception in case of error.
Alias: FROM_BASE64
.
tryBase64Decode
Like base64Decode
but returns an empty string in case of error.
endsWith
Returns whether string str
ends with suffix
.
Syntax
endsWith(str, suffix)
startsWith
Returns whether string str
starts with prefix
.
Syntax
startsWith(str, prefix)
Example
SELECT startsWith('Spider-Man', 'Spi');
trim
Removes the specified characters from the start or end of a string. If not specified otherwise, the function removes whitespace (ASCII-character 32).
Syntax
trim([[LEADING|TRAILING|BOTH] trim_character FROM] input_string)
Arguments
Returned value
A string without leading and/or trailing specified characters.
Type: String
.
Example
SELECT trim(BOTH ' ()' FROM '( Hello, world! )');
Result:
┌─trim(BOTH ' ()' FROM '( Hello, world! )')─┐
│ Hello, world! │
└───────────────────────────────────────────────┘
trimLeft
Removes the consecutive occurrences of whitespace (ASCII-character 32) from the start of a string.
Syntax
trimLeft(input_string)
Alias: ltrim(input_string)
.
Arguments
input_string
— string to trim. String.
Returned value
A string without leading common whitespaces.
Type: String
.
Example
SELECT trimLeft(' Hello, world! ');
Result:
┌─trimLeft(' Hello, world! ')─┐
│ Hello, world! │
└─────────────────────────────────────┘
trimRight
Removes the consecutive occurrences of whitespace (ASCII-character 32) from the end of a string.
Syntax
trimRight(input_string)
Alias: rtrim(input_string)
.
Arguments
input_string
— string to trim. String.
Returned value
A string without trailing common whitespaces.
Type: String
.
Example
SELECT trimRight(' Hello, world! ');
Result:
┌─trimRight(' Hello, world! ')─┐
│ Hello, world! │
└──────────────────────────────────────┘
trimBoth
Removes the consecutive occurrences of whitespace (ASCII-character 32) from both ends of a string.
Syntax
trimBoth(input_string)
Alias: trim(input_string)
.
Arguments
input_string
— string to trim. String.
Returned value
A string without leading and trailing common whitespaces.
Type: String
.
Example
SELECT trimBoth(' Hello, world! ');
Result:
┌─trimBoth(' Hello, world! ')─┐
│ Hello, world! │
└─────────────────────────────────────┘
CRC32
Returns the CRC32 checksum of a string using CRC-32-IEEE 802.3 polynomial and initial value 0xffffffff
(zlib implementation).
The result type is UInt32.
CRC32IEEE
Returns the CRC32 checksum of a string, using CRC-32-IEEE 802.3 polynomial.
The result type is UInt32.
CRC64
Returns the CRC64 checksum of a string, using CRC-64-ECMA polynomial.
The result type is UInt64.
normalizeQuery
Replaces literals, sequences of literals and complex aliases with placeholders.
Syntax
normalizeQuery(x)
Arguments
x
— Sequence of characters. String.
Returned value
- Sequence of characters with placeholders.
Type: String.
Example
SELECT normalizeQuery('[1, 2, 3, x]') AS query;
Result:
┌─query────┐
│ [?.., x] │
└──────────┘
normalizedQueryHash
Returns identical 64bit hash values without the values of literals for similar queries. Can be helpful to analyze query log.
Syntax
normalizedQueryHash(x)
Arguments
x
— Sequence of characters. String.
Returned value
- Hash value.
Type: UInt64.
Example
SELECT normalizedQueryHash('SELECT 1 AS `xyz`') != normalizedQueryHash('SELECT 1 AS `abc`') AS res;
Result:
┌─res─┐
│ 1 │
└─────┘
normalizeUTF8NFC
Converts a string to NFC normalized form, assuming the string is valid UTF8-encoded text.
Syntax
normalizeUTF8NFC(words)
Arguments
words
— UTF8-encoded input string. String.
Returned value
- String transformed to NFC normalization form.
Type: String.
Example
SELECT length('â'), normalizeUTF8NFC('â') AS nfc, length(nfc) AS nfc_len;
Result:
┌─length('â')─┬─nfc─┬─nfc_len─┐
│ 2 │ â │ 2 │
└─────────────┴─────┴─────────┘
normalizeUTF8NFD
Converts a string to NFD normalized form, assuming the string is valid UTF8-encoded text.
Syntax
normalizeUTF8NFD(words)
Arguments
words
— UTF8-encoded input string. String.
Returned value
- String transformed to NFD normalization form.
Type: String.
Example
SELECT length('â'), normalizeUTF8NFD('â') AS nfd, length(nfd) AS nfd_len;
Result:
┌─length('â')─┬─nfd─┬─nfd_len─┐
│ 2 │ â │ 3 │
└─────────────┴─────┴─────────┘
normalizeUTF8NFKC
Converts a string to NFKC normalized form, assuming the string is valid UTF8-encoded text.
Syntax
normalizeUTF8NFKC(words)
Arguments
words
— UTF8-encoded input string. String.
Returned value
- String transformed to NFKC normalization form.
Type: String.
Example
SELECT length('â'), normalizeUTF8NFKC('â') AS nfkc, length(nfkc) AS nfkc_len;
Result:
┌─length('â')─┬─nfkc─┬─nfkc_len─┐
│ 2 │ â │ 2 │
└─────────────┴──────┴──────────┘
normalizeUTF8NFKD
Converts a string to NFKD normalized form, assuming the string is valid UTF8-encoded text.
Syntax
normalizeUTF8NFKD(words)
Arguments
words
— UTF8-encoded input string. String.
Returned value
- String transformed to NFKD normalization form.
Type: String.
Example
SELECT length('â'), normalizeUTF8NFKD('â') AS nfkd, length(nfkd) AS nfkd_len;
Result:
┌─length('â')─┬─nfkd─┬─nfkd_len─┐
│ 2 │ â │ 3 │
└─────────────┴──────┴──────────┘
encodeXMLComponent
Escapes characters with special meaning in XML such that they can afterwards be place into a XML text node or attribute.
The following characters are replaced: <
, &
, >
, "
, '
.
Also see the list of XML and HTML character entity references.
Syntax
encodeXMLComponent(x)
Arguments
x
— An input string. String.
Returned value
- The escaped string.
Type: String.
Example
SELECT encodeXMLComponent('Hello, "world"!');
SELECT encodeXMLComponent('<123>');
SELECT encodeXMLComponent('&clickhouse');
SELECT encodeXMLComponent('\'foo\'');
Result:
Hello, "world"!
<123>
&clickhouse
'foo'
decodeXMLComponent
Un-escapes substrings with special meaning in XML. These substrings are: "
&
'
>
<
This function also replaces numeric character references with Unicode characters. Both decimal (like ✓
) and hexadecimal (✓
) forms are supported.
Syntax
decodeXMLComponent(x)
Arguments
x
— An input string. String.
Returned value
- The un-escaped string.
Type: String.
Example
SELECT decodeXMLComponent(''foo'');
SELECT decodeXMLComponent('< Σ >');
Result:
'foo'
< Σ >
extractTextFromHTML
This function extracts plain text from HTML or XHTML.
It does not conform 100% to the HTML, XML or XHTML specification but the implementation is reasonably accurate and fast. The rules are the following:
- Comments are skipped. Example:
<!-- test -->
. Comment must end with-->
. Nested comments are disallowed. Note: constructions like<!-->
and<!--->
are not valid comments in HTML but they are skipped by other rules. - CDATA is pasted verbatim. Note: CDATA is XML/XHTML-specific and processed on a "best-effort" basis.
script
andstyle
elements are removed with all their content. Note: it is assumed that closing tag cannot appear inside content. For example, in JS string literal has to be escaped like"<\/script>"
. Note: comments and CDATA are possible insidescript
orstyle
- then closing tags are not searched inside CDATA. Example:<script><![CDATA[</script>]]></script>
. But they are still searched inside comments. Sometimes it becomes complicated:<script>var x = "<!--"; </script> var y = "-->"; alert(x + y);</script>
Note:script
andstyle
can be the names of XML namespaces - then they are not treated like usualscript
orstyle
elements. Example:<script:a>Hello</script:a>
. Note: whitespaces are possible after closing tag name:</script >
but not before:< / script>
.- Other tags or tag-like elements are skipped without inner content. Example:
<a>.</a>
Note: it is expected that this HTML is illegal:<a test=">"></a>
Note: it also skips something like tags:<>
,<!>
, etc. Note: tag without end is skipped to the end of input:<hello
- HTML and XML entities are not decoded. They must be processed by separate function.
- Whitespaces in the text are collapsed or inserted by specific rules.
- Whitespaces at the beginning and at the end are removed.
- Consecutive whitespaces are collapsed.
- But if the text is separated by other elements and there is no whitespace, it is inserted.
- It may cause unnatural examples:
Hello<b>world</b>
,Hello<!-- -->world
- there is no whitespace in HTML, but the function inserts it. Also consider:Hello<p>world</p>
,Hello<br>world
. This behavior is reasonable for data analysis, e.g. to convert HTML to a bag of words.
- Also note that correct handling of whitespaces requires the support of
<pre></pre>
and CSSdisplay
andwhite-space
properties.
Syntax
extractTextFromHTML(x)
Arguments
x
— input text. String.
Returned value
- Extracted text.
Type: String.
Example
The first example contains several tags and a comment and also shows whitespace processing.
The second example shows CDATA
and script
tag processing.
In the third example text is extracted from the full HTML response received by the url function.
SELECT extractTextFromHTML(' <p> A text <i>with</i><b>tags</b>. <!-- comments --> </p> ');
SELECT extractTextFromHTML('<![CDATA[The content within <b>CDATA</b>]]> <script>alert("Script");</script>');
SELECT extractTextFromHTML(html) FROM url('http://www.donothingfor2minutes.com/', RawBLOB, 'html String');
Result:
A text with tags .
The content within <b>CDATA</b>
Do Nothing for 2 Minutes 2:00
ascii
Returns the ASCII code point (as Int32) of the first character of string s
.
If s
is empty, the result is 0. If the first character is not an ASCII character or not part of the Latin-1 supplement range of UTF-16, the result is undefined.
Syntax
ascii(s)
soundex
Returns the Soundex code of a string.
Syntax
soundex(val)
Arguments
val
- Input value. String
Returned value
- The Soundex code of the input value. String
Example
select soundex('aksel');
Result:
┌─soundex('aksel')─┐
│ A240 │
└──────────────────┘
initcap
Convert the first letter of each word to upper case and the rest to lower case. Words are sequences of alphanumeric characters separated by non-alphanumeric characters.
initcapUTF8
Like initcap, assuming that the string contains valid UTF-8 encoded text. If this assumption is violated, no exception is thrown and the result is undefined.
Does not detect the language, e.g. for Turkish the result might not be exactly correct (i/İ vs. i/I).
If the length of the UTF-8 byte sequence is different for upper and lower case of a code point, the result may be incorrect for this code point.
firstLine
Returns the first line from a multi-line string.
Syntax
firstLine(val)
Arguments
val
- Input value. String
Returned value
- The first line of the input value or the whole value if there is no line separators. String
Example
select firstLine('foo\nbar\nbaz');
Result:
┌─firstLine('foo\nbar\nbaz')─┐
│ foo │
└────────────────────────────┘