Merge pull request #66189 from Blargian/docs_normalize

[Docs] add `normalizeQueryKeepNames`, `normalizedQueryHashKeepNames`
This commit is contained in:
Daniil Ivanik 2024-07-08 13:19:56 +00:00 committed by GitHub
commit 608c13fa9c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 80 additions and 2 deletions

View File

@ -1555,7 +1555,7 @@ The result type is UInt64.
## normalizeQuery
Replaces literals, sequences of literals and complex aliases with placeholders.
Replaces literals, sequences of literals and complex aliases (containing whitespace, more than two digits or at least 36 bytes long such as UUIDs) with placeholder `?`.
**Syntax**
@ -1573,6 +1573,8 @@ normalizeQuery(x)
**Example**
Query:
``` sql
SELECT normalizeQuery('[1, 2, 3, x]') AS query;
```
@ -1585,9 +1587,44 @@ Result:
└──────────┘
```
## normalizeQueryKeepNames
Replaces literals, sequences of literals with placeholder `?` but does not replace complex aliases (containing whitespace, more than two digits
or at least 36 bytes long such as UUIDs). This helps better analyze complex query logs.
**Syntax**
``` sql
normalizeQueryKeepNames(x)
```
**Arguments**
- `x` — Sequence of characters. [String](../data-types/string.md).
**Returned value**
- Sequence of characters with placeholders. [String](../data-types/string.md).
**Example**
Query:
``` sql
SELECT normalizeQuery('SELECT 1 AS aComplexName123'), normalizeQueryKeepNames('SELECT 1 AS aComplexName123');
```
Result:
```result
┌─normalizeQuery('SELECT 1 AS aComplexName123')─┬─normalizeQueryKeepNames('SELECT 1 AS aComplexName123')─┐
│ SELECT ? AS `?` │ SELECT ? AS aComplexName123 │
└───────────────────────────────────────────────┴────────────────────────────────────────────────────────┘
```
## normalizedQueryHash
Returns identical 64bit hash values without the values of literals for similar queries. Can be helpful to analyze query log.
Returns identical 64bit hash values without the values of literals for similar queries. Can be helpful to analyze query logs.
**Syntax**
@ -1605,6 +1642,8 @@ normalizedQueryHash(x)
**Example**
Query:
``` sql
SELECT normalizedQueryHash('SELECT 1 AS `xyz`') != normalizedQueryHash('SELECT 1 AS `abc`') AS res;
```
@ -1617,6 +1656,43 @@ Result:
└─────┘
```
## normalizedQueryHashKeepNames
Like [normalizedQueryHash](#normalizedqueryhash) it returns identical 64bit hash values without the values of literals for similar queries but it does not replace complex aliases (containing whitespace, more than two digits
or at least 36 bytes long such as UUIDs) with a placeholder before hashing. Can be helpful to analyze query logs.
**Syntax**
``` sql
normalizedQueryHashKeepNames(x)
```
**Arguments**
- `x` — Sequence of characters. [String](../data-types/string.md).
**Returned value**
- Hash value. [UInt64](../data-types/int-uint.md#uint-ranges).
**Example**
``` sql
SELECT normalizedQueryHash('SELECT 1 AS `xyz123`') != normalizedQueryHash('SELECT 1 AS `abc123`') AS normalizedQueryHash;
SELECT normalizedQueryHashKeepNames('SELECT 1 AS `xyz123`') != normalizedQueryHashKeepNames('SELECT 1 AS `abc123`') AS normalizedQueryHashKeepNames;
```
Result:
```result
┌─normalizedQueryHash─┐
│ 0 │
└─────────────────────┘
┌─normalizedQueryHashKeepNames─┐
│ 1 │
└──────────────────────────────┘
```
## normalizeUTF8NFC
Converts a string to [NFC normalized form](https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms), assuming the string is valid UTF8-encoded text.

View File

@ -2124,8 +2124,10 @@ noaa
nonNegativeDerivative
noop
normalizeQuery
normalizeQueryKeepNames
normalizeUTF
normalizedQueryHash
normalizedQueryHashKeepNames
notEmpty
notEquals
notILike