mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-22 23:52:03 +00:00
add normalizeQueryKeepNames and normalizedQueryHashKeepNames
This commit is contained in:
parent
2bca1963a7
commit
126417cc61
@ -1556,7 +1556,8 @@ The result type is UInt64.
|
|||||||
|
|
||||||
## normalizeQuery
|
## normalizeQuery
|
||||||
|
|
||||||
Replaces literals, sequences of literals and complex aliases with placeholders.
|
Replaces literals, sequences of literals and complex aliases (containing whitespace, more than two digits
|
||||||
|
or at least 36 bytes long such as UUIDs) with placeholder `?`.
|
||||||
|
|
||||||
**Syntax**
|
**Syntax**
|
||||||
|
|
||||||
@ -1574,6 +1575,8 @@ normalizeQuery(x)
|
|||||||
|
|
||||||
**Example**
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
``` sql
|
``` sql
|
||||||
SELECT normalizeQuery('[1, 2, 3, x]') AS query;
|
SELECT normalizeQuery('[1, 2, 3, x]') AS query;
|
||||||
```
|
```
|
||||||
@ -1586,9 +1589,44 @@ Result:
|
|||||||
└──────────┘
|
└──────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## normalizeQueryKeepNames
|
||||||
|
|
||||||
|
Replaces literals, sequences of literals with placeholder `?` but does not replace complex aliases (containing whitespace, more than two digits
|
||||||
|
or at least 36 bytes long such as UUIDs). This helps better analyze complex query logs.
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
normalizeQueryKeepNames(x)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `x` — Sequence of characters. [String](../data-types/string.md).
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Sequence of characters with placeholders. [String](../data-types/string.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT normalizeQuery('SELECT 1 AS aComplexName123'), normalizeQueryKeepNames('SELECT 1 AS aComplexName123');
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
```result
|
||||||
|
┌─normalizeQuery('SELECT 1 AS aComplexName123')─┬─normalizeQueryKeepNames('SELECT 1 AS aComplexName123')─┐
|
||||||
|
│ SELECT ? AS `?` │ SELECT ? AS aComplexName123 │
|
||||||
|
└───────────────────────────────────────────────┴────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
## normalizedQueryHash
|
## normalizedQueryHash
|
||||||
|
|
||||||
Returns identical 64bit hash values without the values of literals for similar queries. Can be helpful to analyze query log.
|
Returns identical 64bit hash values without the values of literals for similar queries. Can be helpful to analyze query logs.
|
||||||
|
|
||||||
**Syntax**
|
**Syntax**
|
||||||
|
|
||||||
@ -1606,6 +1644,8 @@ normalizedQueryHash(x)
|
|||||||
|
|
||||||
**Example**
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
``` sql
|
``` sql
|
||||||
SELECT normalizedQueryHash('SELECT 1 AS `xyz`') != normalizedQueryHash('SELECT 1 AS `abc`') AS res;
|
SELECT normalizedQueryHash('SELECT 1 AS `xyz`') != normalizedQueryHash('SELECT 1 AS `abc`') AS res;
|
||||||
```
|
```
|
||||||
@ -1618,6 +1658,43 @@ Result:
|
|||||||
└─────┘
|
└─────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## normalizedQueryHashKeepNames
|
||||||
|
|
||||||
|
Like [normalizedQueryHash](#normalizedqueryhash) it returns identical 64bit hash values without the values of literals for similar queries but it does not replace complex aliases (containing whitespace, more than two digits
|
||||||
|
or at least 36 bytes long such as UUIDs) with a placeholder before hashing. Can be helpful to analyze query logs.
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
normalizedQueryHashKeepNames(x)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `x` — Sequence of characters. [String](../data-types/string.md).
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Hash value. [UInt64](../data-types/int-uint.md#uint-ranges).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT normalizedQueryHash('SELECT 1 AS `xyz123`') != normalizedQueryHash('SELECT 1 AS `abc123`') AS normalizedQueryHash;
|
||||||
|
SELECT normalizedQueryHashKeepNames('SELECT 1 AS `xyz123`') != normalizedQueryHashKeepNames('SELECT 1 AS `abc123`') AS normalizedQueryHashKeepNames;
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
```result
|
||||||
|
┌─normalizedQueryHash─┐
|
||||||
|
│ 0 │
|
||||||
|
└─────────────────────┘
|
||||||
|
┌─normalizedQueryHashKeepNames─┐
|
||||||
|
│ 1 │
|
||||||
|
└──────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
## normalizeUTF8NFC
|
## normalizeUTF8NFC
|
||||||
|
|
||||||
Converts a string to [NFC normalized form](https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms), assuming the string is valid UTF8-encoded text.
|
Converts a string to [NFC normalized form](https://en.wikipedia.org/wiki/Unicode_equivalence#Normal_forms), assuming the string is valid UTF8-encoded text.
|
||||||
|
Loading…
Reference in New Issue
Block a user