ClickHouse/tuple-map-functions.md at c37a0710045771ab8382b4a47b2862f6b5f28ef4

mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-13 02:53:38 +00:00

Merge pull request #49466 from ucasfl/str_to_map

add alias str_to_map and mapFromString for extractKeyValuePairs

2023-05-08 10:11:06 +02:00

26 KiB

Raw Blame History

slug	sidebar_position	sidebar_label
/en/sql-reference/functions/tuple-map-functions	120	Maps

map

Arranges key:value pairs into Map(key, value) data type.

Syntax

map(key1, value1[, key2, value2, ...])

Arguments

key — The key part of the pair. String, Integer, LowCardinality, FixedString, UUID, Date, DateTime, Date32, Enum.
value — The value part of the pair. Arbitrary type, including Map and Array.

Returned value

Data structure as key:value pairs.

Type: Map(key, value).

Examples

Query:

SELECT map('key1', number, 'key2', number * 2) FROM numbers(3);

Result:

┌─map('key1', number, 'key2', multiply(number, 2))─┐
│ {'key1':0,'key2':0}                              │
│ {'key1':1,'key2':2}                              │
│ {'key1':2,'key2':4}                              │
└──────────────────────────────────────────────────┘

Query:

CREATE TABLE table_map (a Map(String, UInt64)) ENGINE = MergeTree() ORDER BY a;
INSERT INTO table_map SELECT map('key1', number, 'key2', number * 2) FROM numbers(3);
SELECT a['key2'] FROM table_map;

Result:

┌─arrayElement(a, 'key2')─┐
│                       0 │
│                       2 │
│                       4 │
└─────────────────────────┘

See Also

Map(key, value) data type

mapFromArrays

Merges an Array of keys and an Array of values into a Map(key, value). Notice that the second argument could also be a Map, thus it is casted to an Array when executing.

The function is a more convenient alternative to CAST((key_array, value_array_or_map), 'Map(key_type, value_type)'). For example, instead of writing CAST((['aa', 'bb'], [4, 5]), 'Map(String, UInt32)'), you can write mapFromArrays(['aa', 'bb'], [4, 5]).

Syntax

mapFromArrays(keys, values)

Alias: MAP_FROM_ARRAYS(keys, values)

Arguments

keys — Given key array to create a map from. The nested type of array must be: String, Integer, LowCardinality, FixedString, UUID, Date, DateTime, Date32, Enum
values - Given value array or map to create a map from.

Returned value

A map whose keys and values are constructed from the key array and value array/map.

Example

Query:

select mapFromArrays(['a', 'b', 'c'], [1, 2, 3])


┌─mapFromArrays(['a', 'b', 'c'], [1, 2, 3])─┐
│ {'a':1,'b':2,'c':3}                       │
└───────────────────────────────────────────┘

SELECT mapFromArrays([1, 2, 3], map('a', 1, 'b', 2, 'c', 3))

┌─mapFromArrays([1, 2, 3], map('a', 1, 'b', 2, 'c', 3))─┐
│ {1:('a',1),2:('b',2),3:('c',3)}                       │
└───────────────────────────────────────────────────────┘

extractKeyValuePairs

Extracts key-value pairs, i.e. a Map(String, String), from a string. Parsing is robust towards noise (e.g. log files).

A key-value pair consists of a key, followed by a key_value_delimiter and a value. Key value pairs must be separated by pair_delimiter. Quoted keys and values are also supported.

Syntax

extractKeyValuePairs(data[, key_value_delimiter[, pair_delimiter[, quoting_character]]])

Alias:

str_to_map
mapFromString

Arguments

data - String to extract key-value pairs from. String or FixedString.
key_value_delimiter - Character to be used as delimiter between the key and the value. Defaults to :. String or FixedString.
pair_delimiters - Set of character to be used as delimiters between pairs. Defaults to , , and ;. String or FixedString.
quoting_character - Character to be used as quoting character. Defaults to ". String or FixedString.

Returned values

A Map(String, String) of key-value pairs.

Examples

Simple case:

SELECT extractKeyValuePairs('name:neymar, age:31 team:psg,nationality:brazil') as kv

Result:

┌─kv──────────────────────────────────────────────────────────────────────┐
│ {'name':'neymar','age':'31','team':'psg','nationality':'brazil'}        │
└─────────────────────────────────────────────────────────────────────────┘

Single quote as quoting character:

SELECT extractKeyValuePairs('name:\'neymar\';\'age\':31;team:psg;nationality:brazil,last_key:last_value', ':', ';,', '\'') as kv

Result:

┌─kv───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ {'name':'neymar','age':'31','team':'psg','nationality':'brazil','last_key':'last_value'}                                 │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Escape sequences without escape sequences support:

SELECT extractKeyValuePairs('age:a\\x0A\\n\\0') AS kv

Result:

┌─kv─────────────────────┐
│ {'age':'a\\x0A\\n\\0'} │
└────────────────────────┘

extractKeyValuePairsWithEscaping

Same as extractKeyValuePairs but with escaping support.

Supported escape sequences: \x, \N, \a, \b, \e, \f, \n, \r, \t, \v and \0. Non standard escape sequences are returned as it is (including the backslash) unless they are one of the following: \\, ', ", backtick, /, = or ASCII control characters (c <= 31).

This function will satisfy the use case where pre-escaping and post-escaping are not suitable. For instance, consider the following input string: a: "aaaa\"bbb". The expected output is: a: aaaa\"bbbb.

Pre-escaping: Pre-escaping it will output: a: "aaaa"bbb" and extractKeyValuePairs will then output: a: aaaa
Post-escaping: extractKeyValuePairs will output a: aaaa\ and post-escaping will keep it as it is.

Leading escape sequences will be skipped in keys and will be considered invalid for values.

Examples

Escape sequences with escape sequence support turned on:

SELECT extractKeyValuePairsWithEscaping('age:a\\x0A\\n\\0') AS kv

Result:

┌─kv────────────────┐
│ {'age':'a\n\n\0'} │
└───────────────────┘

mapAdd

Collect all the keys and sum corresponding values.

Syntax

mapAdd(arg1, arg2 [, ...])

Arguments

Arguments are maps or tuples of two arrays, where items in the first array represent keys, and the second array contains values for the each key. All key arrays should have same type, and all value arrays should contain items which are promoted to the one type (Int64, UInt64 or Float64). The common promoted type is used as a type for the result array.

Returned value

Depending on the arguments returns one map or tuple, where the first array contains the sorted keys and the second array contains values.

Example

Query with a tuple:

SELECT mapAdd(([toUInt8(1), 2], [1, 1]), ([toUInt8(1), 2], [1, 1])) as res, toTypeName(res) as type;

Result:

┌─res───────────┬─type───────────────────────────────┐
│ ([1,2],[2,2]) │ Tuple(Array(UInt8), Array(UInt64)) │
└───────────────┴────────────────────────────────────┘

Query with Map type:

SELECT mapAdd(map(1,1), map(1,1));

Result:

┌─mapAdd(map(1, 1), map(1, 1))─┐
│ {1:2}                        │
└──────────────────────────────┘

mapSubtract

Collect all the keys and subtract corresponding values.

Syntax

mapSubtract(Tuple(Array, Array), Tuple(Array, Array) [, ...])

Arguments

Arguments are maps or tuples of two arrays, where items in the first array represent keys, and the second array contains values for the each key. All key arrays should have same type, and all value arrays should contain items which are promote to the one type (Int64, UInt64 or Float64). The common promoted type is used as a type for the result array.

Returned value

Depending on the arguments returns one map or tuple, where the first array contains the sorted keys and the second array contains values.

Example

Query with a tuple map:

SELECT mapSubtract(([toUInt8(1), 2], [toInt32(1), 1]), ([toUInt8(1), 2], [toInt32(2), 1])) as res, toTypeName(res) as type;

Result:

┌─res────────────┬─type──────────────────────────────┐
│ ([1,2],[-1,0]) │ Tuple(Array(UInt8), Array(Int64)) │
└────────────────┴───────────────────────────────────┘

Query with Map type:

SELECT mapSubtract(map(1,1), map(1,1));

Result:

┌─mapSubtract(map(1, 1), map(1, 1))─┐
│ {1:0}                             │
└───────────────────────────────────┘

mapPopulateSeries

Fills missing keys in the maps (key and value array pair), where keys are integers. Also, it supports specifying the max key, which is used to extend the keys array.

Syntax

mapPopulateSeries(keys, values[, max])
mapPopulateSeries(map[, max])

Generates a map (a tuple with two arrays or a value of Map type, depending on the arguments), where keys are a series of numbers, from minimum to maximum keys (or max argument if it specified) taken from the map with a step size of one, and corresponding values. If the value is not specified for the key, then it uses the default value in the resulting map. For repeated keys, only the first value (in order of appearing) gets associated with the key.

For array arguments the number of elements in keys and values must be the same for each row.

Arguments

Arguments are maps or two arrays, where the first array represent keys, and the second array contains values for the each key.

Mapped arrays:

keys — Array of keys. Array(Int).
values — Array of values. Array(Int).
max — Maximum key value. Optional. Int8, Int16, Int32, Int64, Int128, Int256.

map — Map with integer keys. Map.

Returned value

Depending on the arguments returns a map or a tuple of two arrays: keys in sorted order, and values the corresponding keys.

Example

Query with mapped arrays:

SELECT mapPopulateSeries([1,2,4], [11,22,44], 5) AS res, toTypeName(res) AS type;

Result:

┌─res──────────────────────────┬─type──────────────────────────────┐
│ ([1,2,3,4,5],[11,22,0,44,0]) │ Tuple(Array(UInt8), Array(UInt8)) │
└──────────────────────────────┴───────────────────────────────────┘

Query with Map type:

SELECT mapPopulateSeries(map(1, 10, 5, 20), 6);

Result:

┌─mapPopulateSeries(map(1, 10, 5, 20), 6)─┐
│ {1:10,2:0,3:0,4:0,5:20,6:0}             │
└─────────────────────────────────────────┘

mapContains

Determines whether the map contains the key parameter.

Syntax

mapContains(map, key)

Arguments

map — Map. Map.
key — Key. Type matches the type of keys of map parameter.

Returned value

1 if map contains key, 0 if not.

Type: UInt8.

Example

Query:

CREATE TABLE test (a Map(String,String)) ENGINE = Memory;

INSERT INTO test VALUES ({'name':'eleven','age':'11'}), ({'number':'twelve','position':'6.0'});

SELECT mapContains(a, 'name') FROM test;

Result:

┌─mapContains(a, 'name')─┐
│                      1 │
│                      0 │
└────────────────────────┘

mapKeys

Returns all keys from the map parameter.

Can be optimized by enabling the optimize_functions_to_subcolumns setting. With optimize_functions_to_subcolumns = 1 the function reads only keys subcolumn instead of reading and processing the whole column data. The query SELECT mapKeys(m) FROM table transforms to SELECT m.keys FROM table.

Syntax

mapKeys(map)

Arguments

map — Map. Map.

Returned value

Array containing all keys from the map.

Type: Array.

Example

Query:

CREATE TABLE test (a Map(String,String)) ENGINE = Memory;

INSERT INTO test VALUES ({'name':'eleven','age':'11'}), ({'number':'twelve','position':'6.0'});

SELECT mapKeys(a) FROM test;

Result:

┌─mapKeys(a)────────────┐
│ ['name','age']        │
│ ['number','position'] │
└───────────────────────┘

mapValues

Returns all values from the map parameter.

Can be optimized by enabling the optimize_functions_to_subcolumns setting. With optimize_functions_to_subcolumns = 1 the function reads only values subcolumn instead of reading and processing the whole column data. The query SELECT mapValues(m) FROM table transforms to SELECT m.values FROM table.

Syntax

mapValues(map)

Arguments

map — Map. Map.

Returned value

Array containing all the values from map.

Type: Array.

Example

Query:

CREATE TABLE test (a Map(String,String)) ENGINE = Memory;

INSERT INTO test VALUES ({'name':'eleven','age':'11'}), ({'number':'twelve','position':'6.0'});

SELECT mapValues(a) FROM test;

Result:

┌─mapValues(a)─────┐
│ ['eleven','11']  │
│ ['twelve','6.0'] │
└──────────────────┘

mapContainsKeyLike

Syntax

mapContainsKeyLike(map, pattern)

Arguments

map — Map. Map.
pattern - String pattern to match.

Returned value

1 if map contains key like specified pattern, 0 if not.

Example

Query:

CREATE TABLE test (a Map(String,String)) ENGINE = Memory;

INSERT INTO test VALUES ({'abc':'abc','def':'def'}), ({'hij':'hij','klm':'klm'});

SELECT mapContainsKeyLike(a, 'a%') FROM test;

Result:

┌─mapContainsKeyLike(a, 'a%')─┐
│                           1 │
│                           0 │
└─────────────────────────────┘

mapExtractKeyLike

Syntax

mapExtractKeyLike(map, pattern)

Arguments

map — Map. Map.
pattern - String pattern to match.

Returned value

A map contained elements the key of which matches the specified pattern. If there are no elements matched the pattern, it will return an empty map.

Example

Query:

CREATE TABLE test (a Map(String,String)) ENGINE = Memory;

INSERT INTO test VALUES ({'abc':'abc','def':'def'}), ({'hij':'hij','klm':'klm'});

SELECT mapExtractKeyLike(a, 'a%') FROM test;

Result:

┌─mapExtractKeyLike(a, 'a%')─┐
│ {'abc':'abc'}              │
│ {}                         │
└────────────────────────────┘

mapApply

Syntax

mapApply(func, map)

Arguments

func - Lambda function.
map — Map.

Returned value

Returns a map obtained from the original map by application of func(map1[i], …, mapN[i]) for each element.

Example

Query:

SELECT mapApply((k, v) -> (k, v * 10), _map) AS r
FROM
(
    SELECT map('key1', number, 'key2', number * 2) AS _map
    FROM numbers(3)
)

Result:

┌─r─────────────────────┐
│ {'key1':0,'key2':0}   │
│ {'key1':10,'key2':20} │
│ {'key1':20,'key2':40} │
└───────────────────────┘

mapFilter

Syntax

mapFilter(func, map)

Arguments

func - Lambda function.
map — Map.

Returned value

Returns a map containing only the elements in map for which func(map1[i], …, mapN[i]) returns something other than 0.

Example

Query:

SELECT mapFilter((k, v) -> ((v % 2) = 0), _map) AS r
FROM
(
    SELECT map('key1', number, 'key2', number * 2) AS _map
    FROM numbers(3)
)

Result:

┌─r───────────────────┐
│ {'key1':0,'key2':0} │
│ {'key2':2}          │
│ {'key1':2,'key2':4} │
└─────────────────────┘

mapUpdate

Syntax

mapUpdate(map1, map2)

Arguments

map1 Map.
map2 Map.

Returned value

Returns a map1 with values updated of values for the corresponding keys in map2.

Example

Query:

SELECT mapUpdate(map('key1', 0, 'key3', 0), map('key1', 10, 'key2', 10)) AS map;

Result:

┌─map────────────────────────────┐
│ {'key3':0,'key1':10,'key2':10} │
└────────────────────────────────┘

mapConcat

Syntax

mapConcat(maps)

Arguments

maps – Arbitrary number of arguments of Map type.

Returned value

Returns a map with concatenated maps passed as arguments. If there are same keys in two or more maps, all of them are added to the result map, but only the first one is accessible via operator []

Examples

Query:

SELECT mapConcat(map('key1', 1, 'key3', 3), map('key2', 2)) AS map;

Result:

┌─map──────────────────────────┐
│ {'key1':1,'key3':3,'key2':2} │
└──────────────────────────────┘

Query:

SELECT mapConcat(map('key1', 1, 'key2', 2), map('key1', 3)) AS map, map['key1'];

Result:

┌─map──────────────────────────┬─elem─┐
│ {'key1':1,'key2':2,'key1':3} │    1 │
└──────────────────────────────┴──────┘

mapExists([func,], map)

Returns 1 if there is at least one key-value pair in map for which func(key, value) returns something other than 0. Otherwise, it returns 0.

Note that the mapExists is a higher-order function. You can pass a lambda function to it as the first argument.

Example

Query:

SELECT mapExists((k, v) -> (v = 1), map('k1', 1, 'k2', 2)) AS res

Result:

┌─res─┐
│   1 │
└─────┘

mapAll([func,] map)

Returns 1 if func(key, value) returns something other than 0 for all key-value pairs in map. Otherwise, it returns 0.

Note that the mapAll is a higher-order function. You can pass a lambda function to it as the first argument.

Example

Query:

SELECT mapAll((k, v) -> (v = 1), map('k1', 1, 'k2', 2)) AS res

Result:

┌─res─┐
│   0 │
└─────┘

mapSort([func,], map)

Sorts the elements of the map in ascending order. If the func function is specified, sorting order is determined by the result of the func function applied to the keys and values of the map.

Examples

SELECT mapSort(map('key2', 2, 'key3', 1, 'key1', 3)) AS map;

┌─map──────────────────────────┐
│ {'key1':3,'key2':2,'key3':1} │
└──────────────────────────────┘

SELECT mapSort((k, v) -> v, map('key2', 2, 'key3', 1, 'key1', 3)) AS map;

┌─map──────────────────────────┐
│ {'key3':1,'key2':2,'key1':3} │
└──────────────────────────────┘

For more details see the reference for arraySort function.

mapReverseSort([func,], map)

Sorts the elements of the map in descending order. If the func function is specified, sorting order is determined by the result of the func function applied to the keys and values of the map.

Examples

SELECT mapReverseSort(map('key2', 2, 'key3', 1, 'key1', 3)) AS map;

┌─map──────────────────────────┐
│ {'key3':1,'key2':2,'key1':3} │
└──────────────────────────────┘

SELECT mapReverseSort((k, v) -> v, map('key2', 2, 'key3', 1, 'key1', 3)) AS map;

┌─map──────────────────────────┐
│ {'key1':3,'key2':2,'key3':1} │
└──────────────────────────────┘

For more details see the reference for arrayReverseSort function.

26 KiB Raw Blame History Unescape Escape

map

mapFromArrays

extractKeyValuePairs

extractKeyValuePairsWithEscaping

mapAdd

mapSubtract

mapPopulateSeries

mapContains

mapKeys

mapValues

mapContainsKeyLike

mapExtractKeyLike

mapApply

mapFilter

mapUpdate

mapConcat

mapExists([func,], map)

mapAll([func,] map)

mapSort([func,], map)

mapReverseSort([func,], map)

26 KiB

Raw Blame History