ClickHouse/docs/en/sql-reference/functions/higher-order-functions.md

263 lines
9.7 KiB
Markdown
Raw Normal View History

2020-04-03 13:23:32 +00:00
---
toc_priority: 57
toc_title: Higher-Order
---
# Higher-order Functions {#higher-order-functions}
2020-03-20 10:10:48 +00:00
## `->` operator, lambda(params, expr) function {#operator-lambdaparams-expr-function}
Allows describing a lambda function for passing to a higher-order function. The left side of the arrow has a formal parameter, which is any ID, or multiple formal parameters any IDs in a tuple. The right side of the arrow has an expression that can use these formal parameters, as well as any table columns.
Examples: `x -> 2 * x, str -> str != Referer.`
Higher-order functions can only accept lambda functions as their functional argument.
A lambda function that accepts multiple arguments can be passed to a higher-order function. In this case, the higher-order function is passed several arrays of identical length that these arguments will correspond to.
2019-04-24 14:35:52 +00:00
For some functions, such as [arrayCount](#higher_order_functions-array-count) or [arraySum](#higher_order_functions-array-count), the first argument (the lambda function) can be omitted. In this case, identical mapping is assumed.
2020-03-20 10:10:48 +00:00
A lambda function cant be omitted for the following functions:
- [arrayMap](#higher_order_functions-array-map)
- [arrayFilter](#higher_order_functions-array-filter)
- [arrayFill](#higher_order_functions-array-fill)
- [arrayReverseFill](#higher_order_functions-array-reverse-fill)
- [arraySplit](#higher_order_functions-array-split)
- [arrayReverseSplit](#higher_order_functions-array-reverse-split)
- [arrayFirst](#higher_order_functions-array-first)
- [arrayFirstIndex](#higher_order_functions-array-first-index)
### arrayMap(func, arr1, …) {#higher_order_functions-array-map}
2019-10-26 13:38:05 +00:00
Returns an array obtained from the original application of the `func` function to each element in the `arr` array.
2019-04-24 14:35:52 +00:00
Examples:
2020-03-20 10:10:48 +00:00
``` sql
2019-04-24 14:35:52 +00:00
SELECT arrayMap(x -> (x + 2), [1, 2, 3]) as res;
DOCAPI-8530: Code blocks markup fix (#7060) * Typo fix. * Links fix. * Fixed links in docs. * More fixes. * docs/en: cleaning some files * docs/en: cleaning data_types * docs/en: cleaning database_engines * docs/en: cleaning development * docs/en: cleaning getting_started * docs/en: cleaning interfaces * docs/en: cleaning operations * docs/en: cleaning query_lamguage * docs/en: cleaning en * docs/ru: cleaning data_types * docs/ru: cleaning index * docs/ru: cleaning database_engines * docs/ru: cleaning development * docs/ru: cleaning general * docs/ru: cleaning getting_started * docs/ru: cleaning interfaces * docs/ru: cleaning operations * docs/ru: cleaning query_language * docs: cleaning interfaces/http * Update docs/en/data_types/array.md decorated ``` Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/getting_started/example_datasets/nyc_taxi.md fixed typo Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/getting_started/example_datasets/ontime.md fixed typo Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/interfaces/formats.md fixed error Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/table_engines/custom_partitioning_key.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/utils/clickhouse-local.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/dicts/external_dicts_dict_sources.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/utils/clickhouse-local.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/json_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/json_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/other_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/other_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/date_time_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/table_engines/jdbc.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * docs: fixed error * docs: fixed error
2019-09-23 15:31:46 +00:00
```
2020-03-20 10:10:48 +00:00
``` text
2019-04-24 14:35:52 +00:00
┌─res─────┐
│ [3,4,5] │
└─────────┘
```
2020-03-20 10:10:48 +00:00
2019-04-24 14:35:52 +00:00
The following example shows how to create a tuple of elements from different arrays:
2020-03-20 10:10:48 +00:00
``` sql
2019-04-24 14:35:52 +00:00
SELECT arrayMap((x, y) -> (x, y), [1, 2, 3], [4, 5, 6]) AS res
DOCAPI-8530: Code blocks markup fix (#7060) * Typo fix. * Links fix. * Fixed links in docs. * More fixes. * docs/en: cleaning some files * docs/en: cleaning data_types * docs/en: cleaning database_engines * docs/en: cleaning development * docs/en: cleaning getting_started * docs/en: cleaning interfaces * docs/en: cleaning operations * docs/en: cleaning query_lamguage * docs/en: cleaning en * docs/ru: cleaning data_types * docs/ru: cleaning index * docs/ru: cleaning database_engines * docs/ru: cleaning development * docs/ru: cleaning general * docs/ru: cleaning getting_started * docs/ru: cleaning interfaces * docs/ru: cleaning operations * docs/ru: cleaning query_language * docs: cleaning interfaces/http * Update docs/en/data_types/array.md decorated ``` Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/getting_started/example_datasets/nyc_taxi.md fixed typo Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/getting_started/example_datasets/ontime.md fixed typo Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/interfaces/formats.md fixed error Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/table_engines/custom_partitioning_key.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/utils/clickhouse-local.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/dicts/external_dicts_dict_sources.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/utils/clickhouse-local.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/json_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/json_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/other_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/other_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/query_language/functions/date_time_functions.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * Update docs/en/operations/table_engines/jdbc.md Co-Authored-By: BayoNet <da-daos@yandex.ru> * docs: fixed error * docs: fixed error
2019-09-23 15:31:46 +00:00
```
2020-03-20 10:10:48 +00:00
``` text
2019-04-24 14:35:52 +00:00
┌─res─────────────────┐
│ [(1,4),(2,5),(3,6)] │
└─────────────────────┘
```
2019-06-06 14:13:23 +00:00
2020-03-20 10:10:48 +00:00
Note that the first argument (lambda function) cant be omitted in the `arrayMap` function.
2019-06-06 14:13:23 +00:00
### arrayFilter(func, arr1, …) {#higher_order_functions-array-filter}
2019-04-24 14:35:52 +00:00
Returns an array containing only the elements in `arr1` for which `func` returns something other than 0.
Examples:
2020-03-20 10:10:48 +00:00
``` sql
SELECT arrayFilter(x -> x LIKE '%World%', ['Hello', 'abc World']) AS res
```
2020-03-20 10:10:48 +00:00
``` text
┌─res───────────┐
│ ['abc World'] │
└───────────────┘
```
2020-03-20 10:10:48 +00:00
``` sql
SELECT
arrayFilter(
(i, x) -> x LIKE '%World%',
arrayEnumerate(arr),
['Hello', 'abc World'] AS arr)
AS res
```
2020-03-20 10:10:48 +00:00
``` text
┌─res─┐
│ [2] │
└─────┘
```
2020-03-20 10:10:48 +00:00
Note that the first argument (lambda function) cant be omitted in the `arrayFilter` function.
2019-06-06 14:13:23 +00:00
### arrayFill(func, arr1, …) {#higher_order_functions-array-fill}
2019-10-26 13:38:05 +00:00
Scan through `arr1` from the first element to the last element and replace `arr1[i]` by `arr1[i - 1]` if `func` returns 0. The first element of `arr1` will not be replaced.
Examples:
2020-03-20 10:10:48 +00:00
``` sql
2019-10-26 13:38:05 +00:00
SELECT arrayFill(x -> not isNull(x), [1, null, 3, 11, 12, null, null, 5, 6, 14, null, null]) AS res
```
2020-03-20 10:10:48 +00:00
``` text
2019-10-26 13:38:05 +00:00
┌─res──────────────────────────────┐
│ [1,1,3,11,12,12,12,5,6,14,14,14] │
└──────────────────────────────────┘
```
2020-03-20 10:10:48 +00:00
Note that the first argument (lambda function) cant be omitted in the `arrayFill` function.
2019-10-26 13:38:05 +00:00
### arrayReverseFill(func, arr1, …) {#higher_order_functions-array-reverse-fill}
2019-10-26 13:38:05 +00:00
Scan through `arr1` from the last element to the first element and replace `arr1[i]` by `arr1[i + 1]` if `func` returns 0. The last element of `arr1` will not be replaced.
Examples:
2020-03-20 10:10:48 +00:00
``` sql
2019-10-26 13:38:05 +00:00
SELECT arrayReverseFill(x -> not isNull(x), [1, null, 3, 11, 12, null, null, 5, 6, 14, null, null]) AS res
```
2020-03-20 10:10:48 +00:00
``` text
2019-10-26 13:38:05 +00:00
┌─res────────────────────────────────┐
│ [1,3,3,11,12,5,5,5,6,14,NULL,NULL] │
└────────────────────────────────────┘
```
2020-03-20 10:10:48 +00:00
Note that the first argument (lambda function) cant be omitted in the `arrayReverseFill` function.
2019-10-26 13:38:05 +00:00
### arraySplit(func, arr1, …) {#higher_order_functions-array-split}
2019-10-26 13:38:05 +00:00
Split `arr1` into multiple arrays. When `func` returns something other than 0, the array will be split on the left hand side of the element. The array will not be split before the first element.
Examples:
2020-03-20 10:10:48 +00:00
``` sql
2019-10-26 13:38:05 +00:00
SELECT arraySplit((x, y) -> y, [1, 2, 3, 4, 5], [1, 0, 0, 1, 0]) AS res
```
2020-03-20 10:10:48 +00:00
``` text
2019-10-26 13:38:05 +00:00
┌─res─────────────┐
│ [[1,2,3],[4,5]] │
└─────────────────┘
```
2020-03-20 10:10:48 +00:00
Note that the first argument (lambda function) cant be omitted in the `arraySplit` function.
2019-10-26 13:38:05 +00:00
### arrayReverseSplit(func, arr1, …) {#higher_order_functions-array-reverse-split}
2019-10-26 13:38:05 +00:00
Split `arr1` into multiple arrays. When `func` returns something other than 0, the array will be split on the right hand side of the element. The array will not be split after the last element.
Examples:
2020-03-20 10:10:48 +00:00
``` sql
2019-10-26 13:38:05 +00:00
SELECT arrayReverseSplit((x, y) -> y, [1, 2, 3, 4, 5], [1, 0, 0, 1, 0]) AS res
```
2020-03-20 10:10:48 +00:00
``` text
2019-10-26 13:38:05 +00:00
┌─res───────────────┐
│ [[1],[2,3,4],[5]] │
└───────────────────┘
```
2020-03-20 10:10:48 +00:00
Note that the first argument (lambda function) cant be omitted in the `arraySplit` function.
2019-10-26 13:38:05 +00:00
### arrayCount(\[func,\] arr1, …) {#higher_order_functions-array-count}
2020-03-20 10:10:48 +00:00
Returns the number of elements in the arr array for which func returns something other than 0. If func is not specified, it returns the number of non-zero elements in the array.
2020-03-20 10:10:48 +00:00
### arrayExists(\[func,\] arr1, …) {#arrayexistsfunc-arr1}
2020-03-20 10:10:48 +00:00
Returns 1 if there is at least one element in arr for which func returns something other than 0. Otherwise, it returns 0.
2020-03-20 10:10:48 +00:00
### arrayAll(\[func,\] arr1, …) {#arrayallfunc-arr1}
2020-03-20 10:10:48 +00:00
Returns 1 if func returns something other than 0 for all the elements in arr. Otherwise, it returns 0.
### arraySum(\[func,\] arr1, …) {#higher-order-functions-array-sum}
2020-03-20 10:10:48 +00:00
Returns the sum of the func values. If the function is omitted, it just returns the sum of the array elements.
### arrayFirst(func, arr1, …) {#higher_order_functions-array-first}
2020-03-20 10:10:48 +00:00
Returns the first element in the arr1 array for which func returns something other than 0.
2020-03-20 10:10:48 +00:00
Note that the first argument (lambda function) cant be omitted in the `arrayFirst` function.
2019-06-06 14:13:23 +00:00
### arrayFirstIndex(func, arr1, …) {#higher_order_functions-array-first-index}
2020-03-20 10:10:48 +00:00
Returns the index of the first element in the arr1 array for which func returns something other than 0.
2020-03-20 10:10:48 +00:00
Note that the first argument (lambda function) cant be omitted in the `arrayFirstIndex` function.
2019-06-06 14:13:23 +00:00
2020-03-20 10:10:48 +00:00
### arrayCumSum(\[func,\] arr1, …) {#arraycumsumfunc-arr1}
2018-02-15 10:13:46 +00:00
2018-03-25 02:04:22 +00:00
Returns an array of partial sums of elements in the source array (a running sum). If the `func` function is specified, then the values of the array elements are converted by this function before summing.
2018-02-15 10:13:46 +00:00
Example:
2020-03-20 10:10:48 +00:00
``` sql
2018-02-15 10:13:46 +00:00
SELECT arrayCumSum([1, 1, 1, 1]) AS res
```
2020-03-20 10:10:48 +00:00
``` text
2018-02-15 10:13:46 +00:00
┌─res──────────┐
│ [1, 2, 3, 4] │
└──────────────┘
```
2020-03-20 10:10:48 +00:00
### arrayCumSumNonNegative(arr) {#arraycumsumnonnegativearr}
2019-04-24 14:35:52 +00:00
Same as `arrayCumSum`, returns an array of partial sums of elements in the source array (a running sum). Different `arrayCumSum`, when then returned value contains a value less than zero, the value is replace with zero and the subsequent calculation is performed with zero parameters. For example:
2020-03-20 10:10:48 +00:00
``` sql
SELECT arrayCumSumNonNegative([1, 1, -4, 1]) AS res
```
2020-03-20 10:10:48 +00:00
``` text
┌─res───────┐
│ [1,2,0,1] │
└───────────┘
```
2020-03-20 10:10:48 +00:00
### arraySort(\[func,\] arr1, …) {#arraysortfunc-arr1}
2019-10-26 13:38:05 +00:00
Returns an array as result of sorting the elements of `arr1` in ascending order. If the `func` function is specified, sorting order is determined by the result of the function `func` applied to the elements of array (arrays)
2019-04-19 15:55:23 +00:00
The [Schwartzian transform](https://en.wikipedia.org/wiki/Schwartzian_transform) is used to improve sorting efficiency.
Example:
2020-03-20 10:10:48 +00:00
``` sql
SELECT arraySort((x, y) -> y, ['hello', 'world'], [2, 1]);
```
2020-03-20 10:10:48 +00:00
``` text
┌─res────────────────┐
│ ['world', 'hello'] │
└────────────────────┘
```
For more information about the `arraySort` method, see the [Functions for Working With Arrays](../../sql-reference/functions/array-functions.md#array_functions-sort) section.
2020-03-20 10:10:48 +00:00
### arrayReverseSort(\[func,\] arr1, …) {#arrayreversesortfunc-arr1}
2019-04-24 14:35:52 +00:00
Returns an array as result of sorting the elements of `arr1` in descending order. If the `func` function is specified, sorting order is determined by the result of the function `func` applied to the elements of array (arrays).
Example:
2018-09-06 18:07:25 +00:00
2020-03-20 10:10:48 +00:00
``` sql
2019-04-24 14:35:52 +00:00
SELECT arrayReverseSort((x, y) -> y, ['hello', 'world'], [2, 1]) as res;
```
2020-03-20 10:10:48 +00:00
``` text
2019-04-24 14:35:52 +00:00
┌─res───────────────┐
│ ['hello','world'] │
└───────────────────┘
```
2018-09-06 18:07:25 +00:00
For more information about the `arrayReverseSort` method, see the [Functions for Working With Arrays](../../sql-reference/functions/array-functions.md#array_functions-reverse-sort) section.
2020-01-30 10:34:55 +00:00
[Original article](https://clickhouse.tech/docs/en/query_language/functions/higher_order_functions/) <!--hide-->