mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-22 23:52:03 +00:00
Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2
This commit is contained in:
commit
853f2ea123
@ -16,6 +16,6 @@ ClickHouse® is an open-source column-oriented database management system that a
|
|||||||
* [Contacts](https://clickhouse.com/company/contact) can help to get your questions answered if there are any.
|
* [Contacts](https://clickhouse.com/company/contact) can help to get your questions answered if there are any.
|
||||||
|
|
||||||
## Upcoming events
|
## Upcoming events
|
||||||
* [**v22.12 Release Webinar**](https://clickhouse.com/company/events/v22-12-release-webinar) 22.12 is the ClickHouse Christmas release. There are plenty of gifts (a new JOIN algorithm among them) and we adopted something from MongoDB. Original creator, co-founder, and CTO of ClickHouse Alexey Milovidov will walk us through the highlights of the release.
|
* **Recording available**: [**v22.12 Release Webinar**](https://www.youtube.com/watch?v=sREupr6uc2k) 22.12 is the ClickHouse Christmas release. There are plenty of gifts (a new JOIN algorithm among them) and we adopted something from MongoDB. Original creator, co-founder, and CTO of ClickHouse Alexey Milovidov will walk us through the highlights of the release.
|
||||||
* [**ClickHouse Meetup at the CHEQ office in Tel Aviv**](https://www.meetup.com/clickhouse-tel-aviv-user-group/events/289599423/) - Jan 16 - We are very excited to be holding our next in-person ClickHouse meetup at the CHEQ office in Tel Aviv! Hear from CHEQ, ServiceNow and Contentsquare, as well as a deep dive presentation from ClickHouse CTO Alexey Milovidov. Join us for a fun evening of talks, food and discussion!
|
* [**ClickHouse Meetup at the CHEQ office in Tel Aviv**](https://www.meetup.com/clickhouse-tel-aviv-user-group/events/289599423/) - Jan 16 - We are very excited to be holding our next in-person ClickHouse meetup at the CHEQ office in Tel Aviv! Hear from CHEQ, ServiceNow and Contentsquare, as well as a deep dive presentation from ClickHouse CTO Alexey Milovidov. Join us for a fun evening of talks, food and discussion!
|
||||||
* [**ClickHouse Meetup at Microsoft Office in Seattle**](https://www.meetup.com/clickhouse-seattle-user-group/events/290310025/) - Jan 18 - Keep an eye on this space as we will be announcing speakers soon!
|
* [**ClickHouse Meetup at Microsoft Office in Seattle**](https://www.meetup.com/clickhouse-seattle-user-group/events/290310025/) - Jan 18 - Keep an eye on this space as we will be announcing speakers soon!
|
||||||
|
@ -1,15 +1,15 @@
|
|||||||
---
|
---
|
||||||
slug: /en/development/build-cross-osx
|
slug: /en/development/build-cross-osx
|
||||||
sidebar_position: 66
|
sidebar_position: 66
|
||||||
title: How to Build ClickHouse on Linux for Mac OS X
|
title: How to Build ClickHouse on Linux for macOS
|
||||||
sidebar_label: Build on Linux for Mac OS X
|
sidebar_label: Build on Linux for macOS
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
||||||
This is for the case when you have a Linux machine and want to use it to build `clickhouse` binary that will run on OS X.
|
This is for the case when you have a Linux machine and want to use it to build `clickhouse` binary that will run on OS X.
|
||||||
This is intended for continuous integration checks that run on Linux servers. If you want to build ClickHouse directly on Mac OS X, then proceed with [another instruction](../development/build-osx.md).
|
This is intended for continuous integration checks that run on Linux servers. If you want to build ClickHouse directly on macOS, then proceed with [another instruction](../development/build-osx.md).
|
||||||
|
|
||||||
The cross-build for Mac OS X is based on the [Build instructions](../development/build.md), follow them first.
|
The cross-build for macOS is based on the [Build instructions](../development/build.md), follow them first.
|
||||||
|
|
||||||
## Install Clang-14
|
## Install Clang-14
|
||||||
|
|
||||||
|
@ -1,9 +1,9 @@
|
|||||||
---
|
---
|
||||||
slug: /en/development/build-osx
|
slug: /en/development/build-osx
|
||||||
sidebar_position: 65
|
sidebar_position: 65
|
||||||
sidebar_label: Build on Mac OS X
|
sidebar_label: Build on macOS
|
||||||
title: How to Build ClickHouse on Mac OS X
|
title: How to Build ClickHouse on macOS
|
||||||
description: How to build ClickHouse on Mac OS X
|
description: How to build ClickHouse on macOS
|
||||||
---
|
---
|
||||||
|
|
||||||
:::info You don't have to build ClickHouse yourself!
|
:::info You don't have to build ClickHouse yourself!
|
||||||
|
@ -7,7 +7,7 @@ description: Prerequisites and an overview of how to build ClickHouse
|
|||||||
|
|
||||||
# Getting Started Guide for Building ClickHouse
|
# Getting Started Guide for Building ClickHouse
|
||||||
|
|
||||||
The building of ClickHouse is supported on Linux, FreeBSD and Mac OS X.
|
The building of ClickHouse is supported on Linux, FreeBSD and macOS.
|
||||||
|
|
||||||
If you use Windows, you need to create a virtual machine with Ubuntu. To start working with a virtual machine please install VirtualBox. You can download Ubuntu from the website: https://www.ubuntu.com/#download. Please create a virtual machine from the downloaded image (you should reserve at least 4GB of RAM for it). To run a command-line terminal in Ubuntu, please locate a program containing the word “terminal” in its name (gnome-terminal, konsole etc.) or just press Ctrl+Alt+T.
|
If you use Windows, you need to create a virtual machine with Ubuntu. To start working with a virtual machine please install VirtualBox. You can download Ubuntu from the website: https://www.ubuntu.com/#download. Please create a virtual machine from the downloaded image (you should reserve at least 4GB of RAM for it). To run a command-line terminal in Ubuntu, please locate a program containing the word “terminal” in its name (gnome-terminal, konsole etc.) or just press Ctrl+Alt+T.
|
||||||
|
|
||||||
@ -194,7 +194,7 @@ In this case, ClickHouse will use config files located in the current directory.
|
|||||||
|
|
||||||
To connect to ClickHouse with clickhouse-client in another terminal navigate to `ClickHouse/build/programs/` and run `./clickhouse client`.
|
To connect to ClickHouse with clickhouse-client in another terminal navigate to `ClickHouse/build/programs/` and run `./clickhouse client`.
|
||||||
|
|
||||||
If you get `Connection refused` message on Mac OS X or FreeBSD, try specifying host address 127.0.0.1:
|
If you get `Connection refused` message on macOS or FreeBSD, try specifying host address 127.0.0.1:
|
||||||
|
|
||||||
clickhouse client --host 127.0.0.1
|
clickhouse client --host 127.0.0.1
|
||||||
|
|
||||||
@ -213,7 +213,7 @@ You can also run your custom-built ClickHouse binary with the config file from t
|
|||||||
|
|
||||||
## IDE (Integrated Development Environment) {#ide-integrated-development-environment}
|
## IDE (Integrated Development Environment) {#ide-integrated-development-environment}
|
||||||
|
|
||||||
If you do not know which IDE to use, we recommend that you use CLion. CLion is commercial software, but it offers 30 days free trial period. It is also free of charge for students. CLion can be used both on Linux and on Mac OS X.
|
If you do not know which IDE to use, we recommend that you use CLion. CLion is commercial software, but it offers 30 days free trial period. It is also free of charge for students. CLion can be used both on Linux and on macOS.
|
||||||
|
|
||||||
KDevelop and QTCreator are other great alternatives of an IDE for developing ClickHouse. KDevelop comes in as a very handy IDE although unstable. If KDevelop crashes after a while upon opening project, you should click “Stop All” button as soon as it has opened the list of project’s files. After doing so KDevelop should be fine to work with.
|
KDevelop and QTCreator are other great alternatives of an IDE for developing ClickHouse. KDevelop comes in as a very handy IDE although unstable. If KDevelop crashes after a while upon opening project, you should click “Stop All” button as soon as it has opened the list of project’s files. After doing so KDevelop should be fine to work with.
|
||||||
|
|
||||||
|
@ -139,7 +139,7 @@ If the system clickhouse-server is already running and you do not want to stop i
|
|||||||
Build tests allow to check that build is not broken on various alternative configurations and on some foreign systems. These tests are automated as well.
|
Build tests allow to check that build is not broken on various alternative configurations and on some foreign systems. These tests are automated as well.
|
||||||
|
|
||||||
Examples:
|
Examples:
|
||||||
- cross-compile for Darwin x86_64 (Mac OS X)
|
- cross-compile for Darwin x86_64 (macOS)
|
||||||
- cross-compile for FreeBSD x86_64
|
- cross-compile for FreeBSD x86_64
|
||||||
- cross-compile for Linux AArch64
|
- cross-compile for Linux AArch64
|
||||||
- build on Ubuntu with libraries from system packages (discouraged)
|
- build on Ubuntu with libraries from system packages (discouraged)
|
||||||
|
@ -9,7 +9,7 @@ slug: /en/install
|
|||||||
You have three options for getting up and running with ClickHouse:
|
You have three options for getting up and running with ClickHouse:
|
||||||
|
|
||||||
- **[ClickHouse Cloud](https://clickhouse.com/cloud/):** The official ClickHouse as a service, - built by, maintained and supported by the creators of ClickHouse
|
- **[ClickHouse Cloud](https://clickhouse.com/cloud/):** The official ClickHouse as a service, - built by, maintained and supported by the creators of ClickHouse
|
||||||
- **[Self-managed ClickHouse](#self-managed-install):** ClickHouse can run on any Linux, FreeBSD, or Mac OS X with x86-64, ARM, or PowerPC64LE CPU architecture
|
- **[Self-managed ClickHouse](#self-managed-install):** ClickHouse can run on any Linux, FreeBSD, or macOS with x86-64, ARM, or PowerPC64LE CPU architecture
|
||||||
- **[Docker Image](https://hub.docker.com/r/clickhouse/clickhouse-server/):** Read the guide with the official image in Docker Hub
|
- **[Docker Image](https://hub.docker.com/r/clickhouse/clickhouse-server/):** Read the guide with the official image in Docker Hub
|
||||||
|
|
||||||
## ClickHouse Cloud
|
## ClickHouse Cloud
|
||||||
@ -257,7 +257,7 @@ To run ClickHouse inside Docker follow the guide on [Docker Hub](https://hub.doc
|
|||||||
|
|
||||||
### From Sources {#from-sources}
|
### From Sources {#from-sources}
|
||||||
|
|
||||||
To manually compile ClickHouse, follow the instructions for [Linux](/docs/en/development/build.md) or [Mac OS X](/docs/en/development/build-osx.md).
|
To manually compile ClickHouse, follow the instructions for [Linux](/docs/en/development/build.md) or [macOS](/docs/en/development/build-osx.md).
|
||||||
|
|
||||||
You can compile packages and install them or use programs without installing packages.
|
You can compile packages and install them or use programs without installing packages.
|
||||||
|
|
||||||
@ -352,7 +352,7 @@ To continue experimenting, you can download one of the test data sets or go thro
|
|||||||
|
|
||||||
## Recommendations for Self-Managed ClickHouse
|
## Recommendations for Self-Managed ClickHouse
|
||||||
|
|
||||||
ClickHouse can run on any Linux, FreeBSD, or Mac OS X with x86-64, ARM, or PowerPC64LE CPU architecture.
|
ClickHouse can run on any Linux, FreeBSD, or macOS with x86-64, ARM, or PowerPC64LE CPU architecture.
|
||||||
|
|
||||||
ClickHouse uses all hardware resources available to process data.
|
ClickHouse uses all hardware resources available to process data.
|
||||||
|
|
||||||
|
@ -890,7 +890,7 @@ The maximum number of open files.
|
|||||||
|
|
||||||
By default: `maximum`.
|
By default: `maximum`.
|
||||||
|
|
||||||
We recommend using this option in Mac OS X since the `getrlimit()` function returns an incorrect value.
|
We recommend using this option in macOS since the `getrlimit()` function returns an incorrect value.
|
||||||
|
|
||||||
**Example**
|
**Example**
|
||||||
|
|
||||||
|
@ -3447,13 +3447,45 @@ Default value: 2.
|
|||||||
|
|
||||||
## compatibility {#compatibility}
|
## compatibility {#compatibility}
|
||||||
|
|
||||||
This setting changes other settings according to provided ClickHouse version.
|
The `compatibility` setting causes ClickHouse to use the default settings of a previous version of ClickHouse, where the previous version is provided as the setting.
|
||||||
If a behaviour in ClickHouse was changed by using a different default value for some setting, this compatibility setting allows you to use default values from previous versions for all the settings that were not set by the user.
|
|
||||||
|
|
||||||
This setting takes ClickHouse version number as a string, like `21.3`, `21.8`. Empty value means that this setting is disabled.
|
If settings are set to non-default values, then those settings are honored (only settings that have not been modified are affected by the `compatibility` setting).
|
||||||
|
|
||||||
|
This setting takes a ClickHouse version number as a string, like `22.3`, `22.8`. An empty value means that this setting is disabled.
|
||||||
|
|
||||||
Disabled by default.
|
Disabled by default.
|
||||||
|
|
||||||
|
:::note
|
||||||
|
In ClickHouse Cloud the compatibility setting must be set by ClickHouse Cloud support. Please [open a case](https://clickhouse.cloud/support) to have it set.
|
||||||
|
:::
|
||||||
|
|
||||||
|
## allow_settings_after_format_in_insert {#allow_settings_after_format_in_insert}
|
||||||
|
|
||||||
|
Control whether `SETTINGS` after `FORMAT` in `INSERT` queries is allowed or not. It is not recommended to use this, since this may interpret part of `SETTINGS` as values.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
INSERT INTO FUNCTION null('foo String') SETTINGS max_threads=1 VALUES ('bar');
|
||||||
|
```
|
||||||
|
|
||||||
|
But the following query will work only with `allow_settings_after_format_in_insert`:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SET allow_settings_after_format_in_insert=1;
|
||||||
|
INSERT INTO FUNCTION null('foo String') VALUES ('bar') SETTINGS max_threads=1;
|
||||||
|
```
|
||||||
|
|
||||||
|
Possible values:
|
||||||
|
|
||||||
|
- 0 — Disallow.
|
||||||
|
- 1 — Allow.
|
||||||
|
|
||||||
|
Default value: `0`.
|
||||||
|
|
||||||
|
!!! note "Warning"
|
||||||
|
Use this setting only for backward compatibility if your use cases depend on old syntax.
|
||||||
|
|
||||||
# Format settings {#format-settings}
|
# Format settings {#format-settings}
|
||||||
|
|
||||||
## input_format_skip_unknown_fields {#input_format_skip_unknown_fields}
|
## input_format_skip_unknown_fields {#input_format_skip_unknown_fields}
|
||||||
|
@ -1104,6 +1104,7 @@ Using replacement fields, you can define a pattern for the resulting string. “
|
|||||||
| %d | day of the month, zero-padded (01-31) | 02 |
|
| %d | day of the month, zero-padded (01-31) | 02 |
|
||||||
| %D | Short MM/DD/YY date, equivalent to %m/%d/%y | 01/02/18 |
|
| %D | Short MM/DD/YY date, equivalent to %m/%d/%y | 01/02/18 |
|
||||||
| %e | day of the month, space-padded ( 1-31) | 2 |
|
| %e | day of the month, space-padded ( 1-31) | 2 |
|
||||||
|
| %f | fractional second from the fractional part of DateTime64 | 1234560 |
|
||||||
| %F | short YYYY-MM-DD date, equivalent to %Y-%m-%d | 2018-01-02 |
|
| %F | short YYYY-MM-DD date, equivalent to %Y-%m-%d | 2018-01-02 |
|
||||||
| %G | four-digit year format for ISO week number, calculated from the week-based year [defined by the ISO 8601](https://en.wikipedia.org/wiki/ISO_8601#Week_dates) standard, normally useful only with %V | 2018 |
|
| %G | four-digit year format for ISO week number, calculated from the week-based year [defined by the ISO 8601](https://en.wikipedia.org/wiki/ISO_8601#Week_dates) standard, normally useful only with %V | 2018 |
|
||||||
| %g | two-digit year format, aligned to ISO 8601, abbreviated from four-digit notation | 18 |
|
| %g | two-digit year format, aligned to ISO 8601, abbreviated from four-digit notation | 18 |
|
||||||
@ -1143,6 +1144,20 @@ Result:
|
|||||||
└────────────────────────────────────────────┘
|
└────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT formatDateTime(toDateTime64('2010-01-04 12:34:56.123456', 7), '%f')
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─formatDateTime(toDateTime64('2010-01-04 12:34:56.123456', 7), '%f')─┐
|
||||||
|
│ 1234560 │
|
||||||
|
└─────────────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
## dateName
|
## dateName
|
||||||
|
|
||||||
Returns specified part of date.
|
Returns specified part of date.
|
||||||
|
@ -595,9 +595,9 @@ SELECT xxHash64('')
|
|||||||
|
|
||||||
**Returned value**
|
**Returned value**
|
||||||
|
|
||||||
A `Uint32` or `Uint64` data type hash value.
|
A `UInt32` or `UInt64` data type hash value.
|
||||||
|
|
||||||
Type: `xxHash`.
|
Type: `UInt32` for `xxHash32` and `UInt64` for `xxHash64`.
|
||||||
|
|
||||||
**Example**
|
**Example**
|
||||||
|
|
||||||
|
@ -68,6 +68,440 @@ Result:
|
|||||||
└────────────┴────────────┴──────────────┴────────────────┴─────────────────┴──────────────────────┘
|
└────────────┴────────────┴──────────────┴────────────────┴─────────────────┴──────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
|
# Functions for Generating Random Numbers based on Distributions
|
||||||
|
|
||||||
|
:::note
|
||||||
|
These functions are available starting from 22.10.
|
||||||
|
:::
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## randNormal
|
||||||
|
|
||||||
|
Return random number based on [normal distribution](https://en.wikipedia.org/wiki/Normal_distribution).
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
randNormal(meam, variance)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `meam` - `Float64` mean value of distribution,
|
||||||
|
- `variance` - `Float64` - [variance](https://en.wikipedia.org/wiki/Variance).
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Pseudo-random number.
|
||||||
|
|
||||||
|
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT randNormal(10, 2) FROM numbers(5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌──randNormal(10, 2)─┐
|
||||||
|
│ 13.389228911709653 │
|
||||||
|
│ 8.622949707401295 │
|
||||||
|
│ 10.801887062682981 │
|
||||||
|
│ 4.5220192605895315 │
|
||||||
|
│ 10.901239123982567 │
|
||||||
|
└────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## randLogNormal
|
||||||
|
|
||||||
|
Return random number based on [log-normal distribution](https://en.wikipedia.org/wiki/Log-normal_distribution).
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
randLogNormal(meam, variance)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `meam` - `Float64` mean value of distribution,
|
||||||
|
- `variance` - `Float64` - [variance](https://en.wikipedia.org/wiki/Variance).
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Pseudo-random number.
|
||||||
|
|
||||||
|
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT randLogNormal(100, 5) FROM numbers(5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌─randLogNormal(100, 5)─┐
|
||||||
|
│ 1.295699673937363e48 │
|
||||||
|
│ 9.719869109186684e39 │
|
||||||
|
│ 6.110868203189557e42 │
|
||||||
|
│ 9.912675872925529e39 │
|
||||||
|
│ 2.3564708490552458e42 │
|
||||||
|
└───────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## randBinomial
|
||||||
|
|
||||||
|
Return random number based on [binomial distribution](https://en.wikipedia.org/wiki/Binomial_distribution).
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
randBinomial(experiments, probability)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `experiments` - `UInt64` number of experiments,
|
||||||
|
- `probability` - `Float64` - probability of success in each experiment (values in `0...1` range only).
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Pseudo-random number.
|
||||||
|
|
||||||
|
Type: [UInt64](/docs/en/sql-reference/data-types/int-uint.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT randBinomial(100, .75) FROM numbers(5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌─randBinomial(100, 0.75)─┐
|
||||||
|
│ 74 │
|
||||||
|
│ 78 │
|
||||||
|
│ 76 │
|
||||||
|
│ 77 │
|
||||||
|
│ 80 │
|
||||||
|
└─────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## randNegativeBinomial
|
||||||
|
|
||||||
|
Return random number based on [negative binomial distribution](https://en.wikipedia.org/wiki/Negative_binomial_distribution).
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
randNegativeBinomial(experiments, probability)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `experiments` - `UInt64` number of experiments,
|
||||||
|
- `probability` - `Float64` - probability of failure in each experiment (values in `0...1` range only).
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Pseudo-random number.
|
||||||
|
|
||||||
|
Type: [UInt64](/docs/en/sql-reference/data-types/int-uint.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT randNegativeBinomial(100, .75) FROM numbers(5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌─randNegativeBinomial(100, 0.75)─┐
|
||||||
|
│ 33 │
|
||||||
|
│ 32 │
|
||||||
|
│ 39 │
|
||||||
|
│ 40 │
|
||||||
|
│ 50 │
|
||||||
|
└─────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## randPoisson
|
||||||
|
|
||||||
|
Return random number based on [Poisson distribution](https://en.wikipedia.org/wiki/Poisson_distribution).
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
randPoisson(n)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `n` - `UInt64` mean number of occurrences.
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Pseudo-random number.
|
||||||
|
|
||||||
|
Type: [UInt64](/docs/en/sql-reference/data-types/int-uint.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT randPoisson(10) FROM numbers(5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌─randPoisson(10)─┐
|
||||||
|
│ 8 │
|
||||||
|
│ 8 │
|
||||||
|
│ 7 │
|
||||||
|
│ 10 │
|
||||||
|
│ 6 │
|
||||||
|
└─────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## randBernoulli
|
||||||
|
|
||||||
|
Return random number based on [Bernoulli distribution](https://en.wikipedia.org/wiki/Bernoulli_distribution).
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
randBernoulli(probability)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `probability` - `Float64` - probability of success (values in `0...1` range only).
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Pseudo-random number.
|
||||||
|
|
||||||
|
Type: [UInt64](/docs/en/sql-reference/data-types/int-uint.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT randBernoulli(.75) FROM numbers(5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌─randBernoulli(0.75)─┐
|
||||||
|
│ 1 │
|
||||||
|
│ 1 │
|
||||||
|
│ 0 │
|
||||||
|
│ 1 │
|
||||||
|
│ 1 │
|
||||||
|
└─────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## randExponential
|
||||||
|
|
||||||
|
Return random number based on [exponential distribution](https://en.wikipedia.org/wiki/Exponential_distribution).
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
randExponential(lambda)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `lambda` - `Float64` lambda value.
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Pseudo-random number.
|
||||||
|
|
||||||
|
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT randExponential(1/10) FROM numbers(5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌─randExponential(divide(1, 10))─┐
|
||||||
|
│ 44.71628934340778 │
|
||||||
|
│ 4.211013337903262 │
|
||||||
|
│ 10.809402553207766 │
|
||||||
|
│ 15.63959406553284 │
|
||||||
|
│ 1.8148392319860158 │
|
||||||
|
└────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## randChiSquared
|
||||||
|
|
||||||
|
Return random number based on [Chi-square distribution](https://en.wikipedia.org/wiki/Chi-squared_distribution) - a distribution of a sum of the squares of k independent standard normal random variables.
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
randChiSquared(degree_of_freedom)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `degree_of_freedom` - `Float64` degree of freedom.
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Pseudo-random number.
|
||||||
|
|
||||||
|
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT randChiSquared(10) FROM numbers(5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌─randChiSquared(10)─┐
|
||||||
|
│ 10.015463656521543 │
|
||||||
|
│ 9.621799919882768 │
|
||||||
|
│ 2.71785015634699 │
|
||||||
|
│ 11.128188665931908 │
|
||||||
|
│ 4.902063104425469 │
|
||||||
|
└────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## randStudentT
|
||||||
|
|
||||||
|
Return random number based on [Student's t-distribution](https://en.wikipedia.org/wiki/Student%27s_t-distribution).
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
randStudentT(degree_of_freedom)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `degree_of_freedom` - `Float64` degree of freedom.
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Pseudo-random number.
|
||||||
|
|
||||||
|
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT randStudentT(10) FROM numbers(5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌─────randStudentT(10)─┐
|
||||||
|
│ 1.2217309938538725 │
|
||||||
|
│ 1.7941971681200541 │
|
||||||
|
│ -0.28192176076784664 │
|
||||||
|
│ 0.2508897721303792 │
|
||||||
|
│ -2.7858432909761186 │
|
||||||
|
└──────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## randFisherF
|
||||||
|
|
||||||
|
Return random number based on [F-distribution](https://en.wikipedia.org/wiki/F-distribution).
|
||||||
|
|
||||||
|
**Syntax**
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
randFisherF(d1, d2)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Arguments**
|
||||||
|
|
||||||
|
- `d1` - `Float64` d1 degree of freedom in `X = (S1 / d1) / (S2 / d2)`,
|
||||||
|
- `d2` - `Float64` d2 degree of freedom in `X = (S1 / d1) / (S2 / d2)`,
|
||||||
|
|
||||||
|
**Returned value**
|
||||||
|
|
||||||
|
- Pseudo-random number.
|
||||||
|
|
||||||
|
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||||
|
|
||||||
|
**Example**
|
||||||
|
|
||||||
|
Query:
|
||||||
|
|
||||||
|
``` sql
|
||||||
|
SELECT randFisherF(10, 3) FROM numbers(5)
|
||||||
|
```
|
||||||
|
|
||||||
|
Result:
|
||||||
|
|
||||||
|
``` text
|
||||||
|
┌──randFisherF(10, 3)─┐
|
||||||
|
│ 7.286287504216609 │
|
||||||
|
│ 0.26590779413050386 │
|
||||||
|
│ 0.22207610901168987 │
|
||||||
|
│ 0.7953362728449572 │
|
||||||
|
│ 0.19278885985221572 │
|
||||||
|
└─────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# Random Functions for Working with Strings
|
# Random Functions for Working with Strings
|
||||||
|
|
||||||
## randomString
|
## randomString
|
||||||
|
@ -21,12 +21,11 @@ Subquery is another `SELECT` query that may be specified in parenthesis inside `
|
|||||||
|
|
||||||
When `FINAL` is specified, ClickHouse fully merges the data before returning the result and thus performs all data transformations that happen during merges for the given table engine.
|
When `FINAL` is specified, ClickHouse fully merges the data before returning the result and thus performs all data transformations that happen during merges for the given table engine.
|
||||||
|
|
||||||
It is applicable when selecting data from tables that use the [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md)-engine family. Also supported for:
|
It is applicable when selecting data from ReplacingMergeTree, SummingMergeTree, AggregatingMergeTree, CollapsingMergeTree and VersionedCollapsingMergeTree tables.
|
||||||
|
|
||||||
- [Replicated](../../../engines/table-engines/mergetree-family/replication.md) versions of `MergeTree` engines.
|
`SELECT` queries with `FINAL` are executed in parallel. The [max_final_threads](../../../operations/settings/settings.md#max-final-threads) setting limits the number of threads used.
|
||||||
- [View](../../../engines/table-engines/special/view.md), [Buffer](../../../engines/table-engines/special/buffer.md), [Distributed](../../../engines/table-engines/special/distributed.md), and [MaterializedView](../../../engines/table-engines/special/materializedview.md) engines that operate over other engines, provided they were created over `MergeTree`-engine tables.
|
|
||||||
|
|
||||||
Now `SELECT` queries with `FINAL` are executed in parallel and slightly faster. But there are drawbacks (see below). The [max_final_threads](../../../operations/settings/settings.md#max-final-threads) setting limits the number of threads used.
|
There are drawbacks to using `FINAL` (see below).
|
||||||
|
|
||||||
### Drawbacks
|
### Drawbacks
|
||||||
|
|
||||||
|
@ -49,14 +49,16 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionThrow(const DataTypes & argument_types_, const Array & parameters_, Float64 throw_probability_)
|
AggregateFunctionThrow(const DataTypes & argument_types_, const Array & parameters_, Float64 throw_probability_)
|
||||||
: IAggregateFunctionDataHelper(argument_types_, parameters_), throw_probability(throw_probability_) {}
|
: IAggregateFunctionDataHelper(argument_types_, parameters_, createResultType())
|
||||||
|
, throw_probability(throw_probability_)
|
||||||
|
{}
|
||||||
|
|
||||||
String getName() const override
|
String getName() const override
|
||||||
{
|
{
|
||||||
return "aggThrow";
|
return "aggThrow";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeUInt8>();
|
return std::make_shared<DataTypeUInt8>();
|
||||||
}
|
}
|
||||||
|
@ -37,10 +37,10 @@ class AggregateFunctionAnalysisOfVariance final : public IAggregateFunctionDataH
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionAnalysisOfVariance(const DataTypes & arguments, const Array & params)
|
explicit AggregateFunctionAnalysisOfVariance(const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionDataHelper(arguments, params)
|
: IAggregateFunctionDataHelper(arguments, params, createResultType())
|
||||||
{}
|
{}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
DataTypePtr createResultType() const
|
||||||
{
|
{
|
||||||
DataTypes types {std::make_shared<DataTypeNumber<Float64>>(), std::make_shared<DataTypeNumber<Float64>>() };
|
DataTypes types {std::make_shared<DataTypeNumber<Float64>>(), std::make_shared<DataTypeNumber<Float64>>() };
|
||||||
Strings names {"f_statistic", "p_value"};
|
Strings names {"f_statistic", "p_value"};
|
||||||
|
@ -38,7 +38,6 @@ template <typename Data>
|
|||||||
class AggregateFunctionArgMinMax final : public IAggregateFunctionDataHelper<Data, AggregateFunctionArgMinMax<Data>>
|
class AggregateFunctionArgMinMax final : public IAggregateFunctionDataHelper<Data, AggregateFunctionArgMinMax<Data>>
|
||||||
{
|
{
|
||||||
private:
|
private:
|
||||||
const DataTypePtr & type_res;
|
|
||||||
const DataTypePtr & type_val;
|
const DataTypePtr & type_val;
|
||||||
const SerializationPtr serialization_res;
|
const SerializationPtr serialization_res;
|
||||||
const SerializationPtr serialization_val;
|
const SerializationPtr serialization_val;
|
||||||
@ -47,10 +46,9 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionArgMinMax(const DataTypePtr & type_res_, const DataTypePtr & type_val_)
|
AggregateFunctionArgMinMax(const DataTypePtr & type_res_, const DataTypePtr & type_val_)
|
||||||
: Base({type_res_, type_val_}, {})
|
: Base({type_res_, type_val_}, {}, type_res_)
|
||||||
, type_res(this->argument_types[0])
|
|
||||||
, type_val(this->argument_types[1])
|
, type_val(this->argument_types[1])
|
||||||
, serialization_res(type_res->getDefaultSerialization())
|
, serialization_res(type_res_->getDefaultSerialization())
|
||||||
, serialization_val(type_val->getDefaultSerialization())
|
, serialization_val(type_val->getDefaultSerialization())
|
||||||
{
|
{
|
||||||
if (!type_val->isComparable())
|
if (!type_val->isComparable())
|
||||||
@ -63,11 +61,6 @@ public:
|
|||||||
return StringRef(Data::ValueData_t::name()) == StringRef("min") ? "argMin" : "argMax";
|
return StringRef(Data::ValueData_t::name()) == StringRef("min") ? "argMin" : "argMax";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return type_res;
|
|
||||||
}
|
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||||
{
|
{
|
||||||
if (this->data(place).value.changeIfBetter(*columns[1], row_num, arena))
|
if (this->data(place).value.changeIfBetter(*columns[1], row_num, arena))
|
||||||
|
@ -30,7 +30,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionArray(AggregateFunctionPtr nested_, const DataTypes & arguments, const Array & params_)
|
AggregateFunctionArray(AggregateFunctionPtr nested_, const DataTypes & arguments, const Array & params_)
|
||||||
: IAggregateFunctionHelper<AggregateFunctionArray>(arguments, params_)
|
: IAggregateFunctionHelper<AggregateFunctionArray>(arguments, params_, createResultType(nested_))
|
||||||
, nested_func(nested_), num_arguments(arguments.size())
|
, nested_func(nested_), num_arguments(arguments.size())
|
||||||
{
|
{
|
||||||
assert(parameters == nested_func->getParameters());
|
assert(parameters == nested_func->getParameters());
|
||||||
@ -44,9 +44,9 @@ public:
|
|||||||
return nested_func->getName() + "Array";
|
return nested_func->getName() + "Array";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const AggregateFunctionPtr & nested_)
|
||||||
{
|
{
|
||||||
return nested_func->getReturnType();
|
return nested_->getResultType();
|
||||||
}
|
}
|
||||||
|
|
||||||
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
||||||
|
@ -10,6 +10,7 @@
|
|||||||
#include <AggregateFunctions/IAggregateFunction.h>
|
#include <AggregateFunctions/IAggregateFunction.h>
|
||||||
#include <AggregateFunctions/AggregateFunctionSum.h>
|
#include <AggregateFunctions/AggregateFunctionSum.h>
|
||||||
#include <Core/DecimalFunctions.h>
|
#include <Core/DecimalFunctions.h>
|
||||||
|
#include <Core/IResolvedFunction.h>
|
||||||
|
|
||||||
#include "config.h"
|
#include "config.h"
|
||||||
|
|
||||||
@ -84,9 +85,19 @@ public:
|
|||||||
|
|
||||||
explicit AggregateFunctionAvgBase(const DataTypes & argument_types_,
|
explicit AggregateFunctionAvgBase(const DataTypes & argument_types_,
|
||||||
UInt32 num_scale_ = 0, UInt32 denom_scale_ = 0)
|
UInt32 num_scale_ = 0, UInt32 denom_scale_ = 0)
|
||||||
: Base(argument_types_, {}), num_scale(num_scale_), denom_scale(denom_scale_) {}
|
: Base(argument_types_, {}, createResultType())
|
||||||
|
, num_scale(num_scale_)
|
||||||
|
, denom_scale(denom_scale_)
|
||||||
|
{}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeNumber<Float64>>(); }
|
AggregateFunctionAvgBase(const DataTypes & argument_types_, const DataTypePtr & result_type_,
|
||||||
|
UInt32 num_scale_ = 0, UInt32 denom_scale_ = 0)
|
||||||
|
: Base(argument_types_, {}, result_type_)
|
||||||
|
, num_scale(num_scale_)
|
||||||
|
, denom_scale(denom_scale_)
|
||||||
|
{}
|
||||||
|
|
||||||
|
DataTypePtr createResultType() const { return std::make_shared<DataTypeNumber<Float64>>(); }
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
@ -135,7 +146,7 @@ public:
|
|||||||
for (const auto & argument : this->argument_types)
|
for (const auto & argument : this->argument_types)
|
||||||
can_be_compiled &= canBeNativeType(*argument);
|
can_be_compiled &= canBeNativeType(*argument);
|
||||||
|
|
||||||
auto return_type = getReturnType();
|
auto return_type = this->getResultType();
|
||||||
can_be_compiled &= canBeNativeType(*return_type);
|
can_be_compiled &= canBeNativeType(*return_type);
|
||||||
|
|
||||||
return can_be_compiled;
|
return can_be_compiled;
|
||||||
|
@ -97,11 +97,12 @@ class AggregateFunctionBitwise final : public IAggregateFunctionDataHelper<Data,
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionBitwise(const DataTypePtr & type)
|
explicit AggregateFunctionBitwise(const DataTypePtr & type)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitwise<T, Data>>({type}, {}) {}
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitwise<T, Data>>({type}, {}, createResultType())
|
||||||
|
{}
|
||||||
|
|
||||||
String getName() const override { return Data::name(); }
|
String getName() const override { return Data::name(); }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeNumber<T>>();
|
return std::make_shared<DataTypeNumber<T>>();
|
||||||
}
|
}
|
||||||
@ -137,7 +138,7 @@ public:
|
|||||||
|
|
||||||
bool isCompilable() const override
|
bool isCompilable() const override
|
||||||
{
|
{
|
||||||
auto return_type = getReturnType();
|
auto return_type = this->getResultType();
|
||||||
return canBeNativeType(*return_type);
|
return canBeNativeType(*return_type);
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -151,7 +152,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
|
|
||||||
auto * value_ptr = aggregate_data_ptr;
|
auto * value_ptr = aggregate_data_ptr;
|
||||||
auto * value = b.CreateLoad(return_type, value_ptr);
|
auto * value = b.CreateLoad(return_type, value_ptr);
|
||||||
@ -166,7 +167,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
|
|
||||||
auto * value_dst_ptr = aggregate_data_dst_ptr;
|
auto * value_dst_ptr = aggregate_data_dst_ptr;
|
||||||
auto * value_dst = b.CreateLoad(return_type, value_dst_ptr);
|
auto * value_dst = b.CreateLoad(return_type, value_dst_ptr);
|
||||||
@ -183,7 +184,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
auto * value_ptr = aggregate_data_ptr;
|
auto * value_ptr = aggregate_data_ptr;
|
||||||
|
|
||||||
return b.CreateLoad(return_type, value_ptr);
|
return b.CreateLoad(return_type, value_ptr);
|
||||||
|
@ -112,7 +112,7 @@ public:
|
|||||||
}
|
}
|
||||||
|
|
||||||
explicit AggregateFunctionBoundingRatio(const DataTypes & arguments)
|
explicit AggregateFunctionBoundingRatio(const DataTypes & arguments)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionBoundingRatioData, AggregateFunctionBoundingRatio>(arguments, {})
|
: IAggregateFunctionDataHelper<AggregateFunctionBoundingRatioData, AggregateFunctionBoundingRatio>(arguments, {}, std::make_shared<DataTypeFloat64>())
|
||||||
{
|
{
|
||||||
const auto * x_arg = arguments.at(0).get();
|
const auto * x_arg = arguments.at(0).get();
|
||||||
const auto * y_arg = arguments.at(1).get();
|
const auto * y_arg = arguments.at(1).get();
|
||||||
@ -122,11 +122,6 @@ public:
|
|||||||
ErrorCodes::BAD_ARGUMENTS);
|
ErrorCodes::BAD_ARGUMENTS);
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeFloat64>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, const size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, const size_t row_num, Arena *) const override
|
||||||
|
@ -46,9 +46,9 @@ private:
|
|||||||
}
|
}
|
||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionCategoricalIV(const DataTypes & arguments_, const Array & params_) :
|
AggregateFunctionCategoricalIV(const DataTypes & arguments_, const Array & params_)
|
||||||
IAggregateFunctionHelper<AggregateFunctionCategoricalIV>{arguments_, params_},
|
: IAggregateFunctionHelper<AggregateFunctionCategoricalIV>{arguments_, params_, createResultType()}
|
||||||
category_count{arguments_.size() - 1}
|
, category_count{arguments_.size() - 1}
|
||||||
{
|
{
|
||||||
// notice: argument types has been checked before
|
// notice: argument types has been checked before
|
||||||
}
|
}
|
||||||
@ -121,7 +121,7 @@ public:
|
|||||||
buf.readStrict(place, sizeOfData());
|
buf.readStrict(place, sizeOfData());
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeArray>(
|
return std::make_shared<DataTypeArray>(
|
||||||
std::make_shared<DataTypeNumber<Float64>>());
|
std::make_shared<DataTypeNumber<Float64>>());
|
||||||
|
@ -39,11 +39,13 @@ namespace ErrorCodes
|
|||||||
class AggregateFunctionCount final : public IAggregateFunctionDataHelper<AggregateFunctionCountData, AggregateFunctionCount>
|
class AggregateFunctionCount final : public IAggregateFunctionDataHelper<AggregateFunctionCountData, AggregateFunctionCount>
|
||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionCount(const DataTypes & argument_types_) : IAggregateFunctionDataHelper(argument_types_, {}) {}
|
explicit AggregateFunctionCount(const DataTypes & argument_types_)
|
||||||
|
: IAggregateFunctionDataHelper(argument_types_, {}, createResultType())
|
||||||
|
{}
|
||||||
|
|
||||||
String getName() const override { return "count"; }
|
String getName() const override { return "count"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeUInt64>();
|
return std::make_shared<DataTypeUInt64>();
|
||||||
}
|
}
|
||||||
@ -167,7 +169,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
|
|
||||||
auto * count_value_ptr = aggregate_data_ptr;
|
auto * count_value_ptr = aggregate_data_ptr;
|
||||||
auto * count_value = b.CreateLoad(return_type, count_value_ptr);
|
auto * count_value = b.CreateLoad(return_type, count_value_ptr);
|
||||||
@ -180,7 +182,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
|
|
||||||
auto * count_value_dst_ptr = aggregate_data_dst_ptr;
|
auto * count_value_dst_ptr = aggregate_data_dst_ptr;
|
||||||
auto * count_value_dst = b.CreateLoad(return_type, count_value_dst_ptr);
|
auto * count_value_dst = b.CreateLoad(return_type, count_value_dst_ptr);
|
||||||
@ -197,7 +199,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
auto * count_value_ptr = aggregate_data_ptr;
|
auto * count_value_ptr = aggregate_data_ptr;
|
||||||
|
|
||||||
return b.CreateLoad(return_type, count_value_ptr);
|
return b.CreateLoad(return_type, count_value_ptr);
|
||||||
@ -214,7 +216,7 @@ class AggregateFunctionCountNotNullUnary final
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
AggregateFunctionCountNotNullUnary(const DataTypePtr & argument, const Array & params)
|
AggregateFunctionCountNotNullUnary(const DataTypePtr & argument, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionCountData, AggregateFunctionCountNotNullUnary>({argument}, params)
|
: IAggregateFunctionDataHelper<AggregateFunctionCountData, AggregateFunctionCountNotNullUnary>({argument}, params, createResultType())
|
||||||
{
|
{
|
||||||
if (!argument->isNullable())
|
if (!argument->isNullable())
|
||||||
throw Exception("Logical error: not Nullable data type passed to AggregateFunctionCountNotNullUnary", ErrorCodes::LOGICAL_ERROR);
|
throw Exception("Logical error: not Nullable data type passed to AggregateFunctionCountNotNullUnary", ErrorCodes::LOGICAL_ERROR);
|
||||||
@ -222,7 +224,7 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return "count"; }
|
String getName() const override { return "count"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeUInt64>();
|
return std::make_shared<DataTypeUInt64>();
|
||||||
}
|
}
|
||||||
@ -311,7 +313,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
|
|
||||||
auto * is_null_value = b.CreateExtractValue(values[0], {1});
|
auto * is_null_value = b.CreateExtractValue(values[0], {1});
|
||||||
auto * increment_value = b.CreateSelect(is_null_value, llvm::ConstantInt::get(return_type, 0), llvm::ConstantInt::get(return_type, 1));
|
auto * increment_value = b.CreateSelect(is_null_value, llvm::ConstantInt::get(return_type, 0), llvm::ConstantInt::get(return_type, 1));
|
||||||
@ -327,7 +329,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
|
|
||||||
auto * count_value_dst_ptr = aggregate_data_dst_ptr;
|
auto * count_value_dst_ptr = aggregate_data_dst_ptr;
|
||||||
auto * count_value_dst = b.CreateLoad(return_type, count_value_dst_ptr);
|
auto * count_value_dst = b.CreateLoad(return_type, count_value_dst_ptr);
|
||||||
@ -344,7 +346,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
auto * count_value_ptr = aggregate_data_ptr;
|
auto * count_value_ptr = aggregate_data_ptr;
|
||||||
|
|
||||||
return b.CreateLoad(return_type, count_value_ptr);
|
return b.CreateLoad(return_type, count_value_ptr);
|
||||||
|
@ -31,7 +31,7 @@ class AggregationFunctionDeltaSum final
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
AggregationFunctionDeltaSum(const DataTypes & arguments, const Array & params)
|
AggregationFunctionDeltaSum(const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<AggregationFunctionDeltaSumData<T>, AggregationFunctionDeltaSum<T>>{arguments, params}
|
: IAggregateFunctionDataHelper<AggregationFunctionDeltaSumData<T>, AggregationFunctionDeltaSum<T>>{arguments, params, createResultType()}
|
||||||
{}
|
{}
|
||||||
|
|
||||||
AggregationFunctionDeltaSum()
|
AggregationFunctionDeltaSum()
|
||||||
@ -40,7 +40,7 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return "deltaSum"; }
|
String getName() const override { return "deltaSum"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeNumber<T>>(); }
|
static DataTypePtr createResultType() { return std::make_shared<DataTypeNumber<T>>(); }
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
|
@ -38,7 +38,7 @@ public:
|
|||||||
: IAggregateFunctionDataHelper<
|
: IAggregateFunctionDataHelper<
|
||||||
AggregationFunctionDeltaSumTimestampData<ValueType, TimestampType>,
|
AggregationFunctionDeltaSumTimestampData<ValueType, TimestampType>,
|
||||||
AggregationFunctionDeltaSumTimestamp<ValueType, TimestampType>
|
AggregationFunctionDeltaSumTimestamp<ValueType, TimestampType>
|
||||||
>{arguments, params}
|
>{arguments, params, createResultType()}
|
||||||
{}
|
{}
|
||||||
|
|
||||||
AggregationFunctionDeltaSumTimestamp()
|
AggregationFunctionDeltaSumTimestamp()
|
||||||
@ -52,7 +52,7 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return "deltaSumTimestamp"; }
|
String getName() const override { return "deltaSumTimestamp"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeNumber<ValueType>>(); }
|
static DataTypePtr createResultType() { return std::make_shared<DataTypeNumber<ValueType>>(); }
|
||||||
|
|
||||||
void NO_SANITIZE_UNDEFINED ALWAYS_INLINE add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
void NO_SANITIZE_UNDEFINED ALWAYS_INLINE add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||||
{
|
{
|
||||||
|
@ -168,7 +168,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionDistinct(AggregateFunctionPtr nested_func_, const DataTypes & arguments, const Array & params_)
|
AggregateFunctionDistinct(AggregateFunctionPtr nested_func_, const DataTypes & arguments, const Array & params_)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionDistinct>(arguments, params_)
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionDistinct>(arguments, params_, nested_func_->getResultType())
|
||||||
, nested_func(nested_func_)
|
, nested_func(nested_func_)
|
||||||
, arguments_num(arguments.size())
|
, arguments_num(arguments.size())
|
||||||
{
|
{
|
||||||
@ -255,11 +255,6 @@ public:
|
|||||||
return nested_func->getName() + "Distinct";
|
return nested_func->getName() + "Distinct";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return nested_func->getReturnType();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override
|
bool allocatesMemoryInArena() const override
|
||||||
{
|
{
|
||||||
return true;
|
return true;
|
||||||
|
@ -92,14 +92,14 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionEntropy(const DataTypes & argument_types_)
|
explicit AggregateFunctionEntropy(const DataTypes & argument_types_)
|
||||||
: IAggregateFunctionDataHelper<EntropyData<Value>, AggregateFunctionEntropy<Value>>(argument_types_, {})
|
: IAggregateFunctionDataHelper<EntropyData<Value>, AggregateFunctionEntropy<Value>>(argument_types_, {}, createResultType())
|
||||||
, num_args(argument_types_.size())
|
, num_args(argument_types_.size())
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
String getName() const override { return "entropy"; }
|
String getName() const override { return "entropy"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeNumber<Float64>>();
|
return std::make_shared<DataTypeNumber<Float64>>();
|
||||||
}
|
}
|
||||||
|
@ -29,7 +29,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionExponentialMovingAverage(const DataTypes & argument_types_, const Array & params)
|
AggregateFunctionExponentialMovingAverage(const DataTypes & argument_types_, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<ExponentiallySmoothedAverage, AggregateFunctionExponentialMovingAverage>(argument_types_, params)
|
: IAggregateFunctionDataHelper<ExponentiallySmoothedAverage, AggregateFunctionExponentialMovingAverage>(argument_types_, params, createResultType())
|
||||||
{
|
{
|
||||||
if (params.size() != 1)
|
if (params.size() != 1)
|
||||||
throw Exception{"Aggregate function " + getName() + " requires exactly one parameter: half decay time.",
|
throw Exception{"Aggregate function " + getName() + " requires exactly one parameter: half decay time.",
|
||||||
@ -43,7 +43,7 @@ public:
|
|||||||
return "exponentialMovingAverage";
|
return "exponentialMovingAverage";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeNumber<Float64>>();
|
return std::make_shared<DataTypeNumber<Float64>>();
|
||||||
}
|
}
|
||||||
|
@ -523,12 +523,12 @@ class AggregateFunctionFlameGraph final : public IAggregateFunctionDataHelper<Ag
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionFlameGraph(const DataTypes & argument_types_)
|
explicit AggregateFunctionFlameGraph(const DataTypes & argument_types_)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionFlameGraphData, AggregateFunctionFlameGraph>(argument_types_, {})
|
: IAggregateFunctionDataHelper<AggregateFunctionFlameGraphData, AggregateFunctionFlameGraph>(argument_types_, {}, createResultType())
|
||||||
{}
|
{}
|
||||||
|
|
||||||
String getName() const override { return "flameGraph"; }
|
String getName() const override { return "flameGraph"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeArray>(std::make_shared<DataTypeString>());
|
return std::make_shared<DataTypeArray>(std::make_shared<DataTypeString>());
|
||||||
}
|
}
|
||||||
|
@ -107,7 +107,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionForEach(AggregateFunctionPtr nested_, const DataTypes & arguments, const Array & params_)
|
AggregateFunctionForEach(AggregateFunctionPtr nested_, const DataTypes & arguments, const Array & params_)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionForEachData, AggregateFunctionForEach>(arguments, params_)
|
: IAggregateFunctionDataHelper<AggregateFunctionForEachData, AggregateFunctionForEach>(arguments, params_, createResultType(nested_))
|
||||||
, nested_func(nested_), num_arguments(arguments.size())
|
, nested_func(nested_), num_arguments(arguments.size())
|
||||||
{
|
{
|
||||||
nested_size_of_data = nested_func->sizeOfData();
|
nested_size_of_data = nested_func->sizeOfData();
|
||||||
@ -125,9 +125,9 @@ public:
|
|||||||
return nested_func->getName() + "ForEach";
|
return nested_func->getName() + "ForEach";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(AggregateFunctionPtr nested_)
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeArray>(nested_func->getReturnType());
|
return std::make_shared<DataTypeArray>(nested_->getResultType());
|
||||||
}
|
}
|
||||||
|
|
||||||
bool isVersioned() const override
|
bool isVersioned() const override
|
||||||
|
@ -121,7 +121,7 @@ public:
|
|||||||
explicit GroupArrayNumericImpl(
|
explicit GroupArrayNumericImpl(
|
||||||
const DataTypePtr & data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max(), UInt64 seed_ = 123456)
|
const DataTypePtr & data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max(), UInt64 seed_ = 123456)
|
||||||
: IAggregateFunctionDataHelper<GroupArrayNumericData<T, Trait::sampler != Sampler::NONE>, GroupArrayNumericImpl<T, Trait>>(
|
: IAggregateFunctionDataHelper<GroupArrayNumericData<T, Trait::sampler != Sampler::NONE>, GroupArrayNumericImpl<T, Trait>>(
|
||||||
{data_type_}, parameters_)
|
{data_type_}, parameters_, std::make_shared<DataTypeArray>(data_type_))
|
||||||
, max_elems(max_elems_)
|
, max_elems(max_elems_)
|
||||||
, seed(seed_)
|
, seed(seed_)
|
||||||
{
|
{
|
||||||
@ -129,8 +129,6 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return getNameByTrait<Trait>(); }
|
String getName() const override { return getNameByTrait<Trait>(); }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(this->argument_types[0]); }
|
|
||||||
|
|
||||||
void insert(Data & a, const T & v, Arena * arena) const
|
void insert(Data & a, const T & v, Arena * arena) const
|
||||||
{
|
{
|
||||||
++a.total_values;
|
++a.total_values;
|
||||||
@ -423,7 +421,7 @@ class GroupArrayGeneralImpl final
|
|||||||
public:
|
public:
|
||||||
GroupArrayGeneralImpl(const DataTypePtr & data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max(), UInt64 seed_ = 123456)
|
GroupArrayGeneralImpl(const DataTypePtr & data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max(), UInt64 seed_ = 123456)
|
||||||
: IAggregateFunctionDataHelper<GroupArrayGeneralData<Node, Trait::sampler != Sampler::NONE>, GroupArrayGeneralImpl<Node, Trait>>(
|
: IAggregateFunctionDataHelper<GroupArrayGeneralData<Node, Trait::sampler != Sampler::NONE>, GroupArrayGeneralImpl<Node, Trait>>(
|
||||||
{data_type_}, parameters_)
|
{data_type_}, parameters_, std::make_shared<DataTypeArray>(data_type_))
|
||||||
, data_type(this->argument_types[0])
|
, data_type(this->argument_types[0])
|
||||||
, max_elems(max_elems_)
|
, max_elems(max_elems_)
|
||||||
, seed(seed_)
|
, seed(seed_)
|
||||||
@ -432,8 +430,6 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return getNameByTrait<Trait>(); }
|
String getName() const override { return getNameByTrait<Trait>(); }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(data_type); }
|
|
||||||
|
|
||||||
void insert(Data & a, const Node * v, Arena * arena) const
|
void insert(Data & a, const Node * v, Arena * arena) const
|
||||||
{
|
{
|
||||||
++a.total_values;
|
++a.total_values;
|
||||||
@ -697,7 +693,7 @@ class GroupArrayGeneralListImpl final
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
GroupArrayGeneralListImpl(const DataTypePtr & data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
GroupArrayGeneralListImpl(const DataTypePtr & data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||||
: IAggregateFunctionDataHelper<GroupArrayGeneralListData<Node>, GroupArrayGeneralListImpl<Node, Trait>>({data_type_}, parameters_)
|
: IAggregateFunctionDataHelper<GroupArrayGeneralListData<Node>, GroupArrayGeneralListImpl<Node, Trait>>({data_type_}, parameters_, std::make_shared<DataTypeArray>(data_type_))
|
||||||
, data_type(this->argument_types[0])
|
, data_type(this->argument_types[0])
|
||||||
, max_elems(max_elems_)
|
, max_elems(max_elems_)
|
||||||
{
|
{
|
||||||
@ -705,8 +701,6 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return getNameByTrait<Trait>(); }
|
String getName() const override { return getNameByTrait<Trait>(); }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(data_type); }
|
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||||
{
|
{
|
||||||
if (limit_num_elems && data(place).elems >= max_elems)
|
if (limit_num_elems && data(place).elems >= max_elems)
|
||||||
|
@ -64,7 +64,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionGroupArrayInsertAtGeneric(const DataTypes & arguments, const Array & params)
|
AggregateFunctionGroupArrayInsertAtGeneric(const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionGroupArrayInsertAtDataGeneric, AggregateFunctionGroupArrayInsertAtGeneric>(arguments, params)
|
: IAggregateFunctionDataHelper<AggregateFunctionGroupArrayInsertAtDataGeneric, AggregateFunctionGroupArrayInsertAtGeneric>(arguments, params, std::make_shared<DataTypeArray>(arguments[0]))
|
||||||
, type(argument_types[0])
|
, type(argument_types[0])
|
||||||
, serialization(type->getDefaultSerialization())
|
, serialization(type->getDefaultSerialization())
|
||||||
{
|
{
|
||||||
@ -101,11 +101,6 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return "groupArrayInsertAt"; }
|
String getName() const override { return "groupArrayInsertAt"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeArray>(type);
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||||
|
@ -93,12 +93,15 @@ public:
|
|||||||
using ColumnResult = ColumnVectorOrDecimal<ResultT>;
|
using ColumnResult = ColumnVectorOrDecimal<ResultT>;
|
||||||
|
|
||||||
explicit MovingImpl(const DataTypePtr & data_type_, UInt64 window_size_ = std::numeric_limits<UInt64>::max())
|
explicit MovingImpl(const DataTypePtr & data_type_, UInt64 window_size_ = std::numeric_limits<UInt64>::max())
|
||||||
: IAggregateFunctionDataHelper<Data, MovingImpl<T, LimitNumElements, Data>>({data_type_}, {})
|
: IAggregateFunctionDataHelper<Data, MovingImpl<T, LimitNumElements, Data>>({data_type_}, {}, createResultType(data_type_))
|
||||||
, window_size(window_size_) {}
|
, window_size(window_size_) {}
|
||||||
|
|
||||||
String getName() const override { return Data::name; }
|
String getName() const override { return Data::name; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(getReturnTypeElement()); }
|
static DataTypePtr createResultType(const DataTypePtr & argument)
|
||||||
|
{
|
||||||
|
return std::make_shared<DataTypeArray>(getReturnTypeElement(argument));
|
||||||
|
}
|
||||||
|
|
||||||
void NO_SANITIZE_UNDEFINED add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
void NO_SANITIZE_UNDEFINED add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||||
{
|
{
|
||||||
@ -183,14 +186,14 @@ public:
|
|||||||
}
|
}
|
||||||
|
|
||||||
private:
|
private:
|
||||||
auto getReturnTypeElement() const
|
static auto getReturnTypeElement(const DataTypePtr & argument)
|
||||||
{
|
{
|
||||||
if constexpr (!is_decimal<ResultT>)
|
if constexpr (!is_decimal<ResultT>)
|
||||||
return std::make_shared<DataTypeNumber<ResultT>>();
|
return std::make_shared<DataTypeNumber<ResultT>>();
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
using Res = DataTypeDecimal<ResultT>;
|
using Res = DataTypeDecimal<ResultT>;
|
||||||
return std::make_shared<Res>(Res::maxPrecision(), getDecimalScale(*this->argument_types.at(0)));
|
return std::make_shared<Res>(Res::maxPrecision(), getDecimalScale(*argument));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
@ -19,13 +19,13 @@ class AggregateFunctionBitmap final : public IAggregateFunctionDataHelper<Data,
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionBitmap(const DataTypePtr & type)
|
explicit AggregateFunctionBitmap(const DataTypePtr & type)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitmap<T, Data>>({type}, {})
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitmap<T, Data>>({type}, {}, createResultType())
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
String getName() const override { return Data::name(); }
|
String getName() const override { return Data::name(); }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeNumber<T>>(); }
|
static DataTypePtr createResultType() { return std::make_shared<DataTypeNumber<T>>(); }
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
@ -59,13 +59,13 @@ private:
|
|||||||
static constexpr size_t STATE_VERSION_1_MIN_REVISION = 54455;
|
static constexpr size_t STATE_VERSION_1_MIN_REVISION = 54455;
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionBitmapL2(const DataTypePtr & type)
|
explicit AggregateFunctionBitmapL2(const DataTypePtr & type)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitmapL2<T, Data, Policy>>({type}, {})
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitmapL2<T, Data, Policy>>({type}, {}, createResultType())
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
String getName() const override { return Policy::name; }
|
String getName() const override { return Policy::name; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeNumber<T>>(); }
|
static DataTypePtr createResultType() { return std::make_shared<DataTypeNumber<T>>(); }
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
|
@ -26,8 +26,8 @@ class AggregateFunctionGroupUniqArrayDate : public AggregateFunctionGroupUniqArr
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionGroupUniqArrayDate(const DataTypePtr & argument_type, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
explicit AggregateFunctionGroupUniqArrayDate(const DataTypePtr & argument_type, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||||
: AggregateFunctionGroupUniqArray<DataTypeDate::FieldType, HasLimit>(argument_type, parameters_, max_elems_) {}
|
: AggregateFunctionGroupUniqArray<DataTypeDate::FieldType, HasLimit>(argument_type, parameters_, createResultType(), max_elems_) {}
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDate>()); }
|
static DataTypePtr createResultType() { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDate>()); }
|
||||||
};
|
};
|
||||||
|
|
||||||
template <typename HasLimit>
|
template <typename HasLimit>
|
||||||
@ -35,8 +35,8 @@ class AggregateFunctionGroupUniqArrayDateTime : public AggregateFunctionGroupUni
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionGroupUniqArrayDateTime(const DataTypePtr & argument_type, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
explicit AggregateFunctionGroupUniqArrayDateTime(const DataTypePtr & argument_type, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||||
: AggregateFunctionGroupUniqArray<DataTypeDateTime::FieldType, HasLimit>(argument_type, parameters_, max_elems_) {}
|
: AggregateFunctionGroupUniqArray<DataTypeDateTime::FieldType, HasLimit>(argument_type, parameters_, createResultType(), max_elems_) {}
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDateTime>()); }
|
static DataTypePtr createResultType() { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDateTime>()); }
|
||||||
};
|
};
|
||||||
|
|
||||||
template <typename HasLimit, typename ... TArgs>
|
template <typename HasLimit, typename ... TArgs>
|
||||||
|
@ -50,15 +50,16 @@ private:
|
|||||||
public:
|
public:
|
||||||
AggregateFunctionGroupUniqArray(const DataTypePtr & argument_type, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
AggregateFunctionGroupUniqArray(const DataTypePtr & argument_type, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionGroupUniqArrayData<T>,
|
: IAggregateFunctionDataHelper<AggregateFunctionGroupUniqArrayData<T>,
|
||||||
AggregateFunctionGroupUniqArray<T, LimitNumElems>>({argument_type}, parameters_),
|
AggregateFunctionGroupUniqArray<T, LimitNumElems>>({argument_type}, parameters_, std::make_shared<DataTypeArray>(argument_type)),
|
||||||
max_elems(max_elems_) {}
|
max_elems(max_elems_) {}
|
||||||
|
|
||||||
String getName() const override { return "groupUniqArray"; }
|
AggregateFunctionGroupUniqArray(const DataTypePtr & argument_type, const Array & parameters_, const DataTypePtr & result_type_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||||
|
: IAggregateFunctionDataHelper<AggregateFunctionGroupUniqArrayData<T>,
|
||||||
|
AggregateFunctionGroupUniqArray<T, LimitNumElems>>({argument_type}, parameters_, result_type_),
|
||||||
|
max_elems(max_elems_) {}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
String getName() const override { return "groupUniqArray"; }
|
||||||
return std::make_shared<DataTypeArray>(this->argument_types[0]);
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
@ -153,17 +154,12 @@ class AggregateFunctionGroupUniqArrayGeneric
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionGroupUniqArrayGeneric(const DataTypePtr & input_data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
AggregateFunctionGroupUniqArrayGeneric(const DataTypePtr & input_data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionGroupUniqArrayGenericData, AggregateFunctionGroupUniqArrayGeneric<is_plain_column, LimitNumElems>>({input_data_type_}, parameters_)
|
: IAggregateFunctionDataHelper<AggregateFunctionGroupUniqArrayGenericData, AggregateFunctionGroupUniqArrayGeneric<is_plain_column, LimitNumElems>>({input_data_type_}, parameters_, std::make_shared<DataTypeArray>(input_data_type_))
|
||||||
, input_data_type(this->argument_types[0])
|
, input_data_type(this->argument_types[0])
|
||||||
, max_elems(max_elems_) {}
|
, max_elems(max_elems_) {}
|
||||||
|
|
||||||
String getName() const override { return "groupUniqArray"; }
|
String getName() const override { return "groupUniqArray"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeArray>(input_data_type);
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override
|
bool allocatesMemoryInArena() const override
|
||||||
{
|
{
|
||||||
return true;
|
return true;
|
||||||
|
@ -307,7 +307,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionHistogram(UInt32 max_bins_, const DataTypes & arguments, const Array & params)
|
AggregateFunctionHistogram(UInt32 max_bins_, const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionHistogramData, AggregateFunctionHistogram<T>>(arguments, params)
|
: IAggregateFunctionDataHelper<AggregateFunctionHistogramData, AggregateFunctionHistogram<T>>(arguments, params, createResultType())
|
||||||
, max_bins(max_bins_)
|
, max_bins(max_bins_)
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
@ -316,7 +316,7 @@ public:
|
|||||||
{
|
{
|
||||||
return Data::structSize(max_bins);
|
return Data::structSize(max_bins);
|
||||||
}
|
}
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
DataTypes types;
|
DataTypes types;
|
||||||
auto mean = std::make_shared<DataTypeNumber<Data::Mean>>();
|
auto mean = std::make_shared<DataTypeNumber<Data::Mean>>();
|
||||||
|
@ -448,7 +448,7 @@ AggregateFunctionPtr AggregateFunctionIf::getOwnNullAdapter(
|
|||||||
|
|
||||||
/// Nullability of the last argument (condition) does not affect the nullability of the result (NULL is processed as false).
|
/// Nullability of the last argument (condition) does not affect the nullability of the result (NULL is processed as false).
|
||||||
/// For other arguments it is as usual (at least one is NULL then the result is NULL if possible).
|
/// For other arguments it is as usual (at least one is NULL then the result is NULL if possible).
|
||||||
bool return_type_is_nullable = !properties.returns_default_when_only_null && getReturnType()->canBeInsideNullable()
|
bool return_type_is_nullable = !properties.returns_default_when_only_null && getResultType()->canBeInsideNullable()
|
||||||
&& std::any_of(arguments.begin(), arguments.end() - 1, [](const auto & element) { return element->isNullable(); });
|
&& std::any_of(arguments.begin(), arguments.end() - 1, [](const auto & element) { return element->isNullable(); });
|
||||||
|
|
||||||
bool need_to_serialize_flag = return_type_is_nullable || properties.returns_default_when_only_null;
|
bool need_to_serialize_flag = return_type_is_nullable || properties.returns_default_when_only_null;
|
||||||
|
@ -36,7 +36,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionIf(AggregateFunctionPtr nested, const DataTypes & types, const Array & params_)
|
AggregateFunctionIf(AggregateFunctionPtr nested, const DataTypes & types, const Array & params_)
|
||||||
: IAggregateFunctionHelper<AggregateFunctionIf>(types, params_)
|
: IAggregateFunctionHelper<AggregateFunctionIf>(types, params_, nested->getResultType())
|
||||||
, nested_func(nested), num_arguments(types.size())
|
, nested_func(nested), num_arguments(types.size())
|
||||||
{
|
{
|
||||||
if (num_arguments == 0)
|
if (num_arguments == 0)
|
||||||
@ -51,11 +51,6 @@ public:
|
|||||||
return nested_func->getName() + "If";
|
return nested_func->getName() + "If";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return nested_func->getReturnType();
|
|
||||||
}
|
|
||||||
|
|
||||||
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
||||||
{
|
{
|
||||||
return nested_func->getBaseAggregateFunctionWithSameStateRepresentation();
|
return nested_func->getBaseAggregateFunctionWithSameStateRepresentation();
|
||||||
|
@ -177,11 +177,11 @@ public:
|
|||||||
String getName() const override { return "intervalLengthSum"; }
|
String getName() const override { return "intervalLengthSum"; }
|
||||||
|
|
||||||
explicit AggregateFunctionIntervalLengthSum(const DataTypes & arguments)
|
explicit AggregateFunctionIntervalLengthSum(const DataTypes & arguments)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionIntervalLengthSum<T, Data>>(arguments, {})
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionIntervalLengthSum<T, Data>>(arguments, {}, createResultType())
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
if constexpr (std::is_floating_point_v<T>)
|
if constexpr (std::is_floating_point_v<T>)
|
||||||
return std::make_shared<DataTypeFloat64>();
|
return std::make_shared<DataTypeFloat64>();
|
||||||
|
@ -309,7 +309,7 @@ public:
|
|||||||
UInt64 batch_size_,
|
UInt64 batch_size_,
|
||||||
const DataTypes & arguments_types,
|
const DataTypes & arguments_types,
|
||||||
const Array & params)
|
const Array & params)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionMLMethod<Data, Name>>(arguments_types, params)
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionMLMethod<Data, Name>>(arguments_types, params, createResultType())
|
||||||
, param_num(param_num_)
|
, param_num(param_num_)
|
||||||
, learning_rate(learning_rate_)
|
, learning_rate(learning_rate_)
|
||||||
, l2_reg_coef(l2_reg_coef_)
|
, l2_reg_coef(l2_reg_coef_)
|
||||||
@ -319,8 +319,7 @@ public:
|
|||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
/// This function is called when SELECT linearRegression(...) is called
|
static DataTypePtr createResultType()
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeArray>(std::make_shared<DataTypeFloat64>());
|
return std::make_shared<DataTypeArray>(std::make_shared<DataTypeFloat64>());
|
||||||
}
|
}
|
||||||
|
@ -133,7 +133,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionMannWhitney(const DataTypes & arguments, const Array & params)
|
explicit AggregateFunctionMannWhitney(const DataTypes & arguments, const Array & params)
|
||||||
:IAggregateFunctionDataHelper<MannWhitneyData, AggregateFunctionMannWhitney> ({arguments}, {})
|
: IAggregateFunctionDataHelper<MannWhitneyData, AggregateFunctionMannWhitney> ({arguments}, {}, createResultType())
|
||||||
{
|
{
|
||||||
if (params.size() > 2)
|
if (params.size() > 2)
|
||||||
throw Exception("Aggregate function " + getName() + " require two parameter or less", ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH);
|
throw Exception("Aggregate function " + getName() + " require two parameter or less", ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH);
|
||||||
@ -174,7 +174,7 @@ public:
|
|||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return true; }
|
bool allocatesMemoryInArena() const override { return true; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
DataTypes types
|
DataTypes types
|
||||||
{
|
{
|
||||||
|
@ -18,6 +18,7 @@
|
|||||||
#include <Functions/FunctionHelpers.h>
|
#include <Functions/FunctionHelpers.h>
|
||||||
#include <IO/ReadHelpers.h>
|
#include <IO/ReadHelpers.h>
|
||||||
#include <IO/WriteHelpers.h>
|
#include <IO/WriteHelpers.h>
|
||||||
|
#include "DataTypes/Serializations/ISerialization.h"
|
||||||
#include "base/types.h"
|
#include "base/types.h"
|
||||||
#include <Common/Arena.h>
|
#include <Common/Arena.h>
|
||||||
#include "AggregateFunctions/AggregateFunctionFactory.h"
|
#include "AggregateFunctions/AggregateFunctionFactory.h"
|
||||||
@ -104,26 +105,32 @@ public:
|
|||||||
return nested_func->getDefaultVersion();
|
return nested_func->getDefaultVersion();
|
||||||
}
|
}
|
||||||
|
|
||||||
AggregateFunctionMap(AggregateFunctionPtr nested, const DataTypes & types) : Base(types, nested->getParameters()), nested_func(nested)
|
AggregateFunctionMap(AggregateFunctionPtr nested, const DataTypes & types)
|
||||||
|
: Base(types, nested->getParameters(), std::make_shared<DataTypeMap>(DataTypes{getKeyType(types, nested), nested->getResultType()}))
|
||||||
|
, nested_func(nested)
|
||||||
{
|
{
|
||||||
if (types.empty())
|
key_type = getKeyType(types, nested_func);
|
||||||
throw Exception(
|
|
||||||
ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH, "Aggregate function " + getName() + " requires at least one argument");
|
|
||||||
|
|
||||||
if (types.size() > 1)
|
|
||||||
throw Exception(
|
|
||||||
ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH, "Aggregate function " + getName() + " requires only one map argument");
|
|
||||||
|
|
||||||
const auto * map_type = checkAndGetDataType<DataTypeMap>(types[0].get());
|
|
||||||
if (!map_type)
|
|
||||||
throw Exception(ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT, "Aggregate function " + getName() + " requires map as argument");
|
|
||||||
|
|
||||||
key_type = map_type->getKeyType();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
String getName() const override { return nested_func->getName() + "Map"; }
|
String getName() const override { return nested_func->getName() + "Map"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeMap>(DataTypes{key_type, nested_func->getReturnType()}); }
|
static DataTypePtr getKeyType(const DataTypes & types, const AggregateFunctionPtr & nested)
|
||||||
|
{
|
||||||
|
if (types.empty())
|
||||||
|
throw Exception(ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
|
||||||
|
"Aggregate function {}Map requires at least one argument", nested->getName());
|
||||||
|
|
||||||
|
if (types.size() > 1)
|
||||||
|
throw Exception(ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
|
||||||
|
"Aggregate function {}Map requires only one map argument", nested->getName());
|
||||||
|
|
||||||
|
const auto * map_type = checkAndGetDataType<DataTypeMap>(types[0].get());
|
||||||
|
if (!map_type)
|
||||||
|
throw Exception(ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
||||||
|
"Aggregate function {}Map requires map as argument", nested->getName());
|
||||||
|
|
||||||
|
return map_type->getKeyType();
|
||||||
|
}
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||||
{
|
{
|
||||||
|
@ -62,7 +62,8 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionIntersectionsMax(AggregateFunctionIntersectionsKind kind_, const DataTypes & arguments)
|
AggregateFunctionIntersectionsMax(AggregateFunctionIntersectionsKind kind_, const DataTypes & arguments)
|
||||||
: IAggregateFunctionDataHelper<MaxIntersectionsData<PointType>, AggregateFunctionIntersectionsMax<PointType>>(arguments, {}), kind(kind_)
|
: IAggregateFunctionDataHelper<MaxIntersectionsData<PointType>, AggregateFunctionIntersectionsMax<PointType>>(arguments, {}, createResultType(kind_))
|
||||||
|
, kind(kind_)
|
||||||
{
|
{
|
||||||
if (!isNativeNumber(arguments[0]))
|
if (!isNativeNumber(arguments[0]))
|
||||||
throw Exception{getName() + ": first argument must be represented by integer", ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT};
|
throw Exception{getName() + ": first argument must be represented by integer", ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT};
|
||||||
@ -81,9 +82,9 @@ public:
|
|||||||
: "maxIntersectionsPosition";
|
: "maxIntersectionsPosition";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(AggregateFunctionIntersectionsKind kind_)
|
||||||
{
|
{
|
||||||
if (kind == AggregateFunctionIntersectionsKind::Count)
|
if (kind_ == AggregateFunctionIntersectionsKind::Count)
|
||||||
return std::make_shared<DataTypeUInt64>();
|
return std::make_shared<DataTypeUInt64>();
|
||||||
else
|
else
|
||||||
return std::make_shared<DataTypeNumber<PointType>>();
|
return std::make_shared<DataTypeNumber<PointType>>();
|
||||||
|
@ -36,7 +36,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionMeanZTest(const DataTypes & arguments, const Array & params)
|
AggregateFunctionMeanZTest(const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionMeanZTest<Data>>({arguments}, params)
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionMeanZTest<Data>>({arguments}, params, createResultType())
|
||||||
{
|
{
|
||||||
pop_var_x = params.at(0).safeGet<Float64>();
|
pop_var_x = params.at(0).safeGet<Float64>();
|
||||||
pop_var_y = params.at(1).safeGet<Float64>();
|
pop_var_y = params.at(1).safeGet<Float64>();
|
||||||
@ -63,7 +63,7 @@ public:
|
|||||||
return Data::name;
|
return Data::name;
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
DataTypes types
|
DataTypes types
|
||||||
{
|
{
|
||||||
|
@ -30,7 +30,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionMerge(const AggregateFunctionPtr & nested_, const DataTypePtr & argument, const Array & params_)
|
AggregateFunctionMerge(const AggregateFunctionPtr & nested_, const DataTypePtr & argument, const Array & params_)
|
||||||
: IAggregateFunctionHelper<AggregateFunctionMerge>({argument}, params_)
|
: IAggregateFunctionHelper<AggregateFunctionMerge>({argument}, params_, createResultType(nested_))
|
||||||
, nested_func(nested_)
|
, nested_func(nested_)
|
||||||
{
|
{
|
||||||
const DataTypeAggregateFunction * data_type = typeid_cast<const DataTypeAggregateFunction *>(argument.get());
|
const DataTypeAggregateFunction * data_type = typeid_cast<const DataTypeAggregateFunction *>(argument.get());
|
||||||
@ -45,9 +45,9 @@ public:
|
|||||||
return nested_func->getName() + "Merge";
|
return nested_func->getName() + "Merge";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const AggregateFunctionPtr & nested_)
|
||||||
{
|
{
|
||||||
return nested_func->getReturnType();
|
return nested_->getResultType();
|
||||||
}
|
}
|
||||||
|
|
||||||
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
||||||
|
@ -1222,7 +1222,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionsSingleValue(const DataTypePtr & type)
|
explicit AggregateFunctionsSingleValue(const DataTypePtr & type)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionsSingleValue<Data>>({type}, {})
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionsSingleValue<Data>>({type}, {}, createResultType(type))
|
||||||
, serialization(type->getDefaultSerialization())
|
, serialization(type->getDefaultSerialization())
|
||||||
{
|
{
|
||||||
if (StringRef(Data::name()) == StringRef("min")
|
if (StringRef(Data::name()) == StringRef("min")
|
||||||
@ -1236,12 +1236,11 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return Data::name(); }
|
String getName() const override { return Data::name(); }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const DataTypePtr & type_)
|
||||||
{
|
{
|
||||||
auto result_type = this->argument_types.at(0);
|
|
||||||
if constexpr (Data::is_nullable)
|
if constexpr (Data::is_nullable)
|
||||||
return makeNullable(result_type);
|
return makeNullable(type_);
|
||||||
return result_type;
|
return type_;
|
||||||
}
|
}
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||||
|
@ -6,6 +6,7 @@
|
|||||||
#include <AggregateFunctions/IAggregateFunction.h>
|
#include <AggregateFunctions/IAggregateFunction.h>
|
||||||
#include <IO/ReadHelpers.h>
|
#include <IO/ReadHelpers.h>
|
||||||
#include <IO/WriteHelpers.h>
|
#include <IO/WriteHelpers.h>
|
||||||
|
#include "DataTypes/IDataType.h"
|
||||||
|
|
||||||
|
|
||||||
namespace DB
|
namespace DB
|
||||||
@ -19,16 +20,16 @@ class AggregateFunctionNothing final : public IAggregateFunctionHelper<Aggregate
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
AggregateFunctionNothing(const DataTypes & arguments, const Array & params)
|
AggregateFunctionNothing(const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionHelper<AggregateFunctionNothing>(arguments, params) {}
|
: IAggregateFunctionHelper<AggregateFunctionNothing>(arguments, params, createResultType(arguments)) {}
|
||||||
|
|
||||||
String getName() const override
|
String getName() const override
|
||||||
{
|
{
|
||||||
return "nothing";
|
return "nothing";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const DataTypes & arguments)
|
||||||
{
|
{
|
||||||
return argument_types.empty() ? std::make_shared<DataTypeNullable>(std::make_shared<DataTypeNothing>()) : argument_types.front();
|
return arguments.empty() ? std::make_shared<DataTypeNullable>(std::make_shared<DataTypeNothing>()) : arguments.front();
|
||||||
}
|
}
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
@ -87,7 +87,7 @@ public:
|
|||||||
transformed_nested_function->getParameters());
|
transformed_nested_function->getParameters());
|
||||||
}
|
}
|
||||||
|
|
||||||
bool return_type_is_nullable = !properties.returns_default_when_only_null && nested_function->getReturnType()->canBeInsideNullable();
|
bool return_type_is_nullable = !properties.returns_default_when_only_null && nested_function->getResultType()->canBeInsideNullable();
|
||||||
bool serialize_flag = return_type_is_nullable || properties.returns_default_when_only_null;
|
bool serialize_flag = return_type_is_nullable || properties.returns_default_when_only_null;
|
||||||
|
|
||||||
if (arguments.size() == 1)
|
if (arguments.size() == 1)
|
||||||
|
@ -85,7 +85,8 @@ protected:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionNullBase(AggregateFunctionPtr nested_function_, const DataTypes & arguments, const Array & params)
|
AggregateFunctionNullBase(AggregateFunctionPtr nested_function_, const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionHelper<Derived>(arguments, params), nested_function{nested_function_}
|
: IAggregateFunctionHelper<Derived>(arguments, params, createResultType(nested_function_))
|
||||||
|
, nested_function{nested_function_}
|
||||||
{
|
{
|
||||||
if constexpr (result_is_nullable)
|
if constexpr (result_is_nullable)
|
||||||
prefix_size = nested_function->alignOfData();
|
prefix_size = nested_function->alignOfData();
|
||||||
@ -99,12 +100,12 @@ public:
|
|||||||
return nested_function->getName();
|
return nested_function->getName();
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const AggregateFunctionPtr & nested_function_)
|
||||||
{
|
{
|
||||||
if constexpr (result_is_nullable)
|
if constexpr (result_is_nullable)
|
||||||
return makeNullable(nested_function->getReturnType());
|
return makeNullable(nested_function_->getResultType());
|
||||||
else
|
else
|
||||||
return nested_function->getReturnType();
|
return nested_function_->getResultType();
|
||||||
}
|
}
|
||||||
|
|
||||||
void create(AggregateDataPtr __restrict place) const override
|
void create(AggregateDataPtr __restrict place) const override
|
||||||
@ -275,7 +276,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, this->getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
|
|
||||||
llvm::Value * result = nullptr;
|
llvm::Value * result = nullptr;
|
||||||
|
|
||||||
|
@ -30,16 +30,14 @@ private:
|
|||||||
AggregateFunctionPtr nested_function;
|
AggregateFunctionPtr nested_function;
|
||||||
|
|
||||||
size_t size_of_data;
|
size_t size_of_data;
|
||||||
DataTypePtr inner_type;
|
|
||||||
bool inner_nullable;
|
bool inner_nullable;
|
||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionOrFill(AggregateFunctionPtr nested_function_, const DataTypes & arguments, const Array & params)
|
AggregateFunctionOrFill(AggregateFunctionPtr nested_function_, const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionHelper<AggregateFunctionOrFill>{arguments, params}
|
: IAggregateFunctionHelper<AggregateFunctionOrFill>{arguments, params, createResultType(nested_function_->getResultType())}
|
||||||
, nested_function{nested_function_}
|
, nested_function{nested_function_}
|
||||||
, size_of_data {nested_function->sizeOfData()}
|
, size_of_data {nested_function->sizeOfData()}
|
||||||
, inner_type {nested_function->getReturnType()}
|
, inner_nullable {nested_function->getResultType()->isNullable()}
|
||||||
, inner_nullable {inner_type->isNullable()}
|
|
||||||
{
|
{
|
||||||
// nothing
|
// nothing
|
||||||
}
|
}
|
||||||
@ -246,22 +244,22 @@ public:
|
|||||||
readChar(place[size_of_data], buf);
|
readChar(place[size_of_data], buf);
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const DataTypePtr & inner_type_)
|
||||||
{
|
{
|
||||||
if constexpr (UseNull)
|
if constexpr (UseNull)
|
||||||
{
|
{
|
||||||
// -OrNull
|
// -OrNull
|
||||||
|
|
||||||
if (inner_nullable)
|
if (inner_type_->isNullable())
|
||||||
return inner_type;
|
return inner_type_;
|
||||||
|
|
||||||
return std::make_shared<DataTypeNullable>(inner_type);
|
return std::make_shared<DataTypeNullable>(inner_type_);
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
// -OrDefault
|
// -OrDefault
|
||||||
|
|
||||||
return inner_type;
|
return inner_type_;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -72,7 +72,7 @@ private:
|
|||||||
public:
|
public:
|
||||||
AggregateFunctionQuantile(const DataTypes & argument_types_, const Array & params)
|
AggregateFunctionQuantile(const DataTypes & argument_types_, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionQuantile<Value, Data, Name, has_second_arg, FloatReturnType, returns_many>>(
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionQuantile<Value, Data, Name, has_second_arg, FloatReturnType, returns_many>>(
|
||||||
argument_types_, params)
|
argument_types_, params, createResultType(argument_types_))
|
||||||
, levels(params, returns_many)
|
, levels(params, returns_many)
|
||||||
, level(levels.levels[0])
|
, level(levels.levels[0])
|
||||||
, argument_type(this->argument_types[0])
|
, argument_type(this->argument_types[0])
|
||||||
@ -83,14 +83,14 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return Name::name; }
|
String getName() const override { return Name::name; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const DataTypes & argument_types_)
|
||||||
{
|
{
|
||||||
DataTypePtr res;
|
DataTypePtr res;
|
||||||
|
|
||||||
if constexpr (returns_float)
|
if constexpr (returns_float)
|
||||||
res = std::make_shared<DataTypeNumber<FloatReturnType>>();
|
res = std::make_shared<DataTypeNumber<FloatReturnType>>();
|
||||||
else
|
else
|
||||||
res = argument_type;
|
res = argument_types_[0];
|
||||||
|
|
||||||
if constexpr (returns_many)
|
if constexpr (returns_many)
|
||||||
return std::make_shared<DataTypeArray>(res);
|
return std::make_shared<DataTypeArray>(res);
|
||||||
|
@ -51,7 +51,7 @@ class AggregateFunctionRankCorrelation :
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionRankCorrelation(const DataTypes & arguments)
|
explicit AggregateFunctionRankCorrelation(const DataTypes & arguments)
|
||||||
:IAggregateFunctionDataHelper<RankCorrelationData, AggregateFunctionRankCorrelation> ({arguments}, {})
|
:IAggregateFunctionDataHelper<RankCorrelationData, AggregateFunctionRankCorrelation> ({arguments}, {}, std::make_shared<DataTypeNumber<Float64>>())
|
||||||
{}
|
{}
|
||||||
|
|
||||||
String getName() const override
|
String getName() const override
|
||||||
@ -61,11 +61,6 @@ public:
|
|||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return true; }
|
bool allocatesMemoryInArena() const override { return true; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeNumber<Float64>>();
|
|
||||||
}
|
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||||
{
|
{
|
||||||
Float64 new_x = columns[0]->getFloat64(row_num);
|
Float64 new_x = columns[0]->getFloat64(row_num);
|
||||||
|
@ -43,7 +43,7 @@ public:
|
|||||||
size_t step_,
|
size_t step_,
|
||||||
const DataTypes & arguments,
|
const DataTypes & arguments,
|
||||||
const Array & params)
|
const Array & params)
|
||||||
: IAggregateFunctionHelper<AggregateFunctionResample<Key>>{arguments, params}
|
: IAggregateFunctionHelper<AggregateFunctionResample<Key>>{arguments, params, createResultType(nested_function_)}
|
||||||
, nested_function{nested_function_}
|
, nested_function{nested_function_}
|
||||||
, last_col{arguments.size() - 1}
|
, last_col{arguments.size() - 1}
|
||||||
, begin{begin_}
|
, begin{begin_}
|
||||||
@ -190,9 +190,9 @@ public:
|
|||||||
nested_function->deserialize(place + i * size_of_data, buf, version, arena);
|
nested_function->deserialize(place + i * size_of_data, buf, version, arena);
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const AggregateFunctionPtr & nested_function_)
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeArray>(nested_function->getReturnType());
|
return std::make_shared<DataTypeArray>(nested_function_->getResultType());
|
||||||
}
|
}
|
||||||
|
|
||||||
template <bool merge>
|
template <bool merge>
|
||||||
|
@ -76,7 +76,7 @@ public:
|
|||||||
}
|
}
|
||||||
|
|
||||||
explicit AggregateFunctionRetention(const DataTypes & arguments)
|
explicit AggregateFunctionRetention(const DataTypes & arguments)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionRetentionData, AggregateFunctionRetention>(arguments, {})
|
: IAggregateFunctionDataHelper<AggregateFunctionRetentionData, AggregateFunctionRetention>(arguments, {}, std::make_shared<DataTypeArray>(std::make_shared<DataTypeUInt8>()))
|
||||||
{
|
{
|
||||||
for (const auto i : collections::range(0, arguments.size()))
|
for (const auto i : collections::range(0, arguments.size()))
|
||||||
{
|
{
|
||||||
@ -90,12 +90,6 @@ public:
|
|||||||
events_size = static_cast<UInt8>(arguments.size());
|
events_size = static_cast<UInt8>(arguments.size());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeArray>(std::make_shared<DataTypeUInt8>());
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, const size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, const size_t row_num, Arena *) const override
|
||||||
|
@ -126,8 +126,8 @@ template <typename T, typename Data, typename Derived>
|
|||||||
class AggregateFunctionSequenceBase : public IAggregateFunctionDataHelper<Data, Derived>
|
class AggregateFunctionSequenceBase : public IAggregateFunctionDataHelper<Data, Derived>
|
||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
AggregateFunctionSequenceBase(const DataTypes & arguments, const Array & params, const String & pattern_)
|
AggregateFunctionSequenceBase(const DataTypes & arguments, const Array & params, const String & pattern_, const DataTypePtr & result_type_)
|
||||||
: IAggregateFunctionDataHelper<Data, Derived>(arguments, params)
|
: IAggregateFunctionDataHelper<Data, Derived>(arguments, params, result_type_)
|
||||||
, pattern(pattern_)
|
, pattern(pattern_)
|
||||||
{
|
{
|
||||||
arg_count = arguments.size();
|
arg_count = arguments.size();
|
||||||
@ -617,14 +617,12 @@ class AggregateFunctionSequenceMatch final : public AggregateFunctionSequenceBas
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
AggregateFunctionSequenceMatch(const DataTypes & arguments, const Array & params, const String & pattern_)
|
AggregateFunctionSequenceMatch(const DataTypes & arguments, const Array & params, const String & pattern_)
|
||||||
: AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceMatch<T, Data>>(arguments, params, pattern_) {}
|
: AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceMatch<T, Data>>(arguments, params, pattern_, std::make_shared<DataTypeUInt8>()) {}
|
||||||
|
|
||||||
using AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceMatch<T, Data>>::AggregateFunctionSequenceBase;
|
using AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceMatch<T, Data>>::AggregateFunctionSequenceBase;
|
||||||
|
|
||||||
String getName() const override { return "sequenceMatch"; }
|
String getName() const override { return "sequenceMatch"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeUInt8>(); }
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void insertResultInto(AggregateDataPtr __restrict place, IColumn & to, Arena *) const override
|
void insertResultInto(AggregateDataPtr __restrict place, IColumn & to, Arena *) const override
|
||||||
@ -655,14 +653,12 @@ class AggregateFunctionSequenceCount final : public AggregateFunctionSequenceBas
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
AggregateFunctionSequenceCount(const DataTypes & arguments, const Array & params, const String & pattern_)
|
AggregateFunctionSequenceCount(const DataTypes & arguments, const Array & params, const String & pattern_)
|
||||||
: AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceCount<T, Data>>(arguments, params, pattern_) {}
|
: AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceCount<T, Data>>(arguments, params, pattern_, std::make_shared<DataTypeUInt64>()) {}
|
||||||
|
|
||||||
using AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceCount<T, Data>>::AggregateFunctionSequenceBase;
|
using AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceCount<T, Data>>::AggregateFunctionSequenceBase;
|
||||||
|
|
||||||
String getName() const override { return "sequenceCount"; }
|
String getName() const override { return "sequenceCount"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeUInt64>(); }
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void insertResultInto(AggregateDataPtr __restrict place, IColumn & to, Arena *) const override
|
void insertResultInto(AggregateDataPtr __restrict place, IColumn & to, Arena *) const override
|
||||||
|
@ -190,7 +190,7 @@ public:
|
|||||||
SequenceDirection seq_direction_,
|
SequenceDirection seq_direction_,
|
||||||
size_t min_required_args_,
|
size_t min_required_args_,
|
||||||
UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||||
: IAggregateFunctionDataHelper<SequenceNextNodeGeneralData<Node>, Self>({data_type_}, parameters_)
|
: IAggregateFunctionDataHelper<SequenceNextNodeGeneralData<Node>, Self>({data_type_}, parameters_, data_type_)
|
||||||
, seq_base_kind(seq_base_kind_)
|
, seq_base_kind(seq_base_kind_)
|
||||||
, seq_direction(seq_direction_)
|
, seq_direction(seq_direction_)
|
||||||
, min_required_args(min_required_args_)
|
, min_required_args(min_required_args_)
|
||||||
@ -202,8 +202,6 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return "sequenceNextNode"; }
|
String getName() const override { return "sequenceNextNode"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override { return data_type; }
|
|
||||||
|
|
||||||
bool haveSameStateRepresentationImpl(const IAggregateFunction & rhs) const override
|
bool haveSameStateRepresentationImpl(const IAggregateFunction & rhs) const override
|
||||||
{
|
{
|
||||||
return this->getName() == rhs.getName() && this->haveEqualArgumentTypes(rhs);
|
return this->getName() == rhs.getName() && this->haveEqualArgumentTypes(rhs);
|
||||||
|
@ -99,7 +99,7 @@ public:
|
|||||||
IAggregateFunctionDataHelper<
|
IAggregateFunctionDataHelper<
|
||||||
AggregateFunctionSimpleLinearRegressionData<Ret>,
|
AggregateFunctionSimpleLinearRegressionData<Ret>,
|
||||||
AggregateFunctionSimpleLinearRegression<X, Y, Ret>
|
AggregateFunctionSimpleLinearRegression<X, Y, Ret>
|
||||||
> {arguments, params}
|
> {arguments, params, createResultType()}
|
||||||
{
|
{
|
||||||
// notice: arguments has been checked before
|
// notice: arguments has been checked before
|
||||||
}
|
}
|
||||||
@ -140,7 +140,7 @@ public:
|
|||||||
this->data(place).deserialize(buf);
|
this->data(place).deserialize(buf);
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
DataTypes types
|
DataTypes types
|
||||||
{
|
{
|
||||||
|
@ -20,28 +20,28 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionSimpleState(AggregateFunctionPtr nested_, const DataTypes & arguments_, const Array & params_)
|
AggregateFunctionSimpleState(AggregateFunctionPtr nested_, const DataTypes & arguments_, const Array & params_)
|
||||||
: IAggregateFunctionHelper<AggregateFunctionSimpleState>(arguments_, params_)
|
: IAggregateFunctionHelper<AggregateFunctionSimpleState>(arguments_, params_, createResultType(nested_, params_))
|
||||||
, nested_func(nested_)
|
, nested_func(nested_)
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
String getName() const override { return nested_func->getName() + "SimpleState"; }
|
String getName() const override { return nested_func->getName() + "SimpleState"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const AggregateFunctionPtr & nested_, const Array & params_)
|
||||||
{
|
{
|
||||||
DataTypeCustomSimpleAggregateFunction::checkSupportedFunctions(nested_func);
|
DataTypeCustomSimpleAggregateFunction::checkSupportedFunctions(nested_);
|
||||||
|
|
||||||
// Need to make a clone to avoid recursive reference.
|
// Need to make a clone to avoid recursive reference.
|
||||||
auto storage_type_out = DataTypeFactory::instance().get(nested_func->getReturnType()->getName());
|
auto storage_type_out = DataTypeFactory::instance().get(nested_->getResultType()->getName());
|
||||||
// Need to make a new function with promoted argument types because SimpleAggregates requires arg_type = return_type.
|
// Need to make a new function with promoted argument types because SimpleAggregates requires arg_type = return_type.
|
||||||
AggregateFunctionProperties properties;
|
AggregateFunctionProperties properties;
|
||||||
auto function
|
auto function
|
||||||
= AggregateFunctionFactory::instance().get(nested_func->getName(), {storage_type_out}, nested_func->getParameters(), properties);
|
= AggregateFunctionFactory::instance().get(nested_->getName(), {storage_type_out}, nested_->getParameters(), properties);
|
||||||
|
|
||||||
// Need to make a clone because it'll be customized.
|
// Need to make a clone because it'll be customized.
|
||||||
auto storage_type_arg = DataTypeFactory::instance().get(nested_func->getReturnType()->getName());
|
auto storage_type_arg = DataTypeFactory::instance().get(nested_->getResultType()->getName());
|
||||||
DataTypeCustomNamePtr custom_name
|
DataTypeCustomNamePtr custom_name
|
||||||
= std::make_unique<DataTypeCustomSimpleAggregateFunction>(function, DataTypes{nested_func->getReturnType()}, parameters);
|
= std::make_unique<DataTypeCustomSimpleAggregateFunction>(function, DataTypes{nested_->getResultType()}, params_);
|
||||||
storage_type_arg->setCustomization(std::make_unique<DataTypeCustomDesc>(std::move(custom_name), nullptr));
|
storage_type_arg->setCustomization(std::make_unique<DataTypeCustomDesc>(std::move(custom_name), nullptr));
|
||||||
return storage_type_arg;
|
return storage_type_arg;
|
||||||
}
|
}
|
||||||
|
@ -261,7 +261,7 @@ private:
|
|||||||
public:
|
public:
|
||||||
AggregateFunctionSparkbar(const DataTypes & arguments, const Array & params)
|
AggregateFunctionSparkbar(const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionSparkbarData<X, Y>, AggregateFunctionSparkbar>(
|
: IAggregateFunctionDataHelper<AggregateFunctionSparkbarData<X, Y>, AggregateFunctionSparkbar>(
|
||||||
arguments, params)
|
arguments, params, std::make_shared<DataTypeString>())
|
||||||
{
|
{
|
||||||
width = params.at(0).safeGet<UInt64>();
|
width = params.at(0).safeGet<UInt64>();
|
||||||
if (params.size() == 3)
|
if (params.size() == 3)
|
||||||
@ -283,11 +283,6 @@ public:
|
|||||||
return "sparkbar";
|
return "sparkbar";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeString>();
|
|
||||||
}
|
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * /*arena*/) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * /*arena*/) const override
|
||||||
{
|
{
|
||||||
X x = assert_cast<const ColumnVector<X> *>(columns[0])->getData()[row_num];
|
X x = assert_cast<const ColumnVector<X> *>(columns[0])->getData()[row_num];
|
||||||
|
@ -23,7 +23,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionState(AggregateFunctionPtr nested_, const DataTypes & arguments_, const Array & params_)
|
AggregateFunctionState(AggregateFunctionPtr nested_, const DataTypes & arguments_, const Array & params_)
|
||||||
: IAggregateFunctionHelper<AggregateFunctionState>(arguments_, params_)
|
: IAggregateFunctionHelper<AggregateFunctionState>(arguments_, params_, nested_->getStateType())
|
||||||
, nested_func(nested_)
|
, nested_func(nested_)
|
||||||
{}
|
{}
|
||||||
|
|
||||||
@ -32,11 +32,6 @@ public:
|
|||||||
return nested_func->getName() + "State";
|
return nested_func->getName() + "State";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return getStateType();
|
|
||||||
}
|
|
||||||
|
|
||||||
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
||||||
{
|
{
|
||||||
return nested_func->getBaseAggregateFunctionWithSameStateRepresentation();
|
return nested_func->getBaseAggregateFunctionWithSameStateRepresentation();
|
||||||
|
@ -115,15 +115,11 @@ class AggregateFunctionVariance final
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionVariance(const DataTypePtr & arg)
|
explicit AggregateFunctionVariance(const DataTypePtr & arg)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionVarianceData<T, Op>, AggregateFunctionVariance<T, Op>>({arg}, {}) {}
|
: IAggregateFunctionDataHelper<AggregateFunctionVarianceData<T, Op>, AggregateFunctionVariance<T, Op>>({arg}, {}, std::make_shared<DataTypeFloat64>())
|
||||||
|
{}
|
||||||
|
|
||||||
String getName() const override { return Op::name; }
|
String getName() const override { return Op::name; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeFloat64>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||||
@ -368,15 +364,11 @@ class AggregateFunctionCovariance final
|
|||||||
public:
|
public:
|
||||||
explicit AggregateFunctionCovariance(const DataTypes & args) : IAggregateFunctionDataHelper<
|
explicit AggregateFunctionCovariance(const DataTypes & args) : IAggregateFunctionDataHelper<
|
||||||
CovarianceData<T, U, Op, compute_marginal_moments>,
|
CovarianceData<T, U, Op, compute_marginal_moments>,
|
||||||
AggregateFunctionCovariance<T, U, Op, compute_marginal_moments>>(args, {}) {}
|
AggregateFunctionCovariance<T, U, Op, compute_marginal_moments>>(args, {}, std::make_shared<DataTypeFloat64>())
|
||||||
|
{}
|
||||||
|
|
||||||
String getName() const override { return Op::name; }
|
String getName() const override { return Op::name; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeFloat64>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||||
|
@ -81,12 +81,12 @@ public:
|
|||||||
using ColVecResult = ColumnVector<ResultType>;
|
using ColVecResult = ColumnVector<ResultType>;
|
||||||
|
|
||||||
explicit AggregateFunctionVarianceSimple(const DataTypes & argument_types_)
|
explicit AggregateFunctionVarianceSimple(const DataTypes & argument_types_)
|
||||||
: IAggregateFunctionDataHelper<typename StatFunc::Data, AggregateFunctionVarianceSimple<StatFunc>>(argument_types_, {})
|
: IAggregateFunctionDataHelper<typename StatFunc::Data, AggregateFunctionVarianceSimple<StatFunc>>(argument_types_, {}, std::make_shared<DataTypeNumber<ResultType>>())
|
||||||
, src_scale(0)
|
, src_scale(0)
|
||||||
{}
|
{}
|
||||||
|
|
||||||
AggregateFunctionVarianceSimple(const IDataType & data_type, const DataTypes & argument_types_)
|
AggregateFunctionVarianceSimple(const IDataType & data_type, const DataTypes & argument_types_)
|
||||||
: IAggregateFunctionDataHelper<typename StatFunc::Data, AggregateFunctionVarianceSimple<StatFunc>>(argument_types_, {})
|
: IAggregateFunctionDataHelper<typename StatFunc::Data, AggregateFunctionVarianceSimple<StatFunc>>(argument_types_, {}, std::make_shared<DataTypeNumber<ResultType>>())
|
||||||
, src_scale(getDecimalScale(data_type))
|
, src_scale(getDecimalScale(data_type))
|
||||||
{}
|
{}
|
||||||
|
|
||||||
@ -117,11 +117,6 @@ public:
|
|||||||
UNREACHABLE();
|
UNREACHABLE();
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeNumber<ResultType>>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||||
|
@ -411,23 +411,21 @@ public:
|
|||||||
}
|
}
|
||||||
|
|
||||||
explicit AggregateFunctionSum(const DataTypes & argument_types_)
|
explicit AggregateFunctionSum(const DataTypes & argument_types_)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionSum<T, TResult, Data, Type>>(argument_types_, {})
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionSum<T, TResult, Data, Type>>(argument_types_, {}, createResultType(0))
|
||||||
, scale(0)
|
|
||||||
{}
|
{}
|
||||||
|
|
||||||
AggregateFunctionSum(const IDataType & data_type, const DataTypes & argument_types_)
|
AggregateFunctionSum(const IDataType & data_type, const DataTypes & argument_types_)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionSum<T, TResult, Data, Type>>(argument_types_, {})
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionSum<T, TResult, Data, Type>>(argument_types_, {}, createResultType(getDecimalScale(data_type)))
|
||||||
, scale(getDecimalScale(data_type))
|
|
||||||
{}
|
{}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(UInt32 scale_)
|
||||||
{
|
{
|
||||||
if constexpr (!is_decimal<T>)
|
if constexpr (!is_decimal<T>)
|
||||||
return std::make_shared<DataTypeNumber<TResult>>();
|
return std::make_shared<DataTypeNumber<TResult>>();
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
using DataType = DataTypeDecimal<TResult>;
|
using DataType = DataTypeDecimal<TResult>;
|
||||||
return std::make_shared<DataType>(DataType::maxPrecision(), scale);
|
return std::make_shared<DataType>(DataType::maxPrecision(), scale_);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -548,7 +546,7 @@ public:
|
|||||||
for (const auto & argument_type : this->argument_types)
|
for (const auto & argument_type : this->argument_types)
|
||||||
can_be_compiled &= canBeNativeType(*argument_type);
|
can_be_compiled &= canBeNativeType(*argument_type);
|
||||||
|
|
||||||
auto return_type = getReturnType();
|
auto return_type = this->getResultType();
|
||||||
can_be_compiled &= canBeNativeType(*return_type);
|
can_be_compiled &= canBeNativeType(*return_type);
|
||||||
|
|
||||||
return can_be_compiled;
|
return can_be_compiled;
|
||||||
@ -558,7 +556,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
auto * aggregate_sum_ptr = aggregate_data_ptr;
|
auto * aggregate_sum_ptr = aggregate_data_ptr;
|
||||||
|
|
||||||
b.CreateStore(llvm::Constant::getNullValue(return_type), aggregate_sum_ptr);
|
b.CreateStore(llvm::Constant::getNullValue(return_type), aggregate_sum_ptr);
|
||||||
@ -568,7 +566,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
|
|
||||||
auto * sum_value_ptr = aggregate_data_ptr;
|
auto * sum_value_ptr = aggregate_data_ptr;
|
||||||
auto * sum_value = b.CreateLoad(return_type, sum_value_ptr);
|
auto * sum_value = b.CreateLoad(return_type, sum_value_ptr);
|
||||||
@ -586,7 +584,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
|
|
||||||
auto * sum_value_dst_ptr = aggregate_data_dst_ptr;
|
auto * sum_value_dst_ptr = aggregate_data_dst_ptr;
|
||||||
auto * sum_value_dst = b.CreateLoad(return_type, sum_value_dst_ptr);
|
auto * sum_value_dst = b.CreateLoad(return_type, sum_value_dst_ptr);
|
||||||
@ -602,7 +600,7 @@ public:
|
|||||||
{
|
{
|
||||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||||
|
|
||||||
auto * return_type = toNativeType(b, getReturnType());
|
auto * return_type = toNativeType(b, this->getResultType());
|
||||||
auto * sum_value_ptr = aggregate_data_ptr;
|
auto * sum_value_ptr = aggregate_data_ptr;
|
||||||
|
|
||||||
return b.CreateLoad(return_type, sum_value_ptr);
|
return b.CreateLoad(return_type, sum_value_ptr);
|
||||||
@ -611,8 +609,6 @@ public:
|
|||||||
#endif
|
#endif
|
||||||
|
|
||||||
private:
|
private:
|
||||||
UInt32 scale;
|
|
||||||
|
|
||||||
static constexpr auto & castColumnToResult(IColumn & to)
|
static constexpr auto & castColumnToResult(IColumn & to)
|
||||||
{
|
{
|
||||||
if constexpr (is_decimal<T>)
|
if constexpr (is_decimal<T>)
|
||||||
|
@ -14,12 +14,13 @@ public:
|
|||||||
using Base = AggregateFunctionAvg<T>;
|
using Base = AggregateFunctionAvg<T>;
|
||||||
|
|
||||||
explicit AggregateFunctionSumCount(const DataTypes & argument_types_, UInt32 num_scale_ = 0)
|
explicit AggregateFunctionSumCount(const DataTypes & argument_types_, UInt32 num_scale_ = 0)
|
||||||
: Base(argument_types_, num_scale_), scale(num_scale_) {}
|
: Base(argument_types_, createResultType(num_scale_), num_scale_)
|
||||||
|
{}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(UInt32 num_scale_)
|
||||||
{
|
{
|
||||||
auto second_elem = std::make_shared<DataTypeUInt64>();
|
auto second_elem = std::make_shared<DataTypeUInt64>();
|
||||||
return std::make_shared<DataTypeTuple>(DataTypes{getReturnTypeFirstElement(), std::move(second_elem)});
|
return std::make_shared<DataTypeTuple>(DataTypes{getReturnTypeFirstElement(num_scale_), std::move(second_elem)});
|
||||||
}
|
}
|
||||||
|
|
||||||
void insertResultInto(AggregateDataPtr __restrict place, IColumn & to, Arena *) const final
|
void insertResultInto(AggregateDataPtr __restrict place, IColumn & to, Arena *) const final
|
||||||
@ -43,9 +44,7 @@ public:
|
|||||||
#endif
|
#endif
|
||||||
|
|
||||||
private:
|
private:
|
||||||
UInt32 scale;
|
static auto getReturnTypeFirstElement(UInt32 num_scale_)
|
||||||
|
|
||||||
auto getReturnTypeFirstElement() const
|
|
||||||
{
|
{
|
||||||
using FieldType = AvgFieldType<T>;
|
using FieldType = AvgFieldType<T>;
|
||||||
|
|
||||||
@ -54,7 +53,7 @@ private:
|
|||||||
else
|
else
|
||||||
{
|
{
|
||||||
using DataType = DataTypeDecimal<FieldType>;
|
using DataType = DataTypeDecimal<FieldType>;
|
||||||
return std::make_shared<DataType>(DataType::maxPrecision(), scale);
|
return std::make_shared<DataType>(DataType::maxPrecision(), num_scale_);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
@ -16,6 +16,7 @@
|
|||||||
#include <Common/FieldVisitorSum.h>
|
#include <Common/FieldVisitorSum.h>
|
||||||
#include <Common/assert_cast.h>
|
#include <Common/assert_cast.h>
|
||||||
#include <AggregateFunctions/IAggregateFunction.h>
|
#include <AggregateFunctions/IAggregateFunction.h>
|
||||||
|
#include <AggregateFunctions/FactoryHelpers.h>
|
||||||
#include <map>
|
#include <map>
|
||||||
#include <Common/logger_useful.h>
|
#include <Common/logger_useful.h>
|
||||||
#include <Common/ClickHouseRevision.h>
|
#include <Common/ClickHouseRevision.h>
|
||||||
@ -80,7 +81,7 @@ public:
|
|||||||
|
|
||||||
AggregateFunctionMapBase(const DataTypePtr & keys_type_,
|
AggregateFunctionMapBase(const DataTypePtr & keys_type_,
|
||||||
const DataTypes & values_types_, const DataTypes & argument_types_)
|
const DataTypes & values_types_, const DataTypes & argument_types_)
|
||||||
: Base(argument_types_, {} /* parameters */)
|
: Base(argument_types_, {} /* parameters */, createResultType(keys_type_, values_types_, getName()))
|
||||||
, keys_type(keys_type_)
|
, keys_type(keys_type_)
|
||||||
, keys_serialization(keys_type->getDefaultSerialization())
|
, keys_serialization(keys_type->getDefaultSerialization())
|
||||||
, values_types(values_types_)
|
, values_types(values_types_)
|
||||||
@ -117,19 +118,22 @@ public:
|
|||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(
|
||||||
|
const DataTypePtr & keys_type_,
|
||||||
|
const DataTypes & values_types_,
|
||||||
|
const String & name_)
|
||||||
{
|
{
|
||||||
DataTypes types;
|
DataTypes types;
|
||||||
types.emplace_back(std::make_shared<DataTypeArray>(keys_type));
|
types.emplace_back(std::make_shared<DataTypeArray>(keys_type_));
|
||||||
|
|
||||||
for (const auto & value_type : values_types)
|
for (const auto & value_type : values_types_)
|
||||||
{
|
{
|
||||||
if constexpr (std::is_same_v<Visitor, FieldVisitorSum>)
|
if constexpr (std::is_same_v<Visitor, FieldVisitorSum>)
|
||||||
{
|
{
|
||||||
if (!value_type->isSummable())
|
if (!value_type->isSummable())
|
||||||
throw Exception{ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
throw Exception{ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
||||||
"Values for {} cannot be summed, passed type {}",
|
"Values for {} cannot be summed, passed type {}",
|
||||||
getName(), value_type->getName()};
|
name_, value_type->getName()};
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr result_type;
|
DataTypePtr result_type;
|
||||||
@ -139,7 +143,7 @@ public:
|
|||||||
if (value_type->onlyNull())
|
if (value_type->onlyNull())
|
||||||
throw Exception{ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
throw Exception{ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
||||||
"Cannot calculate {} of type {}",
|
"Cannot calculate {} of type {}",
|
||||||
getName(), value_type->getName()};
|
name_, value_type->getName()};
|
||||||
|
|
||||||
// Overflow, meaning that the returned type is the same as
|
// Overflow, meaning that the returned type is the same as
|
||||||
// the input type. Nulls are skipped.
|
// the input type. Nulls are skipped.
|
||||||
@ -153,7 +157,7 @@ public:
|
|||||||
if (!value_type_without_nullable->canBePromoted())
|
if (!value_type_without_nullable->canBePromoted())
|
||||||
throw Exception{ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
throw Exception{ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
||||||
"Values for {} are expected to be Numeric, Float or Decimal, passed type {}",
|
"Values for {} are expected to be Numeric, Float or Decimal, passed type {}",
|
||||||
getName(), value_type->getName()};
|
name_, value_type->getName()};
|
||||||
|
|
||||||
WhichDataType value_type_to_check(value_type_without_nullable);
|
WhichDataType value_type_to_check(value_type_without_nullable);
|
||||||
|
|
||||||
@ -424,7 +428,10 @@ public:
|
|||||||
}
|
}
|
||||||
|
|
||||||
bool keepKey(const T & key) const { return static_cast<const Derived &>(*this).keepKey(key); }
|
bool keepKey(const T & key) const { return static_cast<const Derived &>(*this).keepKey(key); }
|
||||||
String getName() const override { return static_cast<const Derived &>(*this).getName(); }
|
String getName() const override { return getNameImpl(); }
|
||||||
|
|
||||||
|
private:
|
||||||
|
static String getNameImpl() { return Derived::getNameImpl(); }
|
||||||
};
|
};
|
||||||
|
|
||||||
template <typename T, bool overflow, bool tuple_argument>
|
template <typename T, bool overflow, bool tuple_argument>
|
||||||
@ -443,10 +450,10 @@ public:
|
|||||||
{
|
{
|
||||||
// The constructor accepts parameters to have a uniform interface with
|
// The constructor accepts parameters to have a uniform interface with
|
||||||
// sumMapFiltered, but this function doesn't have any parameters.
|
// sumMapFiltered, but this function doesn't have any parameters.
|
||||||
assertNoParameters(getName(), params_);
|
assertNoParameters(getNameImpl(), params_);
|
||||||
}
|
}
|
||||||
|
|
||||||
String getName() const override
|
static String getNameImpl()
|
||||||
{
|
{
|
||||||
if constexpr (overflow)
|
if constexpr (overflow)
|
||||||
{
|
{
|
||||||
@ -487,13 +494,13 @@ public:
|
|||||||
if (params_.size() != 1)
|
if (params_.size() != 1)
|
||||||
throw Exception(ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
|
throw Exception(ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
|
||||||
"Aggregate function '{}' requires exactly one parameter "
|
"Aggregate function '{}' requires exactly one parameter "
|
||||||
"of Array type", getName());
|
"of Array type", getNameImpl());
|
||||||
|
|
||||||
Array keys_to_keep_values;
|
Array keys_to_keep_values;
|
||||||
if (!params_.front().tryGet<Array>(keys_to_keep_values))
|
if (!params_.front().tryGet<Array>(keys_to_keep_values))
|
||||||
throw Exception(ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
throw Exception(ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
||||||
"Aggregate function {} requires an Array as a parameter",
|
"Aggregate function {} requires an Array as a parameter",
|
||||||
getName());
|
getNameImpl());
|
||||||
|
|
||||||
keys_to_keep.reserve(keys_to_keep_values.size());
|
keys_to_keep.reserve(keys_to_keep_values.size());
|
||||||
|
|
||||||
@ -501,7 +508,7 @@ public:
|
|||||||
keys_to_keep.emplace(f.safeGet<T>());
|
keys_to_keep.emplace(f.safeGet<T>());
|
||||||
}
|
}
|
||||||
|
|
||||||
String getName() const override
|
static String getNameImpl()
|
||||||
{ return overflow ? "sumMapFilteredWithOverflow" : "sumMapFiltered"; }
|
{ return overflow ? "sumMapFilteredWithOverflow" : "sumMapFiltered"; }
|
||||||
|
|
||||||
bool keepKey(const T & key) const { return keys_to_keep.count(key); }
|
bool keepKey(const T & key) const { return keys_to_keep.count(key); }
|
||||||
@ -606,10 +613,10 @@ public:
|
|||||||
{
|
{
|
||||||
// The constructor accepts parameters to have a uniform interface with
|
// The constructor accepts parameters to have a uniform interface with
|
||||||
// sumMapFiltered, but this function doesn't have any parameters.
|
// sumMapFiltered, but this function doesn't have any parameters.
|
||||||
assertNoParameters(getName(), params_);
|
assertNoParameters(getNameImpl(), params_);
|
||||||
}
|
}
|
||||||
|
|
||||||
String getName() const override { return "minMap"; }
|
static String getNameImpl() { return "minMap"; }
|
||||||
|
|
||||||
bool keepKey(const T &) const { return true; }
|
bool keepKey(const T &) const { return true; }
|
||||||
};
|
};
|
||||||
@ -630,10 +637,10 @@ public:
|
|||||||
{
|
{
|
||||||
// The constructor accepts parameters to have a uniform interface with
|
// The constructor accepts parameters to have a uniform interface with
|
||||||
// sumMapFiltered, but this function doesn't have any parameters.
|
// sumMapFiltered, but this function doesn't have any parameters.
|
||||||
assertNoParameters(getName(), params_);
|
assertNoParameters(getNameImpl(), params_);
|
||||||
}
|
}
|
||||||
|
|
||||||
String getName() const override { return "maxMap"; }
|
static String getNameImpl() { return "maxMap"; }
|
||||||
|
|
||||||
bool keepKey(const T &) const { return true; }
|
bool keepKey(const T &) const { return true; }
|
||||||
};
|
};
|
||||||
|
@ -46,7 +46,7 @@ private:
|
|||||||
Float64 confidence_level;
|
Float64 confidence_level;
|
||||||
public:
|
public:
|
||||||
AggregateFunctionTTest(const DataTypes & arguments, const Array & params)
|
AggregateFunctionTTest(const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionTTest<Data>>({arguments}, params)
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionTTest<Data>>({arguments}, params, createResultType(!params.empty()))
|
||||||
{
|
{
|
||||||
if (!params.empty())
|
if (!params.empty())
|
||||||
{
|
{
|
||||||
@ -71,9 +71,9 @@ public:
|
|||||||
return Data::name;
|
return Data::name;
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(bool need_confidence_interval_)
|
||||||
{
|
{
|
||||||
if (need_confidence_interval)
|
if (need_confidence_interval_)
|
||||||
{
|
{
|
||||||
DataTypes types
|
DataTypes types
|
||||||
{
|
{
|
||||||
|
@ -31,15 +31,33 @@ namespace
|
|||||||
template <bool is_weighted>
|
template <bool is_weighted>
|
||||||
class AggregateFunctionTopKDate : public AggregateFunctionTopK<DataTypeDate::FieldType, is_weighted>
|
class AggregateFunctionTopKDate : public AggregateFunctionTopK<DataTypeDate::FieldType, is_weighted>
|
||||||
{
|
{
|
||||||
|
public:
|
||||||
using AggregateFunctionTopK<DataTypeDate::FieldType, is_weighted>::AggregateFunctionTopK;
|
using AggregateFunctionTopK<DataTypeDate::FieldType, is_weighted>::AggregateFunctionTopK;
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDate>()); }
|
|
||||||
|
AggregateFunctionTopKDate(UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params)
|
||||||
|
: AggregateFunctionTopK<DataTypeDate::FieldType, is_weighted>(
|
||||||
|
threshold_,
|
||||||
|
load_factor,
|
||||||
|
argument_types_,
|
||||||
|
params,
|
||||||
|
std::make_shared<DataTypeArray>(std::make_shared<DataTypeDate>()))
|
||||||
|
{}
|
||||||
};
|
};
|
||||||
|
|
||||||
template <bool is_weighted>
|
template <bool is_weighted>
|
||||||
class AggregateFunctionTopKDateTime : public AggregateFunctionTopK<DataTypeDateTime::FieldType, is_weighted>
|
class AggregateFunctionTopKDateTime : public AggregateFunctionTopK<DataTypeDateTime::FieldType, is_weighted>
|
||||||
{
|
{
|
||||||
|
public:
|
||||||
using AggregateFunctionTopK<DataTypeDateTime::FieldType, is_weighted>::AggregateFunctionTopK;
|
using AggregateFunctionTopK<DataTypeDateTime::FieldType, is_weighted>::AggregateFunctionTopK;
|
||||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDateTime>()); }
|
|
||||||
|
AggregateFunctionTopKDateTime(UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params)
|
||||||
|
: AggregateFunctionTopK<DataTypeDateTime::FieldType, is_weighted>(
|
||||||
|
threshold_,
|
||||||
|
load_factor,
|
||||||
|
argument_types_,
|
||||||
|
params,
|
||||||
|
std::make_shared<DataTypeArray>(std::make_shared<DataTypeDateTime>()))
|
||||||
|
{}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
|
@ -40,14 +40,20 @@ protected:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionTopK(UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params)
|
AggregateFunctionTopK(UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionTopKData<T>, AggregateFunctionTopK<T, is_weighted>>(argument_types_, params)
|
: IAggregateFunctionDataHelper<AggregateFunctionTopKData<T>, AggregateFunctionTopK<T, is_weighted>>(argument_types_, params, createResultType(argument_types_))
|
||||||
, threshold(threshold_), reserved(load_factor * threshold) {}
|
, threshold(threshold_), reserved(load_factor * threshold)
|
||||||
|
{}
|
||||||
|
|
||||||
|
AggregateFunctionTopK(UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params, const DataTypePtr & result_type_)
|
||||||
|
: IAggregateFunctionDataHelper<AggregateFunctionTopKData<T>, AggregateFunctionTopK<T, is_weighted>>(argument_types_, params, result_type_)
|
||||||
|
, threshold(threshold_), reserved(load_factor * threshold)
|
||||||
|
{}
|
||||||
|
|
||||||
String getName() const override { return is_weighted ? "topKWeighted" : "topK"; }
|
String getName() const override { return is_weighted ? "topKWeighted" : "topK"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const DataTypes & argument_types_)
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeArray>(this->argument_types[0]);
|
return std::make_shared<DataTypeArray>(argument_types_[0]);
|
||||||
}
|
}
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
@ -126,21 +132,20 @@ private:
|
|||||||
|
|
||||||
UInt64 threshold;
|
UInt64 threshold;
|
||||||
UInt64 reserved;
|
UInt64 reserved;
|
||||||
DataTypePtr & input_data_type;
|
|
||||||
|
|
||||||
static void deserializeAndInsert(StringRef str, IColumn & data_to);
|
static void deserializeAndInsert(StringRef str, IColumn & data_to);
|
||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionTopKGeneric(
|
AggregateFunctionTopKGeneric(
|
||||||
UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params)
|
UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionTopKGenericData, AggregateFunctionTopKGeneric<is_plain_column, is_weighted>>(argument_types_, params)
|
: IAggregateFunctionDataHelper<AggregateFunctionTopKGenericData, AggregateFunctionTopKGeneric<is_plain_column, is_weighted>>(argument_types_, params, createResultType(argument_types_))
|
||||||
, threshold(threshold_), reserved(load_factor * threshold), input_data_type(this->argument_types[0]) {}
|
, threshold(threshold_), reserved(load_factor * threshold) {}
|
||||||
|
|
||||||
String getName() const override { return is_weighted ? "topKWeighted" : "topK"; }
|
String getName() const override { return is_weighted ? "topKWeighted" : "topK"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType(const DataTypes & argument_types_)
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeArray>(input_data_type);
|
return std::make_shared<DataTypeArray>(argument_types_[0]);
|
||||||
}
|
}
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override
|
bool allocatesMemoryInArena() const override
|
||||||
|
@ -358,17 +358,12 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionUniq(const DataTypes & argument_types_)
|
explicit AggregateFunctionUniq(const DataTypes & argument_types_)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionUniq<T, Data>>(argument_types_, {})
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionUniq<T, Data>>(argument_types_, {}, std::make_shared<DataTypeUInt64>())
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
String getName() const override { return Data::getName(); }
|
String getName() const override { return Data::getName(); }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeUInt64>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
/// ALWAYS_INLINE is required to have better code layout for uniqHLL12 function
|
/// ALWAYS_INLINE is required to have better code layout for uniqHLL12 function
|
||||||
@ -462,7 +457,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionUniqVariadic(const DataTypes & arguments)
|
explicit AggregateFunctionUniqVariadic(const DataTypes & arguments)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionUniqVariadic<Data>>(arguments, {})
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionUniqVariadic<Data>>(arguments, {}, std::make_shared<DataTypeUInt64>())
|
||||||
{
|
{
|
||||||
if (argument_is_tuple)
|
if (argument_is_tuple)
|
||||||
num_args = typeid_cast<const DataTypeTuple &>(*arguments[0]).getElements().size();
|
num_args = typeid_cast<const DataTypeTuple &>(*arguments[0]).getElements().size();
|
||||||
@ -472,11 +467,6 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return Data::getName(); }
|
String getName() const override { return Data::getName(); }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeUInt64>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||||
|
@ -126,7 +126,8 @@ class AggregateFunctionUniqCombined final
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
AggregateFunctionUniqCombined(const DataTypes & argument_types_, const Array & params_)
|
AggregateFunctionUniqCombined(const DataTypes & argument_types_, const Array & params_)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqCombinedData<T, K, HashValueType>, AggregateFunctionUniqCombined<T, K, HashValueType>>(argument_types_, params_) {}
|
: IAggregateFunctionDataHelper<AggregateFunctionUniqCombinedData<T, K, HashValueType>, AggregateFunctionUniqCombined<T, K, HashValueType>>(argument_types_, params_, std::make_shared<DataTypeUInt64>())
|
||||||
|
{}
|
||||||
|
|
||||||
String getName() const override
|
String getName() const override
|
||||||
{
|
{
|
||||||
@ -136,11 +137,6 @@ public:
|
|||||||
return "uniqCombined";
|
return "uniqCombined";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeUInt64>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||||
@ -192,7 +188,7 @@ private:
|
|||||||
public:
|
public:
|
||||||
explicit AggregateFunctionUniqCombinedVariadic(const DataTypes & arguments, const Array & params)
|
explicit AggregateFunctionUniqCombinedVariadic(const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqCombinedData<UInt64, K, HashValueType>,
|
: IAggregateFunctionDataHelper<AggregateFunctionUniqCombinedData<UInt64, K, HashValueType>,
|
||||||
AggregateFunctionUniqCombinedVariadic<is_exact, argument_is_tuple, K, HashValueType>>(arguments, params)
|
AggregateFunctionUniqCombinedVariadic<is_exact, argument_is_tuple, K, HashValueType>>(arguments, params, std::make_shared<DataTypeUInt64>())
|
||||||
{
|
{
|
||||||
if (argument_is_tuple)
|
if (argument_is_tuple)
|
||||||
num_args = typeid_cast<const DataTypeTuple &>(*arguments[0]).getElements().size();
|
num_args = typeid_cast<const DataTypeTuple &>(*arguments[0]).getElements().size();
|
||||||
@ -208,11 +204,6 @@ public:
|
|||||||
return "uniqCombined";
|
return "uniqCombined";
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeUInt64>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||||
|
@ -174,7 +174,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionUniqUpTo(UInt8 threshold_, const DataTypes & argument_types_, const Array & params_)
|
AggregateFunctionUniqUpTo(UInt8 threshold_, const DataTypes & argument_types_, const Array & params_)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqUpToData<T>, AggregateFunctionUniqUpTo<T>>(argument_types_, params_)
|
: IAggregateFunctionDataHelper<AggregateFunctionUniqUpToData<T>, AggregateFunctionUniqUpTo<T>>(argument_types_, params_, std::make_shared<DataTypeUInt64>())
|
||||||
, threshold(threshold_)
|
, threshold(threshold_)
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
@ -186,11 +186,6 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return "uniqUpTo"; }
|
String getName() const override { return "uniqUpTo"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeUInt64>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
/// ALWAYS_INLINE is required to have better code layout for uniqUpTo function
|
/// ALWAYS_INLINE is required to have better code layout for uniqUpTo function
|
||||||
@ -235,7 +230,7 @@ private:
|
|||||||
|
|
||||||
public:
|
public:
|
||||||
AggregateFunctionUniqUpToVariadic(const DataTypes & arguments, const Array & params, UInt8 threshold_)
|
AggregateFunctionUniqUpToVariadic(const DataTypes & arguments, const Array & params, UInt8 threshold_)
|
||||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqUpToData<UInt64>, AggregateFunctionUniqUpToVariadic<is_exact, argument_is_tuple>>(arguments, params)
|
: IAggregateFunctionDataHelper<AggregateFunctionUniqUpToData<UInt64>, AggregateFunctionUniqUpToVariadic<is_exact, argument_is_tuple>>(arguments, params, std::make_shared<DataTypeUInt64>())
|
||||||
, threshold(threshold_)
|
, threshold(threshold_)
|
||||||
{
|
{
|
||||||
if (argument_is_tuple)
|
if (argument_is_tuple)
|
||||||
@ -251,11 +246,6 @@ public:
|
|||||||
|
|
||||||
String getName() const override { return "uniqUpTo"; }
|
String getName() const override { return "uniqUpTo"; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeUInt64>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||||
|
@ -221,7 +221,7 @@ public:
|
|||||||
}
|
}
|
||||||
|
|
||||||
AggregateFunctionWindowFunnel(const DataTypes & arguments, const Array & params)
|
AggregateFunctionWindowFunnel(const DataTypes & arguments, const Array & params)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionWindowFunnel<T, Data>>(arguments, params)
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionWindowFunnel<T, Data>>(arguments, params, std::make_shared<DataTypeUInt8>())
|
||||||
{
|
{
|
||||||
events_size = arguments.size() - 1;
|
events_size = arguments.size() - 1;
|
||||||
window = params.at(0).safeGet<UInt64>();
|
window = params.at(0).safeGet<UInt64>();
|
||||||
@ -245,11 +245,6 @@ public:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
|
||||||
{
|
|
||||||
return std::make_shared<DataTypeUInt8>();
|
|
||||||
}
|
|
||||||
|
|
||||||
bool allocatesMemoryInArena() const override { return false; }
|
bool allocatesMemoryInArena() const override { return false; }
|
||||||
|
|
||||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, const size_t row_num, Arena *) const override
|
void add(AggregateDataPtr __restrict place, const IColumn ** columns, const size_t row_num, Arena *) const override
|
||||||
|
@ -118,7 +118,7 @@ class AggregateFunctionCrossTab : public IAggregateFunctionDataHelper<Data, Aggr
|
|||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
explicit AggregateFunctionCrossTab(const DataTypes & arguments)
|
explicit AggregateFunctionCrossTab(const DataTypes & arguments)
|
||||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionCrossTab<Data>>({arguments}, {})
|
: IAggregateFunctionDataHelper<Data, AggregateFunctionCrossTab<Data>>({arguments}, {}, createResultType())
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -132,7 +132,7 @@ public:
|
|||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
DataTypePtr getReturnType() const override
|
static DataTypePtr createResultType()
|
||||||
{
|
{
|
||||||
return std::make_shared<DataTypeNumber<Float64>>();
|
return std::make_shared<DataTypeNumber<Float64>>();
|
||||||
}
|
}
|
||||||
|
@ -10,6 +10,7 @@
|
|||||||
#include <base/types.h>
|
#include <base/types.h>
|
||||||
#include <Common/Exception.h>
|
#include <Common/Exception.h>
|
||||||
#include <Common/ThreadPool.h>
|
#include <Common/ThreadPool.h>
|
||||||
|
#include <Core/IResolvedFunction.h>
|
||||||
|
|
||||||
#include "config.h"
|
#include "config.h"
|
||||||
|
|
||||||
@ -49,6 +50,7 @@ using ConstAggregateDataPtr = const char *;
|
|||||||
|
|
||||||
class IAggregateFunction;
|
class IAggregateFunction;
|
||||||
using AggregateFunctionPtr = std::shared_ptr<const IAggregateFunction>;
|
using AggregateFunctionPtr = std::shared_ptr<const IAggregateFunction>;
|
||||||
|
|
||||||
struct AggregateFunctionProperties;
|
struct AggregateFunctionProperties;
|
||||||
|
|
||||||
/** Aggregate functions interface.
|
/** Aggregate functions interface.
|
||||||
@ -59,18 +61,18 @@ struct AggregateFunctionProperties;
|
|||||||
* (which can be created in some memory pool),
|
* (which can be created in some memory pool),
|
||||||
* and IAggregateFunction is the external interface for manipulating them.
|
* and IAggregateFunction is the external interface for manipulating them.
|
||||||
*/
|
*/
|
||||||
class IAggregateFunction : public std::enable_shared_from_this<IAggregateFunction>
|
class IAggregateFunction : public std::enable_shared_from_this<IAggregateFunction>, public IResolvedFunction
|
||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
IAggregateFunction(const DataTypes & argument_types_, const Array & parameters_)
|
IAggregateFunction(const DataTypes & argument_types_, const Array & parameters_, const DataTypePtr & result_type_)
|
||||||
: argument_types(argument_types_), parameters(parameters_) {}
|
: result_type(result_type_)
|
||||||
|
, argument_types(argument_types_)
|
||||||
|
, parameters(parameters_)
|
||||||
|
{}
|
||||||
|
|
||||||
/// Get main function name.
|
/// Get main function name.
|
||||||
virtual String getName() const = 0;
|
virtual String getName() const = 0;
|
||||||
|
|
||||||
/// Get the result type.
|
|
||||||
virtual DataTypePtr getReturnType() const = 0;
|
|
||||||
|
|
||||||
/// Get the data type of internal state. By default it is AggregateFunction(name(params), argument_types...).
|
/// Get the data type of internal state. By default it is AggregateFunction(name(params), argument_types...).
|
||||||
virtual DataTypePtr getStateType() const;
|
virtual DataTypePtr getStateType() const;
|
||||||
|
|
||||||
@ -102,7 +104,7 @@ public:
|
|||||||
|
|
||||||
virtual size_t getDefaultVersion() const { return 0; }
|
virtual size_t getDefaultVersion() const { return 0; }
|
||||||
|
|
||||||
virtual ~IAggregateFunction() = default;
|
~IAggregateFunction() override = default;
|
||||||
|
|
||||||
/** Data manipulating functions. */
|
/** Data manipulating functions. */
|
||||||
|
|
||||||
@ -348,8 +350,9 @@ public:
|
|||||||
*/
|
*/
|
||||||
virtual AggregateFunctionPtr getNestedFunction() const { return {}; }
|
virtual AggregateFunctionPtr getNestedFunction() const { return {}; }
|
||||||
|
|
||||||
const DataTypes & getArgumentTypes() const { return argument_types; }
|
const DataTypePtr & getResultType() const override { return result_type; }
|
||||||
const Array & getParameters() const { return parameters; }
|
const DataTypes & getArgumentTypes() const override { return argument_types; }
|
||||||
|
const Array & getParameters() const override { return parameters; }
|
||||||
|
|
||||||
// Any aggregate function can be calculated over a window, but there are some
|
// Any aggregate function can be calculated over a window, but there are some
|
||||||
// window functions such as rank() that require a different interface, e.g.
|
// window functions such as rank() that require a different interface, e.g.
|
||||||
@ -398,6 +401,7 @@ public:
|
|||||||
#endif
|
#endif
|
||||||
|
|
||||||
protected:
|
protected:
|
||||||
|
DataTypePtr result_type;
|
||||||
DataTypes argument_types;
|
DataTypes argument_types;
|
||||||
Array parameters;
|
Array parameters;
|
||||||
};
|
};
|
||||||
@ -414,8 +418,8 @@ private:
|
|||||||
}
|
}
|
||||||
|
|
||||||
public:
|
public:
|
||||||
IAggregateFunctionHelper(const DataTypes & argument_types_, const Array & parameters_)
|
IAggregateFunctionHelper(const DataTypes & argument_types_, const Array & parameters_, const DataTypePtr & result_type_)
|
||||||
: IAggregateFunction(argument_types_, parameters_) {}
|
: IAggregateFunction(argument_types_, parameters_, result_type_) {}
|
||||||
|
|
||||||
AddFunc getAddressOfAddFunction() const override { return &addFree; }
|
AddFunc getAddressOfAddFunction() const override { return &addFree; }
|
||||||
|
|
||||||
@ -695,15 +699,15 @@ public:
|
|||||||
// Derived class can `override` this to flag that DateTime64 is not supported.
|
// Derived class can `override` this to flag that DateTime64 is not supported.
|
||||||
static constexpr bool DateTime64Supported = true;
|
static constexpr bool DateTime64Supported = true;
|
||||||
|
|
||||||
IAggregateFunctionDataHelper(const DataTypes & argument_types_, const Array & parameters_)
|
IAggregateFunctionDataHelper(const DataTypes & argument_types_, const Array & parameters_, const DataTypePtr & result_type_)
|
||||||
: IAggregateFunctionHelper<Derived>(argument_types_, parameters_)
|
: IAggregateFunctionHelper<Derived>(argument_types_, parameters_, result_type_)
|
||||||
{
|
{
|
||||||
/// To prevent derived classes changing the destroy() without updating hasTrivialDestructor() to match it
|
/// To prevent derived classes changing the destroy() without updating hasTrivialDestructor() to match it
|
||||||
/// Enforce that either both of them are changed or none are
|
/// Enforce that either both of them are changed or none are
|
||||||
constexpr bool declares_destroy_and_hasTrivialDestructor =
|
constexpr bool declares_destroy_and_has_trivial_destructor =
|
||||||
std::is_same_v<decltype(&IAggregateFunctionDataHelper::destroy), decltype(&Derived::destroy)> ==
|
std::is_same_v<decltype(&IAggregateFunctionDataHelper::destroy), decltype(&Derived::destroy)> ==
|
||||||
std::is_same_v<decltype(&IAggregateFunctionDataHelper::hasTrivialDestructor), decltype(&Derived::hasTrivialDestructor)>;
|
std::is_same_v<decltype(&IAggregateFunctionDataHelper::hasTrivialDestructor), decltype(&Derived::hasTrivialDestructor)>;
|
||||||
static_assert(declares_destroy_and_hasTrivialDestructor,
|
static_assert(declares_destroy_and_has_trivial_destructor,
|
||||||
"destroy() and hasTrivialDestructor() methods of an aggregate function must be either both overridden or not");
|
"destroy() and hasTrivialDestructor() methods of an aggregate function must be either both overridden or not");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -2,6 +2,7 @@
|
|||||||
|
|
||||||
#include <Common/SipHash.h>
|
#include <Common/SipHash.h>
|
||||||
#include <Common/FieldVisitorToString.h>
|
#include <Common/FieldVisitorToString.h>
|
||||||
|
#include <Analyzer/ConstantNode.h>
|
||||||
|
|
||||||
#include <IO/WriteBufferFromString.h>
|
#include <IO/WriteBufferFromString.h>
|
||||||
#include <IO/Operators.h>
|
#include <IO/Operators.h>
|
||||||
@ -17,6 +18,11 @@
|
|||||||
namespace DB
|
namespace DB
|
||||||
{
|
{
|
||||||
|
|
||||||
|
namespace ErrorCodes
|
||||||
|
{
|
||||||
|
extern const int LOGICAL_ERROR;
|
||||||
|
}
|
||||||
|
|
||||||
FunctionNode::FunctionNode(String function_name_)
|
FunctionNode::FunctionNode(String function_name_)
|
||||||
: IQueryTreeNode(children_size)
|
: IQueryTreeNode(children_size)
|
||||||
, function_name(function_name_)
|
, function_name(function_name_)
|
||||||
@ -25,25 +31,41 @@ FunctionNode::FunctionNode(String function_name_)
|
|||||||
children[arguments_child_index] = std::make_shared<ListNode>();
|
children[arguments_child_index] = std::make_shared<ListNode>();
|
||||||
}
|
}
|
||||||
|
|
||||||
void FunctionNode::resolveAsFunction(FunctionOverloadResolverPtr function_value, DataTypePtr result_type_value)
|
ColumnsWithTypeAndName FunctionNode::getArgumentTypes() const
|
||||||
{
|
{
|
||||||
aggregate_function = nullptr;
|
ColumnsWithTypeAndName argument_types;
|
||||||
|
for (const auto & arg : getArguments().getNodes())
|
||||||
|
{
|
||||||
|
ColumnWithTypeAndName argument;
|
||||||
|
argument.type = arg->getResultType();
|
||||||
|
if (auto * constant = arg->as<ConstantNode>())
|
||||||
|
argument.column = argument.type->createColumnConst(1, constant->getValue());
|
||||||
|
argument_types.push_back(argument);
|
||||||
|
}
|
||||||
|
return argument_types;
|
||||||
|
}
|
||||||
|
|
||||||
|
void FunctionNode::resolveAsFunction(FunctionBasePtr function_value)
|
||||||
|
{
|
||||||
|
function_name = function_value->getName();
|
||||||
function = std::move(function_value);
|
function = std::move(function_value);
|
||||||
result_type = std::move(result_type_value);
|
kind = FunctionKind::ORDINARY;
|
||||||
function_name = function->getName();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
void FunctionNode::resolveAsAggregateFunction(AggregateFunctionPtr aggregate_function_value, DataTypePtr result_type_value)
|
void FunctionNode::resolveAsAggregateFunction(AggregateFunctionPtr aggregate_function_value)
|
||||||
{
|
{
|
||||||
function = nullptr;
|
function_name = aggregate_function_value->getName();
|
||||||
aggregate_function = std::move(aggregate_function_value);
|
function = std::move(aggregate_function_value);
|
||||||
result_type = std::move(result_type_value);
|
kind = FunctionKind::AGGREGATE;
|
||||||
function_name = aggregate_function->getName();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
void FunctionNode::resolveAsWindowFunction(AggregateFunctionPtr window_function_value, DataTypePtr result_type_value)
|
void FunctionNode::resolveAsWindowFunction(AggregateFunctionPtr window_function_value)
|
||||||
{
|
{
|
||||||
resolveAsAggregateFunction(window_function_value, result_type_value);
|
if (!hasWindow())
|
||||||
|
throw Exception(ErrorCodes::LOGICAL_ERROR,
|
||||||
|
"Trying to resolve FunctionNode without window definition as a window function {}", window_function_value->getName());
|
||||||
|
resolveAsAggregateFunction(window_function_value);
|
||||||
|
kind = FunctionKind::WINDOW;
|
||||||
}
|
}
|
||||||
|
|
||||||
void FunctionNode::dumpTreeImpl(WriteBuffer & buffer, FormatState & format_state, size_t indent) const
|
void FunctionNode::dumpTreeImpl(WriteBuffer & buffer, FormatState & format_state, size_t indent) const
|
||||||
@ -63,8 +85,8 @@ void FunctionNode::dumpTreeImpl(WriteBuffer & buffer, FormatState & format_state
|
|||||||
|
|
||||||
buffer << ", function_type: " << function_type;
|
buffer << ", function_type: " << function_type;
|
||||||
|
|
||||||
if (result_type)
|
if (function)
|
||||||
buffer << ", result_type: " + result_type->getName();
|
buffer << ", result_type: " + function->getResultType()->getName();
|
||||||
|
|
||||||
const auto & parameters = getParameters();
|
const auto & parameters = getParameters();
|
||||||
if (!parameters.getNodes().empty())
|
if (!parameters.getNodes().empty())
|
||||||
@ -96,11 +118,19 @@ bool FunctionNode::isEqualImpl(const IQueryTreeNode & rhs) const
|
|||||||
isWindowFunction() != rhs_typed.isWindowFunction())
|
isWindowFunction() != rhs_typed.isWindowFunction())
|
||||||
return false;
|
return false;
|
||||||
|
|
||||||
if (result_type && rhs_typed.result_type && !result_type->equals(*rhs_typed.getResultType()))
|
if (isResolved() != rhs_typed.isResolved())
|
||||||
return false;
|
return false;
|
||||||
else if (result_type && !rhs_typed.result_type)
|
if (!isResolved())
|
||||||
|
return true;
|
||||||
|
|
||||||
|
auto lhs_result_type = getResultType();
|
||||||
|
auto rhs_result_type = rhs.getResultType();
|
||||||
|
|
||||||
|
if (lhs_result_type && rhs_result_type && !lhs_result_type->equals(*rhs_result_type))
|
||||||
return false;
|
return false;
|
||||||
else if (!result_type && rhs_typed.result_type)
|
else if (lhs_result_type && !rhs_result_type)
|
||||||
|
return false;
|
||||||
|
else if (!lhs_result_type && rhs_result_type)
|
||||||
return false;
|
return false;
|
||||||
|
|
||||||
return true;
|
return true;
|
||||||
@ -114,7 +144,10 @@ void FunctionNode::updateTreeHashImpl(HashState & hash_state) const
|
|||||||
hash_state.update(isAggregateFunction());
|
hash_state.update(isAggregateFunction());
|
||||||
hash_state.update(isWindowFunction());
|
hash_state.update(isWindowFunction());
|
||||||
|
|
||||||
if (result_type)
|
if (!isResolved())
|
||||||
|
return;
|
||||||
|
|
||||||
|
if (auto result_type = getResultType())
|
||||||
{
|
{
|
||||||
auto result_type_name = result_type->getName();
|
auto result_type_name = result_type->getName();
|
||||||
hash_state.update(result_type_name.size());
|
hash_state.update(result_type_name.size());
|
||||||
@ -130,8 +163,7 @@ QueryTreeNodePtr FunctionNode::cloneImpl() const
|
|||||||
* because ordinary functions or aggregate functions must be stateless.
|
* because ordinary functions or aggregate functions must be stateless.
|
||||||
*/
|
*/
|
||||||
result_function->function = function;
|
result_function->function = function;
|
||||||
result_function->aggregate_function = aggregate_function;
|
result_function->kind = kind;
|
||||||
result_function->result_type = result_type;
|
|
||||||
|
|
||||||
return result_function;
|
return result_function;
|
||||||
}
|
}
|
||||||
|
@ -1,8 +1,12 @@
|
|||||||
#pragma once
|
#pragma once
|
||||||
|
|
||||||
|
#include <memory>
|
||||||
|
#include <Core/IResolvedFunction.h>
|
||||||
#include <Analyzer/IQueryTreeNode.h>
|
#include <Analyzer/IQueryTreeNode.h>
|
||||||
#include <Analyzer/ListNode.h>
|
#include <Analyzer/ListNode.h>
|
||||||
#include <Analyzer/ConstantValue.h>
|
#include <Analyzer/ConstantValue.h>
|
||||||
|
#include <Common/typeid_cast.h>
|
||||||
|
#include <Core/ColumnsWithTypeAndName.h>
|
||||||
|
|
||||||
namespace DB
|
namespace DB
|
||||||
{
|
{
|
||||||
@ -15,6 +19,9 @@ namespace ErrorCodes
|
|||||||
class IFunctionOverloadResolver;
|
class IFunctionOverloadResolver;
|
||||||
using FunctionOverloadResolverPtr = std::shared_ptr<IFunctionOverloadResolver>;
|
using FunctionOverloadResolverPtr = std::shared_ptr<IFunctionOverloadResolver>;
|
||||||
|
|
||||||
|
class IFunctionBase;
|
||||||
|
using FunctionBasePtr = std::shared_ptr<const IFunctionBase>;
|
||||||
|
|
||||||
class IAggregateFunction;
|
class IAggregateFunction;
|
||||||
using AggregateFunctionPtr = std::shared_ptr<const IAggregateFunction>;
|
using AggregateFunctionPtr = std::shared_ptr<const IAggregateFunction>;
|
||||||
|
|
||||||
@ -39,6 +46,14 @@ using AggregateFunctionPtr = std::shared_ptr<const IAggregateFunction>;
|
|||||||
class FunctionNode;
|
class FunctionNode;
|
||||||
using FunctionNodePtr = std::shared_ptr<FunctionNode>;
|
using FunctionNodePtr = std::shared_ptr<FunctionNode>;
|
||||||
|
|
||||||
|
enum class FunctionKind
|
||||||
|
{
|
||||||
|
UNKNOWN,
|
||||||
|
ORDINARY,
|
||||||
|
AGGREGATE,
|
||||||
|
WINDOW,
|
||||||
|
};
|
||||||
|
|
||||||
class FunctionNode final : public IQueryTreeNode
|
class FunctionNode final : public IQueryTreeNode
|
||||||
{
|
{
|
||||||
public:
|
public:
|
||||||
@ -101,6 +116,8 @@ public:
|
|||||||
return children[arguments_child_index];
|
return children[arguments_child_index];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
ColumnsWithTypeAndName getArgumentTypes() const;
|
||||||
|
|
||||||
/// Returns true if function node has window, false otherwise
|
/// Returns true if function node has window, false otherwise
|
||||||
bool hasWindow() const
|
bool hasWindow() const
|
||||||
{
|
{
|
||||||
@ -129,42 +146,46 @@ public:
|
|||||||
/** Get non aggregate function.
|
/** Get non aggregate function.
|
||||||
* If function is not resolved nullptr returned.
|
* If function is not resolved nullptr returned.
|
||||||
*/
|
*/
|
||||||
const FunctionOverloadResolverPtr & getFunction() const
|
FunctionBasePtr getFunction() const
|
||||||
{
|
{
|
||||||
return function;
|
if (kind != FunctionKind::ORDINARY)
|
||||||
|
return {};
|
||||||
|
return std::reinterpret_pointer_cast<const IFunctionBase>(function);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Get aggregate function.
|
/** Get aggregate function.
|
||||||
* If function is not resolved nullptr returned.
|
* If function is not resolved nullptr returned.
|
||||||
* If function is resolved as non aggregate function nullptr returned.
|
* If function is resolved as non aggregate function nullptr returned.
|
||||||
*/
|
*/
|
||||||
const AggregateFunctionPtr & getAggregateFunction() const
|
AggregateFunctionPtr getAggregateFunction() const
|
||||||
{
|
{
|
||||||
return aggregate_function;
|
if (kind == FunctionKind::UNKNOWN || kind == FunctionKind::ORDINARY)
|
||||||
|
return {};
|
||||||
|
return std::reinterpret_pointer_cast<const IAggregateFunction>(function);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Is function node resolved
|
/// Is function node resolved
|
||||||
bool isResolved() const
|
bool isResolved() const
|
||||||
{
|
{
|
||||||
return result_type != nullptr && (function != nullptr || aggregate_function != nullptr);
|
return function != nullptr;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Is function node window function
|
/// Is function node window function
|
||||||
bool isWindowFunction() const
|
bool isWindowFunction() const
|
||||||
{
|
{
|
||||||
return getWindowNode() != nullptr;
|
return hasWindow();
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Is function node aggregate function
|
/// Is function node aggregate function
|
||||||
bool isAggregateFunction() const
|
bool isAggregateFunction() const
|
||||||
{
|
{
|
||||||
return aggregate_function != nullptr && !isWindowFunction();
|
return kind == FunctionKind::AGGREGATE;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Is function node ordinary function
|
/// Is function node ordinary function
|
||||||
bool isOrdinaryFunction() const
|
bool isOrdinaryFunction() const
|
||||||
{
|
{
|
||||||
return function != nullptr;
|
return kind == FunctionKind::ORDINARY;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Resolve function node as non aggregate function.
|
/** Resolve function node as non aggregate function.
|
||||||
@ -173,19 +194,19 @@ public:
|
|||||||
* Assume we have `multiIf` function with single condition, it can be converted to `if` function.
|
* Assume we have `multiIf` function with single condition, it can be converted to `if` function.
|
||||||
* Function name must be updated accordingly.
|
* Function name must be updated accordingly.
|
||||||
*/
|
*/
|
||||||
void resolveAsFunction(FunctionOverloadResolverPtr function_value, DataTypePtr result_type_value);
|
void resolveAsFunction(FunctionBasePtr function_value);
|
||||||
|
|
||||||
/** Resolve function node as aggregate function.
|
/** Resolve function node as aggregate function.
|
||||||
* It is important that function name is updated with resolved function name.
|
* It is important that function name is updated with resolved function name.
|
||||||
* Main motivation for this is query tree optimizations.
|
* Main motivation for this is query tree optimizations.
|
||||||
*/
|
*/
|
||||||
void resolveAsAggregateFunction(AggregateFunctionPtr aggregate_function_value, DataTypePtr result_type_value);
|
void resolveAsAggregateFunction(AggregateFunctionPtr aggregate_function_value);
|
||||||
|
|
||||||
/** Resolve function node as window function.
|
/** Resolve function node as window function.
|
||||||
* It is important that function name is updated with resolved function name.
|
* It is important that function name is updated with resolved function name.
|
||||||
* Main motivation for this is query tree optimizations.
|
* Main motivation for this is query tree optimizations.
|
||||||
*/
|
*/
|
||||||
void resolveAsWindowFunction(AggregateFunctionPtr window_function_value, DataTypePtr result_type_value);
|
void resolveAsWindowFunction(AggregateFunctionPtr window_function_value);
|
||||||
|
|
||||||
QueryTreeNodeType getNodeType() const override
|
QueryTreeNodeType getNodeType() const override
|
||||||
{
|
{
|
||||||
@ -194,12 +215,11 @@ public:
|
|||||||
|
|
||||||
DataTypePtr getResultType() const override
|
DataTypePtr getResultType() const override
|
||||||
{
|
{
|
||||||
if (!result_type)
|
if (!function)
|
||||||
throw Exception(ErrorCodes::UNSUPPORTED_METHOD,
|
throw Exception(ErrorCodes::UNSUPPORTED_METHOD,
|
||||||
"Function node with name '{}' is not resolved",
|
"Function node with name '{}' is not resolved",
|
||||||
function_name);
|
function_name);
|
||||||
|
return function->getResultType();
|
||||||
return result_type;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
void dumpTreeImpl(WriteBuffer & buffer, FormatState & format_state, size_t indent) const override;
|
void dumpTreeImpl(WriteBuffer & buffer, FormatState & format_state, size_t indent) const override;
|
||||||
@ -215,9 +235,8 @@ protected:
|
|||||||
|
|
||||||
private:
|
private:
|
||||||
String function_name;
|
String function_name;
|
||||||
FunctionOverloadResolverPtr function;
|
FunctionKind kind = FunctionKind::UNKNOWN;
|
||||||
AggregateFunctionPtr aggregate_function;
|
IResolvedFunctionPtr function;
|
||||||
DataTypePtr result_type;
|
|
||||||
|
|
||||||
static constexpr size_t parameters_child_index = 0;
|
static constexpr size_t parameters_child_index = 0;
|
||||||
static constexpr size_t arguments_child_index = 1;
|
static constexpr size_t arguments_child_index = 1;
|
||||||
|
@ -147,7 +147,6 @@ public:
|
|||||||
private:
|
private:
|
||||||
static inline void resolveAggregateFunctionNode(FunctionNode & function_node, const String & aggregate_function_name)
|
static inline void resolveAggregateFunctionNode(FunctionNode & function_node, const String & aggregate_function_name)
|
||||||
{
|
{
|
||||||
auto function_result_type = function_node.getResultType();
|
|
||||||
auto function_aggregate_function = function_node.getAggregateFunction();
|
auto function_aggregate_function = function_node.getAggregateFunction();
|
||||||
|
|
||||||
AggregateFunctionProperties properties;
|
AggregateFunctionProperties properties;
|
||||||
@ -156,7 +155,7 @@ private:
|
|||||||
function_aggregate_function->getParameters(),
|
function_aggregate_function->getParameters(),
|
||||||
properties);
|
properties);
|
||||||
|
|
||||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function), std::move(function_result_type));
|
function_node.resolveAsAggregateFunction(std::move(aggregate_function));
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
@ -71,7 +71,7 @@ public:
|
|||||||
auto result_type = function_node->getResultType();
|
auto result_type = function_node->getResultType();
|
||||||
AggregateFunctionProperties properties;
|
AggregateFunctionProperties properties;
|
||||||
auto aggregate_function = AggregateFunctionFactory::instance().get("count", {}, {}, properties);
|
auto aggregate_function = AggregateFunctionFactory::instance().get("count", {}, {}, properties);
|
||||||
function_node->resolveAsAggregateFunction(std::move(aggregate_function), std::move(result_type));
|
function_node->resolveAsAggregateFunction(std::move(aggregate_function));
|
||||||
function_node->getArguments().getNodes().clear();
|
function_node->getArguments().getNodes().clear();
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
@ -138,7 +138,6 @@ public:
|
|||||||
|
|
||||||
static inline void resolveAggregateOrWindowFunctionNode(FunctionNode & function_node, const String & aggregate_function_name)
|
static inline void resolveAggregateOrWindowFunctionNode(FunctionNode & function_node, const String & aggregate_function_name)
|
||||||
{
|
{
|
||||||
auto function_result_type = function_node.getResultType();
|
|
||||||
auto function_aggregate_function = function_node.getAggregateFunction();
|
auto function_aggregate_function = function_node.getAggregateFunction();
|
||||||
|
|
||||||
AggregateFunctionProperties properties;
|
AggregateFunctionProperties properties;
|
||||||
@ -148,16 +147,15 @@ public:
|
|||||||
properties);
|
properties);
|
||||||
|
|
||||||
if (function_node.isAggregateFunction())
|
if (function_node.isAggregateFunction())
|
||||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function), std::move(function_result_type));
|
function_node.resolveAsAggregateFunction(std::move(aggregate_function));
|
||||||
else if (function_node.isWindowFunction())
|
else if (function_node.isWindowFunction())
|
||||||
function_node.resolveAsWindowFunction(std::move(aggregate_function), std::move(function_result_type));
|
function_node.resolveAsWindowFunction(std::move(aggregate_function));
|
||||||
}
|
}
|
||||||
|
|
||||||
inline void resolveOrdinaryFunctionNode(FunctionNode & function_node, const String & function_name) const
|
inline void resolveOrdinaryFunctionNode(FunctionNode & function_node, const String & function_name) const
|
||||||
{
|
{
|
||||||
auto function_result_type = function_node.getResultType();
|
|
||||||
auto function = FunctionFactory::instance().get(function_name, context);
|
auto function = FunctionFactory::instance().get(function_name, context);
|
||||||
function_node.resolveAsFunction(function, std::move(function_result_type));
|
function_node.resolveAsFunction(function->build(function_node.getArgumentTypes()));
|
||||||
}
|
}
|
||||||
|
|
||||||
private:
|
private:
|
||||||
|
@ -78,11 +78,11 @@ public:
|
|||||||
column.name += ".size0";
|
column.name += ".size0";
|
||||||
column.type = std::make_shared<DataTypeUInt64>();
|
column.type = std::make_shared<DataTypeUInt64>();
|
||||||
|
|
||||||
resolveOrdinaryFunctionNode(*function_node, "equals");
|
|
||||||
|
|
||||||
function_arguments_nodes.clear();
|
function_arguments_nodes.clear();
|
||||||
function_arguments_nodes.push_back(std::make_shared<ColumnNode>(column, column_source));
|
function_arguments_nodes.push_back(std::make_shared<ColumnNode>(column, column_source));
|
||||||
function_arguments_nodes.push_back(std::make_shared<ConstantNode>(static_cast<UInt64>(0)));
|
function_arguments_nodes.push_back(std::make_shared<ConstantNode>(static_cast<UInt64>(0)));
|
||||||
|
|
||||||
|
resolveOrdinaryFunctionNode(*function_node, "equals");
|
||||||
}
|
}
|
||||||
else if (function_name == "notEmpty")
|
else if (function_name == "notEmpty")
|
||||||
{
|
{
|
||||||
@ -90,11 +90,11 @@ public:
|
|||||||
column.name += ".size0";
|
column.name += ".size0";
|
||||||
column.type = std::make_shared<DataTypeUInt64>();
|
column.type = std::make_shared<DataTypeUInt64>();
|
||||||
|
|
||||||
resolveOrdinaryFunctionNode(*function_node, "notEquals");
|
|
||||||
|
|
||||||
function_arguments_nodes.clear();
|
function_arguments_nodes.clear();
|
||||||
function_arguments_nodes.push_back(std::make_shared<ColumnNode>(column, column_source));
|
function_arguments_nodes.push_back(std::make_shared<ColumnNode>(column, column_source));
|
||||||
function_arguments_nodes.push_back(std::make_shared<ConstantNode>(static_cast<UInt64>(0)));
|
function_arguments_nodes.push_back(std::make_shared<ConstantNode>(static_cast<UInt64>(0)));
|
||||||
|
|
||||||
|
resolveOrdinaryFunctionNode(*function_node, "notEquals");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
else if (column_type.isNullable())
|
else if (column_type.isNullable())
|
||||||
@ -112,9 +112,9 @@ public:
|
|||||||
column.name += ".null";
|
column.name += ".null";
|
||||||
column.type = std::make_shared<DataTypeUInt8>();
|
column.type = std::make_shared<DataTypeUInt8>();
|
||||||
|
|
||||||
resolveOrdinaryFunctionNode(*function_node, "not");
|
|
||||||
|
|
||||||
function_arguments_nodes = {std::make_shared<ColumnNode>(column, column_source)};
|
function_arguments_nodes = {std::make_shared<ColumnNode>(column, column_source)};
|
||||||
|
|
||||||
|
resolveOrdinaryFunctionNode(*function_node, "not");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
else if (column_type.isMap())
|
else if (column_type.isMap())
|
||||||
@ -182,9 +182,9 @@ public:
|
|||||||
column.type = data_type_map.getKeyType();
|
column.type = data_type_map.getKeyType();
|
||||||
|
|
||||||
auto has_function_argument = std::make_shared<ColumnNode>(column, column_source);
|
auto has_function_argument = std::make_shared<ColumnNode>(column, column_source);
|
||||||
resolveOrdinaryFunctionNode(*function_node, "has");
|
|
||||||
|
|
||||||
function_arguments_nodes[0] = std::move(has_function_argument);
|
function_arguments_nodes[0] = std::move(has_function_argument);
|
||||||
|
|
||||||
|
resolveOrdinaryFunctionNode(*function_node, "has");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@ -192,9 +192,8 @@ public:
|
|||||||
private:
|
private:
|
||||||
inline void resolveOrdinaryFunctionNode(FunctionNode & function_node, const String & function_name) const
|
inline void resolveOrdinaryFunctionNode(FunctionNode & function_node, const String & function_name) const
|
||||||
{
|
{
|
||||||
auto function_result_type = function_node.getResultType();
|
|
||||||
auto function = FunctionFactory::instance().get(function_name, context);
|
auto function = FunctionFactory::instance().get(function_name, context);
|
||||||
function_node.resolveAsFunction(function, std::move(function_result_type));
|
function_node.resolveAsFunction(function->build(function_node.getArgumentTypes()));
|
||||||
}
|
}
|
||||||
|
|
||||||
ContextPtr & context;
|
ContextPtr & context;
|
||||||
|
@ -59,14 +59,13 @@ private:
|
|||||||
std::unordered_set<String> names_to_collect;
|
std::unordered_set<String> names_to_collect;
|
||||||
};
|
};
|
||||||
|
|
||||||
QueryTreeNodePtr createResolvedFunction(const ContextPtr & context, const String & name, const DataTypePtr & result_type, QueryTreeNodes arguments)
|
QueryTreeNodePtr createResolvedFunction(const ContextPtr & context, const String & name, QueryTreeNodes arguments)
|
||||||
{
|
{
|
||||||
auto function_node = std::make_shared<FunctionNode>(name);
|
auto function_node = std::make_shared<FunctionNode>(name);
|
||||||
|
|
||||||
auto function = FunctionFactory::instance().get(name, context);
|
auto function = FunctionFactory::instance().get(name, context);
|
||||||
function_node->resolveAsFunction(std::move(function), result_type);
|
|
||||||
function_node->getArguments().getNodes() = std::move(arguments);
|
function_node->getArguments().getNodes() = std::move(arguments);
|
||||||
|
function_node->resolveAsFunction(function->build(function_node->getArgumentTypes()));
|
||||||
return function_node;
|
return function_node;
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -74,11 +73,6 @@ FunctionNodePtr createResolvedAggregateFunction(const String & name, const Query
|
|||||||
{
|
{
|
||||||
auto function_node = std::make_shared<FunctionNode>(name);
|
auto function_node = std::make_shared<FunctionNode>(name);
|
||||||
|
|
||||||
AggregateFunctionProperties properties;
|
|
||||||
auto aggregate_function = AggregateFunctionFactory::instance().get(name, {argument->getResultType()}, parameters, properties);
|
|
||||||
function_node->resolveAsAggregateFunction(aggregate_function, aggregate_function->getReturnType());
|
|
||||||
function_node->getArguments().getNodes() = { argument };
|
|
||||||
|
|
||||||
if (!parameters.empty())
|
if (!parameters.empty())
|
||||||
{
|
{
|
||||||
QueryTreeNodes parameter_nodes;
|
QueryTreeNodes parameter_nodes;
|
||||||
@ -86,18 +80,27 @@ FunctionNodePtr createResolvedAggregateFunction(const String & name, const Query
|
|||||||
parameter_nodes.emplace_back(std::make_shared<ConstantNode>(param));
|
parameter_nodes.emplace_back(std::make_shared<ConstantNode>(param));
|
||||||
function_node->getParameters().getNodes() = std::move(parameter_nodes);
|
function_node->getParameters().getNodes() = std::move(parameter_nodes);
|
||||||
}
|
}
|
||||||
|
function_node->getArguments().getNodes() = { argument };
|
||||||
|
|
||||||
|
AggregateFunctionProperties properties;
|
||||||
|
auto aggregate_function = AggregateFunctionFactory::instance().get(
|
||||||
|
name,
|
||||||
|
{ argument->getResultType() },
|
||||||
|
parameters,
|
||||||
|
properties);
|
||||||
|
function_node->resolveAsAggregateFunction(aggregate_function);
|
||||||
|
|
||||||
return function_node;
|
return function_node;
|
||||||
}
|
}
|
||||||
|
|
||||||
QueryTreeNodePtr createTupleElementFunction(const ContextPtr & context, const DataTypePtr & result_type, QueryTreeNodePtr argument, UInt64 index)
|
QueryTreeNodePtr createTupleElementFunction(const ContextPtr & context, QueryTreeNodePtr argument, UInt64 index)
|
||||||
{
|
{
|
||||||
return createResolvedFunction(context, "tupleElement", result_type, {std::move(argument), std::make_shared<ConstantNode>(index)});
|
return createResolvedFunction(context, "tupleElement", {argument, std::make_shared<ConstantNode>(index)});
|
||||||
}
|
}
|
||||||
|
|
||||||
QueryTreeNodePtr createArrayElementFunction(const ContextPtr & context, const DataTypePtr & result_type, QueryTreeNodePtr argument, UInt64 index)
|
QueryTreeNodePtr createArrayElementFunction(const ContextPtr & context, QueryTreeNodePtr argument, UInt64 index)
|
||||||
{
|
{
|
||||||
return createResolvedFunction(context, "arrayElement", result_type, {std::move(argument), std::make_shared<ConstantNode>(index)});
|
return createResolvedFunction(context, "arrayElement", {argument, std::make_shared<ConstantNode>(index)});
|
||||||
}
|
}
|
||||||
|
|
||||||
void replaceWithSumCount(QueryTreeNodePtr & node, const FunctionNodePtr & sum_count_node, ContextPtr context)
|
void replaceWithSumCount(QueryTreeNodePtr & node, const FunctionNodePtr & sum_count_node, ContextPtr context)
|
||||||
@ -115,20 +118,20 @@ void replaceWithSumCount(QueryTreeNodePtr & node, const FunctionNodePtr & sum_co
|
|||||||
if (function_name == "sum")
|
if (function_name == "sum")
|
||||||
{
|
{
|
||||||
assert(node->getResultType()->equals(*sum_count_result_type->getElement(0)));
|
assert(node->getResultType()->equals(*sum_count_result_type->getElement(0)));
|
||||||
node = createTupleElementFunction(context, node->getResultType(), sum_count_node, 1);
|
node = createTupleElementFunction(context, sum_count_node, 1);
|
||||||
}
|
}
|
||||||
else if (function_name == "count")
|
else if (function_name == "count")
|
||||||
{
|
{
|
||||||
assert(node->getResultType()->equals(*sum_count_result_type->getElement(1)));
|
assert(node->getResultType()->equals(*sum_count_result_type->getElement(1)));
|
||||||
node = createTupleElementFunction(context, node->getResultType(), sum_count_node, 2);
|
node = createTupleElementFunction(context, sum_count_node, 2);
|
||||||
}
|
}
|
||||||
else if (function_name == "avg")
|
else if (function_name == "avg")
|
||||||
{
|
{
|
||||||
auto sum_result = createTupleElementFunction(context, sum_count_result_type->getElement(0), sum_count_node, 1);
|
auto sum_result = createTupleElementFunction(context, sum_count_node, 1);
|
||||||
auto count_result = createTupleElementFunction(context, sum_count_result_type->getElement(1), sum_count_node, 2);
|
auto count_result = createTupleElementFunction(context, sum_count_node, 2);
|
||||||
/// To avoid integer division by zero
|
/// To avoid integer division by zero
|
||||||
auto count_float_result = createResolvedFunction(context, "toFloat64", std::make_shared<DataTypeFloat64>(), {count_result});
|
auto count_float_result = createResolvedFunction(context, "toFloat64", {count_result});
|
||||||
node = createResolvedFunction(context, "divide", node->getResultType(), {sum_result, count_float_result});
|
node = createResolvedFunction(context, "divide", {sum_result, count_float_result});
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
@ -238,7 +241,7 @@ void tryFuseQuantiles(QueryTreeNodePtr query_tree_node, ContextPtr context)
|
|||||||
for (size_t i = 0; i < nodes_set.size(); ++i)
|
for (size_t i = 0; i < nodes_set.size(); ++i)
|
||||||
{
|
{
|
||||||
size_t array_index = i + 1;
|
size_t array_index = i + 1;
|
||||||
*nodes[i] = createArrayElementFunction(context, result_array_type->getNestedType(), quantiles_node, array_index);
|
*nodes[i] = createArrayElementFunction(context, quantiles_node, array_index);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@ -55,8 +55,8 @@ public:
|
|||||||
return;
|
return;
|
||||||
|
|
||||||
auto multi_if_function = std::make_shared<FunctionNode>("multiIf");
|
auto multi_if_function = std::make_shared<FunctionNode>("multiIf");
|
||||||
multi_if_function->resolveAsFunction(multi_if_function_ptr, std::make_shared<DataTypeUInt8>());
|
|
||||||
multi_if_function->getArguments().getNodes() = std::move(multi_if_arguments);
|
multi_if_function->getArguments().getNodes() = std::move(multi_if_arguments);
|
||||||
|
multi_if_function->resolveAsFunction(multi_if_function_ptr->build(multi_if_function->getArgumentTypes()));
|
||||||
node = std::move(multi_if_function);
|
node = std::move(multi_if_function);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -47,49 +47,64 @@ QueryTreeNodePtr createCastFunction(QueryTreeNodePtr from, DataTypePtr result_ty
|
|||||||
auto enum_literal_node = std::make_shared<ConstantNode>(std::move(enum_literal));
|
auto enum_literal_node = std::make_shared<ConstantNode>(std::move(enum_literal));
|
||||||
|
|
||||||
auto cast_function = FunctionFactory::instance().get("_CAST", std::move(context));
|
auto cast_function = FunctionFactory::instance().get("_CAST", std::move(context));
|
||||||
QueryTreeNodes arguments{std::move(from), std::move(enum_literal_node)};
|
QueryTreeNodes arguments{ std::move(from), std::move(enum_literal_node) };
|
||||||
|
|
||||||
auto function_node = std::make_shared<FunctionNode>("_CAST");
|
auto function_node = std::make_shared<FunctionNode>("_CAST");
|
||||||
function_node->resolveAsFunction(std::move(cast_function), std::move(result_type));
|
|
||||||
function_node->getArguments().getNodes() = std::move(arguments);
|
function_node->getArguments().getNodes() = std::move(arguments);
|
||||||
|
|
||||||
|
function_node->resolveAsFunction(cast_function->build(function_node->getArgumentTypes()));
|
||||||
|
|
||||||
return function_node;
|
return function_node;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// if(arg1, arg2, arg3) will be transformed to if(arg1, _CAST(arg2, Enum...), _CAST(arg3, Enum...))
|
/// if(arg1, arg2, arg3) will be transformed to if(arg1, _CAST(arg2, Enum...), _CAST(arg3, Enum...))
|
||||||
/// where Enum is generated based on the possible values stored in string_values
|
/// where Enum is generated based on the possible values stored in string_values
|
||||||
void changeIfArguments(
|
void changeIfArguments(
|
||||||
QueryTreeNodePtr & first, QueryTreeNodePtr & second, const std::set<std::string> & string_values, const ContextPtr & context)
|
FunctionNode & if_node, const std::set<std::string> & string_values, const ContextPtr & context)
|
||||||
{
|
{
|
||||||
auto result_type = getEnumType(string_values);
|
auto result_type = getEnumType(string_values);
|
||||||
|
|
||||||
first = createCastFunction(first, result_type, context);
|
auto & argument_nodes = if_node.getArguments().getNodes();
|
||||||
second = createCastFunction(second, result_type, context);
|
|
||||||
|
argument_nodes[1] = createCastFunction(argument_nodes[1], result_type, context);
|
||||||
|
argument_nodes[2] = createCastFunction(argument_nodes[2], result_type, context);
|
||||||
|
|
||||||
|
auto if_resolver = FunctionFactory::instance().get("if", context);
|
||||||
|
|
||||||
|
if_node.resolveAsFunction(if_resolver->build(if_node.getArgumentTypes()));
|
||||||
}
|
}
|
||||||
|
|
||||||
/// transform(value, array_from, array_to, default_value) will be transformed to transform(value, array_from, _CAST(array_to, Array(Enum...)), _CAST(default_value, Enum...))
|
/// transform(value, array_from, array_to, default_value) will be transformed to transform(value, array_from, _CAST(array_to, Array(Enum...)), _CAST(default_value, Enum...))
|
||||||
/// where Enum is generated based on the possible values stored in string_values
|
/// where Enum is generated based on the possible values stored in string_values
|
||||||
void changeTransformArguments(
|
void changeTransformArguments(
|
||||||
QueryTreeNodePtr & array_to,
|
FunctionNode & transform_node,
|
||||||
QueryTreeNodePtr & default_value,
|
|
||||||
const std::set<std::string> & string_values,
|
const std::set<std::string> & string_values,
|
||||||
const ContextPtr & context)
|
const ContextPtr & context)
|
||||||
{
|
{
|
||||||
auto result_type = getEnumType(string_values);
|
auto result_type = getEnumType(string_values);
|
||||||
|
|
||||||
|
auto & arguments = transform_node.getArguments().getNodes();
|
||||||
|
|
||||||
|
auto & array_to = arguments[2];
|
||||||
|
auto & default_value = arguments[3];
|
||||||
|
|
||||||
array_to = createCastFunction(array_to, std::make_shared<DataTypeArray>(result_type), context);
|
array_to = createCastFunction(array_to, std::make_shared<DataTypeArray>(result_type), context);
|
||||||
default_value = createCastFunction(default_value, std::move(result_type), context);
|
default_value = createCastFunction(default_value, std::move(result_type), context);
|
||||||
|
|
||||||
|
auto transform_resolver = FunctionFactory::instance().get("transform", context);
|
||||||
|
|
||||||
|
transform_node.resolveAsFunction(transform_resolver->build(transform_node.getArgumentTypes()));
|
||||||
}
|
}
|
||||||
|
|
||||||
void wrapIntoToString(FunctionNode & function_node, QueryTreeNodePtr arg, ContextPtr context)
|
void wrapIntoToString(FunctionNode & function_node, QueryTreeNodePtr arg, ContextPtr context)
|
||||||
{
|
{
|
||||||
assert(isString(function_node.getResultType()));
|
|
||||||
|
|
||||||
auto to_string_function = FunctionFactory::instance().get("toString", std::move(context));
|
auto to_string_function = FunctionFactory::instance().get("toString", std::move(context));
|
||||||
QueryTreeNodes arguments{std::move(arg)};
|
QueryTreeNodes arguments{ std::move(arg) };
|
||||||
|
|
||||||
function_node.resolveAsFunction(std::move(to_string_function), std::make_shared<DataTypeString>());
|
|
||||||
function_node.getArguments().getNodes() = std::move(arguments);
|
function_node.getArguments().getNodes() = std::move(arguments);
|
||||||
|
|
||||||
|
function_node.resolveAsFunction(to_string_function->build(function_node.getArgumentTypes()));
|
||||||
|
|
||||||
|
assert(isString(function_node.getResultType()));
|
||||||
}
|
}
|
||||||
|
|
||||||
class ConvertStringsToEnumVisitor : public InDepthQueryTreeVisitor<ConvertStringsToEnumVisitor>
|
class ConvertStringsToEnumVisitor : public InDepthQueryTreeVisitor<ConvertStringsToEnumVisitor>
|
||||||
@ -117,7 +132,8 @@ public:
|
|||||||
return;
|
return;
|
||||||
|
|
||||||
auto modified_if_node = function_node->clone();
|
auto modified_if_node = function_node->clone();
|
||||||
auto & argument_nodes = modified_if_node->as<FunctionNode>()->getArguments().getNodes();
|
auto * function_if_node = modified_if_node->as<FunctionNode>();
|
||||||
|
auto & argument_nodes = function_if_node->getArguments().getNodes();
|
||||||
|
|
||||||
const auto * first_literal = argument_nodes[1]->as<ConstantNode>();
|
const auto * first_literal = argument_nodes[1]->as<ConstantNode>();
|
||||||
const auto * second_literal = argument_nodes[2]->as<ConstantNode>();
|
const auto * second_literal = argument_nodes[2]->as<ConstantNode>();
|
||||||
@ -132,7 +148,7 @@ public:
|
|||||||
string_values.insert(first_literal->getValue().get<std::string>());
|
string_values.insert(first_literal->getValue().get<std::string>());
|
||||||
string_values.insert(second_literal->getValue().get<std::string>());
|
string_values.insert(second_literal->getValue().get<std::string>());
|
||||||
|
|
||||||
changeIfArguments(argument_nodes[1], argument_nodes[2], string_values, context);
|
changeIfArguments(*function_if_node, string_values, context);
|
||||||
wrapIntoToString(*function_node, std::move(modified_if_node), context);
|
wrapIntoToString(*function_node, std::move(modified_if_node), context);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@ -143,7 +159,8 @@ public:
|
|||||||
return;
|
return;
|
||||||
|
|
||||||
auto modified_transform_node = function_node->clone();
|
auto modified_transform_node = function_node->clone();
|
||||||
auto & argument_nodes = modified_transform_node->as<FunctionNode>()->getArguments().getNodes();
|
auto * function_modified_transform_node = modified_transform_node->as<FunctionNode>();
|
||||||
|
auto & argument_nodes = function_modified_transform_node->getArguments().getNodes();
|
||||||
|
|
||||||
if (!isString(function_node->getResultType()))
|
if (!isString(function_node->getResultType()))
|
||||||
return;
|
return;
|
||||||
@ -176,7 +193,7 @@ public:
|
|||||||
|
|
||||||
string_values.insert(literal_default->getValue().get<std::string>());
|
string_values.insert(literal_default->getValue().get<std::string>());
|
||||||
|
|
||||||
changeTransformArguments(argument_nodes[2], argument_nodes[3], string_values, context);
|
changeTransformArguments(*function_modified_transform_node, string_values, context);
|
||||||
wrapIntoToString(*function_node, std::move(modified_transform_node), context);
|
wrapIntoToString(*function_node, std::move(modified_transform_node), context);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
@ -27,7 +27,7 @@ public:
|
|||||||
return;
|
return;
|
||||||
|
|
||||||
auto result_type = function_node->getResultType();
|
auto result_type = function_node->getResultType();
|
||||||
function_node->resolveAsFunction(if_function_ptr, std::move(result_type));
|
function_node->resolveAsFunction(if_function_ptr->build(function_node->getArgumentTypes()));
|
||||||
}
|
}
|
||||||
|
|
||||||
private:
|
private:
|
||||||
|
@ -53,12 +53,10 @@ private:
|
|||||||
|
|
||||||
static inline void resolveAsCountAggregateFunction(FunctionNode & function_node)
|
static inline void resolveAsCountAggregateFunction(FunctionNode & function_node)
|
||||||
{
|
{
|
||||||
auto function_result_type = function_node.getResultType();
|
|
||||||
|
|
||||||
AggregateFunctionProperties properties;
|
AggregateFunctionProperties properties;
|
||||||
auto aggregate_function = AggregateFunctionFactory::instance().get("count", {}, {}, properties);
|
auto aggregate_function = AggregateFunctionFactory::instance().get("count", {}, {}, properties);
|
||||||
|
|
||||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function), std::move(function_result_type));
|
function_node.resolveAsAggregateFunction(std::move(aggregate_function));
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
@ -4302,7 +4302,7 @@ ProjectionNames QueryAnalyzer::resolveFunction(QueryTreeNodePtr & node, Identifi
|
|||||||
bool force_grouping_standard_compatibility = scope.context->getSettingsRef().force_grouping_standard_compatibility;
|
bool force_grouping_standard_compatibility = scope.context->getSettingsRef().force_grouping_standard_compatibility;
|
||||||
auto grouping_function = std::make_shared<FunctionGrouping>(force_grouping_standard_compatibility);
|
auto grouping_function = std::make_shared<FunctionGrouping>(force_grouping_standard_compatibility);
|
||||||
auto grouping_function_adaptor = std::make_shared<FunctionToOverloadResolverAdaptor>(std::move(grouping_function));
|
auto grouping_function_adaptor = std::make_shared<FunctionToOverloadResolverAdaptor>(std::move(grouping_function));
|
||||||
function_node.resolveAsFunction(std::move(grouping_function_adaptor), std::make_shared<DataTypeUInt64>());
|
function_node.resolveAsFunction(grouping_function_adaptor->build({}));
|
||||||
return result_projection_names;
|
return result_projection_names;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@ -4327,7 +4327,7 @@ ProjectionNames QueryAnalyzer::resolveFunction(QueryTreeNodePtr & node, Identifi
|
|||||||
AggregateFunctionProperties properties;
|
AggregateFunctionProperties properties;
|
||||||
auto aggregate_function = AggregateFunctionFactory::instance().get(function_name, argument_types, parameters, properties);
|
auto aggregate_function = AggregateFunctionFactory::instance().get(function_name, argument_types, parameters, properties);
|
||||||
|
|
||||||
function_node.resolveAsWindowFunction(aggregate_function, aggregate_function->getReturnType());
|
function_node.resolveAsWindowFunction(aggregate_function);
|
||||||
|
|
||||||
bool window_node_is_identifier = function_node.getWindowNode()->getNodeType() == QueryTreeNodeType::IDENTIFIER;
|
bool window_node_is_identifier = function_node.getWindowNode()->getNodeType() == QueryTreeNodeType::IDENTIFIER;
|
||||||
ProjectionName window_projection_name = resolveWindow(function_node.getWindowNode(), scope);
|
ProjectionName window_projection_name = resolveWindow(function_node.getWindowNode(), scope);
|
||||||
@ -4386,7 +4386,7 @@ ProjectionNames QueryAnalyzer::resolveFunction(QueryTreeNodePtr & node, Identifi
|
|||||||
|
|
||||||
AggregateFunctionProperties properties;
|
AggregateFunctionProperties properties;
|
||||||
auto aggregate_function = AggregateFunctionFactory::instance().get(function_name, argument_types, parameters, properties);
|
auto aggregate_function = AggregateFunctionFactory::instance().get(function_name, argument_types, parameters, properties);
|
||||||
function_node.resolveAsAggregateFunction(aggregate_function, aggregate_function->getReturnType());
|
function_node.resolveAsAggregateFunction(aggregate_function);
|
||||||
return result_projection_names;
|
return result_projection_names;
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -4563,6 +4563,8 @@ ProjectionNames QueryAnalyzer::resolveFunction(QueryTreeNodePtr & node, Identifi
|
|||||||
constant_value = std::make_shared<ConstantValue>(std::move(column_constant_value), result_type);
|
constant_value = std::make_shared<ConstantValue>(std::move(column_constant_value), result_type);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function_node.resolveAsFunction(std::move(function_base));
|
||||||
}
|
}
|
||||||
catch (Exception & e)
|
catch (Exception & e)
|
||||||
{
|
{
|
||||||
@ -4570,8 +4572,6 @@ ProjectionNames QueryAnalyzer::resolveFunction(QueryTreeNodePtr & node, Identifi
|
|||||||
throw;
|
throw;
|
||||||
}
|
}
|
||||||
|
|
||||||
function_node.resolveAsFunction(std::move(function), std::move(result_type));
|
|
||||||
|
|
||||||
if (constant_value)
|
if (constant_value)
|
||||||
node = std::make_shared<ConstantNode>(std::move(constant_value), node);
|
node = std::make_shared<ConstantNode>(std::move(constant_value), node);
|
||||||
|
|
||||||
|
@ -117,11 +117,12 @@ public:
|
|||||||
not_function_result_type = makeNullable(not_function_result_type);
|
not_function_result_type = makeNullable(not_function_result_type);
|
||||||
|
|
||||||
auto not_function = std::make_shared<FunctionNode>("not");
|
auto not_function = std::make_shared<FunctionNode>("not");
|
||||||
not_function->resolveAsFunction(FunctionFactory::instance().get("not", context), std::move(not_function_result_type));
|
|
||||||
|
|
||||||
auto & not_function_arguments = not_function->getArguments().getNodes();
|
auto & not_function_arguments = not_function->getArguments().getNodes();
|
||||||
not_function_arguments.push_back(std::move(nested_if_function_arguments_nodes[0]));
|
not_function_arguments.push_back(std::move(nested_if_function_arguments_nodes[0]));
|
||||||
|
|
||||||
|
not_function->resolveAsFunction(FunctionFactory::instance().get("not", context)->build(not_function->getArgumentTypes()));
|
||||||
|
|
||||||
function_node_arguments_nodes[0] = std::move(not_function);
|
function_node_arguments_nodes[0] = std::move(not_function);
|
||||||
function_node_arguments_nodes.resize(1);
|
function_node_arguments_nodes.resize(1);
|
||||||
|
|
||||||
@ -139,8 +140,7 @@ private:
|
|||||||
function_node.getAggregateFunction()->getParameters(),
|
function_node.getAggregateFunction()->getParameters(),
|
||||||
properties);
|
properties);
|
||||||
|
|
||||||
auto function_result_type = function_node.getResultType();
|
function_node.resolveAsAggregateFunction(std::move(aggregate_function));
|
||||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function), std::move(function_result_type));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
ContextPtr & context;
|
ContextPtr & context;
|
||||||
|
@ -76,7 +76,7 @@ public:
|
|||||||
properties);
|
properties);
|
||||||
|
|
||||||
auto function_result_type = function_node->getResultType();
|
auto function_result_type = function_node->getResultType();
|
||||||
function_node->resolveAsAggregateFunction(std::move(aggregate_function), std::move(function_result_type));
|
function_node->resolveAsAggregateFunction(std::move(aggregate_function));
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
@ -21,6 +21,7 @@
|
|||||||
|
|
||||||
#include <Interpreters/Context.h>
|
#include <Interpreters/Context.h>
|
||||||
#include <Analyzer/ColumnNode.h>
|
#include <Analyzer/ColumnNode.h>
|
||||||
|
#include <Analyzer/FunctionNode.h>
|
||||||
#include <Analyzer/InDepthQueryTreeVisitor.h>
|
#include <Analyzer/InDepthQueryTreeVisitor.h>
|
||||||
#include <Common/Exception.h>
|
#include <Common/Exception.h>
|
||||||
|
|
||||||
@ -44,6 +45,23 @@ namespace
|
|||||||
class ValidationChecker : public InDepthQueryTreeVisitor<ValidationChecker>
|
class ValidationChecker : public InDepthQueryTreeVisitor<ValidationChecker>
|
||||||
{
|
{
|
||||||
String pass_name;
|
String pass_name;
|
||||||
|
|
||||||
|
void visitColumn(ColumnNode * column) const
|
||||||
|
{
|
||||||
|
if (column->getColumnSourceOrNull() == nullptr)
|
||||||
|
throw Exception(ErrorCodes::LOGICAL_ERROR,
|
||||||
|
"Column {} {} query tree node does not have valid source node after running {} pass",
|
||||||
|
column->getColumnName(), column->getColumnType(), pass_name);
|
||||||
|
}
|
||||||
|
|
||||||
|
void visitFunction(FunctionNode * function) const
|
||||||
|
{
|
||||||
|
if (!function->isResolved())
|
||||||
|
throw Exception(ErrorCodes::LOGICAL_ERROR,
|
||||||
|
"Function {} is not resolved after running {} pass",
|
||||||
|
function->dumpTree(), pass_name);
|
||||||
|
}
|
||||||
|
|
||||||
public:
|
public:
|
||||||
explicit ValidationChecker(String pass_name_)
|
explicit ValidationChecker(String pass_name_)
|
||||||
: pass_name(std::move(pass_name_))
|
: pass_name(std::move(pass_name_))
|
||||||
@ -51,13 +69,10 @@ public:
|
|||||||
|
|
||||||
void visitImpl(QueryTreeNodePtr & node) const
|
void visitImpl(QueryTreeNodePtr & node) const
|
||||||
{
|
{
|
||||||
auto * column = node->as<ColumnNode>();
|
if (auto * column = node->as<ColumnNode>())
|
||||||
if (!column)
|
return visitColumn(column);
|
||||||
return;
|
else if (auto * function = node->as<FunctionNode>())
|
||||||
if (column->getColumnSourceOrNull() == nullptr)
|
return visitFunction(function);
|
||||||
throw Exception(ErrorCodes::LOGICAL_ERROR,
|
|
||||||
"Column {} {} query tree node does not have valid source node after running {} pass",
|
|
||||||
column->getColumnName(), column->getColumnType(), pass_name);
|
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
#endif
|
#endif
|
||||||
|
@ -91,7 +91,8 @@ bool SortNode::isEqualImpl(const IQueryTreeNode & rhs) const
|
|||||||
void SortNode::updateTreeHashImpl(HashState & hash_state) const
|
void SortNode::updateTreeHashImpl(HashState & hash_state) const
|
||||||
{
|
{
|
||||||
hash_state.update(sort_direction);
|
hash_state.update(sort_direction);
|
||||||
hash_state.update(nulls_sort_direction);
|
/// use some determined value if `nulls_sort_direction` is `nullopt`
|
||||||
|
hash_state.update(nulls_sort_direction.value_or(sort_direction));
|
||||||
hash_state.update(with_fill);
|
hash_state.update(with_fill);
|
||||||
|
|
||||||
if (collator)
|
if (collator)
|
||||||
|
@ -146,7 +146,7 @@ MutableColumnPtr ColumnAggregateFunction::convertToValues(MutableColumnPtr colum
|
|||||||
/// insertResultInto may invalidate states, so we must unshare ownership of them
|
/// insertResultInto may invalidate states, so we must unshare ownership of them
|
||||||
column_aggregate_func.ensureOwnership();
|
column_aggregate_func.ensureOwnership();
|
||||||
|
|
||||||
MutableColumnPtr res = func->getReturnType()->createColumn();
|
MutableColumnPtr res = func->getResultType()->createColumn();
|
||||||
res->reserve(data.size());
|
res->reserve(data.size());
|
||||||
|
|
||||||
/// If there are references to states in final column, we must hold their ownership
|
/// If there are references to states in final column, we must hold their ownership
|
||||||
|
@ -13,7 +13,7 @@ namespace ErrorCodes
|
|||||||
}
|
}
|
||||||
|
|
||||||
class IFunctionBase;
|
class IFunctionBase;
|
||||||
using FunctionBasePtr = std::shared_ptr<IFunctionBase>;
|
using FunctionBasePtr = std::shared_ptr<const IFunctionBase>;
|
||||||
|
|
||||||
/** A column containing a lambda expression.
|
/** A column containing a lambda expression.
|
||||||
* Behaves like a constant-column. Contains an expression, but not input or output data.
|
* Behaves like a constant-column. Contains an expression, but not input or output data.
|
||||||
|
76
src/Common/BinStringDecodeHelper.h
Normal file
76
src/Common/BinStringDecodeHelper.h
Normal file
@ -0,0 +1,76 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <Common/hex.h>
|
||||||
|
|
||||||
|
namespace DB
|
||||||
|
{
|
||||||
|
|
||||||
|
static void inline hexStringDecode(const char * pos, const char * end, char *& out, size_t word_size = 2)
|
||||||
|
{
|
||||||
|
if ((end - pos) & 1)
|
||||||
|
{
|
||||||
|
*out = unhex(*pos);
|
||||||
|
++out;
|
||||||
|
++pos;
|
||||||
|
}
|
||||||
|
while (pos < end)
|
||||||
|
{
|
||||||
|
*out = unhex2(pos);
|
||||||
|
pos += word_size;
|
||||||
|
++out;
|
||||||
|
}
|
||||||
|
*out = '\0';
|
||||||
|
++out;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void inline binStringDecode(const char * pos, const char * end, char *& out)
|
||||||
|
{
|
||||||
|
if (pos == end)
|
||||||
|
{
|
||||||
|
*out = '\0';
|
||||||
|
++out;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
UInt8 left = 0;
|
||||||
|
|
||||||
|
/// end - pos is the length of input.
|
||||||
|
/// (length & 7) to make remain bits length mod 8 is zero to split.
|
||||||
|
/// e.g. the length is 9 and the input is "101000001",
|
||||||
|
/// first left_cnt is 1, left is 0, right shift, pos is 1, left = 1
|
||||||
|
/// then, left_cnt is 0, remain input is '01000001'.
|
||||||
|
for (UInt8 left_cnt = (end - pos) & 7; left_cnt > 0; --left_cnt)
|
||||||
|
{
|
||||||
|
left = left << 1;
|
||||||
|
if (*pos != '0')
|
||||||
|
left += 1;
|
||||||
|
++pos;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (left != 0 || end - pos == 0)
|
||||||
|
{
|
||||||
|
*out = left;
|
||||||
|
++out;
|
||||||
|
}
|
||||||
|
|
||||||
|
assert((end - pos) % 8 == 0);
|
||||||
|
|
||||||
|
while (end - pos != 0)
|
||||||
|
{
|
||||||
|
UInt8 c = 0;
|
||||||
|
for (UInt8 i = 0; i < 8; ++i)
|
||||||
|
{
|
||||||
|
c = c << 1;
|
||||||
|
if (*pos != '0')
|
||||||
|
c += 1;
|
||||||
|
++pos;
|
||||||
|
}
|
||||||
|
*out = c;
|
||||||
|
++out;
|
||||||
|
}
|
||||||
|
|
||||||
|
*out = '\0';
|
||||||
|
++out;
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
@ -261,3 +261,31 @@ TEST(Common, RWLockPerfTestReaders)
|
|||||||
std::cout << "Threads " << pool_size << ", total_time " << std::setprecision(2) << total_time << "\n";
|
std::cout << "Threads " << pool_size << ", total_time " << std::setprecision(2) << total_time << "\n";
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
TEST(Common, RWLockNotUpgradeableWithNoQuery)
|
||||||
|
{
|
||||||
|
updatePHDRCache();
|
||||||
|
|
||||||
|
static auto rw_lock = RWLockImpl::create();
|
||||||
|
|
||||||
|
std::thread read_thread([&] ()
|
||||||
|
{
|
||||||
|
auto lock = rw_lock->getLock(RWLockImpl::Read, RWLockImpl::NO_QUERY, std::chrono::duration<int, std::milli>(50000));
|
||||||
|
auto sleep_for = std::chrono::duration<int, std::milli>(5000);
|
||||||
|
std::this_thread::sleep_for(sleep_for);
|
||||||
|
});
|
||||||
|
|
||||||
|
{
|
||||||
|
auto sleep_for = std::chrono::duration<int, std::milli>(500);
|
||||||
|
std::this_thread::sleep_for(sleep_for);
|
||||||
|
|
||||||
|
Stopwatch watch(CLOCK_MONOTONIC_COARSE);
|
||||||
|
auto get_lock = rw_lock->getLock(RWLockImpl::Write, RWLockImpl::NO_QUERY, std::chrono::duration<int, std::milli>(50000));
|
||||||
|
|
||||||
|
EXPECT_NE(get_lock.get(), nullptr);
|
||||||
|
/// It took some time
|
||||||
|
EXPECT_GT(watch.elapsedMilliseconds(), 3000);
|
||||||
|
}
|
||||||
|
|
||||||
|
read_thread.join();
|
||||||
|
}
|
||||||
|
29
src/Core/IResolvedFunction.h
Normal file
29
src/Core/IResolvedFunction.h
Normal file
@ -0,0 +1,29 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <memory>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace DB
|
||||||
|
{
|
||||||
|
class IDataType;
|
||||||
|
|
||||||
|
using DataTypePtr = std::shared_ptr<const IDataType>;
|
||||||
|
using DataTypes = std::vector<DataTypePtr>;
|
||||||
|
|
||||||
|
struct Array;
|
||||||
|
|
||||||
|
class IResolvedFunction
|
||||||
|
{
|
||||||
|
public:
|
||||||
|
virtual const DataTypePtr & getResultType() const = 0;
|
||||||
|
|
||||||
|
virtual const DataTypes & getArgumentTypes() const = 0;
|
||||||
|
|
||||||
|
virtual const Array & getParameters() const = 0;
|
||||||
|
|
||||||
|
virtual ~IResolvedFunction() = default;
|
||||||
|
};
|
||||||
|
|
||||||
|
using IResolvedFunctionPtr = std::shared_ptr<const IResolvedFunction>;
|
||||||
|
|
||||||
|
}
|
@ -620,6 +620,7 @@ static constexpr UInt64 operator""_GiB(unsigned long long value)
|
|||||||
M(Bool, enable_filesystem_cache_on_lower_level, true, "If read buffer supports caching inside threadpool, allow it to do it, otherwise cache outside ot threadpool. Do not use this setting, it is needed for testing", 0) \
|
M(Bool, enable_filesystem_cache_on_lower_level, true, "If read buffer supports caching inside threadpool, allow it to do it, otherwise cache outside ot threadpool. Do not use this setting, it is needed for testing", 0) \
|
||||||
M(Bool, skip_download_if_exceeds_query_cache, true, "Skip download from remote filesystem if exceeds query cache size", 0) \
|
M(Bool, skip_download_if_exceeds_query_cache, true, "Skip download from remote filesystem if exceeds query cache size", 0) \
|
||||||
M(UInt64, max_query_cache_size, (128UL * 1024 * 1024 * 1024), "Max remote filesystem cache size that can be used by a single query", 0) \
|
M(UInt64, max_query_cache_size, (128UL * 1024 * 1024 * 1024), "Max remote filesystem cache size that can be used by a single query", 0) \
|
||||||
|
M(Bool, throw_on_error_from_cache_on_write_operations, false, "Ignore error from cache when caching on write operations (INSERT, merges)", 0) \
|
||||||
\
|
\
|
||||||
M(Bool, load_marks_asynchronously, false, "Load MergeTree marks asynchronously", 0) \
|
M(Bool, load_marks_asynchronously, false, "Load MergeTree marks asynchronously", 0) \
|
||||||
\
|
\
|
||||||
|
@ -1025,9 +1025,6 @@ void BaseDaemon::setupWatchdog()
|
|||||||
#if defined(OS_LINUX)
|
#if defined(OS_LINUX)
|
||||||
if (0 != prctl(PR_SET_PDEATHSIG, SIGKILL))
|
if (0 != prctl(PR_SET_PDEATHSIG, SIGKILL))
|
||||||
logger().warning("Cannot do prctl to ask termination with parent.");
|
logger().warning("Cannot do prctl to ask termination with parent.");
|
||||||
|
|
||||||
if (getppid() == 1)
|
|
||||||
throw Poco::Exception("Parent watchdog process has exited.");
|
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
{
|
{
|
||||||
|
@ -30,9 +30,9 @@ private:
|
|||||||
public:
|
public:
|
||||||
static constexpr bool is_parametric = true;
|
static constexpr bool is_parametric = true;
|
||||||
|
|
||||||
DataTypeAggregateFunction(const AggregateFunctionPtr & function_, const DataTypes & argument_types_,
|
DataTypeAggregateFunction(AggregateFunctionPtr function_, const DataTypes & argument_types_,
|
||||||
const Array & parameters_, std::optional<size_t> version_ = std::nullopt)
|
const Array & parameters_, std::optional<size_t> version_ = std::nullopt)
|
||||||
: function(function_)
|
: function(std::move(function_))
|
||||||
, argument_types(argument_types_)
|
, argument_types(argument_types_)
|
||||||
, parameters(parameters_)
|
, parameters(parameters_)
|
||||||
, version(version_)
|
, version(version_)
|
||||||
@ -51,7 +51,7 @@ public:
|
|||||||
|
|
||||||
bool canBeInsideNullable() const override { return false; }
|
bool canBeInsideNullable() const override { return false; }
|
||||||
|
|
||||||
DataTypePtr getReturnType() const { return function->getReturnType(); }
|
DataTypePtr getReturnType() const { return function->getResultType(); }
|
||||||
DataTypePtr getReturnTypeToPredict() const { return function->getReturnTypeToPredict(); }
|
DataTypePtr getReturnTypeToPredict() const { return function->getReturnTypeToPredict(); }
|
||||||
DataTypes getArgumentsDataTypes() const { return argument_types; }
|
DataTypes getArgumentsDataTypes() const { return argument_types; }
|
||||||
|
|
||||||
|
@ -131,9 +131,9 @@ static std::pair<DataTypePtr, DataTypeCustomDescPtr> create(const ASTPtr & argum
|
|||||||
|
|
||||||
DataTypePtr storage_type = DataTypeFactory::instance().get(argument_types[0]->getName());
|
DataTypePtr storage_type = DataTypeFactory::instance().get(argument_types[0]->getName());
|
||||||
|
|
||||||
if (!function->getReturnType()->equals(*removeLowCardinality(storage_type)))
|
if (!function->getResultType()->equals(*removeLowCardinality(storage_type)))
|
||||||
{
|
{
|
||||||
throw Exception("Incompatible data types between aggregate function '" + function->getName() + "' which returns " + function->getReturnType()->getName() + " and column storage type " + storage_type->getName(),
|
throw Exception("Incompatible data types between aggregate function '" + function->getName() + "' which returns " + function->getResultType()->getName() + " and column storage type " + storage_type->getName(),
|
||||||
ErrorCodes::BAD_ARGUMENTS);
|
ErrorCodes::BAD_ARGUMENTS);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -256,6 +256,9 @@ void DatabaseOrdinary::startupTables(ThreadPool & thread_pool, LoadingStrictness
|
|||||||
|
|
||||||
auto startup_one_table = [&](const StoragePtr & table)
|
auto startup_one_table = [&](const StoragePtr & table)
|
||||||
{
|
{
|
||||||
|
/// Since startup() method can use physical paths on disk we don't allow any exclusive actions (rename, drop so on)
|
||||||
|
/// until startup finished.
|
||||||
|
auto table_lock_holder = table->lockForShare(RWLockImpl::NO_QUERY, getContext()->getSettingsRef().lock_acquire_timeout);
|
||||||
table->startup();
|
table->startup();
|
||||||
logAboutProgress(log, ++tables_processed, total_tables, watch);
|
logAboutProgress(log, ++tables_processed, total_tables, watch);
|
||||||
};
|
};
|
||||||
|
@ -44,10 +44,10 @@ FileSegmentRangeWriter::FileSegmentRangeWriter(
|
|||||||
const String & source_path_)
|
const String & source_path_)
|
||||||
: cache(cache_)
|
: cache(cache_)
|
||||||
, key(key_)
|
, key(key_)
|
||||||
|
, log(&Poco::Logger::get("FileSegmentRangeWriter"))
|
||||||
, cache_log(cache_log_)
|
, cache_log(cache_log_)
|
||||||
, query_id(query_id_)
|
, query_id(query_id_)
|
||||||
, source_path(source_path_)
|
, source_path(source_path_)
|
||||||
, current_file_segment_it(file_segments_holder.file_segments.end())
|
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -56,69 +56,68 @@ bool FileSegmentRangeWriter::write(const char * data, size_t size, size_t offset
|
|||||||
if (finalized)
|
if (finalized)
|
||||||
return false;
|
return false;
|
||||||
|
|
||||||
auto & file_segments = file_segments_holder.file_segments;
|
if (expected_write_offset != offset)
|
||||||
|
|
||||||
if (current_file_segment_it == file_segments.end())
|
|
||||||
{
|
|
||||||
current_file_segment_it = allocateFileSegment(current_file_segment_write_offset, is_persistent);
|
|
||||||
}
|
|
||||||
else
|
|
||||||
{
|
|
||||||
auto file_segment = *current_file_segment_it;
|
|
||||||
assert(file_segment->getCurrentWriteOffset() == current_file_segment_write_offset);
|
|
||||||
|
|
||||||
if (current_file_segment_write_offset != offset)
|
|
||||||
{
|
{
|
||||||
throw Exception(
|
throw Exception(
|
||||||
ErrorCodes::LOGICAL_ERROR,
|
ErrorCodes::LOGICAL_ERROR,
|
||||||
"Cannot write file segment at offset {}, because current write offset is: {}",
|
"Cannot write file segment at offset {}, because expected write offset is: {}",
|
||||||
offset, current_file_segment_write_offset);
|
offset, expected_write_offset);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (file_segment->range().size() == file_segment->getDownloadedSize())
|
auto & file_segments = file_segments_holder.file_segments;
|
||||||
|
|
||||||
|
if (file_segments.empty() || file_segments.back()->isDownloaded())
|
||||||
{
|
{
|
||||||
completeFileSegment(*file_segment);
|
allocateFileSegment(expected_write_offset, is_persistent);
|
||||||
current_file_segment_it = allocateFileSegment(current_file_segment_write_offset, is_persistent);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
auto & file_segment = *current_file_segment_it;
|
auto & file_segment = file_segments.back();
|
||||||
|
|
||||||
auto downloader = file_segment->getOrSetDownloader();
|
|
||||||
if (downloader != FileSegment::getCallerId())
|
|
||||||
throw Exception(ErrorCodes::LOGICAL_ERROR, "Failed to set a downloader. ({})", file_segment->getInfoForLog());
|
|
||||||
|
|
||||||
SCOPE_EXIT({
|
SCOPE_EXIT({
|
||||||
if (file_segment->isDownloader())
|
if (file_segments.back()->isDownloader())
|
||||||
file_segment->completePartAndResetDownloader();
|
file_segments.back()->completePartAndResetDownloader();
|
||||||
});
|
});
|
||||||
|
|
||||||
bool reserved = file_segment->reserve(size);
|
while (size > 0)
|
||||||
|
{
|
||||||
|
size_t available_size = file_segment->range().size() - file_segment->getDownloadedSize();
|
||||||
|
if (available_size == 0)
|
||||||
|
{
|
||||||
|
completeFileSegment(*file_segment);
|
||||||
|
file_segment = allocateFileSegment(expected_write_offset, is_persistent);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!file_segment->isDownloader()
|
||||||
|
&& file_segment->getOrSetDownloader() != FileSegment::getCallerId())
|
||||||
|
{
|
||||||
|
throw Exception(ErrorCodes::LOGICAL_ERROR,
|
||||||
|
"Failed to set a downloader. ({})", file_segment->getInfoForLog());
|
||||||
|
}
|
||||||
|
|
||||||
|
size_t size_to_write = std::min(available_size, size);
|
||||||
|
|
||||||
|
bool reserved = file_segment->reserve(size_to_write);
|
||||||
if (!reserved)
|
if (!reserved)
|
||||||
{
|
{
|
||||||
file_segment->completeWithState(FileSegment::State::PARTIALLY_DOWNLOADED_NO_CONTINUATION);
|
file_segment->completeWithState(FileSegment::State::PARTIALLY_DOWNLOADED_NO_CONTINUATION);
|
||||||
appendFilesystemCacheLog(*file_segment);
|
appendFilesystemCacheLog(*file_segment);
|
||||||
|
|
||||||
LOG_DEBUG(
|
LOG_DEBUG(
|
||||||
&Poco::Logger::get("FileSegmentRangeWriter"),
|
log, "Failed to reserve space in cache (size: {}, file segment info: {}",
|
||||||
"Unsuccessful space reservation attempt (size: {}, file segment info: {}",
|
|
||||||
size, file_segment->getInfoForLog());
|
size, file_segment->getInfoForLog());
|
||||||
|
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
try
|
file_segment->write(data, size_to_write, offset);
|
||||||
{
|
|
||||||
file_segment->write(data, size, offset);
|
|
||||||
}
|
|
||||||
catch (...)
|
|
||||||
{
|
|
||||||
file_segment->completePartAndResetDownloader();
|
file_segment->completePartAndResetDownloader();
|
||||||
throw;
|
|
||||||
}
|
|
||||||
|
|
||||||
file_segment->completePartAndResetDownloader();
|
size -= size_to_write;
|
||||||
current_file_segment_write_offset += size;
|
expected_write_offset += size_to_write;
|
||||||
|
offset += size_to_write;
|
||||||
|
data += size_to_write;
|
||||||
|
}
|
||||||
|
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
@ -129,10 +128,10 @@ void FileSegmentRangeWriter::finalize()
|
|||||||
return;
|
return;
|
||||||
|
|
||||||
auto & file_segments = file_segments_holder.file_segments;
|
auto & file_segments = file_segments_holder.file_segments;
|
||||||
if (file_segments.empty() || current_file_segment_it == file_segments.end())
|
if (file_segments.empty())
|
||||||
return;
|
return;
|
||||||
|
|
||||||
completeFileSegment(**current_file_segment_it);
|
completeFileSegment(*file_segments.back());
|
||||||
finalized = true;
|
finalized = true;
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -149,7 +148,7 @@ FileSegmentRangeWriter::~FileSegmentRangeWriter()
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
FileSegments::iterator FileSegmentRangeWriter::allocateFileSegment(size_t offset, bool is_persistent)
|
FileSegmentPtr & FileSegmentRangeWriter::allocateFileSegment(size_t offset, bool is_persistent)
|
||||||
{
|
{
|
||||||
/**
|
/**
|
||||||
* Allocate a new file segment starting `offset`.
|
* Allocate a new file segment starting `offset`.
|
||||||
@ -168,7 +167,8 @@ FileSegments::iterator FileSegmentRangeWriter::allocateFileSegment(size_t offset
|
|||||||
auto file_segment = cache->createFileSegmentForDownload(
|
auto file_segment = cache->createFileSegmentForDownload(
|
||||||
key, offset, cache->max_file_segment_size, create_settings, cache_lock);
|
key, offset, cache->max_file_segment_size, create_settings, cache_lock);
|
||||||
|
|
||||||
return file_segments_holder.add(std::move(file_segment));
|
auto & file_segments = file_segments_holder.file_segments;
|
||||||
|
return *file_segments.insert(file_segments.end(), file_segment);
|
||||||
}
|
}
|
||||||
|
|
||||||
void FileSegmentRangeWriter::appendFilesystemCacheLog(const FileSegment & file_segment)
|
void FileSegmentRangeWriter::appendFilesystemCacheLog(const FileSegment & file_segment)
|
||||||
@ -199,7 +199,7 @@ void FileSegmentRangeWriter::appendFilesystemCacheLog(const FileSegment & file_s
|
|||||||
void FileSegmentRangeWriter::completeFileSegment(FileSegment & file_segment)
|
void FileSegmentRangeWriter::completeFileSegment(FileSegment & file_segment)
|
||||||
{
|
{
|
||||||
/// File segment can be detached if space reservation failed.
|
/// File segment can be detached if space reservation failed.
|
||||||
if (file_segment.isDetached())
|
if (file_segment.isDetached() || file_segment.isCompleted())
|
||||||
return;
|
return;
|
||||||
|
|
||||||
file_segment.completeWithoutState();
|
file_segment.completeWithoutState();
|
||||||
@ -223,6 +223,7 @@ CachedOnDiskWriteBufferFromFile::CachedOnDiskWriteBufferFromFile(
|
|||||||
, is_persistent_cache_file(is_persistent_cache_file_)
|
, is_persistent_cache_file(is_persistent_cache_file_)
|
||||||
, query_id(query_id_)
|
, query_id(query_id_)
|
||||||
, enable_cache_log(!query_id_.empty() && settings_.enable_filesystem_cache_log)
|
, enable_cache_log(!query_id_.empty() && settings_.enable_filesystem_cache_log)
|
||||||
|
, throw_on_error_from_cache(settings_.throw_on_error_from_cache)
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -246,11 +247,11 @@ void CachedOnDiskWriteBufferFromFile::nextImpl()
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Write data to cache.
|
/// Write data to cache.
|
||||||
cacheData(working_buffer.begin(), size);
|
cacheData(working_buffer.begin(), size, throw_on_error_from_cache);
|
||||||
current_download_offset += size;
|
current_download_offset += size;
|
||||||
}
|
}
|
||||||
|
|
||||||
void CachedOnDiskWriteBufferFromFile::cacheData(char * data, size_t size)
|
void CachedOnDiskWriteBufferFromFile::cacheData(char * data, size_t size, bool throw_on_error)
|
||||||
{
|
{
|
||||||
if (cache_in_error_state_or_disabled)
|
if (cache_in_error_state_or_disabled)
|
||||||
return;
|
return;
|
||||||
@ -285,11 +286,17 @@ void CachedOnDiskWriteBufferFromFile::cacheData(char * data, size_t size)
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (throw_on_error)
|
||||||
|
throw;
|
||||||
|
|
||||||
tryLogCurrentException(__PRETTY_FUNCTION__);
|
tryLogCurrentException(__PRETTY_FUNCTION__);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
catch (...)
|
catch (...)
|
||||||
{
|
{
|
||||||
|
if (throw_on_error)
|
||||||
|
throw;
|
||||||
|
|
||||||
tryLogCurrentException(__PRETTY_FUNCTION__);
|
tryLogCurrentException(__PRETTY_FUNCTION__);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user