mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-22 15:42:02 +00:00
Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2
This commit is contained in:
kssenii
commit
853f2ea123
@ -16,6 +16,6 @@ ClickHouse® is an open-source column-oriented database management system that a
|
||||
* [Contacts](https://clickhouse.com/company/contact) can help to get your questions answered if there are any.
|
||||
|
||||
## Upcoming events
|
||||
* [**v22.12 Release Webinar**](https://clickhouse.com/company/events/v22-12-release-webinar) 22.12 is the ClickHouse Christmas release. There are plenty of gifts (a new JOIN algorithm among them) and we adopted something from MongoDB. Original creator, co-founder, and CTO of ClickHouse Alexey Milovidov will walk us through the highlights of the release.
|
||||
* **Recording available**: [**v22.12 Release Webinar**](https://www.youtube.com/watch?v=sREupr6uc2k) 22.12 is the ClickHouse Christmas release. There are plenty of gifts (a new JOIN algorithm among them) and we adopted something from MongoDB. Original creator, co-founder, and CTO of ClickHouse Alexey Milovidov will walk us through the highlights of the release.
|
||||
* [**ClickHouse Meetup at the CHEQ office in Tel Aviv**](https://www.meetup.com/clickhouse-tel-aviv-user-group/events/289599423/) - Jan 16 - We are very excited to be holding our next in-person ClickHouse meetup at the CHEQ office in Tel Aviv! Hear from CHEQ, ServiceNow and Contentsquare, as well as a deep dive presentation from ClickHouse CTO Alexey Milovidov. Join us for a fun evening of talks, food and discussion!
|
||||
* [**ClickHouse Meetup at Microsoft Office in Seattle**](https://www.meetup.com/clickhouse-seattle-user-group/events/290310025/) - Jan 18 - Keep an eye on this space as we will be announcing speakers soon!
|
||||
|
||||
@ -1,15 +1,15 @@
|
||||
---
|
||||
slug: /en/development/build-cross-osx
|
||||
sidebar_position: 66
|
||||
title: How to Build ClickHouse on Linux for Mac OS X
|
||||
sidebar_label: Build on Linux for Mac OS X
|
||||
title: How to Build ClickHouse on Linux for macOS
|
||||
sidebar_label: Build on Linux for macOS
|
||||
---
|
||||
|
||||
|
||||
This is for the case when you have a Linux machine and want to use it to build `clickhouse` binary that will run on OS X.
|
||||
This is intended for continuous integration checks that run on Linux servers. If you want to build ClickHouse directly on Mac OS X, then proceed with [another instruction](../development/build-osx.md).
|
||||
This is intended for continuous integration checks that run on Linux servers. If you want to build ClickHouse directly on macOS, then proceed with [another instruction](../development/build-osx.md).
|
||||
|
||||
The cross-build for Mac OS X is based on the [Build instructions](../development/build.md), follow them first.
|
||||
The cross-build for macOS is based on the [Build instructions](../development/build.md), follow them first.
|
||||
|
||||
## Install Clang-14
|
||||
|
||||
|
||||
@ -1,9 +1,9 @@
|
||||
---
|
||||
slug: /en/development/build-osx
|
||||
sidebar_position: 65
|
||||
sidebar_label: Build on Mac OS X
|
||||
title: How to Build ClickHouse on Mac OS X
|
||||
description: How to build ClickHouse on Mac OS X
|
||||
sidebar_label: Build on macOS
|
||||
title: How to Build ClickHouse on macOS
|
||||
description: How to build ClickHouse on macOS
|
||||
---
|
||||
|
||||
:::info You don't have to build ClickHouse yourself!
|
||||
|
||||
@ -7,7 +7,7 @@ description: Prerequisites and an overview of how to build ClickHouse
|
||||
|
||||
# Getting Started Guide for Building ClickHouse
|
||||
|
||||
The building of ClickHouse is supported on Linux, FreeBSD and Mac OS X.
|
||||
The building of ClickHouse is supported on Linux, FreeBSD and macOS.
|
||||
|
||||
If you use Windows, you need to create a virtual machine with Ubuntu. To start working with a virtual machine please install VirtualBox. You can download Ubuntu from the website: https://www.ubuntu.com/#download. Please create a virtual machine from the downloaded image (you should reserve at least 4GB of RAM for it). To run a command-line terminal in Ubuntu, please locate a program containing the word “terminal” in its name (gnome-terminal, konsole etc.) or just press Ctrl+Alt+T.
|
||||
|
||||
@ -194,7 +194,7 @@ In this case, ClickHouse will use config files located in the current directory.
|
||||
|
||||
To connect to ClickHouse with clickhouse-client in another terminal navigate to `ClickHouse/build/programs/` and run `./clickhouse client`.
|
||||
|
||||
If you get `Connection refused` message on Mac OS X or FreeBSD, try specifying host address 127.0.0.1:
|
||||
If you get `Connection refused` message on macOS or FreeBSD, try specifying host address 127.0.0.1:
|
||||
|
||||
clickhouse client --host 127.0.0.1
|
||||
|
||||
@ -213,7 +213,7 @@ You can also run your custom-built ClickHouse binary with the config file from t
|
||||
|
||||
## IDE (Integrated Development Environment) {#ide-integrated-development-environment}
|
||||
|
||||
If you do not know which IDE to use, we recommend that you use CLion. CLion is commercial software, but it offers 30 days free trial period. It is also free of charge for students. CLion can be used both on Linux and on Mac OS X.
|
||||
If you do not know which IDE to use, we recommend that you use CLion. CLion is commercial software, but it offers 30 days free trial period. It is also free of charge for students. CLion can be used both on Linux and on macOS.
|
||||
|
||||
KDevelop and QTCreator are other great alternatives of an IDE for developing ClickHouse. KDevelop comes in as a very handy IDE although unstable. If KDevelop crashes after a while upon opening project, you should click “Stop All” button as soon as it has opened the list of project’s files. After doing so KDevelop should be fine to work with.
|
||||
|
||||
|
||||
@ -139,7 +139,7 @@ If the system clickhouse-server is already running and you do not want to stop i
|
||||
Build tests allow to check that build is not broken on various alternative configurations and on some foreign systems. These tests are automated as well.
|
||||
|
||||
Examples:
|
||||
- cross-compile for Darwin x86_64 (Mac OS X)
|
||||
- cross-compile for Darwin x86_64 (macOS)
|
||||
- cross-compile for FreeBSD x86_64
|
||||
- cross-compile for Linux AArch64
|
||||
- build on Ubuntu with libraries from system packages (discouraged)
|
||||
|
||||
@ -9,7 +9,7 @@ slug: /en/install
|
||||
You have three options for getting up and running with ClickHouse:
|
||||
|
||||
- **[ClickHouse Cloud](https://clickhouse.com/cloud/):** The official ClickHouse as a service, - built by, maintained and supported by the creators of ClickHouse
|
||||
- **[Self-managed ClickHouse](#self-managed-install):** ClickHouse can run on any Linux, FreeBSD, or Mac OS X with x86-64, ARM, or PowerPC64LE CPU architecture
|
||||
- **[Self-managed ClickHouse](#self-managed-install):** ClickHouse can run on any Linux, FreeBSD, or macOS with x86-64, ARM, or PowerPC64LE CPU architecture
|
||||
- **[Docker Image](https://hub.docker.com/r/clickhouse/clickhouse-server/):** Read the guide with the official image in Docker Hub
|
||||
|
||||
## ClickHouse Cloud
|
||||
@ -257,7 +257,7 @@ To run ClickHouse inside Docker follow the guide on [Docker Hub](https://hub.doc
|
||||
|
||||
### From Sources {#from-sources}
|
||||
|
||||
To manually compile ClickHouse, follow the instructions for [Linux](/docs/en/development/build.md) or [Mac OS X](/docs/en/development/build-osx.md).
|
||||
To manually compile ClickHouse, follow the instructions for [Linux](/docs/en/development/build.md) or [macOS](/docs/en/development/build-osx.md).
|
||||
|
||||
You can compile packages and install them or use programs without installing packages.
|
||||
|
||||
@ -352,7 +352,7 @@ To continue experimenting, you can download one of the test data sets or go thro
|
||||
|
||||
## Recommendations for Self-Managed ClickHouse
|
||||
|
||||
ClickHouse can run on any Linux, FreeBSD, or Mac OS X with x86-64, ARM, or PowerPC64LE CPU architecture.
|
||||
ClickHouse can run on any Linux, FreeBSD, or macOS with x86-64, ARM, or PowerPC64LE CPU architecture.
|
||||
|
||||
ClickHouse uses all hardware resources available to process data.
|
||||
|
||||
|
||||
@ -890,7 +890,7 @@ The maximum number of open files.
|
||||
|
||||
By default: `maximum`.
|
||||
|
||||
We recommend using this option in Mac OS X since the `getrlimit()` function returns an incorrect value.
|
||||
We recommend using this option in macOS since the `getrlimit()` function returns an incorrect value.
|
||||
|
||||
**Example**
|
||||
|
||||
|
||||
@ -3447,13 +3447,45 @@ Default value: 2.
|
||||
|
||||
## compatibility {#compatibility}
|
||||
|
||||
This setting changes other settings according to provided ClickHouse version.
|
||||
If a behaviour in ClickHouse was changed by using a different default value for some setting, this compatibility setting allows you to use default values from previous versions for all the settings that were not set by the user.
|
||||
The `compatibility` setting causes ClickHouse to use the default settings of a previous version of ClickHouse, where the previous version is provided as the setting.
|
||||
|
||||
This setting takes ClickHouse version number as a string, like `21.3`, `21.8`. Empty value means that this setting is disabled.
|
||||
If settings are set to non-default values, then those settings are honored (only settings that have not been modified are affected by the `compatibility` setting).
|
||||
|
||||
This setting takes a ClickHouse version number as a string, like `22.3`, `22.8`. An empty value means that this setting is disabled.
|
||||
|
||||
Disabled by default.
|
||||
|
||||
:::note
|
||||
In ClickHouse Cloud the compatibility setting must be set by ClickHouse Cloud support. Please [open a case](https://clickhouse.cloud/support) to have it set.
|
||||
:::
|
||||
|
||||
## allow_settings_after_format_in_insert {#allow_settings_after_format_in_insert}
|
||||
|
||||
Control whether `SETTINGS` after `FORMAT` in `INSERT` queries is allowed or not. It is not recommended to use this, since this may interpret part of `SETTINGS` as values.
|
||||
|
||||
Example:
|
||||
|
||||
```sql
|
||||
INSERT INTO FUNCTION null('foo String') SETTINGS max_threads=1 VALUES ('bar');
|
||||
```
|
||||
|
||||
But the following query will work only with `allow_settings_after_format_in_insert`:
|
||||
|
||||
```sql
|
||||
SET allow_settings_after_format_in_insert=1;
|
||||
INSERT INTO FUNCTION null('foo String') VALUES ('bar') SETTINGS max_threads=1;
|
||||
```
|
||||
|
||||
Possible values:
|
||||
|
||||
- 0 — Disallow.
|
||||
- 1 — Allow.
|
||||
|
||||
Default value: `0`.
|
||||
|
||||
!!! note "Warning"
|
||||
Use this setting only for backward compatibility if your use cases depend on old syntax.
|
||||
|
||||
# Format settings {#format-settings}
|
||||
|
||||
## input_format_skip_unknown_fields {#input_format_skip_unknown_fields}
|
||||
|
||||
@ -1104,6 +1104,7 @@ Using replacement fields, you can define a pattern for the resulting string. “
|
||||
| %d | day of the month, zero-padded (01-31) | 02 |
|
||||
| %D | Short MM/DD/YY date, equivalent to %m/%d/%y | 01/02/18 |
|
||||
| %e | day of the month, space-padded ( 1-31) | 2 |
|
||||
| %f | fractional second from the fractional part of DateTime64 | 1234560 |
|
||||
| %F | short YYYY-MM-DD date, equivalent to %Y-%m-%d | 2018-01-02 |
|
||||
| %G | four-digit year format for ISO week number, calculated from the week-based year [defined by the ISO 8601](https://en.wikipedia.org/wiki/ISO_8601#Week_dates) standard, normally useful only with %V | 2018 |
|
||||
| %g | two-digit year format, aligned to ISO 8601, abbreviated from four-digit notation | 18 |
|
||||
@ -1143,6 +1144,20 @@ Result:
|
||||
└────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT formatDateTime(toDateTime64('2010-01-04 12:34:56.123456', 7), '%f')
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
```
|
||||
┌─formatDateTime(toDateTime64('2010-01-04 12:34:56.123456', 7), '%f')─┐
|
||||
│ 1234560 │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## dateName
|
||||
|
||||
Returns specified part of date.
|
||||
|
||||
@ -595,9 +595,9 @@ SELECT xxHash64('')
|
||||
|
||||
**Returned value**
|
||||
|
||||
A `Uint32` or `Uint64` data type hash value.
|
||||
A `UInt32` or `UInt64` data type hash value.
|
||||
|
||||
Type: `xxHash`.
|
||||
Type: `UInt32` for `xxHash32` and `UInt64` for `xxHash64`.
|
||||
|
||||
**Example**
|
||||
|
||||
|
||||
@ -68,6 +68,440 @@ Result:
|
||||
└────────────┴────────────┴──────────────┴────────────────┴─────────────────┴──────────────────────┘
|
||||
```
|
||||
|
||||
# Functions for Generating Random Numbers based on Distributions
|
||||
|
||||
:::note
|
||||
These functions are available starting from 22.10.
|
||||
:::
|
||||
|
||||
|
||||
|
||||
## randNormal
|
||||
|
||||
Return random number based on [normal distribution](https://en.wikipedia.org/wiki/Normal_distribution).
|
||||
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
randNormal(meam, variance)
|
||||
```
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `meam` - `Float64` mean value of distribution,
|
||||
- `variance` - `Float64` - [variance](https://en.wikipedia.org/wiki/Variance).
|
||||
|
||||
**Returned value**
|
||||
|
||||
- Pseudo-random number.
|
||||
|
||||
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT randNormal(10, 2) FROM numbers(5)
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
┌──randNormal(10, 2)─┐
|
||||
│ 13.389228911709653 │
|
||||
│ 8.622949707401295 │
|
||||
│ 10.801887062682981 │
|
||||
│ 4.5220192605895315 │
|
||||
│ 10.901239123982567 │
|
||||
└────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
|
||||
## randLogNormal
|
||||
|
||||
Return random number based on [log-normal distribution](https://en.wikipedia.org/wiki/Log-normal_distribution).
|
||||
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
randLogNormal(meam, variance)
|
||||
```
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `meam` - `Float64` mean value of distribution,
|
||||
- `variance` - `Float64` - [variance](https://en.wikipedia.org/wiki/Variance).
|
||||
|
||||
**Returned value**
|
||||
|
||||
- Pseudo-random number.
|
||||
|
||||
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT randLogNormal(100, 5) FROM numbers(5)
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
┌─randLogNormal(100, 5)─┐
|
||||
│ 1.295699673937363e48 │
|
||||
│ 9.719869109186684e39 │
|
||||
│ 6.110868203189557e42 │
|
||||
│ 9.912675872925529e39 │
|
||||
│ 2.3564708490552458e42 │
|
||||
└───────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
|
||||
## randBinomial
|
||||
|
||||
Return random number based on [binomial distribution](https://en.wikipedia.org/wiki/Binomial_distribution).
|
||||
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
randBinomial(experiments, probability)
|
||||
```
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `experiments` - `UInt64` number of experiments,
|
||||
- `probability` - `Float64` - probability of success in each experiment (values in `0...1` range only).
|
||||
|
||||
**Returned value**
|
||||
|
||||
- Pseudo-random number.
|
||||
|
||||
Type: [UInt64](/docs/en/sql-reference/data-types/int-uint.md).
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT randBinomial(100, .75) FROM numbers(5)
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
┌─randBinomial(100, 0.75)─┐
|
||||
│ 74 │
|
||||
│ 78 │
|
||||
│ 76 │
|
||||
│ 77 │
|
||||
│ 80 │
|
||||
└─────────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
|
||||
## randNegativeBinomial
|
||||
|
||||
Return random number based on [negative binomial distribution](https://en.wikipedia.org/wiki/Negative_binomial_distribution).
|
||||
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
randNegativeBinomial(experiments, probability)
|
||||
```
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `experiments` - `UInt64` number of experiments,
|
||||
- `probability` - `Float64` - probability of failure in each experiment (values in `0...1` range only).
|
||||
|
||||
**Returned value**
|
||||
|
||||
- Pseudo-random number.
|
||||
|
||||
Type: [UInt64](/docs/en/sql-reference/data-types/int-uint.md).
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT randNegativeBinomial(100, .75) FROM numbers(5)
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
┌─randNegativeBinomial(100, 0.75)─┐
|
||||
│ 33 │
|
||||
│ 32 │
|
||||
│ 39 │
|
||||
│ 40 │
|
||||
│ 50 │
|
||||
└─────────────────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
|
||||
## randPoisson
|
||||
|
||||
Return random number based on [Poisson distribution](https://en.wikipedia.org/wiki/Poisson_distribution).
|
||||
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
randPoisson(n)
|
||||
```
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `n` - `UInt64` mean number of occurrences.
|
||||
|
||||
**Returned value**
|
||||
|
||||
- Pseudo-random number.
|
||||
|
||||
Type: [UInt64](/docs/en/sql-reference/data-types/int-uint.md).
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT randPoisson(10) FROM numbers(5)
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
┌─randPoisson(10)─┐
|
||||
│ 8 │
|
||||
│ 8 │
|
||||
│ 7 │
|
||||
│ 10 │
|
||||
│ 6 │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
|
||||
|
||||
## randBernoulli
|
||||
|
||||
Return random number based on [Bernoulli distribution](https://en.wikipedia.org/wiki/Bernoulli_distribution).
|
||||
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
randBernoulli(probability)
|
||||
```
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `probability` - `Float64` - probability of success (values in `0...1` range only).
|
||||
|
||||
**Returned value**
|
||||
|
||||
- Pseudo-random number.
|
||||
|
||||
Type: [UInt64](/docs/en/sql-reference/data-types/int-uint.md).
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT randBernoulli(.75) FROM numbers(5)
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
┌─randBernoulli(0.75)─┐
|
||||
│ 1 │
|
||||
│ 1 │
|
||||
│ 0 │
|
||||
│ 1 │
|
||||
│ 1 │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
|
||||
## randExponential
|
||||
|
||||
Return random number based on [exponential distribution](https://en.wikipedia.org/wiki/Exponential_distribution).
|
||||
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
randExponential(lambda)
|
||||
```
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `lambda` - `Float64` lambda value.
|
||||
|
||||
**Returned value**
|
||||
|
||||
- Pseudo-random number.
|
||||
|
||||
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT randExponential(1/10) FROM numbers(5)
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
┌─randExponential(divide(1, 10))─┐
|
||||
│ 44.71628934340778 │
|
||||
│ 4.211013337903262 │
|
||||
│ 10.809402553207766 │
|
||||
│ 15.63959406553284 │
|
||||
│ 1.8148392319860158 │
|
||||
└────────────────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
|
||||
## randChiSquared
|
||||
|
||||
Return random number based on [Chi-square distribution](https://en.wikipedia.org/wiki/Chi-squared_distribution) - a distribution of a sum of the squares of k independent standard normal random variables.
|
||||
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
randChiSquared(degree_of_freedom)
|
||||
```
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `degree_of_freedom` - `Float64` degree of freedom.
|
||||
|
||||
**Returned value**
|
||||
|
||||
- Pseudo-random number.
|
||||
|
||||
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT randChiSquared(10) FROM numbers(5)
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
┌─randChiSquared(10)─┐
|
||||
│ 10.015463656521543 │
|
||||
│ 9.621799919882768 │
|
||||
│ 2.71785015634699 │
|
||||
│ 11.128188665931908 │
|
||||
│ 4.902063104425469 │
|
||||
└────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
|
||||
## randStudentT
|
||||
|
||||
Return random number based on [Student's t-distribution](https://en.wikipedia.org/wiki/Student%27s_t-distribution).
|
||||
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
randStudentT(degree_of_freedom)
|
||||
```
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `degree_of_freedom` - `Float64` degree of freedom.
|
||||
|
||||
**Returned value**
|
||||
|
||||
- Pseudo-random number.
|
||||
|
||||
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT randStudentT(10) FROM numbers(5)
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
┌─────randStudentT(10)─┐
|
||||
│ 1.2217309938538725 │
|
||||
│ 1.7941971681200541 │
|
||||
│ -0.28192176076784664 │
|
||||
│ 0.2508897721303792 │
|
||||
│ -2.7858432909761186 │
|
||||
└──────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
|
||||
## randFisherF
|
||||
|
||||
Return random number based on [F-distribution](https://en.wikipedia.org/wiki/F-distribution).
|
||||
|
||||
**Syntax**
|
||||
|
||||
``` sql
|
||||
randFisherF(d1, d2)
|
||||
```
|
||||
|
||||
**Arguments**
|
||||
|
||||
- `d1` - `Float64` d1 degree of freedom in `X = (S1 / d1) / (S2 / d2)`,
|
||||
- `d2` - `Float64` d2 degree of freedom in `X = (S1 / d1) / (S2 / d2)`,
|
||||
|
||||
**Returned value**
|
||||
|
||||
- Pseudo-random number.
|
||||
|
||||
Type: [Float64](/docs/en/sql-reference/data-types/float.md).
|
||||
|
||||
**Example**
|
||||
|
||||
Query:
|
||||
|
||||
``` sql
|
||||
SELECT randFisherF(10, 3) FROM numbers(5)
|
||||
```
|
||||
|
||||
Result:
|
||||
|
||||
``` text
|
||||
┌──randFisherF(10, 3)─┐
|
||||
│ 7.286287504216609 │
|
||||
│ 0.26590779413050386 │
|
||||
│ 0.22207610901168987 │
|
||||
│ 0.7953362728449572 │
|
||||
│ 0.19278885985221572 │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
# Random Functions for Working with Strings
|
||||
|
||||
## randomString
|
||||
|
||||
@ -21,12 +21,11 @@ Subquery is another `SELECT` query that may be specified in parenthesis inside `
|
||||
|
||||
When `FINAL` is specified, ClickHouse fully merges the data before returning the result and thus performs all data transformations that happen during merges for the given table engine.
|
||||
|
||||
It is applicable when selecting data from tables that use the [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md)-engine family. Also supported for:
|
||||
It is applicable when selecting data from ReplacingMergeTree, SummingMergeTree, AggregatingMergeTree, CollapsingMergeTree and VersionedCollapsingMergeTree tables.
|
||||
|
||||
- [Replicated](../../../engines/table-engines/mergetree-family/replication.md) versions of `MergeTree` engines.
|
||||
- [View](../../../engines/table-engines/special/view.md), [Buffer](../../../engines/table-engines/special/buffer.md), [Distributed](../../../engines/table-engines/special/distributed.md), and [MaterializedView](../../../engines/table-engines/special/materializedview.md) engines that operate over other engines, provided they were created over `MergeTree`-engine tables.
|
||||
`SELECT` queries with `FINAL` are executed in parallel. The [max_final_threads](../../../operations/settings/settings.md#max-final-threads) setting limits the number of threads used.
|
||||
|
||||
Now `SELECT` queries with `FINAL` are executed in parallel and slightly faster. But there are drawbacks (see below). The [max_final_threads](../../../operations/settings/settings.md#max-final-threads) setting limits the number of threads used.
|
||||
There are drawbacks to using `FINAL` (see below).
|
||||
|
||||
### Drawbacks
|
||||
|
||||
|
||||
@ -49,14 +49,16 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionThrow(const DataTypes & argument_types_, const Array & parameters_, Float64 throw_probability_)
|
||||
: IAggregateFunctionDataHelper(argument_types_, parameters_), throw_probability(throw_probability_) {}
|
||||
: IAggregateFunctionDataHelper(argument_types_, parameters_, createResultType())
|
||||
, throw_probability(throw_probability_)
|
||||
{}
|
||||
|
||||
String getName() const override
|
||||
{
|
||||
return "aggThrow";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
return std::make_shared<DataTypeUInt8>();
|
||||
}
|
||||
|
||||
@ -37,10 +37,10 @@ class AggregateFunctionAnalysisOfVariance final : public IAggregateFunctionDataH
|
||||
{
|
||||
public:
|
||||
explicit AggregateFunctionAnalysisOfVariance(const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionDataHelper(arguments, params)
|
||||
: IAggregateFunctionDataHelper(arguments, params, createResultType())
|
||||
{}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
DataTypePtr createResultType() const
|
||||
{
|
||||
DataTypes types {std::make_shared<DataTypeNumber<Float64>>(), std::make_shared<DataTypeNumber<Float64>>() };
|
||||
Strings names {"f_statistic", "p_value"};
|
||||
|
||||
@ -38,7 +38,6 @@ template <typename Data>
|
||||
class AggregateFunctionArgMinMax final : public IAggregateFunctionDataHelper<Data, AggregateFunctionArgMinMax<Data>>
|
||||
{
|
||||
private:
|
||||
const DataTypePtr & type_res;
|
||||
const DataTypePtr & type_val;
|
||||
const SerializationPtr serialization_res;
|
||||
const SerializationPtr serialization_val;
|
||||
@ -47,10 +46,9 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionArgMinMax(const DataTypePtr & type_res_, const DataTypePtr & type_val_)
|
||||
: Base({type_res_, type_val_}, {})
|
||||
, type_res(this->argument_types[0])
|
||||
: Base({type_res_, type_val_}, {}, type_res_)
|
||||
, type_val(this->argument_types[1])
|
||||
, serialization_res(type_res->getDefaultSerialization())
|
||||
, serialization_res(type_res_->getDefaultSerialization())
|
||||
, serialization_val(type_val->getDefaultSerialization())
|
||||
{
|
||||
if (!type_val->isComparable())
|
||||
@ -63,11 +61,6 @@ public:
|
||||
return StringRef(Data::ValueData_t::name()) == StringRef("min") ? "argMin" : "argMax";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return type_res;
|
||||
}
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||
{
|
||||
if (this->data(place).value.changeIfBetter(*columns[1], row_num, arena))
|
||||
|
||||
@ -30,7 +30,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionArray(AggregateFunctionPtr nested_, const DataTypes & arguments, const Array & params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionArray>(arguments, params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionArray>(arguments, params_, createResultType(nested_))
|
||||
, nested_func(nested_), num_arguments(arguments.size())
|
||||
{
|
||||
assert(parameters == nested_func->getParameters());
|
||||
@ -44,9 +44,9 @@ public:
|
||||
return nested_func->getName() + "Array";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const AggregateFunctionPtr & nested_)
|
||||
{
|
||||
return nested_func->getReturnType();
|
||||
return nested_->getResultType();
|
||||
}
|
||||
|
||||
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
||||
|
||||
@ -10,6 +10,7 @@
|
||||
#include <AggregateFunctions/IAggregateFunction.h>
|
||||
#include <AggregateFunctions/AggregateFunctionSum.h>
|
||||
#include <Core/DecimalFunctions.h>
|
||||
#include <Core/IResolvedFunction.h>
|
||||
|
||||
#include "config.h"
|
||||
|
||||
@ -83,10 +84,20 @@ public:
|
||||
using Fraction = AvgFraction<Numerator, Denominator>;
|
||||
|
||||
explicit AggregateFunctionAvgBase(const DataTypes & argument_types_,
|
||||
UInt32 num_scale_ = 0, UInt32 denom_scale_ = 0)
|
||||
: Base(argument_types_, {}), num_scale(num_scale_), denom_scale(denom_scale_) {}
|
||||
UInt32 num_scale_ = 0, UInt32 denom_scale_ = 0)
|
||||
: Base(argument_types_, {}, createResultType())
|
||||
, num_scale(num_scale_)
|
||||
, denom_scale(denom_scale_)
|
||||
{}
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeNumber<Float64>>(); }
|
||||
AggregateFunctionAvgBase(const DataTypes & argument_types_, const DataTypePtr & result_type_,
|
||||
UInt32 num_scale_ = 0, UInt32 denom_scale_ = 0)
|
||||
: Base(argument_types_, {}, result_type_)
|
||||
, num_scale(num_scale_)
|
||||
, denom_scale(denom_scale_)
|
||||
{}
|
||||
|
||||
DataTypePtr createResultType() const { return std::make_shared<DataTypeNumber<Float64>>(); }
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
@ -135,7 +146,7 @@ public:
|
||||
for (const auto & argument : this->argument_types)
|
||||
can_be_compiled &= canBeNativeType(*argument);
|
||||
|
||||
auto return_type = getReturnType();
|
||||
auto return_type = this->getResultType();
|
||||
can_be_compiled &= canBeNativeType(*return_type);
|
||||
|
||||
return can_be_compiled;
|
||||
|
||||
@ -97,11 +97,12 @@ class AggregateFunctionBitwise final : public IAggregateFunctionDataHelper<Data,
|
||||
{
|
||||
public:
|
||||
explicit AggregateFunctionBitwise(const DataTypePtr & type)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitwise<T, Data>>({type}, {}) {}
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitwise<T, Data>>({type}, {}, createResultType())
|
||||
{}
|
||||
|
||||
String getName() const override { return Data::name(); }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
return std::make_shared<DataTypeNumber<T>>();
|
||||
}
|
||||
@ -137,7 +138,7 @@ public:
|
||||
|
||||
bool isCompilable() const override
|
||||
{
|
||||
auto return_type = getReturnType();
|
||||
auto return_type = this->getResultType();
|
||||
return canBeNativeType(*return_type);
|
||||
}
|
||||
|
||||
@ -151,7 +152,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
|
||||
auto * value_ptr = aggregate_data_ptr;
|
||||
auto * value = b.CreateLoad(return_type, value_ptr);
|
||||
@ -166,7 +167,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
|
||||
auto * value_dst_ptr = aggregate_data_dst_ptr;
|
||||
auto * value_dst = b.CreateLoad(return_type, value_dst_ptr);
|
||||
@ -183,7 +184,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
auto * value_ptr = aggregate_data_ptr;
|
||||
|
||||
return b.CreateLoad(return_type, value_ptr);
|
||||
|
||||
@ -112,7 +112,7 @@ public:
|
||||
}
|
||||
|
||||
explicit AggregateFunctionBoundingRatio(const DataTypes & arguments)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionBoundingRatioData, AggregateFunctionBoundingRatio>(arguments, {})
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionBoundingRatioData, AggregateFunctionBoundingRatio>(arguments, {}, std::make_shared<DataTypeFloat64>())
|
||||
{
|
||||
const auto * x_arg = arguments.at(0).get();
|
||||
const auto * y_arg = arguments.at(1).get();
|
||||
@ -122,11 +122,6 @@ public:
|
||||
ErrorCodes::BAD_ARGUMENTS);
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeFloat64>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, const size_t row_num, Arena *) const override
|
||||
|
||||
@ -46,9 +46,9 @@ private:
|
||||
}
|
||||
|
||||
public:
|
||||
AggregateFunctionCategoricalIV(const DataTypes & arguments_, const Array & params_) :
|
||||
IAggregateFunctionHelper<AggregateFunctionCategoricalIV>{arguments_, params_},
|
||||
category_count{arguments_.size() - 1}
|
||||
AggregateFunctionCategoricalIV(const DataTypes & arguments_, const Array & params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionCategoricalIV>{arguments_, params_, createResultType()}
|
||||
, category_count{arguments_.size() - 1}
|
||||
{
|
||||
// notice: argument types has been checked before
|
||||
}
|
||||
@ -121,7 +121,7 @@ public:
|
||||
buf.readStrict(place, sizeOfData());
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(
|
||||
std::make_shared<DataTypeNumber<Float64>>());
|
||||
|
||||
@ -39,11 +39,13 @@ namespace ErrorCodes
|
||||
class AggregateFunctionCount final : public IAggregateFunctionDataHelper<AggregateFunctionCountData, AggregateFunctionCount>
|
||||
{
|
||||
public:
|
||||
explicit AggregateFunctionCount(const DataTypes & argument_types_) : IAggregateFunctionDataHelper(argument_types_, {}) {}
|
||||
explicit AggregateFunctionCount(const DataTypes & argument_types_)
|
||||
: IAggregateFunctionDataHelper(argument_types_, {}, createResultType())
|
||||
{}
|
||||
|
||||
String getName() const override { return "count"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
return std::make_shared<DataTypeUInt64>();
|
||||
}
|
||||
@ -167,7 +169,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
|
||||
auto * count_value_ptr = aggregate_data_ptr;
|
||||
auto * count_value = b.CreateLoad(return_type, count_value_ptr);
|
||||
@ -180,7 +182,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
|
||||
auto * count_value_dst_ptr = aggregate_data_dst_ptr;
|
||||
auto * count_value_dst = b.CreateLoad(return_type, count_value_dst_ptr);
|
||||
@ -197,7 +199,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
auto * count_value_ptr = aggregate_data_ptr;
|
||||
|
||||
return b.CreateLoad(return_type, count_value_ptr);
|
||||
@ -214,7 +216,7 @@ class AggregateFunctionCountNotNullUnary final
|
||||
{
|
||||
public:
|
||||
AggregateFunctionCountNotNullUnary(const DataTypePtr & argument, const Array & params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionCountData, AggregateFunctionCountNotNullUnary>({argument}, params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionCountData, AggregateFunctionCountNotNullUnary>({argument}, params, createResultType())
|
||||
{
|
||||
if (!argument->isNullable())
|
||||
throw Exception("Logical error: not Nullable data type passed to AggregateFunctionCountNotNullUnary", ErrorCodes::LOGICAL_ERROR);
|
||||
@ -222,7 +224,7 @@ public:
|
||||
|
||||
String getName() const override { return "count"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
return std::make_shared<DataTypeUInt64>();
|
||||
}
|
||||
@ -311,7 +313,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
|
||||
auto * is_null_value = b.CreateExtractValue(values[0], {1});
|
||||
auto * increment_value = b.CreateSelect(is_null_value, llvm::ConstantInt::get(return_type, 0), llvm::ConstantInt::get(return_type, 1));
|
||||
@ -327,7 +329,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
|
||||
auto * count_value_dst_ptr = aggregate_data_dst_ptr;
|
||||
auto * count_value_dst = b.CreateLoad(return_type, count_value_dst_ptr);
|
||||
@ -344,7 +346,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
auto * count_value_ptr = aggregate_data_ptr;
|
||||
|
||||
return b.CreateLoad(return_type, count_value_ptr);
|
||||
|
||||
@ -31,7 +31,7 @@ class AggregationFunctionDeltaSum final
|
||||
{
|
||||
public:
|
||||
AggregationFunctionDeltaSum(const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionDataHelper<AggregationFunctionDeltaSumData<T>, AggregationFunctionDeltaSum<T>>{arguments, params}
|
||||
: IAggregateFunctionDataHelper<AggregationFunctionDeltaSumData<T>, AggregationFunctionDeltaSum<T>>{arguments, params, createResultType()}
|
||||
{}
|
||||
|
||||
AggregationFunctionDeltaSum()
|
||||
@ -40,7 +40,7 @@ public:
|
||||
|
||||
String getName() const override { return "deltaSum"; }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeNumber<T>>(); }
|
||||
static DataTypePtr createResultType() { return std::make_shared<DataTypeNumber<T>>(); }
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
|
||||
@ -38,7 +38,7 @@ public:
|
||||
: IAggregateFunctionDataHelper<
|
||||
AggregationFunctionDeltaSumTimestampData<ValueType, TimestampType>,
|
||||
AggregationFunctionDeltaSumTimestamp<ValueType, TimestampType>
|
||||
>{arguments, params}
|
||||
>{arguments, params, createResultType()}
|
||||
{}
|
||||
|
||||
AggregationFunctionDeltaSumTimestamp()
|
||||
@ -52,7 +52,7 @@ public:
|
||||
|
||||
String getName() const override { return "deltaSumTimestamp"; }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeNumber<ValueType>>(); }
|
||||
static DataTypePtr createResultType() { return std::make_shared<DataTypeNumber<ValueType>>(); }
|
||||
|
||||
void NO_SANITIZE_UNDEFINED ALWAYS_INLINE add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||
{
|
||||
|
||||
@ -168,7 +168,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionDistinct(AggregateFunctionPtr nested_func_, const DataTypes & arguments, const Array & params_)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionDistinct>(arguments, params_)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionDistinct>(arguments, params_, nested_func_->getResultType())
|
||||
, nested_func(nested_func_)
|
||||
, arguments_num(arguments.size())
|
||||
{
|
||||
@ -255,11 +255,6 @@ public:
|
||||
return nested_func->getName() + "Distinct";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return nested_func->getReturnType();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override
|
||||
{
|
||||
return true;
|
||||
|
||||
@ -92,14 +92,14 @@ private:
|
||||
|
||||
public:
|
||||
explicit AggregateFunctionEntropy(const DataTypes & argument_types_)
|
||||
: IAggregateFunctionDataHelper<EntropyData<Value>, AggregateFunctionEntropy<Value>>(argument_types_, {})
|
||||
: IAggregateFunctionDataHelper<EntropyData<Value>, AggregateFunctionEntropy<Value>>(argument_types_, {}, createResultType())
|
||||
, num_args(argument_types_.size())
|
||||
{
|
||||
}
|
||||
|
||||
String getName() const override { return "entropy"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
return std::make_shared<DataTypeNumber<Float64>>();
|
||||
}
|
||||
|
||||
@ -29,7 +29,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionExponentialMovingAverage(const DataTypes & argument_types_, const Array & params)
|
||||
: IAggregateFunctionDataHelper<ExponentiallySmoothedAverage, AggregateFunctionExponentialMovingAverage>(argument_types_, params)
|
||||
: IAggregateFunctionDataHelper<ExponentiallySmoothedAverage, AggregateFunctionExponentialMovingAverage>(argument_types_, params, createResultType())
|
||||
{
|
||||
if (params.size() != 1)
|
||||
throw Exception{"Aggregate function " + getName() + " requires exactly one parameter: half decay time.",
|
||||
@ -43,7 +43,7 @@ public:
|
||||
return "exponentialMovingAverage";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
return std::make_shared<DataTypeNumber<Float64>>();
|
||||
}
|
||||
|
||||
@ -523,12 +523,12 @@ class AggregateFunctionFlameGraph final : public IAggregateFunctionDataHelper<Ag
|
||||
{
|
||||
public:
|
||||
explicit AggregateFunctionFlameGraph(const DataTypes & argument_types_)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionFlameGraphData, AggregateFunctionFlameGraph>(argument_types_, {})
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionFlameGraphData, AggregateFunctionFlameGraph>(argument_types_, {}, createResultType())
|
||||
{}
|
||||
|
||||
String getName() const override { return "flameGraph"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(std::make_shared<DataTypeString>());
|
||||
}
|
||||
|
||||
@ -107,7 +107,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionForEach(AggregateFunctionPtr nested_, const DataTypes & arguments, const Array & params_)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionForEachData, AggregateFunctionForEach>(arguments, params_)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionForEachData, AggregateFunctionForEach>(arguments, params_, createResultType(nested_))
|
||||
, nested_func(nested_), num_arguments(arguments.size())
|
||||
{
|
||||
nested_size_of_data = nested_func->sizeOfData();
|
||||
@ -125,9 +125,9 @@ public:
|
||||
return nested_func->getName() + "ForEach";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(AggregateFunctionPtr nested_)
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(nested_func->getReturnType());
|
||||
return std::make_shared<DataTypeArray>(nested_->getResultType());
|
||||
}
|
||||
|
||||
bool isVersioned() const override
|
||||
|
||||
@ -121,7 +121,7 @@ public:
|
||||
explicit GroupArrayNumericImpl(
|
||||
const DataTypePtr & data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max(), UInt64 seed_ = 123456)
|
||||
: IAggregateFunctionDataHelper<GroupArrayNumericData<T, Trait::sampler != Sampler::NONE>, GroupArrayNumericImpl<T, Trait>>(
|
||||
{data_type_}, parameters_)
|
||||
{data_type_}, parameters_, std::make_shared<DataTypeArray>(data_type_))
|
||||
, max_elems(max_elems_)
|
||||
, seed(seed_)
|
||||
{
|
||||
@ -129,8 +129,6 @@ public:
|
||||
|
||||
String getName() const override { return getNameByTrait<Trait>(); }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(this->argument_types[0]); }
|
||||
|
||||
void insert(Data & a, const T & v, Arena * arena) const
|
||||
{
|
||||
++a.total_values;
|
||||
@ -423,7 +421,7 @@ class GroupArrayGeneralImpl final
|
||||
public:
|
||||
GroupArrayGeneralImpl(const DataTypePtr & data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max(), UInt64 seed_ = 123456)
|
||||
: IAggregateFunctionDataHelper<GroupArrayGeneralData<Node, Trait::sampler != Sampler::NONE>, GroupArrayGeneralImpl<Node, Trait>>(
|
||||
{data_type_}, parameters_)
|
||||
{data_type_}, parameters_, std::make_shared<DataTypeArray>(data_type_))
|
||||
, data_type(this->argument_types[0])
|
||||
, max_elems(max_elems_)
|
||||
, seed(seed_)
|
||||
@ -432,8 +430,6 @@ public:
|
||||
|
||||
String getName() const override { return getNameByTrait<Trait>(); }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(data_type); }
|
||||
|
||||
void insert(Data & a, const Node * v, Arena * arena) const
|
||||
{
|
||||
++a.total_values;
|
||||
@ -697,7 +693,7 @@ class GroupArrayGeneralListImpl final
|
||||
|
||||
public:
|
||||
GroupArrayGeneralListImpl(const DataTypePtr & data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||
: IAggregateFunctionDataHelper<GroupArrayGeneralListData<Node>, GroupArrayGeneralListImpl<Node, Trait>>({data_type_}, parameters_)
|
||||
: IAggregateFunctionDataHelper<GroupArrayGeneralListData<Node>, GroupArrayGeneralListImpl<Node, Trait>>({data_type_}, parameters_, std::make_shared<DataTypeArray>(data_type_))
|
||||
, data_type(this->argument_types[0])
|
||||
, max_elems(max_elems_)
|
||||
{
|
||||
@ -705,8 +701,6 @@ public:
|
||||
|
||||
String getName() const override { return getNameByTrait<Trait>(); }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(data_type); }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||
{
|
||||
if (limit_num_elems && data(place).elems >= max_elems)
|
||||
|
||||
@ -64,7 +64,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionGroupArrayInsertAtGeneric(const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionGroupArrayInsertAtDataGeneric, AggregateFunctionGroupArrayInsertAtGeneric>(arguments, params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionGroupArrayInsertAtDataGeneric, AggregateFunctionGroupArrayInsertAtGeneric>(arguments, params, std::make_shared<DataTypeArray>(arguments[0]))
|
||||
, type(argument_types[0])
|
||||
, serialization(type->getDefaultSerialization())
|
||||
{
|
||||
@ -101,11 +101,6 @@ public:
|
||||
|
||||
String getName() const override { return "groupArrayInsertAt"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(type);
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||
|
||||
@ -93,12 +93,15 @@ public:
|
||||
using ColumnResult = ColumnVectorOrDecimal<ResultT>;
|
||||
|
||||
explicit MovingImpl(const DataTypePtr & data_type_, UInt64 window_size_ = std::numeric_limits<UInt64>::max())
|
||||
: IAggregateFunctionDataHelper<Data, MovingImpl<T, LimitNumElements, Data>>({data_type_}, {})
|
||||
: IAggregateFunctionDataHelper<Data, MovingImpl<T, LimitNumElements, Data>>({data_type_}, {}, createResultType(data_type_))
|
||||
, window_size(window_size_) {}
|
||||
|
||||
String getName() const override { return Data::name; }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(getReturnTypeElement()); }
|
||||
static DataTypePtr createResultType(const DataTypePtr & argument)
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(getReturnTypeElement(argument));
|
||||
}
|
||||
|
||||
void NO_SANITIZE_UNDEFINED add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||
{
|
||||
@ -183,14 +186,14 @@ public:
|
||||
}
|
||||
|
||||
private:
|
||||
auto getReturnTypeElement() const
|
||||
static auto getReturnTypeElement(const DataTypePtr & argument)
|
||||
{
|
||||
if constexpr (!is_decimal<ResultT>)
|
||||
return std::make_shared<DataTypeNumber<ResultT>>();
|
||||
else
|
||||
{
|
||||
using Res = DataTypeDecimal<ResultT>;
|
||||
return std::make_shared<Res>(Res::maxPrecision(), getDecimalScale(*this->argument_types.at(0)));
|
||||
return std::make_shared<Res>(Res::maxPrecision(), getDecimalScale(*argument));
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
@ -19,13 +19,13 @@ class AggregateFunctionBitmap final : public IAggregateFunctionDataHelper<Data,
|
||||
{
|
||||
public:
|
||||
explicit AggregateFunctionBitmap(const DataTypePtr & type)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitmap<T, Data>>({type}, {})
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitmap<T, Data>>({type}, {}, createResultType())
|
||||
{
|
||||
}
|
||||
|
||||
String getName() const override { return Data::name(); }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeNumber<T>>(); }
|
||||
static DataTypePtr createResultType() { return std::make_shared<DataTypeNumber<T>>(); }
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
@ -59,13 +59,13 @@ private:
|
||||
static constexpr size_t STATE_VERSION_1_MIN_REVISION = 54455;
|
||||
public:
|
||||
explicit AggregateFunctionBitmapL2(const DataTypePtr & type)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitmapL2<T, Data, Policy>>({type}, {})
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionBitmapL2<T, Data, Policy>>({type}, {}, createResultType())
|
||||
{
|
||||
}
|
||||
|
||||
String getName() const override { return Policy::name; }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeNumber<T>>(); }
|
||||
static DataTypePtr createResultType() { return std::make_shared<DataTypeNumber<T>>(); }
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
|
||||
@ -26,8 +26,8 @@ class AggregateFunctionGroupUniqArrayDate : public AggregateFunctionGroupUniqArr
|
||||
{
|
||||
public:
|
||||
explicit AggregateFunctionGroupUniqArrayDate(const DataTypePtr & argument_type, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||
: AggregateFunctionGroupUniqArray<DataTypeDate::FieldType, HasLimit>(argument_type, parameters_, max_elems_) {}
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDate>()); }
|
||||
: AggregateFunctionGroupUniqArray<DataTypeDate::FieldType, HasLimit>(argument_type, parameters_, createResultType(), max_elems_) {}
|
||||
static DataTypePtr createResultType() { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDate>()); }
|
||||
};
|
||||
|
||||
template <typename HasLimit>
|
||||
@ -35,8 +35,8 @@ class AggregateFunctionGroupUniqArrayDateTime : public AggregateFunctionGroupUni
|
||||
{
|
||||
public:
|
||||
explicit AggregateFunctionGroupUniqArrayDateTime(const DataTypePtr & argument_type, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||
: AggregateFunctionGroupUniqArray<DataTypeDateTime::FieldType, HasLimit>(argument_type, parameters_, max_elems_) {}
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDateTime>()); }
|
||||
: AggregateFunctionGroupUniqArray<DataTypeDateTime::FieldType, HasLimit>(argument_type, parameters_, createResultType(), max_elems_) {}
|
||||
static DataTypePtr createResultType() { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDateTime>()); }
|
||||
};
|
||||
|
||||
template <typename HasLimit, typename ... TArgs>
|
||||
|
||||
@ -50,15 +50,16 @@ private:
|
||||
public:
|
||||
AggregateFunctionGroupUniqArray(const DataTypePtr & argument_type, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionGroupUniqArrayData<T>,
|
||||
AggregateFunctionGroupUniqArray<T, LimitNumElems>>({argument_type}, parameters_),
|
||||
AggregateFunctionGroupUniqArray<T, LimitNumElems>>({argument_type}, parameters_, std::make_shared<DataTypeArray>(argument_type)),
|
||||
max_elems(max_elems_) {}
|
||||
|
||||
String getName() const override { return "groupUniqArray"; }
|
||||
AggregateFunctionGroupUniqArray(const DataTypePtr & argument_type, const Array & parameters_, const DataTypePtr & result_type_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionGroupUniqArrayData<T>,
|
||||
AggregateFunctionGroupUniqArray<T, LimitNumElems>>({argument_type}, parameters_, result_type_),
|
||||
max_elems(max_elems_) {}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(this->argument_types[0]);
|
||||
}
|
||||
|
||||
String getName() const override { return "groupUniqArray"; }
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
@ -153,17 +154,12 @@ class AggregateFunctionGroupUniqArrayGeneric
|
||||
|
||||
public:
|
||||
AggregateFunctionGroupUniqArrayGeneric(const DataTypePtr & input_data_type_, const Array & parameters_, UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionGroupUniqArrayGenericData, AggregateFunctionGroupUniqArrayGeneric<is_plain_column, LimitNumElems>>({input_data_type_}, parameters_)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionGroupUniqArrayGenericData, AggregateFunctionGroupUniqArrayGeneric<is_plain_column, LimitNumElems>>({input_data_type_}, parameters_, std::make_shared<DataTypeArray>(input_data_type_))
|
||||
, input_data_type(this->argument_types[0])
|
||||
, max_elems(max_elems_) {}
|
||||
|
||||
String getName() const override { return "groupUniqArray"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(input_data_type);
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override
|
||||
{
|
||||
return true;
|
||||
|
||||
@ -307,7 +307,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionHistogram(UInt32 max_bins_, const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionHistogramData, AggregateFunctionHistogram<T>>(arguments, params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionHistogramData, AggregateFunctionHistogram<T>>(arguments, params, createResultType())
|
||||
, max_bins(max_bins_)
|
||||
{
|
||||
}
|
||||
@ -316,7 +316,7 @@ public:
|
||||
{
|
||||
return Data::structSize(max_bins);
|
||||
}
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
DataTypes types;
|
||||
auto mean = std::make_shared<DataTypeNumber<Data::Mean>>();
|
||||
|
||||
@ -448,7 +448,7 @@ AggregateFunctionPtr AggregateFunctionIf::getOwnNullAdapter(
|
||||
|
||||
/// Nullability of the last argument (condition) does not affect the nullability of the result (NULL is processed as false).
|
||||
/// For other arguments it is as usual (at least one is NULL then the result is NULL if possible).
|
||||
bool return_type_is_nullable = !properties.returns_default_when_only_null && getReturnType()->canBeInsideNullable()
|
||||
bool return_type_is_nullable = !properties.returns_default_when_only_null && getResultType()->canBeInsideNullable()
|
||||
&& std::any_of(arguments.begin(), arguments.end() - 1, [](const auto & element) { return element->isNullable(); });
|
||||
|
||||
bool need_to_serialize_flag = return_type_is_nullable || properties.returns_default_when_only_null;
|
||||
|
||||
@ -36,7 +36,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionIf(AggregateFunctionPtr nested, const DataTypes & types, const Array & params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionIf>(types, params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionIf>(types, params_, nested->getResultType())
|
||||
, nested_func(nested), num_arguments(types.size())
|
||||
{
|
||||
if (num_arguments == 0)
|
||||
@ -51,11 +51,6 @@ public:
|
||||
return nested_func->getName() + "If";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return nested_func->getReturnType();
|
||||
}
|
||||
|
||||
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
||||
{
|
||||
return nested_func->getBaseAggregateFunctionWithSameStateRepresentation();
|
||||
|
||||
@ -177,11 +177,11 @@ public:
|
||||
String getName() const override { return "intervalLengthSum"; }
|
||||
|
||||
explicit AggregateFunctionIntervalLengthSum(const DataTypes & arguments)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionIntervalLengthSum<T, Data>>(arguments, {})
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionIntervalLengthSum<T, Data>>(arguments, {}, createResultType())
|
||||
{
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
if constexpr (std::is_floating_point_v<T>)
|
||||
return std::make_shared<DataTypeFloat64>();
|
||||
|
||||
@ -309,7 +309,7 @@ public:
|
||||
UInt64 batch_size_,
|
||||
const DataTypes & arguments_types,
|
||||
const Array & params)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionMLMethod<Data, Name>>(arguments_types, params)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionMLMethod<Data, Name>>(arguments_types, params, createResultType())
|
||||
, param_num(param_num_)
|
||||
, learning_rate(learning_rate_)
|
||||
, l2_reg_coef(l2_reg_coef_)
|
||||
@ -319,8 +319,7 @@ public:
|
||||
{
|
||||
}
|
||||
|
||||
/// This function is called when SELECT linearRegression(...) is called
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(std::make_shared<DataTypeFloat64>());
|
||||
}
|
||||
|
||||
@ -133,7 +133,7 @@ private:
|
||||
|
||||
public:
|
||||
explicit AggregateFunctionMannWhitney(const DataTypes & arguments, const Array & params)
|
||||
:IAggregateFunctionDataHelper<MannWhitneyData, AggregateFunctionMannWhitney> ({arguments}, {})
|
||||
: IAggregateFunctionDataHelper<MannWhitneyData, AggregateFunctionMannWhitney> ({arguments}, {}, createResultType())
|
||||
{
|
||||
if (params.size() > 2)
|
||||
throw Exception("Aggregate function " + getName() + " require two parameter or less", ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH);
|
||||
@ -174,7 +174,7 @@ public:
|
||||
|
||||
bool allocatesMemoryInArena() const override { return true; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
DataTypes types
|
||||
{
|
||||
|
||||
@ -18,6 +18,7 @@
|
||||
#include <Functions/FunctionHelpers.h>
|
||||
#include <IO/ReadHelpers.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include "DataTypes/Serializations/ISerialization.h"
|
||||
#include "base/types.h"
|
||||
#include <Common/Arena.h>
|
||||
#include "AggregateFunctions/AggregateFunctionFactory.h"
|
||||
@ -104,26 +105,32 @@ public:
|
||||
return nested_func->getDefaultVersion();
|
||||
}
|
||||
|
||||
AggregateFunctionMap(AggregateFunctionPtr nested, const DataTypes & types) : Base(types, nested->getParameters()), nested_func(nested)
|
||||
AggregateFunctionMap(AggregateFunctionPtr nested, const DataTypes & types)
|
||||
: Base(types, nested->getParameters(), std::make_shared<DataTypeMap>(DataTypes{getKeyType(types, nested), nested->getResultType()}))
|
||||
, nested_func(nested)
|
||||
{
|
||||
if (types.empty())
|
||||
throw Exception(
|
||||
ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH, "Aggregate function " + getName() + " requires at least one argument");
|
||||
|
||||
if (types.size() > 1)
|
||||
throw Exception(
|
||||
ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH, "Aggregate function " + getName() + " requires only one map argument");
|
||||
|
||||
const auto * map_type = checkAndGetDataType<DataTypeMap>(types[0].get());
|
||||
if (!map_type)
|
||||
throw Exception(ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT, "Aggregate function " + getName() + " requires map as argument");
|
||||
|
||||
key_type = map_type->getKeyType();
|
||||
key_type = getKeyType(types, nested_func);
|
||||
}
|
||||
|
||||
String getName() const override { return nested_func->getName() + "Map"; }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeMap>(DataTypes{key_type, nested_func->getReturnType()}); }
|
||||
static DataTypePtr getKeyType(const DataTypes & types, const AggregateFunctionPtr & nested)
|
||||
{
|
||||
if (types.empty())
|
||||
throw Exception(ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
|
||||
"Aggregate function {}Map requires at least one argument", nested->getName());
|
||||
|
||||
if (types.size() > 1)
|
||||
throw Exception(ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
|
||||
"Aggregate function {}Map requires only one map argument", nested->getName());
|
||||
|
||||
const auto * map_type = checkAndGetDataType<DataTypeMap>(types[0].get());
|
||||
if (!map_type)
|
||||
throw Exception(ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
||||
"Aggregate function {}Map requires map as argument", nested->getName());
|
||||
|
||||
return map_type->getKeyType();
|
||||
}
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||
{
|
||||
|
||||
@ -62,7 +62,8 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionIntersectionsMax(AggregateFunctionIntersectionsKind kind_, const DataTypes & arguments)
|
||||
: IAggregateFunctionDataHelper<MaxIntersectionsData<PointType>, AggregateFunctionIntersectionsMax<PointType>>(arguments, {}), kind(kind_)
|
||||
: IAggregateFunctionDataHelper<MaxIntersectionsData<PointType>, AggregateFunctionIntersectionsMax<PointType>>(arguments, {}, createResultType(kind_))
|
||||
, kind(kind_)
|
||||
{
|
||||
if (!isNativeNumber(arguments[0]))
|
||||
throw Exception{getName() + ": first argument must be represented by integer", ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT};
|
||||
@ -81,9 +82,9 @@ public:
|
||||
: "maxIntersectionsPosition";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(AggregateFunctionIntersectionsKind kind_)
|
||||
{
|
||||
if (kind == AggregateFunctionIntersectionsKind::Count)
|
||||
if (kind_ == AggregateFunctionIntersectionsKind::Count)
|
||||
return std::make_shared<DataTypeUInt64>();
|
||||
else
|
||||
return std::make_shared<DataTypeNumber<PointType>>();
|
||||
|
||||
@ -36,7 +36,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionMeanZTest(const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionMeanZTest<Data>>({arguments}, params)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionMeanZTest<Data>>({arguments}, params, createResultType())
|
||||
{
|
||||
pop_var_x = params.at(0).safeGet<Float64>();
|
||||
pop_var_y = params.at(1).safeGet<Float64>();
|
||||
@ -63,7 +63,7 @@ public:
|
||||
return Data::name;
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
DataTypes types
|
||||
{
|
||||
|
||||
@ -30,7 +30,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionMerge(const AggregateFunctionPtr & nested_, const DataTypePtr & argument, const Array & params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionMerge>({argument}, params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionMerge>({argument}, params_, createResultType(nested_))
|
||||
, nested_func(nested_)
|
||||
{
|
||||
const DataTypeAggregateFunction * data_type = typeid_cast<const DataTypeAggregateFunction *>(argument.get());
|
||||
@ -45,9 +45,9 @@ public:
|
||||
return nested_func->getName() + "Merge";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const AggregateFunctionPtr & nested_)
|
||||
{
|
||||
return nested_func->getReturnType();
|
||||
return nested_->getResultType();
|
||||
}
|
||||
|
||||
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
||||
|
||||
@ -1222,7 +1222,7 @@ private:
|
||||
|
||||
public:
|
||||
explicit AggregateFunctionsSingleValue(const DataTypePtr & type)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionsSingleValue<Data>>({type}, {})
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionsSingleValue<Data>>({type}, {}, createResultType(type))
|
||||
, serialization(type->getDefaultSerialization())
|
||||
{
|
||||
if (StringRef(Data::name()) == StringRef("min")
|
||||
@ -1236,12 +1236,11 @@ public:
|
||||
|
||||
String getName() const override { return Data::name(); }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const DataTypePtr & type_)
|
||||
{
|
||||
auto result_type = this->argument_types.at(0);
|
||||
if constexpr (Data::is_nullable)
|
||||
return makeNullable(result_type);
|
||||
return result_type;
|
||||
return makeNullable(type_);
|
||||
return type_;
|
||||
}
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||
|
||||
@ -6,6 +6,7 @@
|
||||
#include <AggregateFunctions/IAggregateFunction.h>
|
||||
#include <IO/ReadHelpers.h>
|
||||
#include <IO/WriteHelpers.h>
|
||||
#include "DataTypes/IDataType.h"
|
||||
|
||||
|
||||
namespace DB
|
||||
@ -19,16 +20,16 @@ class AggregateFunctionNothing final : public IAggregateFunctionHelper<Aggregate
|
||||
{
|
||||
public:
|
||||
AggregateFunctionNothing(const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionHelper<AggregateFunctionNothing>(arguments, params) {}
|
||||
: IAggregateFunctionHelper<AggregateFunctionNothing>(arguments, params, createResultType(arguments)) {}
|
||||
|
||||
String getName() const override
|
||||
{
|
||||
return "nothing";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const DataTypes & arguments)
|
||||
{
|
||||
return argument_types.empty() ? std::make_shared<DataTypeNullable>(std::make_shared<DataTypeNothing>()) : argument_types.front();
|
||||
return arguments.empty() ? std::make_shared<DataTypeNullable>(std::make_shared<DataTypeNothing>()) : arguments.front();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
@ -87,7 +87,7 @@ public:
|
||||
transformed_nested_function->getParameters());
|
||||
}
|
||||
|
||||
bool return_type_is_nullable = !properties.returns_default_when_only_null && nested_function->getReturnType()->canBeInsideNullable();
|
||||
bool return_type_is_nullable = !properties.returns_default_when_only_null && nested_function->getResultType()->canBeInsideNullable();
|
||||
bool serialize_flag = return_type_is_nullable || properties.returns_default_when_only_null;
|
||||
|
||||
if (arguments.size() == 1)
|
||||
|
||||
@ -85,7 +85,8 @@ protected:
|
||||
|
||||
public:
|
||||
AggregateFunctionNullBase(AggregateFunctionPtr nested_function_, const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionHelper<Derived>(arguments, params), nested_function{nested_function_}
|
||||
: IAggregateFunctionHelper<Derived>(arguments, params, createResultType(nested_function_))
|
||||
, nested_function{nested_function_}
|
||||
{
|
||||
if constexpr (result_is_nullable)
|
||||
prefix_size = nested_function->alignOfData();
|
||||
@ -99,12 +100,12 @@ public:
|
||||
return nested_function->getName();
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const AggregateFunctionPtr & nested_function_)
|
||||
{
|
||||
if constexpr (result_is_nullable)
|
||||
return makeNullable(nested_function->getReturnType());
|
||||
return makeNullable(nested_function_->getResultType());
|
||||
else
|
||||
return nested_function->getReturnType();
|
||||
return nested_function_->getResultType();
|
||||
}
|
||||
|
||||
void create(AggregateDataPtr __restrict place) const override
|
||||
@ -275,7 +276,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, this->getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
|
||||
llvm::Value * result = nullptr;
|
||||
|
||||
|
||||
@ -30,16 +30,14 @@ private:
|
||||
AggregateFunctionPtr nested_function;
|
||||
|
||||
size_t size_of_data;
|
||||
DataTypePtr inner_type;
|
||||
bool inner_nullable;
|
||||
|
||||
public:
|
||||
AggregateFunctionOrFill(AggregateFunctionPtr nested_function_, const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionHelper<AggregateFunctionOrFill>{arguments, params}
|
||||
: IAggregateFunctionHelper<AggregateFunctionOrFill>{arguments, params, createResultType(nested_function_->getResultType())}
|
||||
, nested_function{nested_function_}
|
||||
, size_of_data {nested_function->sizeOfData()}
|
||||
, inner_type {nested_function->getReturnType()}
|
||||
, inner_nullable {inner_type->isNullable()}
|
||||
, inner_nullable {nested_function->getResultType()->isNullable()}
|
||||
{
|
||||
// nothing
|
||||
}
|
||||
@ -246,22 +244,22 @@ public:
|
||||
readChar(place[size_of_data], buf);
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const DataTypePtr & inner_type_)
|
||||
{
|
||||
if constexpr (UseNull)
|
||||
{
|
||||
// -OrNull
|
||||
|
||||
if (inner_nullable)
|
||||
return inner_type;
|
||||
if (inner_type_->isNullable())
|
||||
return inner_type_;
|
||||
|
||||
return std::make_shared<DataTypeNullable>(inner_type);
|
||||
return std::make_shared<DataTypeNullable>(inner_type_);
|
||||
}
|
||||
else
|
||||
{
|
||||
// -OrDefault
|
||||
|
||||
return inner_type;
|
||||
return inner_type_;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@ -72,7 +72,7 @@ private:
|
||||
public:
|
||||
AggregateFunctionQuantile(const DataTypes & argument_types_, const Array & params)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionQuantile<Value, Data, Name, has_second_arg, FloatReturnType, returns_many>>(
|
||||
argument_types_, params)
|
||||
argument_types_, params, createResultType(argument_types_))
|
||||
, levels(params, returns_many)
|
||||
, level(levels.levels[0])
|
||||
, argument_type(this->argument_types[0])
|
||||
@ -83,14 +83,14 @@ public:
|
||||
|
||||
String getName() const override { return Name::name; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const DataTypes & argument_types_)
|
||||
{
|
||||
DataTypePtr res;
|
||||
|
||||
if constexpr (returns_float)
|
||||
res = std::make_shared<DataTypeNumber<FloatReturnType>>();
|
||||
else
|
||||
res = argument_type;
|
||||
res = argument_types_[0];
|
||||
|
||||
if constexpr (returns_many)
|
||||
return std::make_shared<DataTypeArray>(res);
|
||||
|
||||
@ -51,7 +51,7 @@ class AggregateFunctionRankCorrelation :
|
||||
{
|
||||
public:
|
||||
explicit AggregateFunctionRankCorrelation(const DataTypes & arguments)
|
||||
:IAggregateFunctionDataHelper<RankCorrelationData, AggregateFunctionRankCorrelation> ({arguments}, {})
|
||||
:IAggregateFunctionDataHelper<RankCorrelationData, AggregateFunctionRankCorrelation> ({arguments}, {}, std::make_shared<DataTypeNumber<Float64>>())
|
||||
{}
|
||||
|
||||
String getName() const override
|
||||
@ -61,11 +61,6 @@ public:
|
||||
|
||||
bool allocatesMemoryInArena() const override { return true; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeNumber<Float64>>();
|
||||
}
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * arena) const override
|
||||
{
|
||||
Float64 new_x = columns[0]->getFloat64(row_num);
|
||||
|
||||
@ -43,7 +43,7 @@ public:
|
||||
size_t step_,
|
||||
const DataTypes & arguments,
|
||||
const Array & params)
|
||||
: IAggregateFunctionHelper<AggregateFunctionResample<Key>>{arguments, params}
|
||||
: IAggregateFunctionHelper<AggregateFunctionResample<Key>>{arguments, params, createResultType(nested_function_)}
|
||||
, nested_function{nested_function_}
|
||||
, last_col{arguments.size() - 1}
|
||||
, begin{begin_}
|
||||
@ -190,9 +190,9 @@ public:
|
||||
nested_function->deserialize(place + i * size_of_data, buf, version, arena);
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const AggregateFunctionPtr & nested_function_)
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(nested_function->getReturnType());
|
||||
return std::make_shared<DataTypeArray>(nested_function_->getResultType());
|
||||
}
|
||||
|
||||
template <bool merge>
|
||||
|
||||
@ -76,7 +76,7 @@ public:
|
||||
}
|
||||
|
||||
explicit AggregateFunctionRetention(const DataTypes & arguments)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionRetentionData, AggregateFunctionRetention>(arguments, {})
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionRetentionData, AggregateFunctionRetention>(arguments, {}, std::make_shared<DataTypeArray>(std::make_shared<DataTypeUInt8>()))
|
||||
{
|
||||
for (const auto i : collections::range(0, arguments.size()))
|
||||
{
|
||||
@ -90,12 +90,6 @@ public:
|
||||
events_size = static_cast<UInt8>(arguments.size());
|
||||
}
|
||||
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(std::make_shared<DataTypeUInt8>());
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, const size_t row_num, Arena *) const override
|
||||
|
||||
@ -126,8 +126,8 @@ template <typename T, typename Data, typename Derived>
|
||||
class AggregateFunctionSequenceBase : public IAggregateFunctionDataHelper<Data, Derived>
|
||||
{
|
||||
public:
|
||||
AggregateFunctionSequenceBase(const DataTypes & arguments, const Array & params, const String & pattern_)
|
||||
: IAggregateFunctionDataHelper<Data, Derived>(arguments, params)
|
||||
AggregateFunctionSequenceBase(const DataTypes & arguments, const Array & params, const String & pattern_, const DataTypePtr & result_type_)
|
||||
: IAggregateFunctionDataHelper<Data, Derived>(arguments, params, result_type_)
|
||||
, pattern(pattern_)
|
||||
{
|
||||
arg_count = arguments.size();
|
||||
@ -617,14 +617,12 @@ class AggregateFunctionSequenceMatch final : public AggregateFunctionSequenceBas
|
||||
{
|
||||
public:
|
||||
AggregateFunctionSequenceMatch(const DataTypes & arguments, const Array & params, const String & pattern_)
|
||||
: AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceMatch<T, Data>>(arguments, params, pattern_) {}
|
||||
: AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceMatch<T, Data>>(arguments, params, pattern_, std::make_shared<DataTypeUInt8>()) {}
|
||||
|
||||
using AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceMatch<T, Data>>::AggregateFunctionSequenceBase;
|
||||
|
||||
String getName() const override { return "sequenceMatch"; }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeUInt8>(); }
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void insertResultInto(AggregateDataPtr __restrict place, IColumn & to, Arena *) const override
|
||||
@ -655,14 +653,12 @@ class AggregateFunctionSequenceCount final : public AggregateFunctionSequenceBas
|
||||
{
|
||||
public:
|
||||
AggregateFunctionSequenceCount(const DataTypes & arguments, const Array & params, const String & pattern_)
|
||||
: AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceCount<T, Data>>(arguments, params, pattern_) {}
|
||||
: AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceCount<T, Data>>(arguments, params, pattern_, std::make_shared<DataTypeUInt64>()) {}
|
||||
|
||||
using AggregateFunctionSequenceBase<T, Data, AggregateFunctionSequenceCount<T, Data>>::AggregateFunctionSequenceBase;
|
||||
|
||||
String getName() const override { return "sequenceCount"; }
|
||||
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeUInt64>(); }
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void insertResultInto(AggregateDataPtr __restrict place, IColumn & to, Arena *) const override
|
||||
|
||||
@ -190,7 +190,7 @@ public:
|
||||
SequenceDirection seq_direction_,
|
||||
size_t min_required_args_,
|
||||
UInt64 max_elems_ = std::numeric_limits<UInt64>::max())
|
||||
: IAggregateFunctionDataHelper<SequenceNextNodeGeneralData<Node>, Self>({data_type_}, parameters_)
|
||||
: IAggregateFunctionDataHelper<SequenceNextNodeGeneralData<Node>, Self>({data_type_}, parameters_, data_type_)
|
||||
, seq_base_kind(seq_base_kind_)
|
||||
, seq_direction(seq_direction_)
|
||||
, min_required_args(min_required_args_)
|
||||
@ -202,8 +202,6 @@ public:
|
||||
|
||||
String getName() const override { return "sequenceNextNode"; }
|
||||
|
||||
DataTypePtr getReturnType() const override { return data_type; }
|
||||
|
||||
bool haveSameStateRepresentationImpl(const IAggregateFunction & rhs) const override
|
||||
{
|
||||
return this->getName() == rhs.getName() && this->haveEqualArgumentTypes(rhs);
|
||||
|
||||
@ -99,7 +99,7 @@ public:
|
||||
IAggregateFunctionDataHelper<
|
||||
AggregateFunctionSimpleLinearRegressionData<Ret>,
|
||||
AggregateFunctionSimpleLinearRegression<X, Y, Ret>
|
||||
> {arguments, params}
|
||||
> {arguments, params, createResultType()}
|
||||
{
|
||||
// notice: arguments has been checked before
|
||||
}
|
||||
@ -140,7 +140,7 @@ public:
|
||||
this->data(place).deserialize(buf);
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
DataTypes types
|
||||
{
|
||||
|
||||
@ -20,28 +20,28 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionSimpleState(AggregateFunctionPtr nested_, const DataTypes & arguments_, const Array & params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionSimpleState>(arguments_, params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionSimpleState>(arguments_, params_, createResultType(nested_, params_))
|
||||
, nested_func(nested_)
|
||||
{
|
||||
}
|
||||
|
||||
String getName() const override { return nested_func->getName() + "SimpleState"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const AggregateFunctionPtr & nested_, const Array & params_)
|
||||
{
|
||||
DataTypeCustomSimpleAggregateFunction::checkSupportedFunctions(nested_func);
|
||||
DataTypeCustomSimpleAggregateFunction::checkSupportedFunctions(nested_);
|
||||
|
||||
// Need to make a clone to avoid recursive reference.
|
||||
auto storage_type_out = DataTypeFactory::instance().get(nested_func->getReturnType()->getName());
|
||||
auto storage_type_out = DataTypeFactory::instance().get(nested_->getResultType()->getName());
|
||||
// Need to make a new function with promoted argument types because SimpleAggregates requires arg_type = return_type.
|
||||
AggregateFunctionProperties properties;
|
||||
auto function
|
||||
= AggregateFunctionFactory::instance().get(nested_func->getName(), {storage_type_out}, nested_func->getParameters(), properties);
|
||||
= AggregateFunctionFactory::instance().get(nested_->getName(), {storage_type_out}, nested_->getParameters(), properties);
|
||||
|
||||
// Need to make a clone because it'll be customized.
|
||||
auto storage_type_arg = DataTypeFactory::instance().get(nested_func->getReturnType()->getName());
|
||||
auto storage_type_arg = DataTypeFactory::instance().get(nested_->getResultType()->getName());
|
||||
DataTypeCustomNamePtr custom_name
|
||||
= std::make_unique<DataTypeCustomSimpleAggregateFunction>(function, DataTypes{nested_func->getReturnType()}, parameters);
|
||||
= std::make_unique<DataTypeCustomSimpleAggregateFunction>(function, DataTypes{nested_->getResultType()}, params_);
|
||||
storage_type_arg->setCustomization(std::make_unique<DataTypeCustomDesc>(std::move(custom_name), nullptr));
|
||||
return storage_type_arg;
|
||||
}
|
||||
|
||||
@ -261,7 +261,7 @@ private:
|
||||
public:
|
||||
AggregateFunctionSparkbar(const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionSparkbarData<X, Y>, AggregateFunctionSparkbar>(
|
||||
arguments, params)
|
||||
arguments, params, std::make_shared<DataTypeString>())
|
||||
{
|
||||
width = params.at(0).safeGet<UInt64>();
|
||||
if (params.size() == 3)
|
||||
@ -283,11 +283,6 @@ public:
|
||||
return "sparkbar";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeString>();
|
||||
}
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena * /*arena*/) const override
|
||||
{
|
||||
X x = assert_cast<const ColumnVector<X> *>(columns[0])->getData()[row_num];
|
||||
|
||||
@ -23,7 +23,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionState(AggregateFunctionPtr nested_, const DataTypes & arguments_, const Array & params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionState>(arguments_, params_)
|
||||
: IAggregateFunctionHelper<AggregateFunctionState>(arguments_, params_, nested_->getStateType())
|
||||
, nested_func(nested_)
|
||||
{}
|
||||
|
||||
@ -32,11 +32,6 @@ public:
|
||||
return nested_func->getName() + "State";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return getStateType();
|
||||
}
|
||||
|
||||
const IAggregateFunction & getBaseAggregateFunctionWithSameStateRepresentation() const override
|
||||
{
|
||||
return nested_func->getBaseAggregateFunctionWithSameStateRepresentation();
|
||||
|
||||
@ -115,15 +115,11 @@ class AggregateFunctionVariance final
|
||||
{
|
||||
public:
|
||||
explicit AggregateFunctionVariance(const DataTypePtr & arg)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionVarianceData<T, Op>, AggregateFunctionVariance<T, Op>>({arg}, {}) {}
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionVarianceData<T, Op>, AggregateFunctionVariance<T, Op>>({arg}, {}, std::make_shared<DataTypeFloat64>())
|
||||
{}
|
||||
|
||||
String getName() const override { return Op::name; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeFloat64>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||
@ -368,15 +364,11 @@ class AggregateFunctionCovariance final
|
||||
public:
|
||||
explicit AggregateFunctionCovariance(const DataTypes & args) : IAggregateFunctionDataHelper<
|
||||
CovarianceData<T, U, Op, compute_marginal_moments>,
|
||||
AggregateFunctionCovariance<T, U, Op, compute_marginal_moments>>(args, {}) {}
|
||||
AggregateFunctionCovariance<T, U, Op, compute_marginal_moments>>(args, {}, std::make_shared<DataTypeFloat64>())
|
||||
{}
|
||||
|
||||
String getName() const override { return Op::name; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeFloat64>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||
|
||||
@ -81,12 +81,12 @@ public:
|
||||
using ColVecResult = ColumnVector<ResultType>;
|
||||
|
||||
explicit AggregateFunctionVarianceSimple(const DataTypes & argument_types_)
|
||||
: IAggregateFunctionDataHelper<typename StatFunc::Data, AggregateFunctionVarianceSimple<StatFunc>>(argument_types_, {})
|
||||
: IAggregateFunctionDataHelper<typename StatFunc::Data, AggregateFunctionVarianceSimple<StatFunc>>(argument_types_, {}, std::make_shared<DataTypeNumber<ResultType>>())
|
||||
, src_scale(0)
|
||||
{}
|
||||
|
||||
AggregateFunctionVarianceSimple(const IDataType & data_type, const DataTypes & argument_types_)
|
||||
: IAggregateFunctionDataHelper<typename StatFunc::Data, AggregateFunctionVarianceSimple<StatFunc>>(argument_types_, {})
|
||||
: IAggregateFunctionDataHelper<typename StatFunc::Data, AggregateFunctionVarianceSimple<StatFunc>>(argument_types_, {}, std::make_shared<DataTypeNumber<ResultType>>())
|
||||
, src_scale(getDecimalScale(data_type))
|
||||
{}
|
||||
|
||||
@ -117,11 +117,6 @@ public:
|
||||
UNREACHABLE();
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeNumber<ResultType>>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||
|
||||
@ -411,23 +411,21 @@ public:
|
||||
}
|
||||
|
||||
explicit AggregateFunctionSum(const DataTypes & argument_types_)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionSum<T, TResult, Data, Type>>(argument_types_, {})
|
||||
, scale(0)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionSum<T, TResult, Data, Type>>(argument_types_, {}, createResultType(0))
|
||||
{}
|
||||
|
||||
AggregateFunctionSum(const IDataType & data_type, const DataTypes & argument_types_)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionSum<T, TResult, Data, Type>>(argument_types_, {})
|
||||
, scale(getDecimalScale(data_type))
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionSum<T, TResult, Data, Type>>(argument_types_, {}, createResultType(getDecimalScale(data_type)))
|
||||
{}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(UInt32 scale_)
|
||||
{
|
||||
if constexpr (!is_decimal<T>)
|
||||
return std::make_shared<DataTypeNumber<TResult>>();
|
||||
else
|
||||
{
|
||||
using DataType = DataTypeDecimal<TResult>;
|
||||
return std::make_shared<DataType>(DataType::maxPrecision(), scale);
|
||||
return std::make_shared<DataType>(DataType::maxPrecision(), scale_);
|
||||
}
|
||||
}
|
||||
|
||||
@ -548,7 +546,7 @@ public:
|
||||
for (const auto & argument_type : this->argument_types)
|
||||
can_be_compiled &= canBeNativeType(*argument_type);
|
||||
|
||||
auto return_type = getReturnType();
|
||||
auto return_type = this->getResultType();
|
||||
can_be_compiled &= canBeNativeType(*return_type);
|
||||
|
||||
return can_be_compiled;
|
||||
@ -558,7 +556,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
auto * aggregate_sum_ptr = aggregate_data_ptr;
|
||||
|
||||
b.CreateStore(llvm::Constant::getNullValue(return_type), aggregate_sum_ptr);
|
||||
@ -568,7 +566,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
|
||||
auto * sum_value_ptr = aggregate_data_ptr;
|
||||
auto * sum_value = b.CreateLoad(return_type, sum_value_ptr);
|
||||
@ -586,7 +584,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
|
||||
auto * sum_value_dst_ptr = aggregate_data_dst_ptr;
|
||||
auto * sum_value_dst = b.CreateLoad(return_type, sum_value_dst_ptr);
|
||||
@ -602,7 +600,7 @@ public:
|
||||
{
|
||||
llvm::IRBuilder<> & b = static_cast<llvm::IRBuilder<> &>(builder);
|
||||
|
||||
auto * return_type = toNativeType(b, getReturnType());
|
||||
auto * return_type = toNativeType(b, this->getResultType());
|
||||
auto * sum_value_ptr = aggregate_data_ptr;
|
||||
|
||||
return b.CreateLoad(return_type, sum_value_ptr);
|
||||
@ -611,8 +609,6 @@ public:
|
||||
#endif
|
||||
|
||||
private:
|
||||
UInt32 scale;
|
||||
|
||||
static constexpr auto & castColumnToResult(IColumn & to)
|
||||
{
|
||||
if constexpr (is_decimal<T>)
|
||||
|
||||
@ -14,12 +14,13 @@ public:
|
||||
using Base = AggregateFunctionAvg<T>;
|
||||
|
||||
explicit AggregateFunctionSumCount(const DataTypes & argument_types_, UInt32 num_scale_ = 0)
|
||||
: Base(argument_types_, num_scale_), scale(num_scale_) {}
|
||||
: Base(argument_types_, createResultType(num_scale_), num_scale_)
|
||||
{}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(UInt32 num_scale_)
|
||||
{
|
||||
auto second_elem = std::make_shared<DataTypeUInt64>();
|
||||
return std::make_shared<DataTypeTuple>(DataTypes{getReturnTypeFirstElement(), std::move(second_elem)});
|
||||
return std::make_shared<DataTypeTuple>(DataTypes{getReturnTypeFirstElement(num_scale_), std::move(second_elem)});
|
||||
}
|
||||
|
||||
void insertResultInto(AggregateDataPtr __restrict place, IColumn & to, Arena *) const final
|
||||
@ -43,9 +44,7 @@ public:
|
||||
#endif
|
||||
|
||||
private:
|
||||
UInt32 scale;
|
||||
|
||||
auto getReturnTypeFirstElement() const
|
||||
static auto getReturnTypeFirstElement(UInt32 num_scale_)
|
||||
{
|
||||
using FieldType = AvgFieldType<T>;
|
||||
|
||||
@ -54,7 +53,7 @@ private:
|
||||
else
|
||||
{
|
||||
using DataType = DataTypeDecimal<FieldType>;
|
||||
return std::make_shared<DataType>(DataType::maxPrecision(), scale);
|
||||
return std::make_shared<DataType>(DataType::maxPrecision(), num_scale_);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
@ -16,6 +16,7 @@
|
||||
#include <Common/FieldVisitorSum.h>
|
||||
#include <Common/assert_cast.h>
|
||||
#include <AggregateFunctions/IAggregateFunction.h>
|
||||
#include <AggregateFunctions/FactoryHelpers.h>
|
||||
#include <map>
|
||||
#include <Common/logger_useful.h>
|
||||
#include <Common/ClickHouseRevision.h>
|
||||
@ -80,7 +81,7 @@ public:
|
||||
|
||||
AggregateFunctionMapBase(const DataTypePtr & keys_type_,
|
||||
const DataTypes & values_types_, const DataTypes & argument_types_)
|
||||
: Base(argument_types_, {} /* parameters */)
|
||||
: Base(argument_types_, {} /* parameters */, createResultType(keys_type_, values_types_, getName()))
|
||||
, keys_type(keys_type_)
|
||||
, keys_serialization(keys_type->getDefaultSerialization())
|
||||
, values_types(values_types_)
|
||||
@ -117,19 +118,22 @@ public:
|
||||
return 0;
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(
|
||||
const DataTypePtr & keys_type_,
|
||||
const DataTypes & values_types_,
|
||||
const String & name_)
|
||||
{
|
||||
DataTypes types;
|
||||
types.emplace_back(std::make_shared<DataTypeArray>(keys_type));
|
||||
types.emplace_back(std::make_shared<DataTypeArray>(keys_type_));
|
||||
|
||||
for (const auto & value_type : values_types)
|
||||
for (const auto & value_type : values_types_)
|
||||
{
|
||||
if constexpr (std::is_same_v<Visitor, FieldVisitorSum>)
|
||||
{
|
||||
if (!value_type->isSummable())
|
||||
throw Exception{ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
||||
"Values for {} cannot be summed, passed type {}",
|
||||
getName(), value_type->getName()};
|
||||
name_, value_type->getName()};
|
||||
}
|
||||
|
||||
DataTypePtr result_type;
|
||||
@ -139,7 +143,7 @@ public:
|
||||
if (value_type->onlyNull())
|
||||
throw Exception{ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
||||
"Cannot calculate {} of type {}",
|
||||
getName(), value_type->getName()};
|
||||
name_, value_type->getName()};
|
||||
|
||||
// Overflow, meaning that the returned type is the same as
|
||||
// the input type. Nulls are skipped.
|
||||
@ -153,7 +157,7 @@ public:
|
||||
if (!value_type_without_nullable->canBePromoted())
|
||||
throw Exception{ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
||||
"Values for {} are expected to be Numeric, Float or Decimal, passed type {}",
|
||||
getName(), value_type->getName()};
|
||||
name_, value_type->getName()};
|
||||
|
||||
WhichDataType value_type_to_check(value_type_without_nullable);
|
||||
|
||||
@ -424,7 +428,10 @@ public:
|
||||
}
|
||||
|
||||
bool keepKey(const T & key) const { return static_cast<const Derived &>(*this).keepKey(key); }
|
||||
String getName() const override { return static_cast<const Derived &>(*this).getName(); }
|
||||
String getName() const override { return getNameImpl(); }
|
||||
|
||||
private:
|
||||
static String getNameImpl() { return Derived::getNameImpl(); }
|
||||
};
|
||||
|
||||
template <typename T, bool overflow, bool tuple_argument>
|
||||
@ -443,10 +450,10 @@ public:
|
||||
{
|
||||
// The constructor accepts parameters to have a uniform interface with
|
||||
// sumMapFiltered, but this function doesn't have any parameters.
|
||||
assertNoParameters(getName(), params_);
|
||||
assertNoParameters(getNameImpl(), params_);
|
||||
}
|
||||
|
||||
String getName() const override
|
||||
static String getNameImpl()
|
||||
{
|
||||
if constexpr (overflow)
|
||||
{
|
||||
@ -487,13 +494,13 @@ public:
|
||||
if (params_.size() != 1)
|
||||
throw Exception(ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
|
||||
"Aggregate function '{}' requires exactly one parameter "
|
||||
"of Array type", getName());
|
||||
"of Array type", getNameImpl());
|
||||
|
||||
Array keys_to_keep_values;
|
||||
if (!params_.front().tryGet<Array>(keys_to_keep_values))
|
||||
throw Exception(ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
|
||||
"Aggregate function {} requires an Array as a parameter",
|
||||
getName());
|
||||
getNameImpl());
|
||||
|
||||
keys_to_keep.reserve(keys_to_keep_values.size());
|
||||
|
||||
@ -501,7 +508,7 @@ public:
|
||||
keys_to_keep.emplace(f.safeGet<T>());
|
||||
}
|
||||
|
||||
String getName() const override
|
||||
static String getNameImpl()
|
||||
{ return overflow ? "sumMapFilteredWithOverflow" : "sumMapFiltered"; }
|
||||
|
||||
bool keepKey(const T & key) const { return keys_to_keep.count(key); }
|
||||
@ -606,10 +613,10 @@ public:
|
||||
{
|
||||
// The constructor accepts parameters to have a uniform interface with
|
||||
// sumMapFiltered, but this function doesn't have any parameters.
|
||||
assertNoParameters(getName(), params_);
|
||||
assertNoParameters(getNameImpl(), params_);
|
||||
}
|
||||
|
||||
String getName() const override { return "minMap"; }
|
||||
static String getNameImpl() { return "minMap"; }
|
||||
|
||||
bool keepKey(const T &) const { return true; }
|
||||
};
|
||||
@ -630,10 +637,10 @@ public:
|
||||
{
|
||||
// The constructor accepts parameters to have a uniform interface with
|
||||
// sumMapFiltered, but this function doesn't have any parameters.
|
||||
assertNoParameters(getName(), params_);
|
||||
assertNoParameters(getNameImpl(), params_);
|
||||
}
|
||||
|
||||
String getName() const override { return "maxMap"; }
|
||||
static String getNameImpl() { return "maxMap"; }
|
||||
|
||||
bool keepKey(const T &) const { return true; }
|
||||
};
|
||||
|
||||
@ -46,7 +46,7 @@ private:
|
||||
Float64 confidence_level;
|
||||
public:
|
||||
AggregateFunctionTTest(const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionTTest<Data>>({arguments}, params)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionTTest<Data>>({arguments}, params, createResultType(!params.empty()))
|
||||
{
|
||||
if (!params.empty())
|
||||
{
|
||||
@ -71,9 +71,9 @@ public:
|
||||
return Data::name;
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(bool need_confidence_interval_)
|
||||
{
|
||||
if (need_confidence_interval)
|
||||
if (need_confidence_interval_)
|
||||
{
|
||||
DataTypes types
|
||||
{
|
||||
|
||||
@ -31,15 +31,33 @@ namespace
|
||||
template <bool is_weighted>
|
||||
class AggregateFunctionTopKDate : public AggregateFunctionTopK<DataTypeDate::FieldType, is_weighted>
|
||||
{
|
||||
public:
|
||||
using AggregateFunctionTopK<DataTypeDate::FieldType, is_weighted>::AggregateFunctionTopK;
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDate>()); }
|
||||
|
||||
AggregateFunctionTopKDate(UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params)
|
||||
: AggregateFunctionTopK<DataTypeDate::FieldType, is_weighted>(
|
||||
threshold_,
|
||||
load_factor,
|
||||
argument_types_,
|
||||
params,
|
||||
std::make_shared<DataTypeArray>(std::make_shared<DataTypeDate>()))
|
||||
{}
|
||||
};
|
||||
|
||||
template <bool is_weighted>
|
||||
class AggregateFunctionTopKDateTime : public AggregateFunctionTopK<DataTypeDateTime::FieldType, is_weighted>
|
||||
{
|
||||
public:
|
||||
using AggregateFunctionTopK<DataTypeDateTime::FieldType, is_weighted>::AggregateFunctionTopK;
|
||||
DataTypePtr getReturnType() const override { return std::make_shared<DataTypeArray>(std::make_shared<DataTypeDateTime>()); }
|
||||
|
||||
AggregateFunctionTopKDateTime(UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params)
|
||||
: AggregateFunctionTopK<DataTypeDateTime::FieldType, is_weighted>(
|
||||
threshold_,
|
||||
load_factor,
|
||||
argument_types_,
|
||||
params,
|
||||
std::make_shared<DataTypeArray>(std::make_shared<DataTypeDateTime>()))
|
||||
{}
|
||||
};
|
||||
|
||||
|
||||
|
||||
@ -40,14 +40,20 @@ protected:
|
||||
|
||||
public:
|
||||
AggregateFunctionTopK(UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionTopKData<T>, AggregateFunctionTopK<T, is_weighted>>(argument_types_, params)
|
||||
, threshold(threshold_), reserved(load_factor * threshold) {}
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionTopKData<T>, AggregateFunctionTopK<T, is_weighted>>(argument_types_, params, createResultType(argument_types_))
|
||||
, threshold(threshold_), reserved(load_factor * threshold)
|
||||
{}
|
||||
|
||||
AggregateFunctionTopK(UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params, const DataTypePtr & result_type_)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionTopKData<T>, AggregateFunctionTopK<T, is_weighted>>(argument_types_, params, result_type_)
|
||||
, threshold(threshold_), reserved(load_factor * threshold)
|
||||
{}
|
||||
|
||||
String getName() const override { return is_weighted ? "topKWeighted" : "topK"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const DataTypes & argument_types_)
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(this->argument_types[0]);
|
||||
return std::make_shared<DataTypeArray>(argument_types_[0]);
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
@ -126,21 +132,20 @@ private:
|
||||
|
||||
UInt64 threshold;
|
||||
UInt64 reserved;
|
||||
DataTypePtr & input_data_type;
|
||||
|
||||
static void deserializeAndInsert(StringRef str, IColumn & data_to);
|
||||
|
||||
public:
|
||||
AggregateFunctionTopKGeneric(
|
||||
UInt64 threshold_, UInt64 load_factor, const DataTypes & argument_types_, const Array & params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionTopKGenericData, AggregateFunctionTopKGeneric<is_plain_column, is_weighted>>(argument_types_, params)
|
||||
, threshold(threshold_), reserved(load_factor * threshold), input_data_type(this->argument_types[0]) {}
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionTopKGenericData, AggregateFunctionTopKGeneric<is_plain_column, is_weighted>>(argument_types_, params, createResultType(argument_types_))
|
||||
, threshold(threshold_), reserved(load_factor * threshold) {}
|
||||
|
||||
String getName() const override { return is_weighted ? "topKWeighted" : "topK"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType(const DataTypes & argument_types_)
|
||||
{
|
||||
return std::make_shared<DataTypeArray>(input_data_type);
|
||||
return std::make_shared<DataTypeArray>(argument_types_[0]);
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override
|
||||
|
||||
@ -358,17 +358,12 @@ private:
|
||||
|
||||
public:
|
||||
explicit AggregateFunctionUniq(const DataTypes & argument_types_)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionUniq<T, Data>>(argument_types_, {})
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionUniq<T, Data>>(argument_types_, {}, std::make_shared<DataTypeUInt64>())
|
||||
{
|
||||
}
|
||||
|
||||
String getName() const override { return Data::getName(); }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeUInt64>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
/// ALWAYS_INLINE is required to have better code layout for uniqHLL12 function
|
||||
@ -462,7 +457,7 @@ private:
|
||||
|
||||
public:
|
||||
explicit AggregateFunctionUniqVariadic(const DataTypes & arguments)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionUniqVariadic<Data>>(arguments, {})
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionUniqVariadic<Data>>(arguments, {}, std::make_shared<DataTypeUInt64>())
|
||||
{
|
||||
if (argument_is_tuple)
|
||||
num_args = typeid_cast<const DataTypeTuple &>(*arguments[0]).getElements().size();
|
||||
@ -472,11 +467,6 @@ public:
|
||||
|
||||
String getName() const override { return Data::getName(); }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeUInt64>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||
|
||||
@ -126,7 +126,8 @@ class AggregateFunctionUniqCombined final
|
||||
{
|
||||
public:
|
||||
AggregateFunctionUniqCombined(const DataTypes & argument_types_, const Array & params_)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqCombinedData<T, K, HashValueType>, AggregateFunctionUniqCombined<T, K, HashValueType>>(argument_types_, params_) {}
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqCombinedData<T, K, HashValueType>, AggregateFunctionUniqCombined<T, K, HashValueType>>(argument_types_, params_, std::make_shared<DataTypeUInt64>())
|
||||
{}
|
||||
|
||||
String getName() const override
|
||||
{
|
||||
@ -136,11 +137,6 @@ public:
|
||||
return "uniqCombined";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeUInt64>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||
@ -192,7 +188,7 @@ private:
|
||||
public:
|
||||
explicit AggregateFunctionUniqCombinedVariadic(const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqCombinedData<UInt64, K, HashValueType>,
|
||||
AggregateFunctionUniqCombinedVariadic<is_exact, argument_is_tuple, K, HashValueType>>(arguments, params)
|
||||
AggregateFunctionUniqCombinedVariadic<is_exact, argument_is_tuple, K, HashValueType>>(arguments, params, std::make_shared<DataTypeUInt64>())
|
||||
{
|
||||
if (argument_is_tuple)
|
||||
num_args = typeid_cast<const DataTypeTuple &>(*arguments[0]).getElements().size();
|
||||
@ -208,11 +204,6 @@ public:
|
||||
return "uniqCombined";
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeUInt64>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||
|
||||
@ -174,7 +174,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionUniqUpTo(UInt8 threshold_, const DataTypes & argument_types_, const Array & params_)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqUpToData<T>, AggregateFunctionUniqUpTo<T>>(argument_types_, params_)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqUpToData<T>, AggregateFunctionUniqUpTo<T>>(argument_types_, params_, std::make_shared<DataTypeUInt64>())
|
||||
, threshold(threshold_)
|
||||
{
|
||||
}
|
||||
@ -186,11 +186,6 @@ public:
|
||||
|
||||
String getName() const override { return "uniqUpTo"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeUInt64>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
/// ALWAYS_INLINE is required to have better code layout for uniqUpTo function
|
||||
@ -235,7 +230,7 @@ private:
|
||||
|
||||
public:
|
||||
AggregateFunctionUniqUpToVariadic(const DataTypes & arguments, const Array & params, UInt8 threshold_)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqUpToData<UInt64>, AggregateFunctionUniqUpToVariadic<is_exact, argument_is_tuple>>(arguments, params)
|
||||
: IAggregateFunctionDataHelper<AggregateFunctionUniqUpToData<UInt64>, AggregateFunctionUniqUpToVariadic<is_exact, argument_is_tuple>>(arguments, params, std::make_shared<DataTypeUInt64>())
|
||||
, threshold(threshold_)
|
||||
{
|
||||
if (argument_is_tuple)
|
||||
@ -251,11 +246,6 @@ public:
|
||||
|
||||
String getName() const override { return "uniqUpTo"; }
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeUInt64>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, size_t row_num, Arena *) const override
|
||||
|
||||
@ -221,7 +221,7 @@ public:
|
||||
}
|
||||
|
||||
AggregateFunctionWindowFunnel(const DataTypes & arguments, const Array & params)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionWindowFunnel<T, Data>>(arguments, params)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionWindowFunnel<T, Data>>(arguments, params, std::make_shared<DataTypeUInt8>())
|
||||
{
|
||||
events_size = arguments.size() - 1;
|
||||
window = params.at(0).safeGet<UInt64>();
|
||||
@ -245,11 +245,6 @@ public:
|
||||
}
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
{
|
||||
return std::make_shared<DataTypeUInt8>();
|
||||
}
|
||||
|
||||
bool allocatesMemoryInArena() const override { return false; }
|
||||
|
||||
void add(AggregateDataPtr __restrict place, const IColumn ** columns, const size_t row_num, Arena *) const override
|
||||
|
||||
@ -118,7 +118,7 @@ class AggregateFunctionCrossTab : public IAggregateFunctionDataHelper<Data, Aggr
|
||||
{
|
||||
public:
|
||||
explicit AggregateFunctionCrossTab(const DataTypes & arguments)
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionCrossTab<Data>>({arguments}, {})
|
||||
: IAggregateFunctionDataHelper<Data, AggregateFunctionCrossTab<Data>>({arguments}, {}, createResultType())
|
||||
{
|
||||
}
|
||||
|
||||
@ -132,7 +132,7 @@ public:
|
||||
return false;
|
||||
}
|
||||
|
||||
DataTypePtr getReturnType() const override
|
||||
static DataTypePtr createResultType()
|
||||
{
|
||||
return std::make_shared<DataTypeNumber<Float64>>();
|
||||
}
|
||||
|
||||
@ -10,6 +10,7 @@
|
||||
#include <base/types.h>
|
||||
#include <Common/Exception.h>
|
||||
#include <Common/ThreadPool.h>
|
||||
#include <Core/IResolvedFunction.h>
|
||||
|
||||
#include "config.h"
|
||||
|
||||
@ -49,6 +50,7 @@ using ConstAggregateDataPtr = const char *;
|
||||
|
||||
class IAggregateFunction;
|
||||
using AggregateFunctionPtr = std::shared_ptr<const IAggregateFunction>;
|
||||
|
||||
struct AggregateFunctionProperties;
|
||||
|
||||
/** Aggregate functions interface.
|
||||
@ -59,18 +61,18 @@ struct AggregateFunctionProperties;
|
||||
* (which can be created in some memory pool),
|
||||
* and IAggregateFunction is the external interface for manipulating them.
|
||||
*/
|
||||
class IAggregateFunction : public std::enable_shared_from_this<IAggregateFunction>
|
||||
class IAggregateFunction : public std::enable_shared_from_this<IAggregateFunction>, public IResolvedFunction
|
||||
{
|
||||
public:
|
||||
IAggregateFunction(const DataTypes & argument_types_, const Array & parameters_)
|
||||
: argument_types(argument_types_), parameters(parameters_) {}
|
||||
IAggregateFunction(const DataTypes & argument_types_, const Array & parameters_, const DataTypePtr & result_type_)
|
||||
: result_type(result_type_)
|
||||
, argument_types(argument_types_)
|
||||
, parameters(parameters_)
|
||||
{}
|
||||
|
||||
/// Get main function name.
|
||||
virtual String getName() const = 0;
|
||||
|
||||
/// Get the result type.
|
||||
virtual DataTypePtr getReturnType() const = 0;
|
||||
|
||||
/// Get the data type of internal state. By default it is AggregateFunction(name(params), argument_types...).
|
||||
virtual DataTypePtr getStateType() const;
|
||||
|
||||
@ -102,7 +104,7 @@ public:
|
||||
|
||||
virtual size_t getDefaultVersion() const { return 0; }
|
||||
|
||||
virtual ~IAggregateFunction() = default;
|
||||
~IAggregateFunction() override = default;
|
||||
|
||||
/** Data manipulating functions. */
|
||||
|
||||
@ -348,8 +350,9 @@ public:
|
||||
*/
|
||||
virtual AggregateFunctionPtr getNestedFunction() const { return {}; }
|
||||
|
||||
const DataTypes & getArgumentTypes() const { return argument_types; }
|
||||
const Array & getParameters() const { return parameters; }
|
||||
const DataTypePtr & getResultType() const override { return result_type; }
|
||||
const DataTypes & getArgumentTypes() const override { return argument_types; }
|
||||
const Array & getParameters() const override { return parameters; }
|
||||
|
||||
// Any aggregate function can be calculated over a window, but there are some
|
||||
// window functions such as rank() that require a different interface, e.g.
|
||||
@ -398,6 +401,7 @@ public:
|
||||
#endif
|
||||
|
||||
protected:
|
||||
DataTypePtr result_type;
|
||||
DataTypes argument_types;
|
||||
Array parameters;
|
||||
};
|
||||
@ -414,8 +418,8 @@ private:
|
||||
}
|
||||
|
||||
public:
|
||||
IAggregateFunctionHelper(const DataTypes & argument_types_, const Array & parameters_)
|
||||
: IAggregateFunction(argument_types_, parameters_) {}
|
||||
IAggregateFunctionHelper(const DataTypes & argument_types_, const Array & parameters_, const DataTypePtr & result_type_)
|
||||
: IAggregateFunction(argument_types_, parameters_, result_type_) {}
|
||||
|
||||
AddFunc getAddressOfAddFunction() const override { return &addFree; }
|
||||
|
||||
@ -695,15 +699,15 @@ public:
|
||||
// Derived class can `override` this to flag that DateTime64 is not supported.
|
||||
static constexpr bool DateTime64Supported = true;
|
||||
|
||||
IAggregateFunctionDataHelper(const DataTypes & argument_types_, const Array & parameters_)
|
||||
: IAggregateFunctionHelper<Derived>(argument_types_, parameters_)
|
||||
IAggregateFunctionDataHelper(const DataTypes & argument_types_, const Array & parameters_, const DataTypePtr & result_type_)
|
||||
: IAggregateFunctionHelper<Derived>(argument_types_, parameters_, result_type_)
|
||||
{
|
||||
/// To prevent derived classes changing the destroy() without updating hasTrivialDestructor() to match it
|
||||
/// Enforce that either both of them are changed or none are
|
||||
constexpr bool declares_destroy_and_hasTrivialDestructor =
|
||||
constexpr bool declares_destroy_and_has_trivial_destructor =
|
||||
std::is_same_v<decltype(&IAggregateFunctionDataHelper::destroy), decltype(&Derived::destroy)> ==
|
||||
std::is_same_v<decltype(&IAggregateFunctionDataHelper::hasTrivialDestructor), decltype(&Derived::hasTrivialDestructor)>;
|
||||
static_assert(declares_destroy_and_hasTrivialDestructor,
|
||||
static_assert(declares_destroy_and_has_trivial_destructor,
|
||||
"destroy() and hasTrivialDestructor() methods of an aggregate function must be either both overridden or not");
|
||||
}
|
||||
|
||||
|
||||
@ -2,6 +2,7 @@
|
||||
|
||||
#include <Common/SipHash.h>
|
||||
#include <Common/FieldVisitorToString.h>
|
||||
#include <Analyzer/ConstantNode.h>
|
||||
|
||||
#include <IO/WriteBufferFromString.h>
|
||||
#include <IO/Operators.h>
|
||||
@ -17,6 +18,11 @@
|
||||
namespace DB
|
||||
{
|
||||
|
||||
namespace ErrorCodes
|
||||
{
|
||||
extern const int LOGICAL_ERROR;
|
||||
}
|
||||
|
||||
FunctionNode::FunctionNode(String function_name_)
|
||||
: IQueryTreeNode(children_size)
|
||||
, function_name(function_name_)
|
||||
@ -25,25 +31,41 @@ FunctionNode::FunctionNode(String function_name_)
|
||||
children[arguments_child_index] = std::make_shared<ListNode>();
|
||||
}
|
||||
|
||||
void FunctionNode::resolveAsFunction(FunctionOverloadResolverPtr function_value, DataTypePtr result_type_value)
|
||||
ColumnsWithTypeAndName FunctionNode::getArgumentTypes() const
|
||||
{
|
||||
aggregate_function = nullptr;
|
||||
ColumnsWithTypeAndName argument_types;
|
||||
for (const auto & arg : getArguments().getNodes())
|
||||
{
|
||||
ColumnWithTypeAndName argument;
|
||||
argument.type = arg->getResultType();
|
||||
if (auto * constant = arg->as<ConstantNode>())
|
||||
argument.column = argument.type->createColumnConst(1, constant->getValue());
|
||||
argument_types.push_back(argument);
|
||||
}
|
||||
return argument_types;
|
||||
}
|
||||
|
||||
void FunctionNode::resolveAsFunction(FunctionBasePtr function_value)
|
||||
{
|
||||
function_name = function_value->getName();
|
||||
function = std::move(function_value);
|
||||
result_type = std::move(result_type_value);
|
||||
function_name = function->getName();
|
||||
kind = FunctionKind::ORDINARY;
|
||||
}
|
||||
|
||||
void FunctionNode::resolveAsAggregateFunction(AggregateFunctionPtr aggregate_function_value, DataTypePtr result_type_value)
|
||||
void FunctionNode::resolveAsAggregateFunction(AggregateFunctionPtr aggregate_function_value)
|
||||
{
|
||||
function = nullptr;
|
||||
aggregate_function = std::move(aggregate_function_value);
|
||||
result_type = std::move(result_type_value);
|
||||
function_name = aggregate_function->getName();
|
||||
function_name = aggregate_function_value->getName();
|
||||
function = std::move(aggregate_function_value);
|
||||
kind = FunctionKind::AGGREGATE;
|
||||
}
|
||||
|
||||
void FunctionNode::resolveAsWindowFunction(AggregateFunctionPtr window_function_value, DataTypePtr result_type_value)
|
||||
void FunctionNode::resolveAsWindowFunction(AggregateFunctionPtr window_function_value)
|
||||
{
|
||||
resolveAsAggregateFunction(window_function_value, result_type_value);
|
||||
if (!hasWindow())
|
||||
throw Exception(ErrorCodes::LOGICAL_ERROR,
|
||||
"Trying to resolve FunctionNode without window definition as a window function {}", window_function_value->getName());
|
||||
resolveAsAggregateFunction(window_function_value);
|
||||
kind = FunctionKind::WINDOW;
|
||||
}
|
||||
|
||||
void FunctionNode::dumpTreeImpl(WriteBuffer & buffer, FormatState & format_state, size_t indent) const
|
||||
@ -63,8 +85,8 @@ void FunctionNode::dumpTreeImpl(WriteBuffer & buffer, FormatState & format_state
|
||||
|
||||
buffer << ", function_type: " << function_type;
|
||||
|
||||
if (result_type)
|
||||
buffer << ", result_type: " + result_type->getName();
|
||||
if (function)
|
||||
buffer << ", result_type: " + function->getResultType()->getName();
|
||||
|
||||
const auto & parameters = getParameters();
|
||||
if (!parameters.getNodes().empty())
|
||||
@ -96,11 +118,19 @@ bool FunctionNode::isEqualImpl(const IQueryTreeNode & rhs) const
|
||||
isWindowFunction() != rhs_typed.isWindowFunction())
|
||||
return false;
|
||||
|
||||
if (result_type && rhs_typed.result_type && !result_type->equals(*rhs_typed.getResultType()))
|
||||
if (isResolved() != rhs_typed.isResolved())
|
||||
return false;
|
||||
else if (result_type && !rhs_typed.result_type)
|
||||
if (!isResolved())
|
||||
return true;
|
||||
|
||||
auto lhs_result_type = getResultType();
|
||||
auto rhs_result_type = rhs.getResultType();
|
||||
|
||||
if (lhs_result_type && rhs_result_type && !lhs_result_type->equals(*rhs_result_type))
|
||||
return false;
|
||||
else if (!result_type && rhs_typed.result_type)
|
||||
else if (lhs_result_type && !rhs_result_type)
|
||||
return false;
|
||||
else if (!lhs_result_type && rhs_result_type)
|
||||
return false;
|
||||
|
||||
return true;
|
||||
@ -114,7 +144,10 @@ void FunctionNode::updateTreeHashImpl(HashState & hash_state) const
|
||||
hash_state.update(isAggregateFunction());
|
||||
hash_state.update(isWindowFunction());
|
||||
|
||||
if (result_type)
|
||||
if (!isResolved())
|
||||
return;
|
||||
|
||||
if (auto result_type = getResultType())
|
||||
{
|
||||
auto result_type_name = result_type->getName();
|
||||
hash_state.update(result_type_name.size());
|
||||
@ -130,8 +163,7 @@ QueryTreeNodePtr FunctionNode::cloneImpl() const
|
||||
* because ordinary functions or aggregate functions must be stateless.
|
||||
*/
|
||||
result_function->function = function;
|
||||
result_function->aggregate_function = aggregate_function;
|
||||
result_function->result_type = result_type;
|
||||
result_function->kind = kind;
|
||||
|
||||
return result_function;
|
||||
}
|
||||
|
||||
@ -1,8 +1,12 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <Core/IResolvedFunction.h>
|
||||
#include <Analyzer/IQueryTreeNode.h>
|
||||
#include <Analyzer/ListNode.h>
|
||||
#include <Analyzer/ConstantValue.h>
|
||||
#include <Common/typeid_cast.h>
|
||||
#include <Core/ColumnsWithTypeAndName.h>
|
||||
|
||||
namespace DB
|
||||
{
|
||||
@ -15,6 +19,9 @@ namespace ErrorCodes
|
||||
class IFunctionOverloadResolver;
|
||||
using FunctionOverloadResolverPtr = std::shared_ptr<IFunctionOverloadResolver>;
|
||||
|
||||
class IFunctionBase;
|
||||
using FunctionBasePtr = std::shared_ptr<const IFunctionBase>;
|
||||
|
||||
class IAggregateFunction;
|
||||
using AggregateFunctionPtr = std::shared_ptr<const IAggregateFunction>;
|
||||
|
||||
@ -39,6 +46,14 @@ using AggregateFunctionPtr = std::shared_ptr<const IAggregateFunction>;
|
||||
class FunctionNode;
|
||||
using FunctionNodePtr = std::shared_ptr<FunctionNode>;
|
||||
|
||||
enum class FunctionKind
|
||||
{
|
||||
UNKNOWN,
|
||||
ORDINARY,
|
||||
AGGREGATE,
|
||||
WINDOW,
|
||||
};
|
||||
|
||||
class FunctionNode final : public IQueryTreeNode
|
||||
{
|
||||
public:
|
||||
@ -101,6 +116,8 @@ public:
|
||||
return children[arguments_child_index];
|
||||
}
|
||||
|
||||
ColumnsWithTypeAndName getArgumentTypes() const;
|
||||
|
||||
/// Returns true if function node has window, false otherwise
|
||||
bool hasWindow() const
|
||||
{
|
||||
@ -129,42 +146,46 @@ public:
|
||||
/** Get non aggregate function.
|
||||
* If function is not resolved nullptr returned.
|
||||
*/
|
||||
const FunctionOverloadResolverPtr & getFunction() const
|
||||
FunctionBasePtr getFunction() const
|
||||
{
|
||||
return function;
|
||||
if (kind != FunctionKind::ORDINARY)
|
||||
return {};
|
||||
return std::reinterpret_pointer_cast<const IFunctionBase>(function);
|
||||
}
|
||||
|
||||
/** Get aggregate function.
|
||||
* If function is not resolved nullptr returned.
|
||||
* If function is resolved as non aggregate function nullptr returned.
|
||||
*/
|
||||
const AggregateFunctionPtr & getAggregateFunction() const
|
||||
AggregateFunctionPtr getAggregateFunction() const
|
||||
{
|
||||
return aggregate_function;
|
||||
if (kind == FunctionKind::UNKNOWN || kind == FunctionKind::ORDINARY)
|
||||
return {};
|
||||
return std::reinterpret_pointer_cast<const IAggregateFunction>(function);
|
||||
}
|
||||
|
||||
/// Is function node resolved
|
||||
bool isResolved() const
|
||||
{
|
||||
return result_type != nullptr && (function != nullptr || aggregate_function != nullptr);
|
||||
return function != nullptr;
|
||||
}
|
||||
|
||||
/// Is function node window function
|
||||
bool isWindowFunction() const
|
||||
{
|
||||
return getWindowNode() != nullptr;
|
||||
return hasWindow();
|
||||
}
|
||||
|
||||
/// Is function node aggregate function
|
||||
bool isAggregateFunction() const
|
||||
{
|
||||
return aggregate_function != nullptr && !isWindowFunction();
|
||||
return kind == FunctionKind::AGGREGATE;
|
||||
}
|
||||
|
||||
/// Is function node ordinary function
|
||||
bool isOrdinaryFunction() const
|
||||
{
|
||||
return function != nullptr;
|
||||
return kind == FunctionKind::ORDINARY;
|
||||
}
|
||||
|
||||
/** Resolve function node as non aggregate function.
|
||||
@ -173,19 +194,19 @@ public:
|
||||
* Assume we have `multiIf` function with single condition, it can be converted to `if` function.
|
||||
* Function name must be updated accordingly.
|
||||
*/
|
||||
void resolveAsFunction(FunctionOverloadResolverPtr function_value, DataTypePtr result_type_value);
|
||||
void resolveAsFunction(FunctionBasePtr function_value);
|
||||
|
||||
/** Resolve function node as aggregate function.
|
||||
* It is important that function name is updated with resolved function name.
|
||||
* Main motivation for this is query tree optimizations.
|
||||
*/
|
||||
void resolveAsAggregateFunction(AggregateFunctionPtr aggregate_function_value, DataTypePtr result_type_value);
|
||||
void resolveAsAggregateFunction(AggregateFunctionPtr aggregate_function_value);
|
||||
|
||||
/** Resolve function node as window function.
|
||||
* It is important that function name is updated with resolved function name.
|
||||
* Main motivation for this is query tree optimizations.
|
||||
*/
|
||||
void resolveAsWindowFunction(AggregateFunctionPtr window_function_value, DataTypePtr result_type_value);
|
||||
void resolveAsWindowFunction(AggregateFunctionPtr window_function_value);
|
||||
|
||||
QueryTreeNodeType getNodeType() const override
|
||||
{
|
||||
@ -194,12 +215,11 @@ public:
|
||||
|
||||
DataTypePtr getResultType() const override
|
||||
{
|
||||
if (!result_type)
|
||||
if (!function)
|
||||
throw Exception(ErrorCodes::UNSUPPORTED_METHOD,
|
||||
"Function node with name '{}' is not resolved",
|
||||
function_name);
|
||||
|
||||
return result_type;
|
||||
return function->getResultType();
|
||||
}
|
||||
|
||||
void dumpTreeImpl(WriteBuffer & buffer, FormatState & format_state, size_t indent) const override;
|
||||
@ -215,9 +235,8 @@ protected:
|
||||
|
||||
private:
|
||||
String function_name;
|
||||
FunctionOverloadResolverPtr function;
|
||||
AggregateFunctionPtr aggregate_function;
|
||||
DataTypePtr result_type;
|
||||
FunctionKind kind = FunctionKind::UNKNOWN;
|
||||
IResolvedFunctionPtr function;
|
||||
|
||||
static constexpr size_t parameters_child_index = 0;
|
||||
static constexpr size_t arguments_child_index = 1;
|
||||
|
||||
@ -147,7 +147,6 @@ public:
|
||||
private:
|
||||
static inline void resolveAggregateFunctionNode(FunctionNode & function_node, const String & aggregate_function_name)
|
||||
{
|
||||
auto function_result_type = function_node.getResultType();
|
||||
auto function_aggregate_function = function_node.getAggregateFunction();
|
||||
|
||||
AggregateFunctionProperties properties;
|
||||
@ -156,7 +155,7 @@ private:
|
||||
function_aggregate_function->getParameters(),
|
||||
properties);
|
||||
|
||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function), std::move(function_result_type));
|
||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function));
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@ -71,7 +71,7 @@ public:
|
||||
auto result_type = function_node->getResultType();
|
||||
AggregateFunctionProperties properties;
|
||||
auto aggregate_function = AggregateFunctionFactory::instance().get("count", {}, {}, properties);
|
||||
function_node->resolveAsAggregateFunction(std::move(aggregate_function), std::move(result_type));
|
||||
function_node->resolveAsAggregateFunction(std::move(aggregate_function));
|
||||
function_node->getArguments().getNodes().clear();
|
||||
}
|
||||
};
|
||||
|
||||
@ -138,7 +138,6 @@ public:
|
||||
|
||||
static inline void resolveAggregateOrWindowFunctionNode(FunctionNode & function_node, const String & aggregate_function_name)
|
||||
{
|
||||
auto function_result_type = function_node.getResultType();
|
||||
auto function_aggregate_function = function_node.getAggregateFunction();
|
||||
|
||||
AggregateFunctionProperties properties;
|
||||
@ -148,16 +147,15 @@ public:
|
||||
properties);
|
||||
|
||||
if (function_node.isAggregateFunction())
|
||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function), std::move(function_result_type));
|
||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function));
|
||||
else if (function_node.isWindowFunction())
|
||||
function_node.resolveAsWindowFunction(std::move(aggregate_function), std::move(function_result_type));
|
||||
function_node.resolveAsWindowFunction(std::move(aggregate_function));
|
||||
}
|
||||
|
||||
inline void resolveOrdinaryFunctionNode(FunctionNode & function_node, const String & function_name) const
|
||||
{
|
||||
auto function_result_type = function_node.getResultType();
|
||||
auto function = FunctionFactory::instance().get(function_name, context);
|
||||
function_node.resolveAsFunction(function, std::move(function_result_type));
|
||||
function_node.resolveAsFunction(function->build(function_node.getArgumentTypes()));
|
||||
}
|
||||
|
||||
private:
|
||||
|
||||
@ -78,11 +78,11 @@ public:
|
||||
column.name += ".size0";
|
||||
column.type = std::make_shared<DataTypeUInt64>();
|
||||
|
||||
resolveOrdinaryFunctionNode(*function_node, "equals");
|
||||
|
||||
function_arguments_nodes.clear();
|
||||
function_arguments_nodes.push_back(std::make_shared<ColumnNode>(column, column_source));
|
||||
function_arguments_nodes.push_back(std::make_shared<ConstantNode>(static_cast<UInt64>(0)));
|
||||
|
||||
resolveOrdinaryFunctionNode(*function_node, "equals");
|
||||
}
|
||||
else if (function_name == "notEmpty")
|
||||
{
|
||||
@ -90,11 +90,11 @@ public:
|
||||
column.name += ".size0";
|
||||
column.type = std::make_shared<DataTypeUInt64>();
|
||||
|
||||
resolveOrdinaryFunctionNode(*function_node, "notEquals");
|
||||
|
||||
function_arguments_nodes.clear();
|
||||
function_arguments_nodes.push_back(std::make_shared<ColumnNode>(column, column_source));
|
||||
function_arguments_nodes.push_back(std::make_shared<ConstantNode>(static_cast<UInt64>(0)));
|
||||
|
||||
resolveOrdinaryFunctionNode(*function_node, "notEquals");
|
||||
}
|
||||
}
|
||||
else if (column_type.isNullable())
|
||||
@ -112,9 +112,9 @@ public:
|
||||
column.name += ".null";
|
||||
column.type = std::make_shared<DataTypeUInt8>();
|
||||
|
||||
resolveOrdinaryFunctionNode(*function_node, "not");
|
||||
|
||||
function_arguments_nodes = {std::make_shared<ColumnNode>(column, column_source)};
|
||||
|
||||
resolveOrdinaryFunctionNode(*function_node, "not");
|
||||
}
|
||||
}
|
||||
else if (column_type.isMap())
|
||||
@ -182,9 +182,9 @@ public:
|
||||
column.type = data_type_map.getKeyType();
|
||||
|
||||
auto has_function_argument = std::make_shared<ColumnNode>(column, column_source);
|
||||
resolveOrdinaryFunctionNode(*function_node, "has");
|
||||
|
||||
function_arguments_nodes[0] = std::move(has_function_argument);
|
||||
|
||||
resolveOrdinaryFunctionNode(*function_node, "has");
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -192,9 +192,8 @@ public:
|
||||
private:
|
||||
inline void resolveOrdinaryFunctionNode(FunctionNode & function_node, const String & function_name) const
|
||||
{
|
||||
auto function_result_type = function_node.getResultType();
|
||||
auto function = FunctionFactory::instance().get(function_name, context);
|
||||
function_node.resolveAsFunction(function, std::move(function_result_type));
|
||||
function_node.resolveAsFunction(function->build(function_node.getArgumentTypes()));
|
||||
}
|
||||
|
||||
ContextPtr & context;
|
||||
|
||||
@ -59,14 +59,13 @@ private:
|
||||
std::unordered_set<String> names_to_collect;
|
||||
};
|
||||
|
||||
QueryTreeNodePtr createResolvedFunction(const ContextPtr & context, const String & name, const DataTypePtr & result_type, QueryTreeNodes arguments)
|
||||
QueryTreeNodePtr createResolvedFunction(const ContextPtr & context, const String & name, QueryTreeNodes arguments)
|
||||
{
|
||||
auto function_node = std::make_shared<FunctionNode>(name);
|
||||
|
||||
auto function = FunctionFactory::instance().get(name, context);
|
||||
function_node->resolveAsFunction(std::move(function), result_type);
|
||||
function_node->getArguments().getNodes() = std::move(arguments);
|
||||
|
||||
function_node->resolveAsFunction(function->build(function_node->getArgumentTypes()));
|
||||
return function_node;
|
||||
}
|
||||
|
||||
@ -74,11 +73,6 @@ FunctionNodePtr createResolvedAggregateFunction(const String & name, const Query
|
||||
{
|
||||
auto function_node = std::make_shared<FunctionNode>(name);
|
||||
|
||||
AggregateFunctionProperties properties;
|
||||
auto aggregate_function = AggregateFunctionFactory::instance().get(name, {argument->getResultType()}, parameters, properties);
|
||||
function_node->resolveAsAggregateFunction(aggregate_function, aggregate_function->getReturnType());
|
||||
function_node->getArguments().getNodes() = { argument };
|
||||
|
||||
if (!parameters.empty())
|
||||
{
|
||||
QueryTreeNodes parameter_nodes;
|
||||
@ -86,18 +80,27 @@ FunctionNodePtr createResolvedAggregateFunction(const String & name, const Query
|
||||
parameter_nodes.emplace_back(std::make_shared<ConstantNode>(param));
|
||||
function_node->getParameters().getNodes() = std::move(parameter_nodes);
|
||||
}
|
||||
function_node->getArguments().getNodes() = { argument };
|
||||
|
||||
AggregateFunctionProperties properties;
|
||||
auto aggregate_function = AggregateFunctionFactory::instance().get(
|
||||
name,
|
||||
{ argument->getResultType() },
|
||||
parameters,
|
||||
properties);
|
||||
function_node->resolveAsAggregateFunction(aggregate_function);
|
||||
|
||||
return function_node;
|
||||
}
|
||||
|
||||
QueryTreeNodePtr createTupleElementFunction(const ContextPtr & context, const DataTypePtr & result_type, QueryTreeNodePtr argument, UInt64 index)
|
||||
QueryTreeNodePtr createTupleElementFunction(const ContextPtr & context, QueryTreeNodePtr argument, UInt64 index)
|
||||
{
|
||||
return createResolvedFunction(context, "tupleElement", result_type, {std::move(argument), std::make_shared<ConstantNode>(index)});
|
||||
return createResolvedFunction(context, "tupleElement", {argument, std::make_shared<ConstantNode>(index)});
|
||||
}
|
||||
|
||||
QueryTreeNodePtr createArrayElementFunction(const ContextPtr & context, const DataTypePtr & result_type, QueryTreeNodePtr argument, UInt64 index)
|
||||
QueryTreeNodePtr createArrayElementFunction(const ContextPtr & context, QueryTreeNodePtr argument, UInt64 index)
|
||||
{
|
||||
return createResolvedFunction(context, "arrayElement", result_type, {std::move(argument), std::make_shared<ConstantNode>(index)});
|
||||
return createResolvedFunction(context, "arrayElement", {argument, std::make_shared<ConstantNode>(index)});
|
||||
}
|
||||
|
||||
void replaceWithSumCount(QueryTreeNodePtr & node, const FunctionNodePtr & sum_count_node, ContextPtr context)
|
||||
@ -115,20 +118,20 @@ void replaceWithSumCount(QueryTreeNodePtr & node, const FunctionNodePtr & sum_co
|
||||
if (function_name == "sum")
|
||||
{
|
||||
assert(node->getResultType()->equals(*sum_count_result_type->getElement(0)));
|
||||
node = createTupleElementFunction(context, node->getResultType(), sum_count_node, 1);
|
||||
node = createTupleElementFunction(context, sum_count_node, 1);
|
||||
}
|
||||
else if (function_name == "count")
|
||||
{
|
||||
assert(node->getResultType()->equals(*sum_count_result_type->getElement(1)));
|
||||
node = createTupleElementFunction(context, node->getResultType(), sum_count_node, 2);
|
||||
node = createTupleElementFunction(context, sum_count_node, 2);
|
||||
}
|
||||
else if (function_name == "avg")
|
||||
{
|
||||
auto sum_result = createTupleElementFunction(context, sum_count_result_type->getElement(0), sum_count_node, 1);
|
||||
auto count_result = createTupleElementFunction(context, sum_count_result_type->getElement(1), sum_count_node, 2);
|
||||
auto sum_result = createTupleElementFunction(context, sum_count_node, 1);
|
||||
auto count_result = createTupleElementFunction(context, sum_count_node, 2);
|
||||
/// To avoid integer division by zero
|
||||
auto count_float_result = createResolvedFunction(context, "toFloat64", std::make_shared<DataTypeFloat64>(), {count_result});
|
||||
node = createResolvedFunction(context, "divide", node->getResultType(), {sum_result, count_float_result});
|
||||
auto count_float_result = createResolvedFunction(context, "toFloat64", {count_result});
|
||||
node = createResolvedFunction(context, "divide", {sum_result, count_float_result});
|
||||
}
|
||||
else
|
||||
{
|
||||
@ -238,7 +241,7 @@ void tryFuseQuantiles(QueryTreeNodePtr query_tree_node, ContextPtr context)
|
||||
for (size_t i = 0; i < nodes_set.size(); ++i)
|
||||
{
|
||||
size_t array_index = i + 1;
|
||||
*nodes[i] = createArrayElementFunction(context, result_array_type->getNestedType(), quantiles_node, array_index);
|
||||
*nodes[i] = createArrayElementFunction(context, quantiles_node, array_index);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -55,8 +55,8 @@ public:
|
||||
return;
|
||||
|
||||
auto multi_if_function = std::make_shared<FunctionNode>("multiIf");
|
||||
multi_if_function->resolveAsFunction(multi_if_function_ptr, std::make_shared<DataTypeUInt8>());
|
||||
multi_if_function->getArguments().getNodes() = std::move(multi_if_arguments);
|
||||
multi_if_function->resolveAsFunction(multi_if_function_ptr->build(multi_if_function->getArgumentTypes()));
|
||||
node = std::move(multi_if_function);
|
||||
}
|
||||
|
||||
|
||||
@ -47,49 +47,64 @@ QueryTreeNodePtr createCastFunction(QueryTreeNodePtr from, DataTypePtr result_ty
|
||||
auto enum_literal_node = std::make_shared<ConstantNode>(std::move(enum_literal));
|
||||
|
||||
auto cast_function = FunctionFactory::instance().get("_CAST", std::move(context));
|
||||
QueryTreeNodes arguments{std::move(from), std::move(enum_literal_node)};
|
||||
QueryTreeNodes arguments{ std::move(from), std::move(enum_literal_node) };
|
||||
|
||||
auto function_node = std::make_shared<FunctionNode>("_CAST");
|
||||
function_node->resolveAsFunction(std::move(cast_function), std::move(result_type));
|
||||
function_node->getArguments().getNodes() = std::move(arguments);
|
||||
|
||||
function_node->resolveAsFunction(cast_function->build(function_node->getArgumentTypes()));
|
||||
|
||||
return function_node;
|
||||
}
|
||||
|
||||
/// if(arg1, arg2, arg3) will be transformed to if(arg1, _CAST(arg2, Enum...), _CAST(arg3, Enum...))
|
||||
/// where Enum is generated based on the possible values stored in string_values
|
||||
void changeIfArguments(
|
||||
QueryTreeNodePtr & first, QueryTreeNodePtr & second, const std::set<std::string> & string_values, const ContextPtr & context)
|
||||
FunctionNode & if_node, const std::set<std::string> & string_values, const ContextPtr & context)
|
||||
{
|
||||
auto result_type = getEnumType(string_values);
|
||||
|
||||
first = createCastFunction(first, result_type, context);
|
||||
second = createCastFunction(second, result_type, context);
|
||||
auto & argument_nodes = if_node.getArguments().getNodes();
|
||||
|
||||
argument_nodes[1] = createCastFunction(argument_nodes[1], result_type, context);
|
||||
argument_nodes[2] = createCastFunction(argument_nodes[2], result_type, context);
|
||||
|
||||
auto if_resolver = FunctionFactory::instance().get("if", context);
|
||||
|
||||
if_node.resolveAsFunction(if_resolver->build(if_node.getArgumentTypes()));
|
||||
}
|
||||
|
||||
/// transform(value, array_from, array_to, default_value) will be transformed to transform(value, array_from, _CAST(array_to, Array(Enum...)), _CAST(default_value, Enum...))
|
||||
/// where Enum is generated based on the possible values stored in string_values
|
||||
void changeTransformArguments(
|
||||
QueryTreeNodePtr & array_to,
|
||||
QueryTreeNodePtr & default_value,
|
||||
FunctionNode & transform_node,
|
||||
const std::set<std::string> & string_values,
|
||||
const ContextPtr & context)
|
||||
{
|
||||
auto result_type = getEnumType(string_values);
|
||||
|
||||
auto & arguments = transform_node.getArguments().getNodes();
|
||||
|
||||
auto & array_to = arguments[2];
|
||||
auto & default_value = arguments[3];
|
||||
|
||||
array_to = createCastFunction(array_to, std::make_shared<DataTypeArray>(result_type), context);
|
||||
default_value = createCastFunction(default_value, std::move(result_type), context);
|
||||
|
||||
auto transform_resolver = FunctionFactory::instance().get("transform", context);
|
||||
|
||||
transform_node.resolveAsFunction(transform_resolver->build(transform_node.getArgumentTypes()));
|
||||
}
|
||||
|
||||
void wrapIntoToString(FunctionNode & function_node, QueryTreeNodePtr arg, ContextPtr context)
|
||||
{
|
||||
assert(isString(function_node.getResultType()));
|
||||
|
||||
auto to_string_function = FunctionFactory::instance().get("toString", std::move(context));
|
||||
QueryTreeNodes arguments{std::move(arg)};
|
||||
|
||||
function_node.resolveAsFunction(std::move(to_string_function), std::make_shared<DataTypeString>());
|
||||
QueryTreeNodes arguments{ std::move(arg) };
|
||||
function_node.getArguments().getNodes() = std::move(arguments);
|
||||
|
||||
function_node.resolveAsFunction(to_string_function->build(function_node.getArgumentTypes()));
|
||||
|
||||
assert(isString(function_node.getResultType()));
|
||||
}
|
||||
|
||||
class ConvertStringsToEnumVisitor : public InDepthQueryTreeVisitor<ConvertStringsToEnumVisitor>
|
||||
@ -117,7 +132,8 @@ public:
|
||||
return;
|
||||
|
||||
auto modified_if_node = function_node->clone();
|
||||
auto & argument_nodes = modified_if_node->as<FunctionNode>()->getArguments().getNodes();
|
||||
auto * function_if_node = modified_if_node->as<FunctionNode>();
|
||||
auto & argument_nodes = function_if_node->getArguments().getNodes();
|
||||
|
||||
const auto * first_literal = argument_nodes[1]->as<ConstantNode>();
|
||||
const auto * second_literal = argument_nodes[2]->as<ConstantNode>();
|
||||
@ -132,7 +148,7 @@ public:
|
||||
string_values.insert(first_literal->getValue().get<std::string>());
|
||||
string_values.insert(second_literal->getValue().get<std::string>());
|
||||
|
||||
changeIfArguments(argument_nodes[1], argument_nodes[2], string_values, context);
|
||||
changeIfArguments(*function_if_node, string_values, context);
|
||||
wrapIntoToString(*function_node, std::move(modified_if_node), context);
|
||||
return;
|
||||
}
|
||||
@ -143,7 +159,8 @@ public:
|
||||
return;
|
||||
|
||||
auto modified_transform_node = function_node->clone();
|
||||
auto & argument_nodes = modified_transform_node->as<FunctionNode>()->getArguments().getNodes();
|
||||
auto * function_modified_transform_node = modified_transform_node->as<FunctionNode>();
|
||||
auto & argument_nodes = function_modified_transform_node->getArguments().getNodes();
|
||||
|
||||
if (!isString(function_node->getResultType()))
|
||||
return;
|
||||
@ -176,7 +193,7 @@ public:
|
||||
|
||||
string_values.insert(literal_default->getValue().get<std::string>());
|
||||
|
||||
changeTransformArguments(argument_nodes[2], argument_nodes[3], string_values, context);
|
||||
changeTransformArguments(*function_modified_transform_node, string_values, context);
|
||||
wrapIntoToString(*function_node, std::move(modified_transform_node), context);
|
||||
return;
|
||||
}
|
||||
|
||||
@ -27,7 +27,7 @@ public:
|
||||
return;
|
||||
|
||||
auto result_type = function_node->getResultType();
|
||||
function_node->resolveAsFunction(if_function_ptr, std::move(result_type));
|
||||
function_node->resolveAsFunction(if_function_ptr->build(function_node->getArgumentTypes()));
|
||||
}
|
||||
|
||||
private:
|
||||
|
||||
@ -53,12 +53,10 @@ private:
|
||||
|
||||
static inline void resolveAsCountAggregateFunction(FunctionNode & function_node)
|
||||
{
|
||||
auto function_result_type = function_node.getResultType();
|
||||
|
||||
AggregateFunctionProperties properties;
|
||||
auto aggregate_function = AggregateFunctionFactory::instance().get("count", {}, {}, properties);
|
||||
|
||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function), std::move(function_result_type));
|
||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function));
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@ -4302,7 +4302,7 @@ ProjectionNames QueryAnalyzer::resolveFunction(QueryTreeNodePtr & node, Identifi
|
||||
bool force_grouping_standard_compatibility = scope.context->getSettingsRef().force_grouping_standard_compatibility;
|
||||
auto grouping_function = std::make_shared<FunctionGrouping>(force_grouping_standard_compatibility);
|
||||
auto grouping_function_adaptor = std::make_shared<FunctionToOverloadResolverAdaptor>(std::move(grouping_function));
|
||||
function_node.resolveAsFunction(std::move(grouping_function_adaptor), std::make_shared<DataTypeUInt64>());
|
||||
function_node.resolveAsFunction(grouping_function_adaptor->build({}));
|
||||
return result_projection_names;
|
||||
}
|
||||
}
|
||||
@ -4327,7 +4327,7 @@ ProjectionNames QueryAnalyzer::resolveFunction(QueryTreeNodePtr & node, Identifi
|
||||
AggregateFunctionProperties properties;
|
||||
auto aggregate_function = AggregateFunctionFactory::instance().get(function_name, argument_types, parameters, properties);
|
||||
|
||||
function_node.resolveAsWindowFunction(aggregate_function, aggregate_function->getReturnType());
|
||||
function_node.resolveAsWindowFunction(aggregate_function);
|
||||
|
||||
bool window_node_is_identifier = function_node.getWindowNode()->getNodeType() == QueryTreeNodeType::IDENTIFIER;
|
||||
ProjectionName window_projection_name = resolveWindow(function_node.getWindowNode(), scope);
|
||||
@ -4386,7 +4386,7 @@ ProjectionNames QueryAnalyzer::resolveFunction(QueryTreeNodePtr & node, Identifi
|
||||
|
||||
AggregateFunctionProperties properties;
|
||||
auto aggregate_function = AggregateFunctionFactory::instance().get(function_name, argument_types, parameters, properties);
|
||||
function_node.resolveAsAggregateFunction(aggregate_function, aggregate_function->getReturnType());
|
||||
function_node.resolveAsAggregateFunction(aggregate_function);
|
||||
return result_projection_names;
|
||||
}
|
||||
|
||||
@ -4563,6 +4563,8 @@ ProjectionNames QueryAnalyzer::resolveFunction(QueryTreeNodePtr & node, Identifi
|
||||
constant_value = std::make_shared<ConstantValue>(std::move(column_constant_value), result_type);
|
||||
}
|
||||
}
|
||||
|
||||
function_node.resolveAsFunction(std::move(function_base));
|
||||
}
|
||||
catch (Exception & e)
|
||||
{
|
||||
@ -4570,8 +4572,6 @@ ProjectionNames QueryAnalyzer::resolveFunction(QueryTreeNodePtr & node, Identifi
|
||||
throw;
|
||||
}
|
||||
|
||||
function_node.resolveAsFunction(std::move(function), std::move(result_type));
|
||||
|
||||
if (constant_value)
|
||||
node = std::make_shared<ConstantNode>(std::move(constant_value), node);
|
||||
|
||||
|
||||
@ -117,11 +117,12 @@ public:
|
||||
not_function_result_type = makeNullable(not_function_result_type);
|
||||
|
||||
auto not_function = std::make_shared<FunctionNode>("not");
|
||||
not_function->resolveAsFunction(FunctionFactory::instance().get("not", context), std::move(not_function_result_type));
|
||||
|
||||
auto & not_function_arguments = not_function->getArguments().getNodes();
|
||||
not_function_arguments.push_back(std::move(nested_if_function_arguments_nodes[0]));
|
||||
|
||||
not_function->resolveAsFunction(FunctionFactory::instance().get("not", context)->build(not_function->getArgumentTypes()));
|
||||
|
||||
function_node_arguments_nodes[0] = std::move(not_function);
|
||||
function_node_arguments_nodes.resize(1);
|
||||
|
||||
@ -139,8 +140,7 @@ private:
|
||||
function_node.getAggregateFunction()->getParameters(),
|
||||
properties);
|
||||
|
||||
auto function_result_type = function_node.getResultType();
|
||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function), std::move(function_result_type));
|
||||
function_node.resolveAsAggregateFunction(std::move(aggregate_function));
|
||||
}
|
||||
|
||||
ContextPtr & context;
|
||||
|
||||
@ -76,7 +76,7 @@ public:
|
||||
properties);
|
||||
|
||||
auto function_result_type = function_node->getResultType();
|
||||
function_node->resolveAsAggregateFunction(std::move(aggregate_function), std::move(function_result_type));
|
||||
function_node->resolveAsAggregateFunction(std::move(aggregate_function));
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@ -21,6 +21,7 @@
|
||||
|
||||
#include <Interpreters/Context.h>
|
||||
#include <Analyzer/ColumnNode.h>
|
||||
#include <Analyzer/FunctionNode.h>
|
||||
#include <Analyzer/InDepthQueryTreeVisitor.h>
|
||||
#include <Common/Exception.h>
|
||||
|
||||
@ -44,6 +45,23 @@ namespace
|
||||
class ValidationChecker : public InDepthQueryTreeVisitor<ValidationChecker>
|
||||
{
|
||||
String pass_name;
|
||||
|
||||
void visitColumn(ColumnNode * column) const
|
||||
{
|
||||
if (column->getColumnSourceOrNull() == nullptr)
|
||||
throw Exception(ErrorCodes::LOGICAL_ERROR,
|
||||
"Column {} {} query tree node does not have valid source node after running {} pass",
|
||||
column->getColumnName(), column->getColumnType(), pass_name);
|
||||
}
|
||||
|
||||
void visitFunction(FunctionNode * function) const
|
||||
{
|
||||
if (!function->isResolved())
|
||||
throw Exception(ErrorCodes::LOGICAL_ERROR,
|
||||
"Function {} is not resolved after running {} pass",
|
||||
function->dumpTree(), pass_name);
|
||||
}
|
||||
|
||||
public:
|
||||
explicit ValidationChecker(String pass_name_)
|
||||
: pass_name(std::move(pass_name_))
|
||||
@ -51,13 +69,10 @@ public:
|
||||
|
||||
void visitImpl(QueryTreeNodePtr & node) const
|
||||
{
|
||||
auto * column = node->as<ColumnNode>();
|
||||
if (!column)
|
||||
return;
|
||||
if (column->getColumnSourceOrNull() == nullptr)
|
||||
throw Exception(ErrorCodes::LOGICAL_ERROR,
|
||||
"Column {} {} query tree node does not have valid source node after running {} pass",
|
||||
column->getColumnName(), column->getColumnType(), pass_name);
|
||||
if (auto * column = node->as<ColumnNode>())
|
||||
return visitColumn(column);
|
||||
else if (auto * function = node->as<FunctionNode>())
|
||||
return visitFunction(function);
|
||||
}
|
||||
};
|
||||
#endif
|
||||
|
||||
@ -91,7 +91,8 @@ bool SortNode::isEqualImpl(const IQueryTreeNode & rhs) const
|
||||
void SortNode::updateTreeHashImpl(HashState & hash_state) const
|
||||
{
|
||||
hash_state.update(sort_direction);
|
||||
hash_state.update(nulls_sort_direction);
|
||||
/// use some determined value if `nulls_sort_direction` is `nullopt`
|
||||
hash_state.update(nulls_sort_direction.value_or(sort_direction));
|
||||
hash_state.update(with_fill);
|
||||
|
||||
if (collator)
|
||||
|
||||
@ -146,7 +146,7 @@ MutableColumnPtr ColumnAggregateFunction::convertToValues(MutableColumnPtr colum
|
||||
/// insertResultInto may invalidate states, so we must unshare ownership of them
|
||||
column_aggregate_func.ensureOwnership();
|
||||
|
||||
MutableColumnPtr res = func->getReturnType()->createColumn();
|
||||
MutableColumnPtr res = func->getResultType()->createColumn();
|
||||
res->reserve(data.size());
|
||||
|
||||
/// If there are references to states in final column, we must hold their ownership
|
||||
|
||||
@ -13,7 +13,7 @@ namespace ErrorCodes
|
||||
}
|
||||
|
||||
class IFunctionBase;
|
||||
using FunctionBasePtr = std::shared_ptr<IFunctionBase>;
|
||||
using FunctionBasePtr = std::shared_ptr<const IFunctionBase>;
|
||||
|
||||
/** A column containing a lambda expression.
|
||||
* Behaves like a constant-column. Contains an expression, but not input or output data.
|
||||
|
||||
76
src/Common/BinStringDecodeHelper.h
Normal file
76
src/Common/BinStringDecodeHelper.h
Normal file
@ -0,0 +1,76 @@
|
||||
#pragma once
|
||||
|
||||
#include <Common/hex.h>
|
||||
|
||||
namespace DB
|
||||
{
|
||||
|
||||
static void inline hexStringDecode(const char * pos, const char * end, char *& out, size_t word_size = 2)
|
||||
{
|
||||
if ((end - pos) & 1)
|
||||
{
|
||||
*out = unhex(*pos);
|
||||
++out;
|
||||
++pos;
|
||||
}
|
||||
while (pos < end)
|
||||
{
|
||||
*out = unhex2(pos);
|
||||
pos += word_size;
|
||||
++out;
|
||||
}
|
||||
*out = '\0';
|
||||
++out;
|
||||
}
|
||||
|
||||
static void inline binStringDecode(const char * pos, const char * end, char *& out)
|
||||
{
|
||||
if (pos == end)
|
||||
{
|
||||
*out = '\0';
|
||||
++out;
|
||||
return;
|
||||
}
|
||||
|
||||
UInt8 left = 0;
|
||||
|
||||
/// end - pos is the length of input.
|
||||
/// (length & 7) to make remain bits length mod 8 is zero to split.
|
||||
/// e.g. the length is 9 and the input is "101000001",
|
||||
/// first left_cnt is 1, left is 0, right shift, pos is 1, left = 1
|
||||
/// then, left_cnt is 0, remain input is '01000001'.
|
||||
for (UInt8 left_cnt = (end - pos) & 7; left_cnt > 0; --left_cnt)
|
||||
{
|
||||
left = left << 1;
|
||||
if (*pos != '0')
|
||||
left += 1;
|
||||
++pos;
|
||||
}
|
||||
|
||||
if (left != 0 || end - pos == 0)
|
||||
{
|
||||
*out = left;
|
||||
++out;
|
||||
}
|
||||
|
||||
assert((end - pos) % 8 == 0);
|
||||
|
||||
while (end - pos != 0)
|
||||
{
|
||||
UInt8 c = 0;
|
||||
for (UInt8 i = 0; i < 8; ++i)
|
||||
{
|
||||
c = c << 1;
|
||||
if (*pos != '0')
|
||||
c += 1;
|
||||
++pos;
|
||||
}
|
||||
*out = c;
|
||||
++out;
|
||||
}
|
||||
|
||||
*out = '\0';
|
||||
++out;
|
||||
}
|
||||
|
||||
}
|
||||
@ -240,24 +240,52 @@ TEST(Common, RWLockPerfTestReaders)
|
||||
|
||||
for (auto pool_size : pool_sizes)
|
||||
{
|
||||
Stopwatch watch(CLOCK_MONOTONIC_COARSE);
|
||||
Stopwatch watch(CLOCK_MONOTONIC_COARSE);
|
||||
|
||||
auto func = [&] ()
|
||||
auto func = [&] ()
|
||||
{
|
||||
for (auto i = 0; i < cycles; ++i)
|
||||
{
|
||||
for (auto i = 0; i < cycles; ++i)
|
||||
{
|
||||
auto lock = fifo_lock->getLock(RWLockImpl::Read, RWLockImpl::NO_QUERY);
|
||||
}
|
||||
};
|
||||
auto lock = fifo_lock->getLock(RWLockImpl::Read, RWLockImpl::NO_QUERY);
|
||||
}
|
||||
};
|
||||
|
||||
std::list<std::thread> threads;
|
||||
for (size_t thread = 0; thread < pool_size; ++thread)
|
||||
threads.emplace_back(func);
|
||||
std::list<std::thread> threads;
|
||||
for (size_t thread = 0; thread < pool_size; ++thread)
|
||||
threads.emplace_back(func);
|
||||
|
||||
for (auto & thread : threads)
|
||||
thread.join();
|
||||
for (auto & thread : threads)
|
||||
thread.join();
|
||||
|
||||
auto total_time = watch.elapsedSeconds();
|
||||
std::cout << "Threads " << pool_size << ", total_time " << std::setprecision(2) << total_time << "\n";
|
||||
auto total_time = watch.elapsedSeconds();
|
||||
std::cout << "Threads " << pool_size << ", total_time " << std::setprecision(2) << total_time << "\n";
|
||||
}
|
||||
}
|
||||
|
||||
TEST(Common, RWLockNotUpgradeableWithNoQuery)
|
||||
{
|
||||
updatePHDRCache();
|
||||
|
||||
static auto rw_lock = RWLockImpl::create();
|
||||
|
||||
std::thread read_thread([&] ()
|
||||
{
|
||||
auto lock = rw_lock->getLock(RWLockImpl::Read, RWLockImpl::NO_QUERY, std::chrono::duration<int, std::milli>(50000));
|
||||
auto sleep_for = std::chrono::duration<int, std::milli>(5000);
|
||||
std::this_thread::sleep_for(sleep_for);
|
||||
});
|
||||
|
||||
{
|
||||
auto sleep_for = std::chrono::duration<int, std::milli>(500);
|
||||
std::this_thread::sleep_for(sleep_for);
|
||||
|
||||
Stopwatch watch(CLOCK_MONOTONIC_COARSE);
|
||||
auto get_lock = rw_lock->getLock(RWLockImpl::Write, RWLockImpl::NO_QUERY, std::chrono::duration<int, std::milli>(50000));
|
||||
|
||||
EXPECT_NE(get_lock.get(), nullptr);
|
||||
/// It took some time
|
||||
EXPECT_GT(watch.elapsedMilliseconds(), 3000);
|
||||
}
|
||||
|
||||
read_thread.join();
|
||||
}
|
||||
|
||||
29
src/Core/IResolvedFunction.h
Normal file
29
src/Core/IResolvedFunction.h
Normal file
@ -0,0 +1,29 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <vector>
|
||||
|
||||
namespace DB
|
||||
{
|
||||
class IDataType;
|
||||
|
||||
using DataTypePtr = std::shared_ptr<const IDataType>;
|
||||
using DataTypes = std::vector<DataTypePtr>;
|
||||
|
||||
struct Array;
|
||||
|
||||
class IResolvedFunction
|
||||
{
|
||||
public:
|
||||
virtual const DataTypePtr & getResultType() const = 0;
|
||||
|
||||
virtual const DataTypes & getArgumentTypes() const = 0;
|
||||
|
||||
virtual const Array & getParameters() const = 0;
|
||||
|
||||
virtual ~IResolvedFunction() = default;
|
||||
};
|
||||
|
||||
using IResolvedFunctionPtr = std::shared_ptr<const IResolvedFunction>;
|
||||
|
||||
}
|
||||
@ -620,6 +620,7 @@ static constexpr UInt64 operator""_GiB(unsigned long long value)
|
||||
M(Bool, enable_filesystem_cache_on_lower_level, true, "If read buffer supports caching inside threadpool, allow it to do it, otherwise cache outside ot threadpool. Do not use this setting, it is needed for testing", 0) \
|
||||
M(Bool, skip_download_if_exceeds_query_cache, true, "Skip download from remote filesystem if exceeds query cache size", 0) \
|
||||
M(UInt64, max_query_cache_size, (128UL * 1024 * 1024 * 1024), "Max remote filesystem cache size that can be used by a single query", 0) \
|
||||
M(Bool, throw_on_error_from_cache_on_write_operations, false, "Ignore error from cache when caching on write operations (INSERT, merges)", 0) \
|
||||
\
|
||||
M(Bool, load_marks_asynchronously, false, "Load MergeTree marks asynchronously", 0) \
|
||||
\
|
||||
|
||||
@ -1025,9 +1025,6 @@ void BaseDaemon::setupWatchdog()
|
||||
#if defined(OS_LINUX)
|
||||
if (0 != prctl(PR_SET_PDEATHSIG, SIGKILL))
|
||||
logger().warning("Cannot do prctl to ask termination with parent.");
|
||||
|
||||
if (getppid() == 1)
|
||||
throw Poco::Exception("Parent watchdog process has exited.");
|
||||
#endif
|
||||
|
||||
{
|
||||
|
||||
@ -30,9 +30,9 @@ private:
|
||||
public:
|
||||
static constexpr bool is_parametric = true;
|
||||
|
||||
DataTypeAggregateFunction(const AggregateFunctionPtr & function_, const DataTypes & argument_types_,
|
||||
DataTypeAggregateFunction(AggregateFunctionPtr function_, const DataTypes & argument_types_,
|
||||
const Array & parameters_, std::optional<size_t> version_ = std::nullopt)
|
||||
: function(function_)
|
||||
: function(std::move(function_))
|
||||
, argument_types(argument_types_)
|
||||
, parameters(parameters_)
|
||||
, version(version_)
|
||||
@ -51,7 +51,7 @@ public:
|
||||
|
||||
bool canBeInsideNullable() const override { return false; }
|
||||
|
||||
DataTypePtr getReturnType() const { return function->getReturnType(); }
|
||||
DataTypePtr getReturnType() const { return function->getResultType(); }
|
||||
DataTypePtr getReturnTypeToPredict() const { return function->getReturnTypeToPredict(); }
|
||||
DataTypes getArgumentsDataTypes() const { return argument_types; }
|
||||
|
||||
|
||||
@ -131,9 +131,9 @@ static std::pair<DataTypePtr, DataTypeCustomDescPtr> create(const ASTPtr & argum
|
||||
|
||||
DataTypePtr storage_type = DataTypeFactory::instance().get(argument_types[0]->getName());
|
||||
|
||||
if (!function->getReturnType()->equals(*removeLowCardinality(storage_type)))
|
||||
if (!function->getResultType()->equals(*removeLowCardinality(storage_type)))
|
||||
{
|
||||
throw Exception("Incompatible data types between aggregate function '" + function->getName() + "' which returns " + function->getReturnType()->getName() + " and column storage type " + storage_type->getName(),
|
||||
throw Exception("Incompatible data types between aggregate function '" + function->getName() + "' which returns " + function->getResultType()->getName() + " and column storage type " + storage_type->getName(),
|
||||
ErrorCodes::BAD_ARGUMENTS);
|
||||
}
|
||||
|
||||
|
||||
@ -256,6 +256,9 @@ void DatabaseOrdinary::startupTables(ThreadPool & thread_pool, LoadingStrictness
|
||||
|
||||
auto startup_one_table = [&](const StoragePtr & table)
|
||||
{
|
||||
/// Since startup() method can use physical paths on disk we don't allow any exclusive actions (rename, drop so on)
|
||||
/// until startup finished.
|
||||
auto table_lock_holder = table->lockForShare(RWLockImpl::NO_QUERY, getContext()->getSettingsRef().lock_acquire_timeout);
|
||||
table->startup();
|
||||
logAboutProgress(log, ++tables_processed, total_tables, watch);
|
||||
};
|
||||
|
||||
@ -44,10 +44,10 @@ FileSegmentRangeWriter::FileSegmentRangeWriter(
|
||||
const String & source_path_)
|
||||
: cache(cache_)
|
||||
, key(key_)
|
||||
, log(&Poco::Logger::get("FileSegmentRangeWriter"))
|
||||
, cache_log(cache_log_)
|
||||
, query_id(query_id_)
|
||||
, source_path(source_path_)
|
||||
, current_file_segment_it(file_segments_holder.file_segments.end())
|
||||
{
|
||||
}
|
||||
|
||||
@ -56,69 +56,68 @@ bool FileSegmentRangeWriter::write(const char * data, size_t size, size_t offset
|
||||
if (finalized)
|
||||
return false;
|
||||
|
||||
if (expected_write_offset != offset)
|
||||
{
|
||||
throw Exception(
|
||||
ErrorCodes::LOGICAL_ERROR,
|
||||
"Cannot write file segment at offset {}, because expected write offset is: {}",
|
||||
offset, expected_write_offset);
|
||||
}
|
||||
|
||||
auto & file_segments = file_segments_holder.file_segments;
|
||||
|
||||
if (current_file_segment_it == file_segments.end())
|
||||
if (file_segments.empty() || file_segments.back()->isDownloaded())
|
||||
{
|
||||
current_file_segment_it = allocateFileSegment(current_file_segment_write_offset, is_persistent);
|
||||
}
|
||||
else
|
||||
{
|
||||
auto file_segment = *current_file_segment_it;
|
||||
assert(file_segment->getCurrentWriteOffset() == current_file_segment_write_offset);
|
||||
|
||||
if (current_file_segment_write_offset != offset)
|
||||
{
|
||||
throw Exception(
|
||||
ErrorCodes::LOGICAL_ERROR,
|
||||
"Cannot write file segment at offset {}, because current write offset is: {}",
|
||||
offset, current_file_segment_write_offset);
|
||||
}
|
||||
|
||||
if (file_segment->range().size() == file_segment->getDownloadedSize())
|
||||
{
|
||||
completeFileSegment(*file_segment);
|
||||
current_file_segment_it = allocateFileSegment(current_file_segment_write_offset, is_persistent);
|
||||
}
|
||||
allocateFileSegment(expected_write_offset, is_persistent);
|
||||
}
|
||||
|
||||
auto & file_segment = *current_file_segment_it;
|
||||
|
||||
auto downloader = file_segment->getOrSetDownloader();
|
||||
if (downloader != FileSegment::getCallerId())
|
||||
throw Exception(ErrorCodes::LOGICAL_ERROR, "Failed to set a downloader. ({})", file_segment->getInfoForLog());
|
||||
auto & file_segment = file_segments.back();
|
||||
|
||||
SCOPE_EXIT({
|
||||
if (file_segment->isDownloader())
|
||||
file_segment->completePartAndResetDownloader();
|
||||
if (file_segments.back()->isDownloader())
|
||||
file_segments.back()->completePartAndResetDownloader();
|
||||
});
|
||||
|
||||
bool reserved = file_segment->reserve(size);
|
||||
if (!reserved)
|
||||
while (size > 0)
|
||||
{
|
||||
file_segment->completeWithState(FileSegment::State::PARTIALLY_DOWNLOADED_NO_CONTINUATION);
|
||||
appendFilesystemCacheLog(*file_segment);
|
||||
size_t available_size = file_segment->range().size() - file_segment->getDownloadedSize();
|
||||
if (available_size == 0)
|
||||
{
|
||||
completeFileSegment(*file_segment);
|
||||
file_segment = allocateFileSegment(expected_write_offset, is_persistent);
|
||||
continue;
|
||||
}
|
||||
|
||||
LOG_DEBUG(
|
||||
&Poco::Logger::get("FileSegmentRangeWriter"),
|
||||
"Unsuccessful space reservation attempt (size: {}, file segment info: {}",
|
||||
size, file_segment->getInfoForLog());
|
||||
if (!file_segment->isDownloader()
|
||||
&& file_segment->getOrSetDownloader() != FileSegment::getCallerId())
|
||||
{
|
||||
throw Exception(ErrorCodes::LOGICAL_ERROR,
|
||||
"Failed to set a downloader. ({})", file_segment->getInfoForLog());
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
size_t size_to_write = std::min(available_size, size);
|
||||
|
||||
try
|
||||
{
|
||||
file_segment->write(data, size, offset);
|
||||
}
|
||||
catch (...)
|
||||
{
|
||||
bool reserved = file_segment->reserve(size_to_write);
|
||||
if (!reserved)
|
||||
{
|
||||
file_segment->completeWithState(FileSegment::State::PARTIALLY_DOWNLOADED_NO_CONTINUATION);
|
||||
appendFilesystemCacheLog(*file_segment);
|
||||
|
||||
LOG_DEBUG(
|
||||
log, "Failed to reserve space in cache (size: {}, file segment info: {}",
|
||||
size, file_segment->getInfoForLog());
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
file_segment->write(data, size_to_write, offset);
|
||||
file_segment->completePartAndResetDownloader();
|
||||
throw;
|
||||
}
|
||||
|
||||
file_segment->completePartAndResetDownloader();
|
||||
current_file_segment_write_offset += size;
|
||||
size -= size_to_write;
|
||||
expected_write_offset += size_to_write;
|
||||
offset += size_to_write;
|
||||
data += size_to_write;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
@ -129,10 +128,10 @@ void FileSegmentRangeWriter::finalize()
|
||||
return;
|
||||
|
||||
auto & file_segments = file_segments_holder.file_segments;
|
||||
if (file_segments.empty() || current_file_segment_it == file_segments.end())
|
||||
if (file_segments.empty())
|
||||
return;
|
||||
|
||||
completeFileSegment(**current_file_segment_it);
|
||||
completeFileSegment(*file_segments.back());
|
||||
finalized = true;
|
||||
}
|
||||
|
||||
@ -149,7 +148,7 @@ FileSegmentRangeWriter::~FileSegmentRangeWriter()
|
||||
}
|
||||
}
|
||||
|
||||
FileSegments::iterator FileSegmentRangeWriter::allocateFileSegment(size_t offset, bool is_persistent)
|
||||
FileSegmentPtr & FileSegmentRangeWriter::allocateFileSegment(size_t offset, bool is_persistent)
|
||||
{
|
||||
/**
|
||||
* Allocate a new file segment starting `offset`.
|
||||
@ -168,7 +167,8 @@ FileSegments::iterator FileSegmentRangeWriter::allocateFileSegment(size_t offset
|
||||
auto file_segment = cache->createFileSegmentForDownload(
|
||||
key, offset, cache->max_file_segment_size, create_settings, cache_lock);
|
||||
|
||||
return file_segments_holder.add(std::move(file_segment));
|
||||
auto & file_segments = file_segments_holder.file_segments;
|
||||
return *file_segments.insert(file_segments.end(), file_segment);
|
||||
}
|
||||
|
||||
void FileSegmentRangeWriter::appendFilesystemCacheLog(const FileSegment & file_segment)
|
||||
@ -199,7 +199,7 @@ void FileSegmentRangeWriter::appendFilesystemCacheLog(const FileSegment & file_s
|
||||
void FileSegmentRangeWriter::completeFileSegment(FileSegment & file_segment)
|
||||
{
|
||||
/// File segment can be detached if space reservation failed.
|
||||
if (file_segment.isDetached())
|
||||
if (file_segment.isDetached() || file_segment.isCompleted())
|
||||
return;
|
||||
|
||||
file_segment.completeWithoutState();
|
||||
@ -223,6 +223,7 @@ CachedOnDiskWriteBufferFromFile::CachedOnDiskWriteBufferFromFile(
|
||||
, is_persistent_cache_file(is_persistent_cache_file_)
|
||||
, query_id(query_id_)
|
||||
, enable_cache_log(!query_id_.empty() && settings_.enable_filesystem_cache_log)
|
||||
, throw_on_error_from_cache(settings_.throw_on_error_from_cache)
|
||||
{
|
||||
}
|
||||
|
||||
@ -246,11 +247,11 @@ void CachedOnDiskWriteBufferFromFile::nextImpl()
|
||||
}
|
||||
|
||||
/// Write data to cache.
|
||||
cacheData(working_buffer.begin(), size);
|
||||
cacheData(working_buffer.begin(), size, throw_on_error_from_cache);
|
||||
current_download_offset += size;
|
||||
}
|
||||
|
||||
void CachedOnDiskWriteBufferFromFile::cacheData(char * data, size_t size)
|
||||
void CachedOnDiskWriteBufferFromFile::cacheData(char * data, size_t size, bool throw_on_error)
|
||||
{
|
||||
if (cache_in_error_state_or_disabled)
|
||||
return;
|
||||
@ -285,11 +286,17 @@ void CachedOnDiskWriteBufferFromFile::cacheData(char * data, size_t size)
|
||||
return;
|
||||
}
|
||||
|
||||
if (throw_on_error)
|
||||
throw;
|
||||
|
||||
tryLogCurrentException(__PRETTY_FUNCTION__);
|
||||
return;
|
||||
}
|
||||
catch (...)
|
||||
{
|
||||
if (throw_on_error)
|
||||
throw;
|
||||
|
||||
tryLogCurrentException(__PRETTY_FUNCTION__);
|
||||
return;
|
||||
}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user