diff --git a/.github/workflows/release_branches.yml b/.github/workflows/release_branches.yml
index a5cd6321e8c..82826794ea3 100644
--- a/.github/workflows/release_branches.yml
+++ b/.github/workflows/release_branches.yml
@@ -482,7 +482,7 @@ jobs:
if: ${{ !failure() }}
run: |
# update overall ci report
- python3 finish_check.py --wf-status ${{ contains(needs.*.result, 'failure') && 'failure' || 'success' }}
+ python3 ./tests/ci/finish_check.py --wf-status ${{ contains(needs.*.result, 'failure') && 'failure' || 'success' }}
- name: Check Workflow results
if: ${{ !cancelled() }}
run: |
@@ -490,5 +490,4 @@ jobs:
cat > "$WORKFLOW_RESULT_FILE" << 'EOF'
${{ toJson(needs) }}
EOF
-
python3 ./tests/ci/ci_buddy.py --check-wf-status
diff --git a/.gitmodules b/.gitmodules
index cdee6a43ad8..53ebde0cd3b 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -108,7 +108,7 @@
url = https://github.com/ClickHouse/icudata
[submodule "contrib/icu"]
path = contrib/icu
- url = https://github.com/unicode-org/icu
+ url = https://github.com/ClickHouse/icu
[submodule "contrib/flatbuffers"]
path = contrib/flatbuffers
url = https://github.com/ClickHouse/flatbuffers
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1793fd14ccd..93dcfa18999 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,4 +1,5 @@
### Table of Contents
+**[ClickHouse release v24.8 LTS, 2024-08-20](#243)**
**[ClickHouse release v24.7, 2024-07-30](#247)**
**[ClickHouse release v24.6, 2024-07-01](#246)**
**[ClickHouse release v24.5, 2024-05-30](#245)**
@@ -10,6 +11,153 @@
# 2024 Changelog
+### ClickHouse release 24.8 LTS, 2024-08-20
+
+#### Backward Incompatible Change
+* `clickhouse-client` and `clickhouse-local` now default to multi-query mode (instead single-query mode). As an example, `clickhouse-client -q "SELECT 1; SELECT 2"` now works, whereas users previously had to add `--multiquery` (or `-n`). The `--multiquery/-n` switch became obsolete. INSERT queries in multi-query statements are treated specially based on their FORMAT clause: If the FORMAT is `VALUES` (the most common case), the end of the INSERT statement is represented by a trailing semicolon `;` at the end of the query. For all other FORMATs (e.g. `CSV` or `JSONEachRow`), the end of the INSERT statement is represented by two newlines `\n\n` at the end of the query. [#63898](https://github.com/ClickHouse/ClickHouse/pull/63898) ([FFish](https://github.com/wxybear)).
+* In previous versions, it was possible to use an alternative syntax for `LowCardinality` data types by appending `WithDictionary` to the name of the data type. It was an initial working implementation, and it was never documented or exposed to the public. Now, it is deprecated. If you have used this syntax, you have to ALTER your tables and rename the data types to `LowCardinality`. [#66842](https://github.com/ClickHouse/ClickHouse/pull/66842) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+* Fix logical errors with storage `Buffer` used with distributed destination table. It's a backward incompatible change: queries using `Buffer` with a distributed destination table may stop working if the table appears more than once in the query (e.g., in a self-join). [#67015](https://github.com/ClickHouse/ClickHouse/pull/67015) ([vdimir](https://github.com/vdimir)).
+* In previous versions, calling functions for random distributions based on the Gamma function (such as Chi-Squared, Student, Fisher) with negative arguments close to zero led to a long computation or an infinite loop. In the new version, calling these functions with zero or negative arguments will produce an exception. This closes [#67297](https://github.com/ClickHouse/ClickHouse/issues/67297). [#67326](https://github.com/ClickHouse/ClickHouse/pull/67326) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+* The system table `text_log` is enabled by default. This is fully compatible with previous versions, but you may notice subtly increased disk usage on the local disk (this system table takes a tiny amount of disk space). [#67428](https://github.com/ClickHouse/ClickHouse/pull/67428) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+* In previous versions, `arrayWithConstant` can be slow if asked to generate very large arrays. In the new version, it is limited to 1 GB per array. This closes [#32754](https://github.com/ClickHouse/ClickHouse/issues/32754). [#67741](https://github.com/ClickHouse/ClickHouse/pull/67741) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+* Fix REPLACE modifier formatting (forbid omitting brackets). [#67774](https://github.com/ClickHouse/ClickHouse/pull/67774) ([Azat Khuzhin](https://github.com/azat)).
+* Backported in [#68349](https://github.com/ClickHouse/ClickHouse/issues/68349): Reimplement `Dynamic` type. Now when the limit of dynamic data types is reached new types are not casted to String but stored in a special data structure in binary format with binary encoded data type. Now any type ever inserted into `Dynamic` column can be read from it as subcolumn. [#68132](https://github.com/ClickHouse/ClickHouse/pull/68132) ([Kruglov Pavel](https://github.com/Avogar)).
+
+#### New Feature
+* Added a new `MergeTree` setting `deduplicate_merge_projection_mode` to control the projections during merges (for specific engines) and `OPTIMIZE DEDUPLICATE` query. Supported options: `throw` (throw an exception in case the projection is not fully supported for *MergeTree engine), `drop` (remove projection during merge if it can't be merged itself consistently) and `rebuild` (rebuild projection from scratch, which is a heavy operation). [#66672](https://github.com/ClickHouse/ClickHouse/pull/66672) ([jsc0218](https://github.com/jsc0218)).
+* Add `_etag` virtual column for S3 table engine. Fixes [#65312](https://github.com/ClickHouse/ClickHouse/issues/65312). [#65386](https://github.com/ClickHouse/ClickHouse/pull/65386) ([skyoct](https://github.com/skyoct)).
+* Added a tagging (namespace) mechanism for the query cache. The same queries with different tags are considered different by the query cache. Example: `SELECT 1 SETTINGS use_query_cache = 1, query_cache_tag = 'abc'` and `SELECT 1 SETTINGS use_query_cache = 1, query_cache_tag = 'def'` now create different query cache entries. [#68235](https://github.com/ClickHouse/ClickHouse/pull/68235) ([sakulali](https://github.com/sakulali)).
+* Support more variants of JOIN strictness (`LEFT/RIGHT SEMI/ANTI/ANY JOIN`) with inequality conditions which involve columns from both left and right table. e.g. `t1.y < t2.y` (see the setting `allow_experimental_join_condition`). [#64281](https://github.com/ClickHouse/ClickHouse/pull/64281) ([lgbo](https://github.com/lgbo-ustc)).
+* Intrpret Hive-style partitioning for different engines (`File`, `URL`, `S3`, `AzureBlobStorage`, `HDFS`). Hive-style partitioning organizes data into partitioned sub-directories, making it efficient to query and manage large datasets. Currently, it only creates virtual columns with the appropriate name and data. The follow-up PR will introduce the appropriate data filtering (performance speedup). [#65997](https://github.com/ClickHouse/ClickHouse/pull/65997) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
+* Add function `printf` for Spark compatiability (but you can use the existing `format` function). [#66257](https://github.com/ClickHouse/ClickHouse/pull/66257) ([李扬](https://github.com/taiyang-li)).
+* Added a new server setting, `disable_insertion_and_mutation`. If it is enabled, the server will deny all insertions and mutations. This includes asynchronous INSERTs. This setting can be used to create read-only replicas. [#66519](https://github.com/ClickHouse/ClickHouse/pull/66519) ([Xu Jia](https://github.com/XuJia0210)).
+* Add options `restore_replace_external_engines_to_null` and `restore_replace_external_table_functions_to_null` to replace external engines and table_engines to `Null` engine that can be useful for testing. It should work for RESTORE and explicit table creation. [#66536](https://github.com/ClickHouse/ClickHouse/pull/66536) ([Ilya Yatsishin](https://github.com/qoega)).
+* Added support for reading `MULTILINESTRING` geometry in `WKT` format using function `readWKTLineString`. [#67647](https://github.com/ClickHouse/ClickHouse/pull/67647) ([Jacob Reckhard](https://github.com/jacobrec)).
+* Add a new table function `fuzzQuery`. This function allows the modification of a given query string with random variations. Example: `SELECT query FROM fuzzQuery('SELECT 1') LIMIT 5;`. [#67655](https://github.com/ClickHouse/ClickHouse/pull/67655) ([pufit](https://github.com/pufit)).
+* Add a query `ALTER TABLE ... DROP DETACHED PARTITION ALL` to drop all detached partitions. [#67885](https://github.com/ClickHouse/ClickHouse/pull/67885) ([Duc Canh Le](https://github.com/canhld94)).
+* Add the `rows_before_aggregation_at_least` statistic to the query response when a new setting, `rows_before_aggregation` is enabled. This statistic represents the number of rows read before aggregation. In the context of a distributed query, when using the `group by` or `max` aggregation function without a `limit`, `rows_before_aggregation_at_least` can reflect the number of rows hit by the query. [#66084](https://github.com/ClickHouse/ClickHouse/pull/66084) ([morning-color](https://github.com/morning-color)).
+* Support `OPTIMIZE` query on `Join` tables to reduce their memory footprint. [#67883](https://github.com/ClickHouse/ClickHouse/pull/67883) ([Duc Canh Le](https://github.com/canhld94)).
+* Allow run query instantly in play if you add `&run=1` in the URL [#66457](https://github.com/ClickHouse/ClickHouse/pull/66457) ([Aleksandr Musorin](https://github.com/AVMusorin)).
+
+#### Experimental Feature
+* Implement a new `JSON` data type. [#66444](https://github.com/ClickHouse/ClickHouse/pull/66444) ([Kruglov Pavel](https://github.com/Avogar)).
+* Add the new `TimeSeries` table engine. [#64183](https://github.com/ClickHouse/ClickHouse/pull/64183) ([Vitaly Baranov](https://github.com/vitlibar)).
+* Add new experimental `Kafka` storage engine to store offsets in Keeper instead of relying on committing them to Kafka. It makes the commit to ClickHouse tables atomic with regard to consumption from the queue. [#57625](https://github.com/ClickHouse/ClickHouse/pull/57625) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)).
+* Use adaptive read task size calculation method (adaptive meaning it depends on read column sizes) for parallel replicas. [#60377](https://github.com/ClickHouse/ClickHouse/pull/60377) ([Nikita Taranov](https://github.com/nickitat)).
+* Added statistics type `count_min` (count-min sketches) which provide selectivity estimations for equality predicates like `col = 'val'`. Supported data types are string, date, datatime and numeric types. [#65521](https://github.com/ClickHouse/ClickHouse/pull/65521) ([JackyWoo](https://github.com/JackyWoo)).
+
+#### Performance Improvement
+* Setting `optimize_functions_to_subcolumns` is enabled by default. [#68053](https://github.com/ClickHouse/ClickHouse/pull/68053) ([Anton Popov](https://github.com/CurtizJ)).
+* Store the `plain_rewritable` disk directory metadata in `__meta` layout, separately from the merge tree data in the object storage. Move the `plain_rewritable` disk to a flat directory structure. [#65751](https://github.com/ClickHouse/ClickHouse/pull/65751) ([Julia Kartseva](https://github.com/jkartseva)).
+* Improve columns squashing (an operation happening in INSERT queries) for `String`/`Array`/`Map`/`Variant`/`Dynamic` types by reserving required memory in advance for all subcolumns. [#67043](https://github.com/ClickHouse/ClickHouse/pull/67043) ([Kruglov Pavel](https://github.com/Avogar)).
+* Speed up `SYSTEM FLUSH LOGS` and flush logs on shutdown. [#67472](https://github.com/ClickHouse/ClickHouse/pull/67472) ([Sema Checherinda](https://github.com/CheSema)).
+* Improved overall performance of merges by reducing the overhead of the scheduling steps of merges. [#68016](https://github.com/ClickHouse/ClickHouse/pull/68016) ([Anton Popov](https://github.com/CurtizJ)).
+* Speed up tables removal for `DROP DATABASE` query, increased the default value for `database_catalog_drop_table_concurrency` to 16. [#67228](https://github.com/ClickHouse/ClickHouse/pull/67228) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)).
+* Avoid allocating too much capacity for array column while writing ORC. Performance speeds up 15% for an Array column. [#67879](https://github.com/ClickHouse/ClickHouse/pull/67879) ([李扬](https://github.com/taiyang-li)).
+* Speed up mutations for non-replicated MergeTree significantly [#66911](https://github.com/ClickHouse/ClickHouse/pull/66911) [#66909](https://github.com/ClickHouse/ClickHouse/pull/66909) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+
+#### Improvement
+* Setting `allow_experimental_analyzer` is renamed to `enable_analyzer`. The old name is preserved in a form of an alias. This signifies that Analyzer is no longer in beta and is fully promoted to production. [#66438](https://github.com/ClickHouse/ClickHouse/pull/66438) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)).
+* Improve schema inference of date times. Now DateTime64 used only when date time has fractional part, otherwise regular DateTime is used. Inference of Date/DateTime is more strict now, especially when `date_time_input_format='best_effort'` to avoid inferring date times from strings in corner cases. [#68382](https://github.com/ClickHouse/ClickHouse/pull/68382) ([Kruglov Pavel](https://github.com/Avogar)).
+* ClickHouse server now supports new setting `max_keep_alive_requests`. For keep-alive HTTP connections to the server it works in tandem with `keep_alive_timeout` - if idle timeout not expired but there already more than `max_keep_alive_requests` requests done through the given connection - it will be closed by the server. [#61793](https://github.com/ClickHouse/ClickHouse/pull/61793) ([Nikita Taranov](https://github.com/nickitat)).
+* Various improvements in the advanced dashboard. This closes [#67697](https://github.com/ClickHouse/ClickHouse/issues/67697). This closes [#63407](https://github.com/ClickHouse/ClickHouse/issues/63407). This closes [#51129](https://github.com/ClickHouse/ClickHouse/issues/51129). This closes [#61204](https://github.com/ClickHouse/ClickHouse/issues/61204). [#67701](https://github.com/ClickHouse/ClickHouse/pull/67701) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+* Do not require a grant for REMOTE when creating a Distributed table: a grant for the Distributed engine is enough. [#65419](https://github.com/ClickHouse/ClickHouse/pull/65419) ([jsc0218](https://github.com/jsc0218)).
+* Do not pass logs for keeper explicitly in the Docker image to allow overriding. [#65564](https://github.com/ClickHouse/ClickHouse/pull/65564) ([Azat Khuzhin](https://github.com/azat)).
+* Introduced `use_same_password_for_base_backup` settings for `BACKUP` and `RESTORE` queries, allowing to create and restore incremental backups to/from password protected archives. [#66214](https://github.com/ClickHouse/ClickHouse/pull/66214) ([Samuele](https://github.com/sguerrini97)).
+* Ignore `async_load_databases` for `ATTACH` query (previously it was possible for ATTACH to return before the tables had been attached). [#66240](https://github.com/ClickHouse/ClickHouse/pull/66240) ([Azat Khuzhin](https://github.com/azat)).
+* Added logs and metrics for rejected connections (where there are not enough resources). [#66410](https://github.com/ClickHouse/ClickHouse/pull/66410) ([Alexander Tokmakov](https://github.com/tavplubix)).
+* Support proper `UUID` type for MongoDB engine. [#66671](https://github.com/ClickHouse/ClickHouse/pull/66671) ([Azat Khuzhin](https://github.com/azat)).
+* Add replication lag and recovery time metrics. [#66703](https://github.com/ClickHouse/ClickHouse/pull/66703) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)).
+* Add `DiskS3NoSuchKeyErrors` metric. [#66704](https://github.com/ClickHouse/ClickHouse/pull/66704) ([Miсhael Stetsyuk](https://github.com/mstetsyuk)).
+* Ensure the `COMMENT` clause works for all table engines. [#66832](https://github.com/ClickHouse/ClickHouse/pull/66832) ([Joe Lynch](https://github.com/joelynch)).
+* Function `mapFromArrays` now accepts `Map(K, V)` as first argument, for example: `SELECT mapFromArrays(map('a', 4, 'b', 4), ['aa', 'bb'])` now works and returns `{('a',4):'aa',('b',4):'bb'}`. Also, if the 1st argument is an Array, it can now also be of type `Array(Nullable(T))` or `Array(LowCardinality(Nullable(T)))` as long as the actual array values are not `NULL`. [#67103](https://github.com/ClickHouse/ClickHouse/pull/67103) ([李扬](https://github.com/taiyang-li)).
+* Read configuration for `clickhouse-local` from `~/.clickhouse-local`. [#67135](https://github.com/ClickHouse/ClickHouse/pull/67135) ([Azat Khuzhin](https://github.com/azat)).
+* Rename setting `input_format_orc_read_use_writer_time_zone` to `input_format_orc_reader_timezone` and allow the user to set the reader timezone. [#67175](https://github.com/ClickHouse/ClickHouse/pull/67175) ([kevinyhzou](https://github.com/KevinyhZou)).
+* Decrease level of the `Socket is not connected` error when HTTP connection immediately reset by peer after connecting, close [#34218](https://github.com/ClickHouse/ClickHouse/issues/34218). [#67177](https://github.com/ClickHouse/ClickHouse/pull/67177) ([vdimir](https://github.com/vdimir)).
+* Add ability to load dashboards for `system.dashboards` from config (once set, they overrides the default dashboards preset). [#67232](https://github.com/ClickHouse/ClickHouse/pull/67232) ([Azat Khuzhin](https://github.com/azat)).
+* The window functions in SQL are traditionally in snake case. ClickHouse uses `camelCase`, so new aliases `denseRank()` and `percentRank()` have been created. These new functions can be called the exact same as the original `dense_rank()` and `percent_rank()` functions. Both snake case and camelCase syntaxes remain usable. A new test for each of the functions has been added as well. This closes [#67042](https://github.com/ClickHouse/ClickHouse/issues/67042) . [#67334](https://github.com/ClickHouse/ClickHouse/pull/67334) ([Peter Nguyen](https://github.com/petern48)).
+* Autodetect configuration file format if is not `.xml`, `.yml` or `.yaml`. If the file begins with < it might be XML, otherwise it might be YAML. It is useful when providing a configuration file from a pipe: `clickhouse-server --config-file <(echo "hello: world")`. [#67391](https://github.com/ClickHouse/ClickHouse/pull/67391) ([sakulali](https://github.com/sakulali)).
+* Functions `formatDateTime` and `formatDateTimeInJodaSyntax` now treat their format parameter as optional. If it is not specified, format strings `%Y-%m-%d %H:%i:%s` and `yyyy-MM-dd HH:mm:ss` are assumed. Example: `SELECT parseDateTime('2021-01-04 23:12:34')` now returns DateTime value `2021-01-04 23:12:34` (previously, this threw an exception). [#67399](https://github.com/ClickHouse/ClickHouse/pull/67399) ([Robert Schulze](https://github.com/rschu1ze)).
+* Automatically retry Keeper requests in KeeperMap if they happen because of timeout or connection loss. [#67448](https://github.com/ClickHouse/ClickHouse/pull/67448) ([Antonio Andelic](https://github.com/antonio2368)).
+* Add `-no-pie` to Aarch64 Linux builds to allow proper introspection and symbolizing of stacktraces after a ClickHouse restart. [#67916](https://github.com/ClickHouse/ClickHouse/pull/67916) ([filimonov](https://github.com/filimonov)).
+* Added profile events for merges and mutations for better introspection. [#68015](https://github.com/ClickHouse/ClickHouse/pull/68015) ([Anton Popov](https://github.com/CurtizJ)).
+* Fix settings and `current_database` in `system.processes` for async BACKUP/RESTORE. [#68163](https://github.com/ClickHouse/ClickHouse/pull/68163) ([Azat Khuzhin](https://github.com/azat)).
+* Remove unnecessary logs for non-replicated `MergeTree`. [#68238](https://github.com/ClickHouse/ClickHouse/pull/68238) ([Daniil Ivanik](https://github.com/divanik)).
+
+#### Build/Testing/Packaging Improvement
+* Integration tests flaky check will not run each test case multiple times to find more issues in tests and make them more reliable. It is using `pytest-repeat` library to run test case multiple times for the same environment. It is important to cleanup tables and other entities in the end of a test case to pass. Repeating works much faster than several pytest runs as it starts necessary containers only once. [#66986](https://github.com/ClickHouse/ClickHouse/pull/66986) ([Ilya Yatsishin](https://github.com/qoega)).
+* Unblock the usage of CLion with ClickHouse. In previous versions, CLion freezed for a minute on every keypress. This closes [#66994](https://github.com/ClickHouse/ClickHouse/issues/66994). [#66995](https://github.com/ClickHouse/ClickHouse/pull/66995) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+* getauxval: avoid a crash under a sanitizer re-exec due to high ASLR entropy in newer Linux kernels. [#67081](https://github.com/ClickHouse/ClickHouse/pull/67081) ([Raúl Marín](https://github.com/Algunenano)).
+* Some parts of client code are extracted to a single file and highest possible level optimization is applied to them even for debug builds. This closes: [#65745](https://github.com/ClickHouse/ClickHouse/issues/65745). [#67215](https://github.com/ClickHouse/ClickHouse/pull/67215) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)).
+
+#### Bug Fix
+* Only relevant to the experimental Variant data type. Fix crash with Variant + AggregateFunction type. [#67122](https://github.com/ClickHouse/ClickHouse/pull/67122) ([Kruglov Pavel](https://github.com/Avogar)).
+* Fix crash in DistributedAsyncInsert when connection is empty. [#67219](https://github.com/ClickHouse/ClickHouse/pull/67219) ([Pablo Marcos](https://github.com/pamarcos)).
+* Fix crash of `uniq` and `uniqTheta ` with `tuple()` argument. Closes [#67303](https://github.com/ClickHouse/ClickHouse/issues/67303). [#67306](https://github.com/ClickHouse/ClickHouse/pull/67306) ([flynn](https://github.com/ucasfl)).
+* Fixes [#66026](https://github.com/ClickHouse/ClickHouse/issues/66026). Avoid unresolved table function arguments traversal in `ReplaceTableNodeToDummyVisitor`. [#67522](https://github.com/ClickHouse/ClickHouse/pull/67522) ([Dmitry Novik](https://github.com/novikd)).
+* Fix potential stack overflow in `JSONMergePatch` function. Renamed this function from `jsonMergePatch` to `JSONMergePatch` because the previous name was wrong. The previous name is still kept for compatibility. Improved diagnostic of errors in the function. This closes [#67304](https://github.com/ClickHouse/ClickHouse/issues/67304). [#67756](https://github.com/ClickHouse/ClickHouse/pull/67756) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+* Fixed a NULL pointer dereference, triggered by a specially crafted query, that crashed the server via hopEnd, hopStart, tumbleEnd, and tumbleStart. [#68098](https://github.com/ClickHouse/ClickHouse/pull/68098) ([Salvatore Mesoraca](https://github.com/aiven-sal)).
+* Fixed `Not-ready Set` in some system tables when filtering using subqueries. [#66018](https://github.com/ClickHouse/ClickHouse/pull/66018) ([Michael Kolupaev](https://github.com/al13n321)).
+* Fixed reading of subcolumns after `ALTER ADD COLUMN` query. [#66243](https://github.com/ClickHouse/ClickHouse/pull/66243) ([Anton Popov](https://github.com/CurtizJ)).
+* Fix boolean literals in query sent to external database (for engines like `PostgreSQL`). [#66282](https://github.com/ClickHouse/ClickHouse/pull/66282) ([vdimir](https://github.com/vdimir)).
+* Fix formatting of query with aliased JOIN ON expression, e.g. `... JOIN t2 ON (x = y) AS e ORDER BY x` should be formatted as `... JOIN t2 ON ((x = y) AS e) ORDER BY x`. [#66312](https://github.com/ClickHouse/ClickHouse/pull/66312) ([vdimir](https://github.com/vdimir)).
+* Fix cluster() for inter-server secret (preserve initial user as before). [#66364](https://github.com/ClickHouse/ClickHouse/pull/66364) ([Azat Khuzhin](https://github.com/azat)).
+* Fix possible runtime error while converting Array field with nulls to Array(Variant). [#66727](https://github.com/ClickHouse/ClickHouse/pull/66727) ([Kruglov Pavel](https://github.com/Avogar)).
+* Fix for occasional deadlock in Context::getDDLWorker. [#66843](https://github.com/ClickHouse/ClickHouse/pull/66843) ([Alexander Gololobov](https://github.com/davenger)).
+* Fix creating KeeperMap table after an incomplete drop. [#66865](https://github.com/ClickHouse/ClickHouse/pull/66865) ([Antonio Andelic](https://github.com/antonio2368)).
+* Fix broken part error while restoring to a `s3_plain_rewritable` disk. [#66881](https://github.com/ClickHouse/ClickHouse/pull/66881) ([Vitaly Baranov](https://github.com/vitlibar)).
+* In rare cases ClickHouse could consider parts as broken because of some unexpected projections on disk. Now it's fixed. [#66898](https://github.com/ClickHouse/ClickHouse/pull/66898) ([alesapin](https://github.com/alesapin)).
+* Fix invalid format detection in schema inference that could lead to logical error Format {} doesn't support schema inference. [#66899](https://github.com/ClickHouse/ClickHouse/pull/66899) ([Kruglov Pavel](https://github.com/Avogar)).
+* Fix possible deadlock on query cancel with parallel replicas. [#66905](https://github.com/ClickHouse/ClickHouse/pull/66905) ([Nikita Taranov](https://github.com/nickitat)).
+* Forbid create as select even when database_replicated_allow_heavy_create is set. It was unconditionally forbidden in 23.12 and accidentally allowed under the setting in unreleased 24.7. [#66980](https://github.com/ClickHouse/ClickHouse/pull/66980) ([vdimir](https://github.com/vdimir)).
+* Reading from the `numbers` could wrongly throw an exception when the `max_rows_to_read` limit was set. This closes [#66992](https://github.com/ClickHouse/ClickHouse/issues/66992). [#66996](https://github.com/ClickHouse/ClickHouse/pull/66996) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+* Add proper type conversion to lagInFrame and leadInFrame window functions - fixes msan test. [#67091](https://github.com/ClickHouse/ClickHouse/pull/67091) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
+* TRUNCATE DATABASE used to stop replication as if it was a DROP DATABASE query, it's fixed. [#67129](https://github.com/ClickHouse/ClickHouse/pull/67129) ([Alexander Tokmakov](https://github.com/tavplubix)).
+* Use a separate client context in `clickhouse-local`. [#67133](https://github.com/ClickHouse/ClickHouse/pull/67133) ([Vitaly Baranov](https://github.com/vitlibar)).
+* Fix error `Cannot convert column because it is non constant in source stream but must be constant in result.` for a query that reads from the `Merge` table over the `Distriburted` table with one shard. [#67146](https://github.com/ClickHouse/ClickHouse/pull/67146) ([Nikolai Kochetov](https://github.com/KochetovNicolai)).
+* Correct behavior of `ORDER BY all` with disabled `enable_order_by_all` and parallel replicas (distributed queries as well). [#67153](https://github.com/ClickHouse/ClickHouse/pull/67153) ([Igor Nikonov](https://github.com/devcrafter)).
+* Fix wrong usage of input_format_max_bytes_to_read_for_schema_inference in schema cache. [#67157](https://github.com/ClickHouse/ClickHouse/pull/67157) ([Kruglov Pavel](https://github.com/Avogar)).
+* Fix the memory leak for count distinct, when exception issued during group by single nullable key. [#67171](https://github.com/ClickHouse/ClickHouse/pull/67171) ([Jet He](https://github.com/compasses)).
+* Fix an error in optimization which converts OUTER JOIN to INNER JOIN. This closes [#67156](https://github.com/ClickHouse/ClickHouse/issues/67156). This closes [#66447](https://github.com/ClickHouse/ClickHouse/issues/66447). The bug was introduced in https://github.com/ClickHouse/ClickHouse/pull/62907. [#67178](https://github.com/ClickHouse/ClickHouse/pull/67178) ([Maksim Kita](https://github.com/kitaisreal)).
+* Fix error `Conversion from AggregateFunction(name, Type) to AggregateFunction(name, Nullable(Type)) is not supported`. The bug was caused by the `optimize_rewrite_aggregate_function_with_if` optimization. Fixes [#67112](https://github.com/ClickHouse/ClickHouse/issues/67112). [#67229](https://github.com/ClickHouse/ClickHouse/pull/67229) ([Nikolai Kochetov](https://github.com/KochetovNicolai)).
+* Fix hung query when using empty tuple as lhs of function IN. [#67295](https://github.com/ClickHouse/ClickHouse/pull/67295) ([Duc Canh Le](https://github.com/canhld94)).
+* It was possible to create a very deep nested JSON data that triggered stack overflow while skipping unknown fields. This closes [#67292](https://github.com/ClickHouse/ClickHouse/issues/67292). [#67324](https://github.com/ClickHouse/ClickHouse/pull/67324) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+* Fix attaching ReplicatedMergeTree table after exception during startup. [#67360](https://github.com/ClickHouse/ClickHouse/pull/67360) ([Antonio Andelic](https://github.com/antonio2368)).
+* Fix segfault caused by incorrectly detaching from thread group in `Aggregator`. [#67385](https://github.com/ClickHouse/ClickHouse/pull/67385) ([Antonio Andelic](https://github.com/antonio2368)).
+* Fix one more case when a non-deterministic function is specified in PK. [#67395](https://github.com/ClickHouse/ClickHouse/pull/67395) ([Nikolai Kochetov](https://github.com/KochetovNicolai)).
+* Fixed `bloom_filter` index breaking queries with mildly weird conditions like `(k=2)=(k=2)` or `has([1,2,3], k)`. [#67423](https://github.com/ClickHouse/ClickHouse/pull/67423) ([Michael Kolupaev](https://github.com/al13n321)).
+* Correctly parse file name/URI containing `::` if it's not an archive. [#67433](https://github.com/ClickHouse/ClickHouse/pull/67433) ([Antonio Andelic](https://github.com/antonio2368)).
+* Fix wait for tasks in ~WriteBufferFromS3 in case WriteBuffer was cancelled. [#67459](https://github.com/ClickHouse/ClickHouse/pull/67459) ([Kseniia Sumarokova](https://github.com/kssenii)).
+* Protect temporary part directories from removing during RESTORE. [#67491](https://github.com/ClickHouse/ClickHouse/pull/67491) ([Vitaly Baranov](https://github.com/vitlibar)).
+* Fix execution of nested short-circuit functions. [#67520](https://github.com/ClickHouse/ClickHouse/pull/67520) ([Kruglov Pavel](https://github.com/Avogar)).
+* Fix `Logical error: Expected the argument №N of type T to have X rows, but it has 0`. The error could happen in a remote query with constant expression in `GROUP BY` (with a new analyzer). [#67536](https://github.com/ClickHouse/ClickHouse/pull/67536) ([Nikolai Kochetov](https://github.com/KochetovNicolai)).
+* Fix join on tuple with NULLs: Some queries with the new analyzer and `NULL` inside the tuple in the `JOIN ON` section returned incorrect results. [#67538](https://github.com/ClickHouse/ClickHouse/pull/67538) ([vdimir](https://github.com/vdimir)).
+* Fix redundant reschedule of FileCache::freeSpaceRatioKeepingThreadFunc() in case of full non-evictable cache. [#67540](https://github.com/ClickHouse/ClickHouse/pull/67540) ([Kseniia Sumarokova](https://github.com/kssenii)).
+* Fix inserting into stream like engines (Kafka, RabbitMQ, NATS) through HTTP interface. [#67554](https://github.com/ClickHouse/ClickHouse/pull/67554) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)).
+* Fix for function `toStartOfWeek` which returned the wrong result with a small `DateTime64` value. [#67558](https://github.com/ClickHouse/ClickHouse/pull/67558) ([Yarik Briukhovetskyi](https://github.com/yariks5s)).
+* Fix creation of view with recursive CTE. [#67587](https://github.com/ClickHouse/ClickHouse/pull/67587) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
+* Fix `Logical error: 'file_offset_of_buffer_end <= read_until_position'` in filesystem cache. Closes [#57508](https://github.com/ClickHouse/ClickHouse/issues/57508). [#67623](https://github.com/ClickHouse/ClickHouse/pull/67623) ([Kseniia Sumarokova](https://github.com/kssenii)).
+* Fixes [#62282](https://github.com/ClickHouse/ClickHouse/issues/62282). Removed the call to `convertFieldToString()` and added datatype specific serialization code. Parameterized view substitution was broken for multiple datatypes when parameter value was a function or expression returning datatype instance. [#67654](https://github.com/ClickHouse/ClickHouse/pull/67654) ([Shankar](https://github.com/shiyer7474)).
+* Fix crash on `percent_rank`. `percent_rank`'s default frame type is changed to `range unbounded preceding and unbounded following`. `IWindowFunction`'s default window frame is considered and now window functions without window frame definition in sql can be put into different `WindowTransfomer`s properly. [#67661](https://github.com/ClickHouse/ClickHouse/pull/67661) ([lgbo](https://github.com/lgbo-ustc)).
+* Fix reloading SQL UDFs with UNION. Previously, restarting the server could make UDF invalid. [#67665](https://github.com/ClickHouse/ClickHouse/pull/67665) ([Antonio Andelic](https://github.com/antonio2368)).
+* Fix possible logical error "Unexpected return type from if" with experimental Variant type and enabled setting `use_variant_as_common_type ` in function if with Tuples and Maps. [#67687](https://github.com/ClickHouse/ClickHouse/pull/67687) ([Kruglov Pavel](https://github.com/Avogar)).
+* Due to a bug in Linux Kernel, a query can hung in `TimerDescriptor::drain`. This closes [#37686](https://github.com/ClickHouse/ClickHouse/issues/37686). [#67702](https://github.com/ClickHouse/ClickHouse/pull/67702) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
+* Fix completion of `RESTORE ON CLUSTER` command. [#67720](https://github.com/ClickHouse/ClickHouse/pull/67720) ([Vitaly Baranov](https://github.com/vitlibar)).
+* Fix dictionary hang in case of CANNOT_SCHEDULE_TASK while loading. [#67751](https://github.com/ClickHouse/ClickHouse/pull/67751) ([Azat Khuzhin](https://github.com/azat)).
+* Queries like `SELECT count() FROM t WHERE cast(c = 1 or c = 9999 AS Bool) SETTINGS use_skip_indexes=1` with bloom filter indexes on `c` now work correctly. [#67781](https://github.com/ClickHouse/ClickHouse/pull/67781) ([jsc0218](https://github.com/jsc0218)).
+* Fix wrong aggregation result in some queries with aggregation without keys and filter, close [#67419](https://github.com/ClickHouse/ClickHouse/issues/67419). [#67804](https://github.com/ClickHouse/ClickHouse/pull/67804) ([vdimir](https://github.com/vdimir)).
+* Validate experimental/suspicious data types in ALTER ADD/MODIFY COLUMN. [#67911](https://github.com/ClickHouse/ClickHouse/pull/67911) ([Kruglov Pavel](https://github.com/Avogar)).
+* Fix DateTime64 parsing after constant folding in distributed queries, close [#66773](https://github.com/ClickHouse/ClickHouse/issues/66773). [#67920](https://github.com/ClickHouse/ClickHouse/pull/67920) ([vdimir](https://github.com/vdimir)).
+* Fix wrong `count()` result when there is non-deterministic function in predicate. [#67922](https://github.com/ClickHouse/ClickHouse/pull/67922) ([János Benjamin Antal](https://github.com/antaljanosbenjamin)).
+* Fixed the calculation of the maximum thread soft limit in containerized environments where the usable CPU count is limited. [#67963](https://github.com/ClickHouse/ClickHouse/pull/67963) ([Robert Schulze](https://github.com/rschu1ze)).
+* Now ClickHouse doesn't consider part as broken if projection doesn't exist on disk but exists in `checksums.txt`. [#68003](https://github.com/ClickHouse/ClickHouse/pull/68003) ([alesapin](https://github.com/alesapin)).
+* Fixed skipping of untouched parts in mutations with new analyzer. Previously with enabled analyzer data in part could be rewritten by mutation even if mutation doesn't affect this part according to predicate. [#68052](https://github.com/ClickHouse/ClickHouse/pull/68052) ([Anton Popov](https://github.com/CurtizJ)).
+* Removes an incorrect optimization to remove sorting in subqueries that use `OFFSET`. Fixes [#67906](https://github.com/ClickHouse/ClickHouse/issues/67906). [#68099](https://github.com/ClickHouse/ClickHouse/pull/68099) ([Graham Campbell](https://github.com/GrahamCampbell)).
+* Attempt to fix `Block structure mismatch in AggregatingStep stream: different types` for aggregate projection optimization. [#68107](https://github.com/ClickHouse/ClickHouse/pull/68107) ([Nikolai Kochetov](https://github.com/KochetovNicolai)).
+* Try fix postgres crash when query is cancelled. [#68288](https://github.com/ClickHouse/ClickHouse/pull/68288) ([Kseniia Sumarokova](https://github.com/kssenii)).
+* Fix missing sync replica mode in query `SYSTEM SYNC REPLICA`. [#68326](https://github.com/ClickHouse/ClickHouse/pull/68326) ([Duc Canh Le](https://github.com/canhld94)).
+
+
### ClickHouse release 24.7, 2024-07-30
#### Backward Incompatible Change
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 6fa91fa002f..6abf48a6927 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -605,7 +605,9 @@ if (NATIVE_BUILD_TARGETS
execute_process(
COMMAND ${CMAKE_COMMAND} -E make_directory "${NATIVE_BUILD_DIR}"
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
execute_process(
COMMAND ${CMAKE_COMMAND}
@@ -617,9 +619,13 @@ if (NATIVE_BUILD_TARGETS
"-DENABLE_CLICKHOUSE_SELF_EXTRACTING=${ENABLE_CLICKHOUSE_SELF_EXTRACTING}"
${PROJECT_SOURCE_DIR}
WORKING_DIRECTORY "${NATIVE_BUILD_DIR}"
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
execute_process(
COMMAND ${CMAKE_COMMAND} --build "${NATIVE_BUILD_DIR}" --target ${NATIVE_BUILD_TARGETS}
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
endif ()
diff --git a/PreLoad.cmake b/PreLoad.cmake
index e0fd37b2fd6..92b221c9f63 100644
--- a/PreLoad.cmake
+++ b/PreLoad.cmake
@@ -51,8 +51,14 @@ if (NOT "$ENV{CFLAGS}" STREQUAL ""
endif()
# Default toolchain - this is needed to avoid dependency on OS files.
-execute_process(COMMAND uname -s OUTPUT_VARIABLE OS)
-execute_process(COMMAND uname -m OUTPUT_VARIABLE ARCH)
+execute_process(COMMAND uname -s
+ OUTPUT_VARIABLE OS
+ COMMAND_ERROR_IS_FATAL ANY
+)
+execute_process(COMMAND uname -m
+ OUTPUT_VARIABLE ARCH
+ COMMAND_ERROR_IS_FATAL ANY
+)
# By default, prefer clang on Linux
# But note, that you still may change the compiler with -DCMAKE_C_COMPILER/-DCMAKE_CXX_COMPILER.
diff --git a/base/base/CMakeLists.txt b/base/base/CMakeLists.txt
index 247028b96e0..3d236f52c36 100644
--- a/base/base/CMakeLists.txt
+++ b/base/base/CMakeLists.txt
@@ -8,6 +8,8 @@ endif ()
# when instantiated from JSON.cpp. Try again when libcxx(abi) and Clang are upgraded to 16.
set (CMAKE_CXX_STANDARD 20)
+configure_file(GitHash.cpp.in GitHash.generated.cpp)
+
set (SRCS
argsToConfig.cpp
cgroupsv2.cpp
@@ -33,6 +35,7 @@ set (SRCS
safeExit.cpp
throwError.cpp
Numa.cpp
+ GitHash.generated.cpp
)
add_library (common ${SRCS})
diff --git a/src/Daemon/GitHash.cpp.in b/base/base/GitHash.cpp.in
similarity index 100%
rename from src/Daemon/GitHash.cpp.in
rename to base/base/GitHash.cpp.in
diff --git a/cmake/freebsd/default_libs.cmake b/cmake/freebsd/default_libs.cmake
index 6bde75f8c9a..3f5b3829877 100644
--- a/cmake/freebsd/default_libs.cmake
+++ b/cmake/freebsd/default_libs.cmake
@@ -9,10 +9,18 @@ endif ()
file(GLOB bprefix "/usr/local/llvm${COMPILER_VERSION_MAJOR}/lib/clang/${COMPILER_VERSION_MAJOR}/lib/${system_processor}-portbld-freebsd*/")
message(STATUS "-Bprefix: ${bprefix}")
-execute_process(COMMAND ${CMAKE_CXX_COMPILER} -Bprefix=${bprefix} --print-file-name=libclang_rt.builtins-${system_processor}.a OUTPUT_VARIABLE BUILTINS_LIBRARY OUTPUT_STRIP_TRAILING_WHITESPACE)
+execute_process(COMMAND
+ ${CMAKE_CXX_COMPILER} -Bprefix=${bprefix} --print-file-name=libclang_rt.builtins-${system_processor}.a
+ OUTPUT_VARIABLE BUILTINS_LIBRARY
+ COMMAND_ERROR_IS_FATAL ANY
+ OUTPUT_STRIP_TRAILING_WHITESPACE)
# --print-file-name simply prints what you passed in case of nothing was resolved, so let's try one other possible option
if (BUILTINS_LIBRARY STREQUAL "libclang_rt.builtins-${system_processor}.a")
- execute_process(COMMAND ${CMAKE_CXX_COMPILER} -Bprefix=${bprefix} --print-file-name=libclang_rt.builtins.a OUTPUT_VARIABLE BUILTINS_LIBRARY OUTPUT_STRIP_TRAILING_WHITESPACE)
+ execute_process(COMMAND
+ ${CMAKE_CXX_COMPILER} -Bprefix=${bprefix} --print-file-name=libclang_rt.builtins.a
+ OUTPUT_VARIABLE BUILTINS_LIBRARY
+ COMMAND_ERROR_IS_FATAL ANY
+ OUTPUT_STRIP_TRAILING_WHITESPACE)
endif()
if (BUILTINS_LIBRARY STREQUAL "libclang_rt.builtins.a")
message(FATAL_ERROR "libclang_rt.builtins had not been found")
diff --git a/cmake/limit_jobs.cmake b/cmake/limit_jobs.cmake
index 17d8dd42a2c..8e48fc9b9d8 100644
--- a/cmake/limit_jobs.cmake
+++ b/cmake/limit_jobs.cmake
@@ -42,19 +42,9 @@ endif ()
# But use 2 parallel jobs, since:
# - this is what llvm does
# - and I've verfied that lld-11 does not use all available CPU time (in peak) while linking one binary
-if (CMAKE_BUILD_TYPE_UC STREQUAL "RELWITHDEBINFO" AND ENABLE_THINLTO)
- if (ARCH_AARCH64)
- # aarch64 builds start to often fail with OOMs (reason not yet clear), for now let's limit the concurrency
- message(STATUS "ThinLTO provides its own parallel linking - limiting parallel link jobs to 1.")
- set (PARALLEL_LINK_JOBS 1)
- if (LINKER_NAME MATCHES "lld")
- math(EXPR LTO_JOBS ${NUMBER_OF_LOGICAL_CORES}/4)
- set (CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO "${CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO} -Wl,--thinlto-jobs=${LTO_JOBS}")
- endif()
- elseif (PARALLEL_LINK_JOBS GREATER 2)
- message(STATUS "ThinLTO provides its own parallel linking - limiting parallel link jobs to 2.")
- set (PARALLEL_LINK_JOBS 2)
- endif ()
+if (CMAKE_BUILD_TYPE_UC STREQUAL "RELWITHDEBINFO" AND ENABLE_THINLTO AND PARALLEL_LINK_JOBS GREATER 2)
+ message(STATUS "ThinLTO provides its own parallel linking - limiting parallel link jobs to 2.")
+ set (PARALLEL_LINK_JOBS 2)
endif()
message(STATUS "Building sub-tree with ${PARALLEL_COMPILE_JOBS} compile jobs and ${PARALLEL_LINK_JOBS} linker jobs (system: ${NUMBER_OF_LOGICAL_CORES} cores, ${TOTAL_PHYSICAL_MEMORY} MB RAM, 'OFF' means the native core count).")
diff --git a/cmake/linux/default_libs.cmake b/cmake/linux/default_libs.cmake
index 4a06243243e..51620bc9f33 100644
--- a/cmake/linux/default_libs.cmake
+++ b/cmake/linux/default_libs.cmake
@@ -5,7 +5,11 @@ set (DEFAULT_LIBS "-nodefaultlibs")
# We need builtins from Clang's RT even without libcxx - for ubsan+int128.
# See https://bugs.llvm.org/show_bug.cgi?id=16404
-execute_process (COMMAND ${CMAKE_CXX_COMPILER} --target=${CMAKE_CXX_COMPILER_TARGET} --print-libgcc-file-name --rtlib=compiler-rt OUTPUT_VARIABLE BUILTINS_LIBRARY OUTPUT_STRIP_TRAILING_WHITESPACE)
+execute_process (COMMAND
+ ${CMAKE_CXX_COMPILER} --target=${CMAKE_CXX_COMPILER_TARGET} --print-libgcc-file-name --rtlib=compiler-rt
+ OUTPUT_VARIABLE BUILTINS_LIBRARY
+ COMMAND_ERROR_IS_FATAL ANY
+ OUTPUT_STRIP_TRAILING_WHITESPACE)
# Apparently, in clang-19, the UBSan support library for C++ was moved out into ubsan_standalone_cxx.a, so we have to include both.
if (SANITIZE STREQUAL undefined)
diff --git a/cmake/tools.cmake b/cmake/tools.cmake
index 7aa5d4c51ce..5c7da54b779 100644
--- a/cmake/tools.cmake
+++ b/cmake/tools.cmake
@@ -5,7 +5,11 @@ if (NOT CMAKE_CXX_COMPILER_ID MATCHES "Clang")
endif ()
# Print details to output
-execute_process(COMMAND ${CMAKE_CXX_COMPILER} --version OUTPUT_VARIABLE COMPILER_SELF_IDENTIFICATION OUTPUT_STRIP_TRAILING_WHITESPACE)
+execute_process(COMMAND ${CMAKE_CXX_COMPILER} --version
+ OUTPUT_VARIABLE COMPILER_SELF_IDENTIFICATION
+ COMMAND_ERROR_IS_FATAL ANY
+ OUTPUT_STRIP_TRAILING_WHITESPACE
+)
message (STATUS "Using compiler:\n${COMPILER_SELF_IDENTIFICATION}")
# Require minimum compiler versions
diff --git a/cmake/utils.cmake b/cmake/utils.cmake
index a318408098a..a99d8e050a8 100644
--- a/cmake/utils.cmake
+++ b/cmake/utils.cmake
@@ -90,7 +90,10 @@ endfunction()
# Function get_cmake_properties returns list of all propreties that cmake supports
function(get_cmake_properties outvar)
- execute_process(COMMAND cmake --help-property-list OUTPUT_VARIABLE cmake_properties)
+ execute_process(COMMAND cmake --help-property-list
+ OUTPUT_VARIABLE cmake_properties
+ COMMAND_ERROR_IS_FATAL ANY
+ )
# Convert command output into a CMake list
string(REGEX REPLACE ";" "\\\\;" cmake_properties "${cmake_properties}")
string(REGEX REPLACE "\n" ";" cmake_properties "${cmake_properties}")
diff --git a/contrib/aws b/contrib/aws
index 1c2946bfcb7..d5450d76abd 160000
--- a/contrib/aws
+++ b/contrib/aws
@@ -1 +1 @@
-Subproject commit 1c2946bfcb7f1e3ae0a858de0b59d4f1a7b4ccaf
+Subproject commit d5450d76abda556ce145ddabe7e0cc6a7644ec59
diff --git a/contrib/aws-crt-cpp b/contrib/aws-crt-cpp
index f532d6abc0d..e5aa45cacfd 160000
--- a/contrib/aws-crt-cpp
+++ b/contrib/aws-crt-cpp
@@ -1 +1 @@
-Subproject commit f532d6abc0d2b0d8b5d6fe9e7c51eaedbe4afbd0
+Subproject commit e5aa45cacfdcda7719ead38760e7c61076f5745f
diff --git a/contrib/cctz-cmake/CMakeLists.txt b/contrib/cctz-cmake/CMakeLists.txt
index 7161f743de1..fadf948b053 100644
--- a/contrib/cctz-cmake/CMakeLists.txt
+++ b/contrib/cctz-cmake/CMakeLists.txt
@@ -37,7 +37,9 @@ message(STATUS "Packaging with tzdata version: ${TZDATA_VERSION}")
execute_process(COMMAND
bash -c "cd ${TZDIR} && find * -type f -and ! -name '*.tab' -and ! -name 'localtime' | LC_ALL=C sort | paste -sd ';' -"
OUTPUT_STRIP_TRAILING_WHITESPACE
- OUTPUT_VARIABLE TIMEZONES)
+ OUTPUT_VARIABLE TIMEZONES
+ COMMAND_ERROR_IS_FATAL ANY
+)
file(APPEND ${TIMEZONES_FILE} "// autogenerated by ClickHouse/contrib/cctz-cmake/CMakeLists.txt\n")
file(APPEND ${TIMEZONES_FILE} "#include \n")
diff --git a/contrib/google-protobuf-cmake/CMakeLists.txt b/contrib/google-protobuf-cmake/CMakeLists.txt
index e44f737cfc3..f1a744f851f 100644
--- a/contrib/google-protobuf-cmake/CMakeLists.txt
+++ b/contrib/google-protobuf-cmake/CMakeLists.txt
@@ -359,7 +359,9 @@ else ()
execute_process(
COMMAND mkdir -p ${PROTOC_BUILD_DIR}
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
execute_process(
COMMAND ${CMAKE_COMMAND}
@@ -375,11 +377,15 @@ else ()
"-DABSL_ENABLE_INSTALL=0"
"${protobuf_source_dir}"
WORKING_DIRECTORY "${PROTOC_BUILD_DIR}"
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
execute_process(
COMMAND ${CMAKE_COMMAND} --build "${PROTOC_BUILD_DIR}"
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
endif ()
add_executable(protoc IMPORTED GLOBAL)
diff --git a/contrib/grpc-cmake/CMakeLists.txt b/contrib/grpc-cmake/CMakeLists.txt
index 1c0bf41ff78..975774d1990 100644
--- a/contrib/grpc-cmake/CMakeLists.txt
+++ b/contrib/grpc-cmake/CMakeLists.txt
@@ -51,8 +51,9 @@ if (NOT CMAKE_HOST_SYSTEM_NAME STREQUAL CMAKE_SYSTEM_NAME
set(OPENSSL_SOURCE_DIR "${ClickHouse_SOURCE_DIR}/contrib/openssl-cmake")
execute_process(
- COMMAND mkdir -p ${OPENSSL_BUILD_DIR}
- COMMAND_ECHO STDOUT
+ COMMAND mkdir -p ${OPENSSL_BUILD_DIR}
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
)
if (CMAKE_HOST_SYSTEM_PROCESSOR MATCHES "amd64|x86_64")
@@ -89,15 +90,21 @@ if (NOT CMAKE_HOST_SYSTEM_NAME STREQUAL CMAKE_SYSTEM_NAME
"-DClickHouse_SOURCE_DIR=${ClickHouse_SOURCE_DIR}"
"${OPENSSL_SOURCE_DIR}"
WORKING_DIRECTORY "${OPENSSL_BUILD_DIR}"
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
execute_process(
COMMAND ${CMAKE_COMMAND} --build "${OPENSSL_BUILD_DIR}"
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
execute_process(
COMMAND ${CMAKE_COMMAND} --install "${OPENSSL_BUILD_DIR}"
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
# It's not important on which file we depend, we just want to specify right order
add_library(openssl_for_grpc STATIC IMPORTED GLOBAL)
@@ -108,8 +115,9 @@ if (NOT CMAKE_HOST_SYSTEM_NAME STREQUAL CMAKE_SYSTEM_NAME
set (GRPC_CPP_PLUGIN_BUILD_DIR "${_gRPC_BINARY_DIR}/build")
execute_process(
- COMMAND mkdir -p ${GRPC_CPP_PLUGIN_BUILD_DIR}
- COMMAND_ECHO STDOUT
+ COMMAND mkdir -p ${GRPC_CPP_PLUGIN_BUILD_DIR}
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
)
set(abseil_source_dir "${ClickHouse_SOURCE_DIR}/contrib/abseil-cpp")
@@ -140,11 +148,15 @@ if (NOT CMAKE_HOST_SYSTEM_NAME STREQUAL CMAKE_SYSTEM_NAME
"-DgRPC_SSL_PROVIDER=package"
"${_gRPC_SOURCE_DIR}"
WORKING_DIRECTORY "${GRPC_CPP_PLUGIN_BUILD_DIR}"
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
execute_process(
COMMAND ${CMAKE_COMMAND} --build "${GRPC_CPP_PLUGIN_BUILD_DIR}"
- COMMAND_ECHO STDOUT)
+ COMMAND_ECHO STDOUT
+ COMMAND_ERROR_IS_FATAL ANY
+ )
add_executable(grpc_cpp_plugin IMPORTED GLOBAL)
set_target_properties (grpc_cpp_plugin PROPERTIES IMPORTED_LOCATION "${GRPC_CPP_PLUGIN_BUILD_DIR}/grpc_cpp_plugin")
diff --git a/contrib/icu b/contrib/icu
index 7750081bda4..4216173eeeb 160000
--- a/contrib/icu
+++ b/contrib/icu
@@ -1 +1 @@
-Subproject commit 7750081bda4b3bc1768ae03849ec70f67ea10625
+Subproject commit 4216173eeeb39c1d4caaa54a68860e800412d273
diff --git a/docker/test/fasttest/run.sh b/docker/test/fasttest/run.sh
index 394d31addb1..9920326b11c 100755
--- a/docker/test/fasttest/run.sh
+++ b/docker/test/fasttest/run.sh
@@ -256,22 +256,6 @@ function configure
rm -f "$FASTTEST_DATA/config.d/secure_ports.xml"
}
-function timeout_with_logging() {
- local exit_code=0
-
- timeout -s TERM --preserve-status "${@}" || exit_code="${?}"
-
- echo "Checking if it is a timeout. The code 124 will indicate a timeout."
- if [[ "${exit_code}" -eq "124" ]]
- then
- echo "The command 'timeout ${*}' has been killed by timeout."
- else
- echo "No, it isn't a timeout."
- fi
-
- return $exit_code
-}
-
function run_tests
{
clickhouse-server --version
@@ -340,7 +324,7 @@ case "$stage" in
configure 2>&1 | ts '%Y-%m-%d %H:%M:%S' | tee "$FASTTEST_OUTPUT/install_log.txt"
;&
"run_tests")
- timeout_with_logging 35m bash -c run_tests ||:
+ run_tests ||:
/process_functional_tests_result.py --in-results-dir "$FASTTEST_OUTPUT/" \
--out-results-file "$FASTTEST_OUTPUT/test_results.tsv" \
--out-status-file "$FASTTEST_OUTPUT/check_status.tsv" || echo -e "failure\tCannot parse results" > "$FASTTEST_OUTPUT/check_status.tsv"
diff --git a/docker/test/sqllogic/Dockerfile b/docker/test/sqllogic/Dockerfile
index 1425e12cd84..6397526388e 100644
--- a/docker/test/sqllogic/Dockerfile
+++ b/docker/test/sqllogic/Dockerfile
@@ -35,7 +35,6 @@ RUN mkdir -p /tmp/clickhouse-odbc-tmp \
ENV TZ=Europe/Amsterdam
-ENV MAX_RUN_TIME=9000
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
ARG sqllogic_test_repo="https://github.com/gregrahn/sqllogictest.git"
diff --git a/docker/test/sqllogic/run.sh b/docker/test/sqllogic/run.sh
index ccba344035e..32368980f9b 100755
--- a/docker/test/sqllogic/run.sh
+++ b/docker/test/sqllogic/run.sh
@@ -94,7 +94,7 @@ function run_tests()
export -f run_tests
-timeout "${MAX_RUN_TIME:-9000}" bash -c run_tests || echo "timeout reached" >&2
+run_tests
#/process_functional_tests_result.py || echo -e "failure\tCannot parse results" > /test_output/check_status.tsv
diff --git a/docker/test/sqltest/Dockerfile b/docker/test/sqltest/Dockerfile
index 71d915b0c7a..b805bb03c2b 100644
--- a/docker/test/sqltest/Dockerfile
+++ b/docker/test/sqltest/Dockerfile
@@ -22,7 +22,6 @@ ARG sqltest_repo="https://github.com/elliotchance/sqltest/"
RUN git clone ${sqltest_repo}
ENV TZ=UTC
-ENV MAX_RUN_TIME=900
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
COPY run.sh /
diff --git a/docker/test/stateful/run.sh b/docker/test/stateful/run.sh
index 3a4f0d97993..a489a4b6e14 100755
--- a/docker/test/stateful/run.sh
+++ b/docker/test/stateful/run.sh
@@ -4,9 +4,6 @@
source /setup_export_logs.sh
set -e -x
-MAX_RUN_TIME=${MAX_RUN_TIME:-3600}
-MAX_RUN_TIME=$((MAX_RUN_TIME == 0 ? 3600 : MAX_RUN_TIME))
-
# Choose random timezone for this test run
TZ="$(rg -v '#' /usr/share/zoneinfo/zone.tab | awk '{print $3}' | shuf | head -n1)"
echo "Choosen random timezone $TZ"
@@ -118,14 +115,11 @@ if [[ -n "$USE_DATABASE_REPLICATED" ]] && [[ "$USE_DATABASE_REPLICATED" -eq 1 ]]
clickhouse-client --query "CREATE TABLE test.hits AS datasets.hits_v1"
clickhouse-client --query "CREATE TABLE test.visits AS datasets.visits_v1"
- clickhouse-client --query "INSERT INTO test.hits SELECT * FROM datasets.hits_v1"
- clickhouse-client --query "INSERT INTO test.visits SELECT * FROM datasets.visits_v1"
+ clickhouse-client --max_memory_usage 10G --query "INSERT INTO test.hits SELECT * FROM datasets.hits_v1"
+ clickhouse-client --max_memory_usage 10G --query "INSERT INTO test.visits SELECT * FROM datasets.visits_v1"
clickhouse-client --query "DROP TABLE datasets.hits_v1"
clickhouse-client --query "DROP TABLE datasets.visits_v1"
-
- MAX_RUN_TIME=$((MAX_RUN_TIME < 9000 ? MAX_RUN_TIME : 9000)) # min(MAX_RUN_TIME, 2.5 hours)
- MAX_RUN_TIME=$((MAX_RUN_TIME != 0 ? MAX_RUN_TIME : 9000)) # set to 2.5 hours if 0 (unlimited)
else
clickhouse-client --query "CREATE DATABASE test"
clickhouse-client --query "SHOW TABLES FROM test"
@@ -191,8 +185,8 @@ else
ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID)
SAMPLE BY intHash32(UserID) SETTINGS index_granularity = 8192, storage_policy='s3_cache'"
- clickhouse-client --query "INSERT INTO test.hits SELECT * FROM datasets.hits_v1 SETTINGS enable_filesystem_cache_on_write_operations=0, max_insert_threads=16"
- clickhouse-client --query "INSERT INTO test.visits SELECT * FROM datasets.visits_v1 SETTINGS enable_filesystem_cache_on_write_operations=0, max_insert_threads=16"
+ clickhouse-client --max_memory_usage 10G --query "INSERT INTO test.hits SELECT * FROM datasets.hits_v1 SETTINGS enable_filesystem_cache_on_write_operations=0, max_insert_threads=16"
+ clickhouse-client --max_memory_usage 10G --query "INSERT INTO test.visits SELECT * FROM datasets.visits_v1 SETTINGS enable_filesystem_cache_on_write_operations=0, max_insert_threads=16"
clickhouse-client --query "DROP TABLE datasets.visits_v1 SYNC"
clickhouse-client --query "DROP TABLE datasets.hits_v1 SYNC"
else
@@ -200,7 +194,8 @@ else
clickhouse-client --query "RENAME TABLE datasets.visits_v1 TO test.visits"
fi
clickhouse-client --query "CREATE TABLE test.hits_s3 (WatchID UInt64, JavaEnable UInt8, Title String, GoodEvent Int16, EventTime DateTime, EventDate Date, CounterID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RegionID UInt32, UserID UInt64, CounterClass Int8, OS UInt8, UserAgent UInt8, URL String, Referer String, URLDomain String, RefererDomain String, Refresh UInt8, IsRobot UInt8, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), ResolutionWidth UInt16, ResolutionHeight UInt16, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, FlashMinor2 String, NetMajor UInt8, NetMinor UInt8, UserAgentMajor UInt16, UserAgentMinor FixedString(2), CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, MobilePhone UInt8, MobilePhoneModel String, Params String, IPNetworkID UInt32, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, IsArtifical UInt8, WindowClientWidth UInt16, WindowClientHeight UInt16, ClientTimeZone Int16, ClientEventTime DateTime, SilverlightVersion1 UInt8, SilverlightVersion2 UInt8, SilverlightVersion3 UInt32, SilverlightVersion4 UInt16, PageCharset String, CodeVersion UInt32, IsLink UInt8, IsDownload UInt8, IsNotBounce UInt8, FUniqID UInt64, HID UInt32, IsOldCounter UInt8, IsEvent UInt8, IsParameter UInt8, DontCountHits UInt8, WithHash UInt8, HitColor FixedString(1), UTCEventTime DateTime, Age UInt8, Sex UInt8, Income UInt8, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), RemoteIP UInt32, RemoteIP6 FixedString(16), WindowName Int32, OpenerName Int32, HistoryLength Int16, BrowserLanguage FixedString(2), BrowserCountry FixedString(2), SocialNetwork String, SocialAction String, HTTPError UInt16, SendTiming Int32, DNSTiming Int32, ConnectTiming Int32, ResponseStartTiming Int32, ResponseEndTiming Int32, FetchTiming Int32, RedirectTiming Int32, DOMInteractiveTiming Int32, DOMContentLoadedTiming Int32, DOMCompleteTiming Int32, LoadEventStartTiming Int32, LoadEventEndTiming Int32, NSToDOMContentLoadedTiming Int32, FirstPaintTiming Int32, RedirectCount Int8, SocialSourceNetworkID UInt8, SocialSourcePage String, ParamPrice Int64, ParamOrderID String, ParamCurrency FixedString(3), ParamCurrencyID UInt16, GoalsReached Array(UInt32), OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, RefererHash UInt64, URLHash UInt64, CLID UInt32, YCLID UInt64, ShareService String, ShareURL String, ShareTitle String, ParsedParams Nested(Key1 String, Key2 String, Key3 String, Key4 String, Key5 String, ValueDouble Float64), IslandID FixedString(16), RequestNum UInt32, RequestTry UInt8) ENGINE = MergeTree() PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) SAMPLE BY intHash32(UserID) SETTINGS index_granularity = 8192, storage_policy='s3_cache'"
- clickhouse-client --query "INSERT INTO test.hits_s3 SELECT * FROM test.hits SETTINGS enable_filesystem_cache_on_write_operations=0, max_insert_threads=16"
+ # AWS S3 is very inefficient, so increase memory even further:
+ clickhouse-client --max_memory_usage 30G --max_memory_usage_for_user 30G --query "INSERT INTO test.hits_s3 SELECT * FROM test.hits SETTINGS enable_filesystem_cache_on_write_operations=0, max_insert_threads=16"
fi
clickhouse-client --query "SHOW TABLES FROM test"
@@ -257,24 +252,7 @@ function run_tests()
export -f run_tests
-function timeout_with_logging() {
- local exit_code=0
-
- timeout -s TERM --preserve-status "${@}" || exit_code="${?}"
-
- echo "Checking if it is a timeout. The code 124 will indicate a timeout."
- if [[ "${exit_code}" -eq "124" ]]
- then
- echo "The command 'timeout ${*}' has been killed by timeout."
- else
- echo "No, it isn't a timeout."
- fi
-
- return $exit_code
-}
-
-TIMEOUT=$((MAX_RUN_TIME - 700))
-timeout_with_logging "$TIMEOUT" bash -c run_tests ||:
+run_tests ||:
echo "Files in current directory"
ls -la ./
diff --git a/docker/test/stateless/Dockerfile b/docker/test/stateless/Dockerfile
index d8eb072328f..b0c4914a4e8 100644
--- a/docker/test/stateless/Dockerfile
+++ b/docker/test/stateless/Dockerfile
@@ -65,7 +65,6 @@ ENV TZ=Europe/Amsterdam
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
ENV NUM_TRIES=1
-ENV MAX_RUN_TIME=0
# Unrelated to vars in setup_minio.sh, but should be the same there
# to have the same binaries for local running scenario
diff --git a/docker/test/stateless/run.sh b/docker/test/stateless/run.sh
index c70cbe1fe45..6973eebbac3 100755
--- a/docker/test/stateless/run.sh
+++ b/docker/test/stateless/run.sh
@@ -12,9 +12,6 @@ dmesg --clear
# fail on errors, verbose and export all env variables
set -e -x -a
-MAX_RUN_TIME=${MAX_RUN_TIME:-9000}
-MAX_RUN_TIME=$((MAX_RUN_TIME == 0 ? 9000 : MAX_RUN_TIME))
-
USE_DATABASE_REPLICATED=${USE_DATABASE_REPLICATED:=0}
USE_SHARED_CATALOG=${USE_SHARED_CATALOG:=0}
@@ -308,8 +305,6 @@ function run_tests()
try_run_with_retry 10 clickhouse-client -q "insert into system.zookeeper (name, path, value) values ('auxiliary_zookeeper2', '/test/chroot/', '')"
- TIMEOUT=$((MAX_RUN_TIME - 800 > 8400 ? 8400 : MAX_RUN_TIME - 800))
- START_TIME=${SECONDS}
set +e
TEST_ARGS=(
@@ -324,32 +319,22 @@ function run_tests()
--test-runs "$NUM_TRIES"
"${ADDITIONAL_OPTIONS[@]}"
)
- timeout --preserve-status --signal TERM --kill-after 60m ${TIMEOUT}s clickhouse-test "${TEST_ARGS[@]}" 2>&1 \
+ clickhouse-test "${TEST_ARGS[@]}" 2>&1 \
| ts '%Y-%m-%d %H:%M:%S' \
| tee -a test_output/test_result.txt
set -e
- DURATION=$((SECONDS - START_TIME))
-
- echo "Elapsed ${DURATION} seconds."
- if [[ $DURATION -ge $TIMEOUT ]]
- then
- echo "It looks like the command is terminated by the timeout, which is ${TIMEOUT} seconds."
- fi
}
export -f run_tests
-
-# This should be enough to setup job and collect artifacts
-TIMEOUT=$((MAX_RUN_TIME - 700))
if [ "$NUM_TRIES" -gt "1" ]; then
# We don't run tests with Ordinary database in PRs, only in master.
# So run new/changed tests with Ordinary at least once in flaky check.
- timeout_with_logging "$TIMEOUT" bash -c 'NUM_TRIES=1; USE_DATABASE_ORDINARY=1; run_tests' \
+ NUM_TRIES=1; USE_DATABASE_ORDINARY=1; run_tests \
| sed 's/All tests have finished/Redacted: a message about tests finish is deleted/' | sed 's/No tests were run/Redacted: a message about no tests run is deleted/' ||:
fi
-timeout_with_logging "$TIMEOUT" bash -c run_tests ||:
+run_tests ||:
echo "Files in current directory"
ls -la ./
@@ -391,8 +376,8 @@ done
# wait for minio to flush its batch if it has any
sleep 1
clickhouse-client -q "SYSTEM FLUSH ASYNC INSERT QUEUE"
-clickhouse-client -q "SELECT log FROM minio_audit_logs ORDER BY event_time INTO OUTFILE '/test_output/minio_audit_logs.jsonl.zst' FORMAT JSONEachRow"
-clickhouse-client -q "SELECT log FROM minio_server_logs ORDER BY event_time INTO OUTFILE '/test_output/minio_server_logs.jsonl.zst' FORMAT JSONEachRow"
+clickhouse-client --max_block_size 8192 --max_memory_usage 10G --max_threads 1 --max_result_bytes 0 --max_result_rows 0 --max_rows_to_read 0 --max_bytes_to_read 0 -q "SELECT log FROM minio_audit_logs ORDER BY event_time INTO OUTFILE '/test_output/minio_audit_logs.jsonl.zst' FORMAT JSONEachRow"
+clickhouse-client --max_block_size 8192 --max_memory_usage 10G --max_threads 1 --max_result_bytes 0 --max_result_rows 0 --max_rows_to_read 0 --max_bytes_to_read 0 -q "SELECT log FROM minio_server_logs ORDER BY event_time INTO OUTFILE '/test_output/minio_server_logs.jsonl.zst' FORMAT JSONEachRow"
# Stop server so we can safely read data with clickhouse-local.
# Why do we read data with clickhouse-local?
diff --git a/docker/test/stateless/utils.lib b/docker/test/stateless/utils.lib
index cb257536c36..31cd67254b4 100644
--- a/docker/test/stateless/utils.lib
+++ b/docker/test/stateless/utils.lib
@@ -40,22 +40,6 @@ function fn_exists() {
declare -F "$1" > /dev/null;
}
-function timeout_with_logging() {
- local exit_code=0
-
- timeout -s TERM --preserve-status "${@}" || exit_code="${?}"
-
- echo "Checking if it is a timeout. The code 124 will indicate a timeout."
- if [[ "${exit_code}" -eq "124" ]]
- then
- echo "The command 'timeout ${*}' has been killed by timeout."
- else
- echo "No, it isn't a timeout."
- fi
-
- return $exit_code
-}
-
function collect_core_dumps()
{
find . -type f -maxdepth 1 -name 'core.*' | while read -r core; do
diff --git a/docker/test/util/process_functional_tests_result.py b/docker/test/util/process_functional_tests_result.py
index aa2ea686c46..ec9e14b1430 100755
--- a/docker/test/util/process_functional_tests_result.py
+++ b/docker/test/util/process_functional_tests_result.py
@@ -116,7 +116,7 @@ def process_test_log(log_path, broken_tests):
test[0],
test[1],
test[2],
- "".join(test[3])[:4096].replace("\t", "\\t").replace("\n", "\\n"),
+ "".join(test[3])[:8192].replace("\t", "\\t").replace("\n", "\\n"),
]
for test in test_results
]
diff --git a/docs/changelogs/v23.8.16.16-lts.md b/docs/changelogs/v23.8.16.16-lts.md
new file mode 100644
index 00000000000..9532db4fb0a
--- /dev/null
+++ b/docs/changelogs/v23.8.16.16-lts.md
@@ -0,0 +1,38 @@
+---
+sidebar_position: 1
+sidebar_label: 2024
+---
+
+# 2024 Changelog
+
+### ClickHouse release v23.8.16.16-lts (b80cac57ead) FIXME as compared to v23.8.15.35-lts (060ff8e813a)
+
+#### Improvement
+* Backported in [#66962](https://github.com/ClickHouse/ClickHouse/issues/66962): Added support for parameterized view with analyzer to not analyze create parameterized view. Refactor existing parameterized view logic to not analyze create parameterized view. [#54211](https://github.com/ClickHouse/ClickHouse/pull/54211) ([SmitaRKulkarni](https://github.com/SmitaRKulkarni)).
+* Backported in [#65461](https://github.com/ClickHouse/ClickHouse/issues/65461): Reload certificate chain during certificate reload. [#61671](https://github.com/ClickHouse/ClickHouse/pull/61671) ([Pervakov Grigorii](https://github.com/GrigoryPervakov)).
+* Backported in [#65880](https://github.com/ClickHouse/ClickHouse/issues/65880): Always start Keeper with sufficient amount of threads in global thread pool. [#64444](https://github.com/ClickHouse/ClickHouse/pull/64444) ([Duc Canh Le](https://github.com/canhld94)).
+* Backported in [#65912](https://github.com/ClickHouse/ClickHouse/issues/65912): Respect cgroup CPU limit in Keeper. [#65819](https://github.com/ClickHouse/ClickHouse/pull/65819) ([Antonio Andelic](https://github.com/antonio2368)).
+
+#### Bug Fix (user-visible misbehavior in an official stable release)
+* Backported in [#65351](https://github.com/ClickHouse/ClickHouse/issues/65351): Fix possible abort on uncaught exception in ~WriteBufferFromFileDescriptor in StatusFile. [#64206](https://github.com/ClickHouse/ClickHouse/pull/64206) ([Kruglov Pavel](https://github.com/Avogar)).
+* Backported in [#66037](https://github.com/ClickHouse/ClickHouse/issues/66037): Fix crash on destroying AccessControl: add explicit shutdown. [#64993](https://github.com/ClickHouse/ClickHouse/pull/64993) ([Vitaly Baranov](https://github.com/vitlibar)).
+* Backported in [#65281](https://github.com/ClickHouse/ClickHouse/issues/65281): Fix crash with UniqInjectiveFunctionsEliminationPass and uniqCombined. [#65188](https://github.com/ClickHouse/ClickHouse/pull/65188) ([Raúl Marín](https://github.com/Algunenano)).
+* Backported in [#65368](https://github.com/ClickHouse/ClickHouse/issues/65368): Fix a bug in ClickHouse Keeper that causes digest mismatch during closing session. [#65198](https://github.com/ClickHouse/ClickHouse/pull/65198) ([Aleksei Filatov](https://github.com/aalexfvk)).
+* Backported in [#65782](https://github.com/ClickHouse/ClickHouse/issues/65782): Fixed bug in MergeJoin. Column in sparse serialisation might be treated as a column of its nested type though the required conversion wasn't performed. [#65632](https://github.com/ClickHouse/ClickHouse/pull/65632) ([Nikita Taranov](https://github.com/nickitat)).
+* Backported in [#65743](https://github.com/ClickHouse/ClickHouse/issues/65743): Fix crash in maxIntersections. [#65689](https://github.com/ClickHouse/ClickHouse/pull/65689) ([Raúl Marín](https://github.com/Algunenano)).
+* Backported in [#65926](https://github.com/ClickHouse/ClickHouse/issues/65926): For queries that read from `PostgreSQL`, cancel the internal `PostgreSQL` query if the ClickHouse query is finished. Otherwise, `ClickHouse` query cannot be canceled until the internal `PostgreSQL` query is finished. [#65771](https://github.com/ClickHouse/ClickHouse/pull/65771) ([Maksim Kita](https://github.com/kitaisreal)).
+* Backported in [#65822](https://github.com/ClickHouse/ClickHouse/issues/65822): Fix a bug in short circuit logic when old analyzer and dictGetOrDefault is used. [#65802](https://github.com/ClickHouse/ClickHouse/pull/65802) ([jsc0218](https://github.com/jsc0218)).
+* Backported in [#66322](https://github.com/ClickHouse/ClickHouse/issues/66322): Add missing settings `input_format_csv_skip_first_lines/input_format_tsv_skip_first_lines/input_format_csv_try_infer_numbers_from_strings/input_format_csv_try_infer_strings_from_quoted_tuples` in schema inference cache because they can change the resulting schema. It prevents from incorrect result of schema inference with these settings changed. [#65980](https://github.com/ClickHouse/ClickHouse/pull/65980) ([Kruglov Pavel](https://github.com/Avogar)).
+* Backported in [#66449](https://github.com/ClickHouse/ClickHouse/issues/66449): Fixed a bug in ZooKeeper client: a session could get stuck in unusable state after receiving a hardware error from ZooKeeper. For example, this might happen due to "soft memory limit" in ClickHouse Keeper. [#66140](https://github.com/ClickHouse/ClickHouse/pull/66140) ([Alexander Tokmakov](https://github.com/tavplubix)).
+* Backported in [#66717](https://github.com/ClickHouse/ClickHouse/issues/66717): Correctly track memory for `Allocator::realloc`. [#66548](https://github.com/ClickHouse/ClickHouse/pull/66548) ([Antonio Andelic](https://github.com/antonio2368)).
+* Backported in [#67320](https://github.com/ClickHouse/ClickHouse/issues/67320): TRUNCATE DATABASE used to stop replication as if it was a DROP DATABASE query, it's fixed. [#67129](https://github.com/ClickHouse/ClickHouse/pull/67129) ([Alexander Tokmakov](https://github.com/tavplubix)).
+
+#### NOT FOR CHANGELOG / INSIGNIFICANT
+
+* Backported in [#65080](https://github.com/ClickHouse/ClickHouse/issues/65080): Follow up to [#56541](https://github.com/ClickHouse/ClickHouse/issues/56541). [#57141](https://github.com/ClickHouse/ClickHouse/pull/57141) ([Kseniia Sumarokova](https://github.com/kssenii)).
+* Backported in [#64997](https://github.com/ClickHouse/ClickHouse/issues/64997): Fix crash with DISTINCT and window functions. [#64767](https://github.com/ClickHouse/ClickHouse/pull/64767) ([Igor Nikonov](https://github.com/devcrafter)).
+* Backported in [#65913](https://github.com/ClickHouse/ClickHouse/issues/65913): Fix bug with session closing in Keeper. [#65735](https://github.com/ClickHouse/ClickHouse/pull/65735) ([Antonio Andelic](https://github.com/antonio2368)).
+* Backported in [#66853](https://github.com/ClickHouse/ClickHouse/issues/66853): Fix data race in S3::ClientCache. [#66644](https://github.com/ClickHouse/ClickHouse/pull/66644) ([Konstantin Morozov](https://github.com/k-morozov)).
+* Backported in [#67072](https://github.com/ClickHouse/ClickHouse/issues/67072): Increase asio pool size in case the server is tiny. [#66761](https://github.com/ClickHouse/ClickHouse/pull/66761) ([alesapin](https://github.com/alesapin)).
+* Update version after release. [#67691](https://github.com/ClickHouse/ClickHouse/pull/67691) ([robot-clickhouse](https://github.com/robot-clickhouse)).
+
diff --git a/docs/changelogs/v24.3.8.13-lts.md b/docs/changelogs/v24.3.8.13-lts.md
new file mode 100644
index 00000000000..6fbceacd624
--- /dev/null
+++ b/docs/changelogs/v24.3.8.13-lts.md
@@ -0,0 +1,16 @@
+---
+sidebar_position: 1
+sidebar_label: 2024
+---
+
+# 2024 Changelog
+
+### ClickHouse release v24.3.8.13-lts (84bbfc70f5d) FIXME as compared to v24.3.7.30-lts (c8a28cf4331)
+
+#### Bug Fix (user-visible misbehavior in an official stable release)
+* Backported in [#68562](https://github.com/ClickHouse/ClickHouse/issues/68562): Fix indexHint function case found by fuzzer. [#66286](https://github.com/ClickHouse/ClickHouse/pull/66286) ([Anton Popov](https://github.com/CurtizJ)).
+* Backported in [#68114](https://github.com/ClickHouse/ClickHouse/issues/68114): Fix possible PARAMETER_OUT_OF_BOUND error during reading variant subcolumn. [#66659](https://github.com/ClickHouse/ClickHouse/pull/66659) ([Kruglov Pavel](https://github.com/Avogar)).
+* Backported in [#67989](https://github.com/ClickHouse/ClickHouse/issues/67989): Validate experimental/suspicious data types in ALTER ADD/MODIFY COLUMN. [#67911](https://github.com/ClickHouse/ClickHouse/pull/67911) ([Kruglov Pavel](https://github.com/Avogar)).
+* Backported in [#68335](https://github.com/ClickHouse/ClickHouse/issues/68335): Try fix postgres crash when query is cancelled. [#68288](https://github.com/ClickHouse/ClickHouse/pull/68288) ([Kseniia Sumarokova](https://github.com/kssenii)).
+* Backported in [#68392](https://github.com/ClickHouse/ClickHouse/issues/68392): Fix missing sync replica mode in query `SYSTEM SYNC REPLICA`. [#68326](https://github.com/ClickHouse/ClickHouse/pull/68326) ([Duc Canh Le](https://github.com/canhld94)).
+
diff --git a/docs/en/operations/settings/settings.md b/docs/en/operations/settings/settings.md
index ea58959c123..19db4be17db 100644
--- a/docs/en/operations/settings/settings.md
+++ b/docs/en/operations/settings/settings.md
@@ -1381,7 +1381,7 @@ Default value: `2`.
Close connection before returning connection to the pool.
-Default value: true.
+Default value: false.
## odbc_bridge_connection_pool_size {#odbc-bridge-connection-pool-size}
diff --git a/docs/en/sql-reference/functions/type-conversion-functions.md b/docs/en/sql-reference/functions/type-conversion-functions.md
index 9e4b39e95e4..edd04580f27 100644
--- a/docs/en/sql-reference/functions/type-conversion-functions.md
+++ b/docs/en/sql-reference/functions/type-conversion-functions.md
@@ -6103,30 +6103,23 @@ Result:
└───────┴───────────────┴──────┴──────────────┴──────────────┴──────────────────────┘
```
-## toInterval(Year\|Quarter\|Month\|Week\|Day\|Hour\|Minute\|Second)
+## toIntervalYear
-Converts a Number type argument to an [Interval](../data-types/special-data-types/interval.md) data type.
+Returns an interval of `n` years of data type [IntervalYear](../data-types/special-data-types/interval.md).
**Syntax**
``` sql
-toIntervalSecond(number)
-toIntervalMinute(number)
-toIntervalHour(number)
-toIntervalDay(number)
-toIntervalWeek(number)
-toIntervalMonth(number)
-toIntervalQuarter(number)
-toIntervalYear(number)
+toIntervalYear(n)
```
**Arguments**
-- `number` — Duration of interval. Positive integer number.
+- `n` — Number of years. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
**Returned values**
-- The value in `Interval` data type.
+- Interval of `n` years. [IntervalYear](../data-types/special-data-types/interval.md).
**Example**
@@ -6134,20 +6127,387 @@ Query:
``` sql
WITH
- toDate('2019-01-01') AS date,
- INTERVAL 1 WEEK AS interval_week,
- toIntervalWeek(1) AS interval_to_week
-SELECT
- date + interval_week,
- date + interval_to_week;
+ toDate('2024-06-15') AS date,
+ toIntervalYear(1) AS interval_to_year
+SELECT date + interval_to_year AS result
```
Result:
```response
-┌─plus(date, interval_week)─┬─plus(date, interval_to_week)─┐
-│ 2019-01-08 │ 2019-01-08 │
-└───────────────────────────┴──────────────────────────────┘
+┌─────result─┐
+│ 2025-06-15 │
+└────────────┘
+```
+
+## toIntervalQuarter
+
+Returns an interval of `n` quarters of data type [IntervalQuarter](../data-types/special-data-types/interval.md).
+
+**Syntax**
+
+``` sql
+toIntervalQuarter(n)
+```
+
+**Arguments**
+
+- `n` — Number of quarters. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
+
+**Returned values**
+
+- Interval of `n` quarters. [IntervalQuarter](../data-types/special-data-types/interval.md).
+
+**Example**
+
+Query:
+
+``` sql
+WITH
+ toDate('2024-06-15') AS date,
+ toIntervalQuarter(1) AS interval_to_quarter
+SELECT date + interval_to_quarter AS result
+```
+
+Result:
+
+```response
+┌─────result─┐
+│ 2024-09-15 │
+└────────────┘
+```
+
+## toIntervalMonth
+
+Returns an interval of `n` months of data type [IntervalMonth](../data-types/special-data-types/interval.md).
+
+**Syntax**
+
+``` sql
+toIntervalMonth(n)
+```
+
+**Arguments**
+
+- `n` — Number of months. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
+
+**Returned values**
+
+- Interval of `n` months. [IntervalMonth](../data-types/special-data-types/interval.md).
+
+**Example**
+
+Query:
+
+``` sql
+WITH
+ toDate('2024-06-15') AS date,
+ toIntervalMonth(1) AS interval_to_month
+SELECT date + interval_to_month AS result
+```
+
+Result:
+
+```response
+┌─────result─┐
+│ 2024-07-15 │
+└────────────┘
+```
+
+## toIntervalWeek
+
+Returns an interval of `n` weeks of data type [IntervalWeek](../data-types/special-data-types/interval.md).
+
+**Syntax**
+
+``` sql
+toIntervalWeek(n)
+```
+
+**Arguments**
+
+- `n` — Number of weeks. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
+
+**Returned values**
+
+- Interval of `n` weeks. [IntervalWeek](../data-types/special-data-types/interval.md).
+
+**Example**
+
+Query:
+
+``` sql
+WITH
+ toDate('2024-06-15') AS date,
+ toIntervalWeek(1) AS interval_to_week
+SELECT date + interval_to_week AS result
+```
+
+Result:
+
+```response
+┌─────result─┐
+│ 2024-06-22 │
+└────────────┘
+```
+
+## toIntervalDay
+
+Returns an interval of `n` days of data type [IntervalDay](../data-types/special-data-types/interval.md).
+
+**Syntax**
+
+``` sql
+toIntervalDay(n)
+```
+
+**Arguments**
+
+- `n` — Number of days. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
+
+**Returned values**
+
+- Interval of `n` days. [IntervalDay](../data-types/special-data-types/interval.md).
+
+**Example**
+
+Query:
+
+``` sql
+WITH
+ toDate('2024-06-15') AS date,
+ toIntervalDay(5) AS interval_to_days
+SELECT date + interval_to_days AS result
+```
+
+Result:
+
+```response
+┌─────result─┐
+│ 2024-06-20 │
+└────────────┘
+```
+
+## toIntervalHour
+
+Returns an interval of `n` hours of data type [IntervalHour](../data-types/special-data-types/interval.md).
+
+**Syntax**
+
+``` sql
+toIntervalHour(n)
+```
+
+**Arguments**
+
+- `n` — Number of hours. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
+
+**Returned values**
+
+- Interval of `n` hours. [IntervalHour](../data-types/special-data-types/interval.md).
+
+**Example**
+
+Query:
+
+``` sql
+WITH
+ toDate('2024-06-15') AS date,
+ toIntervalHour(12) AS interval_to_hours
+SELECT date + interval_to_hours AS result
+```
+
+Result:
+
+```response
+┌──────────────result─┐
+│ 2024-06-15 12:00:00 │
+└─────────────────────┘
+```
+
+## toIntervalMinute
+
+Returns an interval of `n` minutes of data type [IntervalMinute](../data-types/special-data-types/interval.md).
+
+**Syntax**
+
+``` sql
+toIntervalMinute(n)
+```
+
+**Arguments**
+
+- `n` — Number of minutes. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
+
+**Returned values**
+
+- Interval of `n` minutes. [IntervalMinute](../data-types/special-data-types/interval.md).
+
+**Example**
+
+Query:
+
+``` sql
+WITH
+ toDate('2024-06-15') AS date,
+ toIntervalMinute(12) AS interval_to_minutes
+SELECT date + interval_to_minutes AS result
+```
+
+Result:
+
+```response
+┌──────────────result─┐
+│ 2024-06-15 00:12:00 │
+└─────────────────────┘
+```
+
+## toIntervalSecond
+
+Returns an interval of `n` seconds of data type [IntervalSecond](../data-types/special-data-types/interval.md).
+
+**Syntax**
+
+``` sql
+toIntervalSecond(n)
+```
+
+**Arguments**
+
+- `n` — Number of seconds. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
+
+**Returned values**
+
+- Interval of `n` seconds. [IntervalSecond](../data-types/special-data-types/interval.md).
+
+**Example**
+
+Query:
+
+``` sql
+WITH
+ toDate('2024-06-15') AS date,
+ toIntervalSecond(30) AS interval_to_seconds
+SELECT date + interval_to_seconds AS result
+```
+
+Result:
+
+```response
+┌──────────────result─┐
+│ 2024-06-15 00:00:30 │
+└─────────────────────┘
+```
+
+## toIntervalMillisecond
+
+Returns an interval of `n` milliseconds of data type [IntervalMillisecond](../data-types/special-data-types/interval.md).
+
+**Syntax**
+
+``` sql
+toIntervalMillisecond(n)
+```
+
+**Arguments**
+
+- `n` — Number of milliseconds. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
+
+**Returned values**
+
+- Interval of `n` milliseconds. [IntervalMilliseconds](../data-types/special-data-types/interval.md).
+
+**Example**
+
+Query:
+
+``` sql
+WITH
+ toDateTime('2024-06-15') AS date,
+ toIntervalMillisecond(30) AS interval_to_milliseconds
+SELECT date + interval_to_milliseconds AS result
+```
+
+Result:
+
+```response
+┌──────────────────result─┐
+│ 2024-06-15 00:00:00.030 │
+└─────────────────────────┘
+```
+
+## toIntervalMicrosecond
+
+Returns an interval of `n` microseconds of data type [IntervalMicrosecond](../data-types/special-data-types/interval.md).
+
+**Syntax**
+
+``` sql
+toIntervalMicrosecond(n)
+```
+
+**Arguments**
+
+- `n` — Number of microseconds. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
+
+**Returned values**
+
+- Interval of `n` microseconds. [IntervalMicrosecond](../data-types/special-data-types/interval.md).
+
+**Example**
+
+Query:
+
+``` sql
+WITH
+ toDateTime('2024-06-15') AS date,
+ toIntervalMicrosecond(30) AS interval_to_microseconds
+SELECT date + interval_to_microseconds AS result
+```
+
+Result:
+
+```response
+┌─────────────────────result─┐
+│ 2024-06-15 00:00:00.000030 │
+└────────────────────────────┘
+```
+
+## toIntervalNanosecond
+
+Returns an interval of `n` nanoseconds of data type [IntervalNanosecond](../data-types/special-data-types/interval.md).
+
+**Syntax**
+
+``` sql
+toIntervalNanosecond(n)
+```
+
+**Arguments**
+
+- `n` — Number of nanoseconds. Integer numbers or string representations thereof, and float numbers. [(U)Int*](../data-types/int-uint.md)/[Float*](../data-types/float.md)/[String](../data-types/string.md).
+
+**Returned values**
+
+- Interval of `n` nanoseconds. [IntervalNanosecond](../data-types/special-data-types/interval.md).
+
+**Example**
+
+Query:
+
+``` sql
+WITH
+ toDateTime('2024-06-15') AS date,
+ toIntervalNanosecond(30) AS interval_to_nanoseconds
+SELECT date + interval_to_nanoseconds AS result
+```
+
+Result:
+
+```response
+┌────────────────────────result─┐
+│ 2024-06-15 00:00:00.000000030 │
+└───────────────────────────────┘
```
## parseDateTime
diff --git a/docs/en/sql-reference/statements/grant.md b/docs/en/sql-reference/statements/grant.md
index 43fa344a16d..6118f4c1d36 100644
--- a/docs/en/sql-reference/statements/grant.md
+++ b/docs/en/sql-reference/statements/grant.md
@@ -200,6 +200,7 @@ Hierarchy of privileges:
- `JDBC`
- `HDFS`
- `S3`
+ - `POSTGRES`
- [dictGet](#dictget)
- [displaySecretsInShowAndSelect](#displaysecretsinshowandselect)
- [NAMED COLLECTION ADMIN](#named-collection-admin)
@@ -476,6 +477,7 @@ Allows using external data sources. Applies to [table engines](../../engines/tab
- `JDBC`. Level: `GLOBAL`
- `HDFS`. Level: `GLOBAL`
- `S3`. Level: `GLOBAL`
+ - `POSTGRES`. Level: `GLOBAL`
The `SOURCES` privilege enables use of all the sources. Also you can grant a privilege for each source individually. To use sources, you need additional privileges.
diff --git a/programs/bash-completion/completions/CMakeLists.txt b/programs/bash-completion/completions/CMakeLists.txt
index d364e07ef6e..2e911e81981 100644
--- a/programs/bash-completion/completions/CMakeLists.txt
+++ b/programs/bash-completion/completions/CMakeLists.txt
@@ -6,6 +6,7 @@ macro(configure_bash_completion)
COMMAND ${PKG_CONFIG_BIN} --variable=completionsdir bash-completion
OUTPUT_VARIABLE ${out}
OUTPUT_STRIP_TRAILING_WHITESPACE
+ COMMAND_ERROR_IS_FATAL ANY
)
endif()
string(REPLACE /usr "${CMAKE_INSTALL_PREFIX}" out "${out}")
diff --git a/programs/client/Client.cpp b/programs/client/Client.cpp
index 25c94c56aa6..39edaf3497e 100644
--- a/programs/client/Client.cpp
+++ b/programs/client/Client.cpp
@@ -1164,6 +1164,9 @@ void Client::processOptions(const OptionsDescription & options_description,
/// (There is no need to copy the context because clickhouse-client has no background tasks so it won't use that context in parallel.)
client_context = global_context;
initClientContext();
+
+ /// Allow to pass-through unknown settings to the server.
+ client_context->getAccessControl().allowAllSettings();
}
diff --git a/programs/client/clickhouse-client.xml b/programs/client/clickhouse-client.xml
index 9ce7d1cb223..6eb8976a6ef 100644
--- a/programs/client/clickhouse-client.xml
+++ b/programs/client/clickhouse-client.xml
@@ -53,6 +53,9 @@
-->
+
+
+
diff --git a/tests/config/config.d/storage_conf_02944.xml b/tests/config/config.d/storage_conf_02944.xml
index 5f45640a923..08d78900229 100644
--- a/tests/config/config.d/storage_conf_02944.xml
+++ b/tests/config/config.d/storage_conf_02944.xml
@@ -19,6 +19,7 @@
10
100
0
+ 0
diff --git a/tests/config/install.sh b/tests/config/install.sh
index 7c4b36dc4bd..fda74bd7a8d 100755
--- a/tests/config/install.sh
+++ b/tests/config/install.sh
@@ -94,6 +94,7 @@ ln -sf $SRC_PATH/users.d/prefetch_settings.xml $DEST_SERVER_PATH/users.d/
ln -sf $SRC_PATH/users.d/nonconst_timezone.xml $DEST_SERVER_PATH/users.d/
ln -sf $SRC_PATH/users.d/allow_introspection_functions.yaml $DEST_SERVER_PATH/users.d/
ln -sf $SRC_PATH/users.d/replicated_ddl_entry.xml $DEST_SERVER_PATH/users.d/
+ln -sf $SRC_PATH/users.d/limits.yaml $DEST_SERVER_PATH/users.d/
if [[ -n "$USE_OLD_ANALYZER" ]] && [[ "$USE_OLD_ANALYZER" -eq 1 ]]; then
ln -sf $SRC_PATH/users.d/analyzer.xml $DEST_SERVER_PATH/users.d/
diff --git a/tests/config/users.d/limits.yaml b/tests/config/users.d/limits.yaml
new file mode 100644
index 00000000000..53cbbfa744a
--- /dev/null
+++ b/tests/config/users.d/limits.yaml
@@ -0,0 +1,56 @@
+profiles:
+ default:
+ max_memory_usage: 5G
+ max_rows_to_read: 20000000
+
+ # Also set every other limit to a high value, so it will not limit anything, but we will test that code around it.
+ s3_max_get_rps: 1000000
+ s3_max_get_burst: 2000000
+ s3_max_put_rps: 1000000
+ s3_max_put_burst: 2000000
+ max_remote_read_network_bandwidth: 1T
+ max_remote_write_network_bandwidth: 1T
+ max_local_read_bandwidth: 1T
+ max_local_write_bandwidth: 1T
+ use_index_for_in_with_subqueries_max_values: 1G
+ max_bytes_to_read: 1T
+ max_bytes_to_read_leaf: 1T
+ max_rows_to_group_by: 10G
+ max_bytes_before_external_group_by: 10G
+ max_rows_to_sort: 10G
+ max_bytes_to_sort: 10G
+ max_bytes_before_external_sort: 10G
+ max_result_rows: 1G
+ max_result_bytes: 1G
+ max_execution_time: 600
+ max_execution_time_leaf: 600
+ max_execution_speed: 100G
+ max_execution_speed_bytes: 10T
+ timeout_before_checking_execution_speed: 300
+ max_estimated_execution_time: 600
+ max_columns_to_read: 20K
+ max_temporary_columns: 20K
+ max_temporary_non_const_columns: 20K
+ max_rows_in_set: 10G
+ max_bytes_in_set: 10G
+ max_rows_in_join: 10G
+ max_bytes_in_join: 10G
+ max_rows_to_transfer: 1G
+ max_bytes_to_transfer: 1G
+ max_rows_in_distinct: 10G
+ max_bytes_in_distinct: 10G
+ max_memory_usage_for_user: 32G
+ max_network_bandwidth: 100G
+ max_network_bytes: 1T
+ max_network_bandwidth_for_user: 100G
+ max_network_bandwidth_for_all_users: 100G
+ max_temporary_data_on_disk_size_for_user: 100G
+ max_temporary_data_on_disk_size_for_query: 100G
+ max_backup_bandwidth: 100G
+ max_hyperscan_regexp_length: 1M
+ max_hyperscan_regexp_total_length: 10M
+ query_cache_max_size_in_bytes: 10M
+ query_cache_max_entries: 100K
+ external_storage_max_read_rows: 10G
+ external_storage_max_read_bytes: 10G
+ max_streams_for_merge_tree_reading: 1000
diff --git a/tests/integration/parallel_skip.json b/tests/integration/parallel_skip.json
index fca2126d824..507894534d4 100644
--- a/tests/integration/parallel_skip.json
+++ b/tests/integration/parallel_skip.json
@@ -1,6 +1,7 @@
[
"test_dns_cache/test.py::test_dns_cache_update",
"test_dns_cache/test.py::test_ip_change_drop_dns_cache",
+ "test_dns_cache/test.py::test_dns_resolver_filter",
"test_dns_cache/test.py::test_ip_change_update_dns_cache",
"test_dns_cache/test.py::test_user_access_ip_change[node0]",
"test_dns_cache/test.py::test_user_access_ip_change[node1]",
diff --git a/tests/integration/test_dictionaries_postgresql/test.py b/tests/integration/test_dictionaries_postgresql/test.py
index 516ac27ea26..010ecdb5084 100644
--- a/tests/integration/test_dictionaries_postgresql/test.py
+++ b/tests/integration/test_dictionaries_postgresql/test.py
@@ -530,10 +530,61 @@ def test_bad_configuration(started_cluster):
"""
)
- node1.query_and_get_error(
+ assert "Unexpected key `dbbb`" in node1.query_and_get_error(
"SELECT dictGetUInt32(postgres_dict, 'value', toUInt64(1))"
)
- assert node1.contains_in_log("Unexpected key `dbbb`")
+
+
+def test_named_collection_from_ddl(started_cluster):
+ cursor = started_cluster.postgres_conn.cursor()
+ cursor.execute("DROP TABLE IF EXISTS test_table")
+ cursor.execute("CREATE TABLE test_table (id integer, value integer)")
+
+ node1.query(
+ """
+ DROP NAMED COLLECTION IF EXISTS pg_conn;
+ CREATE NAMED COLLECTION pg_conn
+ AS user = 'postgres', password = 'mysecretpassword', host = 'postgres1', port = 5432, database = 'postgres', table = 'test_table';
+ """
+ )
+
+ cursor.execute(
+ "INSERT INTO test_table SELECT i, i FROM generate_series(0, 99) as t(i)"
+ )
+
+ node1.query(
+ """
+ DROP DICTIONARY IF EXISTS postgres_dict;
+ CREATE DICTIONARY postgres_dict (id UInt32, value UInt32)
+ PRIMARY KEY id
+ SOURCE(POSTGRESQL(NAME pg_conn))
+ LIFETIME(MIN 1 MAX 2)
+ LAYOUT(HASHED());
+ """
+ )
+ result = node1.query("SELECT dictGetUInt32(postgres_dict, 'value', toUInt64(99))")
+ assert int(result.strip()) == 99
+
+ node1.query(
+ """
+ DROP NAMED COLLECTION IF EXISTS pg_conn_2;
+ CREATE NAMED COLLECTION pg_conn_2
+ AS user = 'postgres', password = 'mysecretpassword', host = 'postgres1', port = 5432, dbbb = 'postgres', table = 'test_table';
+ """
+ )
+ node1.query(
+ """
+ DROP DICTIONARY IF EXISTS postgres_dict;
+ CREATE DICTIONARY postgres_dict (id UInt32, value UInt32)
+ PRIMARY KEY id
+ SOURCE(POSTGRESQL(NAME pg_conn_2))
+ LIFETIME(MIN 1 MAX 2)
+ LAYOUT(HASHED());
+ """
+ )
+ assert "Unexpected key `dbbb`" in node1.query_and_get_error(
+ "SELECT dictGetUInt32(postgres_dict, 'value', toUInt64(99))"
+ )
if __name__ == "__main__":
diff --git a/tests/integration/test_dns_cache/test.py b/tests/integration/test_dns_cache/test.py
index a6db26c8575..36401517429 100644
--- a/tests/integration/test_dns_cache/test.py
+++ b/tests/integration/test_dns_cache/test.py
@@ -317,3 +317,74 @@ def test_host_is_drop_from_cache_after_consecutive_failures(
assert node4.wait_for_log_line(
"Cached hosts dropped:.*InvalidHostThatDoesNotExist.*"
)
+
+
+node7 = cluster.add_instance(
+ "node7",
+ main_configs=["configs/listen_host.xml", "configs/dns_update_long.xml"],
+ with_zookeeper=True,
+ ipv6_address="2001:3984:3989::1:1117",
+ ipv4_address="10.5.95.17",
+)
+
+
+def _render_filter_config(allow_ipv4, allow_ipv6):
+ config = f"""
+
+ {int(allow_ipv4)}
+ {int(allow_ipv6)}
+
+ """
+ return config
+
+
+@pytest.mark.parametrize(
+ "allow_ipv4, allow_ipv6",
+ [
+ (True, False),
+ (False, True),
+ (False, False),
+ ],
+)
+def test_dns_resolver_filter(cluster_without_dns_cache_update, allow_ipv4, allow_ipv6):
+ node = node7
+ host_ipv6 = node.ipv6_address
+ host_ipv4 = node.ipv4_address
+
+ node.set_hosts(
+ [
+ (host_ipv6, "test_host"),
+ (host_ipv4, "test_host"),
+ ]
+ )
+ node.replace_config(
+ "/etc/clickhouse-server/config.d/dns_filter.xml",
+ _render_filter_config(allow_ipv4, allow_ipv6),
+ )
+
+ node.query("SYSTEM RELOAD CONFIG")
+ node.query("SYSTEM DROP DNS CACHE")
+ node.query("SYSTEM DROP CONNECTIONS CACHE")
+
+ if not allow_ipv4 and not allow_ipv6:
+ with pytest.raises(QueryRuntimeException):
+ node.query("SELECT * FROM remote('lost_host', 'system', 'one')")
+ else:
+ node.query("SELECT * FROM remote('test_host', system, one)")
+ assert (
+ node.query(
+ "SELECT ip_address FROM system.dns_cache WHERE hostname='test_host'"
+ )
+ == f"{host_ipv4 if allow_ipv4 else host_ipv6}\n"
+ )
+
+ node.exec_in_container(
+ [
+ "bash",
+ "-c",
+ "rm /etc/clickhouse-server/config.d/dns_filter.xml",
+ ],
+ privileged=True,
+ user="root",
+ )
+ node.query("SYSTEM RELOAD CONFIG")
diff --git a/tests/integration/test_filesystem_cache/test.py b/tests/integration/test_filesystem_cache/test.py
index 17a8dd8b6e1..aee8bd25c2e 100644
--- a/tests/integration/test_filesystem_cache/test.py
+++ b/tests/integration/test_filesystem_cache/test.py
@@ -1,6 +1,7 @@
import logging
import time
import os
+import random
import pytest
from helpers.cluster import ClickHouseCluster
@@ -30,14 +31,6 @@ def cluster():
"config.d/storage_conf_2.xml",
],
)
- cluster.add_instance(
- "node_no_filesystem_caches_path",
- main_configs=[
- "config.d/storage_conf.xml",
- "config.d/remove_filesystem_caches_path.xml",
- ],
- stay_alive=True,
- )
cluster.add_instance(
"node_force_read_through_cache_on_merge",
main_configs=[
@@ -59,6 +52,51 @@ def cluster():
cluster.shutdown()
+@pytest.fixture(scope="function")
+def non_shared_cluster():
+ """
+ For tests that cannot run in parallel against the same node/cluster (see test_custom_cached_disk, which relies on
+ changing server settings at runtime)
+ """
+ try:
+ # Randomize the cluster name
+ cluster = ClickHouseCluster(f"{__file__}_non_shared_{random.randint(0, 10**7)}")
+ cluster.add_instance(
+ "node_no_filesystem_caches_path",
+ main_configs=[
+ "config.d/storage_conf.xml",
+ "config.d/remove_filesystem_caches_path.xml",
+ ],
+ stay_alive=True,
+ )
+
+ logging.info("Starting test-exclusive cluster...")
+ cluster.start()
+ logging.info("Cluster started")
+
+ yield cluster
+ finally:
+ cluster.shutdown()
+
+
+def wait_for_cache_initialized(node, cache_path, max_attempts=50):
+ initialized = False
+ attempts = 0
+ while not initialized:
+ query_result = node.query(
+ "SELECT path FROM system.filesystem_cache_settings WHERE is_initialized"
+ )
+ initialized = cache_path in query_result
+
+ if initialized:
+ break
+
+ time.sleep(0.1)
+ attempts += 1
+ if attempts >= max_attempts:
+ raise "Stopped waiting for cache to be initialized"
+
+
@pytest.mark.parametrize("node_name", ["node"])
def test_parallel_cache_loading_on_startup(cluster, node_name):
node = cluster.instances[node_name]
@@ -71,14 +109,21 @@ def test_parallel_cache_loading_on_startup(cluster, node_name):
ORDER BY value
SETTINGS disk = disk(
type = cache,
- path = 'paralel_loading_test',
+ name = 'parallel_loading_test',
+ path = 'parallel_loading_test',
disk = 'hdd_blob',
max_file_segment_size = '1Ki',
boundary_alignment = '1Ki',
max_size = '1Gi',
max_elements = 10000000,
load_metadata_threads = 30);
+ """
+ )
+ wait_for_cache_initialized(node, "parallel_loading_test")
+
+ node.query(
+ """
SYSTEM DROP FILESYSTEM CACHE;
INSERT INTO test SELECT * FROM generateRandom('a Int32, b String') LIMIT 1000000;
SELECT * FROM test FORMAT Null;
@@ -103,6 +148,7 @@ def test_parallel_cache_loading_on_startup(cluster, node_name):
)
node.restart_clickhouse()
+ wait_for_cache_initialized(node, "parallel_loading_test")
# < because of additional files loaded into cache on server startup.
assert cache_count <= int(node.query("SELECT count() FROM system.filesystem_cache"))
@@ -131,7 +177,7 @@ def test_caches_with_the_same_configuration(cluster, node_name):
node = cluster.instances[node_name]
cache_path = "cache1"
- node.query(f"SYSTEM DROP FILESYSTEM CACHE;")
+ node.query("SYSTEM DROP FILESYSTEM CACHE;")
for table in ["test", "test2"]:
node.query(
f"""
@@ -142,14 +188,20 @@ def test_caches_with_the_same_configuration(cluster, node_name):
ORDER BY value
SETTINGS disk = disk(
type = cache,
- name = {table},
+ name = '{table}',
path = '{cache_path}',
disk = 'hdd_blob',
max_file_segment_size = '1Ki',
boundary_alignment = '1Ki',
cache_on_write_operations=1,
max_size = '1Mi');
+ """
+ )
+ wait_for_cache_initialized(node, cache_path)
+
+ node.query(
+ f"""
SET enable_filesystem_cache_on_write_operations=1;
INSERT INTO {table} SELECT * FROM generateRandom('a Int32, b String')
LIMIT 1000;
@@ -195,9 +247,8 @@ def test_caches_with_the_same_configuration(cluster, node_name):
@pytest.mark.parametrize("node_name", ["node_caches_with_same_path"])
def test_caches_with_the_same_configuration_2(cluster, node_name):
node = cluster.instances[node_name]
- cache_path = "cache1"
- node.query(f"SYSTEM DROP FILESYSTEM CACHE;")
+ node.query("SYSTEM DROP FILESYSTEM CACHE;")
for table in ["cache1", "cache2"]:
node.query(
f"""
@@ -207,7 +258,13 @@ def test_caches_with_the_same_configuration_2(cluster, node_name):
Engine=MergeTree()
ORDER BY value
SETTINGS disk = '{table}';
+ """
+ )
+ wait_for_cache_initialized(node, "cache1")
+
+ node.query(
+ f"""
SET enable_filesystem_cache_on_write_operations=1;
INSERT INTO {table} SELECT * FROM generateRandom('a Int32, b String')
LIMIT 1000;
@@ -227,8 +284,8 @@ def test_caches_with_the_same_configuration_2(cluster, node_name):
)
-def test_custom_cached_disk(cluster):
- node = cluster.instances["node_no_filesystem_caches_path"]
+def test_custom_cached_disk(non_shared_cluster):
+ node = non_shared_cluster.instances["node_no_filesystem_caches_path"]
assert "Cannot create cached custom disk without" in node.query_and_get_error(
f"""
@@ -377,6 +434,7 @@ def test_force_filesystem_cache_on_merges(cluster):
ORDER BY value
SETTINGS disk = disk(
type = cache,
+ name = 'force_cache_on_merges',
path = 'force_cache_on_merges',
disk = 'hdd_blob',
max_file_segment_size = '1Ki',
@@ -385,7 +443,13 @@ def test_force_filesystem_cache_on_merges(cluster):
max_size = '10Gi',
max_elements = 10000000,
load_metadata_threads = 30);
+ """
+ )
+ wait_for_cache_initialized(node, "force_cache_on_merges")
+
+ node.query(
+ """
SYSTEM DROP FILESYSTEM CACHE;
INSERT INTO test SELECT * FROM generateRandom('a Int32, b String') LIMIT 1000000;
INSERT INTO test SELECT * FROM generateRandom('a Int32, b String') LIMIT 1000000;
@@ -441,7 +505,13 @@ SETTINGS disk = disk(type = cache,
path = "test_system_sync_filesystem_cache",
delayed_cleanup_interval_ms = 10000000, disk = hdd_blob),
min_bytes_for_wide_part = 10485760;
+ """
+ )
+ wait_for_cache_initialized(node, "test_system_sync_filesystem_cache")
+
+ node.query(
+ """
INSERT INTO test SELECT 1, 'test';
"""
)
@@ -525,7 +595,13 @@ SETTINGS disk = disk(type = cache,
keep_free_space_elements_ratio = {elements_ratio},
disk = hdd_blob),
min_bytes_for_wide_part = 10485760;
+ """
+ )
+ wait_for_cache_initialized(node, "test_keep_up_size_ratio")
+
+ node.query(
+ """
INSERT INTO test SELECT randomString(200);
"""
)
diff --git a/tests/queries/0_stateless/00002_log_and_exception_messages_formatting.sql b/tests/queries/0_stateless/00002_log_and_exception_messages_formatting.sql
index 53321afc94c..0f13217c236 100644
--- a/tests/queries/0_stateless/00002_log_and_exception_messages_formatting.sql
+++ b/tests/queries/0_stateless/00002_log_and_exception_messages_formatting.sql
@@ -9,6 +9,8 @@ system flush logs;
drop table if exists logs;
create view logs as select * from system.text_log where now() - toIntervalMinute(120) < event_time;
+SET max_rows_to_read = 0;
+
-- Check that we don't have too many messages formatted with fmt::runtime or strings concatenation.
-- 0.001 threshold should be always enough, the value was about 0.00025
WITH 0.001 AS threshold
diff --git a/tests/queries/0_stateless/00086_concat_nary_const_with_nonconst_segfault.sql b/tests/queries/0_stateless/00086_concat_nary_const_with_nonconst_segfault.sql
index 2f0ef648983..4b87b2af28d 100644
--- a/tests/queries/0_stateless/00086_concat_nary_const_with_nonconst_segfault.sql
+++ b/tests/queries/0_stateless/00086_concat_nary_const_with_nonconst_segfault.sql
@@ -1 +1 @@
-SELECT extract(toString(number), '10000000') FROM system.numbers_mt WHERE concat(materialize('1'), '...', toString(number)) LIKE '%10000000%' LIMIT 1
+SELECT extract(toString(number), '10000000') FROM system.numbers_mt WHERE concat(materialize('1'), '...', toString(number)) LIKE '%10000000%' LIMIT 1 SETTINGS max_rows_to_read = 0;
diff --git a/tests/queries/0_stateless/00111_shard_external_sort_distributed.reference b/tests/queries/0_stateless/00111_shard_external_sort_distributed.reference
index df5aa77af60..7534c12a0d8 100644
--- a/tests/queries/0_stateless/00111_shard_external_sort_distributed.reference
+++ b/tests/queries/0_stateless/00111_shard_external_sort_distributed.reference
@@ -1,20 +1,20 @@
-7040546
-7040546
-4327029
-4327029
-1613512
-1613512
-8947307
-8947307
-6233790
-6233790
-3520273
-3520273
-806756
-806756
-8140551
-8140551
-5427034
-5427034
-2713517
-2713517
+4437158
+4437158
+1723641
+1723641
+3630402
+3630402
+916885
+916885
+2823646
+2823646
+110129
+110129
+4730407
+4730407
+2016890
+2016890
+3923651
+3923651
+1210134
+1210134
diff --git a/tests/queries/0_stateless/00111_shard_external_sort_distributed.sql b/tests/queries/0_stateless/00111_shard_external_sort_distributed.sql
index 112f5edae36..9e06654195d 100644
--- a/tests/queries/0_stateless/00111_shard_external_sort_distributed.sql
+++ b/tests/queries/0_stateless/00111_shard_external_sort_distributed.sql
@@ -1,11 +1,12 @@
--- Tags: distributed
+-- Tags: distributed, long, no-flaky-check
+-- ^ no-flaky-check - sometimes longer than 600s with ThreadFuzzer.
-SET max_memory_usage = 300000000;
-SET max_bytes_before_external_sort = 20000000;
+SET max_memory_usage = 150000000;
+SET max_bytes_before_external_sort = 10000000;
DROP TABLE IF EXISTS numbers10m;
-CREATE VIEW numbers10m AS SELECT number FROM system.numbers LIMIT 10000000;
+CREATE VIEW numbers10m AS SELECT number FROM system.numbers LIMIT 5000000;
-SELECT number FROM remote('127.0.0.{2,3}', currentDatabase(), numbers10m) ORDER BY number * 1234567890123456789 LIMIT 19999980, 20;
+SELECT number FROM remote('127.0.0.{2,3}', currentDatabase(), numbers10m) ORDER BY number * 1234567890123456789 LIMIT 4999980, 20;
DROP TABLE numbers10m;
diff --git a/tests/queries/0_stateless/00170_lower_upper_utf8.reference b/tests/queries/0_stateless/00170_lower_upper_utf8.reference
index f202cb75513..3c644f22b9b 100644
--- a/tests/queries/0_stateless/00170_lower_upper_utf8.reference
+++ b/tests/queries/0_stateless/00170_lower_upper_utf8.reference
@@ -22,3 +22,7 @@
1
1
1
+1
+1
+1
+1
diff --git a/tests/queries/0_stateless/00170_lower_upper_utf8.sql b/tests/queries/0_stateless/00170_lower_upper_utf8.sql
index 4caba2033ff..85b6c5c6095 100644
--- a/tests/queries/0_stateless/00170_lower_upper_utf8.sql
+++ b/tests/queries/0_stateless/00170_lower_upper_utf8.sql
@@ -1,3 +1,6 @@
+-- Tags: no-fasttest
+-- no-fasttest: upper/lowerUTF8 use ICU
+
select lower('aaaaaaaaaaaaaaa012345789,.!aaaa' as str) = str;
select lowerUTF8('aaaaaaaaaaaaaaa012345789,.!aaaa' as str) = str;
select lower('AaAaAaAaAaAaAaA012345789,.!aAaA') = 'aaaaaaaaaaaaaaa012345789,.!aaaa';
@@ -27,3 +30,11 @@ select sum(lower(materialize('aaaaАБВГAAAAaaAA')) = materialize('aaaaАБВ
select sum(upper(materialize('aaaaАБВГAAAAaaAA')) = materialize('AAAAАБВГAAAAAAAA')) = count() from system.one array join range(16384) as n;
select sum(lowerUTF8(materialize('aaaaАБВГAAAAaaAA')) = materialize('aaaaабвгaaaaaaaa')) = count() from system.one array join range(16384) as n;
select sum(upperUTF8(materialize('aaaaАБВГAAAAaaAA')) = materialize('AAAAАБВГAAAAAAAA')) = count() from system.one array join range(16384) as n;
+
+-- Turkish language
+select upperUTF8('ır') = 'IR';
+select lowerUTF8('ır') = 'ır';
+
+-- German language
+select upper('öäüß') = 'öäüß';
+select lower('ÖÄÜẞ') = 'ÖÄÜẞ';
diff --git a/tests/queries/0_stateless/00233_position_function_family.sql b/tests/queries/0_stateless/00233_position_function_family.sql
index dd7394bc39a..d6668cb7ba4 100644
--- a/tests/queries/0_stateless/00233_position_function_family.sql
+++ b/tests/queries/0_stateless/00233_position_function_family.sql
@@ -1,3 +1,6 @@
+-- Tags: no-fasttest
+-- no-fasttest: upper/lowerUTF8 use ICU
+
SET send_logs_level = 'fatal';
select 1 = position('', '');
diff --git a/tests/queries/0_stateless/00375_shard_group_uniq_array_of_string.sql b/tests/queries/0_stateless/00375_shard_group_uniq_array_of_string.sql
index 8a310cb8fc9..c8a243d9b27 100644
--- a/tests/queries/0_stateless/00375_shard_group_uniq_array_of_string.sql
+++ b/tests/queries/0_stateless/00375_shard_group_uniq_array_of_string.sql
@@ -1,4 +1,4 @@
--- Tags: shard
+-- Tags: shard, long
DROP TABLE IF EXISTS group_uniq_str;
CREATE TABLE group_uniq_str ENGINE = Memory AS SELECT number % 10 as id, toString(intDiv((number%10000), 10)) as v FROM system.numbers LIMIT 10000000;
@@ -7,7 +7,7 @@ INSERT INTO group_uniq_str SELECT 2 as id, toString(number % 100) as v FROM syst
INSERT INTO group_uniq_str SELECT 5 as id, toString(number % 100) as v FROM system.numbers LIMIT 10000000;
SELECT length(groupUniqArray(v)) FROM group_uniq_str GROUP BY id ORDER BY id;
-SELECT length(groupUniqArray(v)) FROM remote('127.0.0.{2,3,4,5}', currentDatabase(), 'group_uniq_str') GROUP BY id ORDER BY id;
+SELECT length(groupUniqArray(v)) FROM remote('127.0.0.{2,3,4,5}', currentDatabase(), 'group_uniq_str') GROUP BY id ORDER BY id SETTINGS max_rows_to_read = '100M';
SELECT length(groupUniqArray(10)(v)) FROM group_uniq_str GROUP BY id ORDER BY id;
SELECT length(groupUniqArray(10000)(v)) FROM group_uniq_str GROUP BY id ORDER BY id;
diff --git a/tests/queries/0_stateless/00376_shard_group_uniq_array_of_int_array.sql b/tests/queries/0_stateless/00376_shard_group_uniq_array_of_int_array.sql
index abd0e6e6a45..4453c26283c 100644
--- a/tests/queries/0_stateless/00376_shard_group_uniq_array_of_int_array.sql
+++ b/tests/queries/0_stateless/00376_shard_group_uniq_array_of_int_array.sql
@@ -1,4 +1,6 @@
--- Tags: shard
+-- Tags: long
+
+SET max_rows_to_read = '55M';
DROP TABLE IF EXISTS group_uniq_arr_int;
CREATE TABLE group_uniq_arr_int ENGINE = Memory AS
diff --git a/tests/queries/0_stateless/00377_shard_group_uniq_array_of_string_array.sql b/tests/queries/0_stateless/00377_shard_group_uniq_array_of_string_array.sql
index e9cfff211f8..1ec91ac2396 100644
--- a/tests/queries/0_stateless/00377_shard_group_uniq_array_of_string_array.sql
+++ b/tests/queries/0_stateless/00377_shard_group_uniq_array_of_string_array.sql
@@ -1,4 +1,5 @@
--- Tags: shard
+-- Tags: shard, long
+SET max_rows_to_read = '55M';
DROP TABLE IF EXISTS group_uniq_arr_str;
CREATE TABLE group_uniq_arr_str ENGINE = Memory AS
diff --git a/tests/queries/0_stateless/00463_long_sessions_in_http_interface.sh b/tests/queries/0_stateless/00463_long_sessions_in_http_interface.sh
index 6ee1649c9ed..86902fca4aa 100755
--- a/tests/queries/0_stateless/00463_long_sessions_in_http_interface.sh
+++ b/tests/queries/0_stateless/00463_long_sessions_in_http_interface.sh
@@ -74,7 +74,7 @@ ${CLICKHOUSE_CLIENT} --query "DROP TABLE t"
echo "A session cannot be used by concurrent connections:"
-${CLICKHOUSE_CURL} -sS -X POST "${CLICKHOUSE_URL}&session_id=${CLICKHOUSE_DATABASE}_9&query_id=${CLICKHOUSE_DATABASE}_9" --data-binary "SELECT count() FROM system.numbers" >/dev/null &
+${CLICKHOUSE_CURL} -sS -X POST "${CLICKHOUSE_URL}&session_id=${CLICKHOUSE_DATABASE}_9&query_id=${CLICKHOUSE_DATABASE}_9&max_rows_to_read=0" --data-binary "SELECT count() FROM system.numbers" >/dev/null &
# An infinite loop is required to make the test reliable. We will ensure that at least once the query on the line above has started before this check
while true
diff --git a/tests/queries/0_stateless/00600_replace_running_query.sh b/tests/queries/0_stateless/00600_replace_running_query.sh
index 80e2ecf5d5b..7d4125eea69 100755
--- a/tests/queries/0_stateless/00600_replace_running_query.sh
+++ b/tests/queries/0_stateless/00600_replace_running_query.sh
@@ -8,7 +8,7 @@ CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
TEST_PREFIX="${CLICKHOUSE_DATABASE}"
${CLICKHOUSE_CLIENT} -q "drop user if exists u_00600${TEST_PREFIX}"
-${CLICKHOUSE_CLIENT} -q "create user u_00600${TEST_PREFIX} settings max_execution_time=60, readonly=1"
+${CLICKHOUSE_CLIENT} -q "create user u_00600${TEST_PREFIX} settings max_execution_time=60, readonly=1, max_rows_to_read=0"
${CLICKHOUSE_CLIENT} -q "grant select on system.numbers to u_00600${TEST_PREFIX}"
function wait_for_query_to_start()
@@ -28,7 +28,7 @@ function wait_for_queries_to_finish()
}
-$CLICKHOUSE_CURL -sS "$CLICKHOUSE_URL&query_id=${CLICKHOUSE_DATABASE}hello&replace_running_query=1" -d 'SELECT 1, count() FROM system.numbers' > /dev/null 2>&1 &
+$CLICKHOUSE_CURL -sS "$CLICKHOUSE_URL&query_id=${CLICKHOUSE_DATABASE}hello&replace_running_query=1&max_rows_to_read=0" -d 'SELECT 1, count() FROM system.numbers' > /dev/null 2>&1 &
wait_for_query_to_start "${CLICKHOUSE_DATABASE}hello"
# Replace it
@@ -51,7 +51,7 @@ $CLICKHOUSE_CURL -sS "$CLICKHOUSE_URL" -d "KILL QUERY WHERE query_id = '${CLICKH
wait
wait_for_queries_to_finish
-${CLICKHOUSE_CLIENT} --query_id="${CLICKHOUSE_DATABASE}42" --query='SELECT 3, count() FROM system.numbers' 2>&1 | grep -cF 'QUERY_WAS_CANCELLED' &
+${CLICKHOUSE_CLIENT} --query_id="${CLICKHOUSE_DATABASE}42" --max_rows_to_read=0 --query='SELECT 3, count() FROM system.numbers' 2>&1 | grep -cF 'QUERY_WAS_CANCELLED' &
wait_for_query_to_start "${CLICKHOUSE_DATABASE}42"
${CLICKHOUSE_CLIENT} --query_id="${CLICKHOUSE_DATABASE}42" --replace_running_query=1 --replace_running_query_max_wait_ms=500 --query='SELECT 43' 2>&1 | grep -F "can't be stopped" > /dev/null
wait
diff --git a/tests/queries/0_stateless/00601_kill_running_query.reference b/tests/queries/0_stateless/00601_kill_running_query.reference
index 3917ff89482..7824d5804bc 100644
--- a/tests/queries/0_stateless/00601_kill_running_query.reference
+++ b/tests/queries/0_stateless/00601_kill_running_query.reference
@@ -1 +1 @@
-waiting test_00601_default default SELECT sum(ignore(*)) FROM (SELECT number % 1000 AS k, groupArray(number) FROM numbers(50000000) GROUP BY k)
+waiting test_00601_default default SELECT sum(ignore(*)) FROM (SELECT number % 1000 AS k, groupArray(number) FROM numbers(50000000) GROUP BY k) SETTINGS max_rows_to_read = 0
diff --git a/tests/queries/0_stateless/00601_kill_running_query.sh b/tests/queries/0_stateless/00601_kill_running_query.sh
index 3163f8146d0..be0fff49129 100755
--- a/tests/queries/0_stateless/00601_kill_running_query.sh
+++ b/tests/queries/0_stateless/00601_kill_running_query.sh
@@ -11,7 +11,7 @@ function wait_for_query_to_start()
while [[ $($CLICKHOUSE_CURL -sS "$CLICKHOUSE_URL" -d "SELECT count() FROM system.processes WHERE query_id = '$1'") == 0 ]]; do sleep 0.1; done
}
-${CLICKHOUSE_CURL_COMMAND} -q --max-time 30 -sS "$CLICKHOUSE_URL&query_id=test_00601_$CLICKHOUSE_DATABASE" -d 'SELECT sum(ignore(*)) FROM (SELECT number % 1000 AS k, groupArray(number) FROM numbers(50000000) GROUP BY k)' > /dev/null &
+${CLICKHOUSE_CURL_COMMAND} -q --max-time 30 -sS "$CLICKHOUSE_URL&query_id=test_00601_$CLICKHOUSE_DATABASE" -d 'SELECT sum(ignore(*)) FROM (SELECT number % 1000 AS k, groupArray(number) FROM numbers(50000000) GROUP BY k) SETTINGS max_rows_to_read = 0' > /dev/null &
wait_for_query_to_start "test_00601_$CLICKHOUSE_DATABASE"
$CLICKHOUSE_CURL -sS "$CLICKHOUSE_URL" -d "KILL QUERY WHERE query_id = 'test_00601_$CLICKHOUSE_DATABASE'"
wait
diff --git a/tests/queries/0_stateless/00632_get_sample_block_cache.sql b/tests/queries/0_stateless/00632_get_sample_block_cache.sql
index ae9b6bb7b2c..a631cbb8b86 100644
--- a/tests/queries/0_stateless/00632_get_sample_block_cache.sql
+++ b/tests/queries/0_stateless/00632_get_sample_block_cache.sql
@@ -2,6 +2,9 @@
SET joined_subquery_requires_alias = 0;
+-- We are no longer interested in the old analyzer.
+SET allow_experimental_analyzer = 1;
+
-- This test (SELECT) without cache can take tens minutes
DROP TABLE IF EXISTS dict_string;
DROP TABLE IF EXISTS dict_ui64;
@@ -41,8 +44,6 @@ SETTINGS index_granularity = 8192;
CREATE TABLE dict_string (entityIri String) ENGINE = Memory;
CREATE TABLE dict_ui64 (learnerId UInt64) ENGINE = Memory;
---SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count`, `overall-watchers-count`, `overall-full-watched-learners-count`, `views-count`, `time-before-full-watched-average`, if (isNaN((`overall-full-watched-learners-count`/`overall-watchers-count`) * 100), 0, (`overall-full-watched-learners-count`/`overall-watchers-count`) * 100) as `overall-watched-part`, if (isNaN((`full-watched-learners-count`/`watchers-count` * 100)), 0, (`full-watched-learners-count`/`watchers-count` * 100)) as `full-watched-part`, if (isNaN((`rejects-count`/`views-count` * 100)), 0, (`rejects-count`/`views-count` * 100)) as `rejects-part` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count`, `overall-watchers-count`, `overall-full-watched-learners-count`, `views-count`, `time-before-full-watched-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count`, `overall-watchers-count`, `overall-full-watched-learners-count`, `views-count` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count`, `overall-watchers-count`, `overall-full-watched-learners-count` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count`, `overall-watchers-count` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average` FROM (SELECT `entityIri`, `watchers-count` FROM (SELECT `entityIri` FROM `CloM8CwMR2`) ANY LEFT JOIN (SELECT uniq(learnerId) as `watchers-count`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(viewDurationSum) as `time-repeating-average`, `entityIri` FROM (SELECT sum(views.viewDuration) as viewDurationSum, `entityIri`, `learnerId` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `views`.`repeatingView` = 1 AND `learnerId` IN `tkRpHxGqM1` GROUP BY `learnerId`, `entityIri`) GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(views.viewDuration) as `reject-views-duration-average`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `views`.`reject` = 1 AND `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(viewsCount) as `repeating-views-count-average`, `entityIri` FROM (SELECT count() as viewsCount, `learnerId`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `courseId` = 1 AND `entityIri` IN `CloM8CwMR2` WHERE `views`.`repeatingView` = 1 AND `learnerId` IN `tkRpHxGqM1` GROUP BY `learnerId`, `entityIri`) GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(views.viewDuration) as `views-duration-average`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(views.watchedPart) as `watched-part-average`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT count() as `rejects-count`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `views`.`reject` = 1 AND `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(progressMax) as `progress-average`, `entityIri` FROM (SELECT max(views.progress) as progressMax, `entityIri`, `learnerId` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `learnerId` IN `tkRpHxGqM1` GROUP BY `learnerId`, `entityIri`) GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(fullWatchedViews) as `views-count-before-full-watched-average`, `entityIri` FROM `video_views` FINAL PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT any(duration) as `duration`, `entityIri` FROM `video_views` FINAL PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT uniq(learnerId) as `full-watched-learners-count`, `entityIri` FROM `video_views` FINAL PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `fullWatched` = 1 AND `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT uniq(learnerId) as `overall-watchers-count`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT uniq(learnerId) as `overall-full-watched-learners-count`, `entityIri` FROM `video_views` FINAL PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `fullWatched` = 1 AND `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT count() as `views-count`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(fullWatchedTime) as `time-before-full-watched-average`, `entityIri` FROM `video_views` FINAL PREWHERE `entityIri` IN `CloM8CwMR2` AND `courseId` = 1 WHERE `learnerId` IN `tkRpHxGqM1` GROUP BY `entityIri`) USING `entityIri`) FORMAT JSON;
-
SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count`, `overall-watchers-count`, `overall-full-watched-learners-count`, `views-count`, `time-before-full-watched-average`, if (isNaN((`overall-full-watched-learners-count`/`overall-watchers-count`) * 100), 0, (`overall-full-watched-learners-count`/`overall-watchers-count`) * 100) as `overall-watched-part`, if (isNaN((`full-watched-learners-count`/`watchers-count` * 100)), 0, (`full-watched-learners-count`/`watchers-count` * 100)) as `full-watched-part`, if (isNaN((`rejects-count`/`views-count` * 100)), 0, (`rejects-count`/`views-count` * 100)) as `rejects-part` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count`, `overall-watchers-count`, `overall-full-watched-learners-count`, `views-count`, `time-before-full-watched-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count`, `overall-watchers-count`, `overall-full-watched-learners-count`, `views-count` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count`, `overall-watchers-count`, `overall-full-watched-learners-count` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count`, `overall-watchers-count` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration`, `full-watched-learners-count` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average`, `duration` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average`, `views-count-before-full-watched-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count`, `progress-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average`, `rejects-count` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average`, `watched-part-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average`, `repeating-views-count-average`, `views-duration-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`,
`reject-views-duration-average`, `repeating-views-count-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average`, `reject-views-duration-average` FROM (SELECT `entityIri`, `watchers-count`, `time-repeating-average` FROM (SELECT `entityIri`, `watchers-count` FROM (SELECT `entityIri` FROM dict_string) ANY LEFT JOIN (SELECT uniq(learnerId) as `watchers-count`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(viewDurationSum) as `time-repeating-average`, `entityIri` FROM (SELECT sum(views.viewDuration) as viewDurationSum, `entityIri`, `learnerId` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `views`.`repeatingView` = 1 AND `learnerId` IN dict_ui64 GROUP BY `learnerId`, `entityIri`) GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(views.viewDuration) as `reject-views-duration-average`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `views`.`reject` = 1 AND `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(viewsCount) as `repeating-views-count-average`, `entityIri` FROM (SELECT count() as viewsCount, `learnerId`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `courseId` = 1 AND `entityIri` IN dict_string WHERE `views`.`repeatingView` = 1 AND `learnerId` IN dict_ui64 GROUP BY `learnerId`, `entityIri`) GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(views.viewDuration) as `views-duration-average`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(views.watchedPart) as `watched-part-average`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT count() as `rejects-count`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `views`.`reject` = 1 AND `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(progressMax) as `progress-average`, `entityIri` FROM (SELECT max(views.progress) as progressMax, `entityIri`, `learnerId` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `learnerId` IN dict_ui64 GROUP BY `learnerId`, `entityIri`) GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(fullWatchedViews) as `views-count-before-full-watched-average`, `entityIri` FROM `video_views` FINAL PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT any(duration) as `duration`, `entityIri` FROM `video_views` FINAL PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT uniq(learnerId) as `full-watched-learners-count`, `entityIri` FROM `video_views` FINAL PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `fullWatched` = 1 AND `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT uniq(learnerId) as `overall-watchers-count`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT uniq(learnerId) as `overall-full-watched-learners-count`,
`entityIri` FROM `video_views` FINAL PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `fullWatched` = 1 AND `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT count() as `views-count`, `entityIri` FROM `video_views` FINAL ARRAY JOIN `views` PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`) ANY LEFT JOIN (SELECT avg(fullWatchedTime) as `time-before-full-watched-average`, `entityIri` FROM `video_views` FINAL PREWHERE `entityIri` IN dict_string AND `courseId` = 1 WHERE `learnerId` IN dict_ui64 GROUP BY `entityIri`) USING `entityIri`);
@@ -55,8 +56,8 @@ DROP TABLE video_views;
--- Test for tsan: Ensure cache used from one thread
-SET max_threads = 32;
+-- Test for tsan: Ensure cache is used from one thread
+SET max_threads = 32, max_memory_usage = '10G';
DROP TABLE IF EXISTS sample_00632;
@@ -173,7 +174,6 @@ FROM
UNION ALL SELECT * FROM ( SELECT * FROM sample_00632 WHERE x > 0 )
)
GROUP BY x
- --HAVING c = 1
ORDER BY x ASC
);
DROP TABLE sample_00632;
diff --git a/tests/queries/0_stateless/00761_lower_utf8_bug.sql b/tests/queries/0_stateless/00761_lower_utf8_bug.sql
index de20b894331..a0ab55edc15 100644
--- a/tests/queries/0_stateless/00761_lower_utf8_bug.sql
+++ b/tests/queries/0_stateless/00761_lower_utf8_bug.sql
@@ -1 +1,4 @@
+-- Tags: no-fasttest
+-- no-fasttest: upper/lowerUTF8 use ICU
+
SELECT lowerUTF8('\xF0') = lowerUTF8('\xF0');
diff --git a/tests/queries/0_stateless/00804_test_deflate_qpl_codec_compression.reference b/tests/queries/0_stateless/00804_test_deflate_qpl_codec_compression.reference
index a2178f5eda7..a6e03404f2b 100644
--- a/tests/queries/0_stateless/00804_test_deflate_qpl_codec_compression.reference
+++ b/tests/queries/0_stateless/00804_test_deflate_qpl_codec_compression.reference
@@ -1,4 +1,4 @@
-CREATE TABLE default.compression_codec\n(\n `id` UInt64 CODEC(DEFLATE_QPL),\n `data` String CODEC(DEFLATE_QPL),\n `ddd` Date CODEC(DEFLATE_QPL),\n `ddd32` Date32 CODEC(DEFLATE_QPL),\n `somenum` Float64 CODEC(DEFLATE_QPL),\n `somestr` FixedString(3) CODEC(DEFLATE_QPL),\n `othernum` Int64 CODEC(DEFLATE_QPL),\n `somearray` Array(UInt8) CODEC(DEFLATE_QPL),\n `somemap` Map(String, UInt32) CODEC(DEFLATE_QPL),\n `sometuple` Tuple(UInt16, UInt64) CODEC(DEFLATE_QPL)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS index_granularity = 8192
+CREATE TABLE default.compression_codec\n(\n `id` UInt64 CODEC(DEFLATE_QPL),\n `data` String CODEC(DEFLATE_QPL),\n `ddd` Date CODEC(DEFLATE_QPL),\n `ddd32` Date32 CODEC(DEFLATE_QPL),\n `somenum` Float64 CODEC(DEFLATE_QPL),\n `somestr` FixedString(3) CODEC(DEFLATE_QPL),\n `othernum` Int64 CODEC(DEFLATE_QPL),\n `somearray` Array(UInt8) CODEC(DEFLATE_QPL),\n `somemap` Map(String, UInt32) CODEC(DEFLATE_QPL),\n `sometuple` Tuple(\n UInt16,\n UInt64) CODEC(DEFLATE_QPL)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS index_granularity = 8192
1 hello 2018-12-14 2018-12-14 1.1 aaa 5 [1,2,3] {'k1':1,'k2':2} (1,2)
2 world 2018-12-15 2018-12-15 2.2 bbb 6 [4,5,6] {'k3':3,'k4':4} (3,4)
3 ! 2018-12-16 2018-12-16 3.3 ccc 7 [7,8,9] {'k5':5,'k6':6} (5,6)
diff --git a/tests/queries/0_stateless/00804_test_zstd_qat_codec_compression.reference b/tests/queries/0_stateless/00804_test_zstd_qat_codec_compression.reference
index 31a4360469f..ff70403ce7a 100644
--- a/tests/queries/0_stateless/00804_test_zstd_qat_codec_compression.reference
+++ b/tests/queries/0_stateless/00804_test_zstd_qat_codec_compression.reference
@@ -1,4 +1,4 @@
-CREATE TABLE default.compression_codec\n(\n `id` UInt64 CODEC(ZSTD_QAT(1)),\n `data` String CODEC(ZSTD_QAT(1)),\n `ddd` Date CODEC(ZSTD_QAT(1)),\n `ddd32` Date32 CODEC(ZSTD_QAT(1)),\n `somenum` Float64 CODEC(ZSTD_QAT(1)),\n `somestr` FixedString(3) CODEC(ZSTD_QAT(1)),\n `othernum` Int64 CODEC(ZSTD_QAT(1)),\n `somearray` Array(UInt8) CODEC(ZSTD_QAT(1)),\n `somemap` Map(String, UInt32) CODEC(ZSTD_QAT(1)),\n `sometuple` Tuple(UInt16, UInt64) CODEC(ZSTD_QAT(1))\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS index_granularity = 8192
+CREATE TABLE default.compression_codec\n(\n `id` UInt64 CODEC(ZSTD_QAT(1)),\n `data` String CODEC(ZSTD_QAT(1)),\n `ddd` Date CODEC(ZSTD_QAT(1)),\n `ddd32` Date32 CODEC(ZSTD_QAT(1)),\n `somenum` Float64 CODEC(ZSTD_QAT(1)),\n `somestr` FixedString(3) CODEC(ZSTD_QAT(1)),\n `othernum` Int64 CODEC(ZSTD_QAT(1)),\n `somearray` Array(UInt8) CODEC(ZSTD_QAT(1)),\n `somemap` Map(String, UInt32) CODEC(ZSTD_QAT(1)),\n `sometuple` Tuple(\n UInt16,\n UInt64) CODEC(ZSTD_QAT(1))\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS index_granularity = 8192
1 hello 2018-12-14 2018-12-14 1.1 aaa 5 [1,2,3] {'k1':1,'k2':2} (1,2)
2 world 2018-12-15 2018-12-15 2.2 bbb 6 [4,5,6] {'k3':3,'k4':4} (3,4)
3 ! 2018-12-16 2018-12-16 3.3 ccc 7 [7,8,9] {'k5':5,'k6':6} (5,6)
diff --git a/tests/queries/0_stateless/00834_cancel_http_readonly_queries_on_client_close.sh b/tests/queries/0_stateless/00834_cancel_http_readonly_queries_on_client_close.sh
index 5c21c70e06a..dd3735f27b1 100755
--- a/tests/queries/0_stateless/00834_cancel_http_readonly_queries_on_client_close.sh
+++ b/tests/queries/0_stateless/00834_cancel_http_readonly_queries_on_client_close.sh
@@ -5,7 +5,7 @@ CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
. "$CURDIR"/../shell_config.sh
-${CLICKHOUSE_CURL} --max-time 1 -sS "${CLICKHOUSE_URL}&query_id=cancel_http_readonly_queries_on_client_close&cancel_http_readonly_queries_on_client_close=1&query=SELECT+count()+FROM+system.numbers" 2>&1 | grep -cF 'curl: (28)'
+${CLICKHOUSE_CURL} --max-time 1 -sS "${CLICKHOUSE_URL}&query_id=cancel_http_readonly_queries_on_client_close&cancel_http_readonly_queries_on_client_close=1&max_rows_to_read=0&query=SELECT+count()+FROM+system.numbers" 2>&1 | grep -cF 'curl: (28)'
i=0 retries=300
while [[ $i -lt $retries ]]; do
diff --git a/tests/queries/0_stateless/00857_global_joinsavel_table_alias.sql b/tests/queries/0_stateless/00857_global_joinsavel_table_alias.sql
index 2044a9b8d22..092b071cb48 100644
--- a/tests/queries/0_stateless/00857_global_joinsavel_table_alias.sql
+++ b/tests/queries/0_stateless/00857_global_joinsavel_table_alias.sql
@@ -1,4 +1,3 @@
-
DROP TABLE IF EXISTS local_table;
DROP TABLE IF EXISTS other_table;
diff --git a/tests/queries/0_stateless/00906_low_cardinality_cache.sql b/tests/queries/0_stateless/00906_low_cardinality_cache.sql
index 55eacd0db44..337fba865fd 100644
--- a/tests/queries/0_stateless/00906_low_cardinality_cache.sql
+++ b/tests/queries/0_stateless/00906_low_cardinality_cache.sql
@@ -1,3 +1,6 @@
+-- Tags: long
+
+SET max_rows_to_read = '100M';
drop table if exists lc_00906;
create table lc_00906 (b LowCardinality(String)) engine=MergeTree order by b SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi';
insert into lc_00906 select '0123456789' from numbers(100000000);
diff --git a/tests/queries/0_stateless/00974_query_profiler.sql b/tests/queries/0_stateless/00974_query_profiler.sql
index cd2f65eb94a..1f90641726f 100644
--- a/tests/queries/0_stateless/00974_query_profiler.sql
+++ b/tests/queries/0_stateless/00974_query_profiler.sql
@@ -17,6 +17,7 @@ SELECT count() > 0 FROM system.trace_log t WHERE query_id = (SELECT query_id FRO
SET query_profiler_real_time_period_ns = 0;
SET query_profiler_cpu_time_period_ns = 1000000;
SET log_queries = 1;
+SET max_rows_to_read = 0;
SELECT count(), ignore('test cpu time query profiler') FROM numbers_mt(10000000000);
SET log_queries = 0;
SYSTEM FLUSH LOGS;
diff --git a/tests/queries/0_stateless/00976_max_execution_speed.sql b/tests/queries/0_stateless/00976_max_execution_speed.sql
index 52c3f05ff43..41374712724 100644
--- a/tests/queries/0_stateless/00976_max_execution_speed.sql
+++ b/tests/queries/0_stateless/00976_max_execution_speed.sql
@@ -1,2 +1,2 @@
-SET max_execution_speed = 1, max_execution_time = 3;
+SET max_execution_speed = 1, max_execution_time = 3, max_rows_to_read = 0;
SELECT count() FROM system.numbers; -- { serverError TIMEOUT_EXCEEDED }
diff --git a/tests/queries/0_stateless/01010_pmj_right_table_memory_limits.sql b/tests/queries/0_stateless/01010_pmj_right_table_memory_limits.sql
index a090be85221..b8f2596f3d5 100644
--- a/tests/queries/0_stateless/01010_pmj_right_table_memory_limits.sql
+++ b/tests/queries/0_stateless/01010_pmj_right_table_memory_limits.sql
@@ -1,5 +1,7 @@
-- Tags: no-parallel, no-fasttest, no-random-settings
+SET max_bytes_in_join = 0;
+SET max_rows_in_join = 0;
SET max_memory_usage = 32000000;
SET join_on_disk_max_files_to_merge = 4;
diff --git a/tests/queries/0_stateless/01017_uniqCombined_memory_usage.sql b/tests/queries/0_stateless/01017_uniqCombined_memory_usage.sql
index c13a0859183..eca370d94af 100644
--- a/tests/queries/0_stateless/01017_uniqCombined_memory_usage.sql
+++ b/tests/queries/0_stateless/01017_uniqCombined_memory_usage.sql
@@ -7,7 +7,8 @@
-- sizeof(HLL) is (2^K * 6 / 8)
-- hence max_memory_usage for 100 rows = (96<<10)*100 = 9830400
-SET use_uncompressed_cache = 0;
+SET use_uncompressed_cache = 0;
+SET memory_profiler_step = 1;
-- HashTable for UInt32 (used until (1<<13) elements), hence 8192 elements
SELECT 'UInt32';
@@ -31,14 +32,14 @@ SELECT 'K=16';
SELECT 'UInt32';
SET max_memory_usage = 2000000;
SELECT sum(u) FROM (SELECT intDiv(number, 4096) AS k, uniqCombined(16)(number % 4096) u FROM numbers(4096 * 100) GROUP BY k); -- { serverError MEMORY_LIMIT_EXCEEDED }
-SET max_memory_usage = 4915200;
+SET max_memory_usage = 5230000;
SELECT sum(u) FROM (SELECT intDiv(number, 4096) AS k, uniqCombined(16)(number % 4096) u FROM numbers(4096 * 100) GROUP BY k);
-- HashTable for UInt64 (used until (1<<11) elements), hence 2048 elements
SELECT 'UInt64';
SET max_memory_usage = 2000000;
SELECT sum(u) FROM (SELECT intDiv(number, 2048) AS k, uniqCombined(16)(reinterpretAsString(number % 2048)) u FROM numbers(2048 * 100) GROUP BY k); -- { serverError MEMORY_LIMIT_EXCEEDED }
-SET max_memory_usage = 4915200;
+SET max_memory_usage = 5900000;
SELECT sum(u) FROM (SELECT intDiv(number, 2048) AS k, uniqCombined(16)(reinterpretAsString(number % 2048)) u FROM numbers(2048 * 100) GROUP BY k);
SELECT 'K=18';
diff --git a/tests/queries/0_stateless/01091_query_profiler_does_not_hang.sql b/tests/queries/0_stateless/01091_query_profiler_does_not_hang.sql
index 21a84bdd691..45f1a00ae23 100644
--- a/tests/queries/0_stateless/01091_query_profiler_does_not_hang.sql
+++ b/tests/queries/0_stateless/01091_query_profiler_does_not_hang.sql
@@ -1,4 +1,4 @@
-- Tags: no-tsan, no-asan, no-ubsan, no-msan, no-debug
-SET query_profiler_cpu_time_period_ns = 1;
+SET query_profiler_cpu_time_period_ns = 1, max_rows_to_read = 0;
SELECT count() FROM numbers_mt(1000000000);
diff --git a/tests/queries/0_stateless/01119_optimize_trivial_insert_select.sql b/tests/queries/0_stateless/01119_optimize_trivial_insert_select.sql
index a53b60a5ad3..2b301d7aced 100644
--- a/tests/queries/0_stateless/01119_optimize_trivial_insert_select.sql
+++ b/tests/queries/0_stateless/01119_optimize_trivial_insert_select.sql
@@ -1,8 +1,9 @@
drop table if exists t;
create table t(n int, a Int64, s String) engine = MergeTree() order by a;
-set enable_positional_arguments=0;
-set optimize_trivial_insert_select=1;
+set enable_positional_arguments = 0;
+set optimize_trivial_insert_select = 1;
+set max_rows_to_read = 0;
-- due to aggregate functions, optimize_trivial_insert_select will not be applied
insert into t select 1, sum(number) as c, getSetting('max_threads') from numbers_mt(100000000) settings max_insert_threads=4, max_threads=2;
diff --git a/tests/queries/0_stateless/01119_session_log.sql b/tests/queries/0_stateless/01119_session_log.sh
old mode 100644
new mode 100755
similarity index 74%
rename from tests/queries/0_stateless/01119_session_log.sql
rename to tests/queries/0_stateless/01119_session_log.sh
index 55f6228797a..2d17b545276
--- a/tests/queries/0_stateless/01119_session_log.sql
+++ b/tests/queries/0_stateless/01119_session_log.sh
@@ -1,5 +1,20 @@
--- Tags: no-fasttest
+#!/usr/bin/env bash
+# Tags: no-fasttest
+CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+# shellcheck source=../shell_config.sh
+. "$CURDIR"/../shell_config.sh
+
+
+session_log_query_prefix="
+system flush logs;
+select distinct type, user, auth_type, toString(client_address)!='::ffff:0.0.0.0' as a, client_port!=0 as b, interface from system.session_log
+where user in ('default', 'nonexistsnt_user_1119', ' ', ' INTERSERVER SECRET ')
+and interface in ('HTTP', 'TCP', 'TCP_Interserver')
+and (user != 'default' or (a=1 and b=1)) -- FIXME: we should not write uninitialized address and port (but we do sometimes)
+and event_time >= now() - interval 5 minute"
+
+$CLICKHOUSE_CLIENT -nm -q "
select * from remote('127.0.0.2', system, one, 'default', '');
select * from remote('127.0.0.2', system, one, 'default', 'wrong password'); -- { serverError AUTHENTICATION_FAILED }
select * from remote('127.0.0.2', system, one, 'nonexistsnt_user_1119', ''); -- { serverError AUTHENTICATION_FAILED }
@@ -16,9 +31,17 @@ select * from url('http://127.0.0.1:8123/?query=select+1&user=+++', LineAsString
select * from cluster('test_cluster_interserver_secret', system, one);
-system flush logs;
-select distinct type, user, auth_type, toString(client_address)!='::ffff:0.0.0.0' as a, client_port!=0 as b, interface from system.session_log
-where user in ('default', 'nonexistsnt_user_1119', ' ', ' INTERSERVER SECRET ')
-and interface in ('HTTP', 'TCP', 'TCP_Interserver')
-and (user != 'default' or (a=1 and b=1)) -- FIXME: we should not write uninitialized address and port (but we do sometimes)
-and event_time >= now() - interval 5 minute order by type, user, interface;
+$session_log_query_prefix and type != 'Logout' order by type, user, interface;
+"
+
+# Wait for logout events.
+for _ in {1..10}
+do
+ if [ "`$CLICKHOUSE_CLIENT -q "$session_log_query_prefix and type = 'Logout'" | wc -l`" -eq 3 ]
+ then
+ break
+ fi
+ sleep 2
+done
+
+$CLICKHOUSE_CLIENT -q "$session_log_query_prefix and type = 'Logout' order by user, interface"
diff --git a/tests/queries/0_stateless/01161_all_system_tables.sh b/tests/queries/0_stateless/01161_all_system_tables.sh
index 739df782a39..d4a80d074dc 100755
--- a/tests/queries/0_stateless/01161_all_system_tables.sh
+++ b/tests/queries/0_stateless/01161_all_system_tables.sh
@@ -19,7 +19,7 @@ function run_selects()
thread_num=$1
readarray -t tables_arr < <(${CLICKHOUSE_CLIENT} -q "SELECT database || '.' || name FROM system.tables
WHERE database in ('system', 'information_schema', 'INFORMATION_SCHEMA') and name != 'zookeeper' and name != 'models'
- AND sipHash64(name || toString($RAND)) % $THREADS = $thread_num AND name NOT LIKE '%\\_sender' AND name NOT LIKE '%\\_watcher'")
+ AND sipHash64(name || toString($RAND)) % $THREADS = $thread_num AND name NOT LIKE '%\\_sender' AND name NOT LIKE '%\\_watcher' AND name != 'coverage_log'")
for t in "${tables_arr[@]}"
do
diff --git a/tests/queries/0_stateless/01191_rename_dictionary.sql b/tests/queries/0_stateless/01191_rename_dictionary.sql
index c5012dabc81..be95e5a7d4b 100644
--- a/tests/queries/0_stateless/01191_rename_dictionary.sql
+++ b/tests/queries/0_stateless/01191_rename_dictionary.sql
@@ -27,6 +27,7 @@ RENAME DICTIONARY test_01191.t TO test_01191.dict1; -- {serverError INCORRECT_QU
DROP DICTIONARY test_01191.t; -- {serverError INCORRECT_QUERY}
DROP TABLE test_01191.t;
+DROP DATABASE IF EXISTS dummy_db;
CREATE DATABASE dummy_db ENGINE=Atomic;
RENAME DICTIONARY test_01191.dict TO dummy_db.dict1;
RENAME DICTIONARY dummy_db.dict1 TO test_01191.dict;
diff --git a/tests/queries/0_stateless/01245_limit_infinite_sources.sql b/tests/queries/0_stateless/01245_limit_infinite_sources.sql
index 05680d86a33..69c93baf8a8 100644
--- a/tests/queries/0_stateless/01245_limit_infinite_sources.sql
+++ b/tests/queries/0_stateless/01245_limit_infinite_sources.sql
@@ -9,3 +9,4 @@ FROM
)
WHERE number = 1
LIMIT 1
+SETTINGS max_rows_to_read = 0;
diff --git a/tests/queries/0_stateless/01249_flush_interactive.sh b/tests/queries/0_stateless/01249_flush_interactive.sh
index 551e11c8c8d..775b7825a16 100755
--- a/tests/queries/0_stateless/01249_flush_interactive.sh
+++ b/tests/queries/0_stateless/01249_flush_interactive.sh
@@ -14,10 +14,10 @@ CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
function test()
{
- timeout 5 ${CLICKHOUSE_LOCAL} --max_execution_time 10 --query "
+ timeout 5 ${CLICKHOUSE_LOCAL} --max_execution_time 10 --max_rows_to_read 0 --query "
SELECT DISTINCT number % 5 FROM system.numbers" ||:
echo -e '---'
- timeout 5 ${CLICKHOUSE_CURL} -sS --no-buffer "${CLICKHOUSE_URL}&max_execution_time=10" --data-binary "
+ timeout 5 ${CLICKHOUSE_CURL} -sS --no-buffer "${CLICKHOUSE_URL}&max_execution_time=10&max_rows_to_read=0" --data-binary "
SELECT DISTINCT number % 5 FROM system.numbers" ||:
echo -e '---'
}
diff --git a/tests/queries/0_stateless/01278_random_string_utf8.sql b/tests/queries/0_stateless/01278_random_string_utf8.sql
index da2dc48c3e1..290d6a0c759 100644
--- a/tests/queries/0_stateless/01278_random_string_utf8.sql
+++ b/tests/queries/0_stateless/01278_random_string_utf8.sql
@@ -1,3 +1,6 @@
+-- Tags: no-fasttest
+-- no-fasttest: upper/lowerUTF8 use ICU
+
SELECT randomStringUTF8('string'); -- { serverError ILLEGAL_TYPE_OF_ARGUMENT }
SELECT lengthUTF8(randomStringUTF8(100));
SELECT toTypeName(randomStringUTF8(10));
diff --git a/tests/queries/0_stateless/01293_show_settings.reference b/tests/queries/0_stateless/01293_show_settings.reference
index 187f55697e4..c4c3473ee18 100644
--- a/tests/queries/0_stateless/01293_show_settings.reference
+++ b/tests/queries/0_stateless/01293_show_settings.reference
@@ -5,5 +5,7 @@ connect_timeout_with_failover_secure_ms Milliseconds 3000
external_storage_connect_timeout_sec UInt64 10
s3_connect_timeout_ms UInt64 1000
filesystem_prefetch_max_memory_usage UInt64 1073741824
+max_memory_usage UInt64 5000000000
+max_memory_usage_for_user UInt64 32000000000
max_untracked_memory UInt64 1048576
memory_profiler_step UInt64 1048576
diff --git a/tests/queries/0_stateless/01301_aggregate_state_exception_memory_leak.reference b/tests/queries/0_stateless/01301_aggregate_state_exception_memory_leak.reference
index b20e7415f52..6282bf366d0 100644
--- a/tests/queries/0_stateless/01301_aggregate_state_exception_memory_leak.reference
+++ b/tests/queries/0_stateless/01301_aggregate_state_exception_memory_leak.reference
@@ -1,2 +1,2 @@
-Memory limit (for query) exceeded
+Memory limit exceeded
Ok
diff --git a/tests/queries/0_stateless/01301_aggregate_state_exception_memory_leak.sh b/tests/queries/0_stateless/01301_aggregate_state_exception_memory_leak.sh
index d74092d828d..5b7cba77432 100755
--- a/tests/queries/0_stateless/01301_aggregate_state_exception_memory_leak.sh
+++ b/tests/queries/0_stateless/01301_aggregate_state_exception_memory_leak.sh
@@ -8,7 +8,7 @@ CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
start=$SECONDS
# If the memory leak exists, it will lead to OOM fairly quickly.
for _ in {1..1000}; do
- $CLICKHOUSE_CLIENT --max_memory_usage 1G <<< "SELECT uniqExactState(number) FROM system.numbers_mt GROUP BY number % 10";
+ $CLICKHOUSE_CLIENT --max_memory_usage 1G --max_rows_to_read 0 <<< "SELECT uniqExactState(number) FROM system.numbers_mt GROUP BY number % 10";
# NOTE: we cannot use timeout here since this will not guarantee that the query will be executed at least once.
# (since graceful wait of clickhouse-client had been reverted)
@@ -16,5 +16,5 @@ for _ in {1..1000}; do
if [[ $elapsed -gt 30 ]]; then
break
fi
-done 2>&1 | grep -o -F 'Memory limit (for query) exceeded' | uniq
+done 2>&1 | grep -o -P 'Memory limit .+ exceeded' | sed -r -e 's/(Memory limit)(.+)( exceeded)/\1\3/' | uniq
echo 'Ok'
diff --git a/tests/queries/0_stateless/01304_direct_io_long.sh b/tests/queries/0_stateless/01304_direct_io_long.sh
index 1241f299d94..867c37667fe 100755
--- a/tests/queries/0_stateless/01304_direct_io_long.sh
+++ b/tests/queries/0_stateless/01304_direct_io_long.sh
@@ -1,18 +1,19 @@
#!/usr/bin/env bash
-# Tags: long, no-object-storage-with-slow-build
+# Tags: long, no-object-storage-with-slow-build, no-flaky-check
+# It can be too long with ThreadFuzzer
CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
. "$CURDIR"/../shell_config.sh
-$CLICKHOUSE_CLIENT --query "
+$CLICKHOUSE_CLIENT --max_rows_to_read 50M --query "
DROP TABLE IF EXISTS bug;
CREATE TABLE bug (UserID UInt64, Date Date) ENGINE = MergeTree ORDER BY Date
SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi', merge_max_block_size = 8192;
INSERT INTO bug SELECT rand64(), '2020-06-07' FROM numbers(50000000);
OPTIMIZE TABLE bug FINAL;"
LOG="$CLICKHOUSE_TMP/err-$CLICKHOUSE_DATABASE"
-$CLICKHOUSE_BENCHMARK --iterations 10 --max_threads 100 --min_bytes_to_use_direct_io 1 <<< "SELECT sum(UserID) FROM bug PREWHERE NOT ignore(Date)" 1>/dev/null 2>"$LOG"
+$CLICKHOUSE_BENCHMARK --max_rows_to_read 51M --iterations 10 --max_threads 100 --min_bytes_to_use_direct_io 1 <<< "SELECT sum(UserID) FROM bug PREWHERE NOT ignore(Date)" 1>/dev/null 2>"$LOG"
cat "$LOG" | grep Exception
cat "$LOG" | grep Loaded
diff --git a/tests/queries/0_stateless/01401_FORMAT_SETTINGS.reference b/tests/queries/0_stateless/01401_FORMAT_SETTINGS.reference
index 22405bf1866..a8b99666654 100644
--- a/tests/queries/0_stateless/01401_FORMAT_SETTINGS.reference
+++ b/tests/queries/0_stateless/01401_FORMAT_SETTINGS.reference
@@ -1,7 +1,7 @@
1
1
1
-1
-1
+2
+1
2
2
diff --git a/tests/queries/0_stateless/01401_FORMAT_SETTINGS.sh b/tests/queries/0_stateless/01401_FORMAT_SETTINGS.sh
index b70c28422c9..173cc949500 100755
--- a/tests/queries/0_stateless/01401_FORMAT_SETTINGS.sh
+++ b/tests/queries/0_stateless/01401_FORMAT_SETTINGS.sh
@@ -13,7 +13,7 @@ ${CLICKHOUSE_CURL} -sS "${CLICKHOUSE_URL}" -d 'SELECT DISTINCT blockSize() FROM
${CLICKHOUSE_CURL} -sS "${CLICKHOUSE_URL}" -d 'SELECT DISTINCT blockSize() FROM numbers(2) FORMAT CSV SETTINGS max_block_size = 1'
# push down append
${CLICKHOUSE_CURL} -sS "${CLICKHOUSE_URL}" -d 'SELECT DISTINCT blockSize() FROM numbers(2) SETTINGS max_compress_block_size = 1 FORMAT CSV SETTINGS max_block_size = 1'
-# overwrite on push down (since these settings goes latest)
+# not overwrite on push down
${CLICKHOUSE_CURL} -sS "${CLICKHOUSE_URL}" -d 'SELECT DISTINCT blockSize() FROM numbers(2) SETTINGS max_block_size = 2 FORMAT CSV SETTINGS max_block_size = 1'
# on push-down
${CLICKHOUSE_CURL} -sS "${CLICKHOUSE_URL}" -d 'SELECT DISTINCT blockSize() FROM numbers(2) SETTINGS max_block_size = 1 FORMAT CSV'
diff --git a/tests/queries/0_stateless/01431_utf8_ubsan.reference b/tests/queries/0_stateless/01431_utf8_ubsan.reference
index c98c950d535..dc785e57851 100644
--- a/tests/queries/0_stateless/01431_utf8_ubsan.reference
+++ b/tests/queries/0_stateless/01431_utf8_ubsan.reference
@@ -1,2 +1,2 @@
-FF
-FF
+EFBFBD
+EFBFBD
diff --git a/tests/queries/0_stateless/01431_utf8_ubsan.sql b/tests/queries/0_stateless/01431_utf8_ubsan.sql
index d6a299225b1..3a28e023805 100644
--- a/tests/queries/0_stateless/01431_utf8_ubsan.sql
+++ b/tests/queries/0_stateless/01431_utf8_ubsan.sql
@@ -1,2 +1,5 @@
+-- Tags: no-fasttest
+-- no-fasttest: upper/lowerUTF8 use ICU
+
SELECT hex(lowerUTF8('\xFF'));
SELECT hex(upperUTF8('\xFF'));
diff --git a/tests/queries/0_stateless/01458_named_tuple_millin.reference b/tests/queries/0_stateless/01458_named_tuple_millin.reference
index 954dfe36563..86561570985 100644
--- a/tests/queries/0_stateless/01458_named_tuple_millin.reference
+++ b/tests/queries/0_stateless/01458_named_tuple_millin.reference
@@ -1,12 +1,16 @@
CREATE TABLE default.tuple
(
- `j` Tuple(a Int8, b String)
+ `j` Tuple(
+ a Int8,
+ b String)
)
ENGINE = Memory
j Tuple(\n a Int8,\n b String)
CREATE TABLE default.tuple
(
- `j` Tuple(a Int8, b String)
+ `j` Tuple(
+ a Int8,
+ b String)
)
ENGINE = Memory
j Tuple(\n a Int8,\n b String)
diff --git a/tests/queries/0_stateless/01485_256_bit_multiply.sql b/tests/queries/0_stateless/01485_256_bit_multiply.sql
index 5c8c47c9127..a4e99d51970 100644
--- a/tests/queries/0_stateless/01485_256_bit_multiply.sql
+++ b/tests/queries/0_stateless/01485_256_bit_multiply.sql
@@ -1,5 +1,7 @@
-- Tags: no-random-settings, no-asan, no-msan, no-tsan, no-ubsan, no-debug
+SET max_rows_to_read = '100M';
+
select count() from
(
select toInt128(number) * number x, toInt256(number) * number y from numbers_mt(100000000) where x != y
diff --git a/tests/queries/0_stateless/01504_compression_multiple_streams.reference b/tests/queries/0_stateless/01504_compression_multiple_streams.reference
index 4d3aba66526..14cdce72044 100644
--- a/tests/queries/0_stateless/01504_compression_multiple_streams.reference
+++ b/tests/queries/0_stateless/01504_compression_multiple_streams.reference
@@ -1,20 +1,20 @@
1 1 [[1]] (1,[1])
1 1 [[1]] (1,[1])
-CREATE TABLE default.columns_with_multiple_streams\n(\n `field0` Nullable(Int64) CODEC(Delta(2), LZ4),\n `field1` Nullable(UInt8) CODEC(Delta(8), LZ4),\n `field2` Array(Array(Int64)) CODEC(Delta(8), LZ4),\n `field3` Tuple(UInt32, Array(UInt64)) CODEC(T64, Default)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS min_rows_for_wide_part = 0, min_bytes_for_wide_part = 0, index_granularity = 8192
+CREATE TABLE default.columns_with_multiple_streams\n(\n `field0` Nullable(Int64) CODEC(Delta(2), LZ4),\n `field1` Nullable(UInt8) CODEC(Delta(8), LZ4),\n `field2` Array(Array(Int64)) CODEC(Delta(8), LZ4),\n `field3` Tuple(\n UInt32,\n Array(UInt64)) CODEC(T64, Default)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS min_rows_for_wide_part = 0, min_bytes_for_wide_part = 0, index_granularity = 8192
1 1 [[1]] (1,[1])
2 2 [[2]] (2,[2])
-CREATE TABLE default.columns_with_multiple_streams\n(\n `field0` Nullable(Int64) CODEC(Delta(2), LZ4),\n `field1` Nullable(UInt8) CODEC(Delta(8), LZ4),\n `field2` Array(Array(Int64)) CODEC(Delta(8), LZ4),\n `field3` Tuple(UInt32, Array(UInt64)) CODEC(Delta, Default)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS min_rows_for_wide_part = 0, min_bytes_for_wide_part = 0, index_granularity = 8192
+CREATE TABLE default.columns_with_multiple_streams\n(\n `field0` Nullable(Int64) CODEC(Delta(2), LZ4),\n `field1` Nullable(UInt8) CODEC(Delta(8), LZ4),\n `field2` Array(Array(Int64)) CODEC(Delta(8), LZ4),\n `field3` Tuple(\n UInt32,\n Array(UInt64)) CODEC(Delta, Default)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS min_rows_for_wide_part = 0, min_bytes_for_wide_part = 0, index_granularity = 8192
1 1 [[1]] (1,[1])
2 2 [[2]] (2,[2])
3 3 [[3]] (3,[3])
1 1 [[1]] (1,[1])
1 1 [[1]] (1,[1])
-CREATE TABLE default.columns_with_multiple_streams_compact\n(\n `field0` Nullable(Int64) CODEC(Delta(2), LZ4),\n `field1` Nullable(UInt8) CODEC(Delta(8), LZ4),\n `field2` Array(Array(Int64)) CODEC(Delta(8), LZ4),\n `field3` Tuple(UInt32, Array(UInt64)) CODEC(Delta, Default)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS min_rows_for_wide_part = 100000, min_bytes_for_wide_part = 100000, index_granularity = 8192
+CREATE TABLE default.columns_with_multiple_streams_compact\n(\n `field0` Nullable(Int64) CODEC(Delta(2), LZ4),\n `field1` Nullable(UInt8) CODEC(Delta(8), LZ4),\n `field2` Array(Array(Int64)) CODEC(Delta(8), LZ4),\n `field3` Tuple(\n UInt32,\n Array(UInt64)) CODEC(Delta, Default)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS min_rows_for_wide_part = 100000, min_bytes_for_wide_part = 100000, index_granularity = 8192
1 1 [[1]] (1,[1])
2 2 [[2]] (2,[2])
1 1 [[1]] (1,[1])
2 2 [[2]] (2,[2])
-CREATE TABLE default.columns_with_multiple_streams_compact\n(\n `field0` Nullable(Int64) CODEC(Delta(2), LZ4),\n `field1` Nullable(UInt8) CODEC(Delta(8), LZ4),\n `field2` Array(Array(Int64)) CODEC(Delta(8), LZ4),\n `field3` Tuple(UInt32, Array(UInt64)) CODEC(Delta, Default)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS min_rows_for_wide_part = 100000, min_bytes_for_wide_part = 100000, index_granularity = 8192
+CREATE TABLE default.columns_with_multiple_streams_compact\n(\n `field0` Nullable(Int64) CODEC(Delta(2), LZ4),\n `field1` Nullable(UInt8) CODEC(Delta(8), LZ4),\n `field2` Array(Array(Int64)) CODEC(Delta(8), LZ4),\n `field3` Tuple(\n UInt32,\n Array(UInt64)) CODEC(Delta, Default)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS min_rows_for_wide_part = 100000, min_bytes_for_wide_part = 100000, index_granularity = 8192
1 1 [[1]] (1,[1])
2 2 [[2]] (2,[2])
3 3 [[3]] (3,[3])
diff --git a/tests/queries/0_stateless/01548_create_table_compound_column_format.reference b/tests/queries/0_stateless/01548_create_table_compound_column_format.reference
index 21e31e8f034..c23cc57548b 100644
--- a/tests/queries/0_stateless/01548_create_table_compound_column_format.reference
+++ b/tests/queries/0_stateless/01548_create_table_compound_column_format.reference
@@ -7,6 +7,7 @@ ENGINE = TinyLog
CREATE TABLE test
(
`a` Int64,
- `b` Tuple(a Int64)
+ `b` Tuple(
+ a Int64)
)
ENGINE = TinyLog
diff --git a/tests/queries/0_stateless/01561_mann_whitney_scipy.python b/tests/queries/0_stateless/01561_mann_whitney_scipy.python
index 4713120287d..0f84d510933 100644
--- a/tests/queries/0_stateless/01561_mann_whitney_scipy.python
+++ b/tests/queries/0_stateless/01561_mann_whitney_scipy.python
@@ -19,7 +19,13 @@ def test_and_check(name, a, b, t_stat, p_value):
)
client.query(
"INSERT INTO mann_whitney VALUES {};".format(
- ", ".join(["({},{}), ({},{})".format(i, 0, j, 1) for i, j in zip(a, b)])
+ ", ".join(["({},{})".format(i, 0) for i in a])
+ )
+ )
+
+ client.query(
+ "INSERT INTO mann_whitney VALUES {};".format(
+ ", ".join(["({},{})".format(i, 1) for i in b])
)
)
@@ -59,6 +65,15 @@ def test_mann_whitney():
test_and_check("mannWhitneyUTest('greater')", rvs1, rvs2, s, p)
+def test_mann_whitney_skew():
+ rvs1 = [1]
+ rvs2 = [0, 2, 4]
+ s, p = stats.mannwhitneyu(rvs1, rvs2, alternative="two-sided")
+ test_and_check("mannWhitneyUTest", rvs1, rvs2, s, p)
+ test_and_check("mannWhitneyUTest('two-sided')", rvs1, rvs2, s, p)
+
+
if __name__ == "__main__":
test_mann_whitney()
+ test_mann_whitney_skew()
print("Ok.")
diff --git a/tests/queries/0_stateless/01590_countSubstrings.sql b/tests/queries/0_stateless/01590_countSubstrings.sql
index b38cbb7d188..5ec4f412d7f 100644
--- a/tests/queries/0_stateless/01590_countSubstrings.sql
+++ b/tests/queries/0_stateless/01590_countSubstrings.sql
@@ -1,3 +1,6 @@
+-- Tags: no-fasttest
+-- no-fasttest: upper/lowerUTF8 use ICU
+
--
-- countSubstrings
--
diff --git a/tests/queries/0_stateless/01603_read_with_backoff_bug.sql b/tests/queries/0_stateless/01603_read_with_backoff_bug.sql
index 8b97f3514b3..8a6fa9b7845 100644
--- a/tests/queries/0_stateless/01603_read_with_backoff_bug.sql
+++ b/tests/queries/0_stateless/01603_read_with_backoff_bug.sql
@@ -1,15 +1,17 @@
--- Tags: long, no-tsan, no-distributed-cache
--- Tag no-tsan: Too long for TSan
+-- Tags: long, no-tsan, no-msan, no-distributed-cache
+-- Too long for TSan and MSan
set enable_filesystem_cache=0;
set enable_filesystem_cache_on_write_operations=0;
+set max_rows_to_read = '30M';
+
drop table if exists t;
create table t (x UInt64, s String) engine = MergeTree order by x SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi';
INSERT INTO t SELECT
number,
if(number < (8129 * 1024), arrayStringConcat(arrayMap(x -> toString(x), range(number % 128)), ' '), '')
-FROM numbers_mt((8129 * 1024) * 3) settings max_insert_threads=8;
+FROM numbers_mt((8129 * 1024) * 3) settings max_insert_threads=8, max_rows_to_read=0;
-- optimize table t final;
diff --git a/tests/queries/0_stateless/01730_distributed_group_by_no_merge_order_by_long.sql b/tests/queries/0_stateless/01730_distributed_group_by_no_merge_order_by_long.sql
index 6625ad916e8..83a26c83005 100644
--- a/tests/queries/0_stateless/01730_distributed_group_by_no_merge_order_by_long.sql
+++ b/tests/queries/0_stateless/01730_distributed_group_by_no_merge_order_by_long.sql
@@ -1,6 +1,7 @@
-- Tags: long, distributed, no-random-settings
drop table if exists data_01730;
+SET max_rows_to_read = 0, max_result_rows = 0, max_bytes_before_external_group_by = 0;
-- does not use 127.1 due to prefer_localhost_replica
diff --git a/tests/queries/0_stateless/01825_new_type_json_ghdata.sh b/tests/queries/0_stateless/01825_new_type_json_ghdata.sh
index acb4925ce6e..33940caec29 100755
--- a/tests/queries/0_stateless/01825_new_type_json_ghdata.sh
+++ b/tests/queries/0_stateless/01825_new_type_json_ghdata.sh
@@ -1,5 +1,6 @@
#!/usr/bin/env bash
-# Tags: no-fasttest
+# Tags: no-fasttest, no-s3-storage, long
+# ^ no-s3-storage: too memory hungry
CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
@@ -8,7 +9,8 @@ CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
${CLICKHOUSE_CLIENT} -q "DROP TABLE IF EXISTS ghdata"
${CLICKHOUSE_CLIENT} -q "CREATE TABLE ghdata (data JSON) ENGINE = MergeTree ORDER BY tuple() SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi'" --allow_experimental_json_type 1
-cat $CUR_DIR/data_json/ghdata_sample.json | ${CLICKHOUSE_CLIENT} -q "INSERT INTO ghdata FORMAT JSONAsObject"
+cat $CUR_DIR/data_json/ghdata_sample.json | ${CLICKHOUSE_CLIENT} \
+ --max_memory_usage 10G --query "INSERT INTO ghdata FORMAT JSONAsObject"
${CLICKHOUSE_CLIENT} -q "SELECT count() FROM ghdata WHERE NOT ignore(*)"
@@ -16,7 +18,7 @@ ${CLICKHOUSE_CLIENT} -q \
"SELECT data.repo.name, count() AS stars FROM ghdata \
WHERE data.type = 'WatchEvent' GROUP BY data.repo.name ORDER BY stars DESC, data.repo.name LIMIT 5"
-${CLICKHOUSE_CLIENT} --allow_experimental_analyzer=1 -q \
+${CLICKHOUSE_CLIENT} --enable_analyzer=1 -q \
"SELECT data.payload.commits[].author.name AS name, count() AS c FROM ghdata \
ARRAY JOIN data.payload.commits[].author.name \
GROUP BY name ORDER BY c DESC, name LIMIT 5"
diff --git a/tests/queries/0_stateless/01825_new_type_json_ghdata_insert_select.sh b/tests/queries/0_stateless/01825_new_type_json_ghdata_insert_select.sh
index ef87034ff89..568ba2bd185 100755
--- a/tests/queries/0_stateless/01825_new_type_json_ghdata_insert_select.sh
+++ b/tests/queries/0_stateless/01825_new_type_json_ghdata_insert_select.sh
@@ -1,5 +1,6 @@
#!/usr/bin/env bash
-# Tags: no-fasttest, long
+# Tags: no-fasttest, no-s3-storage, long
+# ^ no-s3-storage: it is memory-hungry
CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
@@ -13,10 +14,10 @@ ${CLICKHOUSE_CLIENT} -q "CREATE TABLE ghdata_2 (data JSON) ENGINE = MergeTree OR
${CLICKHOUSE_CLIENT} -q "CREATE TABLE ghdata_2_string (data String) ENGINE = MergeTree ORDER BY tuple() SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi'"
${CLICKHOUSE_CLIENT} -q "CREATE TABLE ghdata_2_from_string (data JSON) ENGINE = MergeTree ORDER BY tuple() SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi'" --allow_experimental_json_type 1
-cat $CUR_DIR/data_json/ghdata_sample.json | ${CLICKHOUSE_CLIENT} -q "INSERT INTO ghdata_2 FORMAT JSONAsObject"
+cat $CUR_DIR/data_json/ghdata_sample.json | ${CLICKHOUSE_CLIENT} --max_memory_usage 10G -q "INSERT INTO ghdata_2 FORMAT JSONAsObject"
cat $CUR_DIR/data_json/ghdata_sample.json | ${CLICKHOUSE_CLIENT} -q "INSERT INTO ghdata_2_string FORMAT JSONAsString"
-${CLICKHOUSE_CLIENT} -q "INSERT INTO ghdata_2_from_string SELECT data FROM ghdata_2_string"
+${CLICKHOUSE_CLIENT} --max_memory_usage 10G -q "INSERT INTO ghdata_2_from_string SELECT data FROM ghdata_2_string"
${CLICKHOUSE_CLIENT} -q "SELECT \
(SELECT mapSort(groupUniqArrayMap(JSONAllPathsWithTypes(data))), sum(cityHash64(toString(data))) FROM ghdata_2_from_string) = \
diff --git a/tests/queries/0_stateless/01825_type_json_ghdata.sh b/tests/queries/0_stateless/01825_type_json_ghdata.sh
index 2686e2c8eb1..7e952de6c08 100755
--- a/tests/queries/0_stateless/01825_type_json_ghdata.sh
+++ b/tests/queries/0_stateless/01825_type_json_ghdata.sh
@@ -1,5 +1,6 @@
#!/usr/bin/env bash
-# Tags: no-fasttest
+# Tags: no-fasttest, no-s3-storage, long
+# ^ no-s3-storage: it is memory-hungry
CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
diff --git a/tests/queries/0_stateless/01825_type_json_ghdata_insert_select.sh b/tests/queries/0_stateless/01825_type_json_ghdata_insert_select.sh
index 711194e71a1..fc503b345d9 100755
--- a/tests/queries/0_stateless/01825_type_json_ghdata_insert_select.sh
+++ b/tests/queries/0_stateless/01825_type_json_ghdata_insert_select.sh
@@ -1,5 +1,6 @@
#!/usr/bin/env bash
-# Tags: no-fasttest
+# Tags: no-fasttest, no-s3-storage, long
+# ^ no-s3-storage: too memory hungry
CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
diff --git a/tests/queries/0_stateless/01881_aggregate_functions_versioning.reference b/tests/queries/0_stateless/01881_aggregate_functions_versioning.reference
index c30c4ca7e74..e15f312c2c8 100644
--- a/tests/queries/0_stateless/01881_aggregate_functions_versioning.reference
+++ b/tests/queries/0_stateless/01881_aggregate_functions_versioning.reference
@@ -1 +1 @@
-CREATE TABLE default.test_table\n(\n `col1` DateTime,\n `col2` Int64,\n `col3` AggregateFunction(1, sumMap, Tuple(Array(UInt8), Array(UInt8)))\n)\nENGINE = AggregatingMergeTree\nORDER BY (col1, col2)\nSETTINGS index_granularity = 8192
+CREATE TABLE default.test_table\n(\n `col1` DateTime,\n `col2` Int64,\n `col3` AggregateFunction(1, sumMap, Tuple(\n Array(UInt8),\n Array(UInt8)))\n)\nENGINE = AggregatingMergeTree\nORDER BY (col1, col2)\nSETTINGS index_granularity = 8192
diff --git a/tests/queries/0_stateless/01961_roaring_memory_tracking.sql b/tests/queries/0_stateless/01961_roaring_memory_tracking.sql
index 485c8192f69..22eb8e887f2 100644
--- a/tests/queries/0_stateless/01961_roaring_memory_tracking.sql
+++ b/tests/queries/0_stateless/01961_roaring_memory_tracking.sql
@@ -2,5 +2,5 @@
SET max_bytes_before_external_group_by = 0;
-SET max_memory_usage = '100M';
+SET max_memory_usage = '100M', max_rows_to_read = '1G';
SELECT cityHash64(rand() % 1000) as n, groupBitmapState(number) FROM numbers_mt(200000000) GROUP BY n FORMAT Null; -- { serverError MEMORY_LIMIT_EXCEEDED }
diff --git a/tests/queries/0_stateless/02003_memory_limit_in_client.sh b/tests/queries/0_stateless/02003_memory_limit_in_client.sh
index 15cacbff8c5..32e8c32f009 100755
--- a/tests/queries/0_stateless/02003_memory_limit_in_client.sh
+++ b/tests/queries/0_stateless/02003_memory_limit_in_client.sh
@@ -4,11 +4,11 @@ CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
. "$CUR_DIR"/../shell_config.sh
-$CLICKHOUSE_CLIENT --max_memory_usage_in_client=1 -q "SELECT arrayMap(x -> range(x), range(number)) FROM numbers(1000) -- { clientError MEMORY_LIMIT_EXCEEDED }"
+$CLICKHOUSE_CLIENT --max_result_bytes 0 --max_memory_usage_in_client=1 -q "SELECT arrayMap(x -> range(x), range(number)) FROM numbers(1000) -- { clientError MEMORY_LIMIT_EXCEEDED }"
$CLICKHOUSE_CLIENT --max_memory_usage_in_client=0 -q "SELECT * FROM (SELECT * FROM system.numbers LIMIT 600000) as num WHERE num.number=60000"
-$CLICKHOUSE_CLIENT --max_memory_usage_in_client='5K' -q "SELECT arrayMap(x -> range(x), range(number)) FROM numbers(1000) -- { clientError MEMORY_LIMIT_EXCEEDED }"
-$CLICKHOUSE_CLIENT --max_memory_usage_in_client='5k' -q "SELECT arrayMap(x -> range(x), range(number)) FROM numbers(1000) -- { clientError MEMORY_LIMIT_EXCEEDED }"
+$CLICKHOUSE_CLIENT --max_result_bytes 0 --max_memory_usage_in_client='5K' -q "SELECT arrayMap(x -> range(x), range(number)) FROM numbers(1000) -- { clientError MEMORY_LIMIT_EXCEEDED }"
+$CLICKHOUSE_CLIENT --max_result_bytes 0 --max_memory_usage_in_client='5k' -q "SELECT arrayMap(x -> range(x), range(number)) FROM numbers(1000) -- { clientError MEMORY_LIMIT_EXCEEDED }"
$CLICKHOUSE_CLIENT --max_memory_usage_in_client='1M' -q "SELECT * FROM (SELECT * FROM system.numbers LIMIT 600000) as num WHERE num.number=60000"
$CLICKHOUSE_CLIENT --max_memory_usage_in_client='23G' -q "SELECT * FROM (SELECT * FROM system.numbers LIMIT 600000) as num WHERE num.number=60000"
$CLICKHOUSE_CLIENT --max_memory_usage_in_client='11T' -q "SELECT * FROM (SELECT * FROM system.numbers LIMIT 600000) as num WHERE num.number=60000"
diff --git a/tests/queries/0_stateless/02021_exponential_sum.reference b/tests/queries/0_stateless/02021_exponential_sum.reference
index 5bd77479cf7..c9dcee51173 100644
--- a/tests/queries/0_stateless/02021_exponential_sum.reference
+++ b/tests/queries/0_stateless/02021_exponential_sum.reference
@@ -5,4 +5,3 @@
0.0009775171065493646
0.0009775171065493646
0.0009775171065493646
-0.0009775171065493646
diff --git a/tests/queries/0_stateless/02021_exponential_sum.sql b/tests/queries/0_stateless/02021_exponential_sum.sql
index 8ab7638029c..62ec7dcf9f1 100644
--- a/tests/queries/0_stateless/02021_exponential_sum.sql
+++ b/tests/queries/0_stateless/02021_exponential_sum.sql
@@ -6,4 +6,3 @@ WITH number % 10 = 0 AS value, number AS time SELECT exponentialMovingAverage(1)
WITH number % 10 = 0 AS value, number AS time SELECT exponentialMovingAverage(1)(value, time) AS exp_smooth FROM numbers_mt(100000);
WITH number % 10 = 0 AS value, number AS time SELECT exponentialMovingAverage(1)(value, time) AS exp_smooth FROM numbers_mt(1000000);
WITH number % 10 = 0 AS value, number AS time SELECT exponentialMovingAverage(1)(value, time) AS exp_smooth FROM numbers_mt(10000000);
-WITH number % 10 = 0 AS value, number AS time SELECT exponentialMovingAverage(1)(value, time) AS exp_smooth FROM numbers_mt(100000000);
diff --git a/tests/queries/0_stateless/02021_exponential_sum_shard.reference b/tests/queries/0_stateless/02021_exponential_sum_shard.reference
index 8453706a05a..c28e5d7a132 100644
--- a/tests/queries/0_stateless/02021_exponential_sum_shard.reference
+++ b/tests/queries/0_stateless/02021_exponential_sum_shard.reference
@@ -2,4 +2,3 @@
0.009775171065493644
0.009775171065493644
0.009775171065493644
-0.009775171065493644
diff --git a/tests/queries/0_stateless/02021_exponential_sum_shard.sql b/tests/queries/0_stateless/02021_exponential_sum_shard.sql
index 49fde0fe217..8e91637e41d 100644
--- a/tests/queries/0_stateless/02021_exponential_sum_shard.sql
+++ b/tests/queries/0_stateless/02021_exponential_sum_shard.sql
@@ -3,4 +3,3 @@ WITH number % 10 = 0 AS value, number AS time SELECT exponentialMovingAverage(1)
WITH number % 10 = 0 AS value, number AS time SELECT exponentialMovingAverage(1)(value, time) AS exp_smooth FROM remote('127.0.0.{1..10}', numbers_mt(10000));
WITH number % 10 = 0 AS value, number AS time SELECT exponentialMovingAverage(1)(value, time) AS exp_smooth FROM remote('127.0.0.{1..10}', numbers_mt(100000));
WITH number % 10 = 0 AS value, number AS time SELECT exponentialMovingAverage(1)(value, time) AS exp_smooth FROM remote('127.0.0.{1..10}', numbers_mt(1000000));
-WITH number % 10 = 0 AS value, number AS time SELECT exponentialMovingAverage(1)(value, time) AS exp_smooth FROM remote('127.0.0.{1..10}', numbers_mt(10000000));
diff --git a/tests/queries/0_stateless/02071_lower_upper_utf8_row_overlaps.reference b/tests/queries/0_stateless/02071_lower_upper_utf8_row_overlaps.reference
index a3bac432482..deabef61a88 100644
--- a/tests/queries/0_stateless/02071_lower_upper_utf8_row_overlaps.reference
+++ b/tests/queries/0_stateless/02071_lower_upper_utf8_row_overlaps.reference
@@ -5,9 +5,9 @@ insert into utf8_overlap values ('\xe2'), ('Foo⚊BarBazBam'), ('\xe2'), ('Foo
-- MONOGRAM FOR YANG
with lowerUTF8(str) as l_, upperUTF8(str) as u_, '0x' || hex(str) as h_
select length(str), if(l_ == '\xe2', h_, l_), if(u_ == '\xe2', h_, u_) from utf8_overlap format CSV;
-1,"0xE2","0xE2"
+1,"�","�"
15,"foo⚊barbazbam","FOO⚊BARBAZBAM"
-1,"0xE2","0xE2"
+1,"�","�"
15,"foo⚊barbazbam","FOO⚊BARBAZBAM"
-- NOTE: regression test for introduced bug
-- https://github.com/ClickHouse/ClickHouse/issues/42756
diff --git a/tests/queries/0_stateless/02071_lower_upper_utf8_row_overlaps.sql b/tests/queries/0_stateless/02071_lower_upper_utf8_row_overlaps.sql
index 8ca0a3f5f75..d175e0659d0 100644
--- a/tests/queries/0_stateless/02071_lower_upper_utf8_row_overlaps.sql
+++ b/tests/queries/0_stateless/02071_lower_upper_utf8_row_overlaps.sql
@@ -1,3 +1,6 @@
+-- Tags: no-fasttest
+-- no-fasttest: upper/lowerUTF8 use ICU
+
drop table if exists utf8_overlap;
create table utf8_overlap (str String) engine=Memory();
diff --git a/tests/queries/0_stateless/02117_show_create_table_system.reference b/tests/queries/0_stateless/02117_show_create_table_system.reference
index 32e8b2f4312..638a46a142f 100644
--- a/tests/queries/0_stateless/02117_show_create_table_system.reference
+++ b/tests/queries/0_stateless/02117_show_create_table_system.reference
@@ -510,9 +510,15 @@ CREATE TABLE system.parts
`rows_where_ttl_info.max` Array(DateTime),
`projections` Array(String),
`visible` UInt8,
- `creation_tid` Tuple(UInt64, UInt64, UUID),
+ `creation_tid` Tuple(
+ UInt64,
+ UInt64,
+ UUID),
`removal_tid_lock` UInt64,
- `removal_tid` Tuple(UInt64, UInt64, UUID),
+ `removal_tid` Tuple(
+ UInt64,
+ UInt64,
+ UUID),
`creation_csn` UInt64,
`removal_csn` UInt64,
`has_lightweight_delete` UInt8,
diff --git a/tests/queries/0_stateless/02122_join_group_by_timeout.sh b/tests/queries/0_stateless/02122_join_group_by_timeout.sh
index 17d8200f20c..0e89fcf56d9 100755
--- a/tests/queries/0_stateless/02122_join_group_by_timeout.sh
+++ b/tests/queries/0_stateless/02122_join_group_by_timeout.sh
@@ -17,7 +17,7 @@ fi
### Should be cancelled after 1 second and return a 159 exception (timeout)
### However, in the test, the server can be overloaded, so we assert query duration in the interval of 1 to 60 seconds.
query_id=$(random_str 12)
-$CLICKHOUSE_CLIENT --query_id "$query_id" --max_execution_time 1 -q "
+$CLICKHOUSE_CLIENT --max_result_rows 0 --max_result_bytes 0 --query_id "$query_id" --max_execution_time 1 -q "
SELECT * FROM
(
SELECT a.name as n
@@ -39,7 +39,7 @@ ${CLICKHOUSE_CURL} -q -sS "$CLICKHOUSE_URL" -d "select 'query_duration', round(q
### Should stop pulling data and return what has been generated already (return code 0)
query_id=$(random_str 12)
-$CLICKHOUSE_CLIENT --query_id "$query_id" -q "
+$CLICKHOUSE_CLIENT --max_result_rows 0 --max_result_bytes 0 --query_id "$query_id" -q "
SELECT a.name as n
FROM
(
@@ -58,7 +58,7 @@ ${CLICKHOUSE_CURL} -q -sS "$CLICKHOUSE_URL" -d "select 'query_duration', round(q
# HTTP CLIENT: As of today (02/12/21) uses PullingPipelineExecutor
### Should be cancelled after 1 second and return a 159 exception (timeout)
-${CLICKHOUSE_CURL} -q --max-time $TIMEOUT -sS "$CLICKHOUSE_URL&max_execution_time=1" -d "
+${CLICKHOUSE_CURL} -q --max-time $TIMEOUT -sS "$CLICKHOUSE_URL&max_execution_time=1&max_result_rows=0&max_result_bytes=0" -d "
SELECT * FROM
(
SELECT a.name as n
@@ -77,7 +77,7 @@ ${CLICKHOUSE_CURL} -q --max-time $TIMEOUT -sS "$CLICKHOUSE_URL&max_execution_tim
### Should stop pulling data and return what has been generated already (return code 0)
-${CLICKHOUSE_CURL} -q --max-time $TIMEOUT -sS "$CLICKHOUSE_URL" -d "
+${CLICKHOUSE_CURL} -q --max-time $TIMEOUT -sS "$CLICKHOUSE_URL&max_result_rows=0&max_result_bytes=0" -d "
SELECT a.name as n
FROM
(
diff --git a/tests/queries/0_stateless/02136_kill_scalar_queries.sh b/tests/queries/0_stateless/02136_kill_scalar_queries.sh
index c8691b62360..f8bd5a42756 100755
--- a/tests/queries/0_stateless/02136_kill_scalar_queries.sh
+++ b/tests/queries/0_stateless/02136_kill_scalar_queries.sh
@@ -10,7 +10,7 @@ function wait_for_query_to_start()
}
QUERY_1_ID="${CLICKHOUSE_DATABASE}_TEST02132KILL_QUERY1"
-(${CLICKHOUSE_CLIENT} --query_id="${QUERY_1_ID}" --query='select (SELECT max(number) from system.numbers) + 1;' 2>&1 | grep -q "Code: 394." || echo 'FAIL') &
+(${CLICKHOUSE_CLIENT} --max_rows_to_read 0 --query_id="${QUERY_1_ID}" --query='select (SELECT max(number) from system.numbers) + 1;' 2>&1 | grep -q "Code: 394." || echo 'FAIL') &
wait_for_query_to_start "${QUERY_1_ID}"
${CLICKHOUSE_CLIENT} --query="KILL QUERY WHERE query_id='${QUERY_1_ID}' SYNC"
diff --git a/tests/queries/0_stateless/02136_scalar_progress.reference b/tests/queries/0_stateless/02136_scalar_progress.reference
index 5378c52de89..b8957f78e6d 100644
--- a/tests/queries/0_stateless/02136_scalar_progress.reference
+++ b/tests/queries/0_stateless/02136_scalar_progress.reference
@@ -1,6 +1,7 @@
< X-ClickHouse-Progress: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"65505","read_bytes":"524040","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"100000","read_bytes":"800000","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000","result_rows":"0","result_bytes":"0"}
-< X-ClickHouse-Progress: {"read_rows":"100001","read_bytes":"800001","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000","result_rows":"0","result_bytes":"0"}
-< X-ClickHouse-Progress: {"read_rows":"100001","read_bytes":"800001","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000","result_rows":"1","result_bytes":"272"}
-< X-ClickHouse-Summary: {"read_rows":"100001","read_bytes":"800001","written_rows":"0","written_bytes":"0","total_rows_to_read":"100000","result_rows":"1","result_bytes":"272"}
+< X-ClickHouse-Progress: {"read_rows":"100000","read_bytes":"800000","written_rows":"0","written_bytes":"0","total_rows_to_read":"100001","result_rows":"0","result_bytes":"0"}
+< X-ClickHouse-Progress: {"read_rows":"100001","read_bytes":"800001","written_rows":"0","written_bytes":"0","total_rows_to_read":"100001","result_rows":"0","result_bytes":"0"}
+< X-ClickHouse-Progress: {"read_rows":"100001","read_bytes":"800001","written_rows":"0","written_bytes":"0","total_rows_to_read":"100001","result_rows":"1","result_bytes":"272"}
+< X-ClickHouse-Summary: {"read_rows":"100001","read_bytes":"800001","written_rows":"0","written_bytes":"0","total_rows_to_read":"100001","result_rows":"1","result_bytes":"272"}
diff --git a/tests/queries/0_stateless/02141_clickhouse_local_interactive_table.reference b/tests/queries/0_stateless/02141_clickhouse_local_interactive_table.reference
index 0bb8966cbe4..0e74c0a083e 100644
--- a/tests/queries/0_stateless/02141_clickhouse_local_interactive_table.reference
+++ b/tests/queries/0_stateless/02141_clickhouse_local_interactive_table.reference
@@ -1,2 +1,2 @@
-CREATE TABLE default.`table`\n(\n `key` String\n)\nENGINE = File(\'TSVWithNamesAndTypes\', \'/dev/null\')
-CREATE TABLE foo.`table`\n(\n `key` String\n)\nENGINE = File(\'TSVWithNamesAndTypes\', \'/dev/null\')
+CREATE TEMPORARY TABLE `table`\n(\n `key` String\n)\nENGINE = File(TSVWithNamesAndTypes, \'/dev/null\')
+CREATE TEMPORARY TABLE `table`\n(\n `key` String\n)\nENGINE = File(TSVWithNamesAndTypes, \'/dev/null\')
diff --git a/tests/queries/0_stateless/02141_clickhouse_local_interactive_table.sh b/tests/queries/0_stateless/02141_clickhouse_local_interactive_table.sh
index 934d87616ac..3a95e59416a 100755
--- a/tests/queries/0_stateless/02141_clickhouse_local_interactive_table.sh
+++ b/tests/queries/0_stateless/02141_clickhouse_local_interactive_table.sh
@@ -4,5 +4,5 @@ CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
. "$CURDIR"/../shell_config.sh
-$CLICKHOUSE_LOCAL --file /dev/null --structure "key String" --input-format TSVWithNamesAndTypes --interactive --send_logs_level=trace <<<'show create table table'
-$CLICKHOUSE_LOCAL --database foo --file /dev/null --structure "key String" --input-format TSVWithNamesAndTypes --interactive --send_logs_level=trace <<<'show create table table'
+$CLICKHOUSE_LOCAL --file /dev/null --structure "key String" --input-format TSVWithNamesAndTypes --interactive --send_logs_level=trace <<<'show create temporary table table'
+$CLICKHOUSE_LOCAL --database foo --file /dev/null --structure "key String" --input-format TSVWithNamesAndTypes --interactive --send_logs_level=trace <<<'show create temporary table table'
diff --git a/tests/queries/0_stateless/02151_lc_prefetch.sql b/tests/queries/0_stateless/02151_lc_prefetch.sql
index c2b97231145..f8c76038120 100644
--- a/tests/queries/0_stateless/02151_lc_prefetch.sql
+++ b/tests/queries/0_stateless/02151_lc_prefetch.sql
@@ -3,5 +3,6 @@ drop table if exists tab_lc;
CREATE TABLE tab_lc (x UInt64, y LowCardinality(String)) engine = MergeTree order by x SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi';
insert into tab_lc select number, toString(number % 10) from numbers(20000000);
optimize table tab_lc;
+SET max_rows_to_read = '21M';
select count() from tab_lc where y == '0' settings local_filesystem_read_prefetch=1;
drop table if exists tab_lc;
diff --git a/tests/queries/0_stateless/02161_addressToLineWithInlines.sql b/tests/queries/0_stateless/02161_addressToLineWithInlines.sql
index cf400ed34c5..d7ce133f38c 100644
--- a/tests/queries/0_stateless/02161_addressToLineWithInlines.sql
+++ b/tests/queries/0_stateless/02161_addressToLineWithInlines.sql
@@ -6,7 +6,7 @@ SELECT addressToLineWithInlines(1); -- { serverError FUNCTION_NOT_ALLOWED }
SET allow_introspection_functions = 1;
SET query_profiler_real_time_period_ns = 0;
SET query_profiler_cpu_time_period_ns = 1000000;
-SET log_queries = 1;
+SET log_queries = 1, max_rows_to_read = 0;
SELECT count() FROM numbers_mt(10000000000) SETTINGS log_comment='02161_test_case';
SET log_queries = 0;
SET query_profiler_cpu_time_period_ns = 0;
diff --git a/tests/queries/0_stateless/02177_issue_31009.sql b/tests/queries/0_stateless/02177_issue_31009.sql
index f25df59f4b4..5c62b5a9c2f 100644
--- a/tests/queries/0_stateless/02177_issue_31009.sql
+++ b/tests/queries/0_stateless/02177_issue_31009.sql
@@ -8,6 +8,8 @@ DROP TABLE IF EXISTS right;
CREATE TABLE left ( key UInt32, value String ) ENGINE = MergeTree ORDER BY key SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi';
CREATE TABLE right ( key UInt32, value String ) ENGINE = MergeTree ORDER BY tuple() SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi';
+SET max_rows_to_read = '50M';
+
INSERT INTO left SELECT number, toString(number) FROM numbers(25367182);
INSERT INTO right SELECT number, toString(number) FROM numbers(23124707);
diff --git a/tests/queries/0_stateless/02177_issue_31009_pt2.sql.j2 b/tests/queries/0_stateless/02177_issue_31009_pt2.sql.j2
index 47940356302..7df77595347 100644
--- a/tests/queries/0_stateless/02177_issue_31009_pt2.sql.j2
+++ b/tests/queries/0_stateless/02177_issue_31009_pt2.sql.j2
@@ -1,4 +1,5 @@
--- Tags: long
+-- Tags: long, no-flaky-check
+-- It can be too long with ThreadFuzzer
DROP TABLE IF EXISTS left;
DROP TABLE IF EXISTS right;
diff --git a/tests/queries/0_stateless/02226_analyzer_or_like_combine.sql b/tests/queries/0_stateless/02226_analyzer_or_like_combine.sql
index b23e5640b8f..0c150249aeb 100644
--- a/tests/queries/0_stateless/02226_analyzer_or_like_combine.sql
+++ b/tests/queries/0_stateless/02226_analyzer_or_like_combine.sql
@@ -1,3 +1,5 @@
+SET allow_hyperscan = 1, max_hyperscan_regexp_length = 0, max_hyperscan_regexp_total_length = 0;
+
EXPLAIN SYNTAX SELECT materialize('Привет, World') AS s WHERE (s LIKE 'hell%') OR (s ILIKE '%привет%') OR (s ILIKE 'world%') SETTINGS optimize_or_like_chain = 0;
EXPLAIN QUERY TREE run_passes=1 SELECT materialize('Привет, World') AS s WHERE (s LIKE 'hell%') OR (s ILIKE '%привет%') OR (s ILIKE 'world%') SETTINGS optimize_or_like_chain = 0, enable_analyzer = 1;
EXPLAIN SYNTAX SELECT materialize('Привет, World') AS s WHERE (s LIKE 'hell%') OR (s ILIKE '%привет%') OR (s ILIKE 'world%') SETTINGS optimize_or_like_chain = 1;
diff --git a/tests/queries/0_stateless/02234_cast_to_ip_address.reference b/tests/queries/0_stateless/02234_cast_to_ip_address.reference
index fa9c6bd0f94..b9f0a49ec4d 100644
--- a/tests/queries/0_stateless/02234_cast_to_ip_address.reference
+++ b/tests/queries/0_stateless/02234_cast_to_ip_address.reference
@@ -37,7 +37,7 @@ IPv6 functions
::ffff:127.0.0.1
::
\N
-100000000
+20000000
--
::ffff:127.0.0.1
--
diff --git a/tests/queries/0_stateless/02234_cast_to_ip_address.sql b/tests/queries/0_stateless/02234_cast_to_ip_address.sql
index 28f1afff57f..c851cfde927 100644
--- a/tests/queries/0_stateless/02234_cast_to_ip_address.sql
+++ b/tests/queries/0_stateless/02234_cast_to_ip_address.sql
@@ -67,11 +67,11 @@ SELECT toIPv6('::.1.2.3'); --{serverError CANNOT_PARSE_IPV6}
SELECT toIPv6OrDefault('::.1.2.3');
SELECT toIPv6OrNull('::.1.2.3');
-SELECT count() FROM numbers_mt(100000000) WHERE NOT ignore(toIPv6OrZero(randomString(8)));
+SELECT count() FROM numbers_mt(20000000) WHERE NOT ignore(toIPv6OrZero(randomString(8)));
SELECT '--';
-SELECT cast('test' , 'IPv6'); --{serverError CANNOT_PARSE_IPV6}
+SELECT cast('test' , 'IPv6'); -- { serverError CANNOT_PARSE_IPV6 }
SELECT cast('::ffff:127.0.0.1', 'IPv6');
SELECT '--';
diff --git a/tests/queries/0_stateless/02286_tuple_numeric_identifier.reference b/tests/queries/0_stateless/02286_tuple_numeric_identifier.reference
index 21348493d1d..916cdaf83cd 100644
--- a/tests/queries/0_stateless/02286_tuple_numeric_identifier.reference
+++ b/tests/queries/0_stateless/02286_tuple_numeric_identifier.reference
@@ -1,4 +1,4 @@
-CREATE TABLE default.t_tuple_numeric\n(\n `t` Tuple(`1` Tuple(`2` Int32, `3` Int32), `4` Int32)\n)\nENGINE = Memory
+CREATE TABLE default.t_tuple_numeric\n(\n `t` Tuple(\n `1` Tuple(\n `2` Int32,\n `3` Int32),\n `4` Int32)\n)\nENGINE = Memory
{"t":{"1":{"2":2,"3":3},"4":4}}
2 3 4
2 3 4
diff --git a/tests/queries/0_stateless/02293_ttest_large_samples.sql b/tests/queries/0_stateless/02293_ttest_large_samples.sql
index 14baa3fddfe..b4687541360 100644
--- a/tests/queries/0_stateless/02293_ttest_large_samples.sql
+++ b/tests/queries/0_stateless/02293_ttest_large_samples.sql
@@ -1,3 +1,5 @@
+-- Tags: long
+
SELECT roundBankers(result.1, 5), roundBankers(result.2, 5) FROM (
SELECT
studentTTest(sample, variant) as result
@@ -15,6 +17,8 @@ SELECT
FROM system.numbers limit 500000));
+SET max_rows_to_read = 0;
+
SELECT roundBankers(result.1, 5), roundBankers(result.2, 5 ) FROM (
SELECT
studentTTest(sample, variant) as result
diff --git a/tests/queries/0_stateless/02294_floating_point_second_in_settings.sh b/tests/queries/0_stateless/02294_floating_point_second_in_settings.sh
index 7a18b8fea29..27dbd3e3de6 100755
--- a/tests/queries/0_stateless/02294_floating_point_second_in_settings.sh
+++ b/tests/queries/0_stateless/02294_floating_point_second_in_settings.sh
@@ -23,16 +23,16 @@ function check_output() {
# TCP CLIENT
echo "TCP CLIENT"
-OUTPUT=$($CLICKHOUSE_CLIENT --max_execution_time $MAX_TIMEOUT -q "SELECT count() FROM system.numbers" 2>&1 || true)
+OUTPUT=$($CLICKHOUSE_CLIENT --max_rows_to_read 0 --max_execution_time $MAX_TIMEOUT -q "SELECT count() FROM system.numbers" 2>&1 || true)
check_output "${OUTPUT}"
echo "TCP CLIENT WITH SETTINGS IN QUERY"
-OUTPUT=$($CLICKHOUSE_CLIENT -q "SELECT count() FROM system.numbers SETTINGS max_execution_time=$MAX_TIMEOUT" 2>&1 || true)
+OUTPUT=$($CLICKHOUSE_CLIENT --max_rows_to_read 0 -q "SELECT count() FROM system.numbers SETTINGS max_execution_time=$MAX_TIMEOUT" 2>&1 || true)
check_output "${OUTPUT}"
# HTTP CLIENT
echo "HTTP CLIENT"
-OUTPUT=$(${CLICKHOUSE_CURL_COMMAND} -q -sS "$CLICKHOUSE_URL&max_execution_time=$MAX_TIMEOUT" -d \
+OUTPUT=$(${CLICKHOUSE_CURL_COMMAND} -q -sS "$CLICKHOUSE_URL&max_execution_time=${MAX_TIMEOUT}&max_rows_to_read=0" -d \
"SELECT count() FROM system.numbers" || true)
check_output "${OUTPUT}"
diff --git a/tests/queries/0_stateless/02343_aggregation_pipeline.reference b/tests/queries/0_stateless/02343_aggregation_pipeline.reference
index bf61eb6da0a..eb013200a17 100644
--- a/tests/queries/0_stateless/02343_aggregation_pipeline.reference
+++ b/tests/queries/0_stateless/02343_aggregation_pipeline.reference
@@ -1,6 +1,6 @@
-- { echoOn }
-explain pipeline select * from (select * from numbers(1e8) group by number) group by number;
+explain pipeline select * from (select * from numbers(1e8) group by number) group by number settings max_rows_to_read = 0;
(Expression)
ExpressionTransform × 16
(Aggregating)
@@ -16,7 +16,7 @@ ExpressionTransform × 16
ExpressionTransform
(ReadFromSystemNumbers)
NumbersRange 0 → 1
-explain pipeline select * from (select * from numbers_mt(1e8) group by number) group by number;
+explain pipeline select * from (select * from numbers_mt(1e8) group by number) group by number settings max_rows_to_read = 0;
(Expression)
ExpressionTransform × 16
(Aggregating)
@@ -32,7 +32,7 @@ ExpressionTransform × 16
ExpressionTransform × 16
(ReadFromSystemNumbers)
NumbersRange × 16 0 → 1
-explain pipeline select * from (select * from numbers_mt(1e8) group by number) order by number;
+explain pipeline select * from (select * from numbers_mt(1e8) group by number) order by number settings max_rows_to_read = 0;
(Expression)
ExpressionTransform
(Sorting)
diff --git a/tests/queries/0_stateless/02343_aggregation_pipeline.sql b/tests/queries/0_stateless/02343_aggregation_pipeline.sql
index 0f9dbd0247d..24d54293313 100644
--- a/tests/queries/0_stateless/02343_aggregation_pipeline.sql
+++ b/tests/queries/0_stateless/02343_aggregation_pipeline.sql
@@ -13,11 +13,9 @@ set allow_prefetched_read_pool_for_local_filesystem = 0;
-- { echoOn }
-explain pipeline select * from (select * from numbers(1e8) group by number) group by number;
-
-explain pipeline select * from (select * from numbers_mt(1e8) group by number) group by number;
-
-explain pipeline select * from (select * from numbers_mt(1e8) group by number) order by number;
+explain pipeline select * from (select * from numbers(1e8) group by number) group by number settings max_rows_to_read = 0;
+explain pipeline select * from (select * from numbers_mt(1e8) group by number) group by number settings max_rows_to_read = 0;
+explain pipeline select * from (select * from numbers_mt(1e8) group by number) order by number settings max_rows_to_read = 0;
explain pipeline select number from remote('127.0.0.{1,2,3}', system, numbers_mt) group by number settings distributed_aggregation_memory_efficient = 1;
diff --git a/tests/queries/0_stateless/02344_describe_cache.reference b/tests/queries/0_stateless/02344_describe_cache.reference
index 6895606eb2b..13429b14866 100644
--- a/tests/queries/0_stateless/02344_describe_cache.reference
+++ b/tests/queries/0_stateless/02344_describe_cache.reference
@@ -1,2 +1,2 @@
1
-102400 10000000 33554432 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/02344_describe_cache_test 0 5000 0 16
+102400 10000000 33554432 1 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/02344_describe_cache_test 0 5000 0 16
diff --git a/tests/queries/0_stateless/02344_describe_cache.sh b/tests/queries/0_stateless/02344_describe_cache.sh
index d91661db9bc..c5373b4d7e3 100755
--- a/tests/queries/0_stateless/02344_describe_cache.sh
+++ b/tests/queries/0_stateless/02344_describe_cache.sh
@@ -11,7 +11,7 @@ $CLICKHOUSE_CLIENT -nm --query """
DROP TABLE IF EXISTS test;
CREATE TABLE test (a Int32, b String)
ENGINE = MergeTree() ORDER BY tuple()
-SETTINGS disk = disk(name = '$disk_name', type = cache, max_size = '100Ki', path = '$disk_name', disk = 's3_disk');
+SETTINGS disk = disk(name = '$disk_name', type = cache, max_size = '100Ki', path = '$disk_name', disk = 's3_disk', load_metadata_asynchronously = 0);
"""
$CLICKHOUSE_CLIENT -nm --query """
diff --git a/tests/queries/0_stateless/02344_insert_profile_events_stress.sql b/tests/queries/0_stateless/02344_insert_profile_events_stress.sql
index e9a790bea5d..902e1da543c 100644
--- a/tests/queries/0_stateless/02344_insert_profile_events_stress.sql
+++ b/tests/queries/0_stateless/02344_insert_profile_events_stress.sql
@@ -1,4 +1,5 @@
-- Tags: no-parallel, long, no-debug, no-tsan, no-msan, no-asan
+SET max_rows_to_read = 0;
create table data_02344 (key Int) engine=Null;
-- 3e9 rows is enough to fill the socket buffer and cause INSERT hung.
diff --git a/tests/queries/0_stateless/02346_additional_filters.sql b/tests/queries/0_stateless/02346_additional_filters.sql
index f6b665713ec..5a799e1c8c1 100644
--- a/tests/queries/0_stateless/02346_additional_filters.sql
+++ b/tests/queries/0_stateless/02346_additional_filters.sql
@@ -4,6 +4,8 @@ drop table if exists table_2;
drop table if exists v_numbers;
drop table if exists mv_table;
+SET max_rows_to_read = 0;
+
create table table_1 (x UInt32, y String) engine = MergeTree order by x;
insert into table_1 values (1, 'a'), (2, 'bb'), (3, 'ccc'), (4, 'dddd');
diff --git a/tests/queries/0_stateless/02353_simdjson_buffer_overflow.sql b/tests/queries/0_stateless/02353_simdjson_buffer_overflow.sql
index b324f834053..e7c6c272102 100644
--- a/tests/queries/0_stateless/02353_simdjson_buffer_overflow.sql
+++ b/tests/queries/0_stateless/02353_simdjson_buffer_overflow.sql
@@ -2,5 +2,6 @@
SET max_execution_time = 3;
SET timeout_overflow_mode = 'break';
+SET max_rows_to_read = 0, max_bytes_to_read = 0;
SELECT count() FROM system.numbers_mt WHERE NOT ignore(JSONExtract('{' || repeat('"a":"b",', rand() % 10) || '"c":"d"}', 'a', 'String')) FORMAT Null;
diff --git a/tests/queries/0_stateless/02354_distributed_with_external_aggregation_memory_usage.sql b/tests/queries/0_stateless/02354_distributed_with_external_aggregation_memory_usage.sql
index 105fb500461..f9da5b3a73c 100644
--- a/tests/queries/0_stateless/02354_distributed_with_external_aggregation_memory_usage.sql
+++ b/tests/queries/0_stateless/02354_distributed_with_external_aggregation_memory_usage.sql
@@ -1,5 +1,7 @@
-- Tags: long, no-tsan, no-msan, no-asan, no-ubsan, no-debug, no-object-storage
+SET max_rows_to_read = '101M';
+
DROP TABLE IF EXISTS t_2354_dist_with_external_aggr;
create table t_2354_dist_with_external_aggr(a UInt64, b String, c FixedString(100)) engine = MergeTree order by tuple() SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi';
@@ -23,6 +25,6 @@ select a, b, c, sum(a) as s
from remote('127.0.0.{2,3}', currentDatabase(), t_2354_dist_with_external_aggr)
group by a, b, c
format Null
-settings max_memory_usage = '5Gi';
+settings max_memory_usage = '5Gi', max_result_rows = 0, max_result_bytes = 0;
DROP TABLE t_2354_dist_with_external_aggr;
diff --git a/tests/queries/0_stateless/02372_now_in_block.sql b/tests/queries/0_stateless/02372_now_in_block.sql
index aee4572ce8d..d0aec471801 100644
--- a/tests/queries/0_stateless/02372_now_in_block.sql
+++ b/tests/queries/0_stateless/02372_now_in_block.sql
@@ -1,3 +1,4 @@
+SET max_rows_to_read = 0, max_bytes_to_read = 0;
SELECT count() FROM (SELECT DISTINCT nowInBlock(), nowInBlock('Pacific/Pitcairn') FROM system.numbers LIMIT 2);
SELECT nowInBlock(1); -- { serverError ILLEGAL_TYPE_OF_ARGUMENT }
SELECT nowInBlock(NULL) IS NULL;
diff --git a/tests/queries/0_stateless/02415_all_new_functions_must_be_documented.reference b/tests/queries/0_stateless/02415_all_new_functions_must_be_documented.reference
index c39f1fb1ce9..0980e25b70f 100644
--- a/tests/queries/0_stateless/02415_all_new_functions_must_be_documented.reference
+++ b/tests/queries/0_stateless/02415_all_new_functions_must_be_documented.reference
@@ -416,7 +416,6 @@ logTrace
lowCardinalityIndices
lowCardinalityKeys
lower
-lowerUTF8
makeDate
makeDate32
makeDateTime
@@ -897,7 +896,6 @@ tupleToNameValuePairs
unbin
unhex
upper
-upperUTF8
uptime
validateNestedArraySizes
version
diff --git a/tests/queries/0_stateless/02450_kill_distributed_query_deadlock.sh b/tests/queries/0_stateless/02450_kill_distributed_query_deadlock.sh
index 0cd520d8d5d..96692ba325a 100755
--- a/tests/queries/0_stateless/02450_kill_distributed_query_deadlock.sh
+++ b/tests/queries/0_stateless/02450_kill_distributed_query_deadlock.sh
@@ -5,20 +5,24 @@ CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
. "$CUR_DIR"/../shell_config.sh
-# Test that running distributed query and cancel it ASAP,
-# this can trigger a hung/deadlock in ProcessorList.
-for i in {1..50}; do
+# Test that runs a distributed query and cancels it ASAP,
+# this has a chance to trigger a hung/deadlock in ProcessorList.
+for i in {1..50}
+do
query_id="$CLICKHOUSE_TEST_UNIQUE_NAME-$i"
- $CLICKHOUSE_CLIENT --format Null --query_id "$query_id" -q "select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))" 2>/dev/null &
- while :; do
+ $CLICKHOUSE_CLIENT --format Null --query_id "$query_id" --max_rows_to_read 0 --max_bytes_to_read 0 --max_result_rows 0 --max_result_bytes 0 -q "select * from remote('127.{1|2|3|4|5|6}', numbers(1e12))" 2>/dev/null &
+ while true
+ do
killed_queries="$($CLICKHOUSE_CLIENT -q "kill query where query_id = '$query_id' sync" | wc -l)"
- if [[ "$killed_queries" -ge 1 ]]; then
+ if [[ "$killed_queries" -ge 1 ]]
+ then
break
fi
done
wait -n
query_return_status=$?
- if [[ $query_return_status -eq 0 ]]; then
+ if [[ $query_return_status -eq 0 ]]
+ then
echo "Query $query_id should be cancelled, however it returns successfully"
fi
done
diff --git a/tests/queries/0_stateless/02481_parquet_list_monotonically_increasing_offsets.sh b/tests/queries/0_stateless/02481_parquet_list_monotonically_increasing_offsets.sh
index 6fd6da69b70..1027f18fc83 100755
--- a/tests/queries/0_stateless/02481_parquet_list_monotonically_increasing_offsets.sh
+++ b/tests/queries/0_stateless/02481_parquet_list_monotonically_increasing_offsets.sh
@@ -12,7 +12,7 @@ echo "Parquet"
DATA_FILE=$CUR_DIR/data_parquet/list_monotonically_increasing_offsets.parquet
${CLICKHOUSE_CLIENT} --query="DROP TABLE IF EXISTS parquet_load"
${CLICKHOUSE_CLIENT} --query="CREATE TABLE parquet_load (list Array(Int64), json Nullable(String)) ENGINE = Memory"
-cat "$DATA_FILE" | ${CLICKHOUSE_CLIENT} -q "INSERT INTO parquet_load FORMAT Parquet"
-${CLICKHOUSE_CLIENT} --query="SELECT * FROM parquet_load" | md5sum
+cat "$DATA_FILE" | ${CLICKHOUSE_CLIENT} --max_memory_usage 10G -q "INSERT INTO parquet_load FORMAT Parquet"
+${CLICKHOUSE_CLIENT} --max_result_rows 0 --max_result_bytes 0 --query="SELECT * FROM parquet_load" | md5sum
${CLICKHOUSE_CLIENT} --query="SELECT count() FROM parquet_load"
${CLICKHOUSE_CLIENT} --query="drop table parquet_load"
diff --git a/tests/queries/0_stateless/02497_remote_disk_fat_column.sql b/tests/queries/0_stateless/02497_remote_disk_fat_column.sql
index d97109b66f3..65519296602 100644
--- a/tests/queries/0_stateless/02497_remote_disk_fat_column.sql
+++ b/tests/queries/0_stateless/02497_remote_disk_fat_column.sql
@@ -2,7 +2,7 @@
set allow_suspicious_fixed_string_types=1;
create table fat_granularity (x UInt32, fat FixedString(160000)) engine = MergeTree order by x settings storage_policy = 's3_cache';
-insert into fat_granularity select number, toString(number) || '_' from numbers(100000) settings max_block_size = 8192, max_insert_threads=8;
+insert into fat_granularity select number, toString(number) || '_' from numbers(100000) settings max_block_size = 3000, max_insert_threads = 8, min_insert_block_size_rows = 0, min_insert_block_size_bytes = 0;
-- Too large sizes of FixedString to deserialize
select x from fat_granularity prewhere fat like '256\_%' settings max_threads=2;
diff --git a/tests/queries/0_stateless/02514_if_with_lazy_low_cardinality.sql b/tests/queries/0_stateless/02514_if_with_lazy_low_cardinality.sql
index 80e3c0a9ece..b169cfd0ab9 100644
--- a/tests/queries/0_stateless/02514_if_with_lazy_low_cardinality.sql
+++ b/tests/queries/0_stateless/02514_if_with_lazy_low_cardinality.sql
@@ -1,3 +1,6 @@
+-- Tags: no-fasttest
+-- no-fasttest: upper/lowerUTF8 use ICU
+
create table if not exists t (`arr.key` Array(LowCardinality(String)), `arr.value` Array(LowCardinality(String))) engine = Memory;
insert into t (`arr.key`, `arr.value`) values (['a'], ['b']);
select if(true, if(lowerUTF8(arr.key) = 'a', 1, 2), 3) as x from t left array join arr;
diff --git a/tests/queries/0_stateless/02536_delta_gorilla_corruption.sql b/tests/queries/0_stateless/02536_delta_gorilla_corruption.sql
index a4e0965e329..3accc726d08 100644
--- a/tests/queries/0_stateless/02536_delta_gorilla_corruption.sql
+++ b/tests/queries/0_stateless/02536_delta_gorilla_corruption.sql
@@ -12,7 +12,7 @@ create table bug_delta_gorilla
(value_bug UInt64 codec (Delta, Gorilla))
engine = MergeTree
order by tuple() SETTINGS index_granularity = 8192, index_granularity_bytes = '10Mi'
-as (select 0 from numbers(30000000));
+as (select 0 from numbers(20000000));
select count(*)
from bug_delta_gorilla
diff --git a/tests/queries/0_stateless/02585_query_status_deadlock.sh b/tests/queries/0_stateless/02585_query_status_deadlock.sh
index e3e34109cdb..932cf593393 100755
--- a/tests/queries/0_stateless/02585_query_status_deadlock.sh
+++ b/tests/queries/0_stateless/02585_query_status_deadlock.sh
@@ -7,8 +7,8 @@ CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
QUERY_ID="${CLICKHOUSE_DATABASE}_test_02585_query_to_kill_id_1"
-$CLICKHOUSE_CLIENT --query_id="$QUERY_ID" -n -q "
-create temporary table tmp as select * from numbers(500000000);
+$CLICKHOUSE_CLIENT --query_id="$QUERY_ID" --max_rows_to_read 0 -n -q "
+create temporary table tmp as select * from numbers(100000000);
select * from remote('127.0.0.2', 'system.numbers_mt') where number in (select * from tmp);" &> /dev/null &
$CLICKHOUSE_CLIENT -q "SYSTEM FLUSH LOGS"
@@ -19,8 +19,7 @@ do
if [ -n "$res" ]; then
break
fi
- sleep 1
+ sleep 1
done
$CLICKHOUSE_CLIENT -q "kill query where query_id = '$QUERY_ID' sync" &> /dev/null
-
diff --git a/tests/queries/0_stateless/02675_profile_events_from_query_log_and_client.sh b/tests/queries/0_stateless/02675_profile_events_from_query_log_and_client.sh
index 6d770b308b5..ff534a6a2e6 100755
--- a/tests/queries/0_stateless/02675_profile_events_from_query_log_and_client.sh
+++ b/tests/queries/0_stateless/02675_profile_events_from_query_log_and_client.sh
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
-# Tags: no-fasttest
+# Tags: no-fasttest, no-random-merge-tree-settings
# Tag no-fasttest: needs s3
CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
@@ -17,7 +17,9 @@ WITH '(\\w+): (\\d+)' AS pattern,
WHERE line LIKE '% S3%'
AND line NOT LIKE '%Microseconds%'
AND line NOT LIKE '%S3DiskConnections%'
- AND line NOT LIKE '%S3DiskAddresses') AS pe_map
+ AND line NOT LIKE '%S3DiskAddresses%'
+ AND line NOT LIKE '%RequestThrottlerCount%'
+ ) AS pe_map
SELECT * FROM (
SELECT untuple(arrayJoin(pe_map) AS pe)
WHERE tupleElement(pe, 1) not like '%WriteRequests%'
diff --git a/tests/queries/0_stateless/02697_stop_reading_on_first_cancel.sh b/tests/queries/0_stateless/02697_stop_reading_on_first_cancel.sh
index 2be13588453..5a2cec08eca 100755
--- a/tests/queries/0_stateless/02697_stop_reading_on_first_cancel.sh
+++ b/tests/queries/0_stateless/02697_stop_reading_on_first_cancel.sh
@@ -5,7 +5,7 @@ CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
QUERY_ID="${CLICKHOUSE_DATABASE}_read_with_cancel"
-$CLICKHOUSE_CLIENT -n --query_id="$QUERY_ID" --query="SELECT sum(number * 0) FROM numbers(10000000000) SETTINGS partial_result_on_first_cancel=true;" &
+$CLICKHOUSE_CLIENT --max_rows_to_read 0 -n --query_id="$QUERY_ID" --query="SELECT sum(number * 0) FROM numbers(10000000000) SETTINGS partial_result_on_first_cancel=true;" &
pid=$!
for _ in {0..60}
diff --git a/tests/queries/0_stateless/02700_s3_part_INT_MAX.sh b/tests/queries/0_stateless/02700_s3_part_INT_MAX.sh
index a34a480a078..cfb38c60615 100755
--- a/tests/queries/0_stateless/02700_s3_part_INT_MAX.sh
+++ b/tests/queries/0_stateless/02700_s3_part_INT_MAX.sh
@@ -10,7 +10,9 @@ CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# NOTE: .sh test is used over .sql because it needs $CLICKHOUSE_DATABASE to
# avoid truncation, since seems that the version of MinIO that is used on CI
# too slow with this.
-$CLICKHOUSE_CLIENT -nm -q "
+#
+# Unfortunately, the test has to buffer it in memory.
+$CLICKHOUSE_CLIENT --max_memory_usage 16G -nm -q "
INSERT INTO FUNCTION s3('http://localhost:11111/test/$CLICKHOUSE_DATABASE/test_INT_MAX.tsv', '', '', 'TSV')
SELECT repeat('a', 1024) FROM numbers((pow(2, 30) * 2) / 1024)
SETTINGS s3_max_single_part_upload_size = '5Gi';
diff --git a/tests/queries/0_stateless/02786_max_execution_time_leaf.sql b/tests/queries/0_stateless/02786_max_execution_time_leaf.sql
index f678c913b46..2e4623f4ac6 100644
--- a/tests/queries/0_stateless/02786_max_execution_time_leaf.sql
+++ b/tests/queries/0_stateless/02786_max_execution_time_leaf.sql
@@ -1,4 +1,5 @@
-- Tags: no-fasttest
+SET max_rows_to_read = 0;
SELECT count() FROM cluster('test_cluster_two_shards', view( SELECT * FROM numbers(100000000000) )) SETTINGS max_execution_time_leaf = 1; -- { serverError TIMEOUT_EXCEEDED }
-- Can return partial result
SELECT count() FROM cluster('test_cluster_two_shards', view( SELECT * FROM numbers(100000000000) )) FORMAT Null SETTINGS max_execution_time_leaf = 1, timeout_overflow_mode_leaf = 'break';
diff --git a/tests/queries/0_stateless/02807_lower_utf8_msan.sql b/tests/queries/0_stateless/02807_lower_utf8_msan.sql
index e9eb18bf615..95f224577f7 100644
--- a/tests/queries/0_stateless/02807_lower_utf8_msan.sql
+++ b/tests/queries/0_stateless/02807_lower_utf8_msan.sql
@@ -1,2 +1,5 @@
+-- Tags: no-fasttest
+-- no-fasttest: upper/lowerUTF8 use ICU
+
SELECT lowerUTF8(arrayJoin(['©--------------------------------------', '©--------------------'])) ORDER BY 1;
SELECT upperUTF8(materialize('aaaaАБВГaaaaaaaaaaaaАБВГAAAAaaAA')) FROM numbers(2);
diff --git a/tests/queries/0_stateless/02844_subquery_timeout_with_break.sql b/tests/queries/0_stateless/02844_subquery_timeout_with_break.sql
index 511ed0c59de..00b527a9378 100644
--- a/tests/queries/0_stateless/02844_subquery_timeout_with_break.sql
+++ b/tests/queries/0_stateless/02844_subquery_timeout_with_break.sql
@@ -4,7 +4,7 @@ CREATE TABLE t (key UInt64, value UInt64, INDEX value_idx value TYPE bloom_filte
INSERT INTO t SELECT number, rand()%1000 FROM numbers(10000);
SET timeout_overflow_mode='break';
-SET max_execution_time=0.1;
+SET max_execution_time=0.1, max_rows_to_read=0;
SELECT * FROM t WHERE value IN (SELECT number FROM numbers(1000000000));
DROP TABLE t;
diff --git a/tests/queries/0_stateless/02884_parallel_window_functions.reference b/tests/queries/0_stateless/02884_parallel_window_functions.reference
index bac15838dc2..1f5346a1484 100644
--- a/tests/queries/0_stateless/02884_parallel_window_functions.reference
+++ b/tests/queries/0_stateless/02884_parallel_window_functions.reference
@@ -12,7 +12,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
GROUP BY ac, nw
)
GROUP BY nw
@@ -32,7 +32,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
GROUP BY ac, nw
)
GROUP BY nw
@@ -42,6 +42,7 @@ SETTINGS max_threads = 1;
0 2 0
1 2 0
2 2 0
+SET max_rows_to_read = 40000000;
SELECT
nw,
sum(WR) AS R,
@@ -53,7 +54,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
WHERE (ac % 4) = 0
GROUP BY
ac,
@@ -64,7 +65,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
WHERE (ac % 4) = 1
GROUP BY
ac,
@@ -75,7 +76,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
WHERE (ac % 4) = 2
GROUP BY
ac,
@@ -86,7 +87,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
WHERE (ac % 4) = 3
GROUP BY
ac,
diff --git a/tests/queries/0_stateless/02884_parallel_window_functions.sql b/tests/queries/0_stateless/02884_parallel_window_functions.sql
index c5ab013a198..2207c90a4ee 100644
--- a/tests/queries/0_stateless/02884_parallel_window_functions.sql
+++ b/tests/queries/0_stateless/02884_parallel_window_functions.sql
@@ -1,6 +1,6 @@
-- Tags: long, no-tsan, no-asan, no-ubsan, no-msan, no-debug
-CREATE TABLE window_funtion_threading
+CREATE TABLE window_function_threading
Engine = MergeTree
ORDER BY (ac, nw)
AS SELECT
@@ -20,7 +20,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
GROUP BY ac, nw
)
GROUP BY nw
@@ -40,7 +40,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
GROUP BY ac, nw
)
GROUP BY nw
@@ -58,7 +58,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
GROUP BY ac, nw
)
GROUP BY nw
@@ -66,6 +66,8 @@ ORDER BY nw ASC, R DESC
LIMIT 10
SETTINGS max_threads = 1;
+SET max_rows_to_read = 40000000;
+
SELECT
nw,
sum(WR) AS R,
@@ -77,7 +79,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
WHERE (ac % 4) = 0
GROUP BY
ac,
@@ -88,7 +90,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
WHERE (ac % 4) = 1
GROUP BY
ac,
@@ -99,7 +101,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
WHERE (ac % 4) = 2
GROUP BY
ac,
@@ -110,7 +112,7 @@ FROM
AVG(wg) AS WR,
ac,
nw
- FROM window_funtion_threading
+ FROM window_function_threading
WHERE (ac % 4) = 3
GROUP BY
ac,
diff --git a/tests/queries/0_stateless/02896_max_execution_time_with_break_overflow_mode.sql b/tests/queries/0_stateless/02896_max_execution_time_with_break_overflow_mode.sql
index ec86a66c7dd..ecaad62b35a 100644
--- a/tests/queries/0_stateless/02896_max_execution_time_with_break_overflow_mode.sql
+++ b/tests/queries/0_stateless/02896_max_execution_time_with_break_overflow_mode.sql
@@ -1,5 +1,7 @@
-- Tags: no-fasttest
+SET max_rows_to_read = 0, max_execution_time = 0, max_estimated_execution_time = 0;
+
-- Query stops after timeout without an error
SELECT * FROM numbers(100000000) SETTINGS max_block_size=1, max_execution_time=2, timeout_overflow_mode='break' FORMAT Null;
diff --git a/tests/queries/0_stateless/02907_backup_restore_flatten_nested.reference b/tests/queries/0_stateless/02907_backup_restore_flatten_nested.reference
index aa8f22f590a..0db19f0591a 100644
--- a/tests/queries/0_stateless/02907_backup_restore_flatten_nested.reference
+++ b/tests/queries/0_stateless/02907_backup_restore_flatten_nested.reference
@@ -1,8 +1,8 @@
BACKUP_CREATED
-CREATE TABLE default.test\n(\n `test` Array(Tuple(foo String, bar Float64))\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS index_granularity = 8192
+CREATE TABLE default.test\n(\n `test` Array(Tuple(\n foo String,\n bar Float64))\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS index_granularity = 8192
BACKUP_CREATED
CREATE TABLE default.test2\n(\n `test` Nested(foo String, bar Float64)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS index_granularity = 8192
RESTORED
-CREATE TABLE default.test\n(\n `test` Array(Tuple(foo String, bar Float64))\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS index_granularity = 8192
+CREATE TABLE default.test\n(\n `test` Array(Tuple(\n foo String,\n bar Float64))\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS index_granularity = 8192
RESTORED
CREATE TABLE default.test2\n(\n `test` Nested(foo String, bar Float64)\n)\nENGINE = MergeTree\nORDER BY tuple()\nSETTINGS index_granularity = 8192
diff --git a/tests/queries/0_stateless/02908_filesystem_cache_as_collection.reference b/tests/queries/0_stateless/02908_filesystem_cache_as_collection.reference
index d4191af1594..41a60204eab 100644
--- a/tests/queries/0_stateless/02908_filesystem_cache_as_collection.reference
+++ b/tests/queries/0_stateless/02908_filesystem_cache_as_collection.reference
@@ -1,2 +1,2 @@
-1048576 10000000 33554432 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/collection_sql 0 5000 0 16
-1048576 10000000 33554432 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/collection 0 5000 0 16
+1048576 10000000 33554432 1 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/collection_sql 0 5000 0 16
+1048576 10000000 33554432 1 4194304 0 0 0 0 /var/lib/clickhouse/filesystem_caches/collection 0 5000 0 16
diff --git a/tests/queries/0_stateless/02908_filesystem_cache_as_collection.sql b/tests/queries/0_stateless/02908_filesystem_cache_as_collection.sql
index c7216833bc9..127baa8304e 100644
--- a/tests/queries/0_stateless/02908_filesystem_cache_as_collection.sql
+++ b/tests/queries/0_stateless/02908_filesystem_cache_as_collection.sql
@@ -3,8 +3,8 @@
CREATE NAMED COLLECTION IF NOT EXISTS cache_collection_sql AS path = 'collection_sql', max_size = '1Mi';
DROP TABLE IF EXISTS test;
CREATE TABLE test (a Int32, b String)
-ENGINE = MergeTree() ORDER BY a SETTINGS disk = disk(type = cache, disk = 'local_disk', name = '$CLICHOUSE_TEST_UNIQUE_NAME', cache_name='cache_collection_sql');
+ENGINE = MergeTree() ORDER BY a SETTINGS disk = disk(type = cache, disk = 'local_disk', name = '$CLICHOUSE_TEST_UNIQUE_NAME', cache_name='cache_collection_sql', load_metadata_asynchronously = 0);
DESCRIBE FILESYSTEM CACHE '$CLICHOUSE_TEST_UNIQUE_NAME';
CREATE TABLE test2 (a Int32, b String)
-ENGINE = MergeTree() ORDER BY a SETTINGS disk = disk(type = cache, disk = 'local_disk', name = '$CLICHOUSE_TEST_UNIQUE_NAME_2', cache_name='cache_collection');
+ENGINE = MergeTree() ORDER BY a SETTINGS disk = disk(type = cache, disk = 'local_disk', name = '$CLICHOUSE_TEST_UNIQUE_NAME_2', cache_name='cache_collection', load_metadata_asynchronously = 0);
DESCRIBE FILESYSTEM CACHE '$CLICHOUSE_TEST_UNIQUE_NAME_2';
diff --git a/tests/queries/0_stateless/02915_sleep_large_uint.sql b/tests/queries/0_stateless/02915_sleep_large_uint.sql
index f7c04ab6d1f..08b6c580a28 100644
--- a/tests/queries/0_stateless/02915_sleep_large_uint.sql
+++ b/tests/queries/0_stateless/02915_sleep_large_uint.sql
@@ -1,6 +1,7 @@
SELECT sleep(3.40282e+44); -- { serverError BAD_ARGUMENTS }
SELECT sleep((pow(2, 64) / 1000000) - 1); -- { serverError BAD_ARGUMENTS }
SELECT sleepEachRow(184467440737095516) from numbers(10000); -- { serverError BAD_ARGUMENTS }
+SET max_rows_to_read = 0;
SELECT sleepEachRow(pow(2, 31)) from numbers(9007199254740992) settings function_sleep_max_microseconds_per_block = 8589934592000000000; -- { serverError TOO_SLOW }
-- Another corner case, but it requires lots of memory to run (huge block size)
diff --git a/tests/queries/0_stateless/02916_glogal_in_cancel.sql b/tests/queries/0_stateless/02916_glogal_in_cancel.sql
index ad54f1ecdec..dd61795947a 100644
--- a/tests/queries/0_stateless/02916_glogal_in_cancel.sql
+++ b/tests/queries/0_stateless/02916_glogal_in_cancel.sql
@@ -1,2 +1,2 @@
-set max_execution_time = 0.5, timeout_overflow_mode = 'break';
+set max_execution_time = 0.5, timeout_overflow_mode = 'break', max_rows_to_read = 0;
SELECT number FROM remote('127.0.0.{3|2}', numbers(1)) WHERE number GLOBAL IN (SELECT number FROM numbers(10000000000.)) format Null;
diff --git a/tests/queries/0_stateless/02933_change_cache_setting_without_restart.reference b/tests/queries/0_stateless/02933_change_cache_setting_without_restart.reference
index 17a25d82824..0f64d0393b2 100644
--- a/tests/queries/0_stateless/02933_change_cache_setting_without_restart.reference
+++ b/tests/queries/0_stateless/02933_change_cache_setting_without_restart.reference
@@ -1,7 +1,7 @@
-134217728 10000000 33554432 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 0 0 0 16
-134217728 10000000 33554432 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 10 1000 0 16
-134217728 10000000 33554432 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 5 1000 0 16
-134217728 10000000 33554432 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 15 1000 0 16
-134217728 10000000 33554432 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 2 1000 0 16
-134217728 10000000 33554432 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 0 1000 0 16
-134217728 10000000 33554432 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 0 0 0 16
+134217728 10000000 33554432 1 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 0 0 0 16
+134217728 10000000 33554432 1 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 10 1000 0 16
+134217728 10000000 33554432 1 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 5 1000 0 16
+134217728 10000000 33554432 1 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 15 1000 0 16
+134217728 10000000 33554432 1 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 2 1000 0 16
+134217728 10000000 33554432 1 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 0 1000 0 16
+134217728 10000000 33554432 1 4194304 1 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02933/ 0 0 0 16
diff --git a/tests/queries/0_stateless/02944_dynamically_change_filesystem_cache_size.reference b/tests/queries/0_stateless/02944_dynamically_change_filesystem_cache_size.reference
index 298cc908178..c6bbcdc20c2 100644
--- a/tests/queries/0_stateless/02944_dynamically_change_filesystem_cache_size.reference
+++ b/tests/queries/0_stateless/02944_dynamically_change_filesystem_cache_size.reference
@@ -1,20 +1,20 @@
-100 10 10 10 0 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02944/ 0 5000 0 16
+100 10 10 1 10 0 0 0 0 /var/lib/clickhouse/filesystem_caches/s3_cache_02944/ 0 5000 0 16
0
10
98
set max_size from 100 to 10
-10 10 10 10 0 0 8 1 /var/lib/clickhouse/filesystem_caches/s3_cache_02944/ 0 5000 0 16
+10 10 10 1 10 0 0 8 1 /var/lib/clickhouse/filesystem_caches/s3_cache_02944/ 0 5000 0 16
1
8
set max_size from 10 to 100
-100 10 10 10 0 0 8 1 /var/lib/clickhouse/filesystem_caches/s3_cache_02944/ 0 5000 0 16
+100 10 10 1 10 0 0 8 1 /var/lib/clickhouse/filesystem_caches/s3_cache_02944/ 0 5000 0 16
10
98
set max_elements from 10 to 2
-100 2 10 10 0 0 18 2 /var/lib/clickhouse/filesystem_caches/s3_cache_02944/ 0 5000 0 16
+100 2 10 1 10 0 0 18 2 /var/lib/clickhouse/filesystem_caches/s3_cache_02944/ 0 5000 0 16
2
18
set max_elements from 2 to 10
-100 10 10 10 0 0 18 2 /var/lib/clickhouse/filesystem_caches/s3_cache_02944/ 0 5000 0 16
+100 10 10 1 10 0 0 18 2 /var/lib/clickhouse/filesystem_caches/s3_cache_02944/ 0 5000 0 16
10
98
diff --git a/tests/queries/0_stateless/02967_parallel_replicas_join_algo_and_analyzer_3.sh b/tests/queries/0_stateless/02967_parallel_replicas_join_algo_and_analyzer_3.sh
index e954cb0e78e..c4b44ce11c5 100755
--- a/tests/queries/0_stateless/02967_parallel_replicas_join_algo_and_analyzer_3.sh
+++ b/tests/queries/0_stateless/02967_parallel_replicas_join_algo_and_analyzer_3.sh
@@ -30,7 +30,7 @@ SETTINGS enable_analyzer=1,
allow_experimental_parallel_reading_from_replicas = 2, max_parallel_replicas = 2, parallel_replicas_for_non_replicated_merge_tree = 1,
cluster_for_parallel_replicas = 'test_cluster_one_shard_three_replicas_localhost', parallel_replicas_prefer_local_join=0"
-$CLICKHOUSE_CLIENT -q "
+$CLICKHOUSE_CLIENT --max_rows_in_set_to_optimize_join 0 -q "
select * from (select key, value from num_1) l
inner join (select key, value from num_2 inner join
(select number * 7 as key from numbers(1e5)) as nn on num_2.key = nn.key settings parallel_replicas_prefer_local_join=0) r
@@ -55,7 +55,7 @@ SETTINGS enable_analyzer=1,
allow_experimental_parallel_reading_from_replicas = 2, max_parallel_replicas = 2, parallel_replicas_for_non_replicated_merge_tree = 1,
cluster_for_parallel_replicas = 'test_cluster_one_shard_three_replicas_localhost', parallel_replicas_prefer_local_join=0"
-$CLICKHOUSE_CLIENT -q "
+$CLICKHOUSE_CLIENT --max_rows_in_set_to_optimize_join 0 -q "
select * from (select key, value from num_1) l
inner join (select key, value from num_2 inner join
(select number * 7 as key from numbers(1e5)) as nn on num_2.key = nn.key settings parallel_replicas_prefer_local_join=1) r
@@ -81,7 +81,7 @@ SETTINGS enable_analyzer=1, parallel_replicas_prefer_local_join=0,
allow_experimental_parallel_reading_from_replicas = 2, max_parallel_replicas = 2, parallel_replicas_for_non_replicated_merge_tree = 1,
cluster_for_parallel_replicas = 'test_cluster_one_shard_three_replicas_localhost', join_algorithm='full_sorting_merge'"
-$CLICKHOUSE_CLIENT -q "
+$CLICKHOUSE_CLIENT --max_rows_in_set_to_optimize_join 0 -q "
select * from (select key, value from num_1) l
inner join (select key, value from num_2 inner join
(select number * 7 as key from numbers(1e5)) as nn on num_2.key = nn.key settings join_algorithm='full_sorting_merge') r
@@ -106,7 +106,7 @@ SETTINGS enable_analyzer=1, parallel_replicas_prefer_local_join=0,
allow_experimental_parallel_reading_from_replicas = 2, max_parallel_replicas = 2, parallel_replicas_for_non_replicated_merge_tree = 1,
cluster_for_parallel_replicas = 'test_cluster_one_shard_three_replicas_localhost', join_algorithm='full_sorting_merge'"
-$CLICKHOUSE_CLIENT -q "
+$CLICKHOUSE_CLIENT --max_rows_in_set_to_optimize_join 0 -q "
select * from (select key, value from num_1) l
inner join (select key, value from num_2 inner join
(select number * 7 as key from numbers(1e5)) as nn on num_2.key = nn.key settings join_algorithm='hash') r
@@ -131,7 +131,7 @@ SETTINGS enable_analyzer=1, parallel_replicas_prefer_local_join=0,
allow_experimental_parallel_reading_from_replicas = 2, max_parallel_replicas = 2, parallel_replicas_for_non_replicated_merge_tree = 1,
cluster_for_parallel_replicas = 'test_cluster_one_shard_three_replicas_localhost', join_algorithm='hash'"
-$CLICKHOUSE_CLIENT -q "
+$CLICKHOUSE_CLIENT --max_rows_in_set_to_optimize_join 0 -q "
select * from (select key, value from num_1) l
inner join (select key, value from num_2 inner join
(select number * 7 as key from numbers(1e5)) as nn on num_2.key = nn.key settings join_algorithm='full_sorting_merge') r
diff --git a/tests/queries/0_stateless/03002_filter_skip_virtual_columns_with_non_deterministic_functions.sql b/tests/queries/0_stateless/03002_filter_skip_virtual_columns_with_non_deterministic_functions.sql
index 8ccc3cf61da..6714a069246 100644
--- a/tests/queries/0_stateless/03002_filter_skip_virtual_columns_with_non_deterministic_functions.sql
+++ b/tests/queries/0_stateless/03002_filter_skip_virtual_columns_with_non_deterministic_functions.sql
@@ -1,3 +1,5 @@
+-- Tags: long
+SET max_rows_to_read = 0;
create table test (number UInt64) engine=MergeTree order by number;
insert into test select * from numbers(50000000);
select ignore(number) from test where RAND() > 4292390314 limit 10;
diff --git a/tests/queries/0_stateless/03003_compatibility_setting_bad_value.sql b/tests/queries/0_stateless/03003_compatibility_setting_bad_value.sql
index 48e98798c51..3a09eec7452 100644
--- a/tests/queries/0_stateless/03003_compatibility_setting_bad_value.sql
+++ b/tests/queries/0_stateless/03003_compatibility_setting_bad_value.sql
@@ -1,2 +1 @@
-select 42 settings compatibility=NULL; -- {clientError BAD_ARGUMENTS}
-
+select 42 settings compatibility=NULL; -- {clientError BAD_GET}
diff --git a/tests/queries/0_stateless/03015_peder1001.sql b/tests/queries/0_stateless/03015_peder1001.sql
index 810503207f2..df8e4db1536 100644
--- a/tests/queries/0_stateless/03015_peder1001.sql
+++ b/tests/queries/0_stateless/03015_peder1001.sql
@@ -1,3 +1,6 @@
+-- Tags: no-fasttest
+-- no-fasttest: upper/lowerUTF8 use ICU
+
DROP TABLE IF EXISTS test_data;
CREATE TABLE test_data
diff --git a/tests/queries/0_stateless/03143_asof_join_ddb_long.reference b/tests/queries/0_stateless/03143_asof_join_ddb_long.reference
index 2850a8aba98..ae7f7c805f2 100644
--- a/tests/queries/0_stateless/03143_asof_join_ddb_long.reference
+++ b/tests/queries/0_stateless/03143_asof_join_ddb_long.reference
@@ -1,2 +1,2 @@
-49999983751397 10000032
-49999983751397 10000032
+7999995751397 4000032
+7999995751397 4000032
diff --git a/tests/queries/0_stateless/03143_asof_join_ddb_long.sql b/tests/queries/0_stateless/03143_asof_join_ddb_long.sql
index 4b211a6a1e1..c93e6618ba9 100644
--- a/tests/queries/0_stateless/03143_asof_join_ddb_long.sql
+++ b/tests/queries/0_stateless/03143_asof_join_ddb_long.sql
@@ -12,7 +12,7 @@ AS
toDateTime('1990-03-21 13:00:00') + INTERVAL number MINUTE AS begin,
number % 4 AS key,
number AS value
- FROM numbers(0, 10000000);
+ FROM numbers(0, 4000000);
CREATE TABLE skewed_probe ENGINE = MergeTree ORDER BY (key, begin)
AS
@@ -34,8 +34,9 @@ AS
SELECT
toDateTime('1990-03-21 13:00:01') + INTERVAL number MINUTE AS begin,
3 AS key
- FROM numbers(0, 10000000);
+ FROM numbers(0, 4000000);
+SET max_rows_to_read = 0;
SELECT SUM(value), COUNT(*)
FROM skewed_probe
diff --git a/tests/queries/0_stateless/03149_numbers_max_block_size_zero.sh b/tests/queries/0_stateless/03149_numbers_max_block_size_zero.sh
index 6f70a0d2536..7f606d889a7 100755
--- a/tests/queries/0_stateless/03149_numbers_max_block_size_zero.sh
+++ b/tests/queries/0_stateless/03149_numbers_max_block_size_zero.sh
@@ -1,7 +1,9 @@
#!/usr/bin/env bash
+# shellcheck disable=SC2266
CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
. "$CUR_DIR"/../shell_config.sh
-$CLICKHOUSE_CLIENT -q "SELECT count(*) FROM numbers(10) AS a, numbers(11) AS b, numbers(12) AS c SETTINGS max_block_size = 0" 2>&1 | grep -q "Sanity check: 'max_block_size' cannot be 0. Set to default value" && echo "OK" || echo "FAIL"
+$CLICKHOUSE_CLIENT -q "SELECT count(*) FROM numbers(10) AS a, numbers(11) AS b, numbers(12) AS c SETTINGS max_block_size = 0" 2>&1 |
+ [ "$(grep -c "Sanity check: 'max_block_size' cannot be 0. Set to default value")" -gt 0 ] && echo "OK" || echo "FAIL"
diff --git a/tests/queries/0_stateless/03170_ecs_crash.reference b/tests/queries/0_stateless/03170_ecs_crash.reference
new file mode 100644
index 00000000000..acd7c60768b
--- /dev/null
+++ b/tests/queries/0_stateless/03170_ecs_crash.reference
@@ -0,0 +1,4 @@
+1 2 3
+4 5 6
+7 8 9
+0 0 0
diff --git a/tests/queries/0_stateless/03170_ecs_crash.sh b/tests/queries/0_stateless/03170_ecs_crash.sh
new file mode 100755
index 00000000000..fa6870c4cf2
--- /dev/null
+++ b/tests/queries/0_stateless/03170_ecs_crash.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# Tags: no-fasttest
+
+CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+# shellcheck source=../shell_config.sh
+. "$CUR_DIR"/../shell_config.sh
+
+# Previous versions crashed in attempt to use this authentication method (regardless of whether it was able to authenticate):
+AWS_CONTAINER_CREDENTIALS_FULL_URI=http://localhost:1338/latest/meta-data/container/security-credentials $CLICKHOUSE_LOCAL -q "select * from s3('http://localhost:11111/test/a.tsv')"
diff --git a/tests/queries/0_stateless/03172_format_settings_clauses.reference b/tests/queries/0_stateless/03172_format_settings_clauses.reference
new file mode 100644
index 00000000000..8a98b137f4b
--- /dev/null
+++ b/tests/queries/0_stateless/03172_format_settings_clauses.reference
@@ -0,0 +1,14 @@
+1
+2
+1
+2
+1
+2
+1
+1
+3
+3
+3
+3
+3
+1
diff --git a/tests/queries/0_stateless/03172_format_settings_clauses.sql b/tests/queries/0_stateless/03172_format_settings_clauses.sql
new file mode 100644
index 00000000000..0d1aa4dcfbb
--- /dev/null
+++ b/tests/queries/0_stateless/03172_format_settings_clauses.sql
@@ -0,0 +1,30 @@
+SET max_block_size = 10, max_threads = 1;
+
+-- Take the following example:
+SELECT 1 UNION ALL SELECT 2 FORMAT TSV;
+
+-- Each subquery can be put in parentheses and have its own settings:
+(SELECT getSetting('max_block_size') SETTINGS max_block_size = 1) UNION ALL (SELECT getSetting('max_block_size') SETTINGS max_block_size = 2) FORMAT TSV;
+
+-- And the whole query can have settings:
+(SELECT getSetting('max_block_size') SETTINGS max_block_size = 1) UNION ALL (SELECT getSetting('max_block_size') SETTINGS max_block_size = 2) FORMAT TSV SETTINGS max_block_size = 3;
+
+-- A single query with output is parsed in the same way as the UNION ALL chain:
+SELECT getSetting('max_block_size') SETTINGS max_block_size = 1 FORMAT TSV SETTINGS max_block_size = 3;
+
+-- So while these forms have a slightly different meaning, they both exist:
+SELECT getSetting('max_block_size') SETTINGS max_block_size = 1 FORMAT TSV;
+SELECT getSetting('max_block_size') FORMAT TSV SETTINGS max_block_size = 3;
+
+-- And due to this effect, the users expect that the FORMAT and SETTINGS may go in an arbitrary order.
+-- But while this work:
+(SELECT getSetting('max_block_size')) UNION ALL (SELECT getSetting('max_block_size')) FORMAT TSV SETTINGS max_block_size = 3;
+
+-- This does not work automatically, unless we explicitly allow different orders:
+(SELECT getSetting('max_block_size')) UNION ALL (SELECT getSetting('max_block_size')) SETTINGS max_block_size = 3 FORMAT TSV;
+
+-- Inevitably, we allow this:
+SELECT getSetting('max_block_size') SETTINGS max_block_size = 1 SETTINGS max_block_size = 3 FORMAT TSV;
+/*^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^*/
+-- Because this part is consumed into ASTSelectWithUnionQuery
+-- and the rest into ASTQueryWithOutput.
diff --git a/tests/queries/0_stateless/03221_refreshable_matview_progress.reference b/tests/queries/0_stateless/03221_refreshable_matview_progress.reference
new file mode 100644
index 00000000000..5ed392e61c7
--- /dev/null
+++ b/tests/queries/0_stateless/03221_refreshable_matview_progress.reference
@@ -0,0 +1,2 @@
+0
+4 4 1
diff --git a/tests/queries/0_stateless/03221_refreshable_matview_progress.sql b/tests/queries/0_stateless/03221_refreshable_matview_progress.sql
new file mode 100644
index 00000000000..98e1c48478d
--- /dev/null
+++ b/tests/queries/0_stateless/03221_refreshable_matview_progress.sql
@@ -0,0 +1,20 @@
+-- Tags: no-replicated-database, no-ordinary-database
+
+set allow_experimental_refreshable_materialized_view=1;
+
+CREATE MATERIALIZED VIEW 03221_rmv
+REFRESH AFTER 10 SECOND
+(
+x UInt64
+)
+ENGINE = Memory
+AS SELECT number AS x
+FROM numbers(3)
+UNION ALL
+SELECT rand64() AS x;
+
+SELECT sleep(2);
+
+SELECT read_rows, total_rows, progress FROM system.view_refreshes WHERE database = currentDatabase() and view = '03221_rmv';
+
+DROP TABLE 03221_rmv;
diff --git a/tests/queries/0_stateless/03223_parallel_replicas_read_task_size_overflow_bug.reference b/tests/queries/0_stateless/03223_parallel_replicas_read_task_size_overflow_bug.reference
new file mode 100644
index 00000000000..e69de29bb2d
diff --git a/tests/queries/0_stateless/03223_parallel_replicas_read_task_size_overflow_bug.sql b/tests/queries/0_stateless/03223_parallel_replicas_read_task_size_overflow_bug.sql
new file mode 100644
index 00000000000..984c7fe0db7
--- /dev/null
+++ b/tests/queries/0_stateless/03223_parallel_replicas_read_task_size_overflow_bug.sql
@@ -0,0 +1,22 @@
+DROP TABLE IF EXISTS test__fuzz_22 SYNC;
+
+CREATE TABLE test__fuzz_22 (k Float32, v String) ENGINE = MergeTree ORDER BY k SETTINGS index_granularity = 1;
+
+SYSTEM STOP MERGES test__fuzz_22;
+
+INSERT INTO test__fuzz_22 SELECT number, toString(number) FROM numbers(1);
+INSERT INTO test__fuzz_22 SELECT number, toString(number) FROM numbers(1);
+INSERT INTO test__fuzz_22 SELECT number, toString(number) FROM numbers(1);
+INSERT INTO test__fuzz_22 SELECT number, toString(number) FROM numbers(1);
+
+SET allow_experimental_parallel_reading_from_replicas = 2, parallel_replicas_for_non_replicated_merge_tree = 1, max_parallel_replicas = 3, cluster_for_parallel_replicas='test_cluster_one_shard_three_replicas_localhost',
+ merge_tree_min_rows_for_concurrent_read = 9223372036854775806, merge_tree_min_rows_for_concurrent_read_for_remote_filesystem = 9223372036854775806;
+
+ SELECT v
+ FROM test__fuzz_22
+ORDER BY v
+ LIMIT 10, 10
+SETTINGS max_threads = 4
+ FORMAT Null; -- { serverError BAD_ARGUMENTS }
+
+DROP TABLE test__fuzz_22 SYNC;
diff --git a/tests/queries/0_stateless/03223_system_tables_set_not_ready.reference b/tests/queries/0_stateless/03223_system_tables_set_not_ready.reference
new file mode 100644
index 00000000000..e39523ed4f5
--- /dev/null
+++ b/tests/queries/0_stateless/03223_system_tables_set_not_ready.reference
@@ -0,0 +1,5 @@
+system.distribution_queue 1
+system.rocksdb 1
+system.databases 1
+system.mutations 1
+system.replication_queue 1
diff --git a/tests/queries/0_stateless/03223_system_tables_set_not_ready.sql b/tests/queries/0_stateless/03223_system_tables_set_not_ready.sql
new file mode 100644
index 00000000000..907fa47143c
--- /dev/null
+++ b/tests/queries/0_stateless/03223_system_tables_set_not_ready.sql
@@ -0,0 +1,30 @@
+-- Tags: no-fasttest
+-- Tag no-fasttest -- due to EmbeddedRocksDB
+
+drop table if exists null;
+drop table if exists dist;
+create table null as system.one engine=Null;
+create table dist as null engine=Distributed(test_cluster_two_shards, currentDatabase(), 'null', rand());
+insert into dist settings prefer_localhost_replica=0 values (1);
+select 'system.distribution_queue', count() from system.distribution_queue where exists(select 1) and database = currentDatabase();
+
+drop table if exists rocksdb;
+create table rocksdb (key Int) engine=EmbeddedRocksDB() primary key key;
+insert into rocksdb values (1);
+select 'system.rocksdb', count()>0 from system.rocksdb where exists(select 1) and database = currentDatabase();
+
+select 'system.databases', count() from system.databases where exists(select 1) and database = currentDatabase();
+
+drop table if exists mt;
+create table mt (key Int) engine=MergeTree() order by key;
+alter table mt delete where 1;
+select 'system.mutations', count() from system.mutations where exists(select 1) and database = currentDatabase();
+
+drop table if exists rep1;
+drop table if exists rep2;
+create table rep1 (key Int) engine=ReplicatedMergeTree('/{database}/rep', '{table}') order by key;
+create table rep2 (key Int) engine=ReplicatedMergeTree('/{database}/rep', '{table}') order by key;
+system stop fetches rep2;
+insert into rep1 values (1);
+system sync replica rep2 pull;
+select 'system.replication_queue', count()>0 from system.replication_queue where exists(select 1) and database = currentDatabase();
diff --git a/tests/queries/0_stateless/03227_print_pretty_tuples_create_query.reference b/tests/queries/0_stateless/03227_print_pretty_tuples_create_query.reference
new file mode 100644
index 00000000000..afaaaaa6119
--- /dev/null
+++ b/tests/queries/0_stateless/03227_print_pretty_tuples_create_query.reference
@@ -0,0 +1,56 @@
+
+SHOW CREATE TABLE:
+CREATE TABLE default.test
+(
+ `x` Tuple(
+ a String,
+ b Array(Tuple(
+ c Tuple(
+ e String),
+ d String))),
+ `y` String
+)
+ENGINE = MergeTree
+ORDER BY tuple()
+SETTINGS index_granularity = 8192
+CREATE TABLE default.test
+(
+ `x` Tuple(a String, b Array(Tuple(c Tuple(e String), d String))),
+ `y` String
+)
+ENGINE = MergeTree
+ORDER BY tuple()
+SETTINGS index_granularity = 8192
+
+clickhouse-format:
+CREATE TABLE test
+(
+ `x` Tuple(
+ a String,
+ b Array(Tuple(
+ c Tuple(
+ e String),
+ d String))),
+ `y` String
+)
+ORDER BY tuple()
+CREATE TABLE test (`x` Tuple(a String, b Array(Tuple(c Tuple(e String), d String))), `y` String) ORDER BY tuple()
+
+formatQuery:
+CREATE TABLE test
+(
+ `x` Tuple(
+ a String,
+ b Array(Tuple(
+ c Tuple(
+ e String),
+ d String))),
+ `y` String
+)
+ORDER BY tuple()
+CREATE TABLE test
+(
+ `x` Tuple(a String, b Array(Tuple(c Tuple(e String), d String))),
+ `y` String
+)
+ORDER BY tuple()
diff --git a/tests/queries/0_stateless/03227_print_pretty_tuples_create_query.sh b/tests/queries/0_stateless/03227_print_pretty_tuples_create_query.sh
new file mode 100755
index 00000000000..e5614f9f228
--- /dev/null
+++ b/tests/queries/0_stateless/03227_print_pretty_tuples_create_query.sh
@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+# Tags: no-fasttest, no-asan, no-msan, no-tsan
+# ^ requires S3
+
+CUR_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+# shellcheck source=../shell_config.sh
+. "$CUR_DIR"/../shell_config.sh
+
+echo
+echo "SHOW CREATE TABLE:"
+${CLICKHOUSE_CLIENT} --output-format Raw --query "
+ DROP TABLE IF EXISTS test;
+ CREATE TABLE test (x Tuple(a String, b Array(Tuple(c Tuple(e String), d String))), y String) ORDER BY ();
+ SET print_pretty_type_names = 1;
+ SHOW CREATE TABLE test;
+ SET print_pretty_type_names = 0;
+ SHOW CREATE TABLE test;
+ DROP TABLE test;
+"
+
+echo
+echo "clickhouse-format:"
+${CLICKHOUSE_FORMAT} --query "
+ CREATE TABLE test (x Tuple(a String, b Array(Tuple(c Tuple(e String), d String))), y String) ORDER BY ()
+"
+${CLICKHOUSE_FORMAT} --oneline --query "
+ CREATE TABLE test (x Tuple(a String, b Array(Tuple(c Tuple(e String), d String))), y String) ORDER BY ()
+"
+
+echo
+echo "formatQuery:"
+${CLICKHOUSE_CLIENT} --output-format Raw --query "
+ SELECT formatQuery('CREATE TABLE test (x Tuple(a String, b Array(Tuple(c Tuple(e String), d String))), y String) ORDER BY ()') SETTINGS print_pretty_type_names = 1;
+ SELECT formatQuery('CREATE TABLE test (x Tuple(a String, b Array(Tuple(c Tuple(e String), d String))), y String) ORDER BY ()') SETTINGS print_pretty_type_names = 0;
+"
diff --git a/tests/queries/0_stateless/03227_proper_parsing_of_cast_operator.reference b/tests/queries/0_stateless/03227_proper_parsing_of_cast_operator.reference
new file mode 100644
index 00000000000..2127d396bb3
--- /dev/null
+++ b/tests/queries/0_stateless/03227_proper_parsing_of_cast_operator.reference
@@ -0,0 +1,4 @@
+414243
+ABC
+A
+{"a": \'A\'}
diff --git a/tests/queries/0_stateless/03227_proper_parsing_of_cast_operator.sql b/tests/queries/0_stateless/03227_proper_parsing_of_cast_operator.sql
new file mode 100644
index 00000000000..0c2e7dc582a
--- /dev/null
+++ b/tests/queries/0_stateless/03227_proper_parsing_of_cast_operator.sql
@@ -0,0 +1,6 @@
+SELECT '414243'::String;
+SELECT x'414243'::String;
+SELECT b'01000001'::String;
+SELECT '{"a": \'\x41\'}'::String;
+SELECT '{"a": \'\x4\'}'::String; -- { clientError SYNTAX_ERROR }
+SELECT '{"a": \'a\x4\'}'::String; -- { clientError SYNTAX_ERROR }
diff --git a/tests/queries/0_stateless/replication.lib b/tests/queries/0_stateless/replication.lib
index 05651531fba..36309cf0331 100755
--- a/tests/queries/0_stateless/replication.lib
+++ b/tests/queries/0_stateless/replication.lib
@@ -89,7 +89,7 @@ function check_replication_consistency()
# Touch all data to check that it's readable (and trigger PartCheckThread if needed)
# it's important to disable prefer warmed unmerged parts because
# otherwise it can read non-syncrhonized state of replicas
- while ! $CLICKHOUSE_CLIENT --prefer_warmed_unmerged_parts_seconds=0 -q "SELECT * FROM merge(currentDatabase(), '$table_name_prefix') FORMAT Null" 2>/dev/null; do
+ while ! $CLICKHOUSE_CLIENT --prefer_warmed_unmerged_parts_seconds=0 --max_result_rows 0 --max_result_bytes 0 --max_rows_to_read 0 --max_bytes_to_read 0 -q "SELECT * FROM merge(currentDatabase(), '$table_name_prefix') FORMAT Null" 2>/dev/null; do
sleep 1;
num_tries=$((num_tries+1))
if [ $num_tries -eq 250 ]; then
@@ -114,7 +114,8 @@ function check_replication_consistency()
# it's important to disable prefer warmed unmerged parts because
# otherwise it can read non-syncrhonized state of replicas
- res=$($CLICKHOUSE_CLIENT --prefer_warmed_unmerged_parts_seconds=0 -q \
+ # also, disable the limit that is set for tests globally
+ res=$($CLICKHOUSE_CLIENT --prefer_warmed_unmerged_parts_seconds=0 --max_rows_to_read=0 -q \
"SELECT
if((countDistinct(data) as c) == 0, 1, c)
FROM
diff --git a/tests/queries/1_stateful/00067_union_all.sql b/tests/queries/1_stateful/00067_union_all.sql
index 2a1d00e975d..9ee14b36b03 100644
--- a/tests/queries/1_stateful/00067_union_all.sql
+++ b/tests/queries/1_stateful/00067_union_all.sql
@@ -10,4 +10,5 @@ UNION ALL
ORDER BY id DESC
LIMIT 10
)
-ORDER BY id, event;
+ORDER BY id, event
+SETTINGS max_rows_to_read = 40_000_000;
diff --git a/tests/queries/1_stateful/00088_global_in_one_shard_and_rows_before_limit.sql b/tests/queries/1_stateful/00088_global_in_one_shard_and_rows_before_limit.sql
index 52f9c46997f..8f18f3740e4 100644
--- a/tests/queries/1_stateful/00088_global_in_one_shard_and_rows_before_limit.sql
+++ b/tests/queries/1_stateful/00088_global_in_one_shard_and_rows_before_limit.sql
@@ -1,4 +1,4 @@
-- Tags: shard
-SET output_format_write_statistics = 0;
+SET output_format_write_statistics = 0, max_rows_to_read = 50_000_000;
SELECT EventDate, count() FROM remote('127.0.0.1', test.hits) WHERE UserID GLOBAL IN (SELECT UserID FROM test.hits) GROUP BY EventDate ORDER BY EventDate LIMIT 5 FORMAT JSONCompact;
diff --git a/tests/queries/1_stateful/00147_global_in_aggregate_function.sql b/tests/queries/1_stateful/00147_global_in_aggregate_function.sql
index 075c01530c6..f0b249e9af4 100644
--- a/tests/queries/1_stateful/00147_global_in_aggregate_function.sql
+++ b/tests/queries/1_stateful/00147_global_in_aggregate_function.sql
@@ -1,4 +1,5 @@
-- Tags: global
+SET max_rows_to_read = 100_000_000;
SELECT sum(UserID GLOBAL IN (SELECT UserID FROM remote('127.0.0.{1,2}', test.hits))) FROM remote('127.0.0.{1,2}', test.hits);
SELECT sum(UserID GLOBAL IN (SELECT UserID FROM test.hits)) FROM remote('127.0.0.{1,2}', test.hits);
diff --git a/tests/queries/1_stateful/00149_quantiles_timing_distributed.sql b/tests/queries/1_stateful/00149_quantiles_timing_distributed.sql
index 6f910646fb7..16b565985ea 100644
--- a/tests/queries/1_stateful/00149_quantiles_timing_distributed.sql
+++ b/tests/queries/1_stateful/00149_quantiles_timing_distributed.sql
@@ -1,4 +1,5 @@
-- Tags: distributed
+SET max_rows_to_read = 100_000_000;
SELECT sum(cityHash64(*)) FROM (SELECT CounterID, quantileTiming(0.5)(SendTiming), count() FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) WHERE SendTiming != -1 GROUP BY CounterID);
SELECT sum(cityHash64(*)) FROM (SELECT CounterID, quantileTiming(0.5)(SendTiming), count() FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) WHERE SendTiming != -1 GROUP BY CounterID) SETTINGS optimize_aggregation_in_order = 1;
diff --git a/tests/queries/1_stateful/00157_cache_dictionary.sql b/tests/queries/1_stateful/00157_cache_dictionary.sql
index 3621ff82126..f1bee538828 100644
--- a/tests/queries/1_stateful/00157_cache_dictionary.sql
+++ b/tests/queries/1_stateful/00157_cache_dictionary.sql
@@ -9,7 +9,8 @@ ORDER BY (CounterID, EventDate, intHash32(UserID))
SAMPLE BY intHash32(UserID)
SETTINGS storage_policy = 'default';
-INSERT INTO test.hits_1m SELECT * FROM test.hits LIMIT 1000000;
+INSERT INTO test.hits_1m SELECT * FROM test.hits LIMIT 1000000
+ SETTINGS min_insert_block_size_rows = 0, min_insert_block_size_bytes = 0, max_block_size = 8192, max_insert_threads = 1, max_threads = 1;
CREATE DATABASE IF NOT EXISTS db_dict;
DROP DICTIONARY IF EXISTS db_dict.cache_hits;
diff --git a/tests/queries/1_stateful/00158_cache_dictionary_has.sql b/tests/queries/1_stateful/00158_cache_dictionary_has.sql
index 32c109417de..631a7751550 100644
--- a/tests/queries/1_stateful/00158_cache_dictionary_has.sql
+++ b/tests/queries/1_stateful/00158_cache_dictionary_has.sql
@@ -10,6 +10,8 @@ SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() USER 'default' TABLE 'hits' PA
LIFETIME(MIN 300 MAX 600)
LAYOUT(CACHE(SIZE_IN_CELLS 100 QUERY_WAIT_TIMEOUT_MILLISECONDS 600000));
+SET timeout_before_checking_execution_speed = 300;
+
SELECT sum(flag) FROM (SELECT dictHas('db_dict.cache_hits', toUInt64(WatchID)) as flag FROM test.hits PREWHERE WatchID % 1400 == 0 LIMIT 100);
SELECT count() from test.hits PREWHERE WatchID % 1400 == 0;
@@ -20,4 +22,4 @@ SELECT sum(flag) FROM (SELECT dictHas('db_dict.cache_hits', toUInt64(WatchID)) a
SELECT count() from test.hits PREWHERE WatchID % 5 == 0;
DROP DICTIONARY IF EXISTS db_dict.cache_hits;
-DROP DATABASE IF EXISTS db_dict;
+DROP DATABASE IF EXISTS db_dict;
diff --git a/tests/queries/1_stateful/00167_read_bytes_from_fs.sql b/tests/queries/1_stateful/00167_read_bytes_from_fs.sql
index 7b3f50f8141..184a8edcbcb 100644
--- a/tests/queries/1_stateful/00167_read_bytes_from_fs.sql
+++ b/tests/queries/1_stateful/00167_read_bytes_from_fs.sql
@@ -1,5 +1,6 @@
-- Tags: no-random-settings
+SET max_memory_usage = '10G';
SELECT sum(cityHash64(*)) FROM test.hits SETTINGS max_threads=40;
-- We had a bug which lead to additional compressed data read. test.hits compressed size is about 1.2Gb, but we read more then 3Gb.
diff --git a/tests/queries/1_stateful/00171_grouping_aggregated_transform_bug.sql b/tests/queries/1_stateful/00171_grouping_aggregated_transform_bug.sql
index 7068780a1b1..b3e4d749328 100644
--- a/tests/queries/1_stateful/00171_grouping_aggregated_transform_bug.sql
+++ b/tests/queries/1_stateful/00171_grouping_aggregated_transform_bug.sql
@@ -1,4 +1,5 @@
-- Tags: distributed
+SET max_rows_to_read = '100M';
SELECT sum(cityHash64(*)) FROM (SELECT CounterID, quantileTiming(0.5)(SendTiming), count() FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) WHERE SendTiming != -1 GROUP BY CounterID) SETTINGS max_block_size = 63169;
SELECT sum(cityHash64(*)) FROM (SELECT CounterID, quantileTiming(0.5)(SendTiming), count() FROM remote('127.0.0.{1,2,3,4,5,6,7,8,9,10}', test.hits) WHERE SendTiming != -1 GROUP BY CounterID) SETTINGS optimize_aggregation_in_order = 1, max_block_size = 63169;
diff --git a/tests/queries/1_stateful/00182_simple_squashing_transform_bug.sql b/tests/queries/1_stateful/00182_simple_squashing_transform_bug.sql
index e73de4b33fb..26e112cff04 100644
--- a/tests/queries/1_stateful/00182_simple_squashing_transform_bug.sql
+++ b/tests/queries/1_stateful/00182_simple_squashing_transform_bug.sql
@@ -1,6 +1,7 @@
-- Tags: global
set allow_prefetched_read_pool_for_remote_filesystem=0, merge_tree_read_split_ranges_into_intersecting_and_non_intersecting_injection_probability=0, max_threads=2, max_block_size=65387;
+set max_rows_to_read = '100M';
SELECT sum(UserID GLOBAL IN (SELECT UserID FROM remote('127.0.0.{1,2}', test.hits))) FROM remote('127.0.0.{1,2}', test.hits);
SELECT sum(UserID GLOBAL IN (SELECT UserID FROM test.hits)) FROM remote('127.0.0.{1,2}', test.hits);
diff --git a/utils/check-style/aspell-ignore/en/aspell-dict.txt b/utils/check-style/aspell-ignore/en/aspell-dict.txt
index 4a8ef449e72..0be6256aa50 100644
--- a/utils/check-style/aspell-ignore/en/aspell-dict.txt
+++ b/utils/check-style/aspell-ignore/en/aspell-dict.txt
@@ -385,6 +385,18 @@ IntelliJ
IntelliSense
InterserverConnection
InterserverThreads
+IntervalDay
+IntervalHour
+IntervalMicrosecond
+IntervalMillisecond
+IntervalMilliseconds
+IntervalMinute
+IntervalMonth
+IntervalNanosecond
+IntervalQuarter
+IntervalSecond
+IntervalWeek
+IntervalYear
IsPentagon
IsResClassIII
IsValid
@@ -2740,6 +2752,17 @@ toISOWeek
toISOYear
toInt
toInterval
+toIntervalDay
+toIntervalHour
+toIntervalMicrosecond
+toIntervalMillisecond
+toIntervalMinute
+toIntervalMonth
+toIntervalNanosecond
+toIntervalQuarter
+toIntervalSecond
+toIntervalWeek
+toIntervalYear
toJSONString
toLastDayOfMonth
toLastDayOfWeek
diff --git a/utils/check-style/check-style b/utils/check-style/check-style
index 3c959617d02..46593e85e45 100755
--- a/utils/check-style/check-style
+++ b/utils/check-style/check-style
@@ -467,3 +467,7 @@ find $ROOT_PATH/{src,base,programs,utils} -name '*.h' -or -name '*.cpp' |
grep -vP $EXCLUDE_DIRS |
xargs grep -F -i 'ErrorCodes::LOGICAL_ERROR, "Logical error:' &&
echo "If an exception has LOGICAL_ERROR code, there is no need to include the text 'Logical error' in the exception message, because then the phrase 'Logical error' will be printed twice."
+
+PATTERN="allow_";
+DIFF=$(comm -3 <(grep -o "\b$PATTERN\w*\b" $ROOT_PATH/src/Core/Settings.h | sort -u) <(grep -o -h "\b$PATTERN\w*\b" $ROOT_PATH/src/Databases/enableAllExperimentalSettings.cpp $ROOT_PATH/utils/check-style/experimental_settings_ignore.txt | sort -u));
+[ -n "$DIFF" ] && echo "$DIFF" && echo "^^ Detected 'allow_*' settings that might need to be included in src/Databases/enableAllExperimentalSettings.cpp" && echo "Alternatively, consider adding an exception to utils/check-style/experimental_settings_ignore.txt"
diff --git a/utils/check-style/experimental_settings_ignore.txt b/utils/check-style/experimental_settings_ignore.txt
new file mode 100644
index 00000000000..94c46cf562e
--- /dev/null
+++ b/utils/check-style/experimental_settings_ignore.txt
@@ -0,0 +1,48 @@
+allow_aggregate_partitions_independently
+allow_archive_path_syntax
+allow_asynchronous_read_from_io_pool_for_merge_tree
+allow_changing_replica_until_first_data_packet
+allow_custom_error_code_in_throwif
+allow_ddl
+allow_deprecated_database_ordinary
+allow_deprecated_snowflake_conversion_functions
+allow_distributed_ddl
+allow_drop_detached
+allow_execute_multiif_columnar
+allow_experimental_alter_materialized_view_structure
+allow_experimental_analyzer
+allow_experimental_annoy_index
+allow_experimental_database_atomic
+allow_experimental_database_materialized_mysql
+allow_experimental_database_materialized_postgresql
+allow_experimental_database_replicated
+allow_experimental_join_condition
+allow_experimental_kafka_offsets_storage_in_keeper
+allow_experimental_lightweight_delete
+allow_experimental_materialized_postgresql_table
+allow_experimental_parallel_reading_from_replicas
+allow_experimental_projection_optimization
+allow_experimental_query_cache
+allow_experimental_query_deduplication
+allow_experimental_refreshable_materialized_view
+allow_experimental_shared_merge_tree
+allow_experimental_statistic
+allow_experimental_statistics
+allow_experimental_time_series_table
+allow_experimental_undrop_table_query
+allow_experimental_usearch_index
+allow_get_client_http_header
+allow_introspection_functions
+allow_materialized_view_with_bad_select
+allow_named_collection_override_by_default
+allow_non_metadata_alters
+allow_nonconst_timezone_arguments
+allow_nondeterministic_mutations
+allow_nondeterministic_optimize_skip_unused_shards
+allow_prefetched_read_pool_for_local_filesystem
+allow_prefetched_read_pool_for_remote_filesystem
+allow_push_predicate_when_subquery_contains_with
+allow_settings_after_format_in_insert
+allow_statistic_optimize
+allow_statistics_optimize
+allow_unrestricted_reads_from_keeper
diff --git a/utils/list-versions/version_date.tsv b/utils/list-versions/version_date.tsv
index 71a4a722a36..776a53ec01c 100644
--- a/utils/list-versions/version_date.tsv
+++ b/utils/list-versions/version_date.tsv
@@ -13,6 +13,7 @@ v24.4.4.113-stable 2024-08-02
v24.4.3.25-stable 2024-06-14
v24.4.2.141-stable 2024-06-07
v24.4.1.2088-stable 2024-05-01
+v24.3.8.13-lts 2024-08-20
v24.3.7.30-lts 2024-08-14
v24.3.6.48-lts 2024-08-02
v24.3.5.46-lts 2024-07-03
@@ -55,6 +56,7 @@ v23.9.3.12-stable 2023-10-31
v23.9.2.56-stable 2023-10-19
v23.9.1.1854-stable 2023-09-29
v23.8.16.40-lts 2024-08-02
+v23.8.16.16-lts 2024-08-20
v23.8.15.35-lts 2024-06-14
v23.8.14.6-lts 2024-05-02
v23.8.13.25-lts 2024-04-26