thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 20:02:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
99,320 Commits 9,330 Branches 1,783 Tags 2.1 GiB
338f41913c
Commit Graph

7648 Commits

Author SHA1 Message Date
kssenii
b52084265c Merge master 2022-07-04 21:37:43 +02:00
vdimir
c0cb588e45
Fix build 2022-07-04 17:28:34 +00:00
vdimir
43afda408b
do not call setStorageJoin when setting isn't set 2022-07-04 17:28:15 +00:00
vdimir
65ac97ce89
direct join: handle nullable, add tests 2022-07-04 17:28:14 +00:00
vdimir
0c12adc934
fix style 2022-07-04 17:28:14 +00:00
vdimir
e40f799f96
handle right rey rename in direct join 2022-07-04 17:28:13 +00:00
vdimir
87794f9b5b
get/setTotas to IJoin 2022-07-04 17:28:13 +00:00
vdimir
a1388813e4
fix src/Interpreters/ExpressionAnalyzer.cpp 2022-07-04 17:28:12 +00:00
vdimir
8359753a85
Move DirectJoin to separate file 2022-07-04 17:28:11 +00:00
vdimir
51a6c347ad
make join_alorithm settings with multiple choices 2022-07-04 17:28:11 +00:00
vdimir
0dfd400f51
check conditions before using direct join 2022-07-04 17:28:10 +00:00
vdimir
085e70c7a3
wip left kvjoin 2022-07-04 17:28:10 +00:00
vdimir
980000d04c
Change getByKeys, add StorageEmbeddedRocksDB::getByKeysImpl 2022-07-04 17:28:09 +00:00
vdimir
744c692be3
wip key value join 2022-07-04 17:28:09 +00:00
Ilya Yatsishin
867b3aa873 fix 2022-07-04 18:01:24 +02:00
Dmitry Novik
e179449381 Fix some tests 2022-07-04 15:56:42 +00:00
Ilya Yatsishin
f871cbc7d5
Update src/Interpreters/OpenTelemetrySpanLog.cpp 
Co-authored-by: Dmitry Novik <d1mas1k4@yandex.ru>
 2022-07-04 17:52:06 +02:00
Dmitry Novik
c66e2c8095
Merge pull request #37155 from amosbird/projection-fix-two 
Fix intermediate header in aggregate projection
 2022-07-04 16:32:34 +02:00
Yatsishin Ilya
b4e62e06e7 Use native Map 2022-07-04 14:07:29 +00:00
Kruglov Pavel
6c4b6fa834
Merge branch 'master' into better-exception-messages 2022-07-04 15:07:31 +02:00
Alexander Tokmakov
2e073b5cff Merge branch 'master' into replicated_database_improvements 2022-07-04 13:53:47 +02:00
lingpeng0314
cca94f01ae add function tryGetKeeperDispatcher for context 
- to make sure other case will get error if dispatcher not initialized
 2022-07-04 13:35:08 +08:00
Alexey Milovidov
c711012399
Merge pull request #38731 from azat/views-max_insert_threads 
Fix number of threads for pushing to views
 2022-07-04 07:43:26 +03:00
Igor Nikonov
2e2ef08712
Merge pull request #37803 from ClickHouse/dictinct_in_order_optimization 
DISTINCT in order optimization
 2022-07-03 21:59:04 +02:00
Azat Khuzhin
dd3515da98 Fix parallel_view_processing with optimize_trivial_insert_select=1 2022-07-03 15:14:05 +03:00
Azat Khuzhin
4ae7db8369 Fix max_insert_threads while pushing to views 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-03 15:14:05 +03:00
mergify[bot]
12f5250e86
Merge branch 'master' into dictinct_in_order_optimization 2022-07-01 22:51:35 +00:00
Maksim Kita
8177608860 SQL create drop index fix formatting 2022-07-01 22:35:20 +02:00
Yakov Olkhovskiy
776a80bff8
Merge pull request #38136 from bigo-sg/translate_qualified_bug 
Bug fixed in TranslateQualifiedNamesMatcher
 2022-07-01 14:07:00 -04:00
Anton Popov
ef87e1207c better support of read_in_order in case of fixed prefix of sorting key 2022-07-01 16:45:01 +00:00
Dmitry Novik
81dd90893e Merge remote-tracking branch 'origin/master' into group-by-use-nulls 2022-07-01 16:24:05 +00:00
Amos Bird
53f47127e9
Fix only_merge header 2022-07-01 23:31:45 +08:00
Nikolai Kochetov
b7529986c0 Use separate counter for RSS in global memory tracker. 2022-07-01 12:09:32 +00:00
Yatsishin Ilya
202008e80b Add asserts 2022-07-01 11:00:04 +00:00
Vitaly Baranov
cadb496894
Merge pull request #38537 from vitlibar/backup-improvements-8 
Backup Improvements 8
 2022-07-01 11:20:46 +02:00
lingpeng0314
03a5ce542d Remove Exception in getKeeperDispatcher 
- to handle the nil case by caller
 2022-07-01 11:27:40 +08:00
lgbo-ustc
bab954c461 update codes based on review comment 2022-07-01 10:15:35 +08:00
Dmitry Novik
33f601ec0a Commit support use_nulls for GS 2022-06-30 15:14:26 +00:00
Vitaly Baranov
e367d96964 Fix style. 2022-06-30 15:10:33 +02:00
Igor Nikonov
488ee75fc4 + use DistinctSorted for final distinct step 
+ fix performance tests
 2022-06-30 13:03:39 +00:00
Anton Popov
7c721578c7
Merge pull request #38320 from CurtizJ/dynamic-columns-16 
Improve performace of insertion to columns of type JSON
 2022-06-30 14:18:20 +02:00
Kruglov Pavel
57686b8422
Merge pull request #38561 from kitaisreal/sql-create-drop-index-update 
SQL create drop index minor fixes
 2022-06-30 13:38:39 +02:00
Dmitry Novik
98e9bc84d5 Refector ROLLUP and CUBE 2022-06-30 10:13:58 +00:00
alesapin
be213c0be7
Merge pull request #38590 from ClickHouse/proper_fix 
Proper fix for ipv4/ipv6 conversion error
 2022-06-30 11:15:01 +02:00
Vitaly Baranov
aa97bf5125 Improve handling predefined databases and tables. 2022-06-30 08:37:17 +02:00
mergify[bot]
4cbbfb431d
Merge branch 'master' into dictinct_in_order_optimization 2022-06-29 23:32:17 +00:00
Igor Nikonov
d435532c68 Adapt range search algorithm to high cardinality case 
+ range search done in steps of some number of rows.
  Controled by new
  setting `distinct_in_order_range_search_step`. By default 0, i.e.
  whole chunk
+ before start binary search, linear probing is done on each step (32
  rows currently)
 2022-06-29 23:30:35 +00:00
Dmitry Novik
4f7967ab9e
Merge pull request #37887 from azat/union-fix 
Fix incorrect columns order in subqueries of UNION
 2022-06-29 20:40:23 +02:00
alesapin
8f5582f95e Review and style fixes 2022-06-29 20:29:50 +02:00
alesapin
62e7a89f26 Proper fix for ipv4/ipv6 conversion error 2022-06-29 17:53:08 +02:00
Alexander Tokmakov
894acf0c5c fix tests 2022-06-29 16:27:21 +02:00
mergify[bot]
36139eacd7
Merge branch 'master' into dictinct_in_order_optimization 2022-06-29 13:37:16 +00:00
Alexander Tokmakov
c6c22409a2 Merge branch 'master' into replicated_database_improvements 2022-06-29 14:38:07 +02:00
Anton Popov
e78814f3bb
Merge pull request #38371 from CurtizJ/fix-distint-with-limit 
Fix `DISTINCT` with `LIMIT` in distributed queries
 2022-06-29 14:02:41 +02:00
Maksim Kita
eeae73e0cf SQL create drop index update implementation 2022-06-29 13:10:30 +02:00
mergify[bot]
1280fc39a0
Merge branch 'master' into better-exception-messages 2022-06-29 11:09:06 +00:00
Alexander Tokmakov
c3573532c0
Merge pull request #38487 from ClickHouse/add_some_tsa_annotations 
Add some TSA annotations
 2022-06-29 13:57:10 +03:00
Alexander Tokmakov
ceb66ade4b
Merge pull request #38335 from ClickHouse/deprecate_ordinary_database 
Deprecate Ordinary database and old *MergeTree syntax
 2022-06-29 13:42:59 +03:00
Maksim Kita
23e09b5e9b
Merge pull request #35166 from zhangjmruc/sql_compatible_create_drop_index 
Support SQL compatible create/drop index syntax
 2022-06-29 12:30:28 +02:00
Robert Schulze
bb441faa42
Merge pull request #38491 from ClickHouse/clang-tidy-in-branches 
Fix some clang-tidy warnings in headers
 2022-06-29 10:22:04 +02:00
Alexey Milovidov
4cda5491f6
Merge pull request #38542 from ClickHouse/unique_lock-to-lock_guard 
Don't use std::unique_lock unless we need to
 2022-06-29 09:40:56 +03:00
jianmei zhang
8d49aa4697 Support sql standard drop index in seprate parser files 2022-06-29 11:08:05 +08:00
jianmei zhang
589cba8045 Support sql standard create index in seprate parser files 2022-06-29 11:08:05 +08:00
Jianmei Zhang
4d0635371f Support sql stanard create and drop index 2022-06-29 11:08:05 +08:00
Nikita Taranov
f5d26572df
Quick fix for aggregation pipeline (#38295) 2022-06-29 01:16:30 +02:00
Azat Khuzhin
f86ddebada Add sanity check for UNION header 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-28 22:53:03 +03:00
Azat Khuzhin
d98336ad83 Fix incorrect columns order in subqueries of UNION 
Consider the following query:

    SELECT avgWeighted(x, y) FROM (SELECT NULL, 255 AS x, 1 AS y UNION ALL SELECT y, NULL AS x, 1 AS y)

Here is UNION from two SELECT queries
- `SELECT NULL, 255 AS x, 1 AS y`
- `SELECT y, NULL AS x, 1 AS y`

UNION queries matches columns by positions, not names, so the following
columns should be used by `avgWeighted()`:
- `255 AS x,  1 AS y`
- `NULL AS x, 1 AS y`

Result types of arguments should be:
- `x Nullable(UInt8)`
- `y UInt8`

And in case of UNION query is a subselect itself, it will return only
required columns, for the example above it needs only `x` and `y`.
For this it will get positions of these arguments from the first query,
and then use those positions to get required column names from the
second query (since there is no ability to get columns by positions
instead of names internally), and due to duplicated columns the second
query will return (`y`, `x`) not (`x`, `y`), and this will make the
result incorrect:

    EXPLAIN header = 1, optimize = 0, actions=1 SELECT avgWeighted(x, y) FROM (SELECT NULL, 255 AS x, 1 AS y UNION ALL SELECT y, NULL AS x, 1 AS y)

    Aggregates:
        avgWeighted(x, y)
          Function: avgWeighted(Nullable(UInt8), UInt8) → Nullable(Float64)
          Arguments: x, y
          Argument positions: 0, 1
      Expression (Before GROUP BY)
      Header: x UInt8
              y Nullable(UInt8)
      ...
      Union
      Header: x UInt8
              y Nullable(UInt8)
        Expression (Conversion before UNION)
        Header: x UInt8
                y Nullable(UInt8)
        Expression (Conversion before UNION)
        Header: x UInt8
                y Nullable(UInt8)

And the query itself fails with an error:

    Logical error: 'Bad cast from type DB::ColumnVector<char8_t> to DB::ColumnNullable'.

_NOTE: `avgWeighted()` here is required to trigger `LOGICAL_ERROR`_

CI: https://s3.amazonaws.com/clickhouse-test-reports/37796/e637489f81768df582fe7389e57f7ed12893087c/fuzzer_astfuzzerdebug,actions//report.html
Fixes: 02227_union_match_by_name
v2: fix untuple() (reserve space for output_columns_positions too)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-28 22:53:03 +03:00
Azat Khuzhin
ee0f2651ee Revert "Fix converting types for UNION queries (may produce LOGICAL_ERROR)" 
This fix is incorrect, and it introduce new issues, in particular it
may breaks UNION queries w/o column aliases, i.e.:

    SELECT a, b, c FROM (SELECT 3 AS a, 2147483647 AS b, 1048575 AS c UNION ALL SELECT -2, NULL, -2) AS js1 ORDER BY a

CI: https://s3.amazonaws.com/clickhouse-test-reports/37796/e637489f81768df582fe7389e57f7ed12893087c/fuzzer_astfuzzerdebug,actions//report.html
Reverts: #37593/#34775 (2613149f6b)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-28 22:50:01 +03:00
Robert Schulze
f692ead6ad
Don't use std::unique_lock unless we have to 
Replace where possible by std::lock_guard which is more light-weight.
 2022-06-28 19:19:06 +00:00
Alexander Tokmakov
4f54abf67f better names for macros 2022-06-28 16:27:18 +02:00
Alexander Tokmakov
021bdc335e Merge branch 'master' into add_some_tsa_annotations 2022-06-28 15:42:01 +02:00
Alexander Tokmakov
428628238f Merge branch 'master' into deprecate_ordinary_database 2022-06-28 15:40:33 +02:00
Robert Schulze
c22038d48b
More clang-tidy fixes 2022-06-28 11:50:05 +00:00
mergify[bot]
ea416c877f
Merge branch 'master' into better-exception-messages 2022-06-28 11:13:26 +00:00
Alexander Tokmakov
09a0688d80 fix 2022-06-28 00:26:52 +02:00
Alexander Tokmakov
851534c9f7 Merge branch 'master' into deprecate_ordinary_database 2022-06-27 23:47:49 +02:00
Alexander Tokmakov
f4883f1f7e add some TSA aanotations 2022-06-27 22:48:27 +02:00
mergify[bot]
a9c1b68034
Merge branch 'master' into dictinct_in_order_optimization 2022-06-27 20:16:00 +00:00
Dmitry Novik
b629d557ce
Merge pull request #38246 from ClickHouse/overcommit-deadlock-fix 
Fix deadlock in OvercommitTracker logging
 2022-06-27 21:57:31 +02:00
Dmitry Novik
1d15d72211 Support NULLs in ROLLUP 2022-06-27 18:42:26 +00:00
Nikolai Kochetov
6824570ea3 Fix style. 2022-06-27 18:10:29 +00:00
Nikolai Kochetov
99436ab1da Remove unneeded code. 2022-06-27 17:46:09 +00:00
Nikolai Kochetov
79cf4c4f7e Remove commented code. 2022-06-27 17:44:30 +00:00
Nikolai Kochetov
1d5ba77c87 Fix test. 2022-06-27 17:41:55 +00:00
Alexander Tokmakov
2283436d24
Merge pull request #38344 from Algunenano/implicit_transactions 
Add implicit_transaction setting
 2022-06-27 18:50:37 +03:00
Alexander Tokmakov
88dbbcb15d fix tests 2022-06-27 16:10:00 +02:00
Alexander Tokmakov
bec921c9d9
Merge pull request #38265 from ClickHouse/cleanup_garbage_in_store_dir 
Add background cleanup of store/ subdirs
 2022-06-27 16:53:13 +03:00
Alexander Tokmakov
7c37079a96 Merge branch 'master' into deprecate_ordinary_database 2022-06-27 14:46:59 +02:00
Anton Kozlov
5f81bcd84f
CLICKHOUSE-1331 Rewrite tuple functions as literals in backwards-compatibility mode (#38096) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-06-27 14:13:21 +02:00
kssenii
5c26f99bc5 Merge master 2022-06-27 13:36:41 +02:00
Nikita Taranov
2487ba7f00
Move updateInputStream to ITransformingStep (#37393) 2022-06-27 13:16:52 +02:00
mergify[bot]
da52780cb5
Merge branch 'master' into implicit_transactions 2022-06-27 09:46:34 +00:00
mergify[bot]
f63c959679
Merge branch 'master' into cleanup_garbage_in_store_dir 2022-06-26 13:35:10 +00:00
Igor Nikonov
aca6a6f1b0 Merge remote-tracking branch 'origin/master' into dictinct_in_order_optimization 2022-06-26 09:45:59 +00:00
Alexey Milovidov
a45e3d47ad Remove useless codec from system.asynchronous_metric_log 2022-06-25 07:16:25 +02:00
mergify[bot]
b65cf4e1fe
Merge branch 'master' into dictinct_in_order_optimization 2022-06-24 22:52:14 +00:00
kssenii
708b93a03b Merge master 2022-06-25 00:08:24 +02:00
Alexander Gololobov
189d0fffb8
Merge pull request #37165 from ClickHouse/merge_tree_reader 
Merge tree reader support for multiple read/filter steps: row level filter, prewhere, ...
 2022-06-24 21:32:31 +02:00
Alexander Tokmakov
94dd80e900 fix tests 2022-06-24 19:10:33 +02:00
Alexander Tokmakov
31dcc7634e Merge branch 'master' into deprecate_ordinary_database 2022-06-24 18:16:07 +02:00
Alexander Tokmakov
59a14d2a50 add docs 2022-06-24 17:34:41 +02:00
Alexander Tokmakov
379fecf7f9 try suppress failures with MaterializedPostgreSQL 2022-06-24 17:15:15 +02:00
Raúl Marín
072f64c800 Improvements based on review 2022-06-24 16:54:47 +02:00
mergify[bot]
36b93772b9
Merge branch 'master' into merge_tree_reader 2022-06-24 13:22:53 +00:00
Anton Popov
a2158066f2 fix distinct with limit in distributed queries 2022-06-24 02:32:40 +00:00
mergify[bot]
a0eb831cd1
Merge branch 'master' into overcommit-deadlock-fix 2022-06-23 23:02:59 +00:00
avogar
59c1c472cb Better exception messages on wrong table engines/functions argument types 2022-06-23 20:04:06 +00:00
Alexander Tokmakov
74f38710a8 Merge branch 'master' into cleanup_garbage_in_store_dir 2022-06-23 21:43:28 +02:00
kssenii
6fbd49f554 Merge master 2022-06-23 21:40:01 +02:00
Alexander Tokmakov
dbf2763788 automatically convert system database to Atomic 2022-06-23 21:38:43 +02:00
Kseniia Sumarokova
40a4742f93
Merge pull request #38319 from kssenii/fix-access-for-drop-fs-cache 
Add access check and on cluster option for system drop fs cache
 2022-06-23 20:56:44 +02:00
kssenii
468c98ff42 Better 2022-06-23 17:46:27 +02:00
mergify[bot]
174d8577fa
Merge branch 'master' into merge_tree_reader 2022-06-23 15:16:51 +00:00
Raúl Marín
7f7d082fb3 Add implicit_transaction setting 2022-06-23 15:23:37 +02:00
kssenii
f7b329ee57 Merge master 2022-06-23 14:56:48 +02:00
mergify[bot]
b5d3fd50d2
Merge branch 'master' into dictinct_in_order_optimization 2022-06-23 09:48:38 +00:00
Alexander Tokmakov
2c5a88faba deprecate Ordinary database 2022-06-23 10:20:14 +02:00
Antonio Andelic
a5ee918bd2
Merge pull request #38047 from ClickHouse/keeper-sequential-consistency-reads 
Add `SYNC` command to internal ZooKeeper client
 2022-06-23 08:43:24 +02:00
kssenii
e40d9bcf55 Merge master 2022-06-22 23:28:52 +02:00
Anton Popov
52db1b35a1 improve performace of insertion to columns of type JSON 2022-06-22 17:45:51 +00:00
Alexander Tokmakov
ed8341025b make code less bad 2022-06-22 18:31:42 +02:00
kssenii
dce1c76270 Fix 2022-06-22 18:00:25 +02:00
Alexander Gololobov
a5b3f83181 Always remove row filter column 2022-06-22 17:23:09 +02:00
Alexander Gololobov
6a26325fab Test dirty hacks for multiple PREWHERE steps 2022-06-22 17:23:05 +02:00
Alexander Tokmakov
31c4f83469
Merge pull request #38272 from ClickHouse/create_empty_as_select 
Support CREATE ... EMPTY AS SELECT
 2022-06-22 13:25:12 +03:00
Nikolai Kochetov
cc6fdfe0eb
Merge pull request #36555 from ClickHouse/refactor-something-in-part-volumes 
Separate data storage abstraction for MergeTree
 2022-06-22 11:13:36 +02:00
Alexander Tokmakov
dc1f596326 fix 2022-06-22 00:50:16 +02:00
mergify[bot]
f45b4f56d8
Merge branch 'master' into dictinct_in_order_optimization 2022-06-21 21:25:37 +00:00
Igor Nikonov
b0a98bd875 DISTINCT in order optimization 
+ use SortDescription from input data stream in DistinctStep to decide if the optimization is applicable
 2022-06-21 21:23:49 +00:00
Anton Popov
d5d568e235
Merge pull request #37913 from azat/parts-refcnt 
Fix refcnt for unused MergeTree parts in SELECT queries
 2022-06-21 22:56:26 +02:00
kssenii
4178abc0a5 Merge master 2022-06-21 22:38:51 +02:00
kssenii
90ce0b44f0 Fixes 2022-06-21 21:03:18 +02:00
kssenii
481644939a Add describe cache command 2022-06-21 16:07:39 +02:00
Alexander Tokmakov
13e3d40db4 fix 2022-06-21 15:11:03 +02:00
Alexander Tokmakov
f3ea8ad745 support CREATE EMPTY AS SELECT 2022-06-21 15:03:58 +02:00
Nikolai Kochetov
543782436d Process additional filters. 2022-06-21 11:24:46 +00:00
Nikolai Kochetov
1e8c9ecd4c Merge branch 'master' into refactor-something-in-part-volumes 2022-06-21 12:37:21 +02:00
Alexander Tokmakov
ba0fcec993 add background cleanup of store/ subdirs 2022-06-21 12:35:47 +02:00
Nikolai Kochetov
b8d27aa8dd
Merge pull request #37469 from azat/projections-optimize_aggregation_in_order 
Implement in order aggregation (optimize_aggregation_in_order) for projections for tables with fully materialized projections
 2022-06-21 12:17:35 +02:00
mergify[bot]
bdd5b94192
Merge branch 'master' into keeper-sequential-consistency-reads 2022-06-21 08:13:04 +00:00
lingpeng0314
c18dd55400 Resolve comments 
- refine metric description
 2022-06-21 10:30:19 +08:00
Nikolai Kochetov
dccf90b1ea Cleanup. 2022-06-20 18:18:17 +00:00
mergify[bot]
9bdd9e14a6
Merge branch 'master' into fix_flaky_tests_with_transactions 2022-06-20 18:11:30 +00:00
Dmitry Novik
f6692c34e6
Merge pull request #34632 from excitoon-favorites/optimizedprocessing 
Optimized processing of ORDER BY in window functions
 2022-06-20 20:03:26 +02:00
Dmitry Novik
53d656d89f Fix deadlock in OvercommitTracker logging 2022-06-20 17:35:24 +00:00
Antonio Andelic
777ae72259 Merge branch 'master' into keeper-sequential-consistency-reads 2022-06-20 14:06:45 +00:00
Antonio Andelic
df31b562ec Use sync after reconnect to avoid stale reads 2022-06-20 14:05:51 +00:00
Nikolai Kochetov
7452d04e3a Merge branch 'master' into refactor-something-in-part-volumes 2022-06-20 15:31:02 +02:00
kssenii
90be49faa4 Merge master 2022-06-20 14:26:56 +02:00
Dmitry Novik
77177917bd
Merge pull request #37848 from ClickHouse/window-function-expression 
Support expressions with window functions
 2022-06-20 14:21:13 +02:00
Vitaly Baranov
06c4082a7c
Merge pull request #37358 from vitlibar/backup-improvements-6 
Backup Improvements 6
 2022-06-20 14:13:30 +02:00
mergify[bot]
4ed606a275
Merge branch 'master' into fix_flaky_tests_with_transactions 2022-06-20 10:17:51 +00:00
Igor Nikonov
6ac68e8303 DISTINCT in order optimization 
+ optimization for DISTINCT containing primary key columns
 2022-06-20 10:06:15 +00:00
lingpeng0314
1d463718ca Change getKeeperDispatcher error type 2022-06-20 17:11:15 +08:00
lingpeng0314
f06b19bdbf Add keeper monitoring data 2022-06-20 11:32:18 +08:00
Vladimir Chebotarev
bfb67ded2c Rebase fix. 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
b81df2be83 Update src/Interpreters/ExpressionAnalyzer.cpp 
Co-authored-by: Dmitry Novik <d1mas1k4@yandex.ru>
 2022-06-20 05:15:08 +03:00
Vladimir Chebotarev
7f9557f8a3 Added optimize_read_in_window_order setting. 2022-06-20 05:15:07 +03:00
Vladimir Chebotarev
ec22f6d539 Draft. 2022-06-20 05:15:07 +03:00
Vitaly Baranov
638ea23399 Fix build. 2022-06-20 03:44:59 +02:00
mergify[bot]
670a63865e
Merge branch 'master' into window-function-expression 2022-06-19 22:14:54 +00:00
Vitaly Baranov
8a7c970ce0 Fix style. 2022-06-19 15:58:26 +02:00
alesapin
50801e41c5 Merge branch 'master' into refactor-something-in-part-volumes 2022-06-19 14:05:46 +02:00
Vitaly Baranov
01aaaf7395 More accurate access checking for RESTORE. 2022-06-19 11:26:41 +02:00
Antonio Andelic
dd75e9cba5
Merge pull request #38149 from vdimir/multiple_joins_original_names_34697 
Add setting multiple_joins_try_to_keep_original_names
 2022-06-18 14:47:31 +02:00
Dmitry Novik
8b917d12f2 Small refactoring 2022-06-18 01:40:30 +00:00
Maksim Kita
7e759cac39
Merge pull request #38184 from kitaisreal/sql-user-defined-function-loader-allow-symlinks 
SQLUserDefinedFunctionsLoader allow symlinks in user_defined directory
 2022-06-17 23:03:52 +02:00
Antonio Andelic
f72e509b3b
Merge pull request #38052 from amosbird/join_regression_fix 
Fix significant join performance regression
 2022-06-17 19:55:33 +02:00
Maksim Kita
965b706350 SQLUserDefinedFunctionsLoader allow symlinks in user_defined directory 2022-06-17 17:43:42 +02:00
Alexander Tokmakov
e9c2157f01 Merge branch 'master' into fix_flaky_tests_with_transactions 2022-06-17 15:57:30 +02:00
Alexander Gololobov
f69f24ffbd
Merge pull request #38135 from vdimir/issue_37928 
Try to fix temporary name clash in partial merge join
 2022-06-17 15:10:19 +02:00
Alexander Tokmakov
5ffc38d9b9 fix style check, add comments 2022-06-17 13:47:14 +02:00
Maksim Kita
77c519ae08
Merge pull request #38142 from kitaisreal/executable-user-defined-functions-parameters-direct-fix 
Disable parameters for non direct executable user defined functions
 2022-06-17 12:57:27 +02:00
alesapin
1c7a7da487
Merge pull request #38088 from kssenii/diff-with-cache 
Extract some diff from pr #36171
 2022-06-17 12:07:31 +02:00
lgbo-ustc
308ca62518 fixed bugs 2022-06-17 15:37:54 +08:00
lgbo-ustc
4a0a6e8cde small optimization 2022-06-17 15:17:56 +08:00
lgbo-ustc
14609e043c fixed style 2022-06-17 15:15:09 +08:00
lgbo-ustc
55757dfb34 fixed bugs 2022-06-17 15:13:31 +08:00
lgbo-ustc
68c5885851 fixed bugs 2022-06-17 12:19:25 +08:00
lgbo-ustc
285bb44b25 nothing 2022-06-17 09:33:16 +08:00
Anton Popov
1523c9c9e5 fix filling of empty Nested + small refactoring 2022-06-17 01:10:52 +00:00
Alexander Tokmakov
39c0219c11 fixes 2022-06-16 19:41:32 +02:00
Dmitry Novik
500895306f Fix typo 2022-06-16 16:06:59 +00:00
Dmitry Novik
644399db8a Add comment and update docs 2022-06-16 15:59:51 +00:00
vdimir
2a9942ee80
Add setting multiple_joins_try_to_keep_original_names 2022-06-16 15:50:03 +00:00
Dmitry Novik
8479d880f3 Remove redundant code 2022-06-16 15:49:49 +00:00
Dmitry Novik
376412e417 Small refactoring 2022-06-16 15:41:04 +00:00
Dmitry Novik
f9b71551d0 Cleanup ActionsVisitor 2022-06-16 15:11:16 +00:00
kssenii
797482f132 Better 2022-06-16 16:40:29 +02:00
Maksim Kita
aff66742e7 Disable parameters for non direct executable user defined functions 2022-06-16 16:29:04 +02:00
Dmitry Novik
0663f07e67 Rework expressions with window functions 2022-06-16 13:29:56 +00:00
vdimir
d50c51f262
Try to fix temporary name clash in partial merge join 2022-06-16 12:47:55 +00:00
lgbo-ustc
9753ea9157 fixed a bug 
fixed a bug in TranslateQualifiedNamesMatcher to handle nested tuple field with the table qulified name
 2022-06-16 19:27:30 +08:00
Alexey Milovidov
f2b665a799
Merge pull request #38120 from wangdh15/delete-unused-field-fix-build-error 
Remove unused class member
 2022-06-16 11:11:26 +03:00
Azat Khuzhin
0d4f78639e Interpreters/Aggregator: cleaner interface for block release during merge 
Suggested-by: @amosbird
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-16 09:58:36 +03:00
Azat Khuzhin
4694929623 Implement merging only for AggregatingStep 
v2: fill AggregateColumnsConstData only for only_merge
    (fixes 01291_aggregation_in_order and some other tests)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-16 09:58:36 +03:00
wangdh15
02cce40b3a when using clang12 compile, the unused filed shard_count will cause compile error. So delete it. 2022-06-16 10:43:31 +08:00
Anton Popov
5b01f0f341 Merge remote-tracking branch 'upstream/master' into HEAD 2022-06-15 23:01:22 +00:00
Alexander Tokmakov
5e2b42c85a Merge branch 'master' into fix_flaky_tests_with_transactions 2022-06-15 20:47:02 +02:00
Vitaly Baranov
0102626532 Disable the 'BACKUP ALL' command (it's not quite clear what to do with predefined databases). 2022-06-15 20:32:35 +02:00
Vitaly Baranov
1198e86295 Fix storing temporary tables and skipping system tables while making a backup. 2022-06-15 20:32:34 +02:00
Vitaly Baranov
592f568f83 Move backup/restore code to storages and databases - part 2. 2022-06-15 20:32:31 +02:00
Daniel Kutenin
a769dea8ef
Merge branch 'ClickHouse:master' into master 2022-06-15 19:12:10 +01:00
kssenii
416c6008ee Merge master 2022-06-15 16:50:25 +02:00
kssenii
500f49972b Extract diff from PR 36171 2022-06-15 16:40:18 +02:00
Alexander Tokmakov
e4d4d0a854 fix race on queue processing 2022-06-15 16:21:26 +02:00
Nikolai Kochetov
2a9a63ac7b Merge branch 'master' into refactor-something-in-part-volumes 2022-06-15 15:35:26 +02:00
Danila Kutenin
08e3f77a9c Optimize most important parts with NEON SIMD 
First part, updated most UTF8, hashing, memory and codecs. Except
utf8lower and upper, maybe a little later.

That includes huge amount of research with movemask dealing. Exact
details and blog post TBD.
 2022-06-15 13:19:29 +00:00
Yakov Olkhovskiy
e604d31feb
Merge pull request #38074 from ClickHouse/bug-with-fill-extra 
Inconsistency in ORDER BY ... WITH FILL feature.
 2022-06-15 08:19:55 -04:00
mergify[bot]
2cb9579234
Merge branch 'master' into join_regression_fix 2022-06-15 11:53:42 +00:00
Yakov Olkhovskiy
411695bd97 do not fill 'to' boundary 2022-06-14 22:26:50 -04:00
Alexander Tokmakov
b39bd3c4e0 fix 2022-06-14 16:11:16 +02:00
Amos Bird
9a6e6ccfaf
Fix significant join performance regression 2022-06-14 21:14:18 +08:00
Maksim Kita
da8b1b1eba
Merge pull request #38025 from kitaisreal/use-base-sort-instead-of-standard 
Use pdqsort instead of standard sort
 2022-06-14 12:13:35 +02:00
Dmitry Novik
eddd4ecaeb Allow several functions in window expression 2022-06-14 00:57:33 +00:00
Dmitry Novik
0c6ed20b42 Fix window expression evaluation 2022-06-13 23:11:53 +00:00
kssenii
4813b90281 Merge master 2022-06-13 21:32:00 +02:00
Antonio Andelic
b9cf6fe367 Merge branch 'master' into pp-system-unfreeze 2022-06-13 14:46:30 +00:00
Maksim Kita
98a89b50ff Use pdqsort instead of standard sort 2022-06-13 15:31:08 +02:00
Nikolai Kochetov
b537e91487 parse additional filetrs. 2022-06-13 12:18:45 +00:00
kssenii
aaf9a46f07 Merge master 2022-06-11 15:56:05 +02:00
mergify[bot]
9e0d16f21d
Merge branch 'master' into window-function-expression 2022-06-10 15:31:53 +00:00
Kseniia Sumarokova
580a30c6ae
Merge pull request #37859 from KinderRiven/support_max_request_cache_size 
Support to set max cache size for per query in local cache
 2022-06-10 16:17:18 +02:00
Robert Schulze
5f5732a2c4
Merge pull request #37969 from ClickHouse/consistent-macro-usage 
More consistent use of platform macros
 2022-06-10 14:10:01 +02:00
Kseniia Sumarokova
508676ab4e
Merge pull request #37804 from ClickHouse/metadata_storage 
Metadata storage abstraction for Disks
 2022-06-10 11:43:28 +02:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros 
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.

__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
 2022-06-10 10:22:31 +02:00
Alexander Tokmakov
780f7c87c7
Merge pull request #36113 from ClickHouse/remove-useless-code-2 
Remove useless code in ReplicatedMergeTreeRestartingThread
 2022-06-09 17:56:28 +03:00
Ilya Yatsishin
d6427f56f3
Opentracing minimal changes for processors (#37837) 2022-06-09 14:43:50 +02:00
alesapin
84ab147155 Merge branch 'master' into metadata_storage 2022-06-08 22:33:12 +02:00
Vadim Volodin
637d293fbd Add SYSTEM UNFREEZE query 2022-06-08 15:21:14 +03:00
Nikolai Kochetov
5bc9b32025 Merge branch 'master' into refactor-something-in-part-volumes 2022-06-08 11:10:06 +00:00
KinderRiven
6883ecda96 fix 2022-06-08 14:19:50 +08:00
Azat Khuzhin
61a2613127 Fix refcnt for unused MergeTree parts in SELECT queries 
Before this patch SELECT queries hold parts even if they were not
required by select (had been eliminated by partition pruning).

This defers removing parts if you have long running queries.

This had been introduced in #23932, with introduction of
StorageSnapshotPtr.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-08 07:04:53 +03:00
Nikolai Kochetov
678d978acf Merge branch 'master' into refactor-something-in-part-volumes 2022-06-07 15:23:00 +00:00
mergify[bot]
2d01abf871
Merge branch 'master' into revert-37647-Fix-all-CheckTriviallyCopyableMove-Errors 2022-06-07 13:32:30 +00:00
Igor Nikonov
dcad154105
Merge pull request #37866 from ClickHouse/igor_minor_cleanup 
Minor cleanup
 2022-06-07 15:24:56 +02:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type" 2022-06-07 13:53:10 +02:00
Vladimir C
1c184a7e18
Merge pull request #37840 from azat/join-query-fatal-errors 2022-06-07 12:07:37 +02:00
Kseniia Sumarokova
4272ca8836
Merge pull request #36060 from Varinara/master 
Add feature disks (ls, C, list-disks, disk, help)
 2022-06-07 11:22:23 +02:00
Vitaly Baranov
d199478169
Merge pull request #37303 from ClickHouse/fix_trash 
Try to fix some trash
 2022-06-07 10:17:39 +02:00
KinderRiven
3b2a427002 fix 2022-06-07 11:50:56 +08:00
mergify[bot]
7a76a7a606
Merge branch 'master' into master 2022-06-06 19:44:32 +00:00
Anton Popov
d40b23272e
Merge pull request #37755 from CurtizJ/fix-mutations-again 
Return back #37266
 2022-06-06 20:22:03 +02:00
mergify[bot]
58c3f31bf4
Merge branch 'master' into bug-with-fill-date 2022-06-06 15:10:33 +00:00
Maksim Kita
612172def5
Merge pull request #37720 from kitaisreal/executable-user-defined-functions-parameters 
Executable user defined functions support parameters
 2022-06-06 16:51:56 +02:00
Varinara
ed6e8176fe Add basic commands for disk tool (list-disks, list, move, remove, link, copy, read, write) + tests 2022-06-06 16:52:58 +03:00
Yakov Olkhovskiy
52ae3f0e92 full refactoring - move type check to transform 2022-06-05 22:39:21 -04:00
KinderRiven
13e74bef5e support skip_download_if_exceeds_query_cache 2022-06-06 02:36:23 +08:00
Robert Schulze
2d87af2a15
Merge pull request #37647 from DevTeamBK/Fix-all-CheckTriviallyCopyableMove-Errors 
Fix errors of CheckTriviallyCopyableMove type
 2022-06-05 19:58:47 +02:00
Igor Nikonov
13149dc094 Minor cleanup 2022-06-05 14:31:07 +00:00
KinderRiven
3ffb0b3549 support_max_query_cache_size 2022-06-05 16:21:36 +08:00
Yakov Olkhovskiy
07009c27c2 tidy build suggestions 2022-06-04 17:24:48 -04:00
Yakov Olkhovskiy
adf305e1dd search source header for columns 2022-06-04 11:44:20 -04:00
Yakov Olkhovskiy
0a063820c7 WITH FILL of date/time type must be equal to type of ORDER BY column 2022-06-04 01:35:41 -04:00
Dmitry Novik
58d79f8eed Support expressions with window functions 2022-06-04 02:01:49 +00:00
Maksim Kita
d14193b3ad Executable user defined functions extract parameters name and type from command value 2022-06-03 18:06:59 +02:00
Maksim Kita
6c5ec68a7c Executable user defined functions support parameters 2022-06-03 18:05:37 +02:00
Azat Khuzhin
c29768325c Print query in one line on fatal errors 
executeQuery() logging query without new lines in SQL, and it is useful,
since you can grep and see the whole query (usually).

So let's use the same in fatal error handler to make CI reports more
informative.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-03 18:55:53 +03:00
mergify[bot]
ddf7210ecc
Merge branch 'master' into remove-useless-code-2 2022-06-03 13:58:45 +00:00
mergify[bot]
70c4968cc5
Merge branch 'master' into metadata_storage 2022-06-03 13:25:31 +00:00
Anton Popov
3ace077401
Merge pull request #37695 from CurtizJ/replace-multiif-to-if 
Replace `multiIf` to `if` in case of one condition
 2022-06-03 14:56:43 +02:00
alesapin
ac03e3af3e
Merge branch 'master' into metadata_storage 2022-06-03 14:42:21 +02:00
alesapin
5d30a4ef09 Fix typos and buge 2022-06-03 14:35:15 +02:00
Anton Popov
5cf2c67410
Merge branch 'master' into fix-mutations-again 2022-06-03 14:17:05 +02:00
Nikolai Kochetov
176af473c3 Fix build. 2022-06-02 19:38:47 +00:00
Nikolai Kochetov
8991f39412 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-06-02 17:00:08 +00:00
Alexander Tokmakov
58f8c87265
Merge pull request #36937 from bigo-sg/fix_settings 
Fix overrided settings: normalize_function_names
 2022-06-02 17:45:17 +03:00
HeenaBansal2009
e3080f2a97 Merge remote-tracking branch 'origin' into Fix-all-CheckTriviallyCopyableMove-Errors 2022-06-02 07:30:08 -07:00
Alexander Tokmakov
cce9057eef fix style check 2022-06-02 15:27:32 +02:00
mergify[bot]
fbfaaa9c74
Merge branch 'master' into fix-mutations-again 2022-06-02 11:57:19 +00:00
Nikolai Kochetov
85f4295370 Cleanup 2022-06-02 10:34:40 +00:00
Nikolai Kochetov
fc726c236f
Merge pull request #37661 from ClickHouse/no_alias_actions 
Trying to remove prewhere_info.alias_actions as they seem to always be included in prewhere_actions
 2022-06-02 12:31:04 +02:00
Nikolai Kochetov
89c5855d20 Merge branch 'master' into refactor-something-in-part-volumes 2022-06-02 12:19:07 +02:00
Kseniia Sumarokova
b24f34dd93
Merge pull request #37585 from Vxider/fix-temp-table-drop 
Flush and shutdown temporary table before drop
 2022-06-02 12:07:50 +02:00
Kseniia Sumarokova
9f0ebb4eac
Merge pull request #37754 from Avogar/fix-values-func 
Fix possible logical error in values table function
 2022-06-02 12:01:24 +02:00
Nikita Mikhaylov
d34e051c69
Support for simultaneous read from local and remote parallel replica (#37204) 2022-06-02 11:46:33 +02:00
Nikolai Kochetov
edac3d6714 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-06-02 09:36:20 +00:00
Maksim Kita
cb93135326
Merge pull request #37711 from kitaisreal/executable-user-defined-functions-nullable-arguments-fix 
Fix executable user default functions execution with Nullable arguments
 2022-06-02 11:05:34 +02:00
Nikolay Degterinsky
9575a6d048
Merge pull request #37587 from bigo-sg/typo 
Fix a typo
 2022-06-02 02:18:03 +02:00
avogar
7ef02a2e44 Fix possible logical error in values table function 2022-06-01 15:32:33 +00:00
Dmitry Novik
7fbe91ca81
Merge pull request #37460 from ClickHouse/memory-overcommit-improvement 
Memory Overcommit: update defaults, exception message and add ProfileEvent
 2022-06-01 17:06:33 +02:00
Sema Checherinda
16dc3ed97d FR: Expose what triggered the merge in system.part_log #26255 2022-06-01 16:58:07 +02:00
Sema Checherinda
2626a49616 FR: Expose what triggered the merge in system.part_log #26255 2022-06-01 16:58:06 +02:00
Alexander Tokmakov
3d346c766a better code 2022-06-01 16:49:26 +02:00
kssenii
aa60de8f63 Merge master 2022-06-01 15:32:01 +02:00
Alexander Tokmakov
75f49a48e1 Merge branch 'master' into fix_trash 2022-06-01 14:20:46 +02:00
Anton Popov
011131198c Merge remote-tracking branch 'upstream/master' into fix-mutations-again 2022-06-01 12:04:18 +00:00
Robert Schulze
ee302f2d9f
Merge pull request #37643 from amosbird/avoid-useless-context-copy 
Avoid useless context copy when building query interpreters
 2022-06-01 13:49:56 +02:00
Vladimir C
8c0dba7302
Merge pull request #37650 from amosbird/joinget-fix 
Fix joinGet with  join_use_nulls = 1 and Array type
 2022-06-01 13:30:29 +02:00
Vladimir C
c466cdebf4
Merge pull request #37530 from vdimir/join_cond_dict_issue_37386 2022-06-01 13:29:01 +02:00
Anton Popov
cae2478b3f Revert "Merge pull request #37355 from ClickHouse/revert-37266-fix-mutations-with-object" 
This reverts commit a53cfa9fca, reversing
changes made to 9acb42fcdb.
 2022-06-01 10:57:20 +00:00
Alexey Milovidov
31b3350749
Merge pull request #37710 from ClickHouse/fix-grouping-function 
Make GROUPING function skip constant folding
 2022-06-01 12:00:14 +03:00
Alexey Milovidov
a0020cb55c
Merge pull request #37724 from CurtizJ/fix-ast-optimizations-remote 
Fix `optimize_monotonous_functions_in_order_by` in distributed queries
 2022-06-01 11:54:45 +03:00
Anton Popov
6cf9405f09 fix optimize_monotonous_functions_in_order_by in distributed queries 2022-06-01 00:50:28 +00:00
Nikolai Kochetov
86fbb74703 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-31 18:07:47 +00:00
vdimir
284c9bc68b
Rollback some changes from appendFromBlock 2022-05-31 17:40:35 +00:00
Maksim Kita
66f43b9ad3 Fix executable user default functions execution with Nullable arguments 2022-05-31 18:46:33 +02:00
Dmitry Novik
b11749ca2c Make GROUPING function skip constant folding 2022-05-31 16:45:29 +00:00
vdimir
e7be677fca
Assert structure match up to locard in appendFromBlock 2022-05-31 16:02:58 +00:00
vdimir
7f4ddb1667
Fix assert for 02244_lowcardinality_hash_join 2022-05-31 16:02:57 +00:00
vdimir
2476c6a988
Fix error on joining with dictionary on some conditions 2022-05-31 16:02:57 +00:00
Vladimir C
2a38fdb796
Merge pull request #37653 from vdimir/cross_join_dup_col_names 2022-05-31 17:50:19 +02:00
Anton Popov
00f87b0f57 replace multiIf to if in case of one condition 2022-05-31 14:45:12 +00:00
Nikolai Kochetov
147a819221 Refactor a little bit more. 2022-05-31 14:43:38 +00:00
Dmitry Novik
b41fe00f31
Merge pull request #37542 from azat/grouping-sets-fix-optimize_aggregation_in_order 
Prohibit optimize_aggregation_in_order with GROUPING SETS (fixes LOGICAL_ERROR)
 2022-05-31 15:31:45 +02:00
Dmitry Novik
f58623a375
Merge pull request #37593 from azat/union-type-cast-resubmit 
Fix converting types for UNION queries (may produce LOGICAL_ERROR)
 2022-05-31 15:27:50 +02:00
mergify[bot]
ba49c6bb46
Merge branch 'master' into memory-overcommit-improvement 2022-05-31 13:17:06 +00:00
mergify[bot]
f90dddccba
Merge branch 'master' into fix-temp-table-drop 2022-05-31 09:10:00 +00:00
zhanglistar
53020b096d
Merge branch 'ClickHouse:master' into typo 2022-05-31 11:28:12 +08:00
kssenii
bdfd166eb1 Merge master 2022-05-30 21:31:47 +02:00
Nikolai Kochetov
77b07dd0a8
Merge pull request #37163 from ClickHouse/grouping-function 
Add GROUPING function
 2022-05-30 20:45:04 +02:00
Nikolai Kochetov
913e7a91ae Fix limits from subquery. 2022-05-30 18:25:17 +00:00
HeenaBansal2009
b7eb6bbd38 Fixed clang-tidy-CheckTriviallyCopyableMove-errors 2022-05-30 11:09:03 -07:00
Alexander Gololobov
e2dd6f6249 Removed prewhere_info.alias_actions 2022-05-30 19:58:23 +02:00
vdimir
8a3f4bda62
Fix columns number mismatch in cross join 2022-05-30 15:40:15 +00:00
Amos Bird
b68e8efaf0
Fix joinGet with cannot be null type 2022-05-30 21:01:27 +08:00
Nikolai Kochetov
5b4658aa5e Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-30 09:47:35 +00:00
Amos Bird
6525bfc4cd
Avoid context copy for InterpreterSelects 2022-05-30 17:08:12 +08:00
Azat Khuzhin
78bd47d8df Fix excessive LIST requests to coordinator for transactions 
In [1] there was only few transactions, but lots of List for /test/clickhouse/txn/log:

    $ clickhouse-local --format TSVWithNamesAndTypes --file zookeeper_log.tsv.gz -q "select * except('path|session_id|event_time|thread_id|event_date|xid') apply(x->groupUniqArray(x)), path, session_id, min(event_time), max(event_time), count() from table where has_watch and type = 'Request' group by path, session_id order by count() desc limit 1 format Vertical"
    Row 1:
    ──────
    groupUniqArray(type):             ['Request']
    groupUniqArray(query_id):         ['','62d75128-9031-48a5-87ba-aec3f0b591c6']
    groupUniqArray(address):          ['::1']
    groupUniqArray(port):             [9181]
    groupUniqArray(has_watch):        [1]
    groupUniqArray(op_num):           ['List']
    groupUniqArray(data):             ['']
    groupUniqArray(is_ephemeral):     [0]
    groupUniqArray(is_sequential):    [0]
    groupUniqArray(version):          []
    groupUniqArray(requests_size):    [0]
    groupUniqArray(request_idx):      [0]
    groupUniqArray(error):            []
    groupUniqArray(watch_type):       []
    groupUniqArray(watch_state):      []
    groupUniqArray(stat_version):     [0]
    groupUniqArray(stat_cversion):    [0]
    groupUniqArray(stat_dataLength):  [0]
    groupUniqArray(stat_numChildren): [0]
    groupUniqArray(children):         [[]]
    path:                             /test/clickhouse/txn/log
    session_id:                       1
    min(event_time):                  2022-05-27 12:54:09.025897
    max(event_time):                  2022-05-27 13:37:12.846314 <!-- last transaction was at 12:54, see server log
    count():                          3673675 <-- huge

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/37593/2613149f6bf4f242bbbf2c3c8539b5176fd77286/stateless_tests__debug__actions__[1/3].html

Server log:

    $ pigz -cd clickhouse-server.log.gz | fgrep TransactionLog: | head -1
    2022.05.27 12:54:09.026852 [ 5018 ] {62d75128-9031-48a5-87ba-aec3f0b591c6} <Trace> TransactionLog: Loading 33 entries from /test/clickhouse/txn/log: csn-0000000000..csn-0000000032
    $ pigz -cd clickhouse-server.log.gz | fgrep TransactionLog: | tail -1
    2022.05.27 12:54:58.909222 [ 509 ] {} <Test> TransactionLog: Closing readonly transaction (177, 38, 41b51ff1-bcba-43bf-bcea-e97ad05f6040)
    $ pigz -cd clickhouse-server.log.gz | fgrep 62d75128-9031-48a5-87ba-aec3f0b591c6 | tail -1
    2022.05.27 12:54:09.064857 [ 5018 ] {62d75128-9031-48a5-87ba-aec3f0b591c6} <Debug> MemoryTracker: Peak memory usage (for query): 0.00 B.

Fixes: #37398 (cc @tavplubix)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-30 11:39:34 +03:00
taiyang-li
0b0d38b18c Merge branch 'master' into fix_settings 2022-05-30 10:07:09 +08:00
Alexey Milovidov
73e2e63414
Merge pull request #37612 from ClickHouse/clang-tidy-14 
Fix clang-tidy-14, part 1
 2022-05-29 03:16:32 +03:00
Alexander Tokmakov
4e52f45695 Merge branch 'master' into fix_trash 2022-05-28 19:43:19 +02:00
Anton Popov
1d9b3be7da
Merge pull request #37536 from CurtizJ/profile-events-for-part-types 
Add profile events for introspection of part types
 2022-05-28 14:25:21 +02:00
Alexey Milovidov
c50791dd3b Fix clang-tidy-14, part 1 2022-05-27 22:52:14 +02:00
Nikolai Kochetov
b80b1940ce Fix some tests. 2022-05-27 20:47:35 +00:00
Alexander Tokmakov
7e659036f8 fix 2022-05-27 20:30:06 +02:00
Dmitry Novik
60b9d81773 Remove global_memory_usage_overcommit_max_wait_microseconds 2022-05-27 16:30:29 +00:00
kssenii
25b1ed526d Merge master 2022-05-27 16:50:45 +02:00
Azat Khuzhin
8a224239c1 Prohibit optimize_aggregation_in_order with GROUPING SETS 
AggregatingStep ignores it anyway, and it leads to the following error
in getSortDescriptionFromGroupBy(), like in [1]:

    2022.05.24 04:29:29.279431 [ 3395 ] {26543564-8bc8-4a3a-b984-70a2adf0245d} <Fatal> : Logical error: 'Trying to get name of not a column: ExpressionList'.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36914/67d3ac72d26ab74d69f03c03422349d4faae9e19/stateless_tests__ubsan__actions_.html

v2: revert change to getSortDescriptionFromGroupBy() after
    GroupingSetsRewriterVisitor had been introduced
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 17:44:57 +03:00
Azat Khuzhin
1f29b0a901 Rewrite queries GROUPING SETS (foo, bar) to GROUP BY foo, bar 
This is better then introducing separate
SelectQueryExpressionAnalyzer::useGroupingSetKey(), since for
optimize_aggregation_in_order that method will not be enough, because
size of ManyExpressionActions will not match size of SortDescription, in
ReadInOrderOptimizer::ReadInOrderOptimizer()

And plus it is cleaner.

v2: fix clang-tidy
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-27 17:44:51 +03:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
Azat Khuzhin
2613149f6b Fix converting types for UNION queries (may produce LOGICAL_ERROR) 
CI founds [1]:

    2022.02.20 15:14:23.969247 [ 492 ] {} <Fatal> BaseDaemon: (version 22.3.1.1, build id: 6082C357CFA6FF99) (from thread 472) (query_id: a5187ff9-962a-4e7c-86f6-8d48850a47d6) (query: SELECT 0., round(avgWeighted(x, y)) FROM (SELECT toDate(toDate('214748364.8', '-922337203.6854775808', '-0.1', NULL) - NULL, 10.000100135803223, '-2147483647'), 255 AS x, -2147483647 AS y UNION ALL SELECT y, NULL AS x, 2147483646 AS y)) Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/26d0e5438c86e52a145aaaf4cb523c399989a878/fuzzer_astfuzzerdebug,actions//report.html

The problem is that subqueries returns different headers:
- first query  -- x, y
- second query -- y, x

v2: Make order of columns strict only for UNION
    https://s3.amazonaws.com/clickhouse-test-reports/34775/9cc8c01a463d18c471853568b2f0af659a4e643f/stateless_tests__address__actions__[2/2].html
    Fixes: 00597_push_down_predicate_long
v3: add no-backward-compatibility-check for the test
Fixes: #37569
Resubmit: #34775
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit a813f5996e)
 2022-05-27 14:11:57 +03:00
zhanglistar
ca67e67a74 Fix a typo 2022-05-27 15:52:04 +08:00
Vxider
54d6f98122 flush and shutdown temporary table before drop 2022-05-27 04:50:36 +00:00
Dmitry Novik
3a9239b79f
Revert "RFC: Fix converting types for UNION queries (may produce LOGICAL_ERROR)" 2022-05-27 04:05:32 +02:00
Alexander Tokmakov
eb71dd4c78
Merge pull request #37547 from ClickHouse/followup_37398 
Follow-up to #37398
 2022-05-26 20:29:41 +03:00
Dmitry Novik
673a521d0b
Merge pull request #34775 from azat/union-type-cast 
RFC: Fix converting types for UNION queries (may produce LOGICAL_ERROR)
 2022-05-26 17:28:23 +02:00
Alexander Tokmakov
e8f33fb0d9 fix flaky tests 2022-05-26 14:17:05 +02:00
Nikolai Kochetov
bf95541531 Fixing style. 2022-05-26 11:09:36 +00:00
Nikolai Kochetov
84f97b53de Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-26 11:07:45 +00:00
Nikolai Kochetov
fea2401f1f
Merge pull request #37532 from ClickHouse/add-separate-mutex-for-factories-info 
Use a separate mutex for query_factories_info in Context.
 2022-05-26 13:03:28 +02:00
Dmitry Novik
16c6b60703 Introduce AggregationKeysInfo 2022-05-25 23:22:29 +00:00
Alexander Tokmakov
3d259fa0de
Update Context.cpp 2022-05-25 23:18:07 +03:00
kssenii
b89a4edc04 Merge master 2022-05-25 20:55:04 +02:00
alesapin
6f5c86e55e Merge branch 'master' into i_object_storage 2022-05-25 20:49:01 +02:00
Alexander Tokmakov
779e6ea0b9 make it better, fix on cluster queries 2022-05-25 20:17:49 +02:00
Azat Khuzhin
a813f5996e Fix converting types for UNION queries (may produce LOGICAL_ERROR) 
CI founds [1]:

    2022.02.20 15:14:23.969247 [ 492 ] {} <Fatal> BaseDaemon: (version 22.3.1.1, build id: 6082C357CFA6FF99) (from thread 472) (query_id: a5187ff9-962a-4e7c-86f6-8d48850a47d6) (query: SELECT 0., round(avgWeighted(x, y)) FROM (SELECT toDate(toDate('214748364.8', '-922337203.6854775808', '-0.1', NULL) - NULL, 10.000100135803223, '-2147483647'), 255 AS x, -2147483647 AS y UNION ALL SELECT y, NULL AS x, 2147483646 AS y)) Received signal Aborted (6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/26d0e5438c86e52a145aaaf4cb523c399989a878/fuzzer_astfuzzerdebug,actions//report.html

The problem is that subqueries returns different headers:
- first query  -- x, y
- second query -- y, x

v2: Make order of columns strict only for UNION
    https://s3.amazonaws.com/clickhouse-test-reports/34775/9cc8c01a463d18c471853568b2f0af659a4e643f/stateless_tests__address__actions__[2/2].html
    Fixes: 00597_push_down_predicate_long
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-25 20:31:47 +03:00
Nikolai Kochetov
1f9b1cf726 Fixing build. 2022-05-25 18:59:46 +02:00
Nikolai Kochetov
7b681fa8ac Fixing build. 2022-05-25 17:15:23 +02:00
Anton Popov
16e839ac71 add profile events for introspection of part types 2022-05-25 14:54:49 +00:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
Nikolai Kochetov
6370c29049 Use a separate mutex for query_factories_info in Context. 2022-05-25 14:16:59 +00:00
mergify[bot]
f49552d48e
Merge branch 'master' into grouping-sets-optimization-fix 2022-05-25 13:03:54 +00:00
Nikolai Kochetov
3d84aae0ab Better. 2022-05-24 20:06:08 +00:00
Igor Nikonov
04e2737a57
Merge pull request #37337 from ClickHouse/column_decl_null_before_default_value 
Column declaration: [NOT] NULL right after type 

+ fixed: data_type_default_nullable=true, it didn't make columns nullable
         if the column declaration contains default expression w/o type

Issue #37229
 2022-05-24 21:16:25 +02:00
Dmitry Novik
c4dc0f7cda Fix ORDER BY optimization in case of GROUPING SETS 2022-05-24 18:56:22 +00:00
Alexander Tokmakov
4618429201 fixes 2022-05-24 18:53:52 +02:00
Vitaly Baranov
497c70d786
Merge pull request #37269 from vitlibar/rework-access-control-notifications 
Rework AccessControl's notifications.
 2022-05-24 17:47:45 +02:00
Alexander Tokmakov
bb5f04efaf Merge branch 'master' into fix_trash 2022-05-24 17:45:51 +02:00
Vladimir C
0eaf22b370
Merge pull request #37453 from vdimir/join_auto_lc_nullable_bug 2022-05-24 16:28:23 +02:00
Vladimir C
bec4ae87c9
Merge pull request #37472 from amosbird/joinpushdown 2022-05-24 16:08:26 +02:00
Alexander Tokmakov
229d35408b
Merge pull request #37398 from ClickHouse/fixes_for_transactions 
Fixes for transactions
 2022-05-24 15:28:01 +03:00
Alexander Tokmakov
dbde63d275 fixes 2022-05-24 14:24:03 +02:00
Amos Bird
76ddb39d02
refactor format 2022-05-24 12:09:00 +08:00
taiyang-li
c7e68d664d fix building 2022-05-24 10:58:45 +08:00
Igor Nikonov
838a6c6f61 Column declaration: [NOT] NULL right after type 
+ fixed: data_type_default_nullable=true, it didn't make columns nullable
         if the column declaration contains default expression w/o type

Issue #37229
 2022-05-23 22:03:45 +00:00
mergify[bot]
51ff49a0ee
Merge branch 'master' into i_object_storage 2022-05-23 20:29:49 +00:00
Alexander Tokmakov
3f44f34fe1 review suggestions 2022-05-23 21:55:17 +02:00
Nikolai Kochetov
fd97a9d885 Move some resources 2022-05-23 19:47:32 +00:00
Alexander Tokmakov
d0f998fb88 Merge branch 'master' into fix_trash 2022-05-23 21:25:56 +02:00
Alexander Tokmakov
fa21121f77 fix 2022-05-23 21:17:52 +02:00
Alexander Tokmakov
a262492cc1 slightly better 2022-05-23 20:53:33 +02:00
Alexander Tokmakov
6bc68c0cbc Merge branch 'master' into fixes_for_transactions 2022-05-23 18:49:21 +02:00
Vladimir C
14a820c2a9
Merge pull request #37336 from vdimir/fix_clash_const_aggegate_join 2022-05-23 17:09:30 +02:00
Nikolai Kochetov
9756b759c6 Move some resources 2022-05-23 13:46:57 +00:00
vdimir
dabb150a95
Fix cast lowcard of nullable in JoinSwitcher 2022-05-23 13:05:59 +00:00
Antonio Andelic
87d445295f
Merge pull request #37342 from azat/projections-optimize_aggregation_in_order-fix 
Fix projections with GROUP/ORDER BY in query and optimize_aggregation_in_order
 2022-05-23 12:29:16 +02:00
taiyang-li
c120b19802 fix building 2022-05-23 17:42:20 +08:00
taiyang-li
f13dcad58c Merge branch 'master' into fix_settings 2022-05-23 17:36:45 +08:00
mergify[bot]
747aa5575c
Merge branch 'master' into remove-useless-code-2 2022-05-22 17:41:57 +00:00
alesapin
06c3dd69c0 Move directories 2022-05-22 13:51:48 +02:00
alesapin
c8d92b87c8 Merge branch 'master' into i_object_storage 2022-05-22 12:16:10 +02:00
Robert Schulze
19d53c14fa
Merge pull request #37382 from ClickHouse/wc++98-compat-extra-semi 
Enable -Wc++98-compat-extra-semi
 2022-05-22 09:40:45 +02:00
kssenii
9aa20193d5 Merge master 2022-05-22 00:56:20 +02:00
kssenii
ffd8d9723a Continuation 2022-05-22 00:21:40 +02:00
Alexey Milovidov
6322d064b8
Merge pull request #37384 from Avogar/short-circuit-process 
Try to improve short circuit functions processing
 2022-05-21 23:43:56 +03:00
Alexey Milovidov
d1678c9662
Update ExpressionActions.cpp 2022-05-21 23:43:26 +03:00
Vitaly Baranov
58f4a86ec7 Rework notifications used in access management. 2022-05-21 10:15:39 +02:00
Alexander Tokmakov
1b453f517d fix 2022-05-21 02:32:35 +02:00
Alexander Tokmakov
c48410b574 fix trash in my code 2022-05-21 02:05:02 +02:00
Alexander Tokmakov
44f2d4529a better waiting, add fault injection 2022-05-20 22:10:01 +02:00
Nikolai Kochetov
56feef01e7 Move some resources 2022-05-20 19:49:31 +00:00
Alexey Milovidov
c77b078c2c
Merge pull request #37367 from ClickHouse/grouping-sets-with-rollup-cube 
Disable WITH ROLLUP/CUBE for GROUPING SETS
 2022-05-20 22:34:56 +03:00
Alexander Tokmakov
1027798517 handle connection loss on commit 2022-05-20 17:35:29 +02:00
Robert Schulze
0f6715bd91
Follow-up to PR #37300: semicolon warnings 
In PR #37300, Alexej asked why we the compiler does not warn about
unnecessary semicolons, e.g.

  f()
  {
  }; // <-- here

The answer is surprising: In C++98, above syntax was disallowed but by
most compilers accepted it regardless. C++>11 introduced "empty
declarations" which made the syntax legal.

The previous behavior can be restored using flag
-Wc++98-compat-extra-semi. This finds many useless semicolons which were
removed in this change. Unfortunately, there are also false positives
which would require #pragma-s and HAS_* logic (--> check_flags.cmake) to
suppress. In the end, -Wc++98-compat-extra-semi comes with extra effort
for little benefit. Therefore, this change only fixes some semicolons
but does not enable the flag.
 2022-05-20 15:06:34 +02:00
avogar
22b072bc7f Make better 2022-05-20 13:04:43 +00:00
avogar
3884819bd1 Fix 2022-05-20 12:48:58 +00:00
vdimir
339980469f
Revert "Swap order of converting_join_columns and before_join steps" 
This reverts commit 2995b69f4a.
 2022-05-20 11:31:59 +00:00
vdimir
23a85d3406
Perserve constness in ExpressionActionsChain::JoinStep 2022-05-20 11:31:58 +00:00
avogar
5d19150906 Try to improve short circuit functions processing 2022-05-20 10:47:28 +00:00
Alexander Tokmakov
12bbb7de87 fix race on TID allocation 2022-05-20 12:41:44 +02:00
alesapin
654b27e307 Merge branch 'master' into i_object_storage 2022-05-20 11:56:13 +02:00
Robert Schulze
b475fbc9a7
Merge pull request #37300 from ClickHouse/cmake-cleanup-pt3 
Various cmake cleanups
 2022-05-20 10:02:36 +02:00
alesapin
92c15ec97c Get rid of IDiskRemote 2022-05-19 20:07:15 +02:00
Dmitry Novik
b3ccf96c81 Merge remote-tracking branch 'origin/master' into grouping-function 2022-05-19 17:58:33 +00:00
Dmitry Novik
d4c66f4a48 Code cleanup & fix GROUPING() with TOTALS 2022-05-19 16:36:51 +00:00
Dmitry Novik
86d48e1c99 Disable WITH ROLLUP/CUBE for GROUPING SETS 2022-05-19 14:10:04 +00:00
Alexander Tokmakov
a53cfa9fca
Merge pull request #37355 from ClickHouse/revert-37266-fix-mutations-with-object 
Revert "Fix mutations in tables with columns of type `Object`"
 2022-05-19 16:27:22 +03:00
Kseniia Sumarokova
24d0899cb7
Merge pull request #36945 from Vxider/wv-populate 
POPULATE support for WindowView
 2022-05-19 13:47:52 +02:00
Kseniia Sumarokova
791cc5ced1
Merge pull request #37290 from azat/query-kind-client 
Add ability to pass QueryKind via clickhouse-client/local (useful for debugging)
 2022-05-19 13:22:42 +02:00
Alexander Tokmakov
f787dc7097
Revert "Fix mutations in tables with columns of type Object" 2022-05-19 13:24:48 +03:00
Azat Khuzhin
e32b695775 Fix projections with GROUP/ORDER BY in query and optimize_aggregation_in_order 
With projections, GROUP BY/ORDER BY in query, optimize_aggregation_in_order,
GROUP BY's InputOrderInfo was used incorrectly for ORDER BY.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 13:11:22 +03:00
Vitaly Baranov
16c8e93dd0
Merge pull request #36941 from nvartolomei/nv/show-grants-current-roles 
Allow a user to inspect grants from granted roles
 2022-05-19 09:20:52 +02:00
Azat Khuzhin
54a6aaef62 Log query processing stage in executeQuery() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 07:19:26 +03:00
Azat Khuzhin
dea1706d4e
Fix GROUP BY AggregateFunction (#37093) 
* Fix GROUP BY AggregateFunction

finalizeChunk() was unconditionally converting AggregateFunction to the
underlying type, however this should be done only if the aggregate was
applied.

So pass names of aggregates as an argument to the finalizeChunk()

Fuzzer report [1]:

    Logical error: 'Bad cast from type DB::ColumnArray to DB::ColumnAggregateFunction'. Received signal 6 Received signal Aborted (6)

For the following query:

    SELECT
        arraySort(groupArrayArray(grp_simple)),
        grp_aggreg,
        arraySort(groupArrayArray(grp_simple)),
        b,
        arraySort(groupArrayArray(grp_simple)) AS grs
    FROM data_02294
    GROUP BY
        a,
        grp_aggreg,
        b
    SETTINGS optimize_aggregation_in_order = 1

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/37050/323ae98202d80fc4b311be1e7308ef2ac39e6063/fuzzer_astfuzzerdebug,actions//fuzzer.log

v2: fix conflicts in src/Interpreters/InterpreterSelectQuery.cpp
v3: Fix header for GROUP BY AggregateFunction WITH TOTALS
v4: Add sanity check into finalizeBlock()
v5: Use typeid_cast<&> to get more sensible error in case of bad cast (as suggested by @nickitat)
v6: Fix positions passed to finalizeChunk()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Core/ColumnNumbers.h: remove unused <string>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Optimize finalizeChunk()/finalizeBlock()

v2: s/ByPosition/Mask/ s/by_position/mask/
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-18 23:37:43 +02:00
Alexander Tokmakov
3bc39f9929 fixes 2022-05-18 21:57:20 +02:00
vdimir
2995b69f4a
Swap order of converting_join_columns and before_join steps 2022-05-18 19:31:14 +00:00
Vitaly Baranov
9ccddc44c6
Merge pull request #35767 from azat/ON_CLUSTER-grant 
RFC: Add separate CLUSTER grant
 2022-05-18 19:19:54 +02:00
Alexander Tokmakov
9772924d06 Merge branch 'master' into fix_trash 2022-05-18 17:27:54 +02:00
Dmitry Novik
6356112a76 Refactor GROUPING function 2022-05-18 15:23:31 +00:00
Alexander Tokmakov
19afeda4b1 fix style 2022-05-18 14:23:52 +02:00
Alexander Tokmakov
538b5cd536 fix more trash 2022-05-18 14:06:52 +02:00
Vladimir C
7d88b81622
Merge pull request #37193 from azat/fix-limit-by-in-dist-queries 
Fix "Cannot create column of type Set" for distributed queries with LIMIT BY
 2022-05-18 13:28:19 +02:00
Anton Popov
4def736ecb
Merge pull request #37275 from ernado/fix/open-telemetry-span-log-fix-alias 
Fix attribute.values alias for OTEL table
 2022-05-18 13:18:07 +02:00
Nicolae Vartolomei
32b9959dec Allow a user to inspect grants from granted roles 2022-05-18 10:44:18 +00:00
Vladimir C
1779fb08ed
Merge pull request #36271 from yuuch/support_collation_in_create_query 2022-05-18 12:43:49 +02:00
Vladimir C
f979a03fbe
Merge pull request #37256 from vdimir/issue_37217 2022-05-18 12:42:47 +02:00
alesapin
e99c90118e Merge branch 'master' into i_object_storage 2022-05-18 12:36:22 +02:00
Kruglov Pavel
aaace46da2
Merge pull request #37266 from CurtizJ/fix-mutations-with-object 
Fix mutations in tables with columns of type `Object`
 2022-05-18 12:19:55 +02:00
Dmitry Novik
a82239984c Remove include 2022-05-18 01:44:51 +00:00
Alexander Tokmakov
5590adeffa fix more trash 2022-05-17 23:04:24 +02:00
Dmitry Novik
a97ba9741e Fix possible deadlock in OvercommitTracker during logging 2022-05-17 18:07:52 +00:00
Alexander Tokmakov
dea39d8175 fix some trash 2022-05-17 18:22:52 +02:00
Vitaly Baranov
f34a5cdee2
Merge branch 'master' into ON_CLUSTER-grant 2022-05-17 13:21:20 +02:00
taiyang-li
8228bbc36f disable normarlize functions when query is secondary 2022-05-17 16:22:31 +08:00
taiyang-li
9f3dadf17c Merge branch 'master' into fix_settings 2022-05-17 16:19:29 +08:00
Vxider
f74c934a61 Merged with master 2022-05-17 02:53:59 +00:00
Aleksandr Razumov
60693afb56
Fix attribute.values alias for OTEL table 2022-05-17 01:28:36 +03:00
kssenii
6a51dea791 Merge master 2022-05-16 22:13:22 +02:00
kssenii
5e3410f60a Better 2022-05-16 22:09:11 +02:00
Robert Schulze
43945cea1b
Fixing some warnings 2022-05-16 20:59:27 +02:00
Anton Popov
b6c5ab4fcf fix mutations in tables with columns of type Object 2022-05-16 18:26:53 +00:00
Dmitry Novik
e5b395e054 Support ROLLUP and CUBE in GROUPING function 2022-05-16 17:33:38 +00:00
Alexander Tokmakov
712a04d69c Revert "Removed an unnecessary check and extra logging" 
This reverts commit 59ed1f9d8e.
 2022-05-16 18:56:00 +02:00
alesapin
ce6d5f7dcc Merge branch 'master' into i_object_storage 2022-05-16 17:42:22 +02:00
Dmitry Novik
dd1e7b55b8
Merge pull request #37050 from azat/fix-optimize_aggregation_in_order-prefix-Array 
Fix optimize_aggregation_in_order with prefix GROUP BY and *Array aggregate functions
 2022-05-16 17:17:56 +02:00
vdimir
76852ae49c
Pass need_filter, has_null_map to joinRightColumns 2022-05-16 14:27:27 +00:00
Robert Schulze
ed7df7cabd
Merge pull request #36960 from ClickHouse/clangtidies 
Enable clangtidies
 2022-05-16 14:22:50 +02:00
alesapin
1829256ae2
Merge pull request #37178 from ClickHouse/ignore_auto_increment_in_create_table_2 
Ignore AUTO_INCREMENT in CREATE TABLE to simplify migration from MySQL
 2022-05-16 13:29:22 +02:00
Azat Khuzhin
01e1c5345a Add separate CLUSTER grant 
In case you have different roles for the same user on multiple clusters,
ON CLUSTER query can help to overcome some limitations.

Consider the following example:
- cluster_with_data, dev_user (readonly=2)
- stage_cluster, dev_user (readonly=0)

So when you will execute the following query from stage_cluster, it will
be successfully executed, since ON CLUSTER queries has different system
profile:

    DROP DATABASE default ON CLUSTER cluster_with_data

This is not 100% safe, but at least something.

Note, that right now only ON CLUSTER query it self is supported, but
separate clusters are not (i.e. GRANT CLUSTER some_cluster_name TO
default), since right now grants sticked to database+.

v2: on_cluster_queries_require_cluster_grant
v3: fix test and process flags as bit mask
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-16 13:57:45 +03:00
alesapin
b10f26e3ed Fix style 2022-05-16 12:25:25 +02:00
Vladimir C
75008d5903
Merge pull request #37166 from vdimir/issue_37141 2022-05-16 12:21:41 +02:00
Robert Schulze
d66dcdad79
Fix new occurrences of new clang-tidy warnings 2022-05-16 11:31:36 +02:00
Robert Schulze
e3cfec5b09
Merge remote-tracking branch 'origin/master' into clangtidies 2022-05-16 10:12:50 +02:00
Vasily Nemkov
59ed1f9d8e Removed an unnecessary check and extra logging 2022-05-16 09:29:54 +03:00
Alexey Milovidov
8334329dff
Merge pull request #37023 from ClickHouse/chained-or 
Merging #34932.
 2022-05-15 04:32:31 +03:00
kssenii
646cf4cc95 Better 2022-05-14 14:26:04 +02:00
Vxider
f2ef27e5d6 rename windowview header 2022-05-14 09:21:54 +00:00
Dmitry Novik
6fc7dfea80 Support ordinary GROUP BY 2022-05-13 23:04:12 +00:00
alesapin
cc304a3178 Merge branch 'master' into i_object_storage 2022-05-13 23:44:00 +02:00
Nikolai Kochetov
0a715b26db Move some resources. 2022-05-13 20:02:28 +00:00
Dmitry Novik
efb30bdf64 Correctly use __grouping_set_map column 2022-05-13 18:20:12 +00:00
Azat Khuzhin
b29a0f3c84 Fix "Cannot create column of type Set" for distributed queries with LIMIT BY 
The problem was that after #36454 it requires too much columns.

Fixes: #36454
Fixes: #37045
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 20:03:19 +03:00
Azat Khuzhin
751a85521e Fill possible WITH FILL error in SelectQueryExpressionAnalyzer::appendOrderBy() 
If order_by_keys was filled by the code above then WITH FILL will
incorrectly handle constant columns.

Fixes: #16618
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 20:03:19 +03:00
kssenii
58faaf0ea3 Merge master 2022-05-13 18:51:07 +02:00
alesapin
eba60ff38f Fix restorable schema 2022-05-13 17:00:47 +02:00
Dmitry Novik
ae81268d4d Try to compute helper column lazy 2022-05-13 14:55:50 +00:00
Igor Nikonov
74be0946ff Test fixes 2022-05-13 13:33:45 +00:00
Vasily Nemkov
fdd42f19ff Less obscure message on destroying Session 2022-05-13 15:27:59 +03:00
Igor Nikonov
8d5cabc8f7 Fix style warnings and review comments 2022-05-13 11:27:17 +00:00
vdimir
976a274f91
Add comment for visiting children of select in ApplyWithGlobalVisitor 2022-05-13 10:53:26 +00:00
Vasily Nemkov
2e669f69f4 Added checks to validate that authentication happens before creating query or session context; 
Updated logging of user_id, removed unnecessary precautions
Explicitly stating in Session d-tor if user was not authenticated
 2022-05-13 13:46:24 +03:00
vdimir
dc2a90a344
ApplyWithGlobalVisitor for WITH in subquery 2022-05-13 10:40:51 +00:00
Kseniia Sumarokova
6d5137879b
Merge pull request #37054 from kssenii/better-read-from-cache 
Better read from cache buffer
 2022-05-13 12:20:06 +02:00
Alexander Tokmakov
432f1d570c
Merge pull request #37153 from azat/fix-optimize_skip_unused_shards_rewrite_in 
Fix implicit cast for optimize_skip_unused_shards_rewrite_in
 2022-05-13 12:58:28 +03:00
Igor Nikonov
f423e9bf40 Ignore AUTO_INCREMENT in CREATE TABLE to simplify migration from MySQL 
Issue #35893
 2022-05-13 09:22:52 +00:00
Kseniia Sumarokova
320efb2221
Merge branch 'master' into better-read-from-cache 2022-05-12 23:35:39 +02:00
Maksim Kita
2bf6d9c028
Merge pull request #37154 from azat/context-const 
Add const qualifier for few methods from Context
 2022-05-12 22:20:39 +02:00
Kseniia Sumarokova
6e0a910fcd
Merge branch 'master' into better-read-from-cache 2022-05-12 19:44:18 +02:00
Anton Popov
56fffc8681 fix filling of missing columns 2022-05-12 17:21:30 +00:00
Dmitry Novik
c5b40a9c91 WIP on GROUPING function 2022-05-12 16:40:26 +00:00
alesapin
e7296a2b28
Merge pull request #36825 from kssenii/cache-fix-1 
Removed forceful drop cache command, fix detached status state
 2022-05-12 17:18:21 +02:00
Vxider
dc33c3a9bb Merged with master 2022-05-12 23:04:07 +08:00
Azat Khuzhin
a0ba113e5e Mark Context::getInterserverIOHandler() as const 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-12 16:54:01 +03:00
Azat Khuzhin
a04b6ad314 Mark Context::getInterserverCredentials() as const 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-12 16:54:01 +03:00
Azat Khuzhin
586eda5242 Fix implicit cast for optimize_skip_unused_shards_rewrite_in 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #37073
 2022-05-12 16:17:30 +03:00
Vladimir C
4cb6ca2311
Merge pull request #36970 from ClickHouse/concurrent_join 
Follow up for ConcurrentHashJoin
 2022-05-12 15:09:29 +02:00
kssenii
483e056bcc Merge master 2022-05-12 14:58:24 +02:00
Dmitry Novik
df071bb701
Merge pull request #37129 from ClickHouse/enable-overcommit-for-user 
Update documentation and defaults for memory overcommit
 2022-05-12 14:38:43 +02:00
Anton Popov
0dbbca5b3e fix filling of missed Nested columns with multiple levels 2022-05-12 12:04:26 +00:00
mergify[bot]
7ffad69019
Merge branch 'master' into wv-populate 2022-05-12 11:52:04 +00:00
Kseniia Sumarokova
59bfcd917b
Merge pull request #36876 from kssenii/extend-cache-log 
Log into filesystem_cache_log when cache not even attempted
 2022-05-12 12:38:36 +02:00
Nikolai Kochetov
55188665f4
Merge pull request #37071 from ClickHouse/join_dup_columns_issue_contin 
Continue fixing #36199.
 2022-05-12 11:36:15 +02:00
mergify[bot]
f4bae3969d
Merge branch 'master' into enable-overcommit-for-user 2022-05-11 20:04:20 +00:00
Maksim Kita
b1c154ccb4 Fixed style check 2022-05-11 21:59:51 +02:00
Maksim Kita
75555c436b Fix usage of min_count_to_compile_sort_description setting 2022-05-11 21:59:51 +02:00
Maksim Kita
ea8ce3140a Fixed tests 2022-05-11 21:59:51 +02:00
Maksim Kita
4e7d10297b Fixed style 2022-05-11 21:59:51 +02:00
Maksim Kita
ce92f6aab1 Fixed tests 2022-05-11 21:59:51 +02:00
Maksim Kita
b3b46d2e5c Fixed tests 2022-05-11 21:59:51 +02:00
Maksim Kita
8ceb63ee6c Added JIT compilation of SortDescription 2022-05-11 21:59:51 +02:00
Nikolai Kochetov
ec34761d9f
Merge pull request #33631 from ClickHouse/grouping-sets-fix 
Support GROUPING SETS
 2022-05-11 21:28:46 +02:00
Dmitry Novik
17608b3d93 Update documentation and defaults for memory overcommit 2022-05-11 16:18:41 +00:00
Vladimir C
641e2be6ef
Merge pull request #37085 from vdimir/force-cross-to-inner-option 
Option to force cross_to_inner_join_rewrite
 2022-05-11 18:03:53 +02:00
Kruglov Pavel
af3db5a992
Merge pull request #36726 from PolyProgrammist/s3clustermacro 
Handle cluster macro in s3Cluster table function
 2022-05-11 18:00:57 +02:00
Alexander Tokmakov
9e37df7989 better table with DDDL status 2022-05-11 16:09:14 +02:00
Nikolai Kochetov
2d99f0ce13 Simplify code a little bit. 2022-05-11 12:16:15 +00:00
vdimir
2f5959aca6
Upd cross_to_inner_join_rewrite setting, add test 2022-05-11 10:56:43 +00:00
Vxider
e4f43d0b01 update getSampleBlock for windowview 2022-05-11 08:47:08 +00:00
kssenii
18dd74be0c Remove unneede changes 2022-05-11 10:26:30 +02:00
Alexey Milovidov
c0acd5a325 Merge branch 'chained-or' of github.com:ClickHouse/ClickHouse into chained-or 2022-05-11 01:29:22 +02:00
Alexey Milovidov
3fcf63830c Merge branch 'master' into chained-or 2022-05-11 01:28:12 +02:00
Nikolai Kochetov
f7dbd48ee5 Simplify code a little bit. 2022-05-10 16:12:03 +00:00
Nikolai Kochetov
a02e1d2f4a Simplify code a little bit. 2022-05-10 16:00:00 +00:00
vdimir
00cd21cacf
Option to force cross_to_inner_join_rewrite 2022-05-10 15:12:17 +00:00
vdimir
9b24e0d260
Apply optimizeCountConstantAndSumOne before JOINs 2022-05-10 14:30:33 +00:00
vdimir
ff001382fb
Handle non unique column names in NotJoinedBlocks 2022-05-10 14:30:33 +00:00
vdimir
837d4bd13f
Revert "Continue fixing #36199." 
This reverts commit 1feba64a4e.
 2022-05-10 14:30:02 +00:00
Nikolai Kochetov
1feba64a4e
Continue fixing #36199. 2022-05-10 14:26:49 +00:00
kssenii
f4b45a5805 Better profile events 2022-05-10 16:00:53 +02:00
Vladimir C
4065b83ba9
Update ConcurrentHashJoin.cpp 2022-05-10 14:14:58 +02:00
Vladimir C
43935f71f7
Update ConcurrentHashJoin.cpp 2022-05-10 14:09:49 +02:00
Anton Popov
f075070fdf
Merge pull request #37046 from azat/fix-optimize_aggregation_in_order-Array 
Fix optimize_aggregation_in_order with *Array aggregate functions
 2022-05-10 14:01:01 +02:00
Anton Popov
aec30c4076
Merge pull request #37053 from CurtizJ/remove-streams-comments 
Remove last mentions of data streams
 2022-05-10 13:38:13 +02:00
Vasily Nemkov
0cbc9c18e2
No debug messge if user not set 2022-05-10 13:39:23 +03:00
Vxider
5d0a5d34c8 removing function populate in windowview 2022-05-10 15:06:58 +08:00
taiyang-li
fd878f7e7b change as requested 2022-05-10 12:06:50 +08:00
Alexey Milovidov
5d8fad8ab6
Update ConvertFunctionOrLikeVisitor.cpp 2022-05-10 02:23:24 +03:00
kssenii
2f0f76369a Better 2022-05-09 21:25:26 +02:00
Anton Popov
e911900054 remove last mentions of data streams 2022-05-09 19:15:24 +00:00
Azat Khuzhin
323ae98202 Fix optimize_aggregation_in_order with prefix GROUP BY and *Array aggregate functions 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #35111
See-also: #37046
 2022-05-09 21:32:40 +03:00
Azat Khuzhin
6ada8a6337 Fix optimize_aggregation_in_order with *Array aggregate functions 
row_begin was wrong, and before this patch aggregator processing
{row_end, row_end} range, in other words, zero range.

Fixes: #9113 (cc @dimarub2000)
v2: add static_cast to fix UBSan
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-09 20:43:19 +03:00
mergify[bot]
55a6d22ad3
Merge branch 'master' into grouping-sets-fix 2022-05-09 14:02:10 +00:00
Nikolai Kochetov
a34411f51e Disable trivial_insert_select if query has aggregate functions. 
(cherry picked from commit 70ae043455)
 2022-05-09 15:56:39 +02:00
Vasily Nemkov
de9a06df5b
Merge branch 'master' into fix_session_log_crash 2022-05-09 17:00:24 +04:00
Vasily Nemkov
58953b5f28 Fixed issues with concurrent access to User object 
ContextAccess might be updated from another thread, nullifying UserPtr.
Can happen if user was dropped in concurrent query.

This could crash server under certain conditions.
 2022-05-09 15:56:50 +03:00
Alexander Tokmakov
b6be194d3a
Merge pull request #36976 from ClickHouse/replicated_database_fixes 
Fixes for Replicated database
 2022-05-09 14:41:49 +03:00
Nikolai Kochetov
ebfeca2c86 Try to move some code from SourceWithProgress to PipelineExecutor. 2022-05-09 10:28:05 +00:00
Alexey Milovidov
fec7447ab4
Update ConvertFunctionOrLikeVisitor.cpp 2022-05-09 06:16:10 +03:00
Alexey Milovidov
56abce6a75 Merge branch 'chained_or' of github.com:danlark1/ClickHouse into chained-or 2022-05-08 23:37:09 +02:00
Robert Schulze
61cbcbf073
Enable clang-tidy readability-misleading-indentation 
Official docs:

  Correct indentation helps to understand code. Mismatch of the
  syntactical structure and the indentation of the code may hide serious
  problems.
 2022-05-08 19:12:01 +02:00
Vxider
34f1821eb9 function now support for windowview populate 2022-05-08 23:43:08 +08:00
Vxider
e7c4eb2697 Merged with master 2022-05-08 18:15:31 +08:00
Alexey Milovidov
2b88c0898a
Update ConvertFunctionOrLikeVisitor.cpp 2022-05-07 14:11:06 +03:00
Kseniia Sumarokova
0f5f75a784
Merge pull request #36887 from kssenii/fix-current-size-count-in-cache 
Fix current size count in cache
 2022-05-07 11:45:20 +02:00
Dmitry Novik
8d59211f01 Remove redundant ActionType 2022-05-06 18:50:21 +00:00
Alexander Tokmakov
c806869040 fix sync database replica 2022-05-06 18:37:20 +02:00
vdimir
d712985575
Minor style changes for ConcurrentHashJoin 2022-05-06 15:17:46 +00:00
Vladimir C
bd5fab97d9
Merge pull request #36415 from bigo-sg/concurrent_join 2022-05-06 17:11:10 +02:00
Anton Popov
13e8db6299
Merge pull request #36762 from CurtizJ/dynamic-columns-12 
Fix insertion to columns of type `Object` from multiple files
 2022-05-06 14:14:32 +02:00
kssenii
32d004017b Merge origin 2022-05-06 13:54:51 +02:00
kssenii
fd1ab053f8 Merge master 2022-05-06 12:50:33 +02:00
Dmitry Novik
b4e1dbc3c8 Remove whitespaces 2022-05-05 18:35:41 +00:00
Dmitry Novik
9a251e0028 Cleanup code 2022-05-05 18:13:00 +00:00
Dmitry Novik
4cc26aa38b Merge remote-tracking branch 'origin/master' into grouping-sets-fix 
And fix execution of the query with only one grouping set
 2022-05-05 17:14:52 +00:00
Vxider
9eea003d69 Merged with master 2022-05-06 00:12:17 +08:00
mergify[bot]
2a8aad42dc
Merge branch 'master' into add-asynchronous-metrics-for-filesystem-cache 2022-05-05 15:43:38 +00:00
Vxider
a7dc1f48e5 populate support for windowview 2022-05-05 23:31:15 +08:00
Dmitry Novik
161f52292b Support distributed queries 2022-05-05 13:56:16 +00:00
Vitaly Baranov
dd51265218
Merge pull request #36864 from vitlibar/backup-improvements-4 
Backup improvements
 2022-05-05 15:37:51 +02:00
Anton Popov
fa1feb238e
Merge pull request #36853 from qoega/add-other-time-microsecons 
Add Other Query Time Microseconds Profile Event
 2022-05-05 13:58:13 +02:00
lgbo-ustc
595ba6bd9f udpate dispatchBlock() 2022-05-05 12:04:11 +08:00
lgbo-ustc
f9f241e566 fixed fast test. remove unused header 2022-05-05 11:16:33 +08:00
lgbo-ustc
29a8db1345 fixed fast test 2022-05-05 10:32:19 +08:00
lgbo-ustc
2ddccdc21c update codes 2022-05-05 10:02:50 +08:00
alesapin
aba78e5ed0
Merge pull request #36859 from ClickHouse/revert-36858-revert-35637-memory-overcommit-free 
Revert "Revert "Memory overcommit: continue query execution if memory is available""
 2022-05-04 23:43:00 +02:00
Kruglov Pavel
77e55c344c
Merge pull request #36667 from Avogar/mysqldump-format 
Add MySQLDump input format
 2022-05-04 19:49:48 +02:00
Robert Schulze
c48d9a4174
Merge pull request #36844 from rschu1ze/shared_ptr_helper3 
Remove inherited create() method + disallow copying of storages
 2022-05-04 19:16:30 +02:00
Robert Schulze
78cdb3ec64
Merge pull request #36862 from rschu1ze/new-clangtidies 
Activated a bunch of LLVM 12/13/14 clang-tidy warnings
 2022-05-04 15:26:57 +02:00
Kseniia Sumarokova
f11cfdca94
Merge pull request #36874 from kssenii/add-size-to-system-cache-log 
Add column to system.filesystem_cache_log
 2022-05-04 11:53:29 +02:00
Robert Schulze
45c0ec44b0
Fix build, pt. II 2022-05-04 10:15:25 +02:00
mergify[bot]
64084b5e32
Merge branch 'master' into shared_ptr_helper3 2022-05-03 20:46:16 +00:00
mergify[bot]
17aecac7ff
Merge branch 'master' into new-clangtidies 2022-05-03 19:44:01 +00:00
kssenii
8f99046100 Add come asynchronous metrics 2022-05-03 20:02:19 +02:00
kssenii
0266cdf125 Add entries to cache log when cache was not event attempted 2022-05-03 18:29:41 +02:00
Nikolai Kochetov
35095191eb Merge branch 'master' into refactor-something-in-part-volumes 2022-05-03 17:51:47 +02:00
mergify[bot]
ddd1c3e1b3
Merge branch 'master' into add-other-time-microsecons 2022-05-03 15:45:40 +00:00
Dmitry Novik
5ba7a55c18
Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing 
Parallel parsing of hive text format
 2022-05-03 15:56:28 +02:00
kssenii
000f2c9f7a Better 2022-05-03 15:55:36 +02:00
Maksim Kita
36d1c8238a
Merge pull request #36766 from kitaisreal/evaluate-constant-expression-fix 
Fix evaluateConstantExpression for subqueries
 2022-05-03 12:32:14 +02:00
Vitaly Baranov
5257ce31f8 Improved using ThreadPool for making backup or restoring, changed columns in system.backups. 2022-05-03 11:03:13 +02:00
Vitaly Baranov
409edfd3fa Rework RestoreCoordination: make restore deterministic. 2022-05-03 11:01:44 +02:00
Vitaly Baranov
2c754f44fc Make calculation of shard_num & replica_num not dependant on match of the cluster's definitions on nodes. 2022-05-03 11:01:44 +02:00
Vitaly Baranov
2a645bb187 Fix sending 'create_table' and 'create_database' restore settings to cluster. 2022-05-03 11:01:44 +02:00
Vitaly Baranov
bddec55d35 Added ASTBackupQuery::setDatabase(). 2022-05-03 11:01:44 +02:00
Robert Schulze
0a4eccb73e
Activated a bunch of LLVM 12/13/14 clang-tidy warnings 
Omitted new checks which produce too many matches or which are
controversial (e.g. readability-identifier-length).

New checks:

- misc-misleading-bidirectional + misc-misleading-identifier

  Detects potential attack as described in the Trojan Source attack

- modernize-macro-to-enum

  Replaces groups of adjacent macros with an unscoped anonymous enum

- modernize-shrink-to-fit

  Replace copy and swap tricks on shrinkable containers with the
  shrink_to_fit() method call

- modernize-use-transparent-functors

  Prefer transparent functors to non-transparent ones

- modernize-use-uncaught-exceptions

  This check will warn on calls to std::uncaught_exception and replace
  them with calls to std::uncaught_exceptions (uncaught_exception was
  deprecated with C++17)

- performance-no-int-to-ptr

  Diagnoses every integer to pointer cast

- readability-duplicate-include

  Looks for duplicate includes and removes them

- readability-redundant-preprocessor

  Finds potentially redundant preprocessor directives

- bugprone-lambda-function-name

  Checks for attempts to get the name of a function from within a lambda
  expression

- bugprone-redundant-branch-condition

  Finds condition variables in nested if statements that were also
  checked in the outer if statement and were not changed

- bugprone-shared-ptr-array-mismatch

  Finds initializations of C++ shared pointers to non-array type that
  are initialized with an array

- bugprone-stringview-nullptr

  Checks for various ways that the const CharT* constructor of
  std::basic_string_view can be passed a null argument and replaces them
  with the default constructor in most cases

- bugprone-suspicious-memory-comparison

  Finds potentially incorrect calls to memcmp() based on properties of
  the arguments
 2022-05-03 09:22:11 +02:00
Dmitry Novik
2ed5a4013a
Revert "Revert "Memory overcommit: continue query execution if memory is available"" 2022-05-03 00:45:13 +02:00
alesapin
f0b7af0aa2
Revert "Memory overcommit: continue query execution if memory is available" 2022-05-03 00:36:50 +02:00
Dmitry Novik
71b6f89166
Merge pull request #35637 from ClickHouse/memory-overcommit-free 
Memory overcommit: continue query execution if memory is available
 2022-05-02 19:00:18 +02:00
mergify[bot]
93f5aa7488
Merge branch 'master' into aggregator-jit-lock-fix 2022-05-02 16:16:36 +00:00
Dmitry Novik
9be17ef50c
Merge pull request #35111 from azat/optimize_aggregation_in_order-prefix 
Implement partial GROUP BY key for optimize_aggregation_in_order
 2022-05-02 17:49:48 +02:00
Ilya Yatsishin
6c3bbb4933 Add Other Time Microseconds Profile Event 2022-05-02 17:13:57 +02:00
Maksim Kita
c059629f8d Aggregator JIT compilation lock fix 2022-05-02 16:21:10 +02:00
Maksim Kita
2377106370 Fix build 2022-05-02 14:27:00 +02:00
Kruglov Pavel
b98ac57090
Merge pull request #36803 from CurtizJ/fix-unflatten-nested 
Fix creation of tables with `flatten_nested = 0`
 2022-05-02 13:58:16 +02:00
Kruglov Pavel
d613f7eab0
Merge branch 'master' into mysqldump-format 2022-05-02 13:31:57 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying 
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
 2022-05-02 08:46:52 +02:00
Kseniia Sumarokova
e3d2d72b39
Merge pull request #36802 from KinderRiven/local_cache_trace 
system table used for statistical query of remote file access
 2022-05-02 00:16:04 +02:00
Azat Khuzhin
3982638e9a Validate that function had been passed in CREATE FUNCTION 
Otherwise right now it leads to LOGICAL_ERROR, due to bad cast.
 2022-05-01 19:01:03 +03:00
Nikita Mikhaylov
93a65463d0
Added SYSTEM SYNC DATABASE query (#35944) 2022-05-01 15:40:18 +02:00
KinderRiven
9413aa423b fix 2022-05-01 19:45:16 +08:00
KinderRiven
7d0789472c fix 2022-05-01 19:45:16 +08:00
KinderRiven
7db67160fe fix 2022-05-01 19:45:16 +08:00
KinderRiven
e1acacf831 fix 2022-05-01 19:45:16 +08:00
KinderRiven
01210c8d38 fix header 2022-05-01 19:45:16 +08:00
KinderRiven
d80aa0fd17 impl system.filesystem_cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
d595df1604 fix 2022-05-01 19:45:16 +08:00
KinderRiven
6a2e4559c8 impl system.cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
ce9a6965d0 impl system.log_table 2022-05-01 19:45:16 +08:00
KinderRiven
a4b8f9e9cf impl system.cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
9c9870f021 build cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
463b42b4c3 remove unused var 2022-05-01 19:45:16 +08:00
KinderRiven
5a361a0db5 impl cache_log in filecache 2022-05-01 19:45:16 +08:00
KinderRiven
4db54ff6bc add log record 2022-05-01 19:45:16 +08:00
KinderRiven
29f0471caf add cache log 2022-05-01 19:45:16 +08:00
KinderRiven
0059a660c7 impl create cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
02080aad1c impl create cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
44c9996a1e impl create cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
d5f287369e add include 2022-05-01 19:45:16 +08:00
KinderRiven
ee6ef29538 add include 2022-05-01 19:45:16 +08:00
KinderRiven
ef863a7942 add include 2022-05-01 19:45:16 +08:00
KinderRiven
021a3a788b add cache log table in system database 2022-05-01 19:45:16 +08:00
KinderRiven
6f930be28b impl cache trace 2022-05-01 19:45:16 +08:00
Alexey Milovidov
b034146ba4
Merge pull request #36799 from azat/cleanup 
Tiny cleanup
 2022-05-01 14:23:13 +03:00
Alexey Milovidov
321514a7f3
Merge pull request #36816 from azat/is_all_data_sent-on-exception 
Set is_all_data_sent on exceptions too
 2022-05-01 14:22:06 +03:00
Nikita Taranov
0fd9740c72
Log hash table's cache messages with TRACE level (#36830) 2022-05-01 12:54:54 +02:00
Alexey Milovidov
189938e935
Merge pull request #36431 from rschu1ze/bugprone-branch-clone 
Fixed warnings of clang-tidy check "bugprone-branch-clone"
 2022-05-01 13:49:37 +03:00
Alexey Milovidov
91d1ac913c
Merge pull request #36820 from azat/fix-asynchronous_metric_log 
Avoid recreation of system.asynchronous_metric_log (due to difference in codec)
 2022-05-01 13:42:17 +03:00
Robert Schulze
89aa9ae00f
Fixed clang-tidy check "bugprone-branch-clone" 
The check is currently *not* part of .clang-tidy. It complains about:
(1) "switch has multiple consecutive identical branches"
(2) "repeated branch in conditional chain"

About (1): Lots of findings in switches were about redundant
"[[fallthrough]]" in places where the compiler would not warn anyways. I
have cleaned these up.

About (2): In if-else_if-else chains, fixing the warning would usually
mean concatenating multiple if-conditions. As this would reduce
readability in most cases, I did not fix these places.

Because of (2), I also refrained from adding "bugprone-branch-clone" to
.clang-tidy.
 2022-04-30 19:40:28 +02:00
mergify[bot]
cc08ccb420
Merge branch 'master' into remove-useless-code-2 2022-04-30 12:48:15 +00:00
Nikita Mikhaylov
117ffa90cf
Fix ungrammatical error message (#36760) 2022-04-30 13:57:13 +02:00
Azat Khuzhin
d82f2ff162 Set is_all_data_sent on exceptions too 
This will avoid clickhouse-test complains about left queries after the
test, like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36258/9646487c093a75dc31e3e818417cfad83580b40f/stateful_tests__memory__actions_.html

Follow-up for: #36649
 2022-04-30 13:00:44 +03:00
Azat Khuzhin
a75559aa36 Avoid recreation of system.asynchronous_metric_log (due to difference in codec) 
From the log:

    2022.04.29 12:33:40.397528 [ 486 ] {} <Debug> SystemLog (system.asynchronous_metric_log): Existing table system.asynchronous_metric_log for system log has obsolete or different structure. Renaming it to asynchronous_metric_log_352.
    Old: CREATE TABLE system.asynchronous_metric_log (`event_date` Date CODEC(Delta(2), ZSTD(1)), `event_time` DateTime CODEC(Delta(4), ZSTD(1)), `metric` LowCardinality(String) CODEC(ZSTD(1)), `value` Float64 CODEC(Gorilla, ZSTD(3))) ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (metric, event_date, event_time) SETTINGS index_granularity = 8192
    New: CREATE TABLE system.asynchronous_metric_log (`event_date` Date CODEC(Delta(2), ZSTD), `event_time` DateTime CODEC(Delta(4), ZSTD), `metric` LowCardinality(String) CODEC(ZSTD), `value` Float64 CODEC(Gorilla, ZSTD(3))) ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (metric, event_date, event_time) SETTINGS index_granularity = 8192

As you can see the difference in default ZSTD level.

P.S. I decided not to fix the printer, since this may introduce changes
in metadata, that may lead to changes in metadata between ZooKeeper and
clickhouse server.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-30 12:19:29 +03:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase 
Reorganize source files so that base won't depend on Common
 2022-04-30 09:40:58 +03:00
mergify[bot]
e9fde5d067
Merge branch 'master' into memory-overcommit-free 2022-04-29 19:31:59 +00:00
Nikolai Kochetov
5807ca47b0
Merge pull request #36425 from ClickHouse/pool-size-hot-reload 
Allow to increase number of threads and tasks for background executors
 2022-04-29 17:24:12 +02:00
Anton Popov
cef462f83d fix creation of tables with flatten_nested = 0 2022-04-29 14:53:08 +00:00
Azat Khuzhin
16b1d2bee4 Remove outdated comment from ThreadStatusExt 
Refs: https://github.com/ClickHouse/ClickHouse/pull/16121#discussion_r511500850

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 17:04:56 +03:00
Azat Khuzhin
b7b7d91bd1 Remove memory reservation for SquashingTransform 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 17:04:56 +03:00
Vladimir C
e1e7a67670
Merge pull request #36201 from vdimir/join_dup_columns_issue36199 2022-04-29 12:57:10 +02:00
lgbo-ustc
5021cb707c update 2022-04-29 17:41:26 +08:00
vdimir
81c24fc521
Fix assertion in join 2022-04-29 09:15:00 +00:00
Maksim Kita
5b9bb36f80
Merge pull request #36753 from kitaisreal/added-user-defined-path-config-setting 
Added user_defined_path config setting
 2022-04-29 11:10:51 +02:00
Alexey Milovidov
b81da324de
Merge pull request #36765 from ClickHouse/fix_flaky_test_non_leader_replica 
Fix flaky test `test_ddl_worker_non_leader`
 2022-04-29 11:18:49 +03:00
Amos Bird
9d30be2c59
Fix build again 2022-04-29 14:45:27 +08:00
Azat Khuzhin
0ce44f3021 Optimize optimize_aggregation_in_order with a prefix key 
Before it does lots of extra work, now, it will be significantly more
optimal (thousands of rows -> 1-2 million of rows).

v2: s/executeOnBlockSimple/executeOnBlockSmall/
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 06:58:28 +03:00
Azat Khuzhin
190ce217bb Disable GROUP BY statistics for optimize_aggregation_in_order 
This statistics significantly decrease performance of
optimize_aggregation_in_order with a prefix key.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 06:58:27 +03:00
Azat Khuzhin
767acd53fb Add ability to pass range of rows to Aggregator 
v2: fix compiled aggregate functions (seek result to row_start)
v3: fix compiled aggregate functions (seek args to row_start)
v4: change signatures for JIT
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 06:57:55 +03:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Maksim Kita
04429d85db Added user_defined_path config setting 2022-04-28 22:13:33 +02:00
Maksim Kita
55994afe0c Fix evaluateConstantExpression for subqueries 2022-04-28 22:09:29 +02:00
Alexander Tokmakov
6797e7dc76 fix flaky test 2022-04-28 21:39:45 +02:00
Anton Popov
1fc51e09ff fix insertion to column of type Object from multiple files via table function 2022-04-28 18:51:13 +00:00
Alexey Milovidov
d8fa806fca
Merge pull request #36360 from ClickHouse/optimize-asynchronous-metric-log 
Optimize `system.asynchronous_metric_log`
 2022-04-28 20:16:17 +03:00
Vadim Volodin
233f0b4b52 Handle cluster macro in s3Cluster table function 2022-04-28 18:38:17 +03:00
Kruglov Pavel
4d08587559
Merge branch 'master' into mysqldump-format 2022-04-28 15:58:18 +02:00
zhanglistar
7d798b3655
Count distinct optimization by using subquery of group by (#35993) 2022-04-28 14:55:37 +02:00
Maksim Kita
ec47abe9af
Merge pull request #36482 from Enmk/fix_session_log_error_on_local_and_tcp_interserver_session 
Fixed missing enum values for ClientInfo::Interface
 2022-04-28 13:06:19 +02:00
Vitaly Baranov
641a5f5e35
Merge pull request #36198 from vitlibar/backup-improvements-3 
Backup for replicated tables and other improvements
 2022-04-28 09:23:53 +02:00
Alexey Milovidov
60e512ac4b Fix for table structure comparison 2022-04-28 05:10:57 +02:00
mergify[bot]
b7e63154df
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-27 18:27:27 +00:00
Kseniia Sumarokova
0d09bb9897
Merge pull request #36699 from amosbird/bettermysqlhandler 
Slightly better mysql handler
 2022-04-27 17:07:35 +02:00
Amos Bird
1cba00478d
Slightly better mysql handler 2022-04-27 18:28:09 +08:00
lgbo-ustc
10b7e08319 update 2022-04-27 16:41:34 +08:00
lgbo-ustc
7505ce426a update 2022-04-27 13:11:24 +08:00
taiyang-li
0341880250 Merge remote-tracking branch 'origin/master' into hive_text_parallel_parsing 2022-04-27 11:15:16 +08:00
Alexey Milovidov
b8a9f1411a
Merge pull request #36649 from azat/system.processes-is_all_data_sent 
clickhouse-test: improve left queries after the test hardening
 2022-04-27 05:21:22 +03:00
lgbo-ustc
520b05b9f1 update test case tests/queries/0_stateless/02236_explain_pipeline_join.sql 2022-04-27 10:08:22 +08:00
Nikolai Kochetov
e44af67fee Merge branch 'master' into refactor-something-in-part-volumes 2022-04-26 21:08:00 +02:00
Vitaly Baranov
a8e924caf6 Make BACKUP & RESTORE synchronous by default. 2022-04-26 18:45:39 +02:00
kssenii
0c43b7b08a Merge master 2022-04-26 16:10:47 +02:00
avogar
33d845dade Add MySQLDump input format 2022-04-26 10:42:56 +00:00
lgbo-ustc
20fc676bff fixed bug: resize on left pipeline cause the order by result wrong 2022-04-26 18:27:42 +08:00
mergify[bot]
7571387a6e
Merge branch 'master' into memory-overcommit-free 2022-04-26 10:15:18 +00:00
lgbo-ustc
d96c29810a fixed bug: resize on left pipeline cause the order by result wrong 2022-04-26 18:12:14 +08:00
lgbo-ustc
0b0fa8453b fixed bug: resize on left pipeline cause the order by result wrong 2022-04-26 18:06:16 +08:00
Kseniia Sumarokova
81db081b88
Merge pull request #36639 from kssenii/better-drop-cache 
Improve drop cache query
 2022-04-26 11:25:46 +02:00
Azat Khuzhin
6339a48923 Add is_all_data_sent into system.processes 
v2: fix SHOW PROCESSLIST (does not have process list entry)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-26 12:15:00 +03:00
Vitaly Baranov
78bcb96098 Rename backup & restore setting 'async' -> 'sync', and make backup & restore async by default. 2022-04-26 09:51:19 +02:00
taiyang-li
99dee35b6e parallel parsing of hive text format 2022-04-26 14:33:10 +08:00
mergify[bot]
4e82b56a8d
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-26 05:40:49 +00:00
lgbo-ustc
c7841d226b udpate 2022-04-26 11:29:18 +08:00
lgbo-ustc
74ccc233d2 Merge remote-tracking branch 'ck/master' into concurrent_join 2022-04-26 09:21:02 +08:00
alesapin
92296484e7
Merge pull request #36348 from rschu1ze/erase_if3 
Replace remove-erase idiom by C++20 erase()/erase_if()
 2022-04-25 23:34:18 +02:00
Vitaly Baranov
000b184691 Fix style & compilation. 2022-04-25 23:05:35 +02:00
alesapin
04afe552f8 Trying to avoid segfaults 2022-04-25 22:51:09 +02:00
alesapin
1fffa56f63 Merge branch 'master' into memory-overcommit-free 2022-04-25 20:21:08 +02:00
mergify[bot]
e79de5a577
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-25 17:29:11 +00:00
kssenii
a76a35d3ce Better drop cache 2022-04-25 19:20:25 +02:00
Vitaly Baranov
ed0af9d4a0 Improve shutdown with async backup/restore. 2022-04-25 16:34:34 +02:00
Vitaly Baranov
042dc4abb2 Rename settings shard -> shard_num, replica -> replica_num. 2022-04-25 16:34:34 +02:00
Vitaly Baranov
129eca7307 CREATE queries generated by RESTORE command is now considered as internal. 2022-04-25 16:34:34 +02:00
Vitaly Baranov
68a020ecea Implement BACKUP/RESTORE ON CLUSTER. 2022-04-25 16:34:33 +02:00
Vitaly Baranov
030f3e488c Add shard_index and replica_index to params of executeDDLQueryOnCluster(). 2022-04-25 16:34:33 +02:00
Vitaly Baranov
3477665659 Add shard_index and replica_index to params of getRewrittenASTWithoutOnCluster(). 2022-04-25 16:34:33 +02:00
Vitaly Baranov
144d3aefeb Add system table system.backups; implement async mode for commands BACKUP & RESTORE 2022-04-25 16:34:33 +02:00
Kseniia Sumarokova
2106a7b895
Merge pull request #36489 from awakeljw/fork_chmaster 
fix ConvertDecimalType unexpected behavior
 2022-04-25 13:23:54 +02:00
lgbo-ustc
c68806fbff fixed typos 2022-04-25 16:16:50 +08:00
lgbo-ustc
981d560553 Merge remote-tracking branch 'ck/master' into concurrent_join 2022-04-25 13:00:04 +08:00
awakeljw
d58a21678a fix ConvertDecimalType unexpected behavior 2022-04-24 23:35:58 +08:00
lgbo-ustc
221bc9c970 update 2022-04-24 17:14:13 +08:00
lgbo-ustc
a38b580c6f update 2022-04-24 15:19:16 +08:00
Alexey Milovidov
90297e225c
Merge pull request #36430 from amosbird/fixcolumnmatcher 
Fix column matcher and column transformer
 2022-04-24 09:11:21 +03:00
Alexey Milovidov
fe004c3486
Merge pull request #36486 from kitaisreal/executable-user-defined-functions-fix-group-by 
ExecutableUserDefinedFunction fix usage in GROUP BY
 2022-04-24 06:11:57 +03:00
Alexey Milovidov
909646247d
Update TranslateQualifiedNamesVisitor.cpp 2022-04-24 05:04:13 +03:00
lgbo-ustc
056b4d1b61 update 2022-04-24 09:26:57 +08:00
Nikolai Kochetov
8c00692844 Part 8 2022-04-22 16:58:09 +00:00
Nikita Mikhaylov
9baf7b14dc Fix build 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
aaa7112405 Even better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
36bdee0499 Deleted settings 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
4818b63979 Better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
8c325c2cdd Better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
31ccb9c1c3 Done 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
224f4dc620
Made parallel_reading_from_replicas work with localhost replica (#36281) 2022-04-22 15:52:38 +02:00
yuuch
d126de6dda
Apply suggestions from code review 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-04-22 20:00:12 +08:00
lgbo-ustc
681d7d6799 update dispatchBlock() 2022-04-22 11:00:00 +08:00
Dmitry Novik
6ee62007d3 Handle GROUPING SETS with size < 2 2022-04-21 23:52:43 +00:00
Maksim Kita
583cbcb834
Merge branch 'master' into fix_session_log_error_on_local_and_tcp_interserver_session 2022-04-21 21:15:11 +02:00
Maksim Kita
9713441650
Merge pull request #36480 from Joeywzr/session_log 
add missing enum values in session_log
 2022-04-21 21:14:12 +02:00
Maksim Kita
485c1d4072 Fixed tests 2022-04-21 19:10:11 +02:00
Dmitry Novik
b5e2b38529 Optimize pipeline in case of single input 2022-04-21 14:11:58 +00:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes 
Clang tidy fixes
 2022-04-21 16:11:27 +02:00
Dmitry Novik
df63a9b205 Depricate TOTALS with GROUPING SETS 2022-04-21 13:41:59 +00:00
Dmitry Novik
77a82cc090
Merge pull request #35631 from amosbird/projection-fix1 
Fix broken SET reuse during projection analysis.
 2022-04-21 15:32:52 +02:00
mergify[bot]
be6914f6f9
Merge branch 'master' into session_log 2022-04-21 11:52:20 +00:00
Maksim Kita
397603e9be ExecutableUserDefinedFunction fix usage in GROUP BY 2022-04-21 13:44:56 +02:00
Dmitry Novik
ea6a25b8e7 Fix nullptr dereference 2022-04-21 11:14:22 +00:00
kssenii
24e376337d Merge with master 2022-04-21 12:33:56 +02:00
Vasily Nemkov
e2866b0541 Fixed missing enum values for ClientInfo::Interface 2022-04-21 12:16:12 +03:00
Memo
ce91a261e8 add missing enum values in session_log 2022-04-21 17:01:44 +08:00
lgbo-ustc
a9d9abb7a1 use column::scatter() 2022-04-21 16:59:30 +08:00
lgbo-ustc
e27d67ce8a update type name 2022-04-21 14:28:22 +08:00
lgbo-ustc
eac810ed85 use concatenateBlocks 2022-04-21 14:10:09 +08:00
lgbo-ustc
60964b207d update addJoinedBlock() 2022-04-21 13:19:33 +08:00
lgbo-ustc
bfd1a0e33a update codes 2022-04-21 12:14:36 +08:00
Dmitry Novik
61deae7105
Merge branch 'master' into grouping-sets-fix 2022-04-21 03:34:42 +02:00
Dmitry Novik
6e73cd6929 Implement parallel grouping sets processing 2022-04-21 01:18:40 +00:00
Alexey Milovidov
ec9b86fd03
Merge branch 'master' into fixcolumnmatcher 2022-04-21 02:15:25 +03:00
Azat Khuzhin
c6ffec1fec Fix "Cannot find column" error for distributed queries with LIMIT BY 
Consider the following example:

    SELECT k FROM (SELECT materialize('foo') AS k, -1 AS v) ORDER BY abs(v) AS _v ASC LIMIT 1 BY k

The problem here is that in case of query has LIMIT BY for
WithMergeableState* (which is the final state on the remote shard for
distributd queries) it returns the following columns:
- k
- v

While it should return:
- k
- abs(v)

So as query w/o LIMIT BY, so that initiator will be able to do the sorting

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-20 15:23:24 +03:00
lgbo-ustc
f461d18de5 update method name 2022-04-20 19:51:35 +08:00
lgbo-ustc
c844415578 add new join_algorithm parallel_hash 2022-04-20 19:47:16 +08:00
Robert Schulze
7550d9b281
Fix build 2022-04-20 13:34:17 +02:00
Alexander Tokmakov
1d30a97fd2 Merge branch 'master' into remove-useless-code-2 2022-04-20 11:45:56 +02:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings 
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
 2022-04-20 10:29:05 +02:00
Kseniia Sumarokova
766a84e64f
Merge pull request #35915 from kssenii/better-s3-settings 
Better settings configuration for s3
 2022-04-20 10:22:06 +02:00
Robert Schulze
b6d7367538
Merge remote-tracking branch 'origin/master' into erase_if3 
Conflicts:
- Interpreters/ActionsDAG.cpp
 2022-04-20 10:02:59 +02:00
Antonio Andelic
bbb0be6a44
Merge pull request #36402 from rschu1ze/clang-tidy-contains 
Activate clang-tidy warning "readability-container-contains"
 2022-04-20 08:15:46 +02:00
Kruglov Pavel
894b1b163e
Merge pull request #36055 from helifu/master2 
Don't materialize external tables if it's explain statement
 2022-04-19 20:26:27 +02:00
Kruglov Pavel
7fb7fc9daa
Merge pull request #36275 from Avogar/fix-insert-select 
Place new logic of schema inference in insert select from table function under setting
 2022-04-19 20:19:28 +02:00
Amos Bird
158a25d5fa
Fix column matcher and column transformer 2022-04-20 01:22:04 +08:00
kssenii
7cfc0ac433 Merge with master 2022-04-19 15:13:45 +02:00
Robert Schulze
72a331189e
Fix glitch 2022-04-19 14:59:47 +02:00
mergify[bot]
3c1dd44b80
Merge branch 'master' into fix-insert-select 2022-04-19 12:17:31 +00:00
lgbo-ustc
26ff749a32 remove unused header 2022-04-19 19:12:08 +08:00
lgbo-ustc
3f2fe3ff66 fixed code style 2022-04-19 18:47:29 +08:00
kssenii
1f9a597b77 Merge master 2022-04-19 11:31:13 +02:00
lgbo-ustc
9dfe2a6682 fixed header style 2022-04-19 16:53:24 +08:00
lgbo-ustc
3d7338581b Improve join 
now adding joined blocks from right table can be run parallelly, speedup the join process
 2022-04-19 16:07:30 +08:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains" 
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
 2022-04-18 23:53:11 +02:00
Alexey Milovidov
1e1ffbbfe2
Update SessionLog.cpp 2022-04-18 23:07:34 +03:00
mergify[bot]
daca157111
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-18 17:31:59 +00:00
Alexey Milovidov
0af183826d
Merge pull request #36358 from ClickHouse/don-t-randomize-priority 
Do not randomize "priority" setting
 2022-04-18 07:04:36 +03:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia 
Remove remaining parts of Arcadia
 2022-04-18 07:02:54 +03:00
Alexey Milovidov
36595e4206
Merge pull request #36320 from ClickHouse/fix-clang-tidy-14 
Fix clang-tidy-14 (part 1)
 2022-04-18 07:02:10 +03:00
Alexey Milovidov
70d7ba5d61 Fix clang-tidy 2022-04-18 02:13:28 +02:00
Alexey Milovidov
6de5d8a554 Merge branch 'master' into fix-clang-tidy-14 2022-04-18 02:12:45 +02:00
Alexey Milovidov
0051e651c6
Update AsynchronousMetricLog.h 2022-04-18 03:01:57 +03:00
Alexey Milovidov
242919eddd Remove abbreviation 2022-04-18 01:02:49 +02:00
Alexey Milovidov
8ea14d2664 Custom ORDER BY for system.asynchronous_metric_log 2022-04-17 23:59:05 +02:00
Alexey Milovidov
e36e22812a Custom column list for system.asynchronous_metric_log 2022-04-17 23:53:28 +02:00
Alexey Milovidov
300ef741cc Custom column list for system.asynchronous_metric_log 2022-04-17 23:51:56 +02:00
Alexey Milovidov
4c09e2d30a Custom column list for system.asynchronous_metric_log 2022-04-17 23:49:39 +02:00
Alexey Milovidov
37c61d9f0a Round values for the log 2022-04-17 22:07:50 +02:00
Alexey Milovidov
1f5df287c0 Remove useless code 2022-04-17 22:02:55 +02:00
Alexey Milovidov
6807f8428d Remove microseconds from system.asynchronous_metric_log 2022-04-17 22:01:58 +02:00
Alexey Milovidov
250ca83058 Allow custom default ORDER BY for system logs 2022-04-17 21:57:44 +02:00
Alexey Milovidov
a64765bc6f Allow custom default ORDER BY for system logs 2022-04-17 21:56:36 +02:00
Alexey Milovidov
aa93f4a074 Do not randomize "priority" setting 2022-04-17 20:48:02 +02:00
Alexey Milovidov
89deaf0af1
Merge pull request #36056 from azat/client-trailing-space 
Remove trailing whitespace in default client_name
 2022-04-17 17:37:43 +03:00
Robert Schulze
1e1df8e101
Replace remove-erase idiom by C++20 erase()/erase_if() 
- makes the code less verbose while being as efficient
 2022-04-17 12:04:47 +02:00
Alexey Milovidov
294efeccfe Fix clang-tidy-14 (part 1) 2022-04-16 04:54:04 +02:00
Alexey Milovidov
cbeeb7ec4f Remove Arcadia 2022-04-16 00:20:47 +02:00
Alexey Milovidov
a0fe29e243 Remove Arcadia 2022-04-15 23:59:49 +02:00
yuuch
6919ca9a6e remove parse about the not null in ParserCollation 2022-04-15 21:31:25 +08:00
mergify[bot]
08e885d517
Merge branch 'master' into exact-rows-before-limit 2022-04-15 06:27:54 +00:00
avogar
ae01646136 Place new logic of schema inference in insert select from table function under setting 2022-04-14 19:05:20 +00:00
Dmitry Novik
a16710c750 Merge remote-tracking branch 'origin/master' into grouping-sets-fix 2022-04-14 17:29:51 +00:00
yuuch
29e0b4c3bb support collation in create query 2022-04-15 00:12:31 +08:00
Alexander Tokmakov
4ce0b280e6 wait for node to disappear instead of removing 2022-04-14 15:31:05 +02:00
Anton Popov
305dd57262
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-14 14:51:15 +02:00
mergify[bot]
95901466e7
Merge branch 'master' into master2 2022-04-14 10:05:03 +00:00
Yakov Olkhovskiy
c1a06ac63a
Merge pull request #24896 from hexiaoting/dev_materialized_view 
Add type check when create materialized view with To clause
 2022-04-13 22:33:54 -04:00
Maksim Kita
ca89af89a1
Merge pull request #36088 from israelg99/fix_executable 
Fix CH silently failing when it cannot execute a file
 2022-04-13 16:53:00 +02:00
kssenii
0912b038fa Merge master 2022-04-13 15:30:01 +02:00
tavplubix
43de4827d7
Merge pull request #36074 from ClickHouse/zookeeper_log_thread_query_id 
Add thread_id and query_id to zookeeper_log
 2022-04-13 15:34:04 +03:00
mergify[bot]
ba22cb76bb
Merge branch 'master' into master2 2022-04-13 11:51:43 +00:00
Kruglov Pavel
03fc08cec3
Merge pull request #36137 from azat/client-profile-events 
ProfileEvents fixes
 2022-04-13 12:27:39 +02:00
Yakov Olkhovskiy
3d52afed71 only check columns sets intersection, some refactoring 2022-04-12 15:10:11 -04:00
tavplubix
303ecc6f7d
Merge pull request #36136 from ClickHouse/transactions_fix_1 
Minor fix for transactions
 2022-04-12 21:38:47 +03:00
Azat Khuzhin
e38057ddb0 Fix code alignment in ProfileEventsExt 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 20:54:56 +03:00
mergify[bot]
d5b2355944
Merge branch 'master' into memory-overcommit-free 2022-04-12 16:00:12 +00:00
Julian Gilyadov
a4f56f3330
Throw exception when file cant be executed instead of displaying success 2022-04-12 17:52:44 +02:00
kssenii
37242a0103 Allow non-evictable file segments based on predicate 2022-04-12 16:55:25 +02:00
Yakov Olkhovskiy
5868539605 check only if TO table exists 2022-04-12 10:17:39 -04:00
mergify[bot]
0b3c15c07a
Merge branch 'master' into projection-fix1 2022-04-12 13:49:28 +00:00
Yakov Olkhovskiy
25101de7e2 check only selected columns against destination table 2022-04-12 09:29:32 -04:00
kssenii
9804c39de7 Composable cache 2022-04-12 14:33:13 +02:00
tavplubix
2e95e0db3b
Merge pull request #36133 from Algunenano/osx_atomic_exchange 
Add support for atomic exchange in OSX
 2022-04-12 14:10:36 +03:00
Alexander Tokmakov
6e4787560b fix another issue 2022-04-12 11:44:25 +02:00
Yakov Olkhovskiy
dc314d53b3 do not create view before check, use MatchColumnsMode::Name mode 2022-04-11 20:28:11 -04:00
mergify[bot]
46b9a1ef80
Merge branch 'master' into client-trailing-space 2022-04-11 22:07:37 +00:00
Yakov Olkhovskiy
783dbb2e63
Merge branch 'master' into dev_materialized_view 2022-04-11 17:47:35 -04:00
Yakov Olkhovskiy
155a2a0d42
Merge pull request #35349 from yakov-olkhovskiy/interpolate-feature 
Interpolate feature
 2022-04-11 11:15:50 -04:00
Yakov Olkhovskiy
2588f8011f comment fix 2022-04-11 11:13:29 -04:00
Alexander Tokmakov
38c80940fd slightly beter fix 2022-04-11 16:25:59 +02:00
Yakov Olkhovskiy
7293e01647 some comments added 2022-04-11 09:47:27 -04:00
Raúl Marín
6e346a7fc3 Support atomic replace under OSX 2022-04-11 14:32:55 +02:00
Vladimir C
1a604cd279
Merge pull request #36058 from xiedeyantu/master 2022-04-11 10:45:50 +02:00
Vitaly Baranov
8dcb1358b7
Merge pull request #36072 from vitlibar/backups-improve-arguments-handling-and-file-removing 
Backups: improve arguments handling and file removing
 2022-04-10 16:51:04 +02:00
xiedeyantu
5efd8a4ec0 If data type has DEFAULT NULL in table definition, make data type Nullable 2022-04-09 15:00:41 +08:00
Vitaly Baranov
758f8235c2 Collect backup entries before opening an archive for writing. 2022-04-09 01:44:46 +02:00
Alexander Tokmakov
50a32a74b7 add thread_id and query_id to zookeeper_log 2022-04-08 17:10:08 +02:00
Yakov Olkhovskiy
87c2b3e2cf fixed Nullable, tests added 2022-04-08 10:52:10 -04:00
Alexander Tokmakov
141fbc6980 Merge branch 'master' into mvcc_prototype 2022-04-08 13:38:11 +02:00
Alexander Tokmakov
49c35f3261 Merge branch 'master' into mvcc_prototype 2022-04-08 13:34:40 +02:00
Vitaly Baranov
45dea9933b
Merge pull request #35883 from azat/format-settings 
Do not allow SETTINGS after FORMAT for INSERT queries
 2022-04-08 12:38:14 +02:00
kssenii
87855c4ae7 Merge master 2022-04-08 12:32:48 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements 
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
 2022-04-08 12:06:26 +02:00
Vladimir C
265fa90689
Merge pull request #36033 from vdimir/cross-join-empty-list 2022-04-08 11:59:21 +02:00
Azat Khuzhin
44bf8ba568 Remove trailing whitespace in default client_name 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-08 08:09:47 +03:00
helifu
1196dc3dbd Don't materialize external tables if it's explain statement 
The explain statement of GLOBAL JOIN two distributed tables can speed up 100x:
  explain plan select ... from t1_dist global join t2_dist on ...
  explain pipeline select ... from t1_dist global join t2_dist on ...
 2022-04-08 09:00:55 +08:00
Yakov Olkhovskiy
64dcddc6e3 fixed ASTInterpolateElement::clone, fixed QueryNormalizer to exclude ASTInterpolateElement::children 2022-04-07 17:41:05 -04:00
Alexander Tokmakov
6a46da93ae Merge branch 'master' into mvcc_prototype 2022-04-07 23:22:19 +02:00
kssenii
5dce2f18b5 Better 2022-04-07 18:48:35 +02:00
Alexander Tokmakov
2e2e2b9190 fix a race condition 2022-04-07 18:17:43 +02:00
Azat Khuzhin
33d99c8ffb Introduce compatiblity mode for SETTINGS after FORMAT 
Add allow_settings_after_format_in_insert setting, OFF by default.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: s/parser_settings_after_format_compact/allow_settings_after_format_in_insert/ (suggested by vitlibar)
v3: replace ParserSettings with a flag (requested by vitlibar)
 2022-04-07 16:36:34 +03:00
Alexander Tokmakov
7f54e7b422 Merge branch 'master' into mvcc_prototype 2022-04-07 15:14:06 +02:00
Kruglov Pavel
73adbb4c15
Merge pull request #35986 from amosbird/better-scalar1 
Fix performance regression of scalar query
 2022-04-07 14:07:59 +02:00
Alexander Tokmakov
8290ffa88d Merge branch 'master' into mvcc_prototype 2022-04-07 13:50:42 +02:00
vdimir
eb7ac4c3a6
Handle empty expression list in cross join 2022-04-07 11:32:32 +00:00
vdimir
69160fa3e3
remove dead code in needRewrite function 2022-04-07 10:46:39 +00:00
tavplubix
6e9c028bbb
Merge pull request #35956 from Algunenano/limit_dns_retries 
DNS cache: Add option to drop elements after several consecutive failures
 2022-04-07 13:20:36 +03:00
Nikolai Kochetov
3e1b3f14c0
Merge pull request #34355 from azat/processors-profiling 
Profiling on Processors level
 2022-04-07 12:13:14 +02:00
Amos Bird
1238babb6f
Make SelectQueryInfo pseudo-copyable 2022-04-07 17:46:50 +08:00
Amos Bird
69022850b9
Better 2022-04-07 17:46:50 +08:00
Amos Bird
9cf5935604
Fix broken SET reuse during projection analysis. 2022-04-07 17:46:49 +08:00
Yakov Olkhovskiy
7dbe8bc2dc major bugs fixed, tests added, docs updated 2022-04-07 01:21:24 -04:00
mergify[bot]
585cff1e95
Merge branch 'master' into limit_dns_retries 2022-04-06 18:23:17 +00:00
tavplubix
58db8cd9b3
Merge pull request #35946 from ClickHouse/faster_replicated_ddl 
Make some replicated DDL queries faster
 2022-04-06 19:25:25 +03:00
Amos Bird
df06f9f974
Fix performance regression of scalar query 2022-04-06 17:50:22 +08:00