thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-11 17:02:25 +00:00
Code Issues Packages Projects Releases Wiki Activity
136,468 Commits 9,299 Branches 1,782 Tags 2.1 GiB
cd4b60b51a
Commit Graph

12605 Commits

Author SHA1 Message Date
yariks5s
88f06886ac enhance tests, review fixes 2024-02-16 18:48:49 +00:00
Maksim Kita
96d9782439 Temporary table already exists exception message fix 2024-02-16 19:16:02 +03:00
Dmitry Novik
a3c34a7cc2
Merge pull request #54211 from ClickHouse/Support_parameterized_view_with_analyzer 
Refactor parameterized view & add support for new analyzer
 2024-02-16 16:13:49 +01:00
Alexander Tokmakov
228a29b93f fix 2024-02-16 14:05:22 +01:00
Alexey Milovidov
25f2501c7f
Merge pull request #60013 from ClickHouse/verify-parser-on-server-side 
Verify formatting consistency on the server-side
 2024-02-16 13:48:10 +01:00
Igor Nikonov
118263d8e8 Merge remote-tracking branch 'origin/master' into query-log-on-insert-exception-over-http 2024-02-16 12:40:10 +00:00
Alexey Milovidov
00c28d97ed Merge branch 'master' into verify-parser-on-server-side 2024-02-16 01:29:08 +01:00
Alexey Milovidov
15683ae6a0 Revert wrong modification 2024-02-15 22:58:07 +01:00
Alexander Tokmakov
22c97be2a4 ability to detect undead ZooKeeper sessions 2024-02-15 22:17:31 +01:00
Robert Schulze
058461e786
Merge remote-tracking branch 'rschu1ze/master' into order-by-all-ambiguities 2024-02-15 16:21:16 +00:00
Robert Schulze
1edf3b2254
Replace ORDER BY ALL by ORDER BY * to get rid of ambiguities 2024-02-15 16:19:56 +00:00
Dmitry Novik
a11b9553ca Fix implementation in new analyzer 2024-02-15 17:16:27 +01:00
Nikita Mikhaylov
3c3d7e52b0 Better 2024-02-15 13:52:30 +00:00
avogar
efa823400b Add IDataType::forEachChild and use it in nested types validation 2024-02-15 13:19:02 +00:00
Raúl Marín
5baae9326c Fix optimize_uniq_to_count removing the column alias 2024-02-15 14:10:55 +01:00
Igor Nikonov
1464830791 Merge remote-tracking branch 'origin/master' into query-log-on-insert-exception-over-http 2024-02-15 12:56:32 +00:00
Igor Nikonov
c580cdb96e Fix review comment 2024-02-15 12:55:43 +00:00
avogar
109720d162 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-bad-types-check 2024-02-15 12:10:49 +00:00
Alexey Milovidov
a86d31a005
Merge pull request #59998 from ClickHouse/revert-59699-disable_async_inserts_for_dependent_mv_dedup 
Revert "Insert synchronously if dependent MV deduplication is enabled"
 2024-02-15 07:52:13 +01:00
Alexey Milovidov
0258aba5c0 Verify formatting consistency on the server-side 2024-02-15 07:49:13 +01:00
Alexey Milovidov
47d8248b9d
Merge pull request #60005 from ClickHouse/non-significant-changes-3 
Fix obviously wrong (but non significant) error in dictionaries
 2024-02-15 05:59:07 +01:00
Alexey Milovidov
0b58f16938
Merge pull request #59964 from ClickHouse/cleanups 
Cleanups
 2024-02-15 02:59:54 +01:00
Alexey Milovidov
5e5528a890
Merge pull request #59972 from jrdi/fix-max-num-to-warn 
Fix max num to warn message
 2024-02-15 02:59:29 +01:00
Alexey Milovidov
46fb01c4f6 Fix trash 2024-02-15 02:31:46 +01:00
yariks5s
1cbfaf6f0d init 2024-02-14 23:06:04 +00:00
Vitaly Baranov
5591d74076 Add new config setting "backups.remove_backup_files_after_failure". 2024-02-14 23:19:21 +01:00
Julia Kartseva
b1b58ef9a3
Revert "Insert synchronously if dependent MV deduplication is enabled" 2024-02-14 12:54:42 -08:00
Nikita Taranov
4e5cfd11d0 upd comments 2024-02-14 19:53:27 +01:00
Nikolai Kochetov
9c626f9c36
Merge pull request #59899 from ClickHouse/refactor-squashing-for-inserts 
Refactor Squashing for inserts.
 2024-02-14 19:29:11 +01:00
Raúl Marín
11519f949b
Merge pull request #59731 from kitaisreal/asof-join-try-sort-with-radix-sort 
ASOF JOIN use trySort with RadixSort
 2024-02-14 15:54:22 +01:00
Igor Nikonov
fd5b572225 Fix 2024-02-14 13:00:22 +00:00
Nikolai Kochetov
3d86e87f7f Fixing tests. 2024-02-14 12:28:54 +00:00
Jordi Villar
83a184d928 Fix max num to warn message 2024-02-14 10:11:21 +01:00
Alexey Milovidov
2b4949bd18 Cleanups 2024-02-14 00:21:13 +01:00
Alexey Milovidov
b9d3ae3b0a
Merge pull request #58195 from Unalian/feat-58127 
Add InitialQuery event
 2024-02-13 23:51:03 +01:00
Igor Nikonov
3d91b12a2f Fix typo 2024-02-13 22:34:10 +00:00
Igor Nikonov
e9dcc751ac Merge remote-tracking branch 'origin/master' into query-log-on-insert-exception-over-http 2024-02-13 22:08:44 +00:00
Igor Nikonov
a27639a7f9 Fix: call first exception callback on exception 
it'll update query_log
 2024-02-13 22:06:00 +00:00
Kruglov Pavel
48a3b8e675
Merge pull request #58292 from ucasfl/negtive-position 
Resubmit: Support negtive position arguments
 2024-02-13 18:35:28 +01:00
Nikolai Kochetov
134837d6fb Fixing tests. 2024-02-13 16:16:53 +00:00
Nikolai Kochetov
253abd1643 Refactor Squashing for inserts. 2024-02-13 16:16:53 +00:00
Maksim Kita
4577975923 Fixed code review issues 2024-02-13 15:41:17 +03:00
Maksim Kita
bfa7938df9 Fixed tests 2024-02-13 15:41:17 +03:00
Maksim Kita
a359ceecb5 ASOF JOIN use trySort with RadixSort 2024-02-13 15:41:17 +03:00
Smita Kulkarni
3526d7982c Merge branch 'master' into Support_parameterized_view_with_analyzer 2024-02-13 13:08:35 +01:00
Nikita Taranov
fe173075e7 fix tests 2024-02-12 23:55:03 +01:00
Igor Nikonov
d1167d0b04 Fix: corrupted source + fail point 2024-02-12 20:56:13 +00:00
Igor Nikonov
eb44faf867 Test to reproduce absence of closing record in query_log 2024-02-12 20:41:46 +00:00
Nikita Taranov
91a38e73b4 impl 2024-02-12 19:36:12 +01:00
jsc0218
c2b2a7f903
Merge branch 'ClickHouse:master' into DictShortCircuit 2024-02-12 12:48:08 -05:00
Nikolai Kochetov
fd2ee19f65 Follow up for #58554. Cleanup. 2024-02-12 13:37:25 +00:00
Nikolai Kochetov
09d3b3c0d0
Merge pull request #58554 from ClickHouse/try-to-always-push-down-prewhere-from-query-plan 
Simplify optimize-push-to-prewhere from query plan
 2024-02-12 12:52:39 +01:00
Igor Nikonov
cb812d84cf Fix: check if std::function is set before calling it 2024-02-11 22:31:33 +00:00
Alexey Milovidov
c36397100b
Merge pull request #59662 from Algunenano/shard_query_start_time 
Fix query start time on non initial queries
 2024-02-10 03:09:06 +01:00
Nikita Mikhaylov
ab019e88d9 Better 2024-02-09 23:45:19 +01:00
Nikita Mikhaylov
d726f28b6c Better 2024-02-09 23:25:02 +01:00
Nikita Mikhaylov
64596c6b3b Merge branch 'master' of github.com:ClickHouse/ClickHouse into shabroo/master 2024-02-09 16:50:20 +01:00
Nikolai Kochetov
862c6cd79c
Merge pull request #57430 from ClickHouse/non-ready-set-ttl 
Non ready set in TTL WHERE.
 2024-02-09 16:31:26 +01:00
Igor Nikonov
effaaceb26
Merge pull request #59501 from ClickHouse/pr-better-replicas-failover-2 
Parallel replicas: better initial replicas failover (2)
 2024-02-09 15:50:56 +01:00
Nikolai Kochetov
1063d9ca50 Use opd prewhere optimizaer for old analyzer. 2024-02-09 14:47:24 +00:00
Michael Kolupaev
359dda5693
Fix DB type check - now it'll refuse to create in Replicated databases (#59743) 
Fix DB type check - now it'll refuse to create in Replicated databases
 2024-02-08 22:38:53 -08:00
Julia Kartseva
93bcf0b09c
Merge pull request #59781 from jkartseva/adaptive-async-timeout-better-logging 
Better logging for adaptive async timeouts
 2024-02-08 20:32:42 -08:00
Julia Kartseva
2286184885
Merge pull request #59699 from jkartseva/disable_async_inserts_for_dependent_mv_dedup 
Insert synchronously if dependent MV deduplication is enabled
 2024-02-08 19:23:51 -08:00
Ilya Yatsishin
1c58cafcf3
Merge pull request #59741 from vitlibar/fix-skipping-unused-shards-with_analyzer 
Fix skipping unused shards with analyzer
 2024-02-09 02:44:59 +04:00
Julia Kartseva
e598ad9157 Better logging for adaptive async timeouts 
Log the adaptive async timeout value after normalization.
 2024-02-08 21:42:44 +00:00
Raúl Marín
74dbb06d3f Merge remote-tracking branch 'blessed/master' into shard_query_start_time 2024-02-08 19:43:20 +01:00
Vitaly Baranov
1b9620001b Move ColumnArray-specific code out of the ActionsDAG. 2024-02-08 15:31:32 +01:00
Igor Nikonov
3fb30a23d9 Merge remote-tracking branch 'origin/master' into pr-better-replicas-failover-2 2024-02-08 09:09:11 +00:00
Julia Kartseva
ece4febe32 Insert synchronously if dependent MV deduplication is enabled 2024-02-08 08:42:04 +00:00
Alexander Tokmakov
510f90edf8
Merge pull request #59688 from ClickHouse/revert_39507_again 
Temporarily remove a feature that doesn't work
 2024-02-08 02:27:55 +01:00
Vitaly Baranov
12a71375b8 Fix skipping unused shards with analyzer. 2024-02-08 00:44:59 +01:00
Nikolai Kochetov
41d6243170 Fix PreparedSets 2024-02-07 12:43:26 +00:00
Nikolai Kochetov
d1902cdba0 Fix some tests. 2024-02-07 12:36:24 +01:00
Nikolai Kochetov
06d112135e Simplify prewhere push down from query plan. Try to always use it. 2024-02-07 12:36:20 +01:00
Nikolai Kochetov
a3d1b3f502 Merge branch 'master' into non-ready-set-ttl 2024-02-07 11:29:04 +00:00
Alexander Tokmakov
77e4b0c8d9 Revert "Merge pull request #59122 from arthurpassos/bring_back_attach_partition_pr" 
This reverts commit ac90637170, reversing
changes made to fe2b5cde05.
 2024-02-07 02:52:43 +01:00
Raúl Marín
b43f90fce2 Fix query start time on non initial queries 2024-02-06 19:55:50 +01:00
Raúl Marín
5c68f9cabd Maintain function alias in RewriteSumFunctionWithSumAndCountVisitor 2024-02-06 19:09:28 +01:00
Igor Nikonov
8748d8c537 UNEXPECTED_CLUSTER error for cluster with more than 1 shard 2024-02-05 15:10:51 +00:00
Robert Schulze
e414384727
Merge remote-tracking branch 'rschu1ze/master' into order-by-all-ambiguities 2024-02-05 13:17:02 +00:00
Igor Nikonov
92a36109c3 Merge remote-tracking branch 'origin/master' into pr-better-replicas-failover-2 2024-02-05 12:07:44 +00:00
Nikita Mikhaylov
6ba35b5459
Make comments for system tables also available in local mode and enforce comments not to be empty (#59493) 
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2024-02-05 11:17:58 +01:00
Igor Nikonov
df4bc398fb Merge remote-tracking branch 'origin/master' into fix-messed-up-progress 2024-02-03 10:25:10 +00:00
Julia Kartseva
f41f4e4bd0
Merge pull request #58486 from jkartseva/adjustable-async-timeouts 
Adjustable asynchronous insert timeouts
 2024-02-02 08:53:58 -08:00
Igor Nikonov
74e04e1819 Merge remote-tracking branch 'origin/master' into fix-messed-up-progress 2024-02-02 13:11:07 +00:00
Alexander Gololobov
095648d7bc
Merge pull request #59388 from ClickHouse/system_zookeeper_retries 
Retry disconnects and expired sessions when reading system.zookeeper
 2024-02-02 11:52:15 +01:00
serxa
1d31ac2a0c add FORGET PARTITION query to remove old parition nodes from ZooKeeper 2024-02-02 10:03:31 +00:00
Robert Schulze
dd484fc312
Simplify what happens in SELECT all, ... [...] ORDER BY ALL 
Previously setting `enable_order_by_all` distinguished for ORDER BY ALL
whether we should sort by column 'all' (if given in the SELECT clause)
or by all columns. The actual behavior was not always intuitive.

Now, we throw unconditionally an exception which also simplifies the
handling a bit. Only an edge case is affected and if users really want
to run ORDER BY ALL on a column names 'all', they can alias it.
 2024-02-02 09:53:35 +00:00
Julia Kartseva
689c368b76 Adaptive async timeouts: consider queue flush history 
In addition to the time since the most recent insert,
consider the elapsed time between the two recent queue
flushes when decreasing the timeout or processing an
entry synchronously.
 2024-02-01 22:45:10 +00:00
Julia Kartseva
4ef02a1895 Log timeout for entries processed asynchronously 2024-02-01 22:45:09 +00:00
Julia Kartseva
c04e5a4f35 Asynchronous insert queue size and bytes metrics 
Add AsynchronousInsertQueueSize and AsynchronousInsertQueueBytes
metrics to improve observability of asynchronous inserts.
The metrics do not account for tasks dispatched for immediate processing
(as opposed to, e.g., PendingAsyncInsert).

```
SELECT value
FROM system.metrics
WHERE metric IN ('AsynchronousInsertQueueSize', 'PendingAsyncInsert')

Query id: a711dd83-b48d-4ad5-8031-fa59b21a7c38

┌─value─┐
│    18 │
│    23 │
└───────┘
```

```
SELECT value
FROM system.metrics
WHERE metric IN ('AsynchronousInsertQueueSize', 'AsynchronousInsertQueueBytes')

Query id: b35a7ceb-2bb5-46ad-b301-e6cf03508699

┌─value─┐
│    28 │
│  1372 │
└───────┘
```
 2024-02-01 22:45:09 +00:00
Julia Kartseva
04c8bd1be5 Adaptive asynchronous insert timeouts 
Implement the algorithm described in #56783 for adaptive asynchronous
insert timeouts.
- The adaptive async insert timeout can take values within
[async_insert_busy_timeout_min_ms, async_insert_busy_timeout_max_ms].
- The initial value is set to async_insert_busy_timeout_min_ms.
- If the elapsed time since the most recent queue insert was
  greater than the maximum timeout, process the queue content immediately,
  and reduce the timeout.
- If the elapsed time was long enough (longer than a would-be decreased
  timeout), decrease the timeout.
- The adaptive timeout is changes exponentially based on the
  async_insert_busy_timeout_{increase|decrease}_rate.

Fixes: https://github.com/ClickHouse/ClickHouse/issues/56783
 2024-02-01 22:45:06 +00:00
Igor Nikonov
82d7b22144 Parallel replicas: better replicas failover 2024-02-01 21:06:29 +00:00
Kruglov Pavel
d485e36f20
Fix style 2024-02-01 16:57:46 +01:00
jsc0218
4ce770aebf Merge remote-tracking branch 'origin/master' into DictShortCircuit 2024-02-01 15:29:27 +00:00
Raúl Marín
18386f7fb9 Merge remote-tracking branch 'blessed/master' into parallel_replicas_cte_fix 2024-02-01 12:04:46 +01:00
Raúl Marín
b93f003b3e
Merge pull request #58216 from bigo-sg/fix_quantiles_gk_state 
Fix quantilesGK bug
 2024-02-01 11:58:02 +01:00
Raúl Marín
a0222c0479 Fix problem detected by UBSAN 2024-02-01 11:46:59 +01:00
Julia Kartseva
bbaa08199a Introduce polling timeout for asynchronous queue 
The current timeout for checking updates in the asynchronous queue is
equal to the timeout used for queue entry
(async_insert_busy_timeout_ms).
That means that, in the worst case, an entry spends twice the time of the
asynchronous timeout in the queue.
 2024-02-01 08:27:41 +00:00
Vitaly Baranov
a193e01c8f
Merge pull request #54737 from arenadata/ADQM-1150 
Added maximum sequential login failures to the quota.
 2024-01-31 16:06:17 +01:00
Raúl Marín
dad6ea0930
Merge pull request #59239 from Algunenano/parallel_replicas_cte 
Disable parallel replicas JOIN with CTE (not analyzer)
 2024-01-31 14:45:13 +01:00
avogar
aaed835415 Validate variants, use new validation on create queries 2024-01-31 12:59:26 +00:00
Raúl Marín
f875151b97
Merge pull request #59379 from Algunenano/fix_translate_asserts 
Add sanity checks for function return types
 2024-01-31 11:56:11 +01:00
taiyang-li
a89890babe fix bug of quantileGK 2024-01-31 12:14:04 +08:00
jsc0218
8583192948 refactior short circuit setting 2024-01-30 21:26:43 +00:00
Igor Nikonov
d8a6e53d0b Use FQDN as host name by default for TCP 
- instead of display_name setting value
 2024-01-30 20:27:30 +00:00
Alexander Gololobov
c348c4e828 Move ZooKeeperRetries.h to Common 2024-01-30 19:02:17 +01:00
avogar
7a1458c922 Fix validating suspicious/experimental types in nested types 2024-01-30 15:21:58 +00:00
Raúl Marín
f79a402e6d Add sanity checks for function return types 2024-01-30 14:10:20 +01:00
Kseniia Sumarokova
e16aaa8d81
Merge branch 'master' into broken-projections-better-handling 2024-01-30 10:54:22 +01:00
flynn
4f12ca249d Fix typo 2024-01-30 07:01:07 +00:00
flynn
9858c41849 Merge branch 'master' of github.com:ClickHouse/ClickHouse into negtive-position 2024-01-30 04:03:35 +00:00
flynn
0ded580011 Fix 2024-01-30 04:03:27 +00:00
Alexey Milovidov
ac90637170
Merge pull request #59122 from arthurpassos/bring_back_attach_partition_pr 
Revert revert attach partition PR
 2024-01-30 04:02:31 +01:00
Alexey Gerasimchuck
559de083a3 Implemented failed login attempt counting using quota 2024-01-29 23:20:38 +00:00
Yarik Briukhovetskyi
5ba82eed81
Merge pull request #58841 from yariks5s/dictionary_table_exception_fix 
Fixing exceptions when table/dictionary already exist
 2024-01-29 22:41:06 +01:00
Arthur Passos
15d04d0a78
Merge branch 'master' into bring_back_attach_partition_pr 2024-01-29 15:16:49 -03:00
Raúl Marín
a6de4a21a3 Merge remote-tracking branch 'blessed/master' into parallel_replicas_cte 2024-01-29 13:33:22 +01:00
Kruglov Pavel
6858d2f4ca
Merge pull request #58047 from Avogar/variant-data-type 
Implement Variant data type
 2024-01-29 11:36:08 +01:00
Arthur Passos
0bbfef884c
Merge branch 'ClickHouse:master' into bring_back_attach_partition_pr 2024-01-28 17:43:38 -03:00
avogar
92db86b829 Merge branch 'master' of github.com:ClickHouse/ClickHouse into variant-data-type 2024-01-28 14:33:49 +00:00
Alexey Milovidov
267b077038 Merge remote-tracking branch 'origin/master' into remove-kql 2024-01-28 14:24:26 +01:00
Alexey Milovidov
e0f0b21e90 Remove KQL 2024-01-28 14:24:13 +01:00
Smita Kulkarni
b7a6dbfbb9 Check engine to avoid possible seg fault 2024-01-27 18:22:37 +01:00
Azat Khuzhin
4248ae509a Support specifying settings for Distributed engine in config (like for MergeTree) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-27 18:06:16 +01:00
avogar
5833641fa5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into variant-data-type 2024-01-26 16:54:02 +00:00
Kseniia Sumarokova
99653fc778
Merge branch 'master' into broken-projections-better-handling 2024-01-26 16:27:23 +01:00
Arthur Passos
ce6877cbb6
Merge branch 'ClickHouse:master' into bring_back_attach_partition_pr 2024-01-26 10:57:11 -03:00
Raúl Marín
176d8beac9 Disable parallel replicas JOIN with CTE (not analyzer) 2024-01-26 14:36:38 +01:00
Nikolai Kochetov
10d4f1fd87
Merge pull request #59228 from kitaisreal/actions-dag-build-filter-actions-refactoring 
Actions dag build filter actions refactoring
 2024-01-26 11:34:19 +01:00
pufit
92e63bf45c
Load server configs in clickhouse local (#59231) 
Load server configs in clickhouse local
 2024-01-25 23:13:31 -05:00
Maksim Kita
5bb734a4bb ActionsDAG buildFilterActionsDAG refactoring 2024-01-25 18:24:14 +03:00
Kseniia Sumarokova
70514e3d14
Merge pull request #57615 from ClickHouse/filecache-small-optimization 
fs cache: small optimization
 2024-01-25 15:50:09 +01:00
Maksim Kita
2a327107b6 Updated implementation 2024-01-25 14:31:49 +03:00
Kseniia Sumarokova
b13be4fd55
Merge pull request #59175 from ClickHouse/filecache-minor 
Move some headers
 2024-01-25 11:33:57 +01:00
Dmitry Novik
a2d7bcbd5f
Merge pull request #58958 from ClickHouse/analyzer-insert-block 
Analyzer: fix storage replacement with insertion block
 2024-01-25 09:25:50 +01:00
Igor Nikonov
7ae7086a2d
Merge pull request #59153 from ClickHouse/fix-pr-limit-by 
Fix: LIMIT BY and LIMIT in distributed query
 2024-01-25 01:51:52 +01:00
jsc0218
311276252e Merge remote-tracking branch 'origin/master' into DictShortCircuit 2024-01-24 19:29:56 +00:00
kssenii
9038f0710a Minor 2024-01-24 18:44:13 +01:00
Kseniia Sumarokova
d674a5ce24
Merge pull request #59168 from ClickHouse/kssenii-patch-9 
Decrease log level for one log message
 2024-01-24 18:39:55 +01:00
Kseniia Sumarokova
235980d52e
Update FileCache.cpp 2024-01-24 17:28:01 +01:00
Dmitry Novik
95e2a3d6da
Merge pull request #58911 from ClickHouse/fix-index-hint-crash 
Fix crash when indexHint() is used
 2024-01-24 17:16:22 +01:00
Kseniia Sumarokova
a4186dc4d3
Decrease log level for one log message 2024-01-24 16:34:29 +01:00
Igor Nikonov
698aef50c2 Fix: LIMIT BY in distriubted query 2024-01-24 12:43:01 +00:00
kssenii
0d7710ee4b Fix build 2024-01-24 12:39:01 +01:00
kssenii
ed607fbd2c Merge remote-tracking branch 'origin/master' into filecache-small-optimization 2024-01-24 11:58:44 +01:00
Arthur Passos
e6b17377d0
Merge branch 'master' into bring_back_attach_partition_pr 2024-01-23 14:52:47 -03:00
Nikolai Kochetov
a26f8f5425
Merge pull request #59076 from ClickHouse/revert-59059-revert-58838-allow-parallel-replicas-for-join-with-analyzer 
Revert "Revert "Allow parallel replicas for JOIN with analyzer [part 1].""
 2024-01-23 17:01:19 +01:00
Arthur Passos
799a94081b Revert "Revert "Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition"" 
This reverts commit a1c83e2f51.
 2024-01-23 11:40:42 -03:00
Robert Schulze
a1c83e2f51
Revert "Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition" 2024-01-23 15:26:04 +01:00
Alexander Gololobov
9a0e5d0aac
Merge pull request #58914 from ClickHouse/ot_spans_in_insert 
Opentelemetry spans to analyze CPU and S3 bottlenecks on inserts
 2024-01-23 13:53:13 +01:00
Nikolai Kochetov
fcbab407b8
Merge branch 'master' into revert-59059-revert-58838-allow-parallel-replicas-for-join-with-analyzer 2024-01-23 12:21:44 +01:00
Dmitry Novik
7a95eb8bd1
Review fixes 2024-01-23 11:59:41 +01:00
Robert Schulze
e23e7e6b73
Merge pull request #53710 from rschu1ze/system-reload-asynchronous-metrics 
Add statement `SYSTEM RELOAD ASYNCHRONOUS METRICS`
 2024-01-23 11:56:25 +01:00
kssenii
b76630c955 Merge remote-tracking branch 'origin/master' into broken-projections-better-handling 2024-01-23 11:34:58 +01:00
Dmitry Novik
6962859e43
Merge pull request #59068 from ClickHouse/check-optimization-setting 
Followup to 57853
 2024-01-23 11:24:31 +01:00
Kseniia Sumarokova
f6ffbb4c0a
Merge pull request #59049 from ClickHouse/fix-data-race-in-factories 
Fix data race on query_factories_info
 2024-01-23 11:22:40 +01:00
Robert Schulze
e67076ea2b
Merge pull request #55201 from canhld94/column_level_compress_block 
Column-level compression block sizes
 2024-01-23 11:22:03 +01:00
Nikita Mikhaylov
bd7024ff9b
Merge pull request #59083 from ClickHouse/fix-startup-failure 
Fix server shutdown due to exception while loading metadata
 2024-01-23 09:48:38 +01:00
Arthur Passos
24b8bbe9fa
Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition (#39507) 
* temp commit

* temp commit

* draft impl for feedback

* fix weird style changes

* fix weird style changes

* fix weird style changes

* fix weird style changes

* fix weird style changes

* aa

* aa

* Add integ tests and remove partition key restriction

* fix small incosistency in partition id

* style fix

* style fix

* style fix

* use existing DataPartStorageBuilder instead of new one

* Refactor part clone to make it more readable and maintainable

* Add MergeTreeDataPartCloner docs

* define ErrorCodes::BAD_ARGUMENTS

* Rebase

* camel case methods

* address some comments

* yet another rebase?

* Move from integ tests to stateless tests

* address more comments

* add finalize on min_max_idx files

* Add sync option to DistinctPartitionExpCloner

* just a temp test

* revert temp change

* Use echoOn to distinguish test queries

* remove comment

* fix build issue during rebase

* atempt to fix build after rebase

* finally fix build

* clear minmaxidx hyperrectangle before loading it

* Fix error on min_max files deletion where it was being assumed that partition expression contained all columns

* get it to the state it was previously

* add missing include

* getting functional?

* refactoring and renaming

* some more refactoring

* extern bad arguments

* try to fix style

* improvements and docs

* remove duplicate includes

* fix crash

* make tests more stable by ordering

* rebase once again..

* fix

* make ci happy?

* fix rebase issues

* docs

* rebase, but prolly needs to be improved

* refactor out from nasty inheritance to static methods

* fix style

* work around optional

* refactor & integrate some changes

* update column_type

* add tests by dencrane

* set utc

* fix ref file

* fix tests

* use MergeTree instead of SummingMergeTree

* mark MergeTreeDataPart::getBlock as const

* address a few comments

* compute module function name size at compile time

* simplify branching in getPartitionAstFieldsCount

* remove column_indexes argument

* merge getBlock with buildBlock

* add some const specifiers

* small adjustments

* remove no longer needed isNull check

* use std::min and max to update global min max idx

* add some assertions

* forward declare some symbols

* fix grammar

* forward decl

* try to fix build..

* remove IFunction forward decl

* Revert "use std::min and max to update global min max idx"

This reverts commit b2fe79dda7.

* Revert "remove no longer needed isNull check"

This reverts commit 129db2610f.

* Revert "Revert "remove no longer needed isNull check""

This reverts commit 9416087dd8.

* Revert "Revert "use std::min and max to update global min max idx""

This reverts commit 20246d4416.

* remove some comments

* partial use of MonotonicityCheckMatcher

* ranges

* remove KeyDescriptionMonotonicityChecker

* remove duplication of applyfunction

* move functions to anonymous namespace

* move functions to cpp

* Relax partition compatibility requirements by accepting subset, add tests from partitioned to unpartitioned

* updte reference file

* Support for partition by a, b, c to partition by a, b

* refactoring part 1

* refactoring part 2, use hyperrectangle, still not complete

* refactoring part 3, build hyperrectangle with intersection of source & destination min max columns

* Support attaching to table with partition expression of multiple expressions

* add tests

* rename method

* remove some code duplication

* draft impl for replicatedmergetree, need to dive deeper

* ship ref file

* fix impl for replicatedmergetree..

* forbid attach empty partition replicatedmergetree

* Add replicated merge tree integration tests

* add test missing files

* fix black

* do not check for monotonicity of empty partition

* add empty tests & fix replicated

* remove no longer needed buildBlockWithMinMaxINdexes

* remove column logic in buildHyperrectangle

* simplify implementation by using existing methods

* further simplify implementation

* move all MergeTreeDataPartClone private methods to .cpp file

* decrease decomposition

* use different namespaces

* reduce code duplication

* fix style

* address a few comments

* add chassert to assert arguments size on MonotonicityCheckVisitor

* remove deleteMinMaxFiles method

* remove useless checks from sanitycheck

* add tests for attach partition (not id)

* Remove sanityCheckASTPartition and bring back conditional getPartitionIDFromQuery

* remove empty block comment

* small fixes

* fix formatting

* add missing include

* remove duplicate iuncludes

* trigger ci

* reduce some code duplication

* use updated partition id on replicatedmergetree

* fix build

* fix build

* small refactor

* do not use insert increment on fetch part

* remove duplicate includes

* add one more integ test

* black

* black

* rely on partition exp instead of partition id on replicated part fetch to decide if it is a different partition exp

* add one more integ test

* add order by clause

* fix black

---------

Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2024-01-22 21:00:04 +01:00
serxa
1d6d1182fd Fix server shutdown due to exception while loading metadata 2024-01-22 19:38:26 +00:00
Robert Schulze
90a0ea36b3
Add statement 'SYSTEM RELOAD ASYNCHRONOUS METRICS' 2024-01-22 17:14:58 +00:00
Robert Schulze
1262945df7
Cosmetics 2024-01-22 16:51:02 +00:00
Nikolai Kochetov
65c5ea31a8
Revert "Revert "Allow parallel replicas for JOIN with analyzer [part 1]."" 2024-01-22 17:09:34 +01:00
Dmitry Novik
3d0e915088
Followup to 57853 2024-01-22 14:44:07 +01:00
Alexander Tokmakov
0f0785cf20
Merge pull request #59059 from ClickHouse/revert-58838-allow-parallel-replicas-for-join-with-analyzer 
Revert "Allow parallel replicas for JOIN with analyzer [part 1]."
 2024-01-22 14:07:26 +01:00
Dmitry Novik
a18a8d8ea3
Merge pull request #59009 from nickitat/uniq_optimisation_for_distributed 
`uniqExact` state parallel merging for distributed queries
 2024-01-22 14:05:02 +01:00
kssenii
4b4eac7c06 Fix 2024-01-22 13:55:15 +01:00
Alexander Tokmakov
4315d44517
Revert "Allow parallel replicas for JOIN with analyzer [part 1]." 2024-01-22 13:30:35 +01:00
kssenii
3faf4087e3 Fix 2024-01-22 12:07:49 +01:00
Raúl Marín
d9ab300300
Merge pull request #57853 from jiebinn/rewriteSumFunction 
Rewrite the AST/Analyzer of sum(column +/- literal) function
 2024-01-22 10:41:30 +01:00
jsc0218
6f3124acdb
Merge pull request #58402 from zhangyifan27/add_max_estimated_execution_time 
Seperate max_execution_time and max_estimated_execution_time
 2024-01-21 19:57:06 -05:00
Vitaly Baranov
01aafda27b
Merge pull request #58804 from vitlibar/allow-kill-query-for-backups 
Allow KILL QUERY for backups
 2024-01-20 23:48:18 +01:00
Igor Nikonov
4be068c73b
Merge pull request #58993 from ClickHouse/revert-58992-revert_flaky 
Revive: Parallel replicas custom key: skip unavailable replicas
 2024-01-20 19:44:56 +01:00
robot-clickhouse
273a06856a
Merge pull request #59025 from ClickHouse/cpu-scheduler 
Split resource scheduler off `IO/` into `Common/Scheduler/`
 2024-01-20 15:43:50 +01:00
Kseniia Sumarokova
f3a1a2f3ad
Fix build 2024-01-20 15:02:13 +01:00
Kseniia Sumarokova
5d67e26fc4
Merge pull request #58869 from ClickHouse/force-check-for-cached-custom-disk 
Fix for custom cached disk creation
 2024-01-20 14:47:36 +01:00
serxa
824deab761 split resource scheduler off IO/ into Common/Scheduler/ 2024-01-20 12:06:26 +00:00
Igor Nikonov
59025b2d3a Merge remote-tracking branch 'origin/master' into revert-58992-revert_flaky 2024-01-19 23:38:24 +00:00
Kruglov Pavel
20379d34d5
Merge branch 'master' into variant-data-type 2024-01-19 23:17:42 +01:00
Jiebin Sun
e9662d4d62 polish the code and adjust some code style 2024-01-20 01:43:06 +08:00
Alexey Milovidov
b249416671
Merge pull request #58968 from ClickHouse/clickhouse-local-less-threads 
Less threads in `clickhouse-local`
 2024-01-19 18:03:36 +01:00
Nikita Taranov
a47d15ab62 impl 2024-01-19 16:59:01 +01:00
Dmitry Novik
728276ab3f Fix crash when indexHint() is used 2024-01-19 14:59:33 +00:00
Kseniia Sumarokova
facbd89d48
Merge pull request #58986 from ClickHouse/fix-typo 
Fix typo
 2024-01-19 14:12:50 +01:00
Igor Nikonov
131f71b50a
Revert "Revert flaky" 2024-01-19 13:51:30 +01:00
Raúl Marín
477489a05d Revert "Merge pull request #57235 from ClickHouse/pr-custom-key-failover" 
This reverts commit 8ba9b4a7ef, reversing
changes made to 0f39245cb8.
 2024-01-19 12:46:05 +00:00
kssenii
5113b719e9 Fix typo 2024-01-19 12:48:48 +01:00
Kseniia Sumarokova
b6094ac233
Merge branch 'master' into filecache-small-optimization 2024-01-19 11:59:04 +01:00
kssenii
48e489682b Merge remote-tracking branch 'origin/master' into broken-projections-better-handling 2024-01-19 10:49:27 +01:00
Kseniia Sumarokova
1c74e7fa08
Merge pull request #58902 from ClickHouse/fs-cache-refactorings 
fs cache: minor refactoring
 2024-01-19 10:25:45 +01:00
Alexey Milovidov
0c1e92aea5 Less threads in clickhouse-local 2024-01-19 04:53:14 +01:00
Robert Schulze
15700592f7
Merge remote-tracking branch 'ClickHouse/master' into column_level_compress_block 2024-01-18 19:12:57 +00:00
Yakov Olkhovskiy
1144f60515 clenup 2024-01-18 18:17:07 +00:00
Yakov Olkhovskiy
b914938c4c clenup 2024-01-18 18:10:54 +00:00
Yakov Olkhovskiy
bb1058e2b2 fix storage replacement with insertion block for analyzer 2024-01-18 17:59:23 +00:00
Kruglov Pavel
5444cde408
Merge branch 'master' into variant-data-type 2024-01-18 18:31:27 +01:00
Kseniia Sumarokova
bfe8037653
Merge branch 'master' into fix-data-race-in-slru-2 2024-01-18 16:54:52 +01:00
kssenii
d20a33424f Fix race 2024-01-18 16:53:10 +01:00
Alexander Gololobov
e9919f47c1 Move paramaters from span name to attributes 2024-01-18 14:26:19 +01:00
Nikolai Kochetov
9db8b50753
Merge pull request #58838 from ClickHouse/allow-parallel-replicas-for-join-with-analyzer 
Allow parallel replicas for JOIN with analyzer [part 1].
 2024-01-18 14:01:08 +01:00
Vitaly Baranov
4c6d3e7efa Add server settings "shutdown_wait_backups_and_restores" (default=true) 
to set whether shutdown should wait for running backups to finish or just cancel them.
 2024-01-18 13:27:14 +01:00
kssenii
dd396b678a Fix data race 2024-01-18 12:23:40 +01:00
Jiebin Sun
206e1c87e0 revise the comments 2024-01-18 19:07:02 +08:00
Duc Canh Le
22afcd46f4 add ALTER COLUMN MODIFY/RESET SETTING 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2024-01-18 05:17:07 +00:00
Smita Kulkarni
d3b4dea805 Fix clang tidy build 2024-01-17 20:02:17 +01:00
Alexey Milovidov
1161dbbd54
Merge pull request #57993 from mkmkme/mkmkme/reload-config-again 
Update `query_masking_rules` when reloading the config, attempt 2
 2024-01-17 17:09:36 +01:00
Jiebin Sun
784cab638a rename some function_name and add more tests 2024-01-17 22:10:08 +08:00
kssenii
591a9f9661 Update unit tests 2024-01-17 14:52:24 +01:00
kssenii
c905937ef6 Refactoring 2024-01-17 13:22:59 +01:00
Antonio Andelic
c8a313da5a
Merge pull request #58873 from ClickHouse/fix-00089_group_by_arrays_of_fixed 
Fix `00089_group_by_arrays_of_fixed` with external aggregation
 2024-01-17 12:50:30 +01:00
Kseniia Sumarokova
0b3ebcda08
Update SLRUFileCachePriority.cpp 2024-01-17 11:13:57 +01:00
Smita Kulkarni
8d88f4cf87 Update setting is_parameterized_view & settings columns for view 2024-01-17 09:42:53 +01:00
Mikhail Koviazin
9e0414f557
fix data race in SensitiveDataMasker 
This commit fixes the data race that was introduced in 3067ca6. That commit
added scope lock for `getInstance` and `setInstance`, but `getInstance`
returned a pointer that could be invalidated between call for `getInstance` and
actual usage of the pointer.

This commit changes `SensitiveDataMasker` singleton from `unique_ptr` to
`shared_ptr`, so even if `setInstance` will be called right after
`getInstance`, the pointer returned from `getInstance` will live long enough to
call `wipeSensitiveData` on it.
 2024-01-17 08:10:50 +02:00
Mikhail Koviazin
6e6197019c
Reintroduce commit 3067ca6 
Update `query_masking_rules` when reloading the config

This reverts commit 7e55eab1c3.
 2024-01-17 08:10:49 +02:00
Vitaly Baranov
c15edebadf Use the same row in system.processes and the same query_id for both a BACKUP/RESTORE query started by user and its asynchronous execution in a separate thread. 2024-01-16 18:49:37 +01:00
Antonio Andelic
4ce6b64d62 Fix 00089_group_by_arrays_of_fixed with external aggregation 2024-01-16 15:16:13 +00:00
Vitaly Baranov
185f6cc37c Add tests for cancelling backup/restore queries. 2024-01-16 15:07:20 +01:00
Kseniia Sumarokova
f2a31db9d2
Merge branch 'master' into filecache-small-optimization 2024-01-16 15:04:02 +01:00
kssenii
8decf5039a Fix for custom cached disk 2024-01-16 14:41:12 +01:00
Smita Kulkarni
76ba60aa24 Merge branch 'master' into Support_parameterized_view_with_analyzer 2024-01-16 11:26:41 +01:00
vdimir
de3e7012d5
Merge pull request #58652 from ClickHouse/vdimir/analyzer_hash_join_duplicate_columns 
Analyzer: Fix assertion in HashJoin with duplicate columns
 2024-01-16 11:12:31 +01:00
vdimir
54a79c1f3d
Merge pull request #58123 from ClickHouse/vdimir/cluster_discovery_password 
ClusterDiscovery supports setting username and password
 2024-01-16 11:05:54 +01:00
Kseniia Sumarokova
5fcf1b216b
Merge pull request #58842 from ClickHouse/fix-data-race-in-slru 
fs cache: fix data race in slru
 2024-01-16 08:53:41 +01:00
Alexey Milovidov
ebfea07f93
Merge pull request #58832 from kitaisreal/database-catalog-background-tasks-add-log-names 
DatabaseCatalog background tasks add log names
 2024-01-16 08:42:38 +01:00
Alexey Milovidov
8973e7d873
Merge pull request #58833 from ClickHouse/fix-analyzer-remote-aggregation 
Analyzer: Resolve GROUPING function on shards
 2024-01-15 23:33:14 +01:00
kssenii
e2c5e5fa70 Fix data race in slru 2024-01-15 21:19:11 +01:00
yariks5s
3eba767805 init 2024-01-15 19:17:13 +00:00
Nikolai Kochetov
733ceaa775 Allow parallel replicas for JOIN with analyzer. 2024-01-15 18:48:12 +00:00
Dmitry Novik
0f7daeffc8 Analyzer: Resolve GROUPING function on shards 2024-01-15 16:02:37 +00:00
Maksim Kita
573fa8c375 DatabaseCatalog background tasks add log names 2024-01-15 18:59:34 +03:00
jsc0218
0d46bfe66a Merge remote-tracking branch 'origin/master' into DictShortCircuit 2024-01-15 14:52:49 +00:00
vdimir
61f2ab3ec5
Update HashJoin::getNonJoinedBlocks 2024-01-15 11:37:33 +00:00
vdimir
6a1b005d7f
Fix build 2024-01-15 11:32:23 +00:00
vdimir
4adeb249d3
Analyzer: Fix assertion in HashJoin with duplicate columns 2024-01-15 11:32:22 +00:00
vdimir
89fadd7769
Update src/Interpreters/ClusterDiscovery.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2024-01-15 11:22:54 +00:00
vdimir
8dc1756532
Support <secret> in cluster discovery 2024-01-15 11:22:53 +00:00
vdimir
012713fc31
ClusterDiscovery supports setting username and password 2024-01-15 11:22:48 +00:00
Azat Khuzhin
48a700edba Support query parameters in ALTER TABLE ... PART 
But, to make this happen it is not enough to simply fix the parser, the
whole ASTs for ALTER should be fixed as well, since they store `ASTPtr`
instead of `IAST *`, and so because of `ReplaceQueryParameterVisitor`
does not work for ALTER query.

So this patch converts ASTPtr to `IAST *` for ASTAlterQuery and
ASTAlterCommand (plus it also fixes when the children for
ASTAlterCommand had not been updated sometimes).

And only after this it is possible to adjust the parser (see
places with `parser_string_and_substituion` in the parser code).

v2: also fix ASTAlterCommand::clone (add missing components) and ParserAlterCommand::parseImpl
v3: fix APPLY DELETED MASK
v4: fix mutations_execute_nondeterministic_on_initiator/mutations_execute_subqueries_on_initiator
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2024-01-15 14:16:54 +03:00
Alexander Tokmakov
5128607995
Merge pull request #58393 from jaymebrd/feat/add-from-modifier-sync-replica-lightweight 
add `from srcreplicas` modifier  for `SYSTEM SYNC REPLICA LIGHTWEIGHT`
 2024-01-14 20:53:16 +01:00
Yakov Olkhovskiy
62660e6e84
Merge pull request #58703 from ClickHouse/imp-error-message 
Add identifier name to the error INVALID_IDENTIFIER
 2024-01-14 07:46:33 -05:00
Duc Canh Le
f755e77534 move column level setting to the end of declaration 
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2024-01-14 09:36:32 +00:00
Duc Canh Le
ac76eecb1e Merge branch 'master' into column_level_compress_block 
Resolve conflicts

Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
 2024-01-14 09:32:43 +00:00
Alexey Milovidov
65ac905032
Merge pull request #54366 from canhld94/final_no_copy 
Implement vertical replacing FINAL
 2024-01-14 04:21:07 +01:00
Alexey Milovidov
776ea26ce7
Update PreparedSets.h 2024-01-14 02:45:51 +03:00
Alexey Milovidov
e92e3615e3
Merge branch 'master' into non-ready-set-ttl 2024-01-13 21:11:51 +01:00
Alexey Milovidov
d46ee32e06
Merge pull request #58724 from Algunenano/promote_f32 
Do not promote float32 automatically
 2024-01-13 20:29:10 +01:00
Alexey Milovidov
d112492c56 Remove some code 2024-01-13 03:48:04 +01:00
Raúl Marín
0684cc7b76 Merge remote-tracking branch 'blessed/master' into i58727 2024-01-12 16:10:43 +00:00
Nikita Mikhaylov
8edab24189
Add comments to all columns of system tables (#58356) 2024-01-12 16:39:22 +01:00