Commit Graph

8103 Commits

Author SHA1 Message Date
Anton Popov
312dd725b4 allow to send profile events to trace_log 2022-11-24 20:12:16 +00:00
Igor Nikonov
20e67b7140 Merge remote-tracking branch 'origin/master' into HEAD 2022-11-24 13:10:37 +00:00
Igor Nikonov
38b7c494d8 Remove unnecessary changes 2022-11-24 13:09:07 +00:00
Alexander Tokmakov
52d33fff00
Merge pull request #43366 from zzsmdfj/issue_15357_MaterializeMySQL_support_drop_mulit_table
MaterializeMySQL_support_drop_mulit_table
2022-11-24 14:54:24 +03:00
Sema Checherinda
6fd7dcf26f
Merge branch 'master' into lock-free-drop-partition 2022-11-24 12:34:19 +01:00
Antonio Andelic
a6f38cb1cc
Merge branch 'master' into keeper-prometheus 2022-11-24 12:12:08 +01:00
Antonio Andelic
b065f9747d Fix test 2022-11-24 10:12:06 +00:00
Salvatore Mesoraca
0488132133 Do not assume that the original query was valid when transforming JOINs
Before this change, during the JOIN transformation, the query
was rewritten in such a way that some references to non-existing
columns were replaced with references to columns with the same
name in other tables.

We now check that the original query was correct before making
changes that may fix it accidentally.
2022-11-24 09:48:51 +01:00
zzsmdfj
11b9cd761f to MaterializeMySQL_support_drop_mulit_table-clean up unnecessary code 2022-11-24 12:08:40 +08:00
Alexander Tokmakov
f0fc71f09f
Merge pull request #43489 from azat/drop-fix
Avoid possible DROP hung due to attached web disk
2022-11-23 23:54:55 +03:00
Nikolai Kochetov
62debf42b4 Aggregating in order [in progress]. 2022-11-23 18:38:12 +00:00
Antonio Andelic
c52ca6e10e Merge branch 'master' into substitute-udf-create 2022-11-23 16:08:37 +00:00
Sema Checherinda
eccb62e4c3 work with the latest remarks 2022-11-23 15:16:09 +00:00
Sema Checherinda
06ac225425 canBeRemoved take into account creation_csn then no txn 2022-11-23 15:16:09 +00:00
Sema Checherinda
6b4cb25e25 fixed 01130_in_memory_parts_partitons, remove flushed tmp part from in memory part 2022-11-23 15:16:09 +00:00
Sema Checherinda
cf90692fb8 add more log for NamedSessionsStorage 2022-11-23 15:16:09 +00:00
Sema Checherinda
00e7b4e195 fix race on remove, add creation csn for removed parts 2022-11-23 15:16:09 +00:00
Sema Checherinda
db86cd0f0a fix several critical races, fix tests 2022-11-23 15:16:08 +00:00
Sema Checherinda
15012c7070 address the review comments 2022-11-23 15:16:08 +00:00
Sema Checherinda
fb86019db9 Delete parts over replacing them with empty parts 2022-11-23 15:16:08 +00:00
Alexander Tokmakov
c0133199c8
Revert "Add table_uuid to system.parts" 2022-11-23 16:41:43 +03:00
Antonio Andelic
981e8a3a27 Modifiy visitor 2022-11-23 13:32:36 +00:00
Igor Nikonov
6ef83f455c
Merge pull request #43410 from ClickHouse/igor/fix-distinct-limitby-limit
Fix: limit hint for DISTINCT ignores LIMIT BY
2022-11-23 12:43:15 +01:00
kssenii
6044a9257c Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-23 12:00:55 +01:00
Antonio Andelic
e84e05d849 Modify visitor 2022-11-23 10:27:38 +00:00
Antonio Andelic
84884df595 Substitute UDFs in CREATE queries 2022-11-23 09:28:08 +00:00
Maksim Kita
db1b971643
Merge pull request #43451 from ClickHouse/explain-query-tree-run
Set run_passes to 1 by default
2022-11-23 01:30:49 +03:00
Alexey Milovidov
25780be0c6
Merge pull request #42265 from taofengliu/group_by_all
Support GROUP BY ALL
2022-11-22 20:08:23 +01:00
Alexey Milovidov
c23b8c69db
Merge pull request #42791 from azat/fix-pe-overflow
Fix incorrect UserTimeMicroseconds/SystemTimeMicroseconds accounting
2022-11-22 19:22:51 +01:00
Alexander Tokmakov
2924f3ba7e
Merge branch 'master' into drop-fix 2022-11-22 17:05:12 +03:00
Sergei Trifonov
59f422f10c
Merge branch 'master' into storage-system-moves 2022-11-22 14:46:00 +01:00
Azat Khuzhin
e7d6b9ccca Avoid possible DROP hung due to attached web disk
While dropping the table server scans all disk for existence of data to
remove, but in case of unavailable web disk it will fail, and also not
instantly but after 10 retries (by default, after #42800), that can take
~30 seconds.

But scanning read only disks does not makes any sense, so reorder
conditions to avoid this.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-22 14:44:56 +01:00
Kseniia Sumarokova
392963012a
Merge pull request #43404 from azat/part.log.uuid
Add table_uuid to system.parts
2022-11-22 14:25:34 +01:00
kssenii
894b654f93 Fix bad merge 2022-11-21 21:54:25 +01:00
kssenii
a279bac767 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-21 21:49:34 +01:00
Sergei Trifonov
94a7614c45
Merge branch 'master' into storage-system-moves 2022-11-21 20:10:12 +01:00
Dmitry Novik
239119c499 Set run_passes to 1 by default 2022-11-21 15:05:52 +00:00
Kseniia Sumarokova
a59dac6337
Merge pull request #43143 from azat/disks/s3-check-fix
Add server UUID for disks access checks (read/read-by-offset/write/delete) to avoid possible races
2022-11-21 15:03:46 +01:00
Azat Khuzhin
d0ce500f89 Add table_uuid to system.parts
Can be useful if the table constantly recreated, i.e. in some tests.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-21 12:11:51 +01:00
Antonio Andelic
e78761b66c
Merge branch 'master' into keeper-prometheus 2022-11-21 09:10:10 +01:00
Igor Nikonov
7f8d07e4b7 Fix: limit hint for DISTINCT ignores LIMIT BY
+ test
2022-11-20 23:06:57 +00:00
Azat Khuzhin
44f23c2568 Make disks checks only for clickhouse-server
This will fix clickhouse-disks

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
Antonio Andelic
77eddd702a
Merge branch 'master' into fix-merge-sample-by 2022-11-20 14:38:53 +01:00
taofengliu
a529943d9f Merge remote-tracking branch 'upstream/master' into group_by_all 2022-11-19 21:11:23 +08:00
Azat Khuzhin
ccde92f855 Fix incorrect UserTimeMicroseconds/SystemTimeMicroseconds accounting
After #40732 it became possible that getrusage() (from detachQuery(),
from buildPushingToViewsChain()) will be called for incorrect thread,
and so when the difference will be calculated it will be simply garbage.

But actually the root of this problem is #25714, after which it became
possible to have multiple ThreadStatus for one thread, and this is very
tricky (sigh).

Here are some other thoughts about it:
- Make ThreadStatus nested - decided that complexity does not worth it,
  at least only for this case
- Move some members into ThreadGroupStatus - will break per-thread
  statistics (and hence query_thread_log, BTW does somebody uses it?)
- Move some members into a separate structure

But decided to fix the issue w/o any refactoring, to make easy for
backport.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-18 21:10:43 +01:00
Sergei Trifonov
77100896e3
Merge branch 'master' into storage-system-moves 2022-11-18 16:33:43 +01:00
zzsmdfj
e09092c834 Merge branch 'issue_15357_MaterializeMySQL_support_drop_mulit_table' of github.com:zzsmdfj/ClickHouse into issue_15357_MaterializeMySQL_support_drop_mulit_table 2022-11-18 23:03:48 +08:00
zzsmdfj
47385d259c to MaterializeMySQL_support_drop_mulit_table-fix code-style 2022-11-18 23:03:24 +08:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info
Improve masking sensitive info
2022-11-18 15:37:50 +01:00
zzsmdfj
9239f50a87
Merge branch 'master' into issue_15357_MaterializeMySQL_support_drop_mulit_table 2022-11-18 21:33:31 +08:00
zzsmdfj
91f1df6fc7 to MaterializeMySQL_support_drop_mulit_table 2022-11-18 17:50:44 +08:00
Anton Popov
9569113169
Merge pull request #43269 from CurtizJ/fix-fuzzer-bad-cast
Fix race in `IColumn::dumpStructure`
2022-11-18 01:31:28 +01:00
Vitaly Baranov
86e486d051 Remove unused argument hilite in formatWithSecretHidden(). 2022-11-17 18:16:33 +01:00
Vitaly Baranov
050df6ac7f Move InDepthNodeVisitor.h back to src/Interpreters. 2022-11-17 18:16:32 +01:00
Vitaly Baranov
2b9ed89721 Rename formatWithHiddenSecrets() -> formatWithSecretsHidden(). 2022-11-17 17:30:37 +01:00
kssenii
69d71a8e5f Mask collection values in logs 2022-11-17 16:01:42 +01:00
Nikita Taranov
7beb58b0cf
Optimize merge of uniqExact without_key (#43072)
* impl for uniqExact

* rm unused (read|write)Text methods

* fix style

* small fixes

* impl for variadic uniqExact

* refactor

* fix style

* more agressive inlining

* disable if max_threads=1

* small improvements

* review fixes

* Revert "rm unused (read|write)Text methods"

This reverts commit a7e7480584.

* encapsulate is_able_to_parallelize_merge in Data

* encapsulate is_exact & argument_is_tuple in Data
2022-11-17 13:19:02 +01:00
Antonio Andelic
9af9ef3916 Fix SAMPLE BY with Merge tables 2022-11-17 08:56:51 +00:00
kssenii
f0ce28349d Better 2022-11-16 17:58:21 +01:00
Alexey Milovidov
1ec21e5890
Merge branch 'master' into group_by_all 2022-11-16 06:14:04 +01:00
Anton Popov
11c28abbed fix race in IColumn::dumpStructure 2022-11-16 00:57:22 +00:00
Vitaly Baranov
d365570b27 Hide passwords in SHOW CREATE TABLE and system.tables. 2022-11-16 01:35:11 +01:00
Vitaly Baranov
8d72f75556 Make the password wiping an option of IAST::format(). 2022-11-16 01:35:06 +01:00
kssenii
234f6ee54d Add commands CREATE/ALTER/DROP NAMED COLLECTION 2022-11-15 15:51:56 +01:00
Sema Checherinda
7e73b187cc
Merge pull request #43159 from Algunenano/over-read
Fix several buffer over-reads
2022-11-15 15:04:24 +01:00
Anton Popov
8c66c759d4
Merge pull request #43133 from amosbird/issue_43111
Fix Nullable keys in hyperrectangle
2022-11-15 14:30:13 +01:00
Antonio Andelic
96838f5be7 Merge branch 'master' into keeper-prometheus 2022-11-15 12:00:02 +00:00
Vitaly Baranov
8e99f5fea3 Move maskSensitiveInfoInQueryForLogging() to src/Parsers/ 2022-11-14 18:55:19 +01:00
Vitaly Baranov
e18c97faf7 Remove dependencies maskSensitiveInfo() from Context. 2022-11-14 18:55:19 +01:00
Kruglov Pavel
58b7f6ca04
Merge pull request #42320 from Avogar/improve-schema-inference-insert-select
Improve using structure from insertion table in table functions
2022-11-14 14:51:27 +01:00
Raúl Marín
4988e387f2 Merge remote-tracking branch 'blessed/master' into over-read 2022-11-14 12:28:31 +01:00
Kseniia Sumarokova
b7df9aa3da
Merge pull request #43155 from zzsmdfj/add_oss_function_and_StorageOSS
to add_oss_function_and_StorageOSS
2022-11-14 12:23:57 +01:00
Alexey Milovidov
4012b46a87 Adjustments for Mac OS 2022-11-13 23:11:00 +01:00
Alexey Milovidov
045fd06510 Clarifications 2022-11-13 04:30:43 +01:00
Alexey Milovidov
7aec54b95a Clarifiactions 2022-11-13 04:19:32 +01:00
Alexey Milovidov
c06bff6615 More praise for heroes 2022-11-13 04:12:58 +01:00
Alexey Milovidov
90643aeba1 Clarifiactions 2022-11-13 04:10:20 +01:00
Alexey Milovidov
884603cc5b Clarifiactions 2022-11-13 04:08:17 +01:00
Alexey Milovidov
3096534d6a Clarifiactions 2022-11-13 04:07:29 +01:00
Alexey Milovidov
ba05b7dd2c Add documentation for AsynchronousMetrics 2022-11-13 03:36:20 +01:00
Alexey Milovidov
5a0b7b2947 Add documentation for AsynchronousMetrics 2022-11-13 02:59:01 +01:00
Alexey Milovidov
6599016aec Fix annoying messages in AsynchronousMetrics 2022-11-12 19:09:45 +01:00
Alexey Milovidov
c9d4598962 Fix annoying messages in AsynchronousMetrics 2022-11-12 19:09:37 +01:00
Alexey Milovidov
b8c877db29 Fix annoying messages in AsynchronousMetrics 2022-11-12 19:07:28 +01:00
Antonio Andelic
cbe92bf8c8
Merge branch 'master' into keeper-prometheus 2022-11-11 13:56:27 +01:00
Amos Bird
444ab24792
Fix gtest 2022-11-11 20:12:14 +08:00
Nikolai Kochetov
63d06c8b53
Merge pull request #42829 from ClickHouse/read-in-order-from-query-plan
Read-in-order over query plan
2022-11-11 12:15:20 +01:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
zzsmdfj
3835373644 to add_oss_function_and_StorageOSS 2022-11-11 16:40:10 +08:00
Alexander Tokmakov
e415fc6e95
Revert "Optimize TTL merge, completely expired parts can be removed in time" 2022-11-10 19:38:55 +03:00
Antonio Andelic
94aab4276c Bring back heavy metrics 2022-11-10 12:56:27 +01:00
Vitaly Baranov
c003c90341
Merge pull request #39819 from quickhouse/transparentrowpolicies
Added applied row-level policies to `system.query_log`
2022-11-10 12:47:50 +01:00
Antonio Andelic
89fc3f52f3 Add missing include 2022-11-10 10:41:49 +00:00
Antonio Andelic
f2a9b5933d Merge branch 'master' into keeper-prometheus 2022-11-10 09:28:06 +01:00
Nikolai Kochetov
f2f5c170e4
Merge branch 'master' into read-in-order-from-query-plan 2022-11-09 22:36:36 +01:00
Nikolai Kochetov
997881c7f7 Review fixes. 2022-11-09 16:07:38 +00:00
Antonio Andelic
1ed3930809 Add more metrics for Keeper 2022-11-09 15:51:41 +01:00
Vitaly Baranov
e867f7fc11 Fix build. 2022-11-09 13:51:27 +01:00
Antonio Andelic
c92ae5a385 Add suport for AsyncMetrics in standalone Keeper 2022-11-09 13:46:03 +01:00
Vladimir C
82bf992099
Merge pull request #42869 from zhongyuankai/optimze_ttl_merge 2022-11-09 11:26:46 +01:00