Commit Graph

8103 Commits

Author SHA1 Message Date
zzsmdfj
91f1df6fc7 to MaterializeMySQL_support_drop_mulit_table 2022-11-18 17:50:44 +08:00
Anton Popov
9569113169
Merge pull request #43269 from CurtizJ/fix-fuzzer-bad-cast
Fix race in `IColumn::dumpStructure`
2022-11-18 01:31:28 +01:00
Vitaly Baranov
86e486d051 Remove unused argument hilite in formatWithSecretHidden(). 2022-11-17 18:16:33 +01:00
Vitaly Baranov
050df6ac7f Move InDepthNodeVisitor.h back to src/Interpreters. 2022-11-17 18:16:32 +01:00
Vitaly Baranov
2b9ed89721 Rename formatWithHiddenSecrets() -> formatWithSecretsHidden(). 2022-11-17 17:30:37 +01:00
kssenii
69d71a8e5f Mask collection values in logs 2022-11-17 16:01:42 +01:00
Nikita Taranov
7beb58b0cf
Optimize merge of uniqExact without_key (#43072)
* impl for uniqExact

* rm unused (read|write)Text methods

* fix style

* small fixes

* impl for variadic uniqExact

* refactor

* fix style

* more agressive inlining

* disable if max_threads=1

* small improvements

* review fixes

* Revert "rm unused (read|write)Text methods"

This reverts commit a7e7480584.

* encapsulate is_able_to_parallelize_merge in Data

* encapsulate is_exact & argument_is_tuple in Data
2022-11-17 13:19:02 +01:00
Antonio Andelic
9af9ef3916 Fix SAMPLE BY with Merge tables 2022-11-17 08:56:51 +00:00
kssenii
f0ce28349d Better 2022-11-16 17:58:21 +01:00
Alexey Milovidov
1ec21e5890
Merge branch 'master' into group_by_all 2022-11-16 06:14:04 +01:00
Anton Popov
11c28abbed fix race in IColumn::dumpStructure 2022-11-16 00:57:22 +00:00
Vitaly Baranov
d365570b27 Hide passwords in SHOW CREATE TABLE and system.tables. 2022-11-16 01:35:11 +01:00
Vitaly Baranov
8d72f75556 Make the password wiping an option of IAST::format(). 2022-11-16 01:35:06 +01:00
kssenii
234f6ee54d Add commands CREATE/ALTER/DROP NAMED COLLECTION 2022-11-15 15:51:56 +01:00
Sema Checherinda
7e73b187cc
Merge pull request #43159 from Algunenano/over-read
Fix several buffer over-reads
2022-11-15 15:04:24 +01:00
Anton Popov
8c66c759d4
Merge pull request #43133 from amosbird/issue_43111
Fix Nullable keys in hyperrectangle
2022-11-15 14:30:13 +01:00
Antonio Andelic
96838f5be7 Merge branch 'master' into keeper-prometheus 2022-11-15 12:00:02 +00:00
Vitaly Baranov
8e99f5fea3 Move maskSensitiveInfoInQueryForLogging() to src/Parsers/ 2022-11-14 18:55:19 +01:00
Vitaly Baranov
e18c97faf7 Remove dependencies maskSensitiveInfo() from Context. 2022-11-14 18:55:19 +01:00
Kruglov Pavel
58b7f6ca04
Merge pull request #42320 from Avogar/improve-schema-inference-insert-select
Improve using structure from insertion table in table functions
2022-11-14 14:51:27 +01:00
Raúl Marín
4988e387f2 Merge remote-tracking branch 'blessed/master' into over-read 2022-11-14 12:28:31 +01:00
Kseniia Sumarokova
b7df9aa3da
Merge pull request #43155 from zzsmdfj/add_oss_function_and_StorageOSS
to add_oss_function_and_StorageOSS
2022-11-14 12:23:57 +01:00
Alexey Milovidov
4012b46a87 Adjustments for Mac OS 2022-11-13 23:11:00 +01:00
Alexey Milovidov
045fd06510 Clarifications 2022-11-13 04:30:43 +01:00
Alexey Milovidov
7aec54b95a Clarifiactions 2022-11-13 04:19:32 +01:00
Alexey Milovidov
c06bff6615 More praise for heroes 2022-11-13 04:12:58 +01:00
Alexey Milovidov
90643aeba1 Clarifiactions 2022-11-13 04:10:20 +01:00
Alexey Milovidov
884603cc5b Clarifiactions 2022-11-13 04:08:17 +01:00
Alexey Milovidov
3096534d6a Clarifiactions 2022-11-13 04:07:29 +01:00
Alexey Milovidov
ba05b7dd2c Add documentation for AsynchronousMetrics 2022-11-13 03:36:20 +01:00
Alexey Milovidov
5a0b7b2947 Add documentation for AsynchronousMetrics 2022-11-13 02:59:01 +01:00
Alexey Milovidov
6599016aec Fix annoying messages in AsynchronousMetrics 2022-11-12 19:09:45 +01:00
Alexey Milovidov
c9d4598962 Fix annoying messages in AsynchronousMetrics 2022-11-12 19:09:37 +01:00
Alexey Milovidov
b8c877db29 Fix annoying messages in AsynchronousMetrics 2022-11-12 19:07:28 +01:00
Antonio Andelic
cbe92bf8c8
Merge branch 'master' into keeper-prometheus 2022-11-11 13:56:27 +01:00
Amos Bird
444ab24792
Fix gtest 2022-11-11 20:12:14 +08:00
Nikolai Kochetov
63d06c8b53
Merge pull request #42829 from ClickHouse/read-in-order-from-query-plan
Read-in-order over query plan
2022-11-11 12:15:20 +01:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
zzsmdfj
3835373644 to add_oss_function_and_StorageOSS 2022-11-11 16:40:10 +08:00
Alexander Tokmakov
e415fc6e95
Revert "Optimize TTL merge, completely expired parts can be removed in time" 2022-11-10 19:38:55 +03:00
Antonio Andelic
94aab4276c Bring back heavy metrics 2022-11-10 12:56:27 +01:00
Vitaly Baranov
c003c90341
Merge pull request #39819 from quickhouse/transparentrowpolicies
Added applied row-level policies to `system.query_log`
2022-11-10 12:47:50 +01:00
Antonio Andelic
89fc3f52f3 Add missing include 2022-11-10 10:41:49 +00:00
Antonio Andelic
f2a9b5933d Merge branch 'master' into keeper-prometheus 2022-11-10 09:28:06 +01:00
Nikolai Kochetov
f2f5c170e4
Merge branch 'master' into read-in-order-from-query-plan 2022-11-09 22:36:36 +01:00
Nikolai Kochetov
997881c7f7 Review fixes. 2022-11-09 16:07:38 +00:00
Antonio Andelic
1ed3930809 Add more metrics for Keeper 2022-11-09 15:51:41 +01:00
Vitaly Baranov
e867f7fc11 Fix build. 2022-11-09 13:51:27 +01:00
Antonio Andelic
c92ae5a385 Add suport for AsyncMetrics in standalone Keeper 2022-11-09 13:46:03 +01:00
Vladimir C
82bf992099
Merge pull request #42869 from zhongyuankai/optimze_ttl_merge 2022-11-09 11:26:46 +01:00
SmitaRKulkarni
1c1432c11f
Merge pull request #42827 from ClickHouse/42452_Bug_fix_parent_node_corrupted
42452 Bug fix for parent node corrupted
2022-11-09 08:11:22 +01:00
Sergei Trifonov
8493314137
Merge pull request #43015 from ClickHouse/fix-throttler-init-order
fix bandwidth throttlers initialization order
2022-11-08 12:49:29 +01:00
Robert Schulze
2c568df714
Merge pull request #42968 from ClickHouse/bitcast
Less use of CH-specific bit_cast()
2022-11-08 10:54:10 +01:00
zhongyuankai
37099e57f4
Merge branch 'master' into optimze_ttl_merge 2022-11-08 12:26:46 +08:00
taofengliu
eff3d121e9 Merge remote-tracking branch 'upstream/master' into group_by_all 2022-11-08 10:44:53 +08:00
Sergei Trifonov
6d3b96ee42
Merge branch 'master' into storage-system-moves 2022-11-07 21:31:27 +01:00
serxa
da114e7ab7 fix bandwidth throttlers initialization order 2022-11-07 18:29:00 +00:00
Vitaly Baranov
ab8feb5bff
Merge pull request #42782 from mdonkers/session-log-support-sslcertificate-auth
Add SSL_CERTIFICATE auth method to all places missing
2022-11-07 16:36:42 +01:00
Robert Schulze
cdaf0becfe
Merge branch 'master' into bitcast 2022-11-07 09:24:52 +01:00
zhongyuankai
ade4173f00 Merge branch 'master' into optimze_ttl_merge 2022-11-06 13:40:32 +08:00
taofengliu
1864ed3363 Merge remote-tracking branch 'upstream/master' into group_by_all 2022-11-05 17:22:52 +08:00
Alexey Milovidov
ecd0c4260b
Merge pull request #42929 from Aaaaaaron/minor_fix01
minor fix error msg, replace outdated func name
2022-11-04 19:18:46 +01:00
Nikolay Degterinsky
6995fa1afd
Merge pull request #42883 from zzsmdfj/fix_join_to_subqueries_null_pointer
Fix  join to subqueries rewriter null pointer
2022-11-04 19:56:40 +03:00
Robert Schulze
9c066e964d
Less use of CH-specific bit_cast()
Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e.
when
  sizeof(From) == sizeof(To).
(The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).)

Motivation for this came from #42847 where it is not clear how the
internal bit_cast should behave on big endian systems, so we better
avoid that situation as much as possible.
2022-11-04 15:52:48 +00:00
Vladimir C
f359411668
Merge branch 'master' into optimze_ttl_merge 2022-11-04 16:52:20 +01:00
Nikolai Kochetov
79facdb4f2 Make query plan optimisation respect query settings in EXPLAIN 2022-11-04 15:44:31 +00:00
Vitaly Baranov
52b1f4aed9
Merge pull request #42484 from vitlibar/mask-sensitive-info-in-logs
Mask some information in logs
2022-11-04 14:09:38 +01:00
zhongyuankai
1345f7e73d Merge branch 'master' into optimze_ttl_merge 2022-11-04 20:54:46 +08:00
zhongyuankai
ecf45bb216 fix test 2022-11-04 20:52:21 +08:00
Kruglov Pavel
a7a5a70189
Merge pull request #42814 from kgoralski/failed_metrics_for_async_inserts
Add `FailedAsyncInsertQuery` event metric for async inserts
2022-11-04 13:41:39 +01:00
Vladimir C
9d51119d57
Merge pull request #42905 from ClickHouse/vdimir/cluster-discovery-issue-42763 2022-11-04 11:15:30 +01:00
zhongyuankai
049a593272 Merge branch 'master' into optimze_ttl_merge 2022-11-04 17:22:38 +08:00
taojiatao
721e85a03e minor fix error msg, repalce outdated func name 2022-11-04 11:27:10 +08:00
zzsmdfj
9d8840b8a7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix_join_to_subqueries_null_pointer 2022-11-04 10:05:12 +08:00
zzsmdfj
1a8568473c update code style 2022-11-04 10:04:01 +08:00
Krzysztof Góralski
85d5cbbf19 Review: Failed queries metrics for Async Inserts 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
f019c6b4a1 style check fix 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
e2f35a01de Failed queries metrics for Async Inserts 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
98cc4535f0 Failed queries metrics for Async Inserts 2022-11-03 22:26:24 +01:00
Nikolai Kochetov
51ec95eb82 Try to fix #39157 in a different way. 2022-11-03 20:33:48 +00:00
Nikolai Kochetov
d01ef8ca59
Merge branch 'master' into read-in-order-from-query-plan 2022-11-03 18:59:28 +01:00
Kruglov Pavel
83c5c44b11
Merge branch 'master' into improve-schema-inference-insert-select 2022-11-03 18:41:51 +01:00
Smita Kulkarni
d5b3184e86 Merge branch '42452_Bug_fix_parent_node_corrupted' of github.com:ClickHouse/ClickHouse into 42452_Bug_fix_parent_node_corrupted 2022-11-03 14:43:52 +01:00
Smita Kulkarni
4c70622202 Updated to check size of alias before cloning, so that we throw error early - 42452 Bug fix for parent node corrupted 2022-11-03 14:42:34 +01:00
SmitaRKulkarni
f8ce2b1413
Merge branch 'master' into 42452_Bug_fix_parent_node_corrupted 2022-11-03 14:12:56 +01:00
vdimir
b7fb500d5b Fix incorrect saved_block_sample with multiple disjuncts in hash join 2022-11-03 13:00:37 +01:00
Smita Kulkarni
9a002bd378 Removed finished_asts updation when alias is replaced and updated alias of the replaced ast it its initial name so that we don’t replace it further - 42452 Bug fix for parent node corrupted 2022-11-03 11:57:32 +01:00
SmitaRKulkarni
f26d121f6d
Merge branch 'master' into 42452_Bug_fix_parent_node_corrupted 2022-11-02 22:40:10 +01:00
Smita Kulkarni
816f2319f7 Updated finished_asts to fix fast test fails with Normalise AST is too deep and updated ConvertFunctionOrLikeData to merge conditions by aliasorColumnName instead of ASTPtr - 42452 Bug fix for parent node corrupted 2022-11-02 19:35:22 +01:00
vdimir
503f71b285
Do not set treat_local_port_as_remote in ClusterDiscovery 2022-11-02 15:34:06 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs 2022-11-02 16:26:13 +01:00
zzsmdfj
61123b7f54 to fix join to subqueries rewriter null pointer 2022-11-02 11:34:15 +08:00
Nikolai Kochetov
478d3079c8 Merge branch 'master' into read-in-order-from-query-plan 2022-11-01 21:02:22 +01:00
Nikolai Kochetov
9ffebf4a0c Fixing read-in-order for special storages. 2022-11-01 19:51:52 +00:00
Vitaly Baranov
085fb80190 Spare stack in InDepthNodeVisitor. 2022-11-01 17:20:12 +01:00
zhongyuankai
c1e2b2af74 Optimize TTL merge, completely expired parts can be removed in time 2022-11-01 22:58:15 +08:00
taofengliu
c43dd96f14 Merge remote-tracking branch 'upstream/master' into group_by_all
# Conflicts:
#	src/Analyzer/QueryNode.h
2022-11-01 22:53:44 +08:00
taofengliu
d0767cab1d GROUP BY ALL in Analyzer 2022-11-01 16:18:56 +08:00
taofengliu
5630bc4d44 Merge remote-tracking branch 'upstream/master' into group_by_all 2022-11-01 16:17:04 +08:00
Vitaly Baranov
9d46547daa
Merge branch 'master' into session-log-support-sslcertificate-auth 2022-11-01 08:32:09 +01:00
Vitaly Baranov
d1c69a0753 Avoid changing query if it doesn't contain password. 2022-10-31 17:04:44 +01:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14 2022-10-31 16:15:19 +01:00
Nikolai Kochetov
375db5bd15 Use read-in-order from query plan by default. 2022-10-31 13:54:03 +00:00
SmitaRKulkarni
7e9dd825c0
Merge branch 'master' into 42452_Bug_fix_parent_node_corrupted 2022-10-31 14:03:44 +01:00
Smita Kulkarni
c97c78e3ff Updated normaliser to clone the alias ast.
Implementation:
* Updated QueryNormalizer to clone alias ast, when its replaced. Previously just assigning the same leads to exception in LogicalExpressinsOptimizer as it would be the same parent being inserted again.
* This bug is not seen with new analyser (allow_experimental_analyzer), so no changes for it. I added a test for the same.
Testing:
* Added a test for or function with alias and const where 02475_or_function_alias_and_const_where.sql
2022-10-31 13:49:31 +01:00
Nikolai Kochetov
742d9af397
Merge pull request #42582 from azat/memory-tracker/fixes
Fix frequent memory drift message and clarify things in comments
2022-10-31 11:43:20 +01:00
vdimir
6a4247ca32
Merge branch 'master' into grace_hash_join 2022-10-31 09:54:37 +00:00
vdimir
aa7c97108f
grace join totals fix 2022-10-31 09:53:30 +00:00
Vitaly Baranov
c07fe6921f Add comments. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
854e1e918f Use assert_cast() instead of as() in some places. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
43efbad642 Wipe passwords from distributed queries too. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
a30bfada63 Wipe passwords from backup logs too. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
dcf8724adf Move prepareQueryForLogging() to a separate header. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
c2cc2ccc99 Make as_table_function a child of ASTCreateQuery (to help writing visitors). 2022-10-31 10:50:33 +01:00
Vitaly Baranov
5d2a222fe4 Mask sensitive information in logs. 2022-10-31 10:50:33 +01:00
taofengliu
73a35f1bc4 Merge remote-tracking branch 'upstream/master' into group_by_all 2022-10-29 13:58:27 +08:00
Miel Donkers
fb1623a5f8 Add SSL_CERTIFICATE auth method to all places missing 2022-10-28 21:17:59 +02:00
Alexey Milovidov
f64e632707
Merge pull request #42469 from amosbird/issue_42456
Fix misbehavior of key analysis
2022-10-28 16:18:00 +02:00
taofengliu
82581b73ea fix 2022-10-28 21:52:27 +08:00
taofengliu
7d1fbdbfeb Merge remote-tracking branch 'upstream/master' into group_by_all 2022-10-28 21:48:56 +08:00
taofengliu
56ab264ec9 fix 2022-10-28 21:48:48 +08:00
Nikolai Kochetov
79b30fe133 Merge branch 'master' into read-in-order-from-query-plan 2022-10-28 11:53:51 +00:00
Sergei Trifonov
836595c7ae
Merge branch 'master' into storage-system-moves 2022-10-28 11:39:38 +02:00
Maksim Kita
835daee0f3
Merge pull request #42728 from ucasfl/fix
Fix truncate table does not hold lock correctly
2022-10-28 11:44:24 +03:00
Nikolai Kochetov
5106c24e88 Merge branch 'master' into read-in-order-from-query-plan 2022-10-27 18:12:20 +00:00
vdimir
2eef8c7639
upd gracge hash 2022-10-27 15:25:50 +00:00
vdimir
533c38af27
upd gracge hash 2022-10-27 14:23:55 +00:00
flynn
2b08fe6963 Fix truncate table does not hold lock correctly 2022-10-27 08:56:18 +00:00
Vladimir Chebotaryov
d17b7387f9 Reworked changes to std::shared_ptr<const RowPolicyFilter>. 2022-10-27 10:42:55 +03:00
Vladimir Chebotaryov
6d5d31e49c Removed UUID's. 2022-10-27 10:42:26 +03:00
Vladimir Chebotaryov
0707d88311 Removed extendQueryLogElemImplImpl. 2022-10-27 10:42:25 +03:00
Vladimir Chebotaryov
1ec199b539 Style fix. 2022-10-27 10:42:25 +03:00
Vladimir Chebotaryov
0c485d786c Better includes. 2022-10-27 10:42:24 +03:00
Vladimir Chebotaryov
c28e439c33 Review fixes. 2022-10-27 10:42:22 +03:00
Vladimir Chebotaryov
4cabe1f57c Added applied row-level policies to system.query_log. 2022-10-27 10:41:47 +03:00
vdimir
63dbb9482e
JoinSwitcher don't use grace hash join 2022-10-26 16:09:11 +00:00
Anton Popov
ca4df62e37
Merge branch 'master' into refactor-data-part 2022-10-26 17:49:20 +02:00
Azat Khuzhin
7b69a70e82 Fix frequent memory drift message and clarify things in comments
Somethine like:

    2022.09.28 06:33:34.001433 [ 3133669 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 562.20 MiB, peak 562.21 MiB, will set to 562.20 MiB (RSS), difference: -70.46 MiB
    2022.09.28 06:33:35.001639 [ 3133669 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 562.20 MiB, peak 562.21 MiB, will set to 562.20 MiB (RSS), difference: -70.45 MiB

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-26 17:38:37 +02:00
Azat Khuzhin
2faefc0c09 Add amount of memory used by allocator into logs
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-26 17:38:37 +02:00
vdimir
506bf2d225
Merge branch 'master' into grace_hash_join 2022-10-26 12:25:50 +00:00
Kruglov Pavel
b8ee5600be
Merge pull request #42579 from ucasfl/set
Fix create set with wrong header when data type is lowcardinality
2022-10-26 14:17:14 +02:00
Kruglov Pavel
219553df8d
Merge pull request #42580 from hanfei1991/fei/limit_max_rows
fix behaviour of max_rows_to_read for trival limit queries
2022-10-26 14:09:59 +02:00
vdimir
1ab0c2dc6e
wip grace hash 2022-10-26 09:54:07 +00:00
Maksim Kita
cd2a8966aa
Merge pull request #42649 from kitaisreal/analyzer-change-setting
Analyzer change setting into allow_experimental_analyzer
2022-10-26 12:26:15 +03:00
Sergei Trifonov
34b43f8b4b
Merge branch 'master' into storage-system-moves 2022-10-25 22:56:45 +02:00
serxa
55a86bc2de Add active moves system table 2022-10-25 13:56:46 +00:00
Nikolai Kochetov
c4685f6994
Merge branch 'master' into fix-42575 2022-10-25 15:31:32 +02:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Nikolai Kochetov
5c4444237e
Merge pull request #42617 from ClickHouse/revert-revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3
Revert revert 41268 disable s3 parallel write for part moves to disk s3
2022-10-25 14:29:27 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections (#42541)
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2022-10-25 13:26:29 +02:00