thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 02:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
118,433 Commits 9,328 Branches 1,783 Tags 2.1 GiB
fc19e74ba9
Commit Graph

10166 Commits

Author SHA1 Message Date
Michael Kolupaev
9f80900d6f Changes related to an internal feature 2023-06-05 18:20:38 -07:00
Smita Kulkarni
dedb9067ce WIP : Azure Table Function, added read and StorageAzureSource 2023-06-05 20:36:17 +02:00
Kseniia Sumarokova
0832cb2d7a
Update Metadata.h 2023-06-05 17:51:12 +02:00
Kseniia Sumarokova
2b3db1d33c
Update Metadata.cpp 2023-06-05 17:05:06 +02:00
Dmitry Novik
fe1354f221 Analyzer: Do not apply Query Tree optimizations on shards 2023-06-05 12:32:08 +00:00
kssenii
58932b81cc Merge remote-tracking branch 'upstream/master' into fix-logical-error-in-try-reserve-2 2023-06-05 13:07:12 +02:00
kssenii
f3a8517a44 Fix 2023-06-05 13:07:07 +02:00
Kseniia Sumarokova
22110ac742
Fix exception message 2023-06-05 11:59:49 +02:00
Antonio Andelic
c4873027c3 Merge branch 'master' into keeper-with-disks 2023-06-05 07:08:33 +00:00
Alexey Milovidov
4ef0196a1b
Merge branch 'master' into fix-datetime64-index-scaning 2023-06-05 06:42:35 +03:00
Alexey Milovidov
4b1bb75a07
Merge pull request #50383 from bigo-sg/grace_hash_join_mismatch_right_block 
Without applying `prepareRightBlock` will cause mismatch block structrue
 2023-06-05 05:57:16 +03:00
Amos Bird
a66f68e5df
fix again 2023-06-05 01:48:13 +08:00
Amos Bird
e24c9267bc
fix 2023-06-04 20:06:27 +08:00
kssenii
24e015b961 Fix 2023-06-04 13:34:20 +02:00
kssenii
411cf84db2 Merge remote-tracking branch 'upstream/master' into add-some-assertions-2 2023-06-04 13:32:57 +02:00
Amos Bird
92b2200c55
mutation stages can be empty 2023-06-04 18:32:12 +08:00
Amos Bird
97bd3f0483
Proper mutation of skip indices and projections 2023-06-04 18:32:08 +08:00
kssenii
009fe3d25e Add profile events for eviction 2023-06-03 21:35:27 +02:00
kssenii
05fbebbdac Merge remote-tracking branch 'upstream/master' into add-some-assertions-2 2023-06-03 20:19:14 +02:00
Alexey Milovidov
9e938d7eec
Merge pull request #49287 from nickitat/optimize_reading2 
Optimise hot reading from remote tables
 2023-06-02 23:51:57 +03:00
Nikolai Kochetov
975e58c56d Fix style. 2023-06-02 15:15:44 +00:00
Antonio Andelic
5f625dcdd9 Merge branch 'master' into keeper-with-disks 2023-06-02 13:35:18 +00:00
vdimir
b1f58d765a
Merge pull request #48633 from ClickHouse/vdimir/any_join_single_row 2023-06-02 15:05:13 +02:00
kssenii
8884be2439 Merge remote-tracking branch 'upstream/master' into add-some-assertions-2 2023-06-02 14:58:34 +02:00
kssenii
5cb4363e58 Remove assertion 2023-06-02 14:50:00 +02:00
Nikita Mikhaylov
73db383727
Limit the number of in-flight tasks for loading outdated parts (#50450) 
* Done

* Update programs/local/LocalServer.cpp

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>

* Bump

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-06-02 14:10:26 +02:00
kssenii
79c14c89ee Add some assertions 2023-06-02 12:53:25 +02:00
Nikolai Kochetov
92859ebb3b Fixing more tests. 2023-06-01 21:15:15 +00:00
vdimir
bdb192cf27
Test right join in 02725_any_join_single_row, style code changes 2023-06-01 15:43:37 +00:00
Antonio Andelic
514beb783b Merge branch 'master' into keeper-with-disks 2023-06-01 12:19:43 +00:00
vdimir
62f950ddaf
Keep blocks with nulls for right and full join 2023-06-01 12:03:10 +00:00
vdimir
68d46c81b9
Do not store blocks in hash join if nothing is inseted 2023-06-01 12:03:09 +00:00
Zhiguo Zhou
3d0f336643 Optimize predicate with toYYYYMM converter 
Like toYear(PR #50062), the predicate with toYYYYMM could also be
optimized with its preimage. For example, we could transform the
predicate "toYYYYMM(c) = 202305" to "c >= '2023-05-01' AND c <=
'2023-05-31'", so that the invocation of toYYYYMM is avoided. The
similar transformation also applies to other comparisons.

This commit extends OptimizeDateFilterInPlaceVisitor by having it
recognize the toYYYYMM converter in the WHERE/PREWHERE clauses and
replace it with the above tranformed predicate in-place at the AST
level.
 2023-06-01 10:20:45 +08:00
lgbo-ustc
5dbce62ec3 fixed. without apply prepareRightBlock will cause mismatch block structure 2023-06-01 08:10:28 +08:00
Nikita Mikhaylov
3543d95980
Parallel replicas always skip unavailable ones (#50293) 2023-05-31 22:10:33 +02:00
kssenii
f15769ee6c Fix 2023-05-30 21:29:08 +02:00
Nikolai Kochetov
c740328426 Fix more tests. 2023-05-30 18:51:18 +00:00
Nikolai Kochetov
8cec00dd6e Merge branch 'master' into refactor-subqueries-for-in 2023-05-30 18:08:37 +02:00
Nikolai Kochetov
e91934bcea Fixing style. 2023-05-30 16:02:14 +00:00
Vasily Nemkov
93cf34320c Fixed convertFieldToType case of converting Date and Date32 to DateTime64 Field 
Also added a unit-test that clarifies implementation
 2023-05-30 16:10:52 +02:00
Sema Checherinda
84d6bfe5f6
Merge pull request #50204 from azat/InsertSelectQueries-pe 
Add new profile events for queries with subqueries
 2023-05-30 15:19:04 +02:00
Antonio Andelic
bb2acb50d2
Merge pull request #50186 from LiuYangkuan/support_fqdn_in_cluster_discovery 
support passing fqdn to register cluster node in keeper
 2023-05-30 11:54:09 +02:00
Antonio Andelic
67746bfff2 Merge branch 'master' into keeper-with-disks 2023-05-30 09:50:12 +00:00
SmitaRKulkarni
f2172ab8dd
Merge pull request #46661 from ClickHouse/42192_Lower_parallel_parsing_threads_with_globs 
When reading from multiple files displace parallel parsing
 2023-05-30 08:53:08 +02:00
Lucas Chang
954e76cb7a use getScaleMultiplier 2023-05-30 09:38:50 +08:00
Nikita Taranov
a3652f1727 Merge branch 'master' into optimize_reading2 2023-05-29 15:26:18 +02:00
Robert Schulze
47daaad172
Merge pull request #49778 from helifu/master2 
Support custom 'order by' expression for system log tables
 2023-05-29 10:05:14 +02:00
Alexey Milovidov
9647cfa33d
Merge pull request #50291 from kitaisreal/jit-aggregation-nullable-key-fix 
JIT aggregation nullable key fix
 2023-05-29 02:08:02 +03:00
alesapin
3a3cee586a
Merge pull request #50257 from CurtizJ/useless-refactoring 
Yet another refactoring
 2023-05-28 21:42:16 +02:00
Maksim Kita
e9840bc6e1 JIT aggregation nullable key fix 2023-05-28 21:05:17 +03:00
Nikita Taranov
42a9072888 Merge branch 'master' into optimize_reading2 2023-05-28 18:04:44 +02:00
helifu
6e8136e6f4 Add 'settings' expression for system log tables 2023-05-28 12:59:34 +08:00
helifu
59a815e4e6 Update according to review comments 2023-05-28 12:59:34 +08:00
helifu
57e5a4aa63 Add the 'order_by' description for system log tables 2023-05-28 12:59:34 +08:00
helifu
03e11b6e16 Remove the duplicate pair of parentheses 2023-05-28 12:59:34 +08:00
helifu
90a6437716 Support custom 'order by' expression for system log tables 2023-05-28 12:59:34 +08:00
Lucas Chang
0d873d50a3 format 2023-05-28 10:49:36 +08:00
Lucas Chang
93415789d7 format 2023-05-28 10:42:42 +08:00
Lucas Chang
864ba06029
Merge branch 'master' into fix-datetime64-index-scaning 2023-05-27 20:44:29 +08:00
Lucas Chang
41c599b862 fix date32 2023-05-27 17:59:41 +08:00
Lucas Chang
806b7fc18b fix datetime64 index querying 2023-05-27 17:39:21 +08:00
Nikolai Kochetov
6ed7a3b73f Fixing more tests. 2023-05-26 19:25:33 +00:00
Azat Khuzhin
6b5933a3db Add new profile events for queries with subqueries 
Differs from SelectQuery/InsertQuery that it could be > 1 (in case of
MATERIALIZED VIEW attached to the table), since each local SELECT/INSERT
query is accounted, not only initial.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-26 18:28:40 +02:00
Sergei Trifonov
0d1f2e297b
Unify priorities: lower value means higher priority (#50205) 2023-05-26 15:55:30 +02:00
Anton Popov
f9c7fcbe0a
Merge branch 'master' into useless-refactoring 2023-05-26 13:45:58 +02:00
Antonio Andelic
8a2a63a7bd Merge branch 'master' into keeper-with-disks 2023-05-26 10:48:41 +00:00
Raúl Marín
b3a96de533
Pure parallel replicas: JOIN support (#49544) 2023-05-26 01:04:24 +02:00
Anton Popov
612173e734 refactoring near alter conversions 2023-05-25 22:54:54 +00:00
Nikolai Kochetov
b5b261b22c Merge branch 'master' into refactor-subqueries-for-in 2023-05-25 21:22:06 +02:00
Nikolai Kochetov
7ebe19f5fb Fixing more tests 2023-05-25 19:18:11 +00:00
Nikolai Kochetov
30ff5113d9 Fixing more tests. 2023-05-25 13:33:52 +00:00
Kseniia Sumarokova
f1a3c9cfd5
Merge pull request #50109 from kssenii/abstract-async-prefetched-buffer 
Make async prefetched buffer work with arbitrary impl
 2023-05-25 15:06:44 +02:00
Nikita Mikhaylov
1c3b6738f4
Fixes for parallel replicas (#50195) 2023-05-25 14:41:04 +02:00
Han Fei
55c2dbcc2d
Merge pull request #50062 from ZhiguoZh/20230511-toyear 
Optimize predicate with toYear converter
 2023-05-25 13:25:55 +02:00
vdimir
2df4130d82
Update src/Interpreters/ClusterDiscovery.cpp 2023-05-25 13:06:40 +02:00
Sema Checherinda
23f894b995
Merge pull request #49777 from helifu/master1 
Add 'initial_query_id' field for system.processors_profile_log
 2023-05-25 12:55:32 +02:00
Sema Checherinda
3329a8428d
Merge pull request #49779 from helifu/master3 
Add 'partitions' field for system.query_log
 2023-05-25 12:51:40 +02:00
Igor Nikonov
1c0b02c3c4
Merge pull request #49503 from ClickHouse/fill_with_by_sorting_prefix_2 
WITH FILL by sorting prefix
 2023-05-25 12:37:40 +02:00
何李夫
e4c8c4cecf
Add zookeeper name in endpoint id (#49780) 
* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

Here is some infomation:
1.old table:
  CREATE TABLE a1 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
2.new table:
  CREATE TABLE a2 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('aux1:/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
3.error info:
  <Error> executeQuery: Code: 220. DB::Exception: Duplicate interserver IO endpoint:
          DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (DUPLICATE_INTERSERVER_IO_ENDPOINT)
  <Error> InterserverIOHTTPHandler: Code: 221. DB::Exception: No interserver IO endpoint
          named DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (NO_SUCH_INTERSERVER_IO_ENDPOINT)

* Revert "Add zookeeper name in endpoint id"

This reverts commit 9deb75b249619b7abdd38e3949ca8b3a76c9df8e.

* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

* Fix incompatible with a new setting

* add a test, fix other issues

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

* Update 02735_system_zookeeper_connection.reference

* Update 02735_system_zookeeper_connection.sql

* Update run.sh

* Remove the 'no-fasttest' tag

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-05-25 12:50:14 +03:00
Antonio Andelic
ae84716eba Merge branch 'master' into keeper-with-disks 2023-05-25 08:45:19 +00:00
Zhiguo Zhou
1bc4eb1a6c OptimizeDateFilterVisitor: Revise variable names for clarity 2023-05-25 13:47:03 +08:00
helifu
802b63f2ab Add 'initial_query_id' field for system.processors_profile_log 
Facilitate profile data association and aggregation for the same query
 2023-05-25 09:37:02 +08:00
Zhiguo Zhou
773a5bbbaa Optimize predicate with toYear converter 
The date converters, such as toYear, are widely used in the where
clauses of the SQL queries, however, these conversions are often
expensive due to the complexity of the calendar system.

The function preimage is found an optimization for the predicates
with the converters. Given a predicate, toYear(c) = y, we could
convert it to its equivalent form: c >= b AND c <= e, where b is
"y-01-01" and e is "y-12-31". The similar transformation applies
to other comparisons (<>, <, >, <=, <=).

This commit implemented the above transformation at the AST level
by adding a new pass in the TreeOptimizer and a new AST visitor
for in-place replacing the predicates of toYear with the converted
ones.
 2023-05-25 09:11:51 +08:00
Nikolai Kochetov
d8f39b8df1 Fixing more tests. 2023-05-24 17:53:37 +00:00
helifu
2255b0287a Add 'partitions' field for system.query_log 2023-05-24 20:42:31 +08:00
Igor Nikonov
2f5ed81e0d
Merge branch 'master' into fill_with_by_sorting_prefix_2 2023-05-24 14:40:44 +02:00
vdimir
3f892ceb12
Merge pull request #49816 from bigo-sg/grace_hash_reserve_hash_table 2023-05-24 11:48:19 +02:00
LiuYangkuan
0df4164180 Merge remote-tracking branch 'origin/master' into cluster_discovery 2023-05-24 17:37:01 +08:00
Antonio Andelic
fa3fb49d33 Merge branch 'master' into keeper-with-disks 2023-05-24 09:04:47 +00:00
Antonio Andelic
161afea266 Add support for changelog 2023-05-24 07:43:57 +00:00
alekseygolub
34fe8ba5cd
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-24 09:49:03 +03:00
Igor Nikonov
e9c86527b0
Merge branch 'master' into fill_with_by_sorting_prefix_2 2023-05-23 22:58:21 +02:00
Nikolai Kochetov
6de52e9fce Fixing some tests. 2023-05-23 18:36:02 +00:00
Anton Popov
3a955661da
Merge pull request #50123 from CurtizJ/fix-multiif-crash 
Fix crash with `multiIf` and constant condition and nullable arguments
 2023-05-23 14:29:32 +02:00
SmitaRKulkarni
55af60ea3f
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-23 13:59:15 +02:00
Igor Nikonov
fbcc944d2f Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-23 10:25:38 +00:00
Kseniia Sumarokova
13cd2d6d5f
Merge pull request #50021 from kssenii/fix-logical-error-in-try-reserve 
Fix logical error in stress test "Not enough space to add ..."
 2023-05-23 12:05:01 +02:00
robot-clickhouse-ci-1
a05088ab73
Merge pull request #50105 from ClickHouse/analyzer-table-function-fix 
Analyzer: Do not execute table functions multiple times
 2023-05-23 10:16:34 +02:00
robot-ch-test-poll4
945673565c
Merge pull request #50059 from ClickHouse/cache-try-reserver-cleanup 
FileCache: simple tryReserve() cleanup
 2023-05-23 04:02:37 +02:00
Anton Popov
f8905acb46 fix crash with multiif and constant condition and nullable arguments 2023-05-22 23:31:50 +00:00
Igor Nikonov
4c97ca0a9e Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-22 18:59:58 +00:00
kssenii
295fe3b228 Merge remote-tracking branch 'upstream/master' into abstract-async-prefetched-buffer 2023-05-22 20:01:38 +02:00
kssenii
c4d862a16f Make async reader work with any impl 2023-05-22 19:54:04 +02:00
Dmitry Novik
0ad041e7c3 Remove redundant code 2023-05-22 16:42:14 +00:00
Dmitry Novik
fc10ba871f Analyzer: Do not execute table functions multiple times 2023-05-22 16:36:34 +00:00
Igor Nikonov
48ad2896c7 Remove segments from candidates as soon as handle them 2023-05-22 14:13:37 +00:00
kssenii
6bfbbc94bf A little better 2023-05-22 15:04:27 +02:00
Antonio Andelic
bde2cf96b1 Better 2023-05-22 12:24:16 +00:00
Igor Nikonov
d27b88538d
Fix grammar 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2023-05-22 13:41:50 +02:00
Sergei Trifonov
b1d5ef5440
Merge pull request #49995 from azat/dict/server-memory 
Charge only server memory for dictionaries
 2023-05-22 13:27:13 +02:00
Nikolai Kochetov
ff6400f91a Merge branch 'master' into refactor-subqueries-for-in 2023-05-22 13:07:51 +02:00
Alexander Tokmakov
8dbf7beb32
Merge pull request #50015 from azat/ddl-initial-query-id 
Preserve initial_query_id for ON CLUSTER queries
 2023-05-22 14:03:45 +03:00
kssenii
fa479a870b Merge remote-tracking branch 'upstream/master' into fix-logical-error-in-try-reserve 2023-05-22 13:02:25 +02:00
kssenii
2d0ebba67f Better 2023-05-22 12:58:56 +02:00
Nikolai Kochetov
dc22e90d2d
Merge pull request #49508 from ClickHouse/fix-tracecollector 
Fix bug in TraceCollector destructor.
 2023-05-22 12:28:17 +02:00
Igor Nikonov
85893b1a0b Clarify dropping removal_candidate flag with comment 2023-05-22 09:49:16 +00:00
Alexander Tokmakov
27e12fb5c9
Merge pull request #50058 from azat/replicated-db-fix-crash 
Fix crashing in case of Replicated database without arguments
 2023-05-22 12:46:36 +03:00
lgbo-ustc
0573d79ff9 update 2023-05-22 17:45:18 +08:00
lgbo-ustc
f33f1e4840 roll back 2023-05-22 17:45:18 +08:00
lgbo-ustc
4d24b645f0 fixed: alloca new hash table when bytes is oveflow 2023-05-22 17:44:27 +08:00
lgbo-ustc
02b04fd9bf reuse previous hash table's space directly 2023-05-22 17:43:45 +08:00
lgbo-ustc
826aa8021a fixed: unnecessary hash table allocation 2023-05-22 17:42:38 +08:00
lgbo-ustc
8bc4a3b2c0 try to reserve hash table size 2023-05-22 17:42:38 +08:00
vdimir
fa93c388b1
Merge pull request #49483 from bigo-sg/grace_hash_full_join 2023-05-22 11:16:08 +02:00
Antonio Andelic
dc31396035 Merge branch 'master' into keeper-with-disks 2023-05-22 08:10:24 +00:00
SmitaRKulkarni
bfb5ab8d73
Merge branch 'master' into parametrized-views-multi-use 2023-05-22 08:56:06 +02:00
lgbo-ustc
35d8388705 update 2023-05-22 10:17:41 +08:00
lgbo-ustc
2582399271 fixed: a in-memory bucket contains rows of other buckets 2023-05-22 10:17:41 +08:00
lgbo-ustc
603c024eb0 ensure only the last processor could access non-joined blocks 2023-05-22 10:17:41 +08:00
lgbo-ustc
29ade23397 fixed: return invalid mismatch rows on full/right join 2023-05-22 10:17:41 +08:00
lgbo-ustc
80af345ea6 update 2023-05-22 10:17:41 +08:00
lgbo-ustc
d5efc0e688 update 2023-05-22 10:17:41 +08:00
lgbo-ustc
8efec9bcca add locks for getNonJoinedBlocks 2023-05-22 10:17:41 +08:00
lgbo-ustc
89dd538bea update 2023-05-22 10:17:41 +08:00
lgbo-ustc
5c44e6a562 triger ci 2023-05-22 10:17:40 +08:00
lgbo-ustc
d89beb1bf7 update tests 2023-05-22 10:17:40 +08:00
lgbo-ustc
7772fed161 update 
1. fixed the memoery overflow problem when  handle all delayed buckets parallely
2. resue exists tests
 2023-05-22 10:17:40 +08:00
lgbo-ustc
39db0f84d9 add comment 2023-05-22 10:17:40 +08:00
lgbo-ustc
39ff030a6e grace hash join supports right/full join 2023-05-22 10:17:40 +08:00
Igor Nikonov
f5dc07d052 tryReserve() cleanup 
simplify removing eviction candidates
 2023-05-21 22:01:28 +00:00
Azat Khuzhin
ef06bb8f14 Fix crashing in case of Replicated database without arguments 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-21 23:12:39 +02:00
Azat Khuzhin
66cf16410d Preserve initial_query_id for ON CLUSTER queries 
v2: add proper escaping
v3: set distributed_ddl_output_mode=none for test to fix replicated database build
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-21 23:04:54 +02:00
Azat Khuzhin
b6cc504717 Remove Common/OpenTelemetryTraceContext.h from Context.h 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-21 23:04:33 +02:00
Azat Khuzhin
0586a27432 Charge only server memory for dictionaries 
Right now the memory will be counted for query/user for dictionary, but
only if it load by user (via SYSTEM RELOAD QUERY or via dictGet()), but
it could be also loaded in backgrounad (due to lifetime, or
update_field, so it is like Buffer, only server memory should be
charged.

v2: mark test as long
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Sergei Trifonov <svtrifonov@gmail.com>
 2023-05-21 22:53:52 +02:00
alekseygolub
70b9077bb2
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-21 17:49:29 +03:00
Robert Schulze
2a9ff30a7f
Merge pull request #49380 from azat/dict/hashed-memory 
Improve memory usage and speed of SPARSE_HASHED/HASHED dictionaries
 2023-05-21 15:46:41 +02:00
Sergei Trifonov
3c002755e2
Merge pull request #50036 from ClickHouse/fix-load-balancing 
Load balancing bugfixes
 2023-05-21 11:21:55 +02:00
LiuYangkuan
6e80537ab6 support passing fqdn to register cluster node in keeper 2023-05-20 12:41:48 +08:00
Alexey Milovidov
2323542e47
Merge pull request #50022 from ClickHouse/geo-types-production-ready 
Geo types are production ready
 2023-05-20 02:02:23 +03:00
Sergei Trifonov
14e8132ac4
Merge branch 'master' into fix-load-balancing 2023-05-19 23:05:27 +02:00
serxa
35e77f8e2a fix comment 2023-05-19 17:53:22 +00:00
mateng915
5237dd0245
New system table zookeeper connection (#45245) 
* Feature: Support new system table to show which zookeeper node be connected

Description:
============
Currently we have no place to check which zk node be connected otherwise using
lsof command. It not convenient

Solution:
=========
Implemented a new system table, system.zookeeper_host when CK Server has zk
this table will show the zk node dir which connected by current CK server

Noted: This table can support multi-zookeeper cluster scenario.

* fixed review comments

* added test case

* update test cases

* remove unused code

* fixed review comments and removed unused code

* updated test cases for print host, port and is_expired

* modify the code comments

* fixed CI Failed

* fixed code style check failure

* updated test cases by added Tags

* update test reference

* update test cases

* added system.zookeeper_connection doc

* Update docs/en/operations/system-tables/zookeeper_connection.md

* Update docs/en/operations/system-tables/zookeeper_connection.md

* Update docs/en/operations/system-tables/zookeeper_connection.md

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-05-19 17:06:43 +03:00
Sergei Trifonov
5db5f6e44b
Merge branch 'master' into fix-throttlers 2023-05-19 14:08:36 +02:00
Alexey Milovidov
f5506210d6 Geo types are production ready 2023-05-19 12:43:55 +02:00
alesapin
e5b001abda
Merge branch 'master' into fix_some_tests4 2023-05-19 12:34:03 +02:00
kssenii
d6df009842 Fix 2023-05-19 12:22:46 +02:00
Azat Khuzhin
422cbe08fe Do not use PackedHashMap for non-POD for the purposes of layout 
In clang-16 the behaviour for POD types had been changed in [1], this
does not allows us to use PackedHashMap for some types.

  [1]: 277123376c

Note, that I tried to come up with a more generic solution then
enumeratic types, but failed. Though now I think that this is good,
since this shows which types are not allowed for PackedHashMap

Another option is to use -fclang-abi-compat=13.0 but I doubt it is a
good idea.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-19 06:07:21 +02:00
Azat Khuzhin
b44497fd4c Introduce PackedHashMap (HashMap with structure without padding) 
In case of you have HashMap with <UInt64, UInt16> as <Key, Value> the
overhead of 38% can be crutial, especially if you have tons of keys.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-19 06:07:21 +02:00
Kseniia Sumarokova
adebac1a92
Merge branch 'master' into fix-assertion-in-do-cleanup 2023-05-18 12:22:02 +02:00
Antonio Andelic
bf6afd27a6 build with disks 2023-05-18 07:39:22 +00:00
SmitaRKulkarni
a91c793684
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-18 09:24:25 +02:00
Kseniia Sumarokova
855c95f626
Update src/Interpreters/Cache/Metadata.cpp 
Co-authored-by: Igor Nikonov <954088+devcrafter@users.noreply.github.com>
 2023-05-17 22:46:09 +02:00
kssenii
f2dbcb5146 Better fix 2023-05-17 16:27:06 +02:00
Nikita Taranov
0dd67bacf2
Merge branch 'master' into optimize_reading2 2023-05-17 15:06:41 +02:00
Kseniia Sumarokova
edceda494d
Merge branch 'master' into add-more-logging-for-cache 2023-05-17 12:24:59 +02:00
Kseniia Sumarokova
3787b7f127
Update Metadata.cpp 2023-05-17 12:16:18 +02:00
Azat Khuzhin
3c80e30f02 Fix per-query IO/BACKUPs throttling settings (when default profile has them) 
When some of this settings was set for default profile (in
users.xml/users.yml), then it will be always used regardless of what
user passed.

Fix this by not inherit per-query throttlers, for this they should be
reset before making query context and they should not be initialized as
before in Context::makeQueryContext(), since makeQueryContext() called
too early, when user settings was not read yet.

But there we had also initialization of per-server throttling, move this
into the ContextSharedPart::configureServerWideThrottling(), and call it
once we have ServerSettings set.

Also note, that this patch makes the following settings - server
settings:
- max_replicated_fetches_network_bandwidth_for_server
- max_replicated_sends_network_bandwidth_for_server
But this change should not affect anybody, since it is done with
compatiblity (i.e. if this setting is set in users profile it will be
read from it as well as a fallback).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-17 12:12:39 +02:00
kssenii
724949927b Add logging 2023-05-16 17:36:48 +02:00
vdimir
1f55c320b4 Fix style 2023-05-16 16:23:53 +02:00
vdimir
ca005ecea1 Update comment about filtering nulls in asof join 2023-05-16 16:23:53 +02:00
vdimir
a7bb8f412f Allow ASOF JOIN over nullable right column 2023-05-16 16:23:53 +02:00
Azat Khuzhin
68138395eb Fix parameterized views when query parameter used multiple times in the query 
Example:

    CREATE VIEW view AS
    SELECT *
    FROM system.one
    WHERE dummy = {k1:Int}+1 OR dummy = {k1:Int}+2
                   ^^                    ^^

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-16 15:13:21 +02:00
Alexander Tokmakov
b6716a8f0f Merge branch 'master' into fix_some_tests4 2023-05-16 14:46:27 +02:00
kssenii
d4ea3ea045 Fix 2023-05-16 13:54:13 +02:00
Sergei Trifonov
cbc15bf35a
Add DynamicResourceManager and FairPolicy into scheduling subsystem (#49671) 
* Add `DynamicResourceManager` and `FairPolicy` into scheduling subsystem

* fix test

* fix tidy build
 2023-05-15 23:13:17 +02:00
Alexander Tokmakov
05ae7b2c2d fix some tests 2023-05-15 18:28:12 +02:00
Igor Nikonov
4d168400ae Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-15 14:40:46 +00:00
Kruglov Pavel
558eda4146
Merge pull request #49412 from azat/block-use-dense-hash-map 
Switch Block::NameMap to google::dense_hash_map over HashMap
 2023-05-15 12:22:55 +02:00
Igor Nikonov
e743693fb3 Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-14 15:56:12 +00:00
Igor Nikonov
93384d6546 Fix suffix generation (see added test in commit 3355894) 2023-05-14 15:52:49 +00:00
Igor Nikonov
629a1a6668 Minor cleanup 2023-05-13 22:11:23 +00:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
Robert Schulze
b9c185af44
Merge pull request #49678 from azat/build/llvm-16 
Switch to LLVM/clang 16 (16.0.3)
 2023-05-12 13:47:36 +02:00
Nikita Taranov
536c5ac261
Merge branch 'master' into optimize_reading2 2023-05-12 12:59:32 +02:00
Han Fei
07341a04fd
Merge pull request #49666 from hanfei1991/hanfei/regexp-dict-read 
Support `dictionary` table function for `RegExpTreeDictionary`
 2023-05-12 11:40:46 +02:00
Azat Khuzhin
2c40dd6a4c Switch Block::NameMap to google::dense_hash_map over HashMap 
Since HashMap creates 2^8 elements by default, while dense_hash_map
should be good here.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-12 05:52:57 +02:00
Azat Khuzhin
e5c4eb35e8 Merge remote-tracking branch 'u/master' into build/llvm-16 
* u/master:
  enable used flags's reinit only when the hash talbe rehash
  Fix build of libfiu on clang-16
  fix flaky test 02504_regexp_dictionary_ua_parser
  fix convertation
  Fix test that expected CH to apply a wrong optimization
  ActionsDAG: remove wrong optimization
 2023-05-11 16:32:25 +02:00
SmitaRKulkarni
9bf6175919
Merge pull request #47584 from aiven-sal/aiven-sal/having_pr 
ActionsDAG: fix wrong optimization
 2023-05-11 14:03:46 +02:00
Han Fei
6b3160f7ce trivial change 2023-05-11 13:49:02 +02:00
lgbo-ustc
a07359fbe8 enable used flags's reinit only when the hash talbe rehash 2023-05-11 11:06:13 +08:00
Igor Nikonov
72f4aed64b Add setting use_with_fill_by_sorting_prefix 2023-05-10 21:08:29 +00:00
Han Fei
2273884ded address comments 2023-05-10 17:42:08 +02:00
Nikita Taranov
ec63a6ee7c
Merge branch 'master' into optimize_reading2 2023-05-10 17:24:37 +02:00
Igor Nikonov
49971276d3
Merge branch 'master' into fill_with_by_sorting_prefix_2 2023-05-10 16:29:28 +02:00
Azat Khuzhin
a0cc5b7e47 Merge remote-tracking branch 'u/master' into build/llvm-16 2023-05-10 16:28:36 +02:00
Han Fei
d17aa828b3
Merge pull request #49361 from hanfei1991/hanfei/fiu 
support named fault injection and add a simple test
 2023-05-10 15:51:51 +02:00
Igor Nikonov
a0ef2e272f Fix suffix generation for empty result set 2023-05-10 12:41:33 +00:00
Kseniia Sumarokova
1ea53b5885
Merge pull request #49690 from kssenii/add-logging-for-try-reserve 
Add some logging
 2023-05-10 12:59:56 +02:00
Igor Nikonov
37bc0becc9 Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-10 10:53:10 +00:00
SmitaRKulkarni
5c030c428c
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-10 09:51:38 +02:00
robot-clickhouse-ci-2
c698d2af99
Merge pull request #49702 from ClickHouse/nickitat-patch-11 
Enable `do_not_evict_index_and_mark_files` by default
 2023-05-10 02:26:38 +02:00
Anton Popov
862008da0d
Merge pull request #49663 from CurtizJ/fix-race-context-copy 
Fix race in `Context::createCopy`
 2023-05-10 02:11:35 +02:00
kssenii
e35c669892 Fix 2023-05-09 22:51:47 +02:00
Yakov Olkhovskiy
815cc8de9b removed unnecessary check for thread's joinability 2023-05-09 19:43:39 +00:00
Azat Khuzhin
d03ae2abfa Fix modernize-loop-convert clang-tidy warning 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 21:19:37 +02:00
kssenii
c20aa0786d More logging 2023-05-09 19:29:57 +02:00
kssenii
13b77f1dac Fix 2023-05-09 19:13:39 +02:00
Anton Popov
d88224dc43 fix build 2023-05-09 16:30:43 +00:00
Nikita Taranov
7cd6db3fb3
Enable do_not_evict_index_and_mark_files by default 2023-05-09 18:04:47 +02:00
kssenii
75aa775d2b Add logging 2023-05-09 15:58:20 +02:00
Kseniia Sumarokova
abde991cc4
Merge pull request #49675 from ClickHouse/cache-cleanup-after-locking-rework 
Some cache cleanup after rework locking
 2023-05-09 11:29:45 +02:00
Alexey Milovidov
3556c26075
Merge pull request #49434 from ClickHouse/ins 
Make INSERT do more things in parallel to avoid getting bottlenecked on one thread
 2023-05-09 08:10:15 +03:00
Alexey Milovidov
a8bdb20fc4
Merge pull request #48787 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks [Resubmit]
 2023-05-09 07:58:36 +03:00
Alexey Milovidov
dda6e9175c
Merge pull request #49647 from ClickHouse/optimize-opentelemetry-span-log 
Optimize `opentelemetry_span_log`
 2023-05-09 06:40:49 +03:00
Han Fei
64b67b9a37 Merge branch 'master' into hanfei/fiu 2023-05-09 01:14:30 +02:00
Yakov Olkhovskiy
70516f2f45 close pipe in run() if exception happens 2023-05-08 23:00:20 +00:00
Han Fei
82ce47ce56 address comments 2023-05-08 23:02:36 +02:00
Igor Nikonov
903021e896 Fix review comment 2023-05-08 18:29:47 +00:00
Igor Nikonov
e7fff74797 Cleanup 2023-05-08 17:45:42 +00:00
Han Fei
71c96ceb61
Update src/Interpreters/ExternalDictionariesLoader.h 2023-05-08 16:21:42 +02:00
Han Fei
72fc567d4a
Merge branch 'master' into hanfei/regexp-dict-read 2023-05-08 16:20:12 +02:00
Han Fei
92e57817a2 Support dictionary table function for RegExpTreeDictionary 2023-05-08 16:14:08 +02:00
Anton Popov
713ffb3bda fix race in Context::createCopy 2023-05-08 13:22:31 +00:00
Yakov Olkhovskiy
1e3cd4ed6d close pipe before join 2023-05-08 12:39:01 +00:00
Kseniia Sumarokova
336bb41c5d
Merge branch 'master' into remove-dependency-from-context 2023-05-08 12:46:10 +02:00
Alexey Milovidov
baf162b26b Optimize opentelemetry_span_log 2023-05-08 05:15:28 +02:00
Alexey Milovidov
647472ed46
Merge branch 'master' into aiven-sal/having_pr 2023-05-08 05:53:38 +03:00
Alexey Milovidov
e8228cb0d2
Merge pull request #49530 from ClickHouse/optimize-query-log 
Optimize the system.query_log and system.query_thread_log tables by applying LowCardinality when appropriate
 2023-05-07 03:36:30 +03:00
robot-clickhouse-ci-1
d1ad3ea24e
Merge pull request #49593 from ClickHouse/rs/qc-empty-chunk 
Query Cache: Safeguard against empty chunks
 2023-05-07 02:20:54 +02:00
Igor Nikonov
6e9c519195 Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-06 19:33:26 +00:00
kssenii
2228107134 Better name 2023-05-06 20:12:26 +02:00
Sema Checherinda
2104baabce
Merge pull request #49527 from ClickHouse/thread-detach-fix 
Fix thread MemoryTracker parent after detach
 2023-05-06 19:11:55 +02:00
Alexey Milovidov
54163b87ec
Merge pull request #47977 from ClickHouse/rs/non-experimental-qc 
Mark query cache as non-experimental
 2023-05-06 19:49:23 +03:00
Alexey Milovidov
62d8677773
Update QueryCache.cpp 2023-05-06 18:25:43 +03:00
Alexey Milovidov
2d39acf2e1
Merge pull request #49554 from kssenii/fix-assertions-3 
Fix stress test (assertion 'key_metadata.lock()')
 2023-05-06 18:16:40 +03:00
Robert Schulze
883e0abea1
Query Cache: Safeguard against empty chunks 
Reading from the query cache is implemented using processor
"SourceFromChunks" which inherits from ISource. The latter has logic
which finishes processing (= calls .finish() on the output port +
returns Status::Finished) when the derived class returns an empty chunk.
If this empty chunk is not the last chunk, i.e. if it is followed by
non-empty chunks, the query result will be incorrect. This situation
should theoretically never occur in practice but who knows ... To be on
the safe side, writing into the query cache now rejects empty chunks and
thereby avoids this scenario.
 2023-05-06 14:22:23 +00:00
Robert Schulze
0598bfd79e
Mark query cache as non-experimental 2023-05-06 13:58:40 +00:00
Robert Schulze
983e37048c
Merge pull request #48853 from ClickHouse/rs/qc-totals-extremes2 
QueryCache: Support queries with totals and extremes
 2023-05-06 15:41:10 +02:00
Robert Schulze
7137c8811a
Merge pull request #49158 from ClickHouse/rs/show-indexes 
Implement SHOW INDEX
 2023-05-06 15:30:29 +02:00
Robert Schulze
39bd320e37
Merge pull request #49573 from ClickHouse/rs/qc-no-internal-queries 
Query cache: disable for internal queries
 2023-05-06 15:27:44 +02:00
Han Fei
d2bd0bcd59
Merge branch 'master' into patch-1 2023-05-05 23:39:08 +02:00
Robert Schulze
3dfc0bd265
Merge pull request #49413 from azat/build/headers 
Slightly reduce inter-header dependencies
 2023-05-05 23:37:58 +02:00
Alexey Milovidov
a8c410c21a Merge branch 'master' into optimize-query-log 2023-05-05 23:21:01 +02:00
Nikolai Kochetov
b2c36fc3e5 Fixing style. 2023-05-05 20:51:38 +00:00
Robert Schulze
9cd81400f3
Query cache: disable for internal queries 2023-05-05 19:22:14 +00:00
Nikolai Kochetov
618486815b Merge branch 'master' into refactor-subqueries-for-in 2023-05-05 20:39:09 +02:00
Nikolai Kochetov
54ef6769f1 Refactor PreparedSets [4] 2023-05-05 18:30:08 +00:00
Robert Schulze
bba85c9c25
Improve docs 2023-05-05 18:11:33 +00:00
Robert Schulze
945a5130cf
Simplify simplify simplify 2023-05-05 16:28:51 +00:00
Robert Schulze
aba5db40a1
Merge remote-tracking branch 'origin/master' into rs/qc-totals-extremes2 2023-05-05 14:11:10 +00:00