Commit Graph

9915 Commits

Author SHA1 Message Date
helifu
2255b0287a Add 'partitions' field for system.query_log 2023-05-24 20:42:31 +08:00
Igor Nikonov
2f5ed81e0d
Merge branch 'master' into fill_with_by_sorting_prefix_2 2023-05-24 14:40:44 +02:00
vdimir
3f892ceb12
Merge pull request #49816 from bigo-sg/grace_hash_reserve_hash_table 2023-05-24 11:48:19 +02:00
LiuYangkuan
0df4164180 Merge remote-tracking branch 'origin/master' into cluster_discovery 2023-05-24 17:37:01 +08:00
Igor Nikonov
e9c86527b0
Merge branch 'master' into fill_with_by_sorting_prefix_2 2023-05-23 22:58:21 +02:00
Anton Popov
3a955661da
Merge pull request #50123 from CurtizJ/fix-multiif-crash
Fix crash with `multiIf` and constant condition and nullable arguments
2023-05-23 14:29:32 +02:00
SmitaRKulkarni
55af60ea3f
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-23 13:59:15 +02:00
Igor Nikonov
fbcc944d2f Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-23 10:25:38 +00:00
Kseniia Sumarokova
13cd2d6d5f
Merge pull request #50021 from kssenii/fix-logical-error-in-try-reserve
Fix logical error in stress test "Not enough space to add ..."
2023-05-23 12:05:01 +02:00
robot-clickhouse-ci-1
a05088ab73
Merge pull request #50105 from ClickHouse/analyzer-table-function-fix
Analyzer: Do not execute table functions multiple times
2023-05-23 10:16:34 +02:00
robot-ch-test-poll4
945673565c
Merge pull request #50059 from ClickHouse/cache-try-reserver-cleanup
FileCache: simple tryReserve() cleanup
2023-05-23 04:02:37 +02:00
Anton Popov
f8905acb46 fix crash with multiif and constant condition and nullable arguments 2023-05-22 23:31:50 +00:00
Igor Nikonov
4c97ca0a9e Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-22 18:59:58 +00:00
kssenii
295fe3b228 Merge remote-tracking branch 'upstream/master' into abstract-async-prefetched-buffer 2023-05-22 20:01:38 +02:00
kssenii
c4d862a16f Make async reader work with any impl 2023-05-22 19:54:04 +02:00
Dmitry Novik
0ad041e7c3 Remove redundant code 2023-05-22 16:42:14 +00:00
Dmitry Novik
fc10ba871f Analyzer: Do not execute table functions multiple times 2023-05-22 16:36:34 +00:00
Igor Nikonov
48ad2896c7 Remove segments from candidates as soon as handle them 2023-05-22 14:13:37 +00:00
kssenii
6bfbbc94bf A little better 2023-05-22 15:04:27 +02:00
Igor Nikonov
d27b88538d
Fix grammar
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2023-05-22 13:41:50 +02:00
Sergei Trifonov
b1d5ef5440
Merge pull request #49995 from azat/dict/server-memory
Charge only server memory for dictionaries
2023-05-22 13:27:13 +02:00
Alexander Tokmakov
8dbf7beb32
Merge pull request #50015 from azat/ddl-initial-query-id
Preserve initial_query_id for ON CLUSTER queries
2023-05-22 14:03:45 +03:00
kssenii
fa479a870b Merge remote-tracking branch 'upstream/master' into fix-logical-error-in-try-reserve 2023-05-22 13:02:25 +02:00
kssenii
2d0ebba67f Better 2023-05-22 12:58:56 +02:00
Nikolai Kochetov
dc22e90d2d
Merge pull request #49508 from ClickHouse/fix-tracecollector
Fix bug in TraceCollector destructor.
2023-05-22 12:28:17 +02:00
Igor Nikonov
85893b1a0b Clarify dropping removal_candidate flag with comment 2023-05-22 09:49:16 +00:00
Alexander Tokmakov
27e12fb5c9
Merge pull request #50058 from azat/replicated-db-fix-crash
Fix crashing in case of Replicated database without arguments
2023-05-22 12:46:36 +03:00
lgbo-ustc
0573d79ff9 update 2023-05-22 17:45:18 +08:00
lgbo-ustc
f33f1e4840 roll back 2023-05-22 17:45:18 +08:00
lgbo-ustc
4d24b645f0 fixed: alloca new hash table when bytes is oveflow 2023-05-22 17:44:27 +08:00
lgbo-ustc
02b04fd9bf reuse previous hash table's space directly 2023-05-22 17:43:45 +08:00
lgbo-ustc
826aa8021a fixed: unnecessary hash table allocation 2023-05-22 17:42:38 +08:00
lgbo-ustc
8bc4a3b2c0 try to reserve hash table size 2023-05-22 17:42:38 +08:00
vdimir
fa93c388b1
Merge pull request #49483 from bigo-sg/grace_hash_full_join 2023-05-22 11:16:08 +02:00
SmitaRKulkarni
bfb5ab8d73
Merge branch 'master' into parametrized-views-multi-use 2023-05-22 08:56:06 +02:00
lgbo-ustc
35d8388705 update 2023-05-22 10:17:41 +08:00
lgbo-ustc
2582399271 fixed: a in-memory bucket contains rows of other buckets 2023-05-22 10:17:41 +08:00
lgbo-ustc
603c024eb0 ensure only the last processor could access non-joined blocks 2023-05-22 10:17:41 +08:00
lgbo-ustc
29ade23397 fixed: return invalid mismatch rows on full/right join 2023-05-22 10:17:41 +08:00
lgbo-ustc
80af345ea6 update 2023-05-22 10:17:41 +08:00
lgbo-ustc
d5efc0e688 update 2023-05-22 10:17:41 +08:00
lgbo-ustc
8efec9bcca add locks for getNonJoinedBlocks 2023-05-22 10:17:41 +08:00
lgbo-ustc
89dd538bea update 2023-05-22 10:17:41 +08:00
lgbo-ustc
5c44e6a562 triger ci 2023-05-22 10:17:40 +08:00
lgbo-ustc
d89beb1bf7 update tests 2023-05-22 10:17:40 +08:00
lgbo-ustc
7772fed161 update
1. fixed the memoery overflow problem when  handle all delayed buckets parallely
2. resue exists tests
2023-05-22 10:17:40 +08:00
lgbo-ustc
39db0f84d9 add comment 2023-05-22 10:17:40 +08:00
lgbo-ustc
39ff030a6e grace hash join supports right/full join 2023-05-22 10:17:40 +08:00
Igor Nikonov
f5dc07d052 tryReserve() cleanup
simplify removing eviction candidates
2023-05-21 22:01:28 +00:00
Azat Khuzhin
ef06bb8f14 Fix crashing in case of Replicated database without arguments
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-21 23:12:39 +02:00
Azat Khuzhin
66cf16410d Preserve initial_query_id for ON CLUSTER queries
v2: add proper escaping
v3: set distributed_ddl_output_mode=none for test to fix replicated database build
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-21 23:04:54 +02:00
Azat Khuzhin
b6cc504717 Remove Common/OpenTelemetryTraceContext.h from Context.h
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-21 23:04:33 +02:00
Azat Khuzhin
0586a27432 Charge only server memory for dictionaries
Right now the memory will be counted for query/user for dictionary, but
only if it load by user (via SYSTEM RELOAD QUERY or via dictGet()), but
it could be also loaded in backgrounad (due to lifetime, or
update_field, so it is like Buffer, only server memory should be
charged.

v2: mark test as long
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Sergei Trifonov <svtrifonov@gmail.com>
2023-05-21 22:53:52 +02:00
Robert Schulze
2a9ff30a7f
Merge pull request #49380 from azat/dict/hashed-memory
Improve memory usage and speed of SPARSE_HASHED/HASHED dictionaries
2023-05-21 15:46:41 +02:00
Sergei Trifonov
3c002755e2
Merge pull request #50036 from ClickHouse/fix-load-balancing
Load balancing bugfixes
2023-05-21 11:21:55 +02:00
LiuYangkuan
6e80537ab6 support passing fqdn to register cluster node in keeper 2023-05-20 12:41:48 +08:00
Alexey Milovidov
2323542e47
Merge pull request #50022 from ClickHouse/geo-types-production-ready
Geo types are production ready
2023-05-20 02:02:23 +03:00
Sergei Trifonov
14e8132ac4
Merge branch 'master' into fix-load-balancing 2023-05-19 23:05:27 +02:00
serxa
35e77f8e2a fix comment 2023-05-19 17:53:22 +00:00
mateng915
5237dd0245
New system table zookeeper connection (#45245)
* Feature: Support new system table to show which zookeeper node be connected

Description:
============
Currently we have no place to check which zk node be connected otherwise using
lsof command. It not convenient

Solution:
=========
Implemented a new system table, system.zookeeper_host when CK Server has zk
this table will show the zk node dir which connected by current CK server

Noted: This table can support multi-zookeeper cluster scenario.

* fixed review comments

* added test case

* update test cases

* remove unused code

* fixed review comments and removed unused code

* updated test cases for print host, port and is_expired

* modify the code comments

* fixed CI Failed

* fixed code style check failure

* updated test cases by added Tags

* update test reference

* update test cases

* added system.zookeeper_connection doc

* Update docs/en/operations/system-tables/zookeeper_connection.md

* Update docs/en/operations/system-tables/zookeeper_connection.md

* Update docs/en/operations/system-tables/zookeeper_connection.md

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-05-19 17:06:43 +03:00
Sergei Trifonov
5db5f6e44b
Merge branch 'master' into fix-throttlers 2023-05-19 14:08:36 +02:00
Alexey Milovidov
f5506210d6 Geo types are production ready 2023-05-19 12:43:55 +02:00
alesapin
e5b001abda
Merge branch 'master' into fix_some_tests4 2023-05-19 12:34:03 +02:00
kssenii
d6df009842 Fix 2023-05-19 12:22:46 +02:00
Azat Khuzhin
422cbe08fe Do not use PackedHashMap for non-POD for the purposes of layout
In clang-16 the behaviour for POD types had been changed in [1], this
does not allows us to use PackedHashMap for some types.

  [1]: 277123376c

Note, that I tried to come up with a more generic solution then
enumeratic types, but failed. Though now I think that this is good,
since this shows which types are not allowed for PackedHashMap

Another option is to use -fclang-abi-compat=13.0 but I doubt it is a
good idea.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-19 06:07:21 +02:00
Azat Khuzhin
b44497fd4c Introduce PackedHashMap (HashMap with structure without padding)
In case of you have HashMap with <UInt64, UInt16> as <Key, Value> the
overhead of 38% can be crutial, especially if you have tons of keys.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-19 06:07:21 +02:00
Kseniia Sumarokova
adebac1a92
Merge branch 'master' into fix-assertion-in-do-cleanup 2023-05-18 12:22:02 +02:00
SmitaRKulkarni
a91c793684
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-18 09:24:25 +02:00
Kseniia Sumarokova
855c95f626
Update src/Interpreters/Cache/Metadata.cpp
Co-authored-by: Igor Nikonov <954088+devcrafter@users.noreply.github.com>
2023-05-17 22:46:09 +02:00
kssenii
f2dbcb5146 Better fix 2023-05-17 16:27:06 +02:00
Nikita Taranov
0dd67bacf2
Merge branch 'master' into optimize_reading2 2023-05-17 15:06:41 +02:00
Kseniia Sumarokova
edceda494d
Merge branch 'master' into add-more-logging-for-cache 2023-05-17 12:24:59 +02:00
Kseniia Sumarokova
3787b7f127
Update Metadata.cpp 2023-05-17 12:16:18 +02:00
Azat Khuzhin
3c80e30f02 Fix per-query IO/BACKUPs throttling settings (when default profile has them)
When some of this settings was set for default profile (in
users.xml/users.yml), then it will be always used regardless of what
user passed.

Fix this by not inherit per-query throttlers, for this they should be
reset before making query context and they should not be initialized as
before in Context::makeQueryContext(), since makeQueryContext() called
too early, when user settings was not read yet.

But there we had also initialization of per-server throttling, move this
into the ContextSharedPart::configureServerWideThrottling(), and call it
once we have ServerSettings set.

Also note, that this patch makes the following settings - server
settings:
- max_replicated_fetches_network_bandwidth_for_server
- max_replicated_sends_network_bandwidth_for_server
But this change should not affect anybody, since it is done with
compatiblity (i.e. if this setting is set in users profile it will be
read from it as well as a fallback).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-17 12:12:39 +02:00
kssenii
724949927b Add logging 2023-05-16 17:36:48 +02:00
vdimir
1f55c320b4 Fix style 2023-05-16 16:23:53 +02:00
vdimir
ca005ecea1 Update comment about filtering nulls in asof join 2023-05-16 16:23:53 +02:00
vdimir
a7bb8f412f Allow ASOF JOIN over nullable right column 2023-05-16 16:23:53 +02:00
Azat Khuzhin
68138395eb Fix parameterized views when query parameter used multiple times in the query
Example:

    CREATE VIEW view AS
    SELECT *
    FROM system.one
    WHERE dummy = {k1:Int}+1 OR dummy = {k1:Int}+2
                   ^^                    ^^

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-16 15:13:21 +02:00
Alexander Tokmakov
b6716a8f0f Merge branch 'master' into fix_some_tests4 2023-05-16 14:46:27 +02:00
kssenii
d4ea3ea045 Fix 2023-05-16 13:54:13 +02:00
Sergei Trifonov
cbc15bf35a
Add DynamicResourceManager and FairPolicy into scheduling subsystem (#49671)
* Add `DynamicResourceManager` and `FairPolicy` into scheduling subsystem

* fix test

* fix tidy build
2023-05-15 23:13:17 +02:00
Alexander Tokmakov
05ae7b2c2d fix some tests 2023-05-15 18:28:12 +02:00
Igor Nikonov
4d168400ae Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-15 14:40:46 +00:00
Kruglov Pavel
558eda4146
Merge pull request #49412 from azat/block-use-dense-hash-map
Switch Block::NameMap to google::dense_hash_map over HashMap
2023-05-15 12:22:55 +02:00
Igor Nikonov
e743693fb3 Merge remote-tracking branch 'origin/master' into fill_with_by_sorting_prefix_2 2023-05-14 15:56:12 +00:00
Igor Nikonov
93384d6546 Fix suffix generation (see added test in commit 3355894) 2023-05-14 15:52:49 +00:00
Igor Nikonov
629a1a6668 Minor cleanup 2023-05-13 22:11:23 +00:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
Robert Schulze
b9c185af44
Merge pull request #49678 from azat/build/llvm-16
Switch to LLVM/clang 16 (16.0.3)
2023-05-12 13:47:36 +02:00
Nikita Taranov
536c5ac261
Merge branch 'master' into optimize_reading2 2023-05-12 12:59:32 +02:00
Han Fei
07341a04fd
Merge pull request #49666 from hanfei1991/hanfei/regexp-dict-read
Support `dictionary` table function for `RegExpTreeDictionary`
2023-05-12 11:40:46 +02:00
Azat Khuzhin
2c40dd6a4c Switch Block::NameMap to google::dense_hash_map over HashMap
Since HashMap creates 2^8 elements by default, while dense_hash_map
should be good here.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-12 05:52:57 +02:00
Azat Khuzhin
e5c4eb35e8 Merge remote-tracking branch 'u/master' into build/llvm-16
* u/master:
  enable used flags's reinit only when the hash talbe rehash
  Fix build of libfiu on clang-16
  fix flaky test 02504_regexp_dictionary_ua_parser
  fix convertation
  Fix test that expected CH to apply a wrong optimization
  ActionsDAG: remove wrong optimization
2023-05-11 16:32:25 +02:00
SmitaRKulkarni
9bf6175919
Merge pull request #47584 from aiven-sal/aiven-sal/having_pr
ActionsDAG: fix wrong optimization
2023-05-11 14:03:46 +02:00
Han Fei
6b3160f7ce trivial change 2023-05-11 13:49:02 +02:00
lgbo-ustc
a07359fbe8 enable used flags's reinit only when the hash talbe rehash 2023-05-11 11:06:13 +08:00
Igor Nikonov
72f4aed64b Add setting use_with_fill_by_sorting_prefix 2023-05-10 21:08:29 +00:00
Han Fei
2273884ded address comments 2023-05-10 17:42:08 +02:00
Nikita Taranov
ec63a6ee7c
Merge branch 'master' into optimize_reading2 2023-05-10 17:24:37 +02:00