Commit Graph

115879 Commits

Author SHA1 Message Date
Raúl Marín
b3a96de533
Pure parallel replicas: JOIN support (#49544) 2023-05-26 01:04:24 +02:00
Anton Popov
612173e734 refactoring near alter conversions 2023-05-25 22:54:54 +00:00
Denny Crane
6b98e02857
Update settings.md 2023-05-25 19:21:32 -03:00
serxa
4f0aeee236 fix more conflicts 2023-05-25 20:50:22 +00:00
johanngan
de3b08aa5b Clean up regexp tree dictionary documentation
dictGetOrNull() relies on IDictionary::hasKeys(), which
RegExpTreeDictionary doesn't implement, so this probably never worked.
If you try to use it, an exception is thrown. The docs shouldn't
indicate that this is supported.

Also fix a markdown hyperlink in the docs.
2023-05-25 14:35:24 -05:00
Stanislav Dobrovolschii
8c4f0c763e Update SAMPLE clause documentation links 2023-05-25 18:52:38 +00:00
Robert Schulze
231d52d0e6
Document system.build_options 2023-05-25 18:29:15 +00:00
Robert Schulze
c4f91a1c45
Merge branch 'master' into space 2023-05-25 19:56:20 +02:00
Robert Schulze
eca08438f4
Fix macos build 2023-05-25 17:05:18 +00:00
pufit
a014b4d06f
Merge pull request #50241 from ClickHouse/cherry-pick-url-fix
Fix URL in backport comment
2023-05-25 12:04:40 -04:00
pufit
4ce722854b Fix URL in backport comment 2023-05-25 11:51:47 -04:00
alesapin
e4c1e2f232 Fix build while it's not failing locally 2023-05-25 17:37:09 +02:00
alesapin
e94b0c8e5e Fix bug 2023-05-25 16:38:19 +02:00
Nikita Taranov
243207aa4e
Update S3 sdk to v1.11.61 (#50037) 2023-05-25 16:00:50 +02:00
kssenii
3fefacbf20 Fix 2023-05-25 15:48:56 +02:00
serxa
0ca526c603 Unify priorities: rework IO scheduling subsystem 2023-05-25 13:25:41 +00:00
serxa
3ef6cb7bdc git-apply #50205 2023-05-25 13:24:45 +00:00
Kseniia Sumarokova
f1a3c9cfd5
Merge pull request #50109 from kssenii/abstract-async-prefetched-buffer
Make async prefetched buffer work with arbitrary impl
2023-05-25 15:06:44 +02:00
serxa
b8d3e495e5 add pool_id out-of-bound checks 2023-05-25 12:42:19 +00:00
Nikita Mikhaylov
1c3b6738f4
Fixes for parallel replicas (#50195) 2023-05-25 14:41:04 +02:00
serxa
e3ce2f834a fix style 2023-05-25 12:35:00 +00:00
Sergei Trifonov
2df22d396d
Merge branch 'master' into async-loader-workloads 2023-05-25 14:32:12 +02:00
Mikhail f. Shiryaev
5c59d9a0ed
Merge pull request #50164 from ClickHouse/optimize-style-check
Speed-up the shellcheck with parallel xargs
2023-05-25 14:02:40 +02:00
Robert Schulze
8804dfd4b0
Fix resizing 2023-05-25 11:57:09 +00:00
Sergei Trifonov
78c89da8bb
Update src/Common/AsyncLoader.cpp
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2023-05-25 13:48:03 +02:00
Sergei Trifonov
32cb9931b6
Update src/Common/AsyncLoader.h
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2023-05-25 13:47:56 +02:00
Han Fei
55c2dbcc2d
Merge pull request #50062 from ZhiguoZh/20230511-toyear
Optimize predicate with toYear converter
2023-05-25 13:25:55 +02:00
avogar
ce99825200 Fix skipping spaces at end of row in CustomSeparatedIgnoreSpaces format 2023-05-25 11:19:15 +00:00
vdimir
2df4130d82
Update src/Interpreters/ClusterDiscovery.cpp 2023-05-25 13:06:40 +02:00
alesapin
5b76ab4e03 Fix build 2023-05-25 13:02:40 +02:00
Sema Checherinda
23f894b995
Merge pull request #49777 from helifu/master1
Add 'initial_query_id' field for system.processors_profile_log
2023-05-25 12:55:32 +02:00
Sema Checherinda
3329a8428d
Merge pull request #49779 from helifu/master3
Add 'partitions' field for system.query_log
2023-05-25 12:51:40 +02:00
Igor Nikonov
1c0b02c3c4
Merge pull request #49503 from ClickHouse/fill_with_by_sorting_prefix_2
WITH FILL by sorting prefix
2023-05-25 12:37:40 +02:00
alesapin
b2c9611da6 Fix build 2023-05-25 12:01:24 +02:00
何李夫
e4c8c4cecf
Add zookeeper name in endpoint id (#49780)
* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

Here is some infomation:
1.old table:
  CREATE TABLE a1 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
2.new table:
  CREATE TABLE a2 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('aux1:/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
3.error info:
  <Error> executeQuery: Code: 220. DB::Exception: Duplicate interserver IO endpoint:
          DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (DUPLICATE_INTERSERVER_IO_ENDPOINT)
  <Error> InterserverIOHTTPHandler: Code: 221. DB::Exception: No interserver IO endpoint
          named DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (NO_SUCH_INTERSERVER_IO_ENDPOINT)

* Revert "Add zookeeper name in endpoint id"

This reverts commit 9deb75b249619b7abdd38e3949ca8b3a76c9df8e.

* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

* Fix incompatible with a new setting

* add a test, fix other issues

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

* Update 02735_system_zookeeper_connection.reference

* Update 02735_system_zookeeper_connection.sql

* Update run.sh

* Remove the 'no-fasttest' tag

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-05-25 12:50:14 +03:00
Robert Schulze
1b32348452
Merge pull request #50219 from rschu1ze/docs/client-doc-parameter
Minor adjustment of clickhouse-client/local parameter docs
2023-05-25 11:41:34 +02:00
Robert Schulze
4f5cf656eb
Minor adjustment of clickhouse-client/local parameter docs
Related to #50210
2023-05-25 09:36:03 +00:00
alesapin
8a7c4bee53 Merge branch 'master' into transactions_for_encrypted_disk 2023-05-25 11:35:17 +02:00
Robert Schulze
a06777c389
Merge pull request #50218 from rschu1ze/readme-update
README.md: Remove Berlin Meetup from upcoming events
2023-05-25 10:54:49 +02:00
Robert Schulze
2651330b12
README.md: Remove Berlin Meetup from upcoming events 2023-05-25 08:50:44 +00:00
Azat Khuzhin
b680697cce Initialize POD members of ASTs to make it less error-prone
The cost of initializing members is insignificant in compare to parsing,
while the cost of the error is high.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-25 10:18:55 +02:00
Azat Khuzhin
b30cfe5503 Fix UB in ASTWatchQuery for is_watch_events
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-25 10:15:30 +02:00
Azat Khuzhin
c053d75741 Fix formatting of INTO OUTFILE extensions (APPEND / AND STDOUT)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-25 10:15:30 +02:00
Azat Khuzhin
9582d9e892 Fix UB for INTO OUTFILE extensions (APPEND / AND STDOUT)
MSAn report:

==38627==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x555599f5e114 in std::__1::__unique_if<DB::WriteBufferFromFile>::__unique_single std::__1::make_unique[abi:v15000]<> build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714:32
    1 0x555599f5e114 in DB::ClientBase::initOutputFormat() build_docker/./src/Client/ClientBase.cpp:604:21
    2 0x555599f590a8 in DB::ClientBase::onData() build_docker/./src/Client/ClientBase.cpp:446:5
    3 0x555599f6f36e in DB::ClientBase::receiveAndProcessPacket() build_docker/./src/Client/ClientBase.cpp:1019:17
    4 0x555599f6e863 in DB::ClientBase::receiveResult() build_docker/./src/Client/ClientBase.cpp:987:18
    5 0x555599f6c05b in DB::ClientBase::processOrdinaryQuery() build_docker/./src/Client/ClientBase.cpp:905:13
    6 0x555599f67e05 in DB::ClientBase::processParsedSingleQuery() build_docker/./src/Client/ClientBase.cpp:1711:13
    7 0x555599f86fb6 in DB::ClientBase::executeMultiQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_docker/./src/Client/ClientBase.cpp:1975:21

  Uninitialized value was created by a heap allocation
    8 0x55559bd3e038 in DB::ParserExplainQuery::parseImpl(DB::IParser::Pos&, std::__1::shared_ptr<DB::IAST>&, DB::Expected&) build_docker/./src/Parsers/ParserExplainQuery.cpp:53:26
    9 0x55559bce31f4 in DB::IParserBase::parse(DB::IParser::Pos&, std::__1::shared_ptr<DB::IAST>&, DB::Expected&)::$_0::operator()() const build_docker/./src/Parsers/IParserBase.cpp:13:20
    ..
    21 0x55559be13b5c in DB::parseQueryAndMovePosition(DB::IParser&, char const*&, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, unsigned long, unsigned long) build_docker/./src/Parsers/parseQuery.cpp:357:18
    22 0x555599f5673a in DB::ClientBase::parseQuery(char const*&, char const*, bool) const build_docker/./src/Client/ClientBase.cpp:362:15
    23 0x555599f84a4f in DB::ClientBase::analyzeMultiQueryText() build_docker/./src/Client/ClientBase.cpp:1821:24
    24 0x555599f867b3 in DB::ClientBase::executeMultiQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_docker/./src/Client/ClientBase.cpp:1910:22
    25 0x555599f8a2fd in DB::ClientBase::processQueryText(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_docker/./src/Client/ClientBase.cpp:2120:12
    26 0x555599f94aee in DB::ClientBase::runNonInteractive() build_docker/./src/Client/ClientBase.cpp:2403:9

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-25 10:15:23 +02:00
Zhiguo Zhou
1bc4eb1a6c OptimizeDateFilterVisitor: Revise variable names for clarity 2023-05-25 13:47:03 +08:00
helifu
cb130a1eb3 Add a new line to the reference file 2023-05-25 09:37:02 +08:00
helifu
515d30caea Add test case. 2023-05-25 09:37:02 +08:00
helifu
802b63f2ab Add 'initial_query_id' field for system.processors_profile_log
Facilitate profile data association and aggregation for the same query
2023-05-25 09:37:02 +08:00
Zhiguo Zhou
773a5bbbaa Optimize predicate with toYear converter
The date converters, such as toYear, are widely used in the where
clauses of the SQL queries, however, these conversions are often
expensive due to the complexity of the calendar system.

The function preimage is found an optimization for the predicates
with the converters. Given a predicate, toYear(c) = y, we could
convert it to its equivalent form: c >= b AND c <= e, where b is
"y-01-01" and e is "y-12-31". The similar transformation applies
to other comparisons (<>, <, >, <=, <=).

This commit implemented the above transformation at the AST level
by adding a new pass in the TreeOptimizer and a new AST visitor
for in-place replacing the predicates of toYear with the converted
ones.
2023-05-25 09:11:51 +08:00
Mikhail f. Shiryaev
a9d5b68946
Merge pull request #50197 from ClickHouse/fix-codebrowser
Fix codebrowser by using clang-15 image
2023-05-24 23:39:13 +02:00