Commit Graph

7062 Commits

Author SHA1 Message Date
kssenii
3cd1da1e11 Fix 2022-03-10 11:11:59 +01:00
Azat Khuzhin
c4b6342853
Improvements for parallel_distributed_insert_select (and related) (#34728)
* Add a warning if parallel_distributed_insert_select was ignored

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Respect max_distributed_depth for parallel_distributed_insert_select

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Print warning for non applied parallel_distributed_insert_select only for initial query

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Remove Cluster::getHashOfAddresses()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Forbid parallel_distributed_insert_select for remote()/cluster() with different addresses

Before it uses empty cluster name (getClusterName()) which is not
correct, compare all addresses instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Fix max_distributed_depth check

max_distributed_depth=1 must mean not more then one distributed query,
not two, since max_distributed_depth=0 means no limit, and
distribute_depth is 0 for the first query.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Fix INSERT INTO remote()/cluster() with parallel_distributed_insert_select

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Add a test for parallel_distributed_insert_select with cluster()/remote()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Return <remote> instead of empty cluster name in Distributed engine

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Make user with sharding_key and w/o in remote()/cluster() identical

Before with sharding_key the user was "default", while w/o it it was
empty.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 15:24:39 +01:00
Antonio Andelic
bc5d7aea57
Merge pull request #34876 from azat/long-INSERT-fix
Fix possible "Part directory doesn't exist" during INSERT
2022-03-08 12:44:53 +01:00
Kseniia Sumarokova
1eb2bae792
Merge pull request #34954 from bigo-sg/hive_read_columns_pruning
read columns pruning for hive
2022-03-08 10:17:24 +01:00
Azat Khuzhin
caffc144b5 Fix possible "Part directory doesn't exist" during INSERT
In #33291 final part commit had been defered, and now it can take
significantly more time, that may lead to "Part directory doesn't exist"
error during INSERT:

    2022.02.21 18:18:06.979881 [ 11329 ] {insert} <Debug> executeQuery: (from 127.1:24572, user: default) INSERT INTO db.table (...) VALUES
    2022.02.21 20:58:03.933593 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18044_18044_0 to 20220214_270654_270654_0.
    2022.02.21 21:16:50.961917 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18197_18197_0 to 20220214_270689_270689_0.
    ...
    2022.02.22 21:16:57.632221 [ 64878 ] {} <Warning> db.table: Removing temporary directory /clickhouse/data/db/table/tmp_insert_20220214_18232_18232_0/
    ...
    2022.02.23 12:23:56.277480 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18232_18232_0 to 20220214_273459_273459_0.
    2022.02.23 12:23:56.299218 [ 11329 ] {insert} <Error> executeQuery: Code: 107. DB::Exception: Part directory /clickhouse/data/db/table/tmp_insert_20220214_18232_18232_0/ doesn't exist. Most likely it is a logical error. (FILE_DOESNT_EXIST) (version 22.2.1.1) (from 127.1:24572) (in query: INSERT INTO db.table (...) VALUES), Stack trace (when copying this message, always include the lines below):

Follow-up for: #28760
Refs: #33291

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 07:44:11 +03:00
lgbo-ustc
a8cfc2458a update codes 2022-03-08 11:55:15 +08:00
Maksim Kita
2f9361008b
Merge pull request #35089 from 1lann/1lann/fix-update_lag-typo
Fix typo of update_lag
2022-03-07 23:12:35 +01:00
Kseniia Sumarokova
5511f2f6e6
Merge pull request #34940 from bigo-sg/hive_client_connection_pool
Use connection pool in HiveMetastoreClient
2022-03-07 17:14:56 +01:00
Kseniia Sumarokova
28b9ec01c0
Merge pull request #34945 from bigo-sg/hive_bug_fixed
unexpected result when use `in` in hive query
2022-03-07 17:13:11 +01:00
alesapin
d90bee1df9
Merge pull request #35080 from azat/mutate-exceptions
Do not hide exceptions during mutations
2022-03-07 12:22:49 +01:00
Vladimir C
678f05ca1e
Merge pull request #34912 from kssenii/fix-filelog-metadata-path 2022-03-07 11:45:29 +01:00
1lann
5423c5a45c Fix typo of update_lag
In external dictionary providers, the allowed keys for configuration seemed to have a typo
of "update_lag" as "update_tag", preventing the use of "update_lag". This change fixes that.
2022-03-07 18:31:20 +08:00
lgbo-ustc
8ae5296ee8 fixed compile errors 2022-03-07 17:26:48 +08:00
lgbo-ustc
cfeedd2cb5 fixed code style 2022-03-07 12:28:31 +08:00
lgbo-ustc
c37eedd887 update codes 2022-03-07 10:30:54 +08:00
lgbo-ustc
75a50a30c4 update codes 2022-03-07 09:43:53 +08:00
lgbo-ustc
d907b70cc4 update codes: get actual read block 2022-03-07 09:26:05 +08:00
lgbo-ustc
f4d8fb46c5 update codes 2022-03-07 09:26:05 +08:00
lgbo-ustc
62c1bd5ae9 hive read columns pruning 2022-03-07 09:26:05 +08:00
Azat Khuzhin
bc224dee36 Do not hide exceptions during mutations
system.mutations includes only the message, but not stacktrace, and it
is not always obvious to understand the culprit w/o stacktrace.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-06 13:39:49 +03:00
Kseniia Sumarokova
3ec6cd3128
Update StorageFileLog.cpp 2022-03-06 11:03:22 +01:00
alexey-milovidov
f9b7df6ba1
Merge pull request #35050 from CurtizJ/fix-async-inserts-system-table
Fix reading from `system.asynchronous_inserts` table
2022-03-06 02:25:53 +03:00
Maksim Kita
7ae1f0fa3b
Merge pull request #34911 from larspars/master
Allow LowCardinality strings for ngrambf_v1/tokenbf_v1 indexes. Fixes #21865
2022-03-04 19:17:48 +01:00
Azat Khuzhin
2ef9d32448 Revert "Remove VERSION_DATE from system.build_options"
As requested by @kitaisreal

This reverts commit 4a404532fb.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:32:54 +03:00
Azat Khuzhin
4a404532fb Remove VERSION_DATE from system.build_options
It was set only bu utils/release/release_lib.sh, and seems that this
script is not used anymore, at least that part of it.

Also note, that GIT_DATE is the same, and it is date time, not only
date.

Plus VERSION_DATE is not installed for releases anyway.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:31:37 +03:00
Azat Khuzhin
b0f964a14f Remove LIBRARY_ARCHITECTURE from system.build_options
CMAKE_LIBRARY_ARCHITECTURE and it is useless, since it is reported only
if the compiler reports subdir arch triplet [1]

  [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1531678

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:31:32 +03:00
Azat Khuzhin
494fe91f86 Fix LINK_FLAGS in system.build_options
Fixes: 79f6f5a202
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:31:32 +03:00
Azat Khuzhin
c426eef07d Fix generating USE_* for system.build_options
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:31:32 +03:00
Anton Popov
c836a57000 fix reading from system.asynchronous_inserts table 2022-03-04 11:46:15 +00:00
Anton Popov
aea7bfb59a
Merge pull request #34992 from azat/fix-asynchronous_inserts-race
Fix race between INSERT async_insert=1 and system.asynchronous_inserts
2022-03-03 20:55:19 +03:00
kssenii
d19f199e93 Revert 2022-03-03 15:25:27 +01:00
Kseniia Sumarokova
b11b34dc8c
Merge pull request #34849 from kssenii/fix-too-many-columns
Fix reading too many columns for s3 and url storages
2022-03-03 13:57:22 +01:00
Kseniia Sumarokova
ad09554c4c
Merge pull request #34996 from kssenii/fix-filelog-assertion
Fix possible segfault in filelog storage
2022-03-03 13:52:54 +01:00
Frank Chen
b4829465d9
Improve the opentelemetry span logs for INSERT on distributed table (#34480) 2022-03-03 12:53:29 +01:00
mergify[bot]
e169813004
Merge branch 'master' into fix-too-many-columns 2022-03-02 18:46:35 +00:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
Maksim Kita
1f5837359e clang-tidy check performance-noexcept-move-constructor fix 2022-03-02 18:15:27 +00:00
mreddy017
f893002b69 Fix vulnerable code related to std::move and noexcept
This commit fixes the vulnerable code related to std::move and noexcept identified by clangtidy tool.
2022-03-02 18:15:27 +00:00
mergify[bot]
add225c83e
Merge branch 'master' into fix-filelog-assertion 2022-03-02 17:06:08 +00:00
Amos Bird
d4cdf04683
Add missing locks (#34025) 2022-03-02 16:23:29 +01:00
kssenii
5e84c75942 Fix 2022-03-02 15:31:34 +01:00
kssenii
ef344a581b Fix bug in FileLog storage 2022-03-02 15:28:17 +01:00
alesapin
b1f5805647
Merge pull request #34609 from ClickHouse/unrestricted-zk-reads
allow unrestricted reads from zookeeper
2022-03-02 14:53:12 +01:00
Azat Khuzhin
57f636a1e8 Fix race between INSERT async_insert=1 and system.asynchronous_inserts
CI report [1]:

    [c190f600f8c6] 2022.03.02 01:07:34.553012 [ 23552 ] {76b6113b-1479-46c9-90ab-e78a3c9f3dbb}  executeQuery: Code: 60. DB::Exception: Both table name and UUID are empty. (UNKNOWN_TABLE) (version 22.3.1.1) (from [::1]:42040) (comment: '02015_async_inserts_stress_long.sh') (in query: SELECT * FROM system.asynchronous_inserts FORMAT Null), Stack trace (when copying this message, always include the lines below):

    0. ClickHouse/contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int) @ 0xf50e04c in /fasttest-workspace/build/programs/clickhouse
    1. ClickHouse/src/Common/Exception.cpp:58: DB::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int, bool) @ 0x663ebfa in /fasttest-workspace/build/programs/clickhouse
    2. DB::StorageID::assertNotEmpty() const @ 0xbc08591 in /fasttest-workspace/build/programs/clickhouse
    3. ClickHouse/contrib/libcxx/include/string:1444: DB::StorageID::getDatabaseName() const @ 0xe50d2b6 in /fasttest-workspace/build/programs/clickhouse
    4. ClickHouse/contrib/libcxx/include/string:1957: DB::StorageSystemAsynchronousInserts::fillData(std::__1::vector::mutable_ptr, std::__1::allocator::mutable_ptr > >&, std::__1::shared_ptr, DB::SelectQueryInfo const&) const @ 0xdac636c in /fasttest-workspace/build/programs/clickhouse

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/34973/e6fc6a22d5c018961c18247242dd3a40b8c54ff2/fast_test__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-02 15:28:06 +03:00
alesapin
9249c5d50e Use tryget instead of get 2022-03-02 13:09:12 +03:00
Maksim Kita
53116faeeb
Update MergeTreeIndexFullText.cpp 2022-03-02 11:08:35 +01:00
Kseniia Sumarokova
a9ab149b31
Merge pull request #34859 from Vxider/windowview-multi-column-groupby
Fix bugs for multiple columns group by in WindowView
2022-03-02 10:09:47 +01:00
Filatenkov Artur
f48f35cad0
Merge pull request #34975 from Vector-Similarity-Search-for-ClickHouse/fix-typo
Fix typo
2022-03-02 09:59:06 +03:00
NikitaEvs
06f47673f4 Fix typo 2022-03-01 21:42:27 +00:00
alesapin
e2989c2b85 Fix storage system zookeeper 2022-03-01 21:23:26 +01:00