thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-27 10:02:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
84,113 Commits 9,196 Branches 1,782 Tags 2.1 GiB
bd68d1d6d0
Commit Graph

23353 Commits

Author SHA1 Message Date
Azat Khuzhin
081f9caa04 Avoid possible deadlock on server shutdown 
Here is an example for deadlock during shutting down DDLWorker:

Server main thread:

    6  ThreadFromGlobalPool::join () at ../src/Common/ThreadPool.h:217
    7  DB::DDLWorker::shutdown () at ../src/Interpreters/DDLWorker.cpp:123
    8  DB::DDLWorker::~DDLWorker () at ../src/Interpreters/DDLWorker.cpp:131
    9  DB::DDLWorker::~DDLWorker () at ../src/Interpreters/DDLWorker.cpp:130
    10 std::__1::default_delete<DB::DDLWorker>::operator() () at ../contrib/libcxx/include/memory:1397
    11 std::__1::unique_ptr<>::reset (this=0x7f7521d44fd0, __p=0x0) at ../contrib/libcxx/include/memory:1658
    12 DB::ContextSharedPart::shutdown () at ../src/Interpreters/Context.cpp:380 <!-- holds mutex
    13 DB::Context::shutdown () at ../src/Interpreters/Context.cpp:2677

DDLWorker thread:

    7  DB::Context::getLock () at ../src/Interpreters/Context.cpp:472 <-- trying to acquire shared.mutex
    8  DB::Context::getTCPPortSecure () at ../src/Interpreters/Context.cpp:2120
    9  DB::DDLTask::findCurrentHostID () at ../src/Interpreters/DDLTask.cpp:169
    10 DB::DDLWorker::initAndCheckTask () at ../src/Interpreters/DDLWorker.cpp:191
    11 DB::DDLWorker::scheduleTasks () at ../src/Interpreters/DDLWorker.cpp:367
    12 DB::DDLWorker::runMainThread () at ../src/Interpreters/DDLWorker.cpp:1063

v2: replace getTryLockTimed() with getTryLock() since std::recursive_mutex does not have try_lock_for()
v3: stole objects from context
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #21318
 2022-03-06 14:31:18 +03:00
Azat Khuzhin
bc224dee36 Do not hide exceptions during mutations 
system.mutations includes only the message, but not stacktrace, and it
is not always obvious to understand the culprit w/o stacktrace.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-06 13:39:49 +03:00
Kseniia Sumarokova
3ec6cd3128
Update StorageFileLog.cpp 2022-03-06 11:03:22 +01:00
alexey-milovidov
01f8eab2da
Merge pull request #35020 from ClickHouse/preprocessed_config_in_yaml 
Always write preprocessed config in XML format
 2022-03-06 02:31:37 +03:00
alexey-milovidov
f9b7df6ba1
Merge pull request #35050 from CurtizJ/fix-async-inserts-system-table 
Fix reading from `system.asynchronous_inserts` table
 2022-03-06 02:25:53 +03:00
mergify[bot]
0d5b7c157b
Merge branch 'master' into more_compatible_read 2022-03-05 16:18:22 +00:00
taiyang-li
1b9987ceb1 fix mistake in check-style 2022-03-05 14:22:56 +08:00
Alexey Milovidov
fc572dcb41 Remove useless define 2022-03-04 23:32:38 +01:00
tavplubix
e49521163f
Merge pull request #34860 from zhangjmruc/master 
Support non-table DDLs on cross replicated cluster
 2022-03-04 19:37:51 +01:00
Kruglov Pavel
d45b6ae03c
Extract schema only once on table creation and add it to metadata (#34684) 
* Add columns description to metadata in case of schema inference

* Make better

* Remove unnecessary code

* Fix tests

* More tests

* Add tag no-fasttest

* Fix test

* Fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-03-04 21:23:19 +03:00
mergify[bot]
13858e88d6
Merge branch 'master' into more_compatible_read 2022-03-04 18:19:01 +00:00
Maksim Kita
7ae1f0fa3b
Merge pull request #34911 from larspars/master 
Allow LowCardinality strings for ngrambf_v1/tokenbf_v1 indexes. Fixes #21865
 2022-03-04 19:17:48 +01:00
Azat Khuzhin
2ef9d32448 Revert "Remove VERSION_DATE from system.build_options" 
As requested by @kitaisreal

This reverts commit 4a404532fb.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-04 15:32:54 +03:00
Azat Khuzhin
4a404532fb Remove VERSION_DATE from system.build_options 
It was set only bu utils/release/release_lib.sh, and seems that this
script is not used anymore, at least that part of it.

Also note, that GIT_DATE is the same, and it is date time, not only
date.

Plus VERSION_DATE is not installed for releases anyway.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-04 15:31:37 +03:00
Azat Khuzhin
b0f964a14f Remove LIBRARY_ARCHITECTURE from system.build_options 
CMAKE_LIBRARY_ARCHITECTURE and it is useless, since it is reported only
if the compiler reports subdir arch triplet [1]

  [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1531678

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-04 15:31:32 +03:00
Azat Khuzhin
494fe91f86 Fix LINK_FLAGS in system.build_options 
Fixes: 79f6f5a202
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-04 15:31:32 +03:00
Azat Khuzhin
c426eef07d Fix generating USE_* for system.build_options 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-04 15:31:32 +03:00
alesapin
5db9018ae0 Fix unitialized variable 2022-03-04 13:26:56 +01:00
alesapin
2ab920a404 Review fixes 2022-03-04 13:14:38 +01:00
Vitaly Baranov
115c0c2aba
Merge pull request #34855 from vitlibar/ignore-obsolete-grants-in-attach-grants 
Ignore obsolete grants in ATTACH GRANT statements
 2022-03-04 12:50:09 +01:00
Anton Popov
c836a57000 fix reading from system.asynchronous_inserts table 2022-03-04 11:46:15 +00:00
Kruglov Pavel
cbb913a67e
Merge pull request #34092 from Avogar/random-settings 
Randomize some settings in functional tests
 2022-03-04 13:57:49 +03:00
Vladimir C
0276140d44
Merge pull request #34963 from vdimir/writebufferfroms3-try-finalize 2022-03-04 11:52:30 +01:00
Vladimir C
79b21c80b1
Merge pull request #33698 from hexiaoting/dev-map-funciton 2022-03-04 11:51:17 +01:00
vdimir
e9dee63fe2
style fix 2022-03-04 10:49:43 +00:00
Maksim Kita
c6e0922449
Merge pull request #35018 from kitaisreal/executable-udf-deterministic-in-scope-of-query-fix 
ExecutableUDF function deterministic in scope of query fix
 2022-03-04 10:46:19 +01:00
Maksim Kita
6722227407
Merge pull request #35027 from kitaisreal/functions-hierarchical-dictionaries-const-arguments-fix 
Dictionary hierarchy functions implicit key cast and support for constant arguments
 2022-03-04 10:45:37 +01:00
Kseniia Sumarokova
7a9b3ae50d
Merge pull request #35001 from kssenii/fix-async-buffer-exception 
Fix possible exception for remote fs async read about no last position boundary
 2022-03-04 10:21:17 +01:00
Anton Popov
7d01516202
Merge pull request #34795 from snar/array-compact-higher-order 
make arrayCompact behave as other higher-order functions
 2022-03-03 21:25:43 +03:00
vdimir
2aa6685baf
Add cases to 02169_map_functions, update error messages in FunctionArrayMapped 2022-03-03 18:04:17 +00:00
Anton Popov
aea7bfb59a
Merge pull request #34992 from azat/fix-asynchronous_inserts-race 
Fix race between INSERT async_insert=1 and system.asynchronous_inserts
 2022-03-03 20:55:19 +03:00
Maksim Kita
18fabccff2 Functions dictGetHierarchy, dictIsIn, dictGetChildren, dictGetDescendants support implicit key cast and constant arguments 2022-03-03 16:13:05 +00:00
mergify[bot]
e4fba1dc99
Merge branch 'master' into executable-udf-deterministic-in-scope-of-query-fix 2022-03-03 15:17:16 +00:00
kssenii
d19f199e93 Revert 2022-03-03 15:25:27 +01:00
alesapin
c33808d3db Fix read old records from logs 2022-03-03 15:18:03 +01:00
Nikita Mikhaylov
33baefb12b Done 2022-03-03 14:35:52 +01:00
hexiaoting
95530d0d70
fix style error 2022-03-03 13:22:48 +00:00
vdimir
f7f002139a
Upd FunctionArrayMapped 2022-03-03 13:22:48 +00:00
vdimir
939a15d29a
Upd FunctionArrayMapped for Map 2022-03-03 13:22:48 +00:00
vdimir
82a76d47ff
Use FunctionArrayMapped for Map 2022-03-03 13:22:47 +00:00
hexiaoting
aeec4a62e7
fix bug 2022-03-03 13:22:47 +00:00
hexiaoting
71aa411746
Fix bug 2022-03-03 13:22:46 +00:00
hexiaoting
a435c3a1e8
Fix mapFilter bug 2022-03-03 13:22:46 +00:00
hexiaoting
582e6fa1c5
Fix build error 2022-03-03 13:22:46 +00:00
hexiaoting
36939f1d5c
Update src/Functions/FunctionMapMapped.h 
Co-authored-by: Vladimir C <vdimir@yandex-team.ru>
 2022-03-03 13:22:45 +00:00
hexiaoting
ed39edda67
Update src/Functions/mapFilter.cpp 
Co-authored-by: Vladimir C <vdimir@yandex-team.ru>
 2022-03-03 13:22:45 +00:00
hexiaoting
e953e482ec
Implement more functions for Map data type 2022-03-03 13:22:44 +00:00
mergify[bot]
c90d24e10b
Merge branch 'master' into random-settings 2022-03-03 13:07:12 +00:00
Kseniia Sumarokova
b11b34dc8c
Merge pull request #34849 from kssenii/fix-too-many-columns 
Fix reading too many columns for s3 and url storages
 2022-03-03 13:57:22 +01:00
vdimir
6c29a574a3
Revert "cancel processon in QueryPipeline dtor" 
This reverts commit 7783bb24ea0c109c918faf24343082fc564f22d1.
 2022-03-03 12:56:49 +00:00
vdimir
8bb095c67e
cancel processon in QueryPipeline dtor 2022-03-03 12:56:24 +00:00
vdimir
d3ee3dee85
Catch exception in ~WriteBufferFromS3 2022-03-03 12:54:24 +00:00
Kseniia Sumarokova
ad09554c4c
Merge pull request #34996 from kssenii/fix-filelog-assertion 
Fix possible segfault in filelog storage
 2022-03-03 13:52:54 +01:00
Nikolai Kochetov
32120d5dec
Merge pull request #34993 from ClickHouse/try-fix-delayed-source 
Avoid pushing to port with data inside DelayedSource
 2022-03-03 13:44:45 +01:00
Maksim Kita
a6c510ceb1 ExecutableUDF function deterministic in scope of query fix 2022-03-03 12:40:42 +00:00
Frank Chen
b4829465d9
Improve the opentelemetry span logs for INSERT on distributed table (#34480) 2022-03-03 12:53:29 +01:00
tavplubix
b7d4c78f13
Merge pull request #34487 from zzsmdfj/issue/32977_MaterializedMySQL_add_table_list_settings 
MaterializedMySQL add materialized_mysql_tables_list settings
 2022-03-03 12:44:51 +01:00
alesapin
f24fa16184 Fix address check 2022-03-03 11:29:43 +01:00
Kseniia Sumarokova
299e11bb2b
Update DiskCacheWrapper.cpp 2022-03-03 11:11:57 +01:00
Maksim Kita
f1b1baf56e
Merge pull request #34982 from Cai-Yao/master 
date_time_input_format = 'best_effort_us'
 2022-03-03 09:22:57 +01:00
Maksim Kita
7d90afb3b0
Merge pull request #34988 from azat/safe-exit 
Fix signal-unsafe TSan report in client
 2022-03-03 09:21:26 +01:00
zhanglistar
9ee0d2c8a0 keeper atomic stat 2022-03-03 14:30:22 +08:00
alesapin
04d4c52e01 Add sanity checks for keeper 2022-03-02 20:37:59 +01:00
Azat Khuzhin
a0e590ae6c Fix signal-unsafe TSan report in client 
CI founds [1]:

    WARNING: ThreadSanitizer: signal-unsafe call inside of a signal (pid=2975)
        0 malloc  (clickhouse+0xab36d8d)
        1 _dl_exception_create_format  (ld-linux-x86-64.so.2+0x18ea8)
        2 DB::interruptSignalHandler(int) obj-x86_64-linux-gnu/../src/Client/ClientBase.cpp:236:9 (clickhouse+0x1a1d603e)
        3 __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) crtstuff.c (clickhouse+0xab3ee5f)
        4 unsigned long std::__1::__cxx_atomic_load(std::__1::__cxx_atomic_base_impl const*, std::__1::memory_order) obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1006:12 (clickhouse+0x1da6c41d)
        5 std::__1::__atomic_base::load(std::__1::memory_order) const obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1615:17 (clickhouse+0x1da6c41d)
        6 cctz::TimeZoneInfo::MakeTime(cctz::detail::civil_time const&) const obj-x86_64-linux-gnu/../contrib/cctz/src/time_zone_info.cc:844:47 (clickhouse+0x1da6c41d)
        7 cctz::time_zone::Impl::MakeTime(cctz::detail::civil_time const&) const obj-x86_64-linux-gnu/../contrib/cctz/src/time_zone_impl.h:57:19 (clickhouse+0x1da64777)
        8 cctz::time_zone::lookup(cctz::detail::civil_time const&) const obj-x86_64-linux-gnu/../contrib/cctz/src/time_zone_lookup.cc:72:27 (clickhouse+0x1da64777)
        9 DateLUTImpl::DateLUTImpl(std::__1::basic_string, std::__1::allocator > const&) obj-x86_64-linux-gnu/../src/Common/DateLUTImpl.cpp:70:63 (clickhouse+0xac8910b)
        10 DateLUT::getImplementation(std::__1::basic_string, std::__1::allocator > const&) const obj-x86_64-linux-gnu/../src/Common/DateLUT.cpp:155:55 (clickhouse+0xac87404)
        11 DateLUT::DateLUT() obj-x86_64-linux-gnu/../src/Common/DateLUT.cpp:145:25 (clickhouse+0xac8697f)
        12 DateLUT::getInstance() obj-x86_64-linux-gnu/../src/Common/DateLUT.cpp:162:20 (clickhouse+0xac87530)
        13 DateLUT::instance(std::__1::basic_string, std::__1::allocator > const&) obj-x86_64-linux-gnu/../src/Common/DateLUT.h:29:33 (clickhouse+0x188481f9)
        14 TimezoneMixin::TimezoneMixin(std::__1::basic_string, std::__1::allocator > const&) obj-x86_64-linux-gnu/../src/DataTypes/TimezoneMixin.h:18:21 (clickhouse+0x188481f9)
        15 DB::DataTypeDateTime::DataTypeDateTime(std::__1::basic_string, std::__1::allocator > const&) obj-x86_64-linux-gnu/../src/DataTypes/DataTypeDateTime.cpp:11:7 (clickhouse+0x18847e55)
        16 DB::(anonymous namespace)::FunctionEmptyArray::getNameImpl() emptyArray.cpp (clickhouse+0x1602abb1)
        17 DB::registerFunctionsEmptyArray(DB::FunctionFactory&)  (clickhouse+0x16023b6f)
        18 DB::registerFunctionsArray(DB::FunctionFactory&)  (clickhouse+0x15de7b99)
        19 DB::registerFunctions()  (clickhouse+0xe4e7bc6)
        20 DB::Client::main(std::__1::vector, std::__1::allocator >, std::__1::allocator, std::__1::allocator > > > const&) obj-x86_64-linux-gnu/../programs/client/Client.cpp:402:5 (clickhouse+0xacb2649)
        21 Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8 (clickhouse+0x1d96868a)
        22 mainEntryClickHouseClient(int, char**) obj-x86_64-linux-gnu/../programs/client/Client.cpp:1237:23 (clickhouse+0xacbdd7c)
        23 main obj-x86_64-linux-gnu/../programs/main.cpp:378:12 (clickhouse+0xabae77a)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/34924/66cbb468eb6990b74ef4d08beefbe48d32bf4bd5/stateless_tests__thread__actions__[1/3].html

Fixes: #34923
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-02 22:17:17 +03:00
alesapin
74ff3ee4bc Add sanity check for localhost 2022-03-02 20:02:02 +01:00
mergify[bot]
e169813004
Merge branch 'master' into fix-too-many-columns 2022-03-02 18:46:35 +00:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
Maksim Kita
1f5837359e clang-tidy check performance-noexcept-move-constructor fix 2022-03-02 18:15:27 +00:00
mreddy017
f893002b69 Fix vulnerable code related to std::move and noexcept 
This commit fixes the vulnerable code related to std::move and noexcept identified by clangtidy tool.
 2022-03-02 18:15:27 +00:00
kssenii
192b839cab Fix 2022-03-02 19:11:45 +01:00
Kruglov Pavel
3b92b9037f
Merge branch 'master' into random-settings 2022-03-02 21:09:48 +03:00
kssenii
d5952109fb Merge master 2022-03-02 18:15:25 +01:00
mergify[bot]
add225c83e
Merge branch 'master' into fix-filelog-assertion 2022-03-02 17:06:08 +00:00
avogar
d25d83580a Fix tests 2022-03-02 16:33:21 +00:00
Amos Bird
d4cdf04683
Add missing locks (#34025) 2022-03-02 16:23:29 +01:00
Kruglov Pavel
3e17f6dd4a
Merge pull request #34961 from Avogar/fix-schema-inference 
Fix wrong schema inference for unquoted dates in CSV
 2022-03-02 18:06:52 +03:00
kssenii
5e84c75942 Fix 2022-03-02 15:31:34 +01:00
kssenii
ef344a581b Fix bug in FileLog storage 2022-03-02 15:28:17 +01:00
alesapin
b1f5805647
Merge pull request #34609 from ClickHouse/unrestricted-zk-reads 
allow unrestricted reads from zookeeper
 2022-03-02 14:53:12 +01:00
alesapin
7312b43e57
Merge pull request #34989 from azat/fix-system-log-settings 
Fix non-MergeTree engines for system.*_log
 2022-03-02 14:32:08 +01:00
Nikolai Kochetov
ed8dfc14d4 Avoid pushing to port with data inside DelayedSource 2022-03-02 14:21:58 +01:00
kssenii
2ce4d34964 Fix checks 2022-03-02 14:17:19 +01:00
alesapin
576ff0b40d
Merge pull request #34715 from azat/fix-opened-file-cache-race 
Workaround for a bug in NuRaft library
 2022-03-02 13:58:43 +01:00
Azat Khuzhin
57f636a1e8 Fix race between INSERT async_insert=1 and system.asynchronous_inserts 
CI report [1]:

    [c190f600f8c6] 2022.03.02 01:07:34.553012 [ 23552 ] {76b6113b-1479-46c9-90ab-e78a3c9f3dbb}  executeQuery: Code: 60. DB::Exception: Both table name and UUID are empty. (UNKNOWN_TABLE) (version 22.3.1.1) (from [::1]:42040) (comment: '02015_async_inserts_stress_long.sh') (in query: SELECT * FROM system.asynchronous_inserts FORMAT Null), Stack trace (when copying this message, always include the lines below):

    0. ClickHouse/contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int) @ 0xf50e04c in /fasttest-workspace/build/programs/clickhouse
    1. ClickHouse/src/Common/Exception.cpp:58: DB::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int, bool) @ 0x663ebfa in /fasttest-workspace/build/programs/clickhouse
    2. DB::StorageID::assertNotEmpty() const @ 0xbc08591 in /fasttest-workspace/build/programs/clickhouse
    3. ClickHouse/contrib/libcxx/include/string:1444: DB::StorageID::getDatabaseName() const @ 0xe50d2b6 in /fasttest-workspace/build/programs/clickhouse
    4. ClickHouse/contrib/libcxx/include/string:1957: DB::StorageSystemAsynchronousInserts::fillData(std::__1::vector::mutable_ptr, std::__1::allocator::mutable_ptr > >&, std::__1::shared_ptr, DB::SelectQueryInfo const&) const @ 0xdac636c in /fasttest-workspace/build/programs/clickhouse

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/34973/e6fc6a22d5c018961c18247242dd3a40b8c54ff2/fast_test__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-02 15:28:06 +03:00
Azat Khuzhin
a49106e583 Fix non-MergeTree engines for system.*_log 
Fixes: https://github.com/ClickHouse/ClickHouse/pull/34949
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-02 13:24:40 +03:00
Maksim Kita
d095e7d684
Merge pull request #34931 from azat/keeper-changelog-cleanup-fix 
Avoid busy polling in keeper while searching for changelog files to delete
 2022-03-02 11:10:13 +01:00
alesapin
9249c5d50e Use tryget instead of get 2022-03-02 13:09:12 +03:00
Maksim Kita
53116faeeb
Update MergeTreeIndexFullText.cpp 2022-03-02 11:08:35 +01:00
mergify[bot]
f964cd6122
Merge branch 'master' into master 2022-03-02 10:02:40 +00:00
Kseniia Sumarokova
a9ab149b31
Merge pull request #34859 from Vxider/windowview-multi-column-groupby 
Fix bugs for multiple columns group by in WindowView
 2022-03-02 10:09:47 +01:00
cwkyaoyao
72194bbaf3 Add date_time_input_format = best_effort_us 2022-03-02 16:00:06 +08:00
HaiBo Li
2ea18b3085
Fix the bug that the toDatetime function overflows (#32898) 
* Fixed overflow bug of the function toDatetime/toDate/toDate32/parseDateTimeBestEffort
 2022-03-02 10:06:38 +03:00
Filatenkov Artur
f48f35cad0
Merge pull request #34975 from Vector-Similarity-Search-for-ClickHouse/fix-typo 
Fix typo
 2022-03-02 09:59:06 +03:00
NikitaEvs
06f47673f4 Fix typo 2022-03-01 21:42:27 +00:00
alesapin
e2989c2b85 Fix storage system zookeeper 2022-03-01 21:23:26 +01:00
alesapin
ec3e4251e1 Fix style 2022-03-01 20:34:25 +01:00
kssenii
a594f388a4 Merge master 2022-03-01 19:43:45 +01:00
kssenii
aa1c71a877 Merge master 2022-03-01 19:25:04 +01:00
kssenii
755e63ed03 Keep compatibility 2022-03-01 19:21:59 +01:00
kssenii
b2cab429a7 Move initialization out of constructor 2022-03-01 18:12:34 +01:00