Commit Graph

112802 Commits

Author SHA1 Message Date
Sema Checherinda
4dd86a406a
Merge pull request #48543 from azat/mv-uniq-thread-group
Use one ThreadGroup while pushing to materialized views (and some refactoring for ThreadGroup)
2023-04-11 11:47:46 +02:00
Han Fei
363b97fab8 refine some messages of exception in regexp tree 2023-04-11 11:45:29 +02:00
Robert Schulze
bbbb55cd50
Better handling of values too large for VarInt encoding
PR #48154 introduced a sanity check in the form of a debug assertion
that the input values for VarInt encoding are not too big. Such values
should be exceptionally rare in practice but the AST fuzzer managed to
trigger the assertion regardless. The strategy to deal with such values
until now was to bypass the check by limiting the value to the maximum
allowed value (see #48412). Because a new AST Fuzzer failure appeared
(#48497) and there may be more failures in future, this PR changes the
sanity check from an assert to an exception.

Fixes: #48497
2023-04-11 07:47:33 +00:00
Aleksei Filatov
dd2364361c
Update src/Coordination/tests/gtest_coordination.cpp
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2023-04-11 10:45:30 +03:00
Robert Schulze
4b61685c64
Minor refactoring of formatDateTime()
Don't explicitly specify literal length when the called functions can
compute it.
2023-04-11 07:43:24 +00:00
Nikolay Degterinsky
13c92ade68
Merge pull request #48576 from ClickHouse/evillique-patch-1
Update syntax.md
2023-04-11 08:52:41 +02:00
Robert Schulze
c3f976c58a
Move space check out of loop 2023-04-11 06:19:10 +00:00
Robert Schulze
b7d5df7aa1
Merge branch 'master' into rs/better-f-in-formatdatetime 2023-04-11 08:11:04 +02:00
Robert Schulze
9de55d9ccc
Rename tests to fix style check 2023-04-11 06:07:58 +00:00
FriendLey
910bd8a15f bugfix: fix access illegal address bugs. 2023-04-11 13:01:53 +08:00
Alexey Milovidov
79fa0d5a59
Merge branch 'master' into evillique-patch-1 2023-04-11 06:22:13 +03:00
Alexey Milovidov
9e1db557e0
Merge pull request #48612 from ClickHouse/remove-strange-code-2
Remove strange code
2023-04-11 06:17:45 +03:00
Rich Raposa
530a06a335
Merge branch 'master' into mar_blog_refs 2023-04-10 21:06:46 -06:00
FriendLey
f4361a1584 rename test file's name. 2023-04-11 09:54:56 +08:00
FriendLey
6a3eb19821
Update src/Functions/soundex.cpp
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-04-11 09:49:52 +08:00
FriendLey
3d4c6e5a48
Update src/Functions/soundex.cpp
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-04-11 09:49:42 +08:00
FriendLey
baa29a5a2c
Update src/Functions/soundex.cpp
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-04-11 09:49:34 +08:00
FriendLey
0ac9f78c43
Update src/Functions/soundex.cpp
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-04-11 09:49:16 +08:00
FriendLey
a5d4a2065e
Update src/Functions/soundex.cpp
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-04-11 09:49:06 +08:00
Alexey Milovidov
d259217cf3
Merge pull request #48570 from azat/build/logger_useful
Remove superfluous includes of logger_userful.h from headers
2023-04-11 03:56:39 +03:00
Yakov Olkhovskiy
7581982e0a fix test 2023-04-10 22:50:09 +00:00
Yakov Olkhovskiy
32174e9a61 test is added 2023-04-10 22:47:59 +00:00
Alexey Milovidov
10674e0a1e
Merge pull request #48551 from azat/tests/fix-02477_projection_materialize_and_zero_copy
Fix 02477_projection_materialize_and_zero_copy flakiness (due to index granularity randomization)
2023-04-10 23:12:26 +03:00
Alexey Milovidov
93b8fc74ef
Merge pull request #48571 from azat/dict/hashed/uncaught-exception-fix
Fix uncaught exception in case of parallel loader for hashed dictionaries
2023-04-10 23:11:01 +03:00
Alexey Milovidov
9f420ecad0 Update HDFS 2023-04-10 22:00:03 +02:00
Alexey Milovidov
f4d6596eaa
Merge pull request #48577 from ClickHouse/evillique-patch-2
Don't use type conversion with String query parameters
2023-04-10 22:18:39 +03:00
Alexey Milovidov
23a0879452 Remove strange code 2023-04-10 21:17:08 +02:00
Yakov Olkhovskiy
e6d96c27f2 unnecessary include 2023-04-10 19:16:02 +00:00
Yakov Olkhovskiy
a6c99e8e84 add IPv4 to supertype calculation 2023-04-10 18:59:50 +00:00
Azat Khuzhin
7cd4009c3d Fix flakiness of test_store_cleanup in case of image rebuild
The log level will be substituted from "test" to "trace" in case of the
tag is not "latest", the assumption behind this I guess is that it
should not try to use "test" log level for older versions.

But, it could have per-PR image in case of changes in the Dockerfile, so
it is better to check for self.with_installed_binary, since actually any
parameters except this will use new clickhouse binary anyway.

CI: https://s3.amazonaws.com/clickhouse-test-reports/48596/a1272e8536265929255fdf5020836f057859e425/integration_tests__tsan__[1/6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 20:31:06 +02:00
Alexey Milovidov
51688bb771
Merge pull request #48578 from ClickHouse/slightly-optimize-install
Slightly optimize Install
2023-04-10 21:17:27 +03:00
robot-ch-test-poll1
88516a4558
Merge pull request #48600 from ClickHouse/evillique-patch-3
Docs: Add SETTINGS to executable table function
2023-04-10 19:59:27 +02:00
Alexey Milovidov
438cd4af09
Merge pull request #48579 from ClickHouse/kerberos-fix-tsan
Fix TSan report in Kerberos
2023-04-10 20:58:50 +03:00
Azat Khuzhin
013a3ed269 Fix possible lock order inversion in Context::setTemporaryStorageInCache()
TSan report [1]:

    WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=1)
      Cycle in lock order graph: M0 (0x7b9000000008) => M1 (0x7b90000000a8) => M0

      Mutex M1 acquired here while holding mutex M0 in main thread:
        0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        3 std::__1::lock_guard<std::__1::mutex>::lock_guard[abi:v15000](std::__1::mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:94:27 (clickhouse+0x1d0ccb3a) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        4 DB::Context::getDisk() const build_docker/./src/Interpreters/Context.cpp:3281:21 (clickhouse+0x1d0ccb3a)
        5 DB::Context::setTemporaryStorageInCache() build_docker/./src/Interpreters/Context.cpp:927:21 (clickhouse+0x1d0cc188) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        6 DB::Server::main() build_docker/./programs/server/Server.cpp:1055:25 (clickhouse+0x148504c2) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)

      Mutex M0 previously acquired by the same thread here:
        0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        3 std::__1::unique_lock<std::__1::recursive_mutex>::unique_lock[abi:v15000](std::__1::recursive_mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:122:61 (clickhouse+0x1d0cc148) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        4 DB::Context::getLock() const build_docker/./src/Interpreters/Context.cpp:690:12 (clickhouse+0x1d0cc148)
        5 DB::Context::setTemporaryStorageInCache() build_docker/./src/Interpreters/Context.cpp:922:17 (clickhouse+0x1d0cc148)
        6 DB::Server::main() build_docker/./programs/server/Server.cpp:1055:25 (clickhouse+0x148504c2) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)

      Muex M0 acquired here while holding mutex M1 in main thread:
        0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        3 std::__1::unique_lock<std::__1::recursive_mutex>::unique_lock[abi:v15000](std::__1::recursive_mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:122:61 (clickhouse+0x1d0c7ce1) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        4 DB::Context::getLock() const build_docker/./src/Interpreters/Context.cpp:690:12 (clickhouse+0x1d0c7ce1)
        5 DB::Context::getPath() const build_docker/./src/Interpreters/Context.cpp:713:17 (clickhouse+0x1d0c7ce1)
        6 DB::loadDiskLocalConfig() build_docker/./src/Disks/DiskLocal.cpp:75:30 (clickhouse+0x1ce7ed8a) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        7 DB::DiskLocal::applyNewSettings() build_docker/./src/Disks/DiskLocal.cpp:491:5 (clickhouse+0x1ce7e7ef) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        8 DB::DiskSelector::updateFromConfig() const build_docker/./src/Disks/DiskSelector.cpp:92:19 (clickhouse+0x1d14088e) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        9 DB::Context::updateStorageConfiguration() build_docker/./src/Interpreters/Context.cpp:3388:49 (clickhouse+0x1d0ec002) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        ..
        17 DB::ConfigReloader::reloadIfNewer() build_docker/./src/Common/Config/ConfigReloader.cpp:149:13 (clickhouse+0x1fff4250)

      Mutex M1 previously acquired by the same thread here:
        0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        3 std::__1::lock_guard<std::__1::mutex>::lock_guard[abi:v15000](std::__1::mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:94:27 (clickhouse+0x1d0ebf25) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        4 DB::Context::updateStorageConfiguration() build_docker/./src/Interpreters/Context.cpp:3384:21 (clickhouse+0x1d0ebf25)
        ..
        12 DB::ConfigReloader::reloadIfNewer() build_docker/./src/Common/Config/ConfigReloader.cpp:149:13 (clickhouse+0x1fff4250)

    SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/usr/bin/clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280) in pthread_mutex_lock
    ==================

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/48596/a1272e8536265929255fdf5020836f057859e425/integration_tests__tsan__[6/6].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 19:56:52 +02:00
Nikolai Kochetov
ba468eac45
Merge pull request #48485 from ClickHouse/read-step-for-system-zookeeper
Add reading step for system zookeeper. Analyze path from filter DAG.
2023-04-10 19:28:18 +02:00
Sergei Trifonov
cc5a4c3853
Merge pull request #48568 from ClickHouse/serxa-patch-3
Update formats.md
2023-04-10 19:07:08 +02:00
robot-ch-test-poll4
5776ce6110
Merge pull request #48608 from ClickHouse/thomoco-patch-3
Update formats.md
2023-04-10 18:58:05 +02:00
Kseniia Sumarokova
65259d964b
Merge pull request #48167 from lzydmxy/add_columns_for_system_storage_policies
Add columns for table `system.storage_policies`
2023-04-10 18:32:13 +02:00
Igor Nikonov
e3b5072eb8
Merge pull request #48525 from ClickHouse/parallel-reading-from-file
Parallel processing right after reading `FROM file()`
2023-04-10 18:30:05 +02:00
Thom O'Connor
8b0adf58e5
Update formats.md
Including an Avro example with DESCRIBE url in S3
2023-04-10 10:17:43 -06:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 17:59:30 +02:00
lzydmxy
6015c3be56 update Failed test 2023-04-10 23:53:32 +08:00
Dale Mcdiarmid
d50c6a9e1b update blog references 2023-04-10 15:23:00 +01:00
Nikolay Degterinsky
450e820728
Docs: Add SETTINGS to executable table function 2023-04-10 16:18:48 +02:00
flynn
9868dd750e fix 2023-04-10 13:53:43 +00:00
Azat Khuzhin
a1272e8536 Pass existing TSAN_OPTIONS to integration tests and append new options
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 15:17:43 +02:00
FriendLey
2a0de4bc16
Update src/Functions/soundex.cpp
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
2023-04-10 21:14:49 +08:00
Azat Khuzhin
bbee1a36be Add a comment for use_sigaltstack=0 ASan option
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 15:06:40 +02:00
Azat Khuzhin
61bf70be12 Add second_deadlock_stack=1 for TSan on CI
To improve TSan reports.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 15:05:57 +02:00
FriendLey
a67ea35875 format code and replace std function with Common utils. 2023-04-10 19:12:17 +08:00