thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
112,895 Commits 9,315 Branches 1,783 Tags 2.1 GiB
056268f90b
Commit Graph

40094 Commits

Author SHA1 Message Date
Anton Popov
5b8db57fe8
Merge branch 'master' into retry-loading-of-parts 2023-04-11 15:51:40 +02:00
Robert Schulze
1fc5d92ae2
Merge pull request #48627 from ClickHouse/rs/formatdatetime-refactoring 
Minor refactoring of formatDateTime()
 2023-04-11 15:29:47 +02:00
Kseniia Sumarokova
10bb328d7d
Merge branch 'master' into fix-error-in-cache 2023-04-11 15:25:38 +02:00
Anton Popov
5a77a2aa4c
Merge branch 'master' into CurtizJ-patch-7 2023-04-11 15:06:07 +02:00
FriendLey
e327371297
Merge branch 'master' into test_soundex 2023-04-11 21:02:01 +08:00
Han Fei
7210b50551
Merge pull request #48632 from hanfei1991/hanfei/refine-expmsg 
refine some messages of exception in regexp tree
 2023-04-11 14:45:35 +02:00
Antonio Andelic
b30aff06d8
Merge pull request #48595 from aalexfvk/alexfvk/fix_create_not_auth_problem 
Fix bug in Keeper when a node is not created with scheme `auth` in ACL sometimes.
 2023-04-11 14:41:37 +02:00
Kruglov Pavel
c3482a104f
Merge pull request #48472 from ClickHouse/Avogar-patch-2 
Avoid operation on uninitialised data in readDateTimeTextImpl
 2023-04-11 14:30:26 +02:00
robot-ch-test-poll1
028ccfdc87
Merge pull request #48438 from evillique/add-trailing-commas 
Allow trailing commas in expression list of SELECT query
 2023-04-11 14:21:28 +02:00
Robert Schulze
e08b7c8a53
Merge pull request #48422 from ClickHouse/rs/better-f-in-formatdatetime 
MySQL compat: Align behavior of %f in formatDateTime() with parseDateTime()
 2023-04-11 14:12:16 +02:00
Robert Schulze
4e52dc672e
Merge pull request #48420 from ClickHouse/rs/f 
MySQL compat: Implement %f in parseDateTime()
 2023-04-11 14:12:02 +02:00
SmitaRKulkarni
e2010c0257
Merge branch 'master' into zk_retry_timeout 2023-04-11 14:05:52 +02:00
Han Fei
c857f767fa
Merge branch 'master' into hanfei/refine-expmsg 2023-04-11 14:03:27 +02:00
Robert Schulze
ba67626992
Add comment about used exception code 2023-04-11 11:59:04 +00:00
vdimir
30659c5b48
Fix build ThreadGroupPtr 2023-04-11 11:46:53 +00:00
Kseniia Sumarokova
f4aa62ebc8
Merge branch 'master' into fix-error-in-cache 2023-04-11 13:40:46 +02:00
Sema Checherinda
4b9eff7dab
Merge pull request #48638 from ClickHouse/vdimir/fix-build-48543 
Fix build src/Interpreters/InterpreterInsertQuery.h
 2023-04-11 13:27:51 +02:00
Yatsishin Ilya
b5b65d2149 Merge remote-tracking branch 'origin' into clickhouse-help 2023-04-11 11:24:48 +00:00
Han Fei
6c33180ac8
Merge branch 'master' into hanfei/refine-expmsg 2023-04-11 13:19:38 +02:00
FriendLey
8aa17dbffe remove return in void function. 2023-04-11 19:18:27 +08:00
vdimir
4605d71947
Fix build src/Interpreters/InterpreterInsertQuery.h 2023-04-11 11:17:02 +00:00
FriendLey
c527347736
Merge branch 'master' into test_soundex 2023-04-11 19:16:10 +08:00
Sergei Trifonov
ca6e253fa6
Merge pull request #48526 from ClickHouse/lost-part-count 2023-04-11 13:11:42 +02:00
FriendLey
48463f79fd
Merge branch 'master' into test_soundex 2023-04-11 19:00:51 +08:00
kssenii
6717eab355 Fix 2023-04-11 12:35:59 +02:00
Kruglov Pavel
a0d11fdf53
Merge pull request #48436 from Avogar/fix-generate-random 
Generate valid Decimals and Bools in generateRandom function
 2023-04-11 12:06:08 +02:00
FriendLey
ffa1a75fb5 Merge branch 'master' into test_soundex 2023-04-11 17:52:32 +08:00
Sema Checherinda
4dd86a406a
Merge pull request #48543 from azat/mv-uniq-thread-group 
Use one ThreadGroup while pushing to materialized views (and some refactoring for ThreadGroup)
 2023-04-11 11:47:46 +02:00
Han Fei
363b97fab8 refine some messages of exception in regexp tree 2023-04-11 11:45:29 +02:00
Robert Schulze
bbbb55cd50
Better handling of values too large for VarInt encoding 
PR #48154 introduced a sanity check in the form of a debug assertion
that the input values for VarInt encoding are not too big. Such values
should be exceptionally rare in practice but the AST fuzzer managed to
trigger the assertion regardless. The strategy to deal with such values
until now was to bypass the check by limiting the value to the maximum
allowed value (see #48412). Because a new AST Fuzzer failure appeared
(#48497) and there may be more failures in future, this PR changes the
sanity check from an assert to an exception.

Fixes: #48497
 2023-04-11 07:47:33 +00:00
Aleksei Filatov
dd2364361c
Update src/Coordination/tests/gtest_coordination.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-04-11 10:45:30 +03:00
Robert Schulze
4b61685c64
Minor refactoring of formatDateTime() 
Don't explicitly specify literal length when the called functions can
compute it.
 2023-04-11 07:43:24 +00:00
Robert Schulze
c3f976c58a
Move space check out of loop 2023-04-11 06:19:10 +00:00
Robert Schulze
b7d5df7aa1
Merge branch 'master' into rs/better-f-in-formatdatetime 2023-04-11 08:11:04 +02:00
FriendLey
910bd8a15f bugfix: fix access illegal address bugs. 2023-04-11 13:01:53 +08:00
Alexey Milovidov
9e1db557e0
Merge pull request #48612 from ClickHouse/remove-strange-code-2 
Remove strange code
 2023-04-11 06:17:45 +03:00
FriendLey
6a3eb19821
Update src/Functions/soundex.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-04-11 09:49:52 +08:00
FriendLey
3d4c6e5a48
Update src/Functions/soundex.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-04-11 09:49:42 +08:00
FriendLey
baa29a5a2c
Update src/Functions/soundex.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-04-11 09:49:34 +08:00
FriendLey
0ac9f78c43
Update src/Functions/soundex.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-04-11 09:49:16 +08:00
FriendLey
a5d4a2065e
Update src/Functions/soundex.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-04-11 09:49:06 +08:00
Alexey Milovidov
d259217cf3
Merge pull request #48570 from azat/build/logger_useful 
Remove superfluous includes of logger_userful.h from headers
 2023-04-11 03:56:39 +03:00
Alexey Milovidov
93b8fc74ef
Merge pull request #48571 from azat/dict/hashed/uncaught-exception-fix 
Fix uncaught exception in case of parallel loader for hashed dictionaries
 2023-04-10 23:11:01 +03:00
Alexey Milovidov
f4d6596eaa
Merge pull request #48577 from ClickHouse/evillique-patch-2 
Don't use type conversion with String query parameters
 2023-04-10 22:18:39 +03:00
Alexey Milovidov
23a0879452 Remove strange code 2023-04-10 21:17:08 +02:00
Yakov Olkhovskiy
e6d96c27f2 unnecessary include 2023-04-10 19:16:02 +00:00
Yakov Olkhovskiy
a6c99e8e84 add IPv4 to supertype calculation 2023-04-10 18:59:50 +00:00
Azat Khuzhin
013a3ed269 Fix possible lock order inversion in Context::setTemporaryStorageInCache() 
TSan report [1]:

    WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=1)
      Cycle in lock order graph: M0 (0x7b9000000008) => M1 (0x7b90000000a8) => M0

      Mutex M1 acquired here while holding mutex M0 in main thread:
        0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        3 std::__1::lock_guard<std::__1::mutex>::lock_guard[abi:v15000](std::__1::mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:94:27 (clickhouse+0x1d0ccb3a) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        4 DB::Context::getDisk() const build_docker/./src/Interpreters/Context.cpp:3281:21 (clickhouse+0x1d0ccb3a)
        5 DB::Context::setTemporaryStorageInCache() build_docker/./src/Interpreters/Context.cpp:927:21 (clickhouse+0x1d0cc188) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        6 DB::Server::main() build_docker/./programs/server/Server.cpp:1055:25 (clickhouse+0x148504c2) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)

      Mutex M0 previously acquired by the same thread here:
        0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        3 std::__1::unique_lock<std::__1::recursive_mutex>::unique_lock[abi:v15000](std::__1::recursive_mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:122:61 (clickhouse+0x1d0cc148) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        4 DB::Context::getLock() const build_docker/./src/Interpreters/Context.cpp:690:12 (clickhouse+0x1d0cc148)
        5 DB::Context::setTemporaryStorageInCache() build_docker/./src/Interpreters/Context.cpp:922:17 (clickhouse+0x1d0cc148)
        6 DB::Server::main() build_docker/./programs/server/Server.cpp:1055:25 (clickhouse+0x148504c2) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)

      Muex M0 acquired here while holding mutex M1 in main thread:
        0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        3 std::__1::unique_lock<std::__1::recursive_mutex>::unique_lock[abi:v15000](std::__1::recursive_mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:122:61 (clickhouse+0x1d0c7ce1) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        4 DB::Context::getLock() const build_docker/./src/Interpreters/Context.cpp:690:12 (clickhouse+0x1d0c7ce1)
        5 DB::Context::getPath() const build_docker/./src/Interpreters/Context.cpp:713:17 (clickhouse+0x1d0c7ce1)
        6 DB::loadDiskLocalConfig() build_docker/./src/Disks/DiskLocal.cpp:75:30 (clickhouse+0x1ce7ed8a) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        7 DB::DiskLocal::applyNewSettings() build_docker/./src/Disks/DiskLocal.cpp:491:5 (clickhouse+0x1ce7e7ef) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        8 DB::DiskSelector::updateFromConfig() const build_docker/./src/Disks/DiskSelector.cpp:92:19 (clickhouse+0x1d14088e) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        9 DB::Context::updateStorageConfiguration() build_docker/./src/Interpreters/Context.cpp:3388:49 (clickhouse+0x1d0ec002) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        ..
        17 DB::ConfigReloader::reloadIfNewer() build_docker/./src/Common/Config/ConfigReloader.cpp:149:13 (clickhouse+0x1fff4250)

      Mutex M1 previously acquired by the same thread here:
        0 pthread_mutex_lock <null> (clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        3 std::__1::lock_guard<std::__1::mutex>::lock_guard[abi:v15000](std::__1::mutex&) build_docker/./contrib/llvm-project/libcxx/include/__mutex_base:94:27 (clickhouse+0x1d0ebf25) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280)
        4 DB::Context::updateStorageConfiguration() build_docker/./src/Interpreters/Context.cpp:3384:21 (clickhouse+0x1d0ebf25)
        ..
        12 DB::ConfigReloader::reloadIfNewer() build_docker/./src/Common/Config/ConfigReloader.cpp:149:13 (clickhouse+0x1fff4250)

    SUMMARY: ThreadSanitizer: lock-order-inversion (potential deadlock) (/usr/bin/clickhouse+0xc66a8da) (BuildId: 174c9c4bf3606a5fc390211bdc2feff052246280) in pthread_mutex_lock
    ==================

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/48596/a1272e8536265929255fdf5020836f057859e425/integration_tests__tsan__[6/6].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-10 19:56:52 +02:00
Nikolai Kochetov
ba468eac45
Merge pull request #48485 from ClickHouse/read-step-for-system-zookeeper 
Add reading step for system zookeeper. Analyze path from filter DAG.
 2023-04-10 19:28:18 +02:00
Kseniia Sumarokova
65259d964b
Merge pull request #48167 from lzydmxy/add_columns_for_system_storage_policies 
Add columns for table `system.storage_policies`
 2023-04-10 18:32:13 +02:00
Igor Nikonov
e3b5072eb8
Merge pull request #48525 from ClickHouse/parallel-reading-from-file 
Parallel processing right after reading `FROM file()`
 2023-04-10 18:30:05 +02:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-10 17:59:30 +02:00
FriendLey
2a0de4bc16
Update src/Functions/soundex.cpp 
Co-authored-by: flynn <fenglv15@mails.ucas.ac.cn>
 2023-04-10 21:14:49 +08:00
FriendLey
a67ea35875 format code and replace std function with Common utils. 2023-04-10 19:12:17 +08:00
Nikolai Kochetov
d564fe4e64 Fixing build 2023-04-10 10:54:37 +00:00
Nikolai Kochetov
28484cd762
Merge pull request #48522 from ClickHouse/many-mutations-memory-limit 
Reduce memory usage for multiple alter delete mutations.
 2023-04-10 12:50:42 +02:00
flynn
5a8d10c0e0 fix 2023-04-10 10:46:47 +00:00
Aleksei Filatov
e8bf96d126 Add unit tests 2023-04-10 13:02:43 +03:00
Aleksei Filatov
000c2b5b91 Correct FixupACL for auth scheme 2023-04-10 13:02:31 +03:00
FriendLey
9d5fc02c5e update code style. 2023-04-10 17:22:15 +08:00
FriendLey
890d4655a0 update code styl. 2023-04-10 17:06:31 +08:00
flynn
dfe9ce6ec4 fix 2023-04-10 08:27:55 +00:00
Raúl Marín
45ad555c39
Merge branch 'master' into zk_retry_timeout 2023-04-10 10:04:16 +02:00
Nikolay Degterinsky
8524f6958a
Update src/Interpreters/ReplaceQueryParameterVisitor.cpp 
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2023-04-10 09:09:57 +02:00
flynn
feb1dce1e7
Merge branch 'master' into show-table 2023-04-10 14:55:16 +08:00
flynn
f87782b6b5 support show table 2023-04-10 06:37:47 +00:00
Alexey Milovidov
c279516ac1
Merge branch 'master' into parallel-reading-from-file 2023-04-10 08:02:50 +03:00
Nikolay Degterinsky
e71c7fe744
Fix 2023-04-10 03:48:58 +02:00
Nikolay Degterinsky
6b1c86c6ec
Don't use type conversion with String query parameters 2023-04-10 03:39:13 +02:00
Alexey Milovidov
dddca7d9a4
Merge pull request #48563 from ClickHouse/server-uuid-macro 
Add support for `{server_uuid}` macro
 2023-04-10 03:40:47 +03:00
Alexey Milovidov
1f6b9809dd Better exception messages for unparsed Bool 2023-04-10 02:32:04 +02:00
Igor Nikonov
1187534545 Simpler way to resize pipeline 2023-04-09 21:26:39 +00:00
Azat Khuzhin
211cea5e7c Fix uncaught exception in case of parallel loader for hashed dictionaries 
Since ThreadPool::wait() rethrows the first exception (if any):

<details>

<summary>stacktrace</summary>

    2023.04.09 12:53:33.629333 [ 22361 ] {} <Fatal> BaseDaemon: (version 22.13.1.1, build id: 5FB01DCAAFFF19F0A9A61E253567F90685989D2F) (from thread 23032) Terminate called for uncaught exception:
    2023.04.09 12:53:33.630179 [ 23645 ] {} <Fatal> BaseDaemon:
    2023.04.09 12:53:33.630213 [ 23645 ] {} <Fatal> BaseDaemon: Stack trace: 0x7f68b00baccc 0x7f68b006bef2 0x7f68b0056472 0x112a42fe 0x1c17f2a3 0x1c17f238 0xbf4bc3b 0x13961c6d 0x138ee529 0x138ed6bc 0x138dd2f0 0x138dd9c6 0x1571d0dd 0x16197c1f 0x161a231e 0x1619fc93 0x161a51b9 0x11151759 0x1115454e 0x7f68b00b8fd4 0x7f68b013966c
    2023.04.09 12:53:33.630247 [ 23645 ] {} <Fatal> BaseDaemon: 3. ? @ 0x7f68b00baccc in ?
    2023.04.09 12:53:33.630263 [ 23645 ] {} <Fatal> BaseDaemon: 4. gsignal @ 0x7f68b006bef2 in ?
    2023.04.09 12:53:33.630273 [ 23645 ] {} <Fatal> BaseDaemon: 5. abort @ 0x7f68b0056472 in ?
    2023.04.09 12:53:33.648815 [ 23645 ] {} <Fatal> BaseDaemon: 6. ./.build/./src/Daemon/BaseDaemon.cpp:456: terminate_handler() @ 0x112a42fe in /usr/lib/debug/usr/bin/clickhouse.debug
    2023.04.09 12:53:33.651484 [ 23645 ] {} <Fatal> BaseDaemon: 7. ./.build/./contrib/llvm-project/libcxxabi/src/cxa_handlers.cpp:61: std::__terminate(void (*)()) @ 0x1c17f2a3 in /usr/lib/debug/usr/bin/clickhouse.debug
    2023.04.09 12:53:33.654080 [ 23645 ] {} <Fatal> BaseDaemon: 8. ./.build/./contrib/llvm-project/libcxxabi/src/cxa_handlers.cpp:79: std::terminate() @ 0x1c17f238 in /usr/lib/debug/usr/bin/clickhouse.debug
    2023.04.09 12:53:35.025565 [ 23645 ] {} <Fatal> BaseDaemon: 9. ? @ 0xbf4bc3b in /usr/lib/debug/usr/bin/clickhouse.debug
    2023.04.09 12:53:36.495557 [ 23645 ] {} <Fatal> BaseDaemon: 10. DB::ParallelDictionaryLoader<(DB::DictionaryKeyType)0, true, true>::~ParallelDictionaryLoader() @ 0x13961c6d in /usr/lib/debug/usr/bin/clickhouse.debug
    2023.04.09 12:53:37.833142 [ 23645 ] {} <Fatal> BaseDaemon: 11. DB::HashedDictionary<(DB::DictionaryKeyType)0, true, true>::loadData() @ 0x138ee529 in /usr/lib/debug/usr/bin/clickhouse.debug
    2023.04.09 12:53:39.124989 [ 23645 ] {} <Fatal> BaseDaemon: 12. DB::HashedDictionary<(DB::DictionaryKeyType)0, true, true>::HashedDictionary(DB::StorageID const&, DB::DictionaryStructure const&, std::__1::shared_ptr<DB::IDictionarySource>, DB::HashedDictionaryStorageConfiguration const&, std::__1::shared_ptr<DB::Block>) @ 0x138ed6bc in /usr/lib/debug/usr/bin/clickhouse.debug

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-09 22:52:51 +02:00
Alexey Milovidov
eda4a021a6
Merge branch 'master' into server-uuid-macro 2023-04-09 22:26:58 +03:00
Nikolay Degterinsky
10883d8af5 Allow commas with subqueries, add comments 2023-04-09 12:52:23 +00:00
Nikolay Degterinsky
6debaccd85 Fix build 2023-04-09 11:12:08 +00:00
FriendLey
fe21f301f0 update code style. 2023-04-09 17:24:54 +08:00
FriendLey
20527e03ac add soundex function. (#39880) 2023-04-09 16:41:11 +08:00
Alexey Milovidov
6292a08c9c Add a test 2023-04-09 03:25:38 +02:00
Alexey Milovidov
09ea79aaf7 Add support for {server_uuid} macro 2023-04-09 03:04:26 +02:00
Sergei Trifonov
b319271d59
Remove misleading comment 
It is no longer "significant" after `NOEXCEPT_SCOPE` macro rework #39229
 2023-04-08 22:50:24 +02:00
robot-clickhouse-ci-1
3302f1ff90
Merge pull request #48299 from CurtizJ/do-not-build-unnecessary-sets 
Do not build sets for skip indexes if they are not used
 2023-04-08 21:25:06 +02:00
Alexey Milovidov
5495cb2935
Merge pull request #48521 from ClibMouse/feature/tiny-improvement 
Use std::string::starts_with instead of a roll your own variant
 2023-04-08 22:19:48 +03:00
Alexey Milovidov
16fe18a7b6
Merge pull request #48520 from ClibMouse/feature/use-string-view-instead-of-strlen 
Use std::string_view instead of strlen
 2023-04-08 22:19:28 +03:00
Alexey Milovidov
c77acace86
Merge pull request #48528 from ClickHouse/bracketed 
Fix bracketed-paste mode messing up password input in client reconnect
 2023-04-08 22:16:02 +03:00
Alexey Milovidov
83dd5e0294
Merge pull request #48552 from ClickHouse/better-exception-message-zstd 
Better exception message for ZSTD
 2023-04-08 22:15:14 +03:00
Artem Pershin
2b00c8cb9b Fix typo in word acquired, in comment for lockForAlter method 2023-04-08 18:39:04 +03:00
Alexander Tokmakov
bf20e6bfc6
Merge pull request #48550 from ClickHouse/fix_sync_replica 
Fix some tests
 2023-04-08 14:41:10 +03:00
Yakov Olkhovskiy
054078e06d fix nested map for keys of IP and UUID types 2023-04-08 05:34:35 +00:00
Nikolay Degterinsky
7161d3d380 Fixes 2023-04-08 00:49:53 +00:00
Alexey Milovidov
ec60a10694 Better exception message for ZSTD 2023-04-08 00:54:05 +02:00
Vitaly Baranov
62e9c419de
Merge pull request #48519 from azat/build/fwd-decl-threadpool 
Use forward declaration of ThreadPool
 2023-04-08 00:44:01 +02:00
Alexander Tokmakov
e39ee3f0af suppress a bug in filesystem cache 2023-04-07 22:54:49 +02:00
Nikolay Degterinsky
41891839a0
Merge pull request #48185 from evillique/add-kafka-murmur-hash-function 
Add kafkaMurmurHash function
 2023-04-07 22:06:56 +02:00
Alexander Tokmakov
9929185eab fix REPLACE_RANGE 2023-04-07 20:31:03 +02:00
Michael Kolupaev
6609004ca6
Merge branch 'master' into bracketed 2023-04-07 11:22:56 -07:00
Alexander Tokmakov
18a2eb355e fix a bug in sync replica 2023-04-07 20:16:25 +02:00
Michael Kolupaev
b6261104e7 Fix bracketed-paste mode messing up password input in client reconnect 2023-04-07 18:10:33 +00:00
Antonio Andelic
55a999d6a3
Merge branch 'master' into fix-topk-ubsan 2023-04-07 20:05:36 +02:00
Alexey Milovidov
e09e80b493
Merge pull request #48500 from CurtizJ/fix-sparse-column-query-cache 
Fix query cache with sparse columns
 2023-04-07 19:34:10 +03:00
kssenii
00282483c9 Address remaining review comments 2023-04-07 18:29:55 +02:00
Sergei Trifonov
396f6a6421
Update src/Storages/StorageReplicatedMergeTree.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-04-07 17:07:36 +02:00
Azat Khuzhin
44ae8485f1 Use one ThreadGroup while pushing to materialized views 
Before this patch only the case when ThreadStatus (current_thread)
already has ThreadGroup works that way, after they will be identical in
this aspect.

But this should not affect anything, but it just make sense.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 15:37:35 +02:00
Azat Khuzhin
aacf2a0838 Move ThreadGroupSwitcher to ThreadStatus.h (out from MergeTree code) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 15:32:08 +02:00
Azat Khuzhin
5b2b20a0b0 Rename ThreadGroupStatus to ThreadGroup 
There are methods like getThreadGroup() and ThreadGroupSwitcher class,
so seems that this is logical.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 15:31:48 +02:00
Sema Checherinda
0503ed6fb8
Merge pull request #46668 from azat/fix-query_views_log-from-bg 
Fix system.query_views_log for MVs that are pushed from background threads
 2023-04-07 14:47:05 +02:00
Igor Nikonov
78038a3c2c Fix: do not resize pipeline when there is no files to process (globs expands to empty set) 2023-04-07 11:34:04 +00:00
MikhailBurdukov
54d7a412ec Fixed unit tests 2023-04-07 11:04:00 +00:00
Nikolay Degterinsky
a48655af39
Merge branch 'master' into add-kafka-murmur-hash-function 2023-04-07 12:21:16 +02:00
robot-ch-test-poll2
d13e8f8ddb
Merge pull request #48518 from ClibMouse/feature/remove-dead-code 
Remove dead code and unused dependencies
 2023-04-07 12:07:33 +02:00
alesapin
a0dd1c96bf
Merge pull request #48512 from ClickHouse/serxa-patch-3 
Fix bytesSize() of zk SetRequest
 2023-04-07 11:47:31 +02:00
Kseniia Sumarokova
d5889a47d6
Merge pull request #48311 from kssenii/rabbitmq-fix-cpu-usage 
Fix cpu usage in rabbitmq (was worsened in 23.2 after #44404)
 2023-04-07 11:44:48 +02:00
Azat Khuzhin
011480924a Use forward declaration of ThreadPool 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 11:25:35 +02:00
Antonio Andelic
67e6335250 Fix nullptr to memcpy 2023-04-07 08:32:01 +00:00
Sergei Trifonov
f0de79b21a
Merge branch 'master' into lost-part-count 2023-04-07 01:50:41 +02:00
Yakov Olkhovskiy
8d1924cc9a
Merge pull request #47962 from ClickHouse/imp-use-structure-from-insertion 
Improvement and fix for use_structure_from_insertion_table_in_table_functions
 2023-04-06 19:01:00 -04:00
serxa
343a071798 add lost_part_count column to system.replicas 2023-04-06 22:38:58 +00:00
Igor Nikonov
96213fa464 Fix header 2023-04-06 22:17:09 +00:00
Igor Nikonov
2e139c21d2 Parallel reading in FROM file() 2023-04-06 21:57:03 +00:00
robot-clickhouse-ci-2
2cb8252676
Merge pull request #48487 from azat/memory-tracker-fix-deadlock 
Fix deadlock due to debug tracking of memory allocations
 2023-04-06 23:50:44 +02:00
Dmitry Novik
a1552159e3 Fix style 2023-04-06 21:35:28 +00:00
ltrk2
4449e49e72 Mark a function inline 2023-04-06 13:28:30 -07:00
Nikolai Kochetov
57255eb56c Fix equals. 2023-04-06 19:22:10 +00:00
Dmitry Novik
a60c315b3e Support additional_result_filter 2023-04-06 19:18:26 +00:00
Nikolai Kochetov
e8fb135180 Reduce memory usage for multiple alter delete mutations. 2023-04-06 19:07:53 +00:00
ltrk2
790e570af6 Use std::string::starts_with instead of a roll your own variant 2023-04-06 11:57:44 -07:00
ltrk2
00c8e6e60d Use std::string_view instead of strlen 2023-04-06 11:39:33 -07:00
ltrk2
4544abc7d6 Remove dead code and unused dependencies 2023-04-06 11:37:12 -07:00
Anton Popov
8d1b535efb fix storage Memory with compression 2023-04-06 18:27:53 +00:00
Sergei Trifonov
b6975d36e9
Fix bytesSize() of zk SetRequest 2023-04-06 19:05:32 +02:00
Sema Checherinda
9d8d9e8ec9
Merge branch 'master' into fix-query_views_log-from-bg 2023-04-06 18:16:47 +02:00
Anton Popov
4757d76fbf fix query cache with sparse columns 2023-04-06 16:01:23 +00:00
MikhailBurdukov
5282a20aba Test and fix 2023-04-06 15:50:28 +00:00
Alexey Milovidov
4ab8b35b00
Merge pull request #48444 from ClickHouse/better-exception-messages-from-keeper-client 
Better exception messages from Keeper client
 2023-04-06 18:36:39 +03:00
Alexey Milovidov
3066d5df52
Merge pull request #48195 from ClickHouse/disable-compile-aggregations 
Remove a feature
 2023-04-06 18:09:48 +03:00
Anton Popov
b5a3723c88
Merge pull request #48395 from ClickHouse/with_fill_clarification 
WITH FILL clarification and cleanup
 2023-04-06 17:02:57 +02:00
Robert Schulze
7bcb8c4970
Register trim aliases in system.functions 2023-04-06 14:25:27 +00:00
Anton Popov
e88938d6c8
Change error code. 2023-04-06 16:17:45 +02:00
Robert Schulze
cfd9c4d85e
Register aliases of date_diff in system.functions 2023-04-06 14:04:37 +00:00
Azat Khuzhin
e91c8bfea3 Fix deadlock due to debug tracking of memory allocations 
Since memory tracker should be blocked globally (by default it is blocked only
for VariableContext::User).

<details>

<summary>stacktrace</summary>

    * thread 11, name = 'Collector', stop reason = trace
        frame 1: 0x00007ffff7e37ea2 libc.so.6`pthread_mutex_lock + 274
        frame 6: 0x000000001ee10298 clickhouse`toStringCached(pointers=0x00007fffedfef068, offset=0, size=45) + 88 at StackTrace.cpp:425
        frame 7: 0x000000001ee1128e clickhouse`StackTrace::toString(this=0x00007fffedfef058) const + 46 at StackTrace.cpp:444
        frame 8: 0x000000001ede58ff clickhouse`(anonymous namespace)::debugLogBigAllocationWithoutCheck(size=25187648) + 479 at MemoryTracker.cpp:98
        frame 9: 0x000000001ede5481 clickhouse`MemoryTracker::allocImpl(this=0x0000000030729e68, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x00007ffff3c22440) + 3937 at MemoryTracker.cpp:352
        frame 10: 0x000000001ede477f clickhouse`MemoryTracker::allocImpl(this=0x00007ffff3c1e530, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x00007ffff3c22440) + 607 at MemoryTracker.cpp:198
        frame 11: 0x000000001ede477f clickhouse`MemoryTracker::allocImpl(this=0x00007ffff3c22440, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x0000000000000000) + 607 at MemoryTracker.cpp:198
        frame 12: 0x000000001ede477f clickhouse`MemoryTracker::allocImpl(this=0x00007fffedff3bf0, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x0000000000000000) + 607 at MemoryTracker.cpp:198
        frame 13: 0x000000001ed6c4c9 clickhouse`CurrentMemoryTracker::allocImpl(size=25165824, throw_if_memory_exceeded=false) + 265 at CurrentMemoryTracker.cpp:58
        frame 14: 0x000000001ed6c6a1 clickhouse`CurrentMemoryTracker::allocNoThrow(size=25165824) + 33 at CurrentMemoryTracker.cpp:91
        frame 15: 0x000000001ed63079 clickhouse`operator new(unsigned long) [inlined] void Memory::trackMemory<>(size=25165824) + 89 at memory.h:177
        frame 16: 0x000000001ed63034 clickhouse`operator new(size=25165824) + 20 at new_delete.cpp:74
        ...
        frame 27: 0x000000001ee15895 clickhouse`DB::(anonymous namespace)::collectSymbols(info=0x00007ffff7062000, (null)=<unavailable>, data_ptr=0x00007fffec2b7000) + 53 at SymbolIndex.cpp:483
        frame 28: 0x000000002e1a5d57 clickhouse`::dl_iterate_phdr(callback=(clickhouse`DB::(anonymous namespace)::collectSymbols(dl_phdr_info*, unsigned long, void*) at SymbolIndex.cpp:480), data=0x00007fffec2b7000)(dl_phdr_info *, size_t, void *), void *) + 183 at phdr_cache.cpp:76
        frame 29: 0x000000001ee15549 clickhouse`DB::SymbolIndex::update(this=0x00007fffec2b7000) + 41 at SymbolIndex.cpp:515
        frame 30: 0x000000001ee1be35 clickhouse`DB::SymbolIndex::SymbolIndex(this=<unavailable>) + 53 at SymbolIndex.h:23
        frame 31: 0x000000001ee17a6c clickhouse`DB::SymbolIndex::instanceImpl() + 108 at SymbolIndex.cpp:555
        frame 32: 0x000000001ee17b4a clickhouse`DB::SymbolIndex::instance() + 10 at SymbolIndex.cpp:561
        frame 33: 0x000000001ee1070f clickhouse`void toStringEveryLineImpl<toStringCached(std::__1::array<void*, 45ul> const&, unsigned long, unsigned long)::$_0>(fatal=false, stack_trace=0x00007fffedff1ef8, callback=0x00007fffedff1ee0)::$_0&&) + 207 at StackTrace.cpp:349
        frame 34: 0x000000001ee1040b clickhouse`toStringCached(pointers=0x00007fffedff2928, offset=0, size=43) + 459 at StackTrace.cpp:436
        frame 35: 0x000000001ee1128e clickhouse`StackTrace::toString(this=0x00007fffedff2918) const + 46 at StackTrace.cpp:444
        frame 36: 0x000000001ede58ff clickhouse`(anonymous namespace)::debugLogBigAllocationWithoutCheck(size=25165824) + 479 at MemoryTracker.cpp:98
        frame 37: 0x000000001ede5481 clickhouse`MemoryTracker::allocImpl(this=0x00007fffedff3bf0, size=25165824, throw_if_memory_exceeded=false, query_tracker=0x0000000000000000) + 3937 at MemoryTracker.cpp:352
        frame 38: 0x000000001ed6c4c9 clickhouse`CurrentMemoryTracker::allocImpl(size=25165824, throw_if_memory_exceeded=false) + 265 at CurrentMemoryTracker.cpp:58
        frame 39: 0x000000001ed6c6a1 clickhouse`CurrentMemoryTracker::allocNoThrow(size=25165824) + 33 at CurrentMemoryTracker.cpp:91
        frame 40: 0x000000001ed63079 clickhouse`operator new(unsigned long) [inlined] void Memory::trackMemory<>(size=25165824) + 89 at memory.h:177
        frame 41: 0x000000001ed63034 clickhouse`operator new(size=25165824) + 20 at new_delete.cpp:74
        ...
        frame 55: 0x0000000025badace clickhouse`DB::WriteBufferFromS3::nextImpl(this=0x00007ffff3c58f00) + 366 at WriteBufferFromS3.cpp:110

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-06 16:03:46 +02:00
Sema Checherinda
b8729be97a
Merge pull request #47104 from CheSema/thead-group-for-merge 
merge and mutation make thread group for setting memory trackers right
 2023-04-06 15:46:56 +02:00
kssenii
2d335f82cc Fix another test 2023-04-06 15:43:16 +02:00
Vitaly Baranov
b45b662fe3
Merge pull request #48431 from Algunenano/system_referential_table 
Don't check dependencies when renaming system tables automatically
 2023-04-06 15:28:37 +02:00
Nikolai Kochetov
6c6335b686
Merge pull request #48357 from ClickHouse/fix-projections-analysis-with-analyzer 
Batch fix for projections analysis with analyzer.
 2023-04-06 15:13:56 +02:00
Nikolai Kochetov
54180851ab Fix name 2023-04-06 13:13:17 +00:00
Yakov Olkhovskiy
d2e897d4b8
Merge branch 'master' into imp-use-structure-from-insertion 2023-04-06 08:52:38 -04:00
Nikolai Kochetov
b199219ec1 Add reading step for system zookeeper. Analyze path from filter DAG. 2023-04-06 12:24:22 +00:00
Vitaly Baranov
2fc40834f5
Merge branch 'master' into fix-forgotten-showsecrets-in-semi-copy-ctor 2023-04-06 14:20:57 +02:00
kssenii
0f02349473 Fix 2023-04-06 14:14:22 +02:00
Nikolai Kochetov
e04abef6b3
Merge pull request #48469 from ClickHouse/kssenii-patch-7 
Fix possible segfault in cache
 2023-04-06 13:56:45 +02:00
Jordi Villar
e4977861a0 toTimeZone throw an error when no string constant timezone 2023-04-06 13:41:43 +02:00
Kruglov Pavel
fb33d6bedb
Merge pull request #48126 from Avogar/orc-arrow-parquet-more-types 
Support more ClickHouse types in ORC/Arrow/Parquet formats
 2023-04-06 13:32:46 +02:00
Kruglov Pavel
1ec9f5e42c
Avoid operation on uninitialised data in readDateTimeTextImpl 2023-04-06 13:20:11 +02:00
Kseniia Sumarokova
3fc952a56a
Update CachedOnDiskReadBufferFromFile.cpp 2023-04-06 13:01:24 +02:00
Antonio Andelic
28a33f0a47
Merge pull request #48435 from ClickHouse/kv-update-only-affected-rows 
Update only affected rows in KV storage
 2023-04-06 12:33:54 +02:00
Robert Schulze
df5de4ec78
Merge remote-tracking branch 'origin/master' into rs/better-f-in-formatdatetime 2023-04-06 09:58:09 +00:00
Vladimir C
e7a5c96353
Merge pull request #48219 from ClickHouse/vdimir/lock_order_47072 2023-04-06 11:54:33 +02:00
Vladimir C
a2b71a0054
Merge pull request #48386 from ClickHouse/vdimir/reset-downloader-tmp-data-cache 2023-04-06 11:23:16 +02:00
Robert Schulze
b7be5fd89e
Improve code and tests 2023-04-06 08:47:38 +00:00
Robert Schulze
5eb31bba87
MySQL compat: Align behavior of %f in formatDateTime() with parseDateTime() 2023-04-06 08:13:04 +00:00
Antonio Andelic
a52fc83e14
Merge branch 'master' into dont-replicate-more-keepermap 2023-04-06 09:35:37 +02:00
Robert Schulze
ce2692ba79
Merge pull request #47246 from ClickHouse/rs/m 
Change the behavior of formatter %M in function formatDateTime() from minutes to month name
 2023-04-06 09:19:50 +02:00
Yakov Olkhovskiy
8a19ac4e73
Merge branch 'master' into imp-use-structure-from-insertion 2023-04-06 00:51:44 -04:00
Nikolay Degterinsky
95086536b7 Fix style 2023-04-06 01:22:39 +00:00
Anton Popov
166a63c36c
Merge pull request #46589 from ClickHouse/rs/nonconst-replace 
replace[Regexp]{One,All}(): allow non-const pattern and replacement arguments
 2023-04-06 03:08:28 +02:00
Nikolay Degterinsky
0dfcc84910 Add support for Date/Datetime datatypes to arrayMin, arrayMax, arrayDifference functions 2023-04-06 00:53:53 +00:00
Alexey Milovidov
785ea8213b Better exception messages from Keeper client 2023-04-06 02:29:37 +02:00
robot-ch-test-poll3
3ad0a6ac18
Merge pull request #48433 from ClickHouse/some-fixes-for-parallel-replicas 
Some fixes for parallel replicas
 2023-04-06 02:00:06 +02:00
robot-ch-test-poll1
9466cec1fc
Merge pull request #48342 from ClickHouse/Backup_Restore_concurrency_check_node_2 
Check node for Backup Restore concurrency
 2023-04-05 23:49:32 +02:00
Anton Popov
4fb7e0ec3c add retries to loading of data parts 2023-04-05 21:23:57 +00:00
Alexander Tokmakov
a154fa73f4
Merge pull request #48401 from ClickHouse/fix_00002_log_messages 
Find big allocations without memory limits checks
 2023-04-05 23:52:39 +03:00
Robert Schulze
65609340cc
Yet another fix 2023-04-05 20:23:41 +00:00
robot-ch-test-poll2
bf003c7595
Merge pull request #48390 from Avogar/protobuf-tuple 
Allow write/read unnamed tuple as nested Message in Protobuf format
 2023-04-05 22:14:28 +02:00
Robert Schulze
3e84be5f3a
Fix stuff 2023-04-05 20:10:27 +00:00
Robert Schulze
ce234baf9c
Merge branch 'master' into rs/f 2023-04-05 21:59:13 +02:00
Robert Schulze
809a7fbb7a
Incorporate more review feedback 2023-04-05 19:43:30 +00:00
Sema Checherinda
554c928534
Merge branch 'master' into thead-group-for-merge 2023-04-05 21:09:38 +02:00
Raúl Marín
1fa992f015
Merge branch 'master' into system_referential_table 2023-04-05 20:47:53 +02:00
Nikolay Degterinsky
061be7c9ad Add trailing commas to expression list in SELECT query 2023-04-05 18:08:56 +00:00
Sema Checherinda
c1adbb7aa6 transfer counters to the query directly 2023-04-05 19:51:36 +02:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling 
Per-query/per-server throttling for remote IO/local IO/BACKUPs
 2023-04-05 19:32:37 +02:00
Robert Schulze
90a1a75424
Merge pull request #48412 from ClickHouse/rs/fix-ast-fuzzer 
AST fuzzer: Fix assertion in TopK serialization
 2023-04-05 19:16:50 +02:00
kssenii
be13ce76f4 Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-04-05 18:47:08 +02:00
Kruglov Pavel
11d3d01e50
Fix special build 2023-04-05 18:29:59 +02:00
Kruglov Pavel
faf6d49f1b
Merge pull request #48356 from Avogar/fix-protobuf-abort 
Avoid abort in protobuf library in debug build
 2023-04-05 18:26:59 +02:00
Kruglov Pavel
9331c6c260
Merge pull request #48122 from Avogar/bson-more-types 
Support Enum output/input in BSONEachRow, allow all map key types and avoid extra calculations
 2023-04-05 18:26:19 +02:00
Nikolai Kochetov
ab87cf7ff7
Merge branch 'master' into fix-possible-pipeline-stuck-on-partial-query-result 2023-04-05 18:25:09 +02:00
avogar
1be6bceb2f Generate valid Decimals and Bools in generateRandom function 2023-04-05 16:23:06 +00:00
Vitaly Baranov
82373890e5
Merge branch 'master' into fix-forgotten-showsecrets-in-semi-copy-ctor 2023-04-05 18:17:33 +02:00
Dmitry Novik
4cb1ffa2eb Fix distributed case 2023-04-05 16:13:03 +00:00
Antonio Andelic
0260b84bc3 Update only affected rows in KV storage 2023-04-05 16:12:14 +00:00
Raúl Marín
5d9e006c46 PR comments 2023-04-05 18:04:03 +02:00
Igor Nikonov
46f7016d8c Add search alias by input name in ActionsDAG 2023-04-05 15:56:06 +00:00
Antonio Andelic
7975df538e Don't replicate KeeperMap DELETE and TRUNCATE 2023-04-05 15:28:35 +00:00
Nikita Mikhaylov
3f44c6d936 Better 2023-04-05 15:19:45 +00:00
Alexey Milovidov
5477d67acd
Merge branch 'master' into disable-compile-aggregations 2023-04-05 18:17:17 +03:00
kssenii
62244ee57d Fix test 2023-04-05 17:14:41 +02:00
Vitaly Baranov
862a4d6fcd
Merge pull request #48142 from DimasKovas/json_functions_to_h_file 
Move FunctionsJSON implementation to header file
 2023-04-05 16:52:48 +02:00
Anton Popov
027f8a02c8
Merge branch 'master' into check_sample_dict_key_is_correct 2023-04-05 16:31:57 +02:00
Raúl Marín
d65d00f496 Don't check dependencies when renaming system tables automatically 2023-04-05 16:15:24 +02:00
vdimir
d9abcf2f69
Reduce scope storage_policies_mutex in setTemporaryStoragePolicy 2023-04-05 13:42:49 +00:00
vdimir
01ce3baf0f
Rename Context::getTemporaryVolume to getGlobalTemporaryVolume 2023-04-05 13:42:13 +00:00
Igor Nikonov
41f6e0e7b5 More efficient original node search in ActionsDAG 2023-04-05 13:36:07 +00:00
vdimir
6f3fa33641
Add unbound flag to system.filesystem_cache 2023-04-05 13:12:20 +00:00
Alexander Tokmakov
0e0e89d596 fix 2023-04-05 15:10:09 +02:00
Robert Schulze
62608f74d3
Merge pull request #48396 from ClickHouse/rs/clang-tidy-cleanup 
Cleanup mess in .clang-tidy
 2023-04-05 14:39:28 +02:00
Alexander Tokmakov
3b5ed9242c
Merge pull request #48391 from azat/fix-sync-replica-stuck 
Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION
 2023-04-05 15:20:46 +03:00
kssenii
3fb4cd0f52 Fix s3 test 2023-04-05 14:13:46 +02:00
Robert Schulze
949f7c1839
Fix darwin build 2023-04-05 12:07:22 +00:00
Kruglov Pavel
bd318950b3
Fix special build 2023-04-05 13:35:12 +02:00
Nikita Mikhaylov
7b526d3df9
Always communicate with local replica through loopback interface for parallel replicas (#48246) 2023-04-05 13:32:46 +02:00
avogar
9d3790a4c8 Fix tests 2023-04-05 11:27:38 +00:00
Robert Schulze
1be81db885
Fix build, pt. II 2023-04-05 11:23:09 +00:00
Robert Schulze
a59b53775c
Fix style 2023-04-05 10:43:55 +00:00
Sema Checherinda
f504cd5fed avoid race, do not set thread_group.counters.parent from query 2023-04-05 12:36:55 +02:00
Igor Nikonov
f6c46153fb Rename old_*_columns to input_*_columns 2023-04-05 10:22:26 +00:00
Igor Nikonov
47e7e19ca9 Move suffix generation code into one function 2023-04-05 10:12:42 +00:00
vdimir
bae1286b63
Update temp_data_on_disk in shared context 2023-04-05 10:07:44 +00:00
Kruglov Pavel
96a3307bda
Merge branch 'master' into fix-protobuf-abort 2023-04-05 11:57:18 +02:00
Kruglov Pavel
8d6b709f1a
Merge pull request #48257 from Avogar/capnp-more-types 
Support more types in CapnProto format, allow integer conversions
 2023-04-05 11:56:46 +02:00
avogar
f46f098c78 Better 2023-04-05 09:55:49 +00:00
Vladimir C
1f22fca3c3
Merge pull request #46428 from bigo-sg/quantile_gk 2023-04-05 11:25:48 +02:00
vdimir
7fe06f8435
Do not take lock for shared context in setTempDataOnDisk 2023-04-05 09:22:09 +00:00
Raúl Marín
2276d4feb4 Backups have no context and no process list element 2023-04-05 11:19:04 +02:00
Robert Schulze
0d5d2a9b55
Implement %f in parseDateTime() 
Fixes: #48394

@cc OP
 2023-04-05 09:13:18 +00:00
Azat Khuzhin
4008155a6e Add max_ prefix for backup_bandwidth settings 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 10:54:25 +02:00
Robert Schulze
598b050628
AST Fuzzer: Fix assertion in TopK serialization 
Problem:
  https://s3.amazonaws.com/clickhouse-test-reports/0/fa5b2bd4a5b02336bca8837c473a7124f8ecedf2/fuzzer_astfuzzerasan/report.html

The new assertion in the Varint code was introduced with (*). It rejects
values whose serialization cannot be deserialized (and this behavior
cannot be changed due to historical reasons). Such values should be
exceptionally rare in practice but AST fuzzer managers to trigger them.

The fix is similar to (**): Bypass the check by limiting the value to
the maximum allowed value.

(if AST fuzzer triggers finds more violations of the assertion, we might
consider throwing an exception instead)

(*) https://github.com/ClickHouse/ClickHouse/pull/48154
(**) https://github.com/ClickHouse/ClickHouse/pull/48154/files#diff-653c0a18dfdaa86262c78dc6b25550add0487f165b4ad053e86f530388f6203a
 2023-04-05 08:07:37 +00:00
Azat Khuzhin
f0f58de79c Remove throttler for IDisk::copyFile() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:41:27 +02:00
Azat Khuzhin
dd9f0f409b Remove knowledge about throttling from IBackupWriter::supportNativeCopy() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:19 +02:00
Azat Khuzhin
c8597fbb9a Do not throttle S3-S3 backups if native copy is possible 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
61405b827d Use MAKE_DEPRECATED_BY_SERVER_CONFIG() for deprecated throttle settings 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
684ecc24e2 Ignore deprecated values max_remote_{read,write}_network_bandwidth_for_server 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
ad96d3c249 Do not require proper Poco Application for Context 
$ yes | head | clickhouse-obfuscator --structure 'id String' --seed foo --input-format TSV --output-format TSV
    Poco::Exception. Code: 1000, e.code() = 0, Null pointer: _pInstance in file "/src/ch/clickhouse/base/poco/Util/include/Poco/Util/Application.h", line 446, Stack trace (when copying this message, always include the lines below):

    0. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x15605622 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    1. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x156055ed in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    2. /src/ch/clickhouse/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x2ebd2d80 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    3. /src/ch/clickhouse/base/poco/Foundation/src/Exception.cpp:132: Poco::LogicException::LogicException(String const&, int) @ 0x2ebd3667 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    4. /src/ch/clickhouse/base/poco/Foundation/src/Exception.cpp:134: Poco::NullPointerException::NullPointerException(String const&, int) @ 0x2ebd3da7 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    5. /src/ch/clickhouse/base/poco/Foundation/src/Bugcheck.cpp:42: Poco::Bugcheck::nullPointer(char const*, char const*, int) @ 0x2ebc5851 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    6. /src/ch/clickhouse/base/poco/Util/include/Poco/Util/Application.h:446: Poco::Util::Application::instance() @ 0x1ee3e6cd in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    7. /src/ch/clickhouse/src/Interpreters/Context.cpp:358: DB::ContextSharedPart::ContextSharedPart() @ 0x263b0cca in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    8. /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: std::__unique_if<DB::ContextSharedPart>::__unique_single std::make_unique[abi:v15000]<DB::ContextSharedPart>() @ 0x26387b6e in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    9. /src/ch/clickhouse/src/Interpreters/Context.cpp:652: DB::Context::createShared() @ 0x26361a3c in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    10. /src/ch/clickhouse/programs/obfuscator/Obfuscator.cpp:1293: mainEntryClickHouseObfuscator(int, char**) @ 0x1ef6eb26 in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    11. /src/ch/clickhouse/programs/main.cpp:481: main @ 0x155e569a in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
    12. ? @ 0x7ffff7dd2790 in ?
    13. __libc_start_main @ 0x7ffff7dd284a in ?
    14. _start @ 0x155e51ae in /src/ch/clickhouse/.cmake-debug/programs/clickhouse
     (version 23.3.1.2537)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
a8c1407bc4 Use existing ServerSettings in Context instead of reading it again 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:18 +02:00
Azat Khuzhin
c332d290d8 Keep only one throttler for BACKUPs IO (instead of separate read/write) 
There is no need in separate read/write throttling, because you cannot
write faster then read anyway, and plus this makes the code less cleaner

(and also it will allow avoid implementing throttling backups to S3,
since it does not use common S3 writer).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:40:17 +02:00
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis 
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:48 +02:00
Azat Khuzhin
a25dd1d348 Add ability to throttle local IO on per-query/server basis 
Server settings:
- max_local_read_bandwidth_for_server
- max_local_write_bandwidth_for_server

Query settings:
- max_local_read_bandwidth
- max_local_write_bandwidth

This is the preparation for adding ability to throttle BACKUPs

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
b3406beeb7 Add per-query network throttling 
Controlled with:
- max_remote_read_network_bandwidth
- max_remote_write_network_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
abdb682048 Move some methods for Context into module part 
This will be required later, for per-query bandwidth limiting.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
40cb87c2ac Make remote bandwidth settings server settings 
Right now those two settings are applied only at start:
- max_remote_read_network_bandwidth_for_server
- max_remote_write_network_bandwidth_for_server

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
robot-clickhouse-ci-1
fb3af065f4
Merge pull request #48294 from ClickHouse/update-arrow-2 
Try to update arrow library to release 11.0.0
 2023-04-05 04:43:51 +02:00
FFFFFFFHHHHHHH
6e8f77ee9c
Add statistical aggregate function kolmogorovSmirnovTest (#48325) 2023-04-05 01:18:54 +02:00
kssenii
a3d69694f4 Fix build 2023-04-04 23:13:17 +02:00
Robert Schulze
db864891f8
Merge pull request #47610 from murfel/format-hiliting-fix 
Fix minor hiliting issues in clickhouse-format
 2023-04-04 22:59:46 +02:00
kssenii
c379eb7c29 Fix style check 2023-04-04 22:59:16 +02:00
kssenii
f44c53b97a Merge remote-tracking branch 'upstream/master' into better-tests-for-data-lakes 2023-04-04 22:41:22 +02:00
kssenii
8c0be0c3be Checkpoints 2023-04-04 22:40:05 +02:00
Dmitry Novik
96a1304730 WIP on additional_table_filters 2023-04-04 20:33:59 +00:00
Dmitry Novik
673413dc9f
Merge pull request #48355 from ClickHouse/lambda-type-resolution 
Fix lambda type resolution
 2023-04-04 22:07:57 +02:00
Igor Nikonov
0ff4e70c84 Fix 02579_fill_empty_chunk.sql 2023-04-04 19:11:08 +00:00