thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-27 18:12:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
74,628 Commits 9,188 Branches 1,782 Tags 2.1 GiB
e66735e21e
Commit Graph

18778 Commits

Author SHA1 Message Date
Alexey Milovidov
f919271096 Compatibility 2021-10-01 04:46:41 +03:00
Nikita Tikhomirov
37f76d60ff Added MD4 and SHA384 functions. Refactoring the length of hashes in variables. 2021-10-01 04:19:50 +03:00
Alexey Milovidov
4271f8c738 Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-10-01 01:33:58 +03:00
Azat Khuzhin
91f6cf44b3 Fix SessionLog::addLoginSuccess() for ClientInfo::Interface::TCP_INTERSERVER 
In this case there there can be no user.
CI report [1]:

    Address: 0x1f Access: read. Address not mapped to object.
    Stack trace: 0x24494b2f 0x244784c6 0x260fe1af 0x260ed672 0x260ddffa 0x26104880 0x2d62f3af 0x2d6300a1 0x2d926666 0x2d91fb7a 0x7f8bcea3c609 0x7f8bce963293
    3.1. inlined from ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:0: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::operator=(std::__1::basic_string>
    3. ../src/Interpreters/SessionLog.cpp:216: DB::SessionLog::addLoginSuccess(StrongTypedef<wide::integer<128ul, unsigned int>, DB::UUIDTag> const&, std::__1::optional<std::__1::basic_string<char, st>
    4. ./obj-x86_64-linux-gnu/../src/Interpreters/Session.cpp:0: DB::Session::makeQueryContextImpl(DB::ClientInfo const*, DB::ClientInfo*) const @ 0x244784c6 in /usr/bin/clickhouse
    5.1. inlined from ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3299: std::__1::shared_ptr<DB::Context>::swap(std::__1::shared_ptr<DB::Context>&)
    5.2. inlined from ../contrib/libcxx/include/memory:3243: std::__1::shared_ptr<DB::Context>::operator=(std::__1::shared_ptr<DB::Context>&&)
    5. ../src/Server/TCPHandler.cpp:1208: DB::TCPHandler::receiveQuery() @ 0x260fe1af in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:0: DB::TCPHandler::receivePacket() @ 0x260ed672 in /usr/bin/clickhouse
    7. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x260ddffa in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1643: DB::TCPHandler::run() @ 0x26104880 in /usr/bin/clickhouse

  [1]: https://clickhouse-test-reports.s3.yandex.net/29060/c087fc0ed5fbea133eb3dc3a64b8db93a81d0ece/integration_tests_flaky_check_(asan).html#fail1
 2021-10-01 01:13:15 +03:00
Azat Khuzhin
e8a90b8ff2 Do not allow to reuse previous credentials in case of inter-server secret 
Before this patch INSERT via Buffer/Kafka may re-use previously set user
for that connection, while this is not correct, it should reset the
user, and use global context.

Note, before [1] there was a fallback to default user, but that code had
been removed, and now it got back.

  [1]: 0159c74f21 ("Secure inter-cluster query execution (with initial_user as current query user) [v3]")

Also note, that context for Buffer table (and others) cannot be changed,
since they don't have any user only profile.

I've tested this patch manually using the following:

    create table dist (key Int) engine=Distributed(test_cluster_two_shards_secure, default, data, key);
    create table buffer (key Int) engine=Buffer(default, dist, 1, 0, 0, 0, 0, 0, 0);
    create table data (key Int) engine=Memory();

    # to start the connection with readonly user
    $ clickhouse-client --user readonly -q 'select * from dist'
    $ clickhouse-client -q 'insert into buffer values (1)'
    # before this patch this produces errors like:
    # 2021.09.27 23:46:48.384920 [ 19474 ] {} <Error> default.dist.DirectoryMonitor: Code: 164. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: readonly: Cannot execute query in readonly mode. Stack trace:

v2: reset the authentication instead of using default user (as suggested by @vitlibar)
v3: reset Session::user and introduce ClientInfo::resetAuthentication (as suggested by @vitlibar)
v4: reset the session every time in interserver mode (suggested by @vitlibar)
 2021-10-01 01:13:15 +03:00
Nikita Mikhaylov
9756b8dc33
Added an ability to execute more merges and mutations than threads, added new settings (#29140) 2021-10-01 00:26:24 +03:00
Anton Popov
515cc74530 generalize code in IColumn::updatePermutation 2021-10-01 00:11:54 +03:00
Kseniia Sumarokova
3e0d4faa1e
Merge pull request #29503 from azat/external-db-log-query 
Log queries to external databases (since they may be rewritten)
 2021-09-30 21:08:57 +03:00
Kseniia Sumarokova
3457868f3d
Merge pull request #26231 from kssenii/interactive-mode-for-clickhouse-local 
clickhouse-local interactive mode, merge clickhouse-client and clickhouse-local code
 2021-09-30 19:31:34 +03:00
Maksim Kita
d538871d56 ExpressionJIT fix short-circuit with alias 2021-09-30 19:23:15 +03:00
alesapin
950f63a104 Fix commit 2021-09-30 18:59:49 +03:00
alesapin
f31c35de48 Fix nullpointer dereference in AddDefaultDatabaseVisitor 2021-09-30 18:56:55 +03:00
Alexey Milovidov
551c52c236 Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-09-30 17:45:14 +03:00
alexey-milovidov
3796ccb912
Merge pull request #29540 from myrrc/improvement/monotonicity-aggregate 
Making Monotonicity an aggregate to use with designated initializers
 2021-09-30 17:26:06 +03:00
Raúl Marín
6f2447c027 clang-tidy fix 2021-09-30 15:48:54 +02:00
Nikolai Kochetov
bc91d7ea60 Fix crash. 2021-09-30 16:48:04 +03:00
Maksim Kita
7bc6b8fd70
Merge pull request #28604 from azat/safer-http-handler 
Proper buffer flush while handling HTTP queries (avoids std::terminate)
 2021-09-30 16:44:09 +03:00
Maksim Kita
5c5199ef01
Merge pull request #29502 from azat/fix-INSERT-with-metadata-changes 
Fix LOGICAL_ERROR for INSERT with concurrent ALTER
 2021-09-30 16:11:50 +03:00
Maksim Kita
c2407fee06 Fixed tests 2021-09-30 14:35:24 +03:00
Maksim Kita
87ce29886e Fixed tests 2021-09-30 13:52:07 +03:00
vdimir
4acd8f3147
Do not add extra parentheses for tuple negate unary operator 2021-09-30 13:47:29 +03:00
Maksim Kita
3e0d6a9709
Merge pull request #29531 from kssenii/fix-paths 
Fix pathStartsWith
 2021-09-30 12:49:52 +03:00
Raúl Marín
f58742014c Consistent naming 2021-09-30 11:05:21 +02:00
Vladimir C
bc8d925ac5
Merge pull request #29463 from azat/external-db-null 2021-09-30 11:38:04 +03:00
Nikolay Degterinsky
485e19ff84
Merge pull request #28955 from anneji-dev/alterTable 
Add default database for alter table
 2021-09-30 11:28:47 +03:00
MaxWk
3cbabe2c68 support conversion between date32 and datetime64 2021-09-30 15:09:36 +08:00
mergify[bot]
775d8baa0c
Merge branch 'master' into fix-paths 2021-09-30 06:17:04 +00:00
凌涛
a13d6b0aea Check all requests 2021-09-30 13:40:48 +08:00
Alexey Boykov
192633cb9b
Merge branch 'ClickHouse:master' into numeric_tuple_functions 2021-09-30 01:37:37 +03:00
kssenii
59cbe8579d Interrupt listener 2021-09-29 22:01:52 +00:00
凌涛
bd78af435d update code and add document 2021-09-30 04:51:02 +08:00
kssenii
07b979bda3 Rewrite to processors 2021-09-29 19:17:02 +00:00
Azat Khuzhin
4618636446 Fix LOGICAL_ERROR for INSERT with metadata changes in parallel 
In #28582 the problem was introduced due to it obtains new metadata
snapshot and so it may fail when it will check that headers does not
differs, like in [1]:

<details>

    {7b1256b1-09dc-4cc3-93d9-e9f6e26b0b99} <Fatal> : Logical error: 'Block structure mismatch in  function connect between ConvertingTransform and ReplicatedMergeTreeSink stream: different number of columns:
    a UInt8 UInt8(size = 0), b Int16 Int16(size = 0), c Float32 Float32(size = 0), d String String(size = 0), e Array(UInt8) Array(size = 0, UInt64(size = 0), UInt8(size = 0)), f Nullable(UUID) Nullable(size = 0, UUID(size = 0), UInt8(size = 0)), g Tuple(UInt8, UInt16) Tuple(size = 0, UInt8(size = 0), UInt16(size = 0)), h UInt64 UInt64(size = 0)
    a UInt8 UInt8(size = 0), b Int16 Int16(size = 0), c Float32 Float32(size = 0), d String String(size = 0), e Array(UInt8) Array(size = 0, UInt64(size = 0), UInt8(size = 0)), f Nullable(UUID) Nullable(size = 0, UUID(size = 0), UInt8(size = 0)), g Tuple(UInt8, UInt16) Tuple(size = 0, UInt8(size = 0), UInt16(size = 0))'.
    {7b1256b1-09dc-4cc3-93d9-e9f6e26b0b99} <Fatal> : Logical error: 'Block structure mismatch in  function connect between ConvertingTransform and ReplicatedMergeTreeSink stream: different number of columns:
    {} <Fatal> BaseDaemon: ########################################
    {} <Fatal> BaseDaemon: (version 21.11.1.8227, build id: E012027D3A9E94FD61655974C45C15F58215605B) (from thread 2180) (query_id: 7b1256b1-09dc-4cc3-93d9-e9f6e26b0b99) Received signal Aborted (6)
    {} <Fatal> BaseDaemon:
    {} <Fatal> BaseDaemon: Stack trace: 0x7f5661a3b18b 0x7f5661a1a859 0x1333bcb8 0x1333bdc2 0x1d9f04d1 0x1d9eec3c 0x1d9eeeca 0x1f7cc44c 0x1f79b992 0x1e52123f 0x1e522b8e 0x1eb7e27b 0x1eb7be44 0x1f751e26 0x1f75f0e5 0x23ba1179 0x23ba1988 0x23cef8b4 0x23cec39a 0x23ceb17c 0x7f5661c01609 0x7f5661b17293
    {} <Fatal> BaseDaemon: 4. raise @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    {} <Fatal> BaseDaemon: 5. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:53: DB::handle_error_code(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool, std::__1::vector<void*, std::__1::allocator<void*> > const&) @ 0x1333bcb8 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:60: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x1333bdc2 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Core/Block.cpp:32: void DB::onError<void>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x1d9f04d1 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Core/Block.cpp:86: void DB::checkBlockStructure<void>(DB::Block const&, DB::Block const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) @ 0x1d9eec3c in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Core/Block.cpp:607: DB::assertCompatibleHeader(DB::Block const&, DB::Block const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x1d9eeeca in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Processors/Port.cpp:19: DB::connect(DB::OutputPort&, DB::InputPort&) @ 0x1f7cc44c in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../src/Processors/Chain.cpp:81: DB::Chain::addSource(std::__1::shared_ptr<DB::IProcessor>) @ 0x1f79b992 in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../src/Interpreters/InterpreterInsertQuery.cpp:222: DB::InterpreterInsertQuery::buildChainImpl(std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, DB::Block const&, DB::ThreadStatus*, std::__1::atomic<unsigned long>*) @ 0x1e52123f in /usr/bin/clickhouse
    {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../src/Interpreters/InterpreterInsertQuery.cpp:368: DB::InterpreterInsertQuery::execute() @ 0x1e522b8e in /usr/bin/clickhouse

</details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/29461/d6cd057e2c0c8b04d8a712a7b2a83526507ef512/functional_stateless_tests_(debug).html#fail1

Fixes: #28582 (cc: @KochetovNicolai)
 2021-09-29 21:56:59 +03:00
Azat Khuzhin
a4f2663b82 Log queries to external databases (since they may be rewriten) 
By rewritten means that some expression from WHERE can be removed, yes
there is external_table_strict_query, but sometimes it is too strict.
 2021-09-29 21:56:28 +03:00
Azat Khuzhin
6ddfbf39c8 Cover IS NULL/IS NOT NULL for external dbs in unit tests 2021-09-29 21:45:09 +03:00
Azat Khuzhin
24f55e29d1 Add support for IS NULL/IS NOT NULL for external dbs 2021-09-29 21:45:09 +03:00
Anton Popov
0dfe96fc02 generalize code in IColumn::permute 2021-09-29 20:56:10 +03:00
kssenii
2c005db9ad Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-09-29 17:45:15 +00:00
Nikolai Kochetov
68f8b9d235 Update ColumnGathererStream 2021-09-29 20:45:01 +03:00
Raúl Marín
0ee5c0bff5 Use RWLock in StorageJoin to avoid deadlocks 2021-09-29 19:30:07 +02:00
Mike Kot
61e0c6208b Making Monotonicity an aggregate to use with designated initializers 2021-09-29 18:01:26 +02:00
Maksim Kita
a5151fb49b Function accurateCastOrDefault updated implementation 2021-09-29 17:52:08 +03:00
mergify[bot]
6a2ac6e871
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-29 13:29:08 +00:00
kssenii
46a4525c21 Fix 2021-09-29 12:53:09 +00:00
Maksim Kita
7edf63162f ExternalXMLLoaders reload with server configuration support 2021-09-29 15:52:58 +03:00
Vitaly Baranov
0c33f1121b
Merge pull request #28012 from amosbird/jsonextractstring2 
Handle null column correctly in JSONFunction.
 2021-09-29 14:58:40 +03:00
Artur
fcebf7b985 correct tests 2021-09-29 11:29:24 +00:00
Mike Kot
7670ea50c1 Replacing std::function to Fn in some places 
to avoid dynamic allocating while keeping desired function signature
clear.
Simplifying SimpleCache (CachedFn) using C++20
 2021-09-29 13:09:39 +02:00
Kruglov Pavel
14be2f31f5
Merge pull request #29475 from ClickHouse/fix-29010 
Remove filter column from HAVING when it is not needed.
 2021-09-29 13:03:27 +03:00
Nikolai Kochetov
8d85590900
Merge pull request #29494 from ClickHouse/fix-possible-pushing-executor-hang 
Fix possible hang in PushingAsyncPipelineExecutor.
 2021-09-29 12:16:26 +03:00
Maksim Kita
1a0feaaf8a
Merge pull request #29511 from MaxWk/huawei-obs-support 
support obs storage
 2021-09-29 12:02:01 +03:00
Vladimir C
27f0f9fcf3
Merge pull request #21320 from arenadata/ADQM-138 2021-09-29 11:53:54 +03:00
凌涛
c298fba774 Support HSTS in Clickhouse HTTP server 2021-09-29 16:37:48 +08:00
MaxWk
723ff795a8 support obs storage 2021-09-29 14:11:28 +08:00
alexey-milovidov
be427555e5
Merge pull request #29476 from ClickHouse/fix_29456 
Fix table name escaping in Lazy database
 2021-09-29 02:45:40 +03:00
alexey-milovidov
d46dfd0ddd
Merge pull request #29446 from azat/isNull-isNotNull-format 
Transform isNull/isNotNull to IS NULL/IS NOT NULL (for external dbs)
 2021-09-29 00:56:57 +03:00
alexey-milovidov
f6db7552b6
Merge pull request #29444 from CurtizJ/fix-deadlock-async-inserts 
Fix deadlock in concurrent async inserts and truncates
 2021-09-29 00:56:29 +03:00
Nikita Mikhaylov
340fb049cf
Merge pull request #29497 from nikitamikhaylov/fuzz 
Use `max_parse_depth` in fuzzers
 2021-09-28 23:18:07 +03:00
Nikita Mikhaylov
0e602d85f0 Done 2021-09-28 20:16:32 +00:00
alesapin
7962ea5958
Merge pull request #29417 from ClickHouse/compressed_snapshots 
Compress keeper snapshots with default ZSTD codec
 2021-09-28 23:09:43 +03:00
Maksim Kita
0279afa738
Merge pull request #29474 from kitaisreal/bloom-filter-indexes-updated-tests 
Bloom filter indexes updated tests
 2021-09-28 22:40:27 +03:00
Maksim Kita
9ecd0a3f4f
Merge pull request #29220 from ClickHouse/full-text-bloom-filter-index-map-data-type 
Merging #28511
 2021-09-28 22:40:17 +03:00
Anton Popov
369821634b
Merge pull request #29390 from azat/fix-async-insert-tests 
Fix 02015_async_inserts_2 flakiness
 2021-09-28 21:07:26 +03:00
Nikolai Kochetov
ee2c12b4ab Fix pissble hang in PushingAsyncPipelineExecutor. 2021-09-28 20:47:20 +03:00
alexey-milovidov
1e6f9ac635
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-28 20:26:44 +03:00
Nikita Mikhaylov
4ac80808fe
Merge pull request #29455 from azat/enable-memory-profiler 
Enable per-query memory profiler (set to memory_profiler_step=4MiB) globally
 2021-09-28 20:00:06 +03:00
mo-avatar
6c16ec83eb in memory part lost when freeze 2021-09-28 22:52:01 +08:00
Nikolai Kochetov
ebe4cb0bbb Fix tests. 2021-09-28 15:32:43 +03:00
Anton Popov
83fd853169 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-28 15:03:27 +03:00
Ilya Golshtein
d67bc0bef4 minor fixes in ORs in JOIN 2021-09-28 14:11:33 +03:00
Ilya Golshtein
7ebc16c1b3 get rid of DNF and related features in ORs in JOIN 2021-09-28 14:11:33 +03:00
Ilya Golshtein
1dc7fc5962 fix bug found by PVS in ORs in JOIN 2021-09-28 14:11:33 +03:00
Ilya Golshtein
626bfdf23c compatible filter conditions, fixes and new tests in ORs in JOIN 2021-09-28 14:11:33 +03:00
Ilya Golshtein
bbd548e81d bypass filer conditions in DNF in ORs in JOIN (part 2) 2021-09-28 14:11:33 +03:00
Ilya Golshtein
336b2a4c68 bypass filer conditions in DNF in ORs in JOIN (part 1) 2021-09-28 14:11:33 +03:00
vdimir
760a92c490 Do not allow in optimizeClauses conditions for different table joined via OR 2021-09-28 14:11:32 +03:00
Ilya Golshtein
17e6cfbefb DNF bugfix in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
29b911f86b rebase collisions fixed in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
aa4751a632 checkStackSize moved to the top of DNF::distributed in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
6daef66364 MAX_DISJUNCTS instead of MAX_ORS in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
78ad6bf529 MAX_ORS, checkStackSize and beautification per review in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
fa6c2a64e4 minor merge mistakes fixed in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
8057e052a6 crash fix, style fixes, ASTs moved out of TableJoin in ORs in JOIN 2021-09-28 14:11:32 +03:00
Ilya Golshtein
637ff19f79 optimizeDisjuncts in ORs in JOIN 2021-09-28 14:11:31 +03:00
vdimir
3b35ab6e8c Not implemented for asof and auto join with multiple ORs 2021-09-28 14:11:31 +03:00
vdimir
f8e8f6da14 Use table_join->getAllNames in HashJoin.cpp 2021-09-28 14:11:31 +03:00
vdimir
71b6c9414c Minor changes related to JOIN ON ORs 2021-09-28 14:11:31 +03:00
Ilya Golshtein
300eb5098d fix rebase collisions in ORs in JOIN 2021-09-28 14:11:31 +03:00
vdimir
0a9a028c6f fix 2021-09-28 14:11:31 +03:00
vdimir
46187a73ee wip 2021-09-28 14:11:31 +03:00
vdimir
8e2637aab2 Store all related to one join disjunct in JoinOnClause, pt1 2021-09-28 14:11:31 +03:00
Ilya Golshtein
aa3d8a431a No redundant convertColumnsToNullable if ORs in JOIN 2021-09-28 14:11:30 +03:00
Ilya Golshtein
5ed6b26c9d any_join_distinct_right_table_keys tests and fix for ORs in JOIN 2021-09-28 14:11:30 +03:00
Ilya Golshtein
be034444ba fix any_join_distinct_right_table_keys crash if ORs in JOIN 2021-09-28 14:11:30 +03:00
vdimir
d37b88fb07 fix HashJoin::initRightBlockStructure 2021-09-28 14:11:30 +03:00
vdimir
38374ba07b fix join->getNonJoinedBlocks 2021-09-28 14:11:30 +03:00
vdimir
a8e1de51e1 fix TableJoin::leftToRightKeyRemap 2021-09-28 14:11:30 +03:00
Ilya Golshtein
24e3e04203 BlockWithFlags removed from vdimir's change 2021-09-28 14:11:30 +03:00
vdimir
6b6592fda7 Use JoinUsedFlags with multiple dijuncts 2021-09-28 14:11:30 +03:00
Ilya Golshtein
68d6a0ee85 prohibit storage join if ORs 2021-09-28 14:11:29 +03:00
Ilya Golshtein
4bddd0bc71 make PVS check happy with ORs in JOIN 2021-09-28 14:11:29 +03:00
Ilya Golshtein
d0990e270a making gcc happy, minor improvements in ORs in JOIN 2021-09-28 14:11:29 +03:00
Ilya Golshtein
b03415bd0e Fixed other bug if OR together with filter conditions 2021-09-28 14:11:29 +03:00
Ilya Golshtein
f94b2cba2d Fixed bug if OR together with filter conditions, some comments 2021-09-28 14:11:29 +03:00
Ilya Golshtein
a18c6f7f8e merge error fixed 2021-09-28 14:11:29 +03:00
Ilya Golshtein
db50015eed review changes 1 - ASTPtr, some comments 2021-09-28 14:11:29 +03:00
Ilya Golshtein
3766d47f31 ORs in JOINs 2021-09-28 14:11:28 +03:00
Maksim Kita
1e5e93830b Merge branch 'master' into cast_to_value_or_default 2021-09-28 14:04:10 +03:00
Viachaslau Boben
762904adbd Add nfd and perf test 2021-09-28 13:42:37 +03:00
Alexander Tokmakov
462088f500 fix table name escaping 2021-09-28 13:14:05 +03:00
Maksim Kita
cd1e72e312
Merge pull request #29454 from Avogar/fix-least-greatest 
Fix logical error in functions greatest/least
 2021-09-28 13:13:30 +03:00
Nikolai Kochetov
1a4ee1f77f Remove filter column from HAVING when it is not needed. 2021-09-28 13:01:47 +03:00
Maksim Kita
f2402e1a10 Fixed tests 2021-09-28 12:52:07 +03:00
Maksim Kita
71fb9e6059 Bloom filter indexes updated tests 2021-09-28 12:38:27 +03:00
Maksim Kita
9f9f36b480
Merge pull request #29464 from azat/std-cerr-cout-style-check 
Add std::cerr/std::cout style check
 2021-09-28 12:12:44 +03:00
Vladimir C
6c2db9c29b
Merge pull request #29428 from vdimir/rocksdb-istream-to-source 2021-09-28 12:06:41 +03:00
alesapin
3cd3850edf
Update gtest_coordination.cpp 2021-09-28 11:51:02 +03:00
Azat Khuzhin
9a23b20965 Remove debug from MergeTreeIndexConditionBloomFilter::traverseASTEquals() 2021-09-28 10:28:28 +03:00
alesapin
eb5402622e Better test 2021-09-28 10:24:01 +03:00
Amos Bird
07a278e426
Better 2021-09-28 11:38:29 +08:00
Amos Bird
b25c3fb659
Handle null column correctly in JSONFunction 2021-09-28 11:38:29 +08:00
alexey-milovidov
995913e89c
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-28 02:27:36 +03:00
Pavel Kruglov
cc40e36514 Fix logical error in function greatest/least 2021-09-27 23:13:01 +03:00
Azat Khuzhin
a218c3a28c Enable per-query memory profiler (set to memory_profiler_step=4MiB) globally 
Let's see at performance tests. And yes, they should show the
difference, if any, since they uses clickhouse python driver, which does
not knows about default settings, so new server will have this setting
set, while old server - upstream/master, will not.
 2021-09-27 22:46:35 +03:00
Azat Khuzhin
2378491c04 Transform isNull/isNotNull to IS NULL/IS NOT NULL (for external dbs) 
This is required to support queries (like `foo IS NULL`) to external
databases (i.e. MySQL).

This patch had been submitted separately, due to:
- avoid mixing patches
- run CI (and update tests references)
 2021-09-27 22:24:09 +03:00
Maksim Kita
7861c26eaf
Merge pull request #29431 from kitaisreal/map-bloom-filter-index-map-values-equals-support 
Map bloom filter index mapValues equals function support
 2021-09-27 22:03:53 +03:00
Nikolai Kochetov
236d71ea94
Merge pull request #28582 from ClickHouse/rewrite-pushing-to-views 
Rewrite PushingToViews
 2021-09-27 21:19:11 +03:00
Anton Popov
9a58e4a8ba fix deadlock in concurrent async inserts and truncates 2021-09-27 19:39:32 +03:00
Viachaslau Boben
b3325772f7 Add normalizeUTF8 function with NFC normalization 2021-09-27 18:47:43 +03:00
tavplubix
e9749b0027
Merge pull request #29323 from ClickHouse/fix_assertion_ddl_dependency 
Fix assertion in DDLDependencyVisitor
 2021-09-27 18:06:32 +03:00
alesapin
ddd80c8e72
Merge pull request #29264 from Enmk/governance/table_comment_update 
Implemented modifying table comments with `ALTER TABLE t MODIFY COMMENT 'value'`
 2021-09-27 18:02:14 +03:00
Maksim Kita
e290f19957 Merge branch 'master' into full-text-bloom-filter-index-map-data-type 2021-09-27 17:50:07 +03:00
Maksim Kita
b28d6e76cc Map bloom filter full text index added support for mapKeys, mapValues 2021-09-27 17:26:25 +03:00
Maksim Kita
cd9c796e6f Map bloom filter index mapValues equals function support 2021-09-27 17:23:56 +03:00
alesapin
608644436c Fix tidy 2021-09-27 17:21:10 +03:00
vdimir
aea2b952fd
Merge EmbeddedRocksDBBlockInputStream and EmbeddedRocksDBSource 2021-09-27 16:16:34 +03:00
Maksim Kita
a6a19c6267
Merge pull request #29405 from kitaisreal/added-setting-use-skip-indexes 
Added setting use_skip_indexes
 2021-09-27 16:06:54 +03:00
Kseniia Sumarokova
beda6bee2e
Merge pull request #29276 from kssenii/hdfs-fix-assertion 
Fix !hasPendingData() failed assertion in hdfs read buffer
 2021-09-27 15:47:17 +03:00
alesapin
a73064679d Compress keeper snapshots with default ZSTD codec 2021-09-27 14:54:04 +03:00
Nikita Mikhaylov
3741b8a17e
Merge pull request #29337 from amosbird/projection-fix18 
Fix projection merges and mutations.
 2021-09-27 13:45:13 +03:00
vdimir
8fe17dd3d1
Revert "Rename EmbeddedRocksDBBlockInputStream -> EmbeddedRocksDBSource" 
This reverts commit 3ab217031e50a0b4d19fb24d72088e353cdfa510.
 2021-09-27 12:49:14 +03:00
vdimir
dc7393e810
Rename EmbeddedRocksDBBlockInputStream -> EmbeddedRocksDBSource 2021-09-27 12:49:13 +03:00
vdimir
222ab78931
Convert EmbeddedRocksDBBlockInputStream to ISource 2021-09-27 12:49:13 +03:00
Vladimir C
70dc43a72d
Merge pull request #29388 from aiven/kmichel-fix-replicated-storage-reload 2021-09-27 12:41:24 +03:00
Maksim Kita
d0f53cfc31 Added setting use_skip_indexes 2021-09-27 12:17:00 +03:00
Maksim Kita
92c484f0d3
Merge pull request #29252 from Enmk/fix_some_functions_exception_messages 
Better exception messages for some String-related functions
 2021-09-27 11:50:42 +03:00
Nikolai Kochetov
78a7665f43 Merge branch 'master' into rewrite-pushing-to-views 2021-09-27 10:56:50 +03:00
Nikolai Kochetov
09232788c0 Fix GRPC tests. 2021-09-27 10:55:33 +03:00
Azat Khuzhin
c495297e65 Remove unused reset_timeout/access_timeout from AsynchronousInsertQueue 2021-09-27 10:02:27 +03:00
Vasily Nemkov
544cccb2fb
Forward-declared LOGICAL_ERROR to make style check happy 2021-09-27 09:43:13 +03:00
alexey-milovidov
5a7f30171a
Merge pull request #29355 from ClickHouse/remove-trash-db-regexp 
Fix assert in table function `merge` with database regexp
 2021-09-27 05:50:13 +03:00
alexey-milovidov
319e0c1f91
Merge pull request #29229 from azat/send-UNKNOWN_DATABASE-to-client 
Send UNKNOWN_DATABASE to the client (via TCP)
 2021-09-27 02:46:01 +03:00
Alexey Milovidov
1227a383e3 Merge branch 'remove-trash-db-regexp' of github.com:yandex/ClickHouse into remove-trash-db-regexp 2021-09-27 02:17:19 +03:00
Alexey Milovidov
16f0f0af7f Merge remote-tracking branch 'origin/master' into remove-trash-db-regexp 2021-09-27 02:12:13 +03:00
mergify[bot]
64669a28ce
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-26 22:59:23 +00:00
alexey-milovidov
1c1e2e6b28
Merge pull request #29289 from vdimir/issue-29227-rocksdb-truncate 
Fix race in StorageSystemRocksDB
 2021-09-27 01:55:07 +03:00
kssenii
8e200b68a4 Small fix 2021-09-26 21:22:04 +00:00
Azat Khuzhin
b3d1bfc67a Send UNKNOWN_DATABASE to the client (via TCP) 
Before this patch:

    $ clickhouse-client --database foo -q 'select 1'
    Code: 32. DB::Exception: Attempt to read after eof: while receiving packet from localhost:9000. (ATTEMPT_TO_READ_AFTER_EOF)

After:

    $ clickhouse-client --database foo -q 'select 1'
    Received exception from server (version 21.11.1):
    Code: 81. DB::Exception: Received from localhost:9000. DB::Exception: Database foo doesn't exist. (UNKNOWN_DATABASE)
    (query: select 1)

Fixes: #26864 (cc @vitlibar)
 2021-09-26 23:45:29 +03:00
Kevin Michel
13db65f47c
Fix ReplicatedAccessStorage shutdown without startup 
If ReplicatedAccessStorage startup was not executing or if it
failed before completing (for instance when ZooKeeper was not configured),
its destructor would call shutdown and try to join a missing thread.
 2021-09-26 19:50:29 +02:00
alexey-milovidov
aa48698613
Merge pull request #29368 from bharatnc/ncb/fix-name-h3isResClassIII 
Fix H3 function name (h3ResIsClassIII to h3IsResClassIII)
 2021-09-26 20:36:36 +03:00
Nikolai Kochetov
efbd8e4911 Remove ExceptionHandlingSink. 2021-09-26 17:54:59 +03:00
Vasily Nemkov
2c6b9aa174 Better exception messages for some String-related functions 2021-09-26 08:18:37 +03:00
Vasily Nemkov
121dfe26b6
Fixed resetting nullptr field 2021-09-26 08:13:58 +03:00
bharatnc
0e9af142df Fix one more comment 2021-09-25 18:21:47 -07:00
bharatnc
23b6ac82a7 Fix comments in AggregateFunctionFactory 2021-09-25 18:21:06 -07:00
alexey-milovidov
07eef92c16
Merge pull request #29364 from azat/benchmark-fix-query_id 
Pass through initial query_id for clickhouse-benchmark
 2021-09-26 03:27:27 +03:00
mergify[bot]
15e5204a7c
Merge branch 'master' into issue-29227-rocksdb-truncate 2021-09-26 00:10:30 +00:00
Alexey Milovidov
fb7ce28198 Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-26 02:10:11 +03:00
Kseniia Sumarokova
a7809d3d63
Merge pull request #29365 from kssenii/fix-mysql 
Fix integration test
 2021-09-26 00:52:10 +03:00
bharatnc
59dfc87113 Merge remote-tracking branch 'upstream/master' into ncb/fix-name-h3isResClassIII 2021-09-25 10:19:00 -07:00
bharatnc
2deb9643af Rename h3ResIsClassIII -> h3IsResClassIII 2021-09-25 10:12:32 -07:00
alexey-milovidov
e4363872ca
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-25 19:58:36 +03:00
alexey-milovidov
08e395bbcd
Merge pull request #29342 from ClickHouse/fix-typo-in-comment 
Fix typo in comment
 2021-09-25 19:44:59 +03:00
alexey-milovidov
85fc160eb4
Merge pull request #29346 from ClickHouse/remove-ya-make 
Remove ya.make
 2021-09-25 19:43:40 +03:00
alexey-milovidov
87d9506bb0
Merge pull request #29348 from ClickHouse/remove-nth-value 
Remove window function 'nth_value'
 2021-09-25 19:43:02 +03:00
alexey-milovidov
9dac348893
Update StorageMerge.cpp 2021-09-25 19:41:50 +03:00
Azat Khuzhin
fb584715e1 Pass through initial query_id for clickhouse-benchmark 2021-09-25 18:05:55 +03:00
kssenii
30b92d24f1 Fix 2021-09-25 14:46:03 +00:00
kssenii
2129230b1e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-09-25 09:16:05 +00:00
kssenii
f3784780fd Fixes 2021-09-25 08:46:57 +00:00
Nikolai Kochetov
ef425310ff Try use empty sink 2021-09-25 11:36:52 +03:00
Kseniia Sumarokova
b262070eba
Merge pull request #29340 from ClickHouse/fix-msan-disk-web 
Fix trivial mistake in DiskWebServer
 2021-09-25 11:05:29 +03:00
Amos Bird
34017c942e
More fixes 2021-09-25 13:38:56 +08:00
Alexey Milovidov
4f7830ae3a Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-25 06:58:30 +03:00
Alexey Milovidov
5c5d9afd18 Merge branch 'yandex-to-clickhouse-in-configs' of github.com:yandex/ClickHouse into yandex-to-clickhouse-in-configs 2021-09-25 06:58:23 +03:00
Alexey Milovidov
fe44be522e Fix assert in table function merge with database regexp 2021-09-25 05:48:24 +03:00
Alexey Milovidov
06fc466438 Remove window function 'nth_value' 2021-09-25 04:39:38 +03:00
Alexey Milovidov
cd7f9d981c Remove ya.make 2021-09-25 04:22:54 +03:00
Anton Popov
7628273df4
Merge pull request #29266 from CurtizJ/normalize-asts 
Normalize ASTs in ColumnsDescription
 2021-09-25 04:01:54 +03:00
mergify[bot]
e2d5102259
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-25 00:18:48 +00:00
alexey-milovidov
62d606bb76
Update DiskWebServer.h 2021-09-25 02:58:12 +03:00
Alexey Milovidov
686989f4f1 Remove some uglyness 2021-09-25 02:34:50 +03:00
Alexey Milovidov
566797807f Remove assorted trash 2021-09-25 02:32:03 +03:00
Alexey Milovidov
a2c2b3a866 Whitespaces 2021-09-25 02:12:26 +03:00
Alexey Milovidov
3284b2c8de Fix typo in comment 2021-09-25 02:10:10 +03:00
Maksim Kita
63aef6ff95
Merge pull request #29335 from kitaisreal/function-h3-get-faces-optimization 
Function h3GetFaces optimization
 2021-09-25 01:54:17 +03:00
Alexey Milovidov
82d57c6e47 Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-25 01:38:39 +03:00
Maksim Kita
830eae6d1f
Merge pull request #29310 from kitaisreal/parser-number-binary-literal-update 
Parser number binary literal update
 2021-09-25 01:35:22 +03:00
Alexey Milovidov
4a44bcaced Fix trivial mistake in DiskWebServer 2021-09-25 01:33:40 +03:00
Alexey Milovidov
7e3ed5828e Fix trivial mistake in DiskWebServer 2021-09-25 01:33:17 +03:00
Amos Bird
23d3d894e6
Fix projection merges and mutations. 2021-09-24 22:45:50 +08:00
alesapin
3614fb9e37
Merge pull request #29223 from ClickHouse/compressed_logs 
Compressed logs for keeper
 2021-09-24 16:03:26 +03:00
Filatenkov Artur
aaf6785359
Merge branch 'ClickHouse:master' into add_CORS 2021-09-24 15:40:37 +03:00
Filatenkov Artur
7bbd08cb5d
Update HTTPHandler.cpp 2021-09-24 15:40:27 +03:00
Vladimir C
908ae1dc12
Merge pull request #29075 from vdimir/quantile-overflow 2021-09-24 15:03:15 +03:00
Nikolai Kochetov
193f566568 Fix exception message. 2021-09-24 14:54:39 +03:00
Anton Popov
62dea79b78 better formatting 2021-09-24 14:41:49 +03:00
Maksim Kita
5af3f617f4 Fixed tests 2021-09-24 14:19:46 +03:00
Anton Popov
4017b1eb89
Update src/Storages/ColumnsDescription.cpp 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2021-09-24 14:08:33 +03:00
Maksim Kita
edeaa6d6d8 Function h3GetFaces optimization 2021-09-24 14:06:36 +03:00
Nikolai Kochetov
9d3e8fb9c3 Tiny fix. 2021-09-24 13:41:52 +03:00
Nikolai Kochetov
cc53ba283d Try fix other tests. 2021-09-24 13:23:28 +03:00
Alexander Tokmakov
165765659d fix assertion in DDLDependencyVisitor 2021-09-24 13:22:22 +03:00
Vasily Nemkov
1d20014d5d Fixed clang12 build and updated tests 2021-09-24 13:11:00 +03:00
vdimir
db5477e634
Fix style in QuantileTDigest.h 2021-09-24 13:08:46 +03:00
vdimir
58a756da9e
Handle overflow in quantileTDigest 2021-09-24 13:08:45 +03:00
Vladimir C
604dc8861d
Apply suggestions from code review 
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
 2021-09-24 13:07:24 +03:00
Nikolai Kochetov
e617aeb7a5
Merge pull request #29139 from amosbird/projection-fix17 
Disable projections when ARRAY JOIN is used.
 2021-09-24 11:15:23 +03:00
mergify[bot]
4d983441f1
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-23 22:09:48 +00:00
bharatnc
52289fa41e Merge remote-tracking branch 'upstream/master' into ncb/complete-h3-inspection-funcs 2021-09-23 14:51:13 -07:00
Maksim Kita
5047e1a452 Parser number binary literal update 2021-09-24 00:35:41 +03:00
Maksim Kita
1ebb49d939 Revert parse number binary literal 2021-09-24 00:23:15 +03:00
bharatnc
de65461158 Merge remote-tracking branch 'upstream/master' into ncb/complete-h3-inspection-funcs 2021-09-23 13:44:36 -07:00
bharatnc
b13edbfee8 Cast dst_data to ColumnUInt8 2021-09-23 12:37:12 -07:00
Vladimir Chebotarev
0343106692
Fixed logging level for message in S3Common.cpp. 2021-09-23 21:47:31 +03:00
bharatnc
53a2ebead5 Use const auto * and & 2021-09-23 11:01:38 -07:00
Maksim Kita
193fbf5db0 Parser number binary literal 2021-09-23 21:00:16 +03:00
alexey-milovidov
b8083929a6
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-23 20:49:40 +03:00
Amos Bird
983ee004c7
Add empty key description builder. 2021-09-24 01:35:41 +08:00
bharatnc
f02d81be24 Merge remote-tracking branch 'upstream/master' into ncb/complete-h3-inspection-funcs 2021-09-23 08:59:39 -07:00
bharatnc
494c59d2c0 Add more review fixes for h3GetFaces 2021-09-23 08:58:53 -07:00
vdimir
9ca9a66a70
Fix race in StorageSystemRocksDB truncate 2021-09-23 17:03:33 +03:00
alesapin
eb3a651bab Fix bug found on review 2021-09-23 16:55:50 +03:00
Alexey Boykov
a429cbe4c9 Style 2021-09-23 16:42:06 +03:00
vdimir
81a051e3ee
Use dynamic_pointer_cast instead of dynamic_cast in StorageSystemRocksDB 2021-09-23 16:40:10 +03:00
Anton Popov
0390035963 fix unit test 2021-09-23 16:11:31 +03:00
Alexey Boykov
e0963c7666 Cosine null fix 2021-09-23 15:54:00 +03:00
Maksim Kita
23f865c722
Merge pull request #29282 from azat/fix-connection-timeout 
Fix connection timeouts (send_timeout/receive_timeout)
 2021-09-23 15:30:53 +03:00
Kruglov Pavel
55553026ee
Merge pull request #29206 from azat/strict-external-queries 
Add ability to disable converting expressions to local filter for external queries
 2021-09-23 13:22:12 +03:00
Nikolai Kochetov
998d29ebc7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-23 13:10:27 +03:00
Nikolai Kochetov
562ce8c7c2 Fix virtuals. 2021-09-23 13:01:52 +03:00
Kruglov Pavel
6461391387
Merge pull request #29077 from Avogar/fix-msgpack 
Improve MsgPack input/output format
 2021-09-23 12:46:36 +03:00
Maksim Kita
3d79812427 Merge branch 'master' into full-text-bloom-filter-index-map-data-type 2021-09-23 12:38:03 +03:00
Maksim Kita
0341b99c69
Merge pull request #29280 from kitaisreal/full-text-bloom-filter-added-support-for-array-data-type 
FullText bloom filter index added support for Array data type
 2021-09-23 12:34:06 +03:00
Maksim Kita
313f967204 UserDefinedExecutableFunction fix debug message 2021-09-23 12:14:21 +03:00
Maksim Kita
d9e265df2c Fixed debug message 2021-09-23 12:13:01 +03:00
Maksim Kita
d59986f224
Merge pull request #29278 from kitaisreal/bloom-filter-map-added-support-for-has-function 
Bloom filter map added support for has function
 2021-09-23 12:03:56 +03:00
kssenii
6c1312760f Remove debug 2021-09-23 07:52:52 +00:00
Kseniia Sumarokova
f7ff14b407
Merge pull request #28577 from kssenii/named-collections 
Predefined connections and other parameters for external data sources
 2021-09-23 10:25:23 +03:00
kssenii
d71b6ad657 Fix 2021-09-23 07:21:08 +00:00
Azat Khuzhin
015695b3bf Fix connection timeouts (send_timeout/receive_timeout) 
Query is not executed within sendQuery() function.

INSERT query consist from multiple parts:
- sendQuery()
- write()/writePrepared() -- send Data blocks
- writeSuffix() -- send empty Data block and receive EndOfStream
(see RemoteBlockOutputStream for more info)
So as you can see it is not executed completely from the sendQuery() function.

SELECT query (and others) also send Data blocks to the client after.

And what this means that temporary timeout (via TimeoutSetter) is not
enough, since next query can use timeout from the previous query.

Usually you do not see it, because:
- You don't use custom send_timeout/receive_timeout
- SELECT is fast enough

However it is not the case with INSERT, especially with
insert_distributed_sync=1, since
DistributedBlockOutputStream::writeSuffix() may stack because remote
server still INSERT'ing the data (i.e. creating znodes in zookeeper for
ReplicatedMergeTree, syncing blocks, and similar) that was issued from
DistributedBlockOutputStream::write(), and it will lead to the following
exception:

    Code: 209, e.displayText() = DB::NetException: Timeout exceeded while reading from socket (127.1:9000): while receiving packet from ch-47-drt.dpa.semrush.net:9000: Insertion status:
    Wrote X blocks and Y rows on shard Z replica 0, 127.1:9000 (average 0 ms per block, the slowest block N ms)
    0. DB::Exception::Exception()
    1. DB::ReadBufferFromPocoSocket::nextImpl()
    2. void DB::readVarUIntImpl<false>(unsigned long&, DB::ReadBuffer&)
    3. DB::Connection::receivePacket()
    4. DB::RemoteBlockOutputStream::writeSuffix()
    ...
 2021-09-23 09:03:13 +03:00
bharatnc
727182c3a4 Add review fixes 2021-09-22 22:50:33 -07:00
Maksim Kita
ebbefd49c3 FullText bloom filter index added support for Array data type 2021-09-23 02:06:54 +03:00
Vasily Nemkov
cb1ca9b33e Implemented modifying table comments with ALTER TABLE t MODIFY COMMENT 'value' 2021-09-23 01:14:00 +03:00
Maksim Kita
d5a7e92b97 Bloom filter map added support for has function 2021-09-23 01:10:14 +03:00
kssenii
04575eb12b Review fixes, fix style check and clang-tidy 2021-09-22 21:35:29 +00:00
Maksim Kita
1bf375e2b7
Merge pull request #29254 from kitaisreal/map-bloom-filter-index-updated 
Updated bloom filter index for map type to work with mapKeys, mapValues functions
 2021-09-23 00:31:14 +03:00
Maksim Kita
2920e124f3
Merge pull request #29267 from kitaisreal/function-has-support-map-type 
Function has support map type
 2021-09-23 00:26:58 +03:00
Nikita Mikhaylov
32aa2ca939
Merge pull request #29257 from nikitamikhaylov/race-merge-task 
Fix segfault in fast test
 2021-09-23 00:12:31 +03:00
kssenii
ec24a8f114 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-09-22 20:46:49 +00:00
mergify[bot]
d69fc35dd9
Merge branch 'master' into compressed_logs 2021-09-22 19:43:16 +00:00
Nikolai Kochetov
4f802d1cea Fix test_executable_table_function 2021-09-22 22:31:12 +03:00
hhell
fc166570d8
Fix typo in an error message 
Related: https://github.com/ClickHouse/ClickHouse/issues/20551
 2021-09-22 21:33:15 +03:00
Maksim Kita
7eca0a1486
Merge pull request #29258 from vdimir/fuse-quantileBFloat16Weighted 
Add NameQuantileBFloat16Weighted to GatherFunctionQuantileVisitor
 2021-09-22 20:38:55 +03:00
Maksim Kita
93208fca4b Bloom filter for map data type dded support for IN, mapContains functions 2021-09-22 20:25:10 +03:00
Artur
e5c49c2e86 Merge branch 'add_CORS' of https://github.com/FArthur-cmd/ClickHouse into add_CORS 2021-09-22 17:15:54 +00:00
Artur
ce4193fe95 small refactoring 2021-09-22 17:10:15 +00:00
Filatenkov Artur
1fe4555dec
Merge branch 'ClickHouse:master' into add_CORS 2021-09-22 19:55:15 +03:00
Artur
2ad7641e7f activate add_http_cors_header by default. 2021-09-22 16:54:36 +00:00
Kseniia Sumarokova
b41a446e8b
Delete getEngineConnectionArguments.h 2021-09-22 19:52:55 +03:00
Maksim Kita
c73808615b Function has support map type 2021-09-22 19:08:08 +03:00
Anton Popov
8943389d03 normalize ASTs in ColumnsDescription 2021-09-22 18:55:20 +03:00
Kruglov Pavel
b46e4db70e
Fix PVS check 2.0 2021-09-22 18:22:34 +03:00
kssenii
a84fc77b52 Fixes 2021-09-22 18:10:25 +03:00
tavplubix
ba82781e11
Merge pull request #29061 from ClickHouse/remove_some_trash 
Fix `system.distributed_ddl_queue`
 2021-09-22 16:58:29 +03:00
mergify[bot]
fd39fc58d7
Merge branch 'master' into yandex-to-clickhouse-in-configs 2021-09-22 13:35:20 +00:00
mergify[bot]
c5ef025b64
Merge branch 'master' into fix-msgpack 2021-09-22 13:33:05 +00:00
Nikolai Kochetov
eed4e8c754 Fix progress for insert select. 2021-09-22 16:29:58 +03:00
vdimir
eb1053d931
Add NameQuantileBFloat16Weighted to GatherFunctionQuantileVisitor 2021-09-22 15:53:39 +03:00
mergify[bot]
a3f62af09c
Merge branch 'master' into race-merge-task 2021-09-22 12:42:45 +00:00
Artur
36223b5f23 update comments for filter 2021-09-22 12:30:55 +00:00
Artur
2cffa98a60 add test and comments in config 2021-09-22 12:22:21 +00:00
Nikita Mikhaylov
558c74e826 Done 2021-09-22 11:52:47 +00:00
Nikolai Kochetov
81bf13a247 Refactor pushing to views. 2021-09-22 13:57:00 +03:00
Kruglov Pavel
7545bff76c
Minor update 2021-09-22 13:47:03 +03:00
alesapin
258ce2436a
Merge pull request #29237 from CurtizJ/better-logging 
Better diagnostic in ReplicatedMergeTree
 2021-09-22 13:45:10 +03:00
alesapin
f980a414ee Better tests and ability to work without compression 2021-09-22 13:38:06 +03:00
Artur
c8892ec7a7 add options support 2021-09-22 10:34:48 +00:00
mergify[bot]
627a4fea96
Merge branch 'master' into fix-msgpack 2021-09-22 10:14:18 +00:00
Pavel Kruglov
9ed82fb44a Fix PVS check 2021-09-22 13:13:05 +03:00
Maksim Kita
92fd6b376f Updated bloom filter index for map type to work with mapKeys function 2021-09-22 12:45:48 +03:00
Kseniia Sumarokova
192c5397cc
Merge pull request #29211 from kssenii/better-retries-2 
Fix retries for disk web
 2021-09-22 12:37:25 +03:00
alesapin
3a11cc59b0 Rename back 2021-09-22 12:16:56 +03:00
kssenii
b51840a205 Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections 2021-09-22 11:45:22 +03:00
Nikolai Kochetov
d018cb7adf Fry fix executor again. 2021-09-22 11:30:02 +03:00
alesapin
043ef03d42 revert method name 2021-09-22 10:30:15 +03:00
Nikolai Kochetov
2c7239da66 Fix executor. 2021-09-22 09:49:59 +03:00
bharatnc
bdd9a4a352 Fix style check 2021-09-21 23:40:56 -07:00
万康
7201b2476e fix spell error 2021-09-22 13:26:05 +08:00
bharatnc
8336847be6 Merge remote-tracking branch 'upstream/master' into ncb/complete-h3-inspection-funcs 2021-09-21 17:43:47 -07:00
bharatnc
208dfc998a H3 - fix logic to return icosahedron faces 2021-09-21 17:39:59 -07:00
Alexey Milovidov
e9e77b4403 .tech -> .com 2021-09-22 03:22:57 +03:00
bharatnc
8203c5462a Fix a small comment 2021-09-21 16:45:29 -07:00
Alexey Milovidov
4ade6cc5da Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs 2021-09-22 02:08:29 +03:00
bharatnc
32baa1ddae Fix getH3Faces logic 2021-09-21 14:08:09 -07:00
alexey-milovidov
3203fa4c34
Merge pull request #28914 from ClickHouse/exp-smooth 
Add aggregate function for exponential smoothing
 2021-09-21 23:52:44 +03:00
alesapin
334a7ab8db Change default extension to zst 2021-09-21 23:43:44 +03:00
alexey-milovidov
c6dd891471
Merge pull request #29180 from azat/socket-timeout-to-message 
Add socket timeout values into SOCKET_TIMEOUT exception
 2021-09-21 23:41:30 +03:00
alesapin
eec44929f1 Fix clang tidy 2021-09-21 23:40:14 +03:00
Alexey Milovidov
d5a96d7725 Fix error 2021-09-21 23:34:56 +03:00
Anton Popov
0fbfac056d better diagnostic in ReplicatedMergeTree 2021-09-21 22:42:03 +03:00
Alexey Milovidov
213ecaedde Fix issues with merging configs 2021-09-21 21:51:59 +03:00
Maksim Kita
84e37c2d87 Fixed tests 2021-09-21 21:28:52 +03:00
Alexey Milovidov
1f74a98baf Merge branch 'master' of github.com:yandex/ClickHouse 2021-09-21 21:05:48 +03:00
Alexey Milovidov
087d0683c9 Updates to YAMLParser and FileChecker 2021-09-21 21:05:08 +03:00
Maksim Kita
19d70158fe
Merge pull request #29215 from ClickHouse/bloom-filter-index-map-data-type 
Merging #28634
 2021-09-21 20:31:14 +03:00
tavplubix
e54bd40102
Merge pull request #29216 from ClickHouse/fix_29122 
Fix terminate on uncaught exception
 2021-09-21 20:12:51 +03:00
Nikolai Kochetov
912393a7ed Fix pvs check. 2021-09-21 19:55:32 +03:00
Nikolai Kochetov
db6f1e198c Add cancell callback to completed executor. 2021-09-21 19:37:32 +03:00
alesapin
5a8c801635 Remove unused variable 2021-09-21 18:50:21 +03:00
alesapin
ac7579f187 Compressed logs for keeper 2021-09-21 17:29:05 +03:00
Maksim Kita
6dadc521fb MergeTreeIndexFullText updated for data type map 2021-09-21 16:55:48 +03:00
凌涛
e3e3b84f63 fix tests 2021-09-21 16:55:48 +03:00
凌涛
9461d6dbf2 modify code style 2021-09-21 16:55:47 +03:00
凌涛
e84f1000fb optimze code to make sure only map function will be processed 2021-09-21 16:55:47 +03:00
凌涛
a3e657778a bloom filter skipping index support Map with key of String type 2021-09-21 16:55:07 +03:00
凌涛
8f87d44958 bloom filter skipping index support Map with key of String type 2021-09-21 16:55:07 +03:00
Nikita Mikhaylov
7878d848ab
Merge pull request #29207 from bharatnc/ncb/run-ya-make 
Update outdated ya.make files
 2021-09-21 16:07:56 +03:00
Maksim Kita
15c9315ea1 MergeTreeIndexConditionBloomFilter updated for Map data type 2021-09-21 15:07:59 +03:00
Kseniia Sumarokova
17b1dd4bc6
Merge branch 'master' into better-retries-2 2021-09-21 13:55:53 +03:00
Anton Popov
ea4fd19e28
Merge pull request #29087 from CurtizJ/asyn-inserts-follow-up 
Minor enhancements in async inserts
 2021-09-21 13:38:52 +03:00
Nikolai Kochetov
02e5e662fd Refactor a little bit. 2021-09-21 13:35:41 +03:00
kssenii
975033e6e3 Fix 2021-09-21 09:44:54 +00:00
Vladimir C
bc7c41efff
Merge pull request #26657 from hexiaoting/fuse_quantile 2021-09-21 12:33:07 +03:00
Vladimir C
9abd8aea6d
Merge pull request #29123 from depressed-pho/csv-custom-null 2021-09-21 11:35:21 +03:00
Alexander Tokmakov
7e346ec6ec fix terminate on uncaught exception 2021-09-21 11:04:22 +03:00
Nikolai Kochetov
0308f14bea Fix another one test. 2021-09-21 11:01:00 +03:00
fuwhu
8962672e59 add bloom filter skip index for map data type. 2021-09-21 10:59:16 +03:00
Azat Khuzhin
23b3085c8b Add more tests for external_table_strict_query 2021-09-21 10:53:22 +03:00
Nikolai Kochetov
12a4c74af0 Fix more tests again 2021-09-21 09:57:55 +03:00
bharatnc
b4e86e98db Fix typo 2021-09-20 22:57:43 -07:00
tavplubix
5f4c665c76
Merge pull request #29201 from ClickHouse/follow_up_to_28373 
Follow up to #28373
 2021-09-21 08:26:15 +03:00
bharatnc
49e6023d65 H3 - add func H3GetFaces 2021-09-20 19:20:02 -07:00
bharatnc
536f3a52d9 H3 - Add func H3IsPentagon 2021-09-20 18:50:45 -07:00
bharatnc
8a8ff94b99 Update ya.make file 2021-09-20 18:32:26 -07:00
bharatnc
51e45050c1 Update outdated ya.make files 
This runs the `utils/generate-ya-make/generate-ya-make.sh` script to
update the ya.make files that are not updated.

I wonder why they were not updated and how it was missed.
 2021-09-20 18:28:07 -07:00
bharatnc
1766702a60 H3 - add func ResIsClassIII 2021-09-20 18:18:10 -07:00
Azat Khuzhin
9c20519f33 Add ability to disable converting expressions to local filter for external queries 
Sometimes it is undesirable to remove any expressions from WHERE, since
this may lead to reading the whole table, and this is pretty heavy job
for MySQL and PostgreSQL (even if you read only one column).

So external_table_strict_query had been introduced to prohibit this and
fail the query instead.
 2021-09-21 02:41:07 +03:00
Maksim Kita
1932d29ed1
Merge pull request #29199 from kitaisreal/parser-system-query-fix 
ParserSystemQuery fix
 2021-09-21 02:23:51 +03:00
Maksim Kita
ad2052e8e5 Added comments 2021-09-21 02:23:34 +03:00
Nikolai Kochetov
4e7d5191e3 Fix more tests 2021-09-20 22:07:33 +03:00
Maksim Kita
494ec4aecc Fixed tests 2021-09-20 21:33:25 +03:00
Anton Popov
c48a93a62f fix integer underflow in function 'quantile' 2021-09-20 21:21:40 +03:00
Nikolai Kochetov
d391e3419e Fixing build. 2021-09-20 20:54:01 +03:00
Anton Popov
83e45adad6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-20 20:04:50 +03:00
Anton Popov
55cf857aba
Merge pull request #28707 from azat/max_suspicious_broken_parts_bytes 
Implement max_suspicious_broken_parts_bytes
 2021-09-20 19:55:07 +03:00
Alexander Tokmakov
4087de5439 follow up to 28373 2021-09-20 19:44:00 +03:00
Nikolai Kochetov
a790d391c0 Merge branch 'master' into rewrite-pushing-to-views 2021-09-20 19:43:15 +03:00
Nikolai Kochetov
c7533d58ee Add debug output. 2021-09-20 19:09:39 +03:00
Maksim Kita
56e40e1ff4 ParserSystemQuery fix 2021-09-20 18:15:23 +03:00
Nikolai Kochetov
211bf07e68 Another try to fix crash. 2021-09-20 18:10:21 +03:00
tavplubix
8adfb9b593
Stop ThreadFuzzer before hung check (#29167) 
* stop ThreadFuzzer before hung check

* fix

* fix
 2021-09-20 17:23:10 +03:00
Nikolai Kochetov
00dba6e745 Fix crash in kill 2021-09-20 16:08:52 +03:00
tavplubix
66bc619c3e
Merge pull request #28983 from ClickHouse/func_zookeeper_session_uptime 
Add function zookeeperSessionUptime(), fix some flaky tests
 2021-09-20 14:47:49 +03:00
tavplubix
922cf7ee20
Merge pull request #28373 from ClickHouse/tables_topsort 
Resolve table dependencies on metadata loading
 2021-09-20 14:46:47 +03:00
alesapin
a249dcc5f5
Merge pull request #28301 from kssenii/materialized-postgresql 
MaterializedPostgreSQL: allow dynamically adding/deleting tables, altering settings
 2021-09-20 14:16:29 +03:00
alesapin
f983381e45
Merge pull request #29030 from nicelulu/fix-keeper-converter 
keeper-converter add the last logfile that is less than the zxid.
 2021-09-20 13:49:07 +03:00
Nikolai Kochetov
729e530fe7 Try fix crash. 2021-09-20 13:36:01 +03:00
Nikolai Kochetov
4eba3fcc8a Fix even more tests. 2021-09-20 12:48:53 +03:00
alexey-milovidov
5f5b96aa57
Merge pull request #28982 from vdimir/issue-28660-pmj-fuzz 
Add additional checkTypesOfKeys to JoiningTransform::transformHeader
 2021-09-20 12:30:00 +03:00
Maksim Kita
c2088678ed
Merge pull request #28803 from kitaisreal/executable-user-defined-functions 
Executable user defined functions
 2021-09-20 12:11:15 +03:00
Nikolai Kochetov
9398c22aae Fix more tests. 2021-09-20 12:05:34 +03:00
vdimir
091ce151bf
Avoid cycles in optimizeFuseQuantileFunctions 2021-09-20 11:39:29 +03:00
Azat Khuzhin
e5f66fd103 Add socket timeout values into SOCKET_TIMEOUT exception 2021-09-20 09:51:42 +03:00
kssenii
8d19ed5ee7 Fix remaining tests 2021-09-20 05:39:13 +00:00
Kseniia Sumarokova
2a8f355326
Merge pull request #29179 from kssenii/better-retries 
Better retries for disk web
 2021-09-20 08:25:49 +03:00
kssenii
12b90da5de Better retries 2021-09-19 20:50:50 +00:00
Nikolai Kochetov
2b7d71d3a2 Fix some other tests. 2021-09-19 23:15:10 +03:00
Alexey Milovidov
894f4908fa Add comments 2021-09-19 22:26:16 +03:00
Nikolai Kochetov
c1f0ba1f92 Fix some tests. 2021-09-19 21:53:36 +03:00
kssenii
db45363c19 Fix tests 2021-09-19 18:42:45 +00:00
Alexey Milovidov
edb4c58ecc Fix build 2021-09-19 21:13:00 +03:00
Alexey Milovidov
322b67a533 Clarifications 2021-09-19 20:58:10 +03:00
Alexey Milovidov
04c5574ce6 Add comments 2021-09-19 20:51:35 +03:00
Alexey Milovidov
7d24e110e4 Make something with more sense 2021-09-19 20:23:06 +03:00
Nikolai Kochetov
53e946ec25 Fix executors. 2021-09-19 19:18:38 +03:00
Alexey Milovidov
b444d224b0 Merge branch 'master' into exp-smooth 2021-09-19 17:58:57 +03:00
Anton Popov
e3e6258b95
Merge pull request #29163 from CurtizJ/small-refactoring 
Slightly better code near reading of subcolumns
 2021-09-19 16:14:52 +03:00
Anton Popov
db50a1027e
Merge pull request #29132 from Avogar/fix-avro 
Fix segfault in Avro with LowCardinality(Nullable)
 2021-09-19 16:13:11 +03:00
Anton Popov
f50098a059
fix style check 2021-09-19 00:13:00 +03:00
Anton Popov
1c80a95b36 slightly better code near reading of subcolumns 2021-09-18 22:34:54 +03:00
alexey-milovidov
20d8523a2e
Merge pull request #28888 from azat/mysql-in-fix 
Fix queries to external databases (i.e. MySQL) with multiple columns in IN ( i.e. `(k,v) IN ((1, 2))` )
 2021-09-18 15:53:40 +03:00
alexey-milovidov
e16e8fa286
Update ExpressionElementParsers.cpp 2021-09-18 15:47:57 +03:00
alexey-milovidov
3eed1f06ea
Merge pull request #28952 from fuzzERot/h3-functions 
Implementation of h3ToGeoBoundary
 2021-09-18 14:27:11 +03:00
Artur
82a849ba8e add options method 2021-09-18 09:36:02 +00:00
Amos Bird
6b3cd34cff
Disable projections when ARRAY JOIN is used. 2021-09-18 08:15:59 +08:00
tavplubix
749b91347d
Update PostgreSQLDictionarySource.cpp 2021-09-17 22:39:54 +03:00
Alexander Tokmakov
3bca886174 remove some trash 2021-09-17 22:29:10 +03:00
Azat Khuzhin
b108c5e1b0 Implement max_suspicious_broken_parts_bytes 
Usually broken parts are found from just created parts, which are
usually small enough, so it make sense to have a limit in bytes for
broken parts.

Default value was set to 1G.
 2021-09-17 22:06:32 +03:00
Nikolai Kochetov
a8443bef4d Fix build. 2021-09-17 20:52:26 +03:00
Maksim Kita
784a4a33e0 Fixed tests 2021-09-17 18:43:00 +03:00
Maksim Kita
85a4d4bb50 Added user defined executable functions to system.functions 2021-09-17 18:43:00 +03:00
Maksim Kita
ca395e9678 Fixed tests 2021-09-17 18:43:00 +03:00
Maksim Kita
ca3fc352d6 Fixed tests 2021-09-17 18:43:00 +03:00
Maksim Kita
581f051f37 Fixed build 2021-09-17 18:43:00 +03:00
Maksim Kita
769fab62e3 UserDefinedExecutableFunction added send_chunk_header option 2021-09-17 18:43:00 +03:00
Maksim Kita
3852954b49 Fixed build 2021-09-17 18:43:00 +03:00
Maksim Kita
bb3bc6722a UserDefinedExecutableFunction added support for process pool 2021-09-17 18:43:00 +03:00
Maksim Kita
93ecbf3ae4 Added tests 2021-09-17 18:42:59 +03:00
Maksim Kita
de20e04dfd Added executable user defined functions 2021-09-17 18:42:59 +03:00
alesapin
4adc4e9152
Merge pull request #28817 from ClickHouse/fix_part_checker 
Fix rare replicas diverge
 2021-09-17 18:06:27 +03:00
Pavel Kruglov
aba86b010b Update docs, tests and add some fixes 2021-09-17 17:32:23 +03:00
Vladimir Chebotarev
db516e8c91
Removed sub-optimal mutation notifications in StorageMergeTree when merges are still going (#27552) 
* Removed redundand mutation notifications in `StorageMergeTree` when merges are still going.
 2021-09-17 16:19:36 +03:00
Pavel Kruglov
7bf13c4891 Remove extra space 2021-09-17 16:13:15 +03:00
Pavel Kruglov
48a3a2f75c Fix segfault in Avro with LowCardinality(Nullable) 2021-09-17 16:11:44 +03:00
Maksim Kita
14e4d49601
Merge pull request #29063 from ClickHouse/read-integer-overflow-bug-fix 
Merging #27963
 2021-09-17 16:06:58 +03:00
Nikolai Kochetov
618d4d863e Restore InterpreterInsertQuery 2021-09-17 15:59:40 +03:00
Nikolai Kochetov
a8c3b02598 Merge branch 'master' into rewrite-pushing-to-views 2021-09-17 15:38:11 +03:00
Nikolai Kochetov
6e0ac024a2 Maybe fix some build. 2021-09-17 15:05:54 +03:00
Vladimir C
333fd323f5
Merge pull request #29062 from Avogar/orc-fixes 2021-09-17 14:48:19 +03:00
Vladimir C
4eaf2969e2
Merge pull request #28332 from OodounaikingoO/master 2021-09-17 14:46:20 +03:00
Nikolai Kochetov
58bb5fe462 Fix some tests. 2021-09-17 14:40:03 +03:00
anneji
258159e13d add default database for alter table 2021-09-17 17:30:17 +08:00
Maksim Kita
73368aa351 Fixed tests 2021-09-17 12:17:19 +03:00
PHO
3c4b1ea9c5 New setting: output_format_csv_null_representation 
This is the same as output_format_tsv_null_representation but is for CSV output.
 2021-09-17 17:58:23 +09:00
alesapin
d0b5485773
Merge pull request #29071 from nicelulu/fix_deserializeCheckVersionTxn 
Fix deserializeCheckVersionTxn version.
 2021-09-17 10:40:02 +03:00
mergify[bot]
c23f1fc160
Merge branch 'master' into fix_part_checker 2021-09-17 07:23:12 +00:00
PHO
5a4985fe27 Do not initialize std::pair with a std::tuple<T1, T2> 
GCC/libstdc++ does not allow this and emits a compilation error. The standard does not say they are supposed to be interchangeable.
 2021-09-17 11:12:05 +09:00
Nikita Mikhaylov
c52b8ec083
Introduced MergeTask and MutateTask (#25165) 
Introduced MergeTask and MutateTask
 2021-09-17 00:19:58 +03:00
Kseniia Sumarokova
e9d44d0848
Merge pull request #28797 from kssenii/rabbit-fix 
Rewrite StorageRabbitMQ a little
 2021-09-16 23:58:51 +03:00
Azat Khuzhin
60013cff1a Do not call finalize() for HTTP buffer multiple times 
Before only non-compressed was flushed, and so after compressed will
write to non-compressed and this will trigger multiple finalize() for
raw buffer, fix this, but adding Output::finalize() that take care of
all buffers.

This should fix CI failure [1] and also not only CI failures but also an
assert() that had been added in the previous commit.

    2021-09-05 15:08:43 00429_long_http_bufferization:                                          [ FAIL ] 6.68 sec. - return code 1
    2021-09-05 15:08:43 cmp: EOF on /tmp/clickhouse-test/0_stateless/test_e4xs61/res3 after byte 1048576, in line 4127
    2021-09-05 15:08:43 , result:

    The problem was with the following query:

        curl -q -sS 'http://localhost:8123/?database=default&log_comment=/src/ch/clickhouse/tests/queries/0_stateless/00429_long_http_bufferization.sh&max_block_size=500000&compress=1&buffer_size=1048576&wait_end_of_query=1' -d 'SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 1500000 FORMAT RowBinary' | clickhouse-compressor --decompress | less

    It returns less rows.

  [1]: https://clickhouse-test-reports.s3.yandex.net/28604/1f34fd9a9eaff19fde0199f78b45656ab74f0590/functional_stateless_tests_(release).html#fail1
 2021-09-16 22:36:09 +03:00
Azat Khuzhin
9bb9d5952b Catch write-after-finalize bugs in WriteBufferFromHTTPServerResponse 2021-09-16 22:35:51 +03:00
Azat Khuzhin
7b741c28e4 Do not try to write HTTP response after exception 
CI reports [1]:

    [ 25285 ] {} <Trace> DynamicQueryHandler: Request URI: /?database=test_27&log_comment=/usr/share/clickhouse-test/queries/0_stateless/00429_long_http_bufferization.sh&max_block_size=500000&compress=1&buffer_size=1500000&wait_end_of_query=1
    ...
    [ 25285 ] {ee96eb9c-44cc-4b76-9f30-3cc545a5506f} <Debug> executeQuery: (from [::1]:41578) (comment: /usr/share/clickhouse-test/queries/0_stateless/00429_long_http_bufferization.sh) SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 1048576 FORMAT RowBinary
    [ 25285 ] {ee96eb9c-44cc-4b76-9f30-3cc545a5506f} <Information> executeQuery: Read 1500000 rows, 11.44 MiB in 63.924507821 sec., 23465 rows/sec., 183.32 KiB/sec.
    [ 25285 ] {ee96eb9c-44cc-4b76-9f30-3cc545a5506f} <Debug> DynamicQueryHandler: Done processing query
    ...
    [ 455 ] {} <Fatal> BaseDaemon: Code: 24. DB::Exception: Cannot write to ostream at offset 2097152. (CANNOT_WRITE_TO_OSTREAM), Stack trace (when copying this message, always include the lines below):
    [ 455 ] {} <Fatal> BaseDaemon:
    [ 455 ] {} <Fatal> BaseDaemon: 0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:0: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x18fa949b in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:59: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x99f38dc in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 2. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromOStream.cpp:0: DB::WriteBufferFromOStream::nextImpl() @ 0x9adf185 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 3. ./obj-x86_64-linux-gnu/../src/IO/BufferBase.h:39: DB::WriteBufferFromHTTPServerResponse::nextImpl() @ 0x15b71ec0 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 4. ./obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:0: DB::WriteBuffer::write(char const*, unsigned long) @ 0x99be96a in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 5. ./obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:36: DB::CompressedWriteBuffer::nextImpl() @ 0x1436ca4d in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../src/IO/BufferBase.h:39: DB::CompressedWriteBuffer::~CompressedWriteBuffer() @ 0x1436cd8f in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2615: std::__1::__shared_ptr_emplace<DB::CompressedWriteBuffer, std::__1::allocator<DB::CompressedWriteBuffer> >::__on_zero_shared() @ 0x15a88ee8 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2518: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x15b0566b in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x15b6ba61 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x18ec71a3 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114: Poco::Net::TCPServerDispatcher::run() @ 0x18ec7930 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/ScopedLock.h:36: Poco::PooledThread::run() @ 0x1903e042 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:56: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x1903c4d0 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/SharedPtr.h:277: Poco::ThreadImpl::runnableEntry(void*) @ 0x1903ac48 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 15. __tsan_thread_start_func @ 0x990a76d in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 16. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    [ 455 ] {} <Fatal> BaseDaemon: 17. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    [ 455 ] {} <Fatal> BaseDaemon:  (version 21.10.1.7997)
    [ 25878 ] {} <Fatal> BaseDaemon: ########################################
    [ 25878 ] {} <Fatal> BaseDaemon: (version 21.10.1.7997, build id: DB9812D7EA89EB06D2C7243F5C8BD293AA9BA37D) (from thread 25285) (query_id: ee96eb9c-44cc-4b76-9f30-3cc545a5506f) Received signal Aborted (6)
    [ 25878 ] {} <Fatal> BaseDaemon:
    [ 25878 ] {} <Fatal> BaseDaemon: Stack trace: 0x7efe2a35818b 0x7efe2a337859 0x990ff6e 0x13fa707d 0x1bb17614 0x1bb17507 0x999c97b 0x1436cf11 0x15a88ee8 0x15b0566b 0x15b6ba61 0x18ec71a3 0x18ec7930 0x1903e042 0x1903c4d0 0x1903ac48 0x990a76d 0x7efe2a50d609 0x7efe2a434293
    [ 25878 ] {} <Fatal> BaseDaemon: 5. raise @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    [ 25878 ] {} <Fatal> BaseDaemon: 6. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    [ 450 ] {} <Fatal> Application: Child process was terminated by signal 6.

  [1]: https://clickhouse-test-reports.s3.yandex.net/28604/d6e86dab2754dbea3ed19d9ae72276e7d161aef1/stress_test_(thread).html#fail1
 2021-09-16 22:35:51 +03:00
Azat Khuzhin
230f29ac4e Always flush buffer while handling HTTP queries 
This will avoid possible std::terminate, like in [1]:

  [1]: https://github.com/ClickHouse/ClickHouse/pull/24023#issuecomment-842979875
 2021-09-16 22:35:50 +03:00
Azat Khuzhin
47ad10061b Reduce scope for HTTPHandler::trySendExceptionToClient() 2021-09-16 22:35:50 +03:00
kssenii
3b4843a1a9 Apply changed from pr 27135 2021-09-16 18:44:42 +00:00
Anton Popov
99175f7acc minor enhancements in async inserts 2021-09-16 20:55:34 +03:00
Nikolai Kochetov
341553febd Fix build. 2021-09-16 20:40:42 +03:00
mergify[bot]
e0376e1e77
Merge branch 'master' into fix_deserializeCheckVersionTxn 2021-09-16 16:18:11 +00:00
kssenii
55619dcc1b Correct merge 2021-09-16 16:14:17 +00:00
kssenii
c88dc460d4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-09-16 16:13:04 +00:00
alesapin
2d9fa85c0a
Merge pull request #28889 from azat/fix-mutation-wait 
Fix waiting for mutation with mutations_sync=2
 2021-09-16 19:10:56 +03:00
alesapin
606ab6d3e4 Typo fix 2021-09-16 18:40:25 +03:00
alesapin
8930a0d2bd More precise initialization 2021-09-16 18:39:11 +03:00
Anton Popov
77a995381b
Merge pull request #28975 from ClickHouse/check-what-if-do-not-add-const-agg-keys-with-having 
Don not add const group by key for query with only having.
 2021-09-16 18:33:53 +03:00
alesapin
b4cbc86164 Fix bug 2021-09-16 17:59:55 +03:00
Kruglov Pavel
50c0045e27
Remove unused header 2021-09-16 17:13:07 +03:00
Pavel Kruglov
96dba14213 Improve MsgPack input/output format 2021-09-16 17:05:07 +03:00
kssenii
7b068317c1 Fix build 2021-09-16 13:55:18 +00:00
vdimir
661a54827b
Do not throw LOGICAL_ERROR from fuse quantile for illegal params 2021-09-16 16:06:02 +03:00