Commit Graph

30504 Commits

Author SHA1 Message Date
Azat Khuzhin
80cf86f100 Cover INSERT into Distributed with MATERIALIZED columns and !is_local node
I guess that adding new cluster into server-test.xml is not required,
but it won't harm.
2019-10-23 21:55:08 +03:00
Azat Khuzhin
ab9d9f8997 Fix INSERT into Distributed non local node with MATERIALIZED columns
Previous patch e527def18a ("Fix INSERT
into Distributed() table with MATERIALIZED column") fixes it only for
cases when the node is local, i.e. direct insert.

This patch address the problem when the node is not local
(`is_local == false`), by erasing materialized columns on INSERT into
Distributed.

And this patch fixes two cases, depends on `insert_distributed_sync`
setting:

- `insert_distributed_sync=0`

    ```
    Not found column value in block. There are only columns: date. Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5d6cf6 DB::Block::getByName(...) dbms/src/Core/Block.cpp:187
    4. 0x7fffec2fe067 DB::NativeBlockInputStream::readImpl() dbms/src/DataStreams/NativeBlockInputStream.cpp:159
    5. 0x7fffec2d223f DB::IBlockInputStream::read() dbms/src/DataStreams/IBlockInputStream.cpp:61
    6. 0x7ffff7c6d40d DB::TCPHandler::receiveData() dbms/programs/server/TCPHandler.cpp:971
    7. 0x7ffff7c6cc1d DB::TCPHandler::receivePacket() dbms/programs/server/TCPHandler.cpp:855
    8. 0x7ffff7c6a1ef DB::TCPHandler::readDataNext(unsigned long const&, int const&) dbms/programs/server/TCPHandler.cpp:406
    9. 0x7ffff7c6a41b DB::TCPHandler::readData(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:437
    10. 0x7ffff7c6a5d9 DB::TCPHandler::processInsertQuery(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:464
    11. 0x7ffff7c687b5 DB::TCPHandler::runImpl() dbms/programs/server/TCPHandler.cpp:257
    ```

- `insert_distributed_sync=1`

    ```
    2019.10.18 13:23:22.114578 [ 44 ] {a78f669f-0b08-4337-abf8-d31e958f6d12} <Error> executeQuery: Code: 171, e.displayText() = DB::Exception: Block structure mismatch in RemoteBlockOutputStream stream: different number of columns:
    date Date UInt16(size = 1), value Date UInt16(size = 1)
    date Date UInt16(size = 0): Insertion status:
    Wrote 1 blocks and 0 rows on shard 0 replica 0, 127.0.0.1:59000 (average 0 ms per block)
    Wrote 0 blocks and 0 rows on shard 1 replica 0, 127.0.0.2:59000 (average 2 ms per block)
     (version 19.16.1.1) (from [::1]:3624) (in query: INSERT INTO distributed_00952 VALUES ), Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5da4e9 DB::checkBlockStructure<void>(...)::{...}::operator()(...) const dbms/src/Core/Block.cpp:460
    4. 0x7fffec5da671 void DB::checkBlockStructure<void>(...) dbms/src/Core/Block.cpp:467
    5. 0x7fffec5d8d58 DB::assertBlocksHaveEqualStructure(...) dbms/src/Core/Block.cpp:515
    6. 0x7fffec326630 DB::RemoteBlockOutputStream::write(DB::Block const&) dbms/src/DataStreams/RemoteBlockOutputStream.cpp:68
    7. 0x7fffe98bd154 DB::DistributedBlockOutputStream::runWritingJob(DB::DistributedBlockOutputStream::JobReplica&, DB::Block const&)::{lambda()#1}::operator()() const dbms/src/Storages/Distributed/DistributedBlockOutputStream.cpp:280
    <snip>
    ````

Fixes: #7365
Fixes: #5429
Refs: #6891
2019-10-23 21:54:27 +03:00
Azat Khuzhin
260a9fba2b StorageDistributed: Drop extra inclusion of materializeBlock.h 2019-10-18 23:38:33 +03:00
Alexander Kuzmenkov
3143fa37d7
Merge pull request #7371 from CurtizJ/fix-optimize-read-in-order
Fix optimize_read_in_order with IN and a subquery.
2019-10-18 15:55:53 +03:00
BayoNet
e1e82cb438
DOCS-8299: Added a link with metrics for system.metrics in RU docs (#7379)
* Typo fix.

* Links fix.

* Fixed links in docs.

* More fixes.

* Link fixes.

* DOCAPI-8299: Link update in RU docs.
2019-10-18 12:25:56 +03:00
hcz
502672c973 Add aggregate function combinators: -OrNull & -OrDefault (#7331) 2019-10-18 12:25:39 +03:00
alesapin
71cbe878fc
Merge pull request #7374 from millb/fixed_getMultipleValuesFromConfig
Fixed function getMultipleValuesFromConfig
2019-10-18 11:55:41 +03:00
alexey-milovidov
3803d94db6
Merge pull request #7376 from amosbird/helpakuzm
Fix ubsan introduced by Field refactoring.
2019-10-18 06:29:27 +03:00
Mikhail Korotov
a1f417a537
Update gtest_getMultipleValuesFromConfig.cpp 2019-10-18 00:07:27 +03:00
Amos Bird
16d1b4472e Fix ubsan introduced by Field refactoring. 2019-10-18 04:30:33 +08:00
millb
26d9ee3ae5 Fixed bug with function getMultipleValuesFromConfig and added test which confirms this. 2019-10-17 19:52:47 +03:00
vzakaznikov
66675edf2d Fixing data race with no_users_thread. (#7353) 2019-10-17 17:47:36 +03:00
vzakaznikov
ee1a2ce027 Fixing AddressSanitizer error in the LIVE VIEW getHeader() method. (#7271) 2019-10-17 17:43:11 +03:00
tavplubix
0046b9f137 Wait for jobs to finish on exception (fixes rare segfaults) (#7350) 2019-10-17 17:41:27 +03:00
Alexander Kuzmenkov
b62c9e729d
Merge pull request #7215 from ClickHouse/formats_improvements
Miscellaneous formats improvements
2019-10-17 17:37:38 +03:00
Alexander Kuzmenkov
3e662da6f4
Merge pull request #7351 from Enmk/http_keep_alive_timeout
Fixed issue of using HTTP timeout as TCP timeout value.
2019-10-17 16:14:29 +03:00
Mikhail Korotov
326dab83d9
Merge pull request #2 from ClickHouse/master
refresh_2
2019-10-17 16:13:44 +03:00
CurtizJ
546414c015 fix optimize_read_in_order with in subquery 2019-10-17 16:11:27 +03:00
Ivan
e595df05d9
Fix CREATE TABLE AS from table with virtual columns (#7183)
* Disable undefined instrumentation for rdkafka
2019-10-17 14:12:35 +03:00
BayoNet
7601fafe63
DOCS-413: Links fixes. (#7362)
* Typo fix.

* Links fix.

* Fixed links in docs.

* More fixes.

* Link fixes.

* CLICKHOUSEDOCS-413: Links fixes.
2019-10-17 13:38:32 +03:00
Alexander Kuzmenkov
e9d7da3dd4
Merge pull request #7359 from amosbird/joingetconst
Constness-aware JoinGet.
2019-10-17 13:35:37 +03:00
alexey-milovidov
6191c10abe
Merge pull request #7361 from excitoon/patch-2
Improved readability a little bit (`MergeTreeData::getActiveContainingPart`).
2019-10-17 06:39:52 +03:00
Amos Bird
89247deb64 Constness-aware JoinGet. 2019-10-17 08:52:28 +08:00
tavplubix
c4b46e113e
Merge branch 'master' into formats_improvements 2019-10-16 23:00:17 +03:00
Alexander Tokmakov
e9e8d055af null_as_default for Values 2019-10-16 22:55:24 +03:00
Alexander Tokmakov
102cbfd25a fix old bug with default values 2019-10-16 22:27:33 +03:00
Vladimir Chebotarev
512e97ddb3
Improved readability a little bit (MergeTreeData::getActiveContainingPart). 2019-10-16 22:01:50 +03:00
Alexander Tokmakov
5f6703130e fixes 2019-10-16 18:28:48 +03:00
Nikolai Kochetov
dfe297b93f
Merge pull request #6570 from godfreyd/godfreyd-clickhouse-catboost
Docs(solutions): applying a CatBoost model in ClickHouse
2019-10-16 17:24:11 +03:00
Alexander Tokmakov
aa9d29d6ae Merge branch 'master' into formats_improvements 2019-10-16 17:15:13 +03:00
Alexander Kuzmenkov
91b995212f
Merge pull request #7343 from amosbird/bf10
Build fix
2019-10-16 16:59:46 +03:00
Vasily Nemkov
94095fcb92 Fixed issue of using HTTP timeout as TCP timeout value. 2019-10-16 16:04:26 +03:00
Alexander Kuzmenkov
16f252dce9
Merge pull request #7332 from Enmk/More_Date_and_DateTime_performance_tests
More performance test for Date and DateTime
2019-10-16 16:03:54 +03:00
Artem Zuikov
b70ca30447
Merge pull request #7335 from 4ertus2/ast
Remove asterisk_left_columns_only setting
2019-10-16 13:57:17 +03:00
Artem Zuikov
3ceeb95145
Merge pull request #7282 from 4ertus2/asof
ASOF JOIN inequalities
2019-10-16 13:46:52 +03:00
Amos Bird
dd128e9e1d Build fix 2019-10-16 10:49:29 +08:00
chertus
ccc8c1ad2a restore logic for array join (it's probably a hidden error) 2019-10-15 20:39:23 +03:00
Artem Zuikov
cf99f88bc4 PartialMergeJoin flush data on disk (#7186) 2019-10-15 19:31:49 +03:00
chertus
c727777a9c resolve some for review sentences 2019-10-15 18:49:00 +03:00
chertus
9667e54201 remove asterisk_left_columns_only setting 2019-10-15 17:42:57 +03:00
Vasily Nemkov
48b3c21aaa More performance test for Date and DateTime
Added cases for:
 * binary operations: -,+,>,>=,<,<=,==,!=
 * previously-missing functions: timeSlot, toRelativeQuarterNum, toStartOfTenMinutes, toUnixTimestamp, addYears (and alike).
2019-10-15 15:20:54 +03:00
alexey-milovidov
1b8cb59e4b
Merge pull request #7286 from azat/docs-agg_functions-notes-fixes
Fix notes syntax in documentation
2019-10-14 20:12:53 +03:00
alexey-milovidov
6a44336b86
Merge pull request #7304 from sundy-li/ttl_check
Check TTL in StorageFactory
2019-10-14 20:11:37 +03:00
Alexander Kuzmenkov
831fb2cbfd
Merge pull request #7283 from ClickHouse/aku/in-unconvertible
Ignore non-convertible values at any depth on the right side of the IN operator
2019-10-14 18:32:22 +03:00
Sergei Bocharov
8b27fa975e Fixes 2019-10-14 16:13:26 +03:00
Sergei Bocharov
6920592143 Fix .data to data 2019-10-14 16:08:46 +03:00
Sergei Bocharov
bf90e0eef9 Fixes 2019-10-14 15:39:42 +03:00
Alexander Kuzmenkov
160d8a6416 Ignore non-convertible values at any depth on the right side of IN
operator.

The values that are not convertible to the left argument type can't
match anyway, so it is safe to discard them.
2019-10-14 14:25:03 +03:00
Sergei Bocharov
fe63646cb4 Fixes Integrate CatBoost Topic 2019-10-14 13:34:27 +03:00
Sergei Bocharov
443299fdf5 Fixed step 3 — added info about libcatboostmodel 2019-10-14 13:34:27 +03:00