Commit Graph

5182 Commits

Author SHA1 Message Date
Artem Zuikov
df32d92ee5
Merge pull request #7358 from 4ertus2/ast
Better ambiguous column detection
2019-10-24 20:56:23 +03:00
alesapin
5d8d1df802
Merge pull request #7434 from excitoon-favorites/fix-7424
Allowed to have some parts on destination disk or volume in MOVE PARTITION
2019-10-24 19:45:22 +03:00
alexey-milovidov
e865136b2c
Merge pull request #7450 from excitoon-favorites/test-7414
Added integration test for #7414 (validation of `max_data_part_size_bytes`).
2019-10-24 19:23:21 +03:00
chertus
f2028e901d review related changes 2019-10-24 16:04:50 +03:00
chertus
12cd21f3c3 Merge branch 'master' into ast 2019-10-24 15:20:09 +03:00
Alexander Kuzmenkov
29052b6a37
Merge pull request #7377 from azat/INSERT-Distributed-MATERIALIZED-cols
* Fix INSERT into Distributed non local node with MATERIALIZED columns

Previous patch e527def18a ("Fix INSERT
into Distributed() table with MATERIALIZED column") fixes it only for
cases when the node is local, i.e. direct insert.

This patch address the problem when the node is not local
(`is_local == false`), by erasing materialized columns on INSERT into
Distributed.

And this patch fixes two cases, depends on `insert_distributed_sync`
setting:

- `insert_distributed_sync=0`

    ```
    Not found column value in block. There are only columns: date. Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5d6cf6 DB::Block::getByName(...) dbms/src/Core/Block.cpp:187
    4. 0x7fffec2fe067 DB::NativeBlockInputStream::readImpl() dbms/src/DataStreams/NativeBlockInputStream.cpp:159
    5. 0x7fffec2d223f DB::IBlockInputStream::read() dbms/src/DataStreams/IBlockInputStream.cpp:61
    6. 0x7ffff7c6d40d DB::TCPHandler::receiveData() dbms/programs/server/TCPHandler.cpp:971
    7. 0x7ffff7c6cc1d DB::TCPHandler::receivePacket() dbms/programs/server/TCPHandler.cpp:855
    8. 0x7ffff7c6a1ef DB::TCPHandler::readDataNext(unsigned long const&, int const&) dbms/programs/server/TCPHandler.cpp:406
    9. 0x7ffff7c6a41b DB::TCPHandler::readData(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:437
    10. 0x7ffff7c6a5d9 DB::TCPHandler::processInsertQuery(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:464
    11. 0x7ffff7c687b5 DB::TCPHandler::runImpl() dbms/programs/server/TCPHandler.cpp:257
    ```

- `insert_distributed_sync=1`

    ```
    2019.10.18 13:23:22.114578 [ 44 ] {a78f669f-0b08-4337-abf8-d31e958f6d12} <Error> executeQuery: Code: 171, e.displayText() = DB::Exception: Block structure mismatch in RemoteBlockOutputStream stream: different number of columns:
    date Date UInt16(size = 1), value Date UInt16(size = 1)
    date Date UInt16(size = 0): Insertion status:
    Wrote 1 blocks and 0 rows on shard 0 replica 0, 127.0.0.1:59000 (average 0 ms per block)
    Wrote 0 blocks and 0 rows on shard 1 replica 0, 127.0.0.2:59000 (average 2 ms per block)
     (version 19.16.1.1) (from [::1]:3624) (in query: INSERT INTO distributed_00952 VALUES ), Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5da4e9 DB::checkBlockStructure<void>(...)::{...}::operator()(...) const dbms/src/Core/Block.cpp:460
    4. 0x7fffec5da671 void DB::checkBlockStructure<void>(...) dbms/src/Core/Block.cpp:467
    5. 0x7fffec5d8d58 DB::assertBlocksHaveEqualStructure(...) dbms/src/Core/Block.cpp:515
    6. 0x7fffec326630 DB::RemoteBlockOutputStream::write(DB::Block const&) dbms/src/DataStreams/RemoteBlockOutputStream.cpp:68
    7. 0x7fffe98bd154 DB::DistributedBlockOutputStream::runWritingJob(DB::DistributedBlockOutputStream::JobReplica&, DB::Block const&)::{lambda()#1}::operator()() const dbms/src/Storages/Distributed/DistributedBlockOutputStream.cpp:280
    <snip>
    ````

Fixes: #7365
Fixes: #5429
Refs: #6891

* Cover INSERT into Distributed with MATERIALIZED columns and !is_local node

I guess that adding new cluster into server-test.xml is not required,
but it won't harm.

* Update DistributedBlockOutputStream.cpp
2019-10-24 12:35:09 +03:00
Vladimir Chebotarev
3debdc2119 Added integration tests for ALTER MOVE PARTITION and fixed minor things. 2019-10-24 11:52:33 +03:00
Vladimir Chebotarev
e8e5cefc35 Fixed integration test for #7414. 2019-10-24 08:59:33 +03:00
Azat Khuzhin
80cf86f100 Cover INSERT into Distributed with MATERIALIZED columns and !is_local node
I guess that adding new cluster into server-test.xml is not required,
but it won't harm.
2019-10-23 21:55:08 +03:00
Artem Zuikov
bb1c1d0ed9
Merge pull request #7431 from arenadata/master
fix aggregation (avg and quantiles) over empty decimal columns.
2019-10-23 20:28:35 +03:00
Vladimir Chebotarev
7546c315a6 Added integration test for #7414. 2019-10-23 14:25:51 +03:00
akonyaev
7426542b8b up precision for avg result to max of type 2019-10-23 11:22:51 +03:00
alexey-milovidov
ffc2e4e149
Merge pull request #7396 from nvartolomei/nv/mv-extra-columns
Test materialized view pushing extra columns
2019-10-23 07:30:53 +03:00
Nikolai Kochetov
e22b43c669
Merge pull request #7436 from amosbird/scalarperf
add perf test for subqueries with large scalars
2019-10-22 19:06:20 +03:00
Amos Bird
dd72f62f17
add perf test for subqueries with large scalars 2019-10-22 23:55:11 +08:00
akonyaev
6d0dbf7e95 add test returned datatype from quantile and median result for empty decimal data 2019-10-22 18:51:34 +03:00
akonyaev
71dd3a303e fix result type for avg aggregation 2019-10-22 18:31:56 +03:00
akonyaev
dc7c4c1b9a add tests for aggregation over empty decimal table 2019-10-22 15:30:00 +03:00
Ivan
7ba4e36e73
Don't push to MVs when inserting into Kafka table (#7265)
* Do not insert values into MV when inserting directly to Kafka
* Add method noPushingToViews() to IStorage interface
  To separate InterpreterInsertQuery and StorageKafka
2019-10-22 13:31:28 +03:00
Alexander Kuzmenkov
be4618dd05
Merge pull request #7339 from amosbird/f10
Fix preciseExp10
2019-10-22 12:00:53 +03:00
Vitaly Baranov
f611fd4dfa
Merge pull request #7293 from Vdimir/feature/isvalidjson-5910
Add function isValidJSON
2019-10-21 21:24:18 +03:00
Alexander Kuzmenkov
9c1f8109c7
Merge pull request #7402 from Enmk/odbc_bridge_nullable_fix
Fixed NULL-values in nullable columns through ODBC-bridge
2019-10-21 19:09:27 +03:00
Amos Bird
a860ee7891
Fix preciseExp10 2019-10-22 00:05:57 +08:00
chertus
4f0859a5fa Merge branch 'master' into ast 2019-10-21 15:08:05 +03:00
chertus
0d708569e3 fix test result (disabled push down) 2019-10-21 14:31:16 +03:00
Vasily Nemkov
e080324fd0 Fixed NULL-values in nullable columns through ODBC-bridge 2019-10-21 12:13:33 +03:00
alexey-milovidov
61125e536b
Update 01019_materialized_view_select_extra_columns.sql 2019-10-20 17:59:26 +03:00
Nicolae Vartolomei
b0f2f23a38 Test materialized view pushing extra columns
This test exists to prevent unintended changes to existing behaviour.

Although this behaviour might not be ideal it is can be exploited for 0-downtime changes to materialized views.

Step 1: Add new column to source table.
Step 2: Create new view reading source column.
Step 3: Swap views using `RENAME TABLE`.
Step 4: Add new column to destination table as well.
2019-10-20 10:26:44 +01:00
alexey-milovidov
52bfa9927c
Merge pull request #7384 from ClickHouse/fix_ub_in_system_parts
Fix UB in StoragesInfoStream
2019-10-19 09:59:39 +03:00
VDimir
f1630b4e2e Merge branch 'master' into feature/isvalidjson-5910 2019-10-19 06:21:21 +00:00
chertus
c5a850240c better ambiguous column detection in multiple join rewriter 2019-10-18 20:09:17 +03:00
chertus
37f07213ee better test 2019-10-18 19:45:59 +03:00
chertus
1bb8174285 disable push down for CROSS JOIN tests 2019-10-18 19:34:06 +03:00
chertus
2533652847 columns match priority: table alias > table name> aliased table name 2019-10-18 19:16:57 +03:00
Alexander Kuzmenkov
5e2cc37381
Distinguish Array and Tuple field types when creating from array of Fields. (#7179) 2019-10-18 18:57:05 +03:00
Alexander Tokmakov
06b28f9fe3 fix uninitialized offset for Lazy db 2019-10-18 16:09:24 +03:00
Alexander Kuzmenkov
3143fa37d7
Merge pull request #7371 from CurtizJ/fix-optimize-read-in-order
Fix optimize_read_in_order with IN and a subquery.
2019-10-18 15:55:53 +03:00
hcz
502672c973 Add aggregate function combinators: -OrNull & -OrDefault (#7331) 2019-10-18 12:25:39 +03:00
chertus
0f6176cc2d throw on ambiguous qualified column 2019-10-18 00:08:28 +03:00
Alexander Kuzmenkov
b62c9e729d
Merge pull request #7215 from ClickHouse/formats_improvements
Miscellaneous formats improvements
2019-10-17 17:37:38 +03:00
CurtizJ
546414c015 fix optimize_read_in_order with in subquery 2019-10-17 16:11:27 +03:00
Ivan
e595df05d9
Fix CREATE TABLE AS from table with virtual columns (#7183)
* Disable undefined instrumentation for rdkafka
2019-10-17 14:12:35 +03:00
tavplubix
c4b46e113e
Merge branch 'master' into formats_improvements 2019-10-16 23:00:17 +03:00
Alexander Tokmakov
e9e8d055af null_as_default for Values 2019-10-16 22:55:24 +03:00
Alexander Tokmakov
102cbfd25a fix old bug with default values 2019-10-16 22:27:33 +03:00
Alexander Tokmakov
5f6703130e fixes 2019-10-16 18:28:48 +03:00
Alexander Tokmakov
aa9d29d6ae Merge branch 'master' into formats_improvements 2019-10-16 17:15:13 +03:00
Alexander Kuzmenkov
16f252dce9
Merge pull request #7332 from Enmk/More_Date_and_DateTime_performance_tests
More performance test for Date and DateTime
2019-10-16 16:03:54 +03:00
Artem Zuikov
b70ca30447
Merge pull request #7335 from 4ertus2/ast
Remove asterisk_left_columns_only setting
2019-10-16 13:57:17 +03:00
Artem Zuikov
3ceeb95145
Merge pull request #7282 from 4ertus2/asof
ASOF JOIN inequalities
2019-10-16 13:46:52 +03:00