thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-19 14:11:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
31,139 Commits 9,138 Branches 1,778 Tags 2 GiB
c0630ec3bc
Commit Graph

22428 Commits

Author SHA1 Message Date
alexey-milovidov
0c1510d3e3
Update FunctionsBitmap.h 2019-11-05 02:20:16 +03:00
alexey-milovidov
f9d14f78c0
Update FunctionsBitmap.h 2019-11-05 02:09:28 +03:00
alexey-milovidov
3ee7de2d52
Merge pull request #7620 from millb/integration_test_for_part_log_table 
Integration test for part_log table
 2019-11-05 02:05:33 +03:00
alexey-milovidov
c081c8aacb
Update AggregateFunctionGroupBitmapData.h 2019-11-05 01:59:02 +03:00
Alexey Milovidov
4f83b56c14 Added another test case 2019-11-05 01:47:46 +03:00
Vladimir Chebotarev
8556b2cee3 Added tests to check that ClickHouse don't crash on insufficient arguments of S3, URL and HDFS storages. 2019-11-05 01:47:26 +03:00
Alexey Milovidov
0a3b7068fd Added a test for function "char" 2019-11-05 01:45:51 +03:00
alexey-milovidov
7d2a16ee39
Merge pull request #7486 from sundy-li/char_function 
functions: add char function
 2019-11-05 01:40:18 +03:00
Alexey Milovidov
92824e14e7 Added a test 2019-11-05 01:18:59 +03:00
Alexey Milovidov
979db8c965 Improve "arrayCompact" function 2019-11-05 01:13:10 +03:00
alexey-milovidov
fbed472018
Update 00076_ip_coding_functions.sql 2019-11-05 00:53:35 +03:00
Azat Khuzhin
4dfffdd24a Write current batch for distributed send atomically (using .tmp + rename) 
Otherwise the following can happen after reboot:

    2019.11.01 11:46:12.217143 [ 187 ] {} <Error> dist.Distributed.DirectoryMonitor: Code: 27, e.displayText() = DB::Exception: Cannot parse input: expected \n before: S\'^A\0^]\0\0<BE>4^A\0r<87>\0\0<A2><D7>^D^Y\0<F2>{^E<CD>\0\0Hy\0\0<F2>^_^C\0^_&\0\0<FF><D3>\0\0
    <8D><91>\0\0<C0>9\0\0<C0><B0>^A\0^G<AA>\0\0<B5><FE>^A\0<BF><A7>^A\0<9B><CB>^A\0I^R^A\0<B7><AB>^A\0<BC><8F>\0\0˲^B\0Zy\0\0<94><AA>\0\0<98>
    <8F>\0\0\f<A5>\0\0^QN\0\0<E3><C6>\0\0<B1>6^B\0ɳ\0\0W<99>\0\0<B9><A2>\0\0:<BB>\0\0)<B1>\0\0#<8B>\0\0aW\0\0<ED>#\0\0<F1>@\0\0ˀ^B\0<D7><FC>\0\0<DF>, Stack trace:

    0. 0x559e27222e60 StackTrace::StackTrace() /usr/bin/clickhouse
    1. 0x559e27222c45 DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) /usr/bin/clickhouse
    2. 0x559e26de4473 ? /usr/bin/clickhouse
    3. 0x559e272494b5 DB::assertString(char const*, DB::ReadBuffer&) /usr/bin/clickhouse
    4. 0x559e2a5dab45 DB::StorageDistributedDirectoryMonitor::processFilesWithBatching(std::map<unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&) /usr/bin/clickhouse
    5. 0x559e2a5db5fa DB::StorageDistributedDirectoryMonitor::processFiles() /usr/bin/clickhouse
    6. 0x559e2a5dba78 DB::StorageDistributedDirectoryMonitor::run() /usr/bin/clickhouse
    7. 0x559e2a5ddbbc ThreadFromGlobalPool::ThreadFromGlobalPool<void (DB::StorageDistributedDirectoryMonitor::*)(), DB::StorageDistributedDirectoryMonitor*>(void (DB::StorageDistributedDirectoryMonitor::*&&)(), DB::StorageDistributedDirectoryMonitor*&&)::{lambda()#1}::operator()() const /usr/bin/clickhouse
    8. 0x559e2726b07c ThreadPoolImpl<std::thread>::worker(std::_List_iterator<std::thread>) /usr/bin/clickhouse
    9. 0x559e2bbc3640 ? /usr/bin/clickhouse
    10. 0x7fbd62b3cfb7 start_thread /lib/x86_64-linux-gnu/libpthread-2.29.so
    11. 0x7fbd62a692ef __clone /lib/x86_64-linux-gnu/libc-2.29.so
     (version 19.17.1.1)

v2: remove fsync, to avoid possible stalls (https://github.com/ClickHouse/ClickHouse/pull/7600#discussion_r342010874)
 2019-11-05 00:24:38 +03:00
alexey-milovidov
5aceca9e17
Merge pull request #7615 from amosbird/fix21 
Global context should be unique.
 2019-11-05 00:16:37 +03:00
Mikhail Korotov
0781065bd8 improved changes 2019-11-04 22:10:02 +03:00
Nicolae Vartolomei
ac6472b03f Try and enable test_replicated_alters 2019-11-04 17:15:28 +00:00
Mikhail Korotov
dd99468e7b Created integration test for part_log table 2019-11-04 20:13:49 +03:00
Vladimir Chebotarev
9869ee141c Fixed exception in case of using 1 argument while defining S3, URL and HDFS storages. 2019-11-04 20:11:50 +03:00
Amos Bird
e2cfc3ecab
Global context should be unique. 2019-11-04 23:43:13 +08:00
alexey-milovidov
81e33ca817
Merge pull request #7586 from 4ertus2/joins 
Rewrite NonJoinedBlockInputStream
 2019-11-04 14:49:47 +03:00
alexey-milovidov
b261e6033a
Merge pull request #7602 from excitoon-favorites/cloneAndLoadDataPartOnSameDisk 
Made `MergeTreeData::cloneAndLoadDataPart` only work for the same disk.
 2019-11-04 14:42:30 +03:00
alexey-milovidov
1354d1079a
Merge pull request #7605 from zhang2014/fix/is_local_show 
fix cluster system is_local
 2019-11-04 14:35:37 +03:00
alexey-milovidov
782e2f6c13
Merge pull request #7606 from amosbird/nullformat 
Better Null format for tcp handler.
 2019-11-04 14:27:27 +03:00
Zhichang Yu
da2d05802c added bitmapTransform 2019-11-04 14:55:31 +08:00
Amos Bird
8f902dc2a9
Better null format for tcp handler. 2019-11-04 11:53:26 +08:00
zhang2014
08894484ee fix cluster system is_local 2019-11-04 10:56:12 +08:00
alexey-milovidov
a8d6843723
Merge pull request #7599 from azat/__getScalar-fix 
Check column for __getScalar() (avoids assertion)
 2019-11-04 02:28:42 +03:00
Vladimir Chebotarev
e76bbfc5ad Made MergeTreeData::cloneAndLoadDataPart only work for the same disk. 2019-11-04 01:27:05 +03:00
Azat Khuzhin
920572915e Add regression test for the __getScalar() 2019-11-04 00:31:10 +03:00
alexey-milovidov
d9c02b652f
Merge pull request #7594 from amosbird/fix20 
early constant folding improvement
 2019-11-04 00:30:16 +03:00
chertus
b7b664845d return right x in RIGHT|FULL JOIN .. USING(x) from NonJoinedBlockInputStream if not qualified 2019-11-03 23:30:49 +03:00
chertus
d4e1468022 fix Join.saved_block_sample columns' nullability 2019-11-03 22:33:59 +03:00
Azat Khuzhin
842f4e2ca0 Check column for __getScalar() (avoids assertion) 
Can be triggered using the following query:
  CREATE TABLE foo (key String, macro String MATERIALIZED __getScalar(key)) Engine=Null();

Trace:
    3. 0x00007ffff6d5d526 __assert_fail (libc.so.6)
    4. 0x00007ffff41fd931 boost::intrusive_ptr<DB::IColumn const>::operator*() const (libclickhouse_functionsd.so)
    5. 0x00007ffff41fcd64 COW<DB::IColumn>::IntrusivePtr<DB::IColumn const>::operator*() const & (libclickhouse_functionsd.so)
    6. 0x00007ffff4dc5944 DB::FunctionGetScalar::getReturnTypeImpl() const (libclickhouse_functionsd.so)

(Even though it is internal I guess it is better to fix it)

Refs: #7392
Cc: @amosbird
 2019-11-03 21:44:56 +03:00
Amos Bird
f0b3c1f7c7
early constant folding improvement 2019-11-03 21:17:46 +08:00
alexey-milovidov
d08bfbd3e1
Merge pull request #7328 from Joeywzr/arrayCompact 
implement arrayCompact
 2019-11-03 01:07:19 +03:00
alexey-milovidov
fa0274e223
Merge pull request #7380 from hczhcz/patch-1018 
Add function arrayFill and arrayReverseFill
 2019-11-03 01:06:41 +03:00
alexey-milovidov
ada6ee4137
Merge pull request #7591 from ClickHouse/one-more-attempt-djvu-bugfix 
One more attempt of bugfix of infinite loop in pretty space format
 2019-11-02 23:40:53 +03:00
alexey-milovidov
15c4a19bb1
Merge pull request #7585 from ernestp/macos_catalina_2019_11_01 
Fixed compile errors on macOS Catalina
 2019-11-02 15:46:17 +03:00
alexey-milovidov
f5531fb436
Merge pull request #7590 from amosbird/typetrait 
ubsan fix
 2019-11-02 13:22:39 +03:00
proller
6a573b4092 Allow to use mysql format without ssl - try2 with mysql interface split (#7524) 
* Allow use mysql format without ssl

* fix

* fix

*  fix

* Also disable poco's optional libraries

* clean

* fix

* fix

* Requested changes

* clean

* Requested fixes

* Update MySQLHandler.cpp
 2019-11-02 13:20:46 +03:00
alexey-milovidov
6e4d63e394
Merge pull request #7575 from Alex-Burmak/escaping_fix 
Fixed missing file name escaping in FREEZE command
 2019-11-02 13:20:28 +03:00
Amos Bird
ad12099642
ubsan fix 2019-11-02 17:00:03 +08:00
stavrolia
e58aa4e579 One more attempt of bugfix of infinite loop in pretty space format 2019-11-02 11:53:35 +03:00
stavrolia
d503c38cac Revert "Merge pull request #7587 from ClickHouse/fix-nonsignificant-djvu-bug" 
This reverts commit 102a76ab47, reversing
changes made to dea9220e3c.
 2019-11-02 03:23:27 +03:00
Olga Khvostikova
102a76ab47
Merge pull request #7587 from ClickHouse/fix-nonsignificant-djvu-bug 
Find and retrieve lost bug fix in pretty spaces
 2019-11-02 02:46:24 +03:00
stavrolia
89d8e6de93 Fix 2019-11-02 02:39:43 +03:00
alexey-milovidov
dea9220e3c
Merge pull request #7542 from hczhcz/patch-1030 
Fix "create as select" parsing
 2019-11-02 00:58:09 +03:00
alexey-milovidov
ad01f7b1ac
Merge pull request #7552 from azat/do-not-account-memory-for-Buffer-in-max_memory_usage 
[RFC] Do not account memory for Buffer engine in max_memory_usage limit
 2019-11-02 00:57:24 +03:00
alexey-milovidov
dda93e2275
Merge pull request #7561 from ClickHouse/achimbab-master 
Merge for #7363
 2019-11-02 00:57:13 +03:00
alexey-milovidov
7de9fdc183
Merge pull request #7579 from amosbird/countperftest 
Add perf test for early constant folding
 2019-11-02 00:51:45 +03:00
alesapin
d388565d04
Add create database query in test. 2019-11-01 22:35:50 +03:00
chertus
334b91f351 revrite NonJoinedBlockInputStream (in progress) 2019-11-01 20:41:07 +03:00
Ernest Poletaev
de1e5d0e53 Fixed compile errors on macOS Catalina 2019-11-01 23:33:43 +07:00
Nikolai Kochetov
239d92a94c Code cleanup. 2019-11-01 18:31:02 +03:00
Amos Bird
9626e24206
Add perf test for early constant folding 2019-11-01 22:33:03 +08:00
Nikolai Kochetov
575eabd4ff Add flag to set that set is created. 2019-11-01 16:56:33 +03:00
alexey-milovidov
4081b0be90
Merge pull request #7497 from amosbird/countopt 
Early constant folding.
 2019-11-01 16:54:02 +03:00
Alexander Burmak
9b8fc9a899 Fixed missing file name escaping in FREEZE command 2019-11-01 16:17:08 +03:00
Ivan
21db9157f7
Don't initialize some constant blocks in a loop (#7475) 2019-11-01 14:34:29 +03:00
Nikolai Kochetov
388bbb950b Add flag to set that set is created. 2019-11-01 14:03:35 +03:00
Nikolai Kochetov
1f7a798c97 Add flag to set that set is created. 2019-11-01 13:58:29 +03:00
alesapin
6ef0a92535
Merge pull request #7558 from excitoon/patch-1 
Made mutation to choose the same disk in `ReplicatedMergeTree`.
 2019-11-01 13:52:45 +03:00
alexey-milovidov
b1fe128109
Merge pull request #7562 from amosbird/parser3 
Complement to #7501
 2019-11-01 13:15:36 +03:00
Nikolai Kochetov
ddcbc2f9d3 Enable processors by default. 2019-11-01 12:45:33 +03:00
Nikolai Kochetov
d31cac33b2 Small fixes. 2019-11-01 12:41:20 +03:00
Nikolai Kochetov
5bb47e2484
Merge pull request #7181 from ClickHouse/processors-and-storages 
Processors and storages
 2019-11-01 12:04:49 +03:00
Nikolai Kochetov
c0fccd5b87 Simplified code. 2019-11-01 11:32:41 +03:00
Amos Bird
5333258e7e
Complement to tuple parser 2019-11-01 10:48:15 +08:00
alexey-milovidov
30b2d85fb0
Merge pull request #7501 from amosbird/parser2 
Some parser improvements.
 2019-10-31 23:48:14 +03:00
Azat Khuzhin
dbe45800d9 Do not account memory for Buffer engine in max_memory_usage limit 
Since background flush will not be accounted there, and it can be too
tricky to calclulate this limit, in case you have multiple materialized
views with Buffer engine.

v2: test adjustment
 2019-10-31 22:26:50 +03:00
Nikolai Kochetov
9a5a0a2a3b Fix types for set elements. 2019-10-31 19:14:06 +03:00
Vitaly Baranov
56011d4649
Merge pull request #7553 from vitlibar/add-constraints-columns-to-table-system-settings 
Add constraints columns to table system.settings
 2019-10-31 19:06:46 +03:00
Amos Bird
e04cdd4ebb
Early constant folding. 2019-10-31 23:54:32 +08:00
Amos Bird
4aff9a20c6
Better parser 2019-10-31 23:22:48 +08:00
Vladimir Chebotarev
faf1b070ac
Typo was fixed. 2019-10-31 16:26:11 +03:00
Vladimir Chebotarev
0dea6f5458
Added comment for selecting disk for mutation. 2019-10-31 16:19:38 +03:00
Nikolai Kochetov
be1ccaa890 Fix clang build. 2019-10-31 16:18:21 +03:00
Vladimir Chebotarev
f44cee623a
Made mutation to choose the same disk in ReplicatedMergeTree. 2019-10-31 15:22:25 +03:00
Nikolai Kochetov
18378417df Fix build. 2019-10-31 14:48:35 +03:00
Nikolai Kochetov
a80338e066 Merged with master 2019-10-31 14:41:29 +03:00
Nikolai Kochetov
a38124c68e Review fixes. 2019-10-31 14:32:24 +03:00
Vitaly Baranov
272608762e Fix comment. 2019-10-31 11:45:14 +03:00
alexey-milovidov
417c008700
Update 01021_create_as_select.sql 2019-10-31 11:29:30 +03:00
hcz
134bc80940
Create array_fill.xml 2019-10-31 14:30:11 +08:00
sundy-li
17212afabb fix virtual call 2019-10-31 13:19:28 +08:00
sundy-li
d2a785f759 fix style 2019-10-31 12:59:50 +08:00
hcz
4c7a54113d Add tests 2019-10-31 11:33:23 +08:00
Vitaly Baranov
5601d6ae66 Add constraints columns to table system.settings 2019-10-31 01:45:57 +03:00
alexey-milovidov
194fd85f5c
Update arrayCompact.cpp 2019-10-31 00:29:15 +03:00
alexey-milovidov
0d2a2c332f
Merge pull request #7366 from achimbab/fixbug_inop 
Made IN operator support Array(LowCardinality(Nullable(String)))
 2019-10-31 00:26:22 +03:00
alexey-milovidov
88d43488e1
Merge pull request #7412 from 4ertus2/strange 
merge_max_block_size in settings
 2019-10-31 00:24:43 +03:00
alexey-milovidov
883c868b36
Merge pull request #7523 from ClickHouse/fix-thread-context-attach 
Disable context replacement for internal queries
 2019-10-31 00:07:13 +03:00
alexey-milovidov
8cfc70aad0
Merge pull request #7551 from ClickHouse/merge-fast-count 
Improvements for fast count() after #7510
 2019-10-30 23:55:58 +03:00
alexey-milovidov
1ec7cb3546
Update range.cpp 2019-10-30 23:39:29 +03:00
alexey-milovidov
bb75c7469a
Merge pull request #7545 from ClickHouse/fix_only_tuple_columns 
Fix only tuple columns in table.
 2019-10-30 23:34:23 +03:00
proller
0006f2a370 Fix build without submodules (#7295) 
* Build fixes

*            fix

* Fix

* fix

* Fix install(export..

* Freebsd fixes

* Freebsd fixes

* Fix warning

* fix

* More PRIVATE linking

* Fix review

* Timmy

* Timmy

* fix

* Try fix jemalloc link

* style
 2019-10-30 23:34:06 +03:00
Alexey Milovidov
fe73354320 Improvements after #7510 2019-10-30 22:15:55 +03:00
Amos Bird
4609ca3189
Enable bracketed paste if possible 2019-10-31 02:33:45 +08:00
alexey-milovidov
ddb58eeb8b
Merge pull request #7510 from amosbird/smartcount 
Instant count() for MergeTree
 2019-10-30 20:42:00 +03:00
alesapin
e2424bda4c Fix style 2019-10-30 19:00:40 +03:00
alesapin
7f4e982d13 Add tests 2019-10-30 17:51:30 +03:00
alesapin
2b5ef8bdef Fix bug when table contains only tuple columns 2019-10-30 17:50:59 +03:00
hcz
4dca99fa65 Fix create as select parsing 2019-10-30 21:06:23 +08:00
Artem Zuikov
0f68c5a12a
Merge pull request #7530 from 4ertus2/joins 
Fix right table key nullability (join_use_keys = 0)
 2019-10-30 13:30:27 +03:00
Ivan
4f2f5cca84
Add support for cross-compiling to the CPU architecture AARCH64 (#7370) 2019-10-30 10:01:53 +03:00
sundy-li
7292748012 fix docs 2019-10-30 10:06:30 +08:00
sundy-li
be9da37720 fix zero divide 2019-10-30 09:36:50 +08:00
chertus
deb2406b30 fix JOIN right table keys nullability 2019-10-29 22:39:42 +03:00
Olga Khvostikova
ff4d90f602
Add test for greatCircleDist 2019-10-29 19:49:37 +03:00
alexey-milovidov
6e837e86e6
Merge pull request #7500 from amosbird/parsernit 
Fix DUMP debug.
 2019-10-29 19:25:20 +03:00
Amos Bird
1eb44c6837
Add missing tests 2019-10-29 22:26:43 +08:00
chertus
12b57aedf0 remove error prone code 2019-10-29 16:52:05 +03:00
sundy-li
6f2cce37d7 fix move 2019-10-29 19:22:00 +08:00
sundy-li
718fb99939 fix move 2019-10-29 19:20:35 +08:00
alesapin
cde07dd620 Disable context replacement for internal queries 2019-10-29 13:27:19 +03:00
Amos Bird
2c75a51d4f
Instant count() for MergeTree 
Use (Replicated)MergeTree's metadata to do trivial count()
 2019-10-29 17:45:47 +08:00
sundy-li
00389a67a0 add more tests 2019-10-29 14:55:59 +08:00
sundy-li
66ffec32f7 improve range to support start,step args 2019-10-29 14:33:03 +08:00
sundy-li
9eb77a59e3 fix index 2019-10-29 13:20:18 +08:00
sundy-li
a653f50432 remove logs 2019-10-29 13:01:45 +08:00
sundy-li
dbd3227e5a fixup partial const column 2019-10-29 11:51:27 +08:00
sundy-li
3f1651fa29 improve range 2019-10-29 11:03:47 +08:00
memo
3449e8d107 fix 2019-10-29 10:27:54 +08:00
Alexander Kuzmenkov
6e2af3db41
Merge pull request #7480 from azat/crc-v3 
Add CRC32IEEE()/CRC64() support
 2019-10-28 16:16:37 +03:00
Amos Bird
15d12db16a
Fix DUMP debug. 2019-10-28 19:39:51 +08:00
Alexander Kuzmenkov
c3ea94209b
Merge pull request #7491 from traceon/odbc-bridge-type-mapping-fix 
Add handling of SQL_TINYINT and SQL_BIGINT, fix SQL_FLOAT in ODBC Bridge
 2019-10-28 13:30:18 +03:00
memo
3064e716db add line 2019-10-28 14:48:25 +08:00
memo
0f572e5d7d add line 2019-10-28 12:51:59 +08:00
memo
90333ac9aa modified getReturnType 2019-10-28 11:15:52 +08:00
sundy-li
538b77c123 functions: add char function as mysql behavior 2019-10-28 10:44:56 +08:00
Nikolai Kochetov
43c8d13de6
Merge pull request #7496 from amosbird/fix15 
Fix HAVING without GROUP BY
 2019-10-28 02:44:24 +03:00
Amos Bird
77bdfb3350
Fix HAVING without GROUP BY 2019-10-27 22:10:54 +08:00
Azat Khuzhin
04f1e6b2cc Use crc32_z() over crc32(), since it size_t for length 2019-10-25 23:32:29 +03:00
Denis Glazachev
8e48430159 Add handling of SQL_TINYINT and SQL_BIGINT 
Fix handling of SQL_FLOAT
 2019-10-25 21:49:49 +04:00
Azat Khuzhin
2d2e738085 Add CRC32IEEE()/CRC64() support 
zlib's implementation uses CRC-32-IEEE 802.3 polynomial (0xedb88320) but
with starting value 0xffffffff, so introduce another crc32
implementation - CRC32IEEE that has starting value - 0

Also add CRC64 with ECMA polynomial.

v2: s/crc*_data./crc*_data./ to avoid conflicts with other crc32.h in contrib
v3: join with existing CRC32()
 2019-10-25 15:52:41 +03:00
alesapin
04ca163518 Merge branch 'master' into dictionaries_ddl_loader 2019-10-25 10:58:04 +03:00
hcz
3767cb76de Improve style 2019-10-25 11:25:02 +08:00
alesapin
8755f5ffa9 Add missed config 2019-10-24 23:08:36 +03:00
Artem Zuikov
df32d92ee5
Merge pull request #7358 from 4ertus2/ast 
Better ambiguous column detection
 2019-10-24 20:56:23 +03:00
alesapin
5d8d1df802
Merge pull request #7434 from excitoon-favorites/fix-7424 
Allowed to have some parts on destination disk or volume in MOVE PARTITION
 2019-10-24 19:45:22 +03:00
alexey-milovidov
e865136b2c
Merge pull request #7450 from excitoon-favorites/test-7414 
Added integration test for #7414 (validation of `max_data_part_size_bytes`).
 2019-10-24 19:23:21 +03:00
chertus
f2028e901d review related changes 2019-10-24 16:04:50 +03:00
chertus
12cd21f3c3 Merge branch 'master' into ast 2019-10-24 15:20:09 +03:00
Artem Zuikov
39b64dff87
Merge pull request #7454 from 4ertus2/refactoring 
Refactoring: extract non aliases logic out of QueryNormalizer
 2019-10-24 14:34:53 +03:00
Vladimir Chebotarev
64f158ff28 Fixed message for ALTER MOVE PART. 2019-10-24 13:56:32 +03:00
Alexander Kuzmenkov
29052b6a37
Merge pull request #7377 from azat/INSERT-Distributed-MATERIALIZED-cols 
* Fix INSERT into Distributed non local node with MATERIALIZED columns

Previous patch e527def18a ("Fix INSERT
into Distributed() table with MATERIALIZED column") fixes it only for
cases when the node is local, i.e. direct insert.

This patch address the problem when the node is not local
(`is_local == false`), by erasing materialized columns on INSERT into
Distributed.

And this patch fixes two cases, depends on `insert_distributed_sync`
setting:

- `insert_distributed_sync=0`

    ```
    Not found column value in block. There are only columns: date. Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5d6cf6 DB::Block::getByName(...) dbms/src/Core/Block.cpp:187
    4. 0x7fffec2fe067 DB::NativeBlockInputStream::readImpl() dbms/src/DataStreams/NativeBlockInputStream.cpp:159
    5. 0x7fffec2d223f DB::IBlockInputStream::read() dbms/src/DataStreams/IBlockInputStream.cpp:61
    6. 0x7ffff7c6d40d DB::TCPHandler::receiveData() dbms/programs/server/TCPHandler.cpp:971
    7. 0x7ffff7c6cc1d DB::TCPHandler::receivePacket() dbms/programs/server/TCPHandler.cpp:855
    8. 0x7ffff7c6a1ef DB::TCPHandler::readDataNext(unsigned long const&, int const&) dbms/programs/server/TCPHandler.cpp:406
    9. 0x7ffff7c6a41b DB::TCPHandler::readData(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:437
    10. 0x7ffff7c6a5d9 DB::TCPHandler::processInsertQuery(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:464
    11. 0x7ffff7c687b5 DB::TCPHandler::runImpl() dbms/programs/server/TCPHandler.cpp:257
    ```

- `insert_distributed_sync=1`

    ```
    2019.10.18 13:23:22.114578 [ 44 ] {a78f669f-0b08-4337-abf8-d31e958f6d12} <Error> executeQuery: Code: 171, e.displayText() = DB::Exception: Block structure mismatch in RemoteBlockOutputStream stream: different number of columns:
    date Date UInt16(size = 1), value Date UInt16(size = 1)
    date Date UInt16(size = 0): Insertion status:
    Wrote 1 blocks and 0 rows on shard 0 replica 0, 127.0.0.1:59000 (average 0 ms per block)
    Wrote 0 blocks and 0 rows on shard 1 replica 0, 127.0.0.2:59000 (average 2 ms per block)
     (version 19.16.1.1) (from [::1]:3624) (in query: INSERT INTO distributed_00952 VALUES ), Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5da4e9 DB::checkBlockStructure<void>(...)::{...}::operator()(...) const dbms/src/Core/Block.cpp:460
    4. 0x7fffec5da671 void DB::checkBlockStructure<void>(...) dbms/src/Core/Block.cpp:467
    5. 0x7fffec5d8d58 DB::assertBlocksHaveEqualStructure(...) dbms/src/Core/Block.cpp:515
    6. 0x7fffec326630 DB::RemoteBlockOutputStream::write(DB::Block const&) dbms/src/DataStreams/RemoteBlockOutputStream.cpp:68
    7. 0x7fffe98bd154 DB::DistributedBlockOutputStream::runWritingJob(DB::DistributedBlockOutputStream::JobReplica&, DB::Block const&)::{lambda()#1}::operator()() const dbms/src/Storages/Distributed/DistributedBlockOutputStream.cpp:280
    <snip>
    ````

Fixes: #7365
Fixes: #5429
Refs: #6891

* Cover INSERT into Distributed with MATERIALIZED columns and !is_local node

I guess that adding new cluster into server-test.xml is not required,
but it won't harm.

* Update DistributedBlockOutputStream.cpp
 2019-10-24 12:35:09 +03:00
Alexander Kuzmenkov
050de71ef4
Update DistributedBlockOutputStream.cpp 2019-10-24 12:33:45 +03:00
alesapin
a6199b7e69 Merge with master 2019-10-24 12:33:40 +03:00
alesapin
7edd80c9b7 Add test for existing dictionary 2019-10-24 12:25:28 +03:00
Vladimir Chebotarev
255da8f5e0 Fixed style. 2019-10-24 12:11:06 +03:00
Vladimir Chebotarev
3debdc2119 Added integration tests for ALTER MOVE PARTITION and fixed minor things. 2019-10-24 11:52:33 +03:00