Commit Graph

2592 Commits

Author SHA1 Message Date
alesapin
5d94b7eec0 Remove non working code 2020-12-24 19:30:53 +03:00
Nikita Mikhaylov
6818b24fe5
Merge pull request #18381 from CurtizJ/part-types-merge
Restrict merges from wide to compact parts
2020-12-24 17:47:14 +03:00
Yatsishin Ilya
ce1e2b18a3 Fix codespell warnings. Split style checks. Update style checks docker 2020-12-24 13:17:52 +03:00
alexey-milovidov
c4bbb95b0f
Merge pull request #18405 from ClickHouse/add_test_for_compact_parts
Add test for already working code
2020-12-24 01:24:09 +03:00
alesapin
f3f5466d23
Merge pull request #18306 from amosbird/restrictsettings
Extend mergetree setting alter restriction
2020-12-23 23:09:59 +03:00
Anton Popov
84ec3930ba
Merge pull request #18223 from ClickHouse/delayed_last_mark_for_wide_parts
Better adaptive granularity calculation for wide parts
2020-12-23 19:37:25 +03:00
Amos Bird
6fc225e676
Distributed insertion to one random shard (#18294)
* Distributed insertion to one random shard

* add some tests

* add some documentation

* Respect shards' weights

* fine locking

Co-authored-by: Ivan Lezhankin <ilezhankin@yandex-team.ru>
2020-12-23 19:04:05 +03:00
alexey-milovidov
2c7979dd64
Merge pull request #18379 from vitlibar/fix-filling-table-system-settings-profile-elements
Fix filling table system.settings_profile_elements.
2020-12-23 15:34:00 +03:00
alesapin
fb1c6c4eb4
Update MergeTreeSettings.h 2020-12-23 15:03:55 +03:00
alesapin
54455b4740 Add test for already working code 2020-12-23 14:53:49 +03:00
Anton Popov
518cac6150 fix for review 2020-12-23 13:11:24 +03:00
Nikolai Kochetov
af7f5c9518
Merge pull request #17868 from ClickHouse/async-read-from-socket
Async read from socket
2020-12-23 12:20:42 +03:00
Alexander Kuzmenkov
22a0edc596
Update IStorage.cpp 2020-12-23 09:44:44 +03:00
Alexander Kuzmenkov
51b2329295
Merge branch 'master' into aku/window-prototype 2020-12-23 09:26:11 +03:00
spongedc
a277308ac6 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-23 11:56:46 +08:00
alexey-milovidov
8e48a25607
Merge pull request #18299 from ClickHouse/enable_thread_fuzzer_for_flaky_check
Trying to enable thread fuzzer in flaky check
2020-12-23 05:32:19 +03:00
Nikita Mikhailov
dcfbe782c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-23 05:20:22 +03:00
Nikita Mikhaylov
dae1e9ba31
Merge pull request #17857 from fastio/feature_hints_for_column_names
hints for column names
2020-12-23 04:55:41 +03:00
alexey-milovidov
c72a480d89
Merge pull request #17630 from amosbird/stacktracefix
Fix system.stack_trace when running in daemon mode
2020-12-23 04:40:50 +03:00
alexey-milovidov
aedb1874af
Fix clang-tidy 2020-12-23 04:40:23 +03:00
Anton Popov
78c4b9f2bc do not allow merge from wide to compact parts 2020-12-23 03:16:15 +03:00
Anton Popov
5101804e30
Merge pull request #18366 from CurtizJ/remove-unused-code
Remove unused code
2020-12-23 02:45:33 +03:00
Vitaly Baranov
9e31800ebb Fix filling table system.settings_profile_elements. 2020-12-22 23:24:59 +03:00
alesapin
d1d139a168 Fix comment 2020-12-22 16:48:10 +03:00
alesapin
23156fe6e4 Fix accident stop merges for merge tree 2020-12-22 16:46:09 +03:00
Anton Popov
371050a16e remove unused code 2020-12-22 15:22:54 +03:00
alesapin
35413635e7 Add ability to change some types for primary key 2020-12-22 13:41:12 +03:00
flynn
4e580f7b7c
Add asynchronous metrics on MergeTree tables data volume (#17639)
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2020-12-22 13:34:35 +03:00
Alexey Milovidov
b9df795595 Another code for TSan 2020-12-22 11:16:36 +03:00
Alexey Milovidov
af8022a0b1 Merge branch 'master' into amosbird-stacktracefix 2020-12-22 10:48:28 +03:00
alexey-milovidov
b81e067cd4
Update IStorage.h 2020-12-22 09:51:09 +03:00
alexey-milovidov
c1bd8d5df5
Merge pull request #18329 from azat/buffer-rollback-fix
Fix NULL dereference in Buffer rollback
2020-12-22 09:30:53 +03:00
Azat Khuzhin
6acefa7624 Suppress bugprone-use-after-move in Buffer rollback 2020-12-22 01:22:36 +03:00
alexey-milovidov
69719f743d
Merge branch 'master' into revert-15259-revert-14962-log-avoid-deadlock 2020-12-22 00:41:52 +03:00
Azat Khuzhin
ba57c418d0 Fix NULL dereference in Buffer rollback
<details>

Stacktrace:

```
(gdb) bt
0  DB::appendBlock (from=..., to=...) at ../src/Storages/StorageBuffer.cpp:411
1  DB::BufferBlockOutputStream::insertIntoBuffer (this=<optimized out>, block=..., buffer=...) at ../src/Storages/StorageBuffer.cpp:541
2  0x000000000f2e9d5f in DB::BufferBlockOutputStream::write (this=<optimized out>, block=...) at ../src/Storages/StorageBuffer.cpp:508
3  0x000000000ec426c4 in DB::PushingToViewsBlockOutputStream::write (this=0x7f74660faa18, block=...) at ../src/DataStreams/PushingToViewsBlockOutputStream.cpp:160
4  0x000000000ec49633 in DB::AddingDefaultBlockOutputStream::write (this=0x7f74660f1b18, block=...) at ../src/DataStreams/AddingDefaultBlockOutputStream.cpp:10
5  0x000000000ec483ac in DB::SquashingBlockOutputStream::finalize (this=0x7f74660f1d18) at ../src/DataStreams/SquashingBlockOutputStream.cpp:30
6  0x000000000ec48429 in DB::SquashingBlockOutputStream::writeSuffix (this=0x7f74660f1d18) at ../src/DataStreams/SquashingBlockOutputStream.cpp:50
7  0x000000000ec43f8f in DB::PushingToViewsBlockOutputStream::writeSuffix (this=0x7f74660f8258) at ../src/DataStreams/PushingToViewsBlockOutputStream.cpp:280
8  0x000000000ec43f8f in DB::PushingToViewsBlockOutputStream::writeSuffix (this=0x7f74b7ddea18) at ../src/DataStreams/PushingToViewsBlockOutputStream.cpp:280
9  0x000000000f2e6748 in DB::StorageBuffer::writeBlockToDestination (this=<optimized out>, block=..., table=...) at ../src/Storages/StorageBuffer.cpp:820
10 0x000000000f2ea00b in DB::BufferBlockOutputStream::write (this=0x7f7574e11748, block=...) at ../src/Storages/StorageBuffer.cpp:469
11 0x000000000ec426c4 in DB::PushingToViewsBlockOutputStream::write (this=0x7f7574ed3658, block=...) at ../src/DataStreams/PushingToViewsBlockOutputStream.cpp:160
12 0x000000000ec49633 in DB::AddingDefaultBlockOutputStream::write (this=0x7f7574e84518, block=...) at ../src/DataStreams/AddingDefaultBlockOutputStream.cpp:10
13 0x000000000ec482f4 in DB::SquashingBlockOutputStream::write (this=0x7f7574e84718, block=...) at ../src/DataStreams/SquashingBlockOutputStream.cpp:17
14 0x000000000ebe8bce in DB::CountingBlockOutputStream::write (this=0x7f7574ed3720, block=...) at ../src/DataStreams/CountingBlockOutputStream.cpp:17
15 0x000000000f68e834 in DB::TCPHandler::receiveData (this=<optimized out>, scalar=<optimized out>) at ../src/Server/TCPHandler.cpp:1168
16 0x000000000f68737c in DB::TCPHandler::receivePacket (this=0x7f7574f17000) at ../src/Server/TCPHandler.cpp:918
17 0x000000000f688d2f in DB::TCPHandler::readDataNext (this=0x7f7574f17000, poll_interval=@0x7f6f1dff1f78: 10000000, receive_timeout=@0x7f6f1dff1f68: 300) at ../src/Server/TCPHandler.cpp:460
18 0x000000000f6878be in DB::TCPHandler::readData (this=0x7f7574f17000, connection_settings=...) at ../src/Server/TCPHandler.cpp:490
19 DB::TCPHandler::processInsertQuery (this=0x7f7574f17000, connection_settings=...) at ../src/Server/TCPHandler.cpp:519
20 0x000000000f680ab9 in DB::TCPHandler::runImpl (this=0x7f7574f17000) at ../src/Server/TCPHandler.cpp:268
21 0x000000000f68f297 in DB::TCPHandler::run (this=0x7f7574f17000) at ../src/Server/TCPHandler.cpp:1414
22 0x0000000011fb81cf in Poco::Net::TCPServerConnection::start (this=0x0) at ../contrib/poco/Net/src/TCPServerConnection.cpp:43
23 0x0000000011fb9be1 in Poco::Net::TCPServerDispatcher::run (this=0x7f752ab5fd00) at ../contrib/poco/Net/src/TCPServerDispatcher.cpp:112
24 0x00000000120e71c9 in Poco::PooledThread::run (this=0x7f747d3a4580) at ../contrib/poco/Foundation/src/ThreadPool.cpp:199
25 0x00000000120e315a in Poco::ThreadImpl::runnableEntry (pThread=<optimized out>) at ../contrib/poco/Foundation/src/Thread_POSIX.cpp:345
26 0x00007f760620aea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
27 0x00007f760613aeaf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

(gdb) p to.data.__end_-to.data.__begin_
$17 = 10
(gdb) p to.data.__begin_[9].column.px
$19 = (const DB::IColumn *) 0x7f7328392720
(gdb) p to.data.__begin_[8].column.px
$20 = (const DB::IColumn *) 0x0
(gdb) p to.data.__begin_[7].column.px
$21 = (const DB::IColumn *) 0x7f746f33d360
```

Line numbers matched with this version -
f0e7cb16a7/src/Storages/StorageBuffer.cpp (L411)

</details>
2020-12-21 23:56:56 +03:00
Amos Bird
7a81e0a0f5
Extend mergetree setting alter restriction 2020-12-21 18:45:39 +08:00
alesapin
07b5c03c54 Better adjustment of the last mark 2020-12-21 11:24:52 +03:00
Alexey Milovidov
571520964c Remove useless code 2020-12-21 10:48:15 +03:00
Alexander Kuzmenkov
85c1bc1253 Merge remote-tracking branch 'origin/master' into tmp 2020-12-21 10:46:21 +03:00
Alexey Milovidov
838b146b41 Merge branch 'master' into amosbird-stacktracefix 2020-12-21 04:22:31 +03:00
alesapin
23eebb3f90 Merge branch 'master' into delayed_last_mark_for_wide_parts 2020-12-20 23:45:44 +03:00
alesapin
0041c293ab Fix gcc-10 build 2020-12-20 11:01:39 +03:00
Alexey Milovidov
9be5fa9ef2 Merge branch 'master' into Enmk-Optimize_deduplicate 2020-12-20 09:57:10 +03:00
alesapin
ca6708e594 Fix check and granule adjust for non adaptive parts 2020-12-19 21:59:20 +03:00
Vasily Nemkov
e5ec81f7cd Single quotes around column names 2020-12-19 20:58:23 +02:00
alesapin
b2d53d802d Fix for non-adaptive granularity 2020-12-18 23:32:52 +03:00
Alexander Kuzmenkov
132a0b42eb cleanup 2020-12-18 20:13:28 +03:00
Nikolai Kochetov
d7fc426458
Merge pull request #18216 from ClickHouse/fix-18137
Fix 18137
2020-12-18 18:49:02 +03:00
alesapin
cd598034cd Fix add tests 2020-12-18 17:44:31 +03:00
alesapin
c1e7d5d3c1 Working code 2020-12-18 16:49:45 +03:00
Vasily Nemkov
e166aae3f9 Using CSV-like strings for list of columns to deduplicate by instead of JSON-like notation. 2020-12-18 13:44:56 +02:00
alesapin
26637bd7ee
Merge pull request #17943 from ClickHouse/try_rewrite_writer
Small simplification of MergeTreeDataWriter
2020-12-18 12:41:16 +03:00
Nikolai Kochetov
eef58c85f0 Put thread back to set if it had stolen all tasks of backoff thread. 2020-12-18 11:33:55 +03:00
Alexander Kuzmenkov
5e19eaf2f0 Merge remote-tracking branch 'origin/master' into tmp 2020-12-18 03:49:59 +03:00
Nikita Mikhailov
6a55e018e4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-18 02:25:32 +03:00
alexey-milovidov
6fc3ca8b7b
Merge pull request #16724 from kitaisreal/added-function-accurate-cast-or-null
Added function accurateCastOrNull, allow different types inside IN subquery
2020-12-18 00:45:27 +03:00
alesapin
8140fbd0fb
Merge pull request #18200 from ClickHouse/more_logs_during_quorum
More logs during quorum insert
2020-12-18 00:22:19 +03:00
alesapin
799997db35
Merge pull request #16621 from arenadata/ADQM-148
HDFS configuration and Kerberos Authenticaiton
2020-12-17 23:16:58 +03:00
Vitaly Baranov
b9c537ac37
Merge pull request #18104 from vitlibar/fix-access-rights-for-merge-table-function-2
Fix access rights required for the merge() table function #2.
2020-12-17 22:59:33 +03:00
Vitaly Baranov
d9125a4822
Merge pull request #18014 from vitlibar/fix-showing-temporary-tables-in-system-tables
Fix showing temporary tables in system tables
2020-12-17 22:59:09 +03:00
alexey-milovidov
f82274a722
Merge pull request #16527 from kitaisreal/mmap-read-buffer-from-file-descriptor-use-getpagesize
MMapReadBufferFromFileDescriptor use getpagesize
2020-12-17 22:34:05 +03:00
alexey-milovidov
dd78c9dc75
Merge pull request #17566 from ClickHouse/relax-too-many-parts-1
Relax "Too many parts" threshold, part 1
2020-12-17 20:44:44 +03:00
alesapin
aecdd02be4 Better message 2020-12-17 19:14:42 +03:00
alesapin
f45993cd1e More logs during quorum insert 2020-12-17 19:13:01 +03:00
tavplubix
7e73b6acd8
Merge pull request #17642 from filimonov/detach-permanently
DETACH TABLE PERMANENTLY
2020-12-17 14:26:51 +03:00
alesapin
911a1cd48f Suppress gdb warning 2020-12-17 13:25:31 +03:00
Vitaly Baranov
da18af96a3 Fix access rights required for the merge() table function. 2020-12-17 12:29:07 +03:00
Vitaly Baranov
af3ccb49f1 Show create query for temporary tables in system.tables 2020-12-17 11:42:25 +03:00
Vitaly Baranov
ede755d3bc Show temporary tables in system.columns as tables with empty database. 2020-12-17 11:42:25 +03:00
Vitaly Baranov
d7a3cc8c90 Don't show the "_temporary_and_external_tables" database in system tables
system.databases, system.tables, system.columns.
2020-12-17 11:42:25 +03:00
alesapin
b74e7accf1 Review fixes 2020-12-17 11:17:39 +03:00
alesapin
f2957d6f05 Merge branch 'master' into try_rewrite_writer 2020-12-17 10:40:25 +03:00
Alexey Milovidov
b25f126569 Added a test 2020-12-17 04:52:27 +03:00
Alexey Milovidov
9a017b77df Merge branch 'master' into relax-too-many-parts-1 2020-12-17 04:40:39 +03:00
alexey-milovidov
d2653f91e1
Merge pull request #18112 from GrigoryPervakov/join_set_s3
Use IDisk in Set and Join storages
2020-12-17 03:40:13 +03:00
alexey-milovidov
484f20f77d
Merge pull request #18140 from ClickHouse/fix_flaky_test
Trying to fix 00620_optimize_on_nonleader_test
2020-12-17 03:34:39 +03:00
Alexander Kuzmenkov
a1326414ff Merge remote-tracking branch 'origin/master' into tmp 2020-12-17 00:46:14 +03:00
Alexander Kuzmenkov
fef24c81e3 some grammar 2020-12-17 00:44:05 +03:00
Maksim Kita
484e6e7de4 Removed getPageSize from small files 2020-12-16 17:38:17 +03:00
alesapin
849db47f8a Better exception messages 2020-12-16 17:31:17 +03:00
Pervakov Grigorii
4edbb42ba1 Use IDisk in Set and Join storages 2020-12-16 14:58:44 +03:00
Ilya Golshtein
09d1e89674 yandex/clickhouse-kerberized-hadoop + missed header 2020-12-16 14:48:33 +03:00
Ilya Golshtein
407565b47f fix - wrong error code 2020-12-16 14:48:33 +03:00
Ilya Golshtein
9701d6a2d5 style fix per code review, doc improvement, params consistency check 2020-12-16 14:48:33 +03:00
Ilya Golshtein
815856bc3d pass config, not context to createHDFSBuilder 2020-12-16 14:48:33 +03:00
Ilya Golshtein
7e97814859 cleanup, fixes, new submodules, ShellCommand, WriteBufferFromString 2020-12-16 14:48:33 +03:00
Ilya Golshtein
d9166a0645 HADOOP_SECURE_DN_USER way, kinit thread, docker capabilities 2020-12-16 14:48:33 +03:00
Ilya Golshtein
185ac744b1 kerberized hdfs compiled 2020-12-16 14:48:33 +03:00
Maksim Kita
7d23451e81 Replaced usages of 4096 constant with getPageSize 2020-12-16 13:46:28 +03:00
alesapin
2a4a6b22b1 Fix remove TTL for column 2020-12-16 10:26:18 +03:00
Alexey Milovidov
37363ae89d Comment update 2020-12-16 09:43:16 +03:00
spongedc
38c0827881 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-16 12:26:56 +08:00
Alexey Milovidov
4bd5d3b662 Comment update 2020-12-16 06:36:43 +03:00
Alexey Milovidov
88c5031dfa Add a commit #16595 2020-12-16 06:27:28 +03:00
spongedc
81b0fa2989 1. rename is_view to is_ordinary_view 2. add more tests 2020-12-16 11:19:38 +08:00
Alexey Milovidov
ca9ccf4850 Comment update 2020-12-16 06:03:43 +03:00
Alexey Milovidov
7ca86f176c Fix TSan warning in system.stack_trace 2020-12-16 01:43:07 +03:00
Alexey Milovidov
13ac1bee7a Merge branch 'master' into amosbird-stacktracefix 2020-12-15 22:12:49 +03:00
Nikolai Kochetov
4905201985 Merge branch 'master' into async-read-from-socket 2020-12-15 21:04:47 +03:00
alesapin
1bd2f49fb5 Merge branch 'master' into try_rewrite_writer 2020-12-15 19:05:04 +03:00