thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 18:32:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
60,211 Commits 9,328 Branches 1,783 Tags 2.1 GiB
febb187da0
Commit Graph

3327 Commits

Author SHA1 Message Date
kssenii
d79f7c04f6 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into rabbit-format-settings 2021-02-17 06:03:19 +00:00
kssenii
29362bb483 Support vhost 2021-02-16 22:05:14 +00:00
Alexander Tokmakov
1aac7b3471 Merge branch 'master' into database_replicated 2021-02-17 00:39:56 +03:00
Vitaly Baranov
d357fb9129 Fix reading from the table system.quota_usage. 2021-02-16 23:54:05 +03:00
tavplubix
68b427a99d
Merge pull request #20448 from ClickHouse/better_ddl_queue_cleanup 
Better distributed DDL queue cleanup
 2021-02-16 19:29:50 +03:00
M0r64n
75117389ec Add a couple of QOL file engine settings 2021-02-16 18:50:11 +04:00
Alexander Tokmakov
7b54b892b5 fix 2021-02-16 17:05:58 +03:00
Alexander Tokmakov
bf6f64a3fb Merge branch 'master' into database_replicated 2021-02-16 01:28:19 +03:00
kssenii
2a887b9772 Add missing format factory settings 2021-02-15 22:06:10 +00:00
Nikolai Kochetov
85277d6a41 Comment debug output. 2021-02-15 22:54:47 +03:00
Nikolai Kochetov
ae73600fb0 Refactor row level security actions. 2021-02-15 22:48:06 +03:00
alexey-milovidov
a299790522
Merge pull request #19748 from excitoon-favorites/s3ssec 
Added Server Side Encryption Customer Keys support in S3 client
 2021-02-15 15:23:14 +03:00
tavplubix
3f86ce4c67
Update StorageReplicatedMergeTree.cpp 2021-02-15 15:04:30 +03:00
Azat Khuzhin
02198d091e Add proper checks while parsing directory names for async INSERT (fixes SIGSEGV) 2021-02-15 10:53:41 +03:00
Denis Glazachev
45e90961f7 Store and process alias_actions in FilterInfo 2021-02-14 21:16:40 +04:00
alexey-milovidov
d39398baa6
Merge pull request #20468 from ClickHouse/speedup-memory 
Performance improvement of StorageMemory by Nikolai Kochetov
 2021-02-14 06:07:04 +03:00
Alexey Milovidov
ddb2cbcf6d Merge branch 'master' into in-memory-compression 2021-02-14 04:32:41 +03:00
Denis Glazachev
99a04b6c64 Merge branch 'master' into row-policy-with-prewhere 
* master: (759 commits)
  Suppress UBSan report in Decimal comparison
  Suppress UBSan report in Decimal comparison
  Fix UBSan report in arrayDifference
  Update README.md
  Non significant change in AggregationCommon
  Print stack trace on SIGTRAP
  Fix dependent test
  Fix tests for better parallel run
  Add test for already working code
  Revert "Fix access control manager destruction order"
  Update index.md
  Update index.md
  Update index.md
  Bit more complicated example for isIPv4String - ru
  Bit more complicated example for isIPv4String
  cleanup
  Replace database with ordinary
  Added comments
  Split tests to make them stable
  Fixes
  ...

# Conflicts:
#	src/Storages/MergeTree/MergeTreeRangeReader.cpp
 2021-02-14 02:24:00 +04:00
Denis Glazachev
79592b73f8 Store filter info in prewhere info instead of multiple prewheres 
Some cleanups
 2021-02-14 02:07:13 +04:00
Alexey Milovidov
179a0f9d8b Performance improvement by Nikolai Kochetov 2021-02-13 22:51:38 +03:00
Azat Khuzhin
68f23b7087 Improve logging during MergeTree reading 
- Remove "Not using primary index on part {}" message (too noisy)
- Add number of total marks before filtering by primary key into the
  common message
- Make "Index {} has dropped {} / {} granules." not per-part, but
  per-query
 2021-02-13 18:08:55 +03:00
alexey-milovidov
2fe2190a20
Merge pull request #19721 from azat/buffer-profile 
Add separate config directive for Buffer profile
 2021-02-13 02:43:41 +03:00
Alexey Milovidov
34f5ea00e4 Merge branch 'master' into in-memory-compression 2021-02-13 02:08:40 +03:00
Alexey Milovidov
453450985f Performance improvement by Nikolai Kochetov 2021-02-13 00:26:25 +03:00
Alexey Milovidov
1c656830fc Fix clang-tidy 2021-02-13 00:26:12 +03:00
Pavel Kruglov
f0163c2acf Don't create empty parts on INSERT 2021-02-12 17:02:04 +03:00
Nikolai Kochetov
2ba503a9cc
Merge pull request #20110 from vitlibar/mongodb-doesnt-connect-before-read 
MongoDB table engine now establishes connection only when it reads data.
 2021-02-12 15:49:45 +03:00
alesapin
a2bb586170
Merge pull request #20335 from ClickHouse/fix_scheduling_of_non_successful_background_tasks 
Fix backoff for failed background tasks in replicated merge tree
 2021-02-12 10:06:04 +03:00
Alexey Milovidov
170daa5d65 Generate ya.make 2021-02-12 05:33:39 +03:00
Alexey Milovidov
ed7270dd8b Better interface 2021-02-12 03:25:00 +03:00
Alexey Milovidov
a2da05039e Merge branch 'master' into in-memory-compression 2021-02-11 23:59:14 +03:00
Alexey Milovidov
b61ce427a8 Whitespaces 2021-02-11 23:59:00 +03:00
alexey-milovidov
7b55eb8c58
Merge pull request #20178 from amosbird/delayinactive 
Delay or throw insertion when too many inactive parts
 2021-02-11 20:52:04 +03:00
Alexey Milovidov
280f459f71 Fix quadratic INSERT 2021-02-11 16:41:21 +03:00
Alexey Milovidov
d539948fe7 In memory compression: a prototype 2021-02-11 16:40:15 +03:00
alexey-milovidov
263d751d64
Merge pull request #20169 from ClickHouse/no-excessive-squash-memory 
Disable excessive squashing of blocks for StorageMemory #13052
 2021-02-11 16:38:24 +03:00
alesapin
3253638969 Fix backoff for failed background tasks in replicated merge tree 2021-02-11 14:46:18 +03:00
tavplubix
17af32a59b
Merge pull request #20215 from azat/dist-lockless-SYSTEM-FLUSH-DISTRIBUTED 
Lockless SYSTEM FLUSH DISTRIBUTED
 2021-02-11 12:30:46 +03:00
alesapin
56453256b7
Merge pull request #20300 from ClickHouse/restrict_to_drop_version_column 
Don't allow to drop or rename version column
 2021-02-11 11:04:00 +03:00
alesapin
121c5c9605
Merge pull request #20279 from ClickHouse/relax_check_in_data_writer 
Relax check in data writer for non adaptive tables
 2021-02-11 10:57:07 +03:00
alesapin
c2bb2c2902
Merge pull request #20097 from ClickHouse/remove-adding-missed-step 
Build actions dag to evaluate missing defaults.
 2021-02-11 10:51:21 +03:00
alexey-milovidov
e33477a16e
Merge pull request #20282 from ClickHouse/dont_allow_ttl_for_old_syntax 
Don't allow TTL in old syntax
 2021-02-11 00:46:30 +03:00
Alexey Milovidov
d3dba0e52a Merge branch 'master' into no-excessive-squash-memory 2021-02-10 23:45:12 +03:00
Azat Khuzhin
809fa7e4cc Sync SYSTEM FLUSH DISTRIBUTED with TRUNCATE 2021-02-10 23:10:37 +03:00
Anton Popov
365e2fb746
Merge pull request #20231 from fuwhu/refine-loadDataParts-code 
refine code in MergeTreeData::loadDataParts to avoid parsing WAL file as data part
 2021-02-10 22:42:31 +03:00
alesapin
891fce3275 Don't allow to drop or rename version column 2021-02-10 22:23:48 +03:00
Azat Khuzhin
935870b2c2 Add separate config directive for Buffer profile 
If you push data via Buffer engine then all your queries will be done
from one user, however this is not always desired behavior, since this
will not allow to limit queries with max_concurrent_queries_for_user and
similar.
 2021-02-10 21:40:26 +03:00
Alexey Milovidov
30b2554fa3 Fix error 2021-02-10 20:48:39 +03:00
alesapin
a63d73db58 Don't allow TTL in old syntax 2021-02-10 15:09:12 +03:00
alesapin
420f2958e2
Merge pull request #19991 from ClickHouse/constant-propagation-fix 
Do not use inputs which values are known constants in ActionsDAG.
 2021-02-10 14:13:52 +03:00
alesapin
21faea1f68 Relax check in data writer for non adaptive tables 2021-02-10 13:43:31 +03:00
alesapin
9179fc8d96
Merge pull request #20226 from ClickHouse/fix_race_in_system_parts 
Fix benign race in system.parts
 2021-02-10 12:45:51 +03:00
Amos Bird
afb5846a24
refactor 2021-02-10 11:22:24 +08:00
HuFuwang
4ac8860cea refine code in MergeTreeData::loadDataParts to avoid parsing WAL file as data part. 2021-02-10 10:13:43 +08:00
Vitaly Baranov
9de7a0a779
Add comment 2021-02-09 18:55:36 +03:00
tavplubix
86530e603c
Merge pull request #20027 from bharatnc/ncb/event_time_micro_part_log 
add event_time_microseconds field to system.part_log table
 2021-02-09 14:46:40 +03:00
alesapin
8da9f150de Fix race in system.parts 2021-02-09 11:51:17 +03:00
Nikolai Kochetov
229acbad5d
Merge pull request #20077 from ClickHouse/remove-adding-const-column-transform 
Remove AddingConstColumn step and transform.
 2021-02-09 11:28:44 +03:00
alesapin
accf48a21b
Merge pull request #20212 from nvartolomei/codespace-4851 
Remove dead code
 2021-02-09 11:27:27 +03:00
Alexander Tokmakov
5e0cbfe972 Merge branch 'master' into database_replicated 2021-02-08 23:01:12 +03:00
Alexander Tokmakov
78c1d69b8c better code 2021-02-08 22:36:17 +03:00
tavplubix
ac477d9850
Merge pull request #19771 from ClickHouse/thread_state_improvements 
Minor code improvements around ThreadStatus
 2021-02-08 22:34:55 +03:00
Azat Khuzhin
ce91c257b2 Lockless SYSTEM FLUSH DISTRIBUTED 
Right now SYSTEM FLUSH DISTRIBUTED will block:
- INSERT into this Distributed table (requireDirectoryMonitor())
- SELECT * FROM system.distribution_queue
 2021-02-08 22:07:30 +03:00
Nicolae Vartolomei
c336cac306 Remove dead code 2021-02-08 17:36:58 +00:00
Nicolae Vartolomei
a8f37e7704 Remove dead code 2021-02-08 17:19:35 +00:00
bharatnc
60c8dfa22c PartLog - populate event_time_microseconds for MergeParts 2021-02-08 07:25:44 -08:00
Nikolai Kochetov
d7fa440356
Merge branch 'master' into remove-adding-const-column-transform 2021-02-08 15:49:23 +03:00
Alexey Milovidov
dc01464764 Merge branch 'master' into no-excessive-squash-memory 2021-02-08 15:39:22 +03:00
tavplubix
b31891ce2b
Merge pull request #20105 from nicelulu/zookeeper_table_in_query 
Supports system.zookeeper path IN query.
 2021-02-08 15:01:56 +03:00
Anton Ivashkin
4d44d75bc7 Fix build after merge one more time 2021-02-08 14:45:10 +03:00
Amos Bird
109a392e0c
Fix ubsan 2021-02-08 19:41:16 +08:00
Kruglov Pavel
b256bd455c
Merge pull request #19961 from ucasFL/shard-id 
add setting insert_shard_id
 2021-02-08 14:28:13 +03:00
Anton Ivashkin
1b6ce056a5 Merge master 2021-02-08 12:34:20 +03:00
Alexander Tokmakov
7ce0ef2561 show clusters for replicated db 2021-02-08 12:14:17 +03:00
Amos Bird
ab5c7b75a4
Delay or throw insertion when too many inactive parts 2021-02-07 19:03:55 +08:00
Alexey Milovidov
38ea4af339 Fix quadratic INSERT 2021-02-07 05:40:06 +03:00
Alexey Milovidov
905793a7e4 Disable excessive squashing of blocks for StorageMemory #13052 2021-02-07 04:57:17 +03:00
alexey-milovidov
66b868d865
Merge pull request #20153 from zlobober/customize_prewhere_optimizer 
Allow using MergeTreeWhereOptimizer not only with MergeTree-based storages
 2021-02-07 01:10:11 +03:00
alexey-milovidov
d0a1e69e99
Merge pull request #20094 from ClickHouse/columns-description-fuzzer 
Add fuzzer for ColumnsDescription
 2021-02-06 23:21:41 +03:00
Maxim Akhmedov
0cd3601928 Allow using MergeTreeWhereOptimizer not only with MergeTree-based storages 2021-02-06 18:08:42 +03:00
Kruglov Pavel
d94e8624d7
Merge branch 'master' into shard-id 2021-02-06 16:48:17 +03:00
tavplubix
1c5d6bbe44
Merge pull request #20051 from stigsb/materializemysql-system-tables-fix 
Show details of MaterializeMySQL tables in `system.tables`
 2021-02-06 14:50:11 +03:00
Alexey Milovidov
26d4d52b9b Merge branch 'master' into columns-description-fuzzer 2021-02-06 13:31:50 +03:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID 
CLICKHOUSE-606: query deduplication based on parts' UUID
 2021-02-05 22:47:34 +03:00
Anton Ivashkin
e64c63c611 Merge master 2021-02-05 20:10:06 +03:00
Anton Ivashkin
df6c882aab Fix build after merge 2021-02-05 18:52:40 +03:00
Nikolai Kochetov
85c175883e Rename functions. 2021-02-05 18:11:26 +03:00
Stig Bakken
3fc1641d91 Show details of MaterializeMySQL tables in system.tables 2021-02-05 21:37:29 +08:00
Nikolai Kochetov
9869f70a0d Remove AddMissed step and transform. 2021-02-05 14:41:44 +03:00
alesapin
7cbc135e72 More isolated code 2021-02-05 12:54:34 +03:00
Vitaly Baranov
1c51b4a4e4 MongoDB table engine now establishes connection only when it reads data. 2021-02-05 11:47:31 +03:00
vdimir
1e2c302248
Merge pull request #18377 from Jokser/disk-s3-backup-restore-metadata 2021-02-05 10:44:59 +03:00
zhangxiao871
581495ae23 Supports system.zookeeper path IN query. 2021-02-05 14:45:28 +08:00
Nikolai Kochetov
d9aa1096cf Build actions dag to evaluate missing defaults. 2021-02-04 23:36:50 +03:00
vdimir
91ea99129e
Merge pull request #19999 from kreuzerkrieg/change_the_idisk_interface 
return `DiskType` instead of `String` in IDisk::getType()
 2021-02-04 22:46:54 +03:00
Alexey Milovidov
d2c1a97d86 Minor modification 2021-02-04 22:41:17 +03:00
Alexey Milovidov
19c38c61e6 Add fuzzer for ColumnsDescription 2021-02-04 22:03:56 +03:00
vdimir
30f98795bc
Merge remote-tracking branch 'upstream/master' into disk-s3-backup-restore-metadata 2021-02-04 18:38:28 +03:00
Nikolai Kochetov
e3d6ffd9dc Remove AddingConstColumn step and transform. 2021-02-04 17:25:11 +03:00
alexey-milovidov
c12f2d460e
Merge pull request #19972 from ClickHouse/fix-rocksdb 
Fix missing type check in EmbeddedRocksDB
 2021-02-04 15:06:12 +03:00
kreuzerkrieg
1cc9e03d0c return DiskType instead of String IDisk::getType() as in the rest of storage interfaces 2021-02-04 10:01:21 +02:00
alexey-milovidov
5c1b96dd93
Merge pull request #19886 from azat/writer-partial-buffer 
Allow writing partial buffer
 2021-02-04 10:29:47 +03:00
Alexey Milovidov
55c7fdbbe4 Merge branch 'master' into fix-rocksdb 2021-02-04 08:30:02 +03:00
Alexey Milovidov
474f151707 More docs for MergeTreeSettings 2021-02-04 08:26:22 +03:00
Azat Khuzhin
fb4596f433 Fix clang-analyzer-optin.cplusplus.VirtualCall 2021-02-04 02:08:41 +03:00
Azat Khuzhin
ca071381e1 Merge remote-tracking branch 'upstream/master' into writer-partial-buffer 2021-02-03 23:46:39 +03:00
Azat Khuzhin
46f5268135 Fix incorrect WriteBuffer interface usage in message brokers producers 
WriteBufferToKafkaProducer and WriteBufferToRabbitMQProducer uses
WriteBuffer::set(nullptr, 0) which will leave the WriteBuffer in the
invalid state, since after this available() is 0 and next() will not
call nextImpl().

Stacktrace:

    2021.02.02 05:27:16.248056 [ 97 ] {} <Fatal> BaseDaemon: ########################################
    2021.02.02 05:27:16.248697 [ 97 ] {} <Fatal> BaseDaemon: (version 21.3.1.5880, build id: 03F265087EF651DC4D8E569775FE9E91FFA5DE76) (from thread 93) (query_id: 8c34c220-f66a-45d2-8f4f-01193cede98b) Received signal Segmentation fault (11)
    2021.02.02 05:27:16.249030 [ 97 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: write. Address not mapped to object.
    2021.02.02 05:27:16.249623 [ 97 ] {} <Fatal> BaseDaemon: Stack trace: 0x12f3dbeb 0x12efd075 0x12efc90e 0x12ebb0e9 0x12ebe0cb 0x12e1da18 0x12a8c857 0x1219aa62 0x1218fc82 0x1218fdaa 0x11e3f120 0x12dfa3df 0x12df42a4 0x12e00fa8 0x158cdc63 0x158ce38f 0x15a36b72 0x15a35110 0x15a33918 0x8adf12d 0x7fb5e0fe3609 0x7fb5e0f0a293
    2021.02.02 05:27:16.408985 [ 97 ] {} <Fatal> BaseDaemon: 5. ./obj-x86_64-linux-gnu/../src/IO/WriteHelpers.h:64: DB::JSONEachRowRowOutputFormat::writeRowStartDelimiter() @ 0x12f3dbeb in /usr/bin/clickhouse
    2021.02.02 05:27:16.534797 [ 97 ] {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../src/Processors/Formats/IRowOutputFormat.cpp:80: DB::IRowOutputFormat::write(std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > > const&, unsigned long) @ 0x12efd075 in /usr/bin/clickhouse
    2021.02.02 05:27:16.599210 [ 97 ] {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../src/Processors/Formats/IRowOutputFormat.cpp:0: DB::IRowOutputFormat::consume(DB::Chunk) @ 0x12efc90e in /usr/bin/clickhouse
    2021.02.02 05:27:16.680733 [ 97 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3211: DB::IOutputFormat::write(DB::Block const&) @ 0x12ebb0e9 in /usr/bin/clickhouse
    2021.02.02 05:27:16.798768 [ 97 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Processors/Formats/OutputStreamToOutputFormat.cpp:15: DB::OutputStreamToOutputFormat::write(DB::Block const&) @ 0x12ebe0cb in /usr/bin/clickhouse
    2021.02.02 05:27:16.981593 [ 97 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/DataStreams/MaterializingBlockOutputStream.h:0: DB::MaterializingBlockOutputStream::write(DB::Block const&) @ 0x12e1da18 in /usr/bin/clickhouse
    2021.02.02 05:27:17.090383 [ 97 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Storages/RabbitMQ/RabbitMQBlockOutputStream.cpp:61: DB::RabbitMQBlockOutputStream::write(DB::Block const&) @ 0x12a8c857 in /usr/bin/clickhouse
    2021.02.02 05:27:17.140425 [ 97 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../src/DataStreams/AddingDefaultBlockOutputStream.cpp:0: DB::AddingDefaultBlockOutputStream::write(DB::Block const&) @ 0x1219aa62 in /usr/bin/clickhouse
    2021.02.02 05:27:17.190282 [ 97 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../src/DataStreams/SquashingBlockOutputStream.cpp:0: DB::SquashingBlockOutputStream::finalize() @ 0x1218fc82 in /usr/bin/clickhouse
    2021.02.02 05:27:17.240052 [ 97 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2844: DB::SquashingBlockOutputStream::writeSuffix() @ 0x1218fdaa in /usr/bin/clickhouse
    2021.02.02 05:27:17.294527 [ 97 ] {} <Fatal> BaseDaemon: 15. ./obj-x86_64-linux-gnu/../src/DataStreams/CountingBlockOutputStream.h:37: DB::CountingBlockOutputStream::writeSuffix() @ 0x11e3f120 in /usr/bin/clickhouse
    2021.02.02 05:27:17.405442 [ 97 ] {} <Fatal> BaseDaemon: 16. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:521: DB::TCPHandler::processInsertQuery(DB::Settings const&) @ 0x12dfa3df in /usr/bin/clickhouse
    2021.02.02 05:27:17.475822 [ 97 ] {} <Fatal> BaseDaemon: 17. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x12df42a4 in /usr/bin/clickhouse
    2021.02.02 05:27:17.609845 [ 97 ] {} <Fatal> BaseDaemon: 18. ./obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1419: DB::TCPHandler::run() @ 0x12e00fa8 in /usr/bin/clickhouse
    2021.02.02 05:27:17.695292 [ 97 ] {} <Fatal> BaseDaemon: 19. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x158cdc63 in /usr/bin/clickhouse
    2021.02.02 05:27:17.833612 [ 97 ] {} <Fatal> BaseDaemon: 20. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:0: Poco::Net::TCPServerDispatcher::run() @ 0x158ce38f in /usr/bin/clickhouse
    2021.02.02 05:27:17.907391 [ 97 ] {} <Fatal> BaseDaemon: 21. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/ScopedLock.h:36: Poco::PooledThread::run() @ 0x15a36b72 in /usr/bin/clickhouse
    2021.02.02 05:27:18.033949 [ 97 ] {} <Fatal> BaseDaemon: 22. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:56: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x15a35110 in /usr/bin/clickhouse
    2021.02.02 05:27:18.122761 [ 97 ] {} <Fatal> BaseDaemon: 23. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/SharedPtr.h:277: Poco::ThreadImpl::runnableEntry(void*) @ 0x15a33918 in /usr/bin/clickhouse
    2021.02.02 05:27:18.283663 [ 97 ] {} <Fatal> BaseDaemon: 24. __tsan_thread_start_func @ 0x8adf12d in /usr/bin/clickhouse
    2021.02.02 05:27:18.284153 [ 97 ] {} <Fatal> BaseDaemon: 25. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    2021.02.02 05:27:18.284544 [ 97 ] {} <Fatal> BaseDaemon: 26. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.02.02 05:27:21.675458 [ 97 ] {} <Fatal> BaseDaemon: Calculated checksum of the binary: B53B58DB6CF8186EEC9EA9273F135E44. There is no information about the reference checksum.
    2021.02.02 05:27:28.078805 [ 1 ] {} <Fatal> Application: Child process was terminated by signal 11.
 2021-02-03 23:46:28 +03:00
alexey-milovidov
ca06d3b317
Merge branch 'master' into fix-rocksdb 2021-02-03 21:28:50 +03:00
alexey-milovidov
402c018d1e
Merge pull request #19947 from azat/dist-INSERT-block-structure-mismatch 
Fix "Block structure mismatch" for INSERT into Distributed
 2021-02-03 21:27:45 +03:00
Alexey Milovidov
dc2afc4795 Fix double whitespace #18383 2021-02-03 21:12:14 +03:00
Alexey Milovidov
82ab793731 Fix missing type check in StorageEmbeddedRocksDB 2021-02-03 20:35:23 +03:00
Alexander Tokmakov
87502d0220 Merge branch 'thread_state_improvements' into database_replicated 2021-02-03 20:19:35 +03:00
Alexander Tokmakov
d010f97db0 Merge branch 'master' into database_replicated 2021-02-03 20:13:25 +03:00
Pavel Kovalenko
b7151aa754 Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata 
# Conflicts:
#	src/Disks/DiskDecorator.h
#	src/Disks/IDisk.h
#	src/Disks/S3/DiskS3.cpp
 2021-02-03 14:22:18 +03:00
Azat Khuzhin
fabace394d Cleanup message brokers output streams 2021-02-03 00:32:36 +03:00
Alexander Tokmakov
6456ccf0da better test 2021-02-02 22:39:04 +03:00
Alexey Milovidov
12485eee6b Fix some of the issues found by Coverity 2021-02-02 22:07:23 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
Nikolai Kochetov
b2d7790794 Fix test. 2021-02-02 17:30:54 +03:00
Alexander Tokmakov
0073c87d5d fix 2021-02-02 13:32:42 +03:00
feng lv
0edf65c094 fix test 
fix

update

fix spell
 2021-02-02 09:22:30 +00:00
tavplubix
dbcddbcebd
Merge pull request #19770 from stigsb/materializemysql_system_parts 
Show MaterializeMySQL tables in system.parts
 2021-02-02 11:43:42 +03:00
Maksim Kita
d0151de4bb
Merge pull request #19608 from kreuzerkrieg/Add_IStoragePolicy_interface 
Add IStoragePolicy interface
 2021-02-02 11:03:20 +03:00
feng lv
4279c7da41 add setting insert_shard_id 
add test

fix style

fix
 2021-02-02 04:26:59 +00:00
Pavel Kruglov
a3f1b825cc Fix build 2021-02-01 21:17:12 +03:00
alexey-milovidov
6b2f0435c8
Merge pull request #19375 from Avogar/select-final 
Improve do_not_merge_across_partitions_select_final optimization.
 2021-02-01 20:31:08 +03:00
alesapin
2aa8a6304b
Merge pull request #15450 from CurtizJ/fix-ttl-group-by 
Fix some cases of TTL expressions
 2021-02-01 16:48:07 +03:00
Nikolai Kochetov
19e4a33f9d
Merge pull request #19544 from amosbird/limitconcurrency 
Per MergeTree table query limit
 2021-02-01 16:09:12 +03:00
robot-clickhouse
cd82eed093 Auto version update to [21.3.1.1] [54448] 2021-02-01 12:50:56 +03:00
tavplubix
4f955be832
fix sync 2021-02-01 12:32:36 +03:00
Stig Bakken
bedc472bb0 Show MaterializeMySQL tables in system.parts 2021-02-01 16:59:10 +08:00
alesapin
d9598c47b4
Merge pull request #19702 from ClickHouse/fix_rare_bug_after_part_corruption 
Fix incorrect virtual_parts after part corruption
 2021-02-01 10:09:48 +03:00
alexey-milovidov
08f713f177
Merge pull request #14822 from vzakaznikov/live_view_periodic_refresh 
Adding support for periodically refreshed LIVE VIEW tables
 2021-02-01 06:11:30 +03:00
alexey-milovidov
d93dad0ec3
Merge pull request #19722 from azat/background_message_broker_schedule_pool_size 
Add separate pool for message brokers (RabbitMQ and Kafka)
 2021-01-31 06:07:45 +03:00
Vitaliy Zakaznikov
dc9c3c8514 Fixing styling errors. 
Adding simple tests.
 2021-01-30 09:00:42 -05:00
Vitaliy Zakaznikov
3de4690450 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into live_view_periodic_refresh 2021-01-29 19:57:13 -05:00
Alexander Kuzmenkov
98e88d7305
Merge pull request #19793 from excitoon-favorites/fixcompressions3 
Fixed table function S3 `auto` compression mode
 2021-01-29 23:42:05 +03:00
alexey-milovidov
d19feb724b
Merge pull request #19799 from CurtizJ/fix-uint8-filtering 
Fix filtering by Uint8 greater than 127
 2021-01-29 21:34:42 +03:00
Denis Glazachev
31706e69bc Merge branch 'master' into row-policy-with-prewhere 
* master: (75 commits)
  Temporary retrun dicttoxml to integration tests runner
  Fix option
  Fix dependency on ODBC for Yandex internal build
  Remove useless headers
  Add test-connect tool
  Revert "Remove old non-automated test"
  Fix trivial bug in arrayEnumerateUniq
  Fix UBSan report in "round"
  Translate comment in test
  Added test to ANTLR skip_list.json
  Update Client.cpp
  Fixed race in tests
  Fixed test_kafka_flush_by_time test.
  LowCardinality UUID fix
  Update mergetree.md
  Clickhouse client query param CTE added test
  correct style zh doc typo
  Fixed data race in function DictGetNoType
  add empty line after error messages in client
  Fixed style issues
  ...

# Conflicts:
#	src/Storages/MergeTree/MergeTreeReadPool.h
 2021-01-29 22:26:11 +04:00
Pavel Kruglov
78371e15dc Update test, reduce num_threads_for_lonely_parts if data is small 2021-01-29 21:00:08 +03:00
alesapin
f4236fd765 Fix style 2021-01-29 20:12:53 +03:00
Nikolai Kochetov
47a7273fe3 Fix build 2021-01-29 19:21:52 +03:00
Nikolai Kochetov
afdc9635cb Update MergeTreeRangeReader 2021-01-29 19:12:53 +03:00
alesapin
c373d92a80 Less strict check 2021-01-29 18:50:08 +03:00
Nikolai Kochetov
4929fe2063 Update MergeTreeRangeReader 2021-01-29 18:13:09 +03:00
Pavel Kruglov
71f4acd48b Use one pool for lonely parts, update tests 2021-01-29 17:30:14 +03:00
Pavel Kruglov
a437ee4e31 Merge branch 'master' of github.com:ClickHouse/ClickHouse into select-final 2021-01-29 14:25:47 +03:00
Kruglov Pavel
caef103837
Merge branch 'master' into Add_IStoragePolicy_interface 2021-01-29 14:00:12 +03:00
Anton Popov
031132038b fix filtering by uint8 greater than 127 2021-01-29 10:39:18 +03:00
Alexey Milovidov
5a92e633a1 Remove useless headers 2021-01-29 10:37:46 +03:00
Vladimir Chebotarev
d729aacb09 Fixed table function S3 auto compression mode. 2021-01-29 07:54:52 +03:00
alexey-milovidov
fa7a795c3a
Merge pull request #19588 from ClickHouse/disable-checksums-on-read 
Allow to disable checksums on read
 2021-01-29 02:37:26 +03:00
alesapin
2881c830e3 Merge branch 'master' into fix_rare_bug_after_part_corruption 2021-01-28 23:16:52 +03:00
Denis Glazachev
009d1e3fc5 Merge branch 'master' into row-policy-with-prewhere 
* master: (247 commits)
  Update 01014_lazy_database_basic.sh
  Update 00459_group_array_insert_at.sql
  Same for ALTER
  Syntax and links
  Added test.
  Check where and prewhere identifiers exist.
  Remove redundant lsof
  Add lsof to fasttest
  Add new line to status
  Make integration odbc tests idempotent
  Minor code improvement in JOIN
  Arcadia does not have bitmaps
  Make Fuzzer more reliable
  Add missing lsof for fasttest docker image
  Update test
  The test most likely would not work in Arcadia
  Less parser depth
  Fix stack overflow in coroutine
  Fuzzer: better messages.
  Fixed build.
  ...
 2021-01-28 21:48:12 +04:00
alesapin
be1104d6c1 Fix typos 2021-01-28 19:07:47 +03:00
alesapin
c1d18fc8a2 Fix very rare case 2021-01-28 18:10:53 +03:00
tavplubix
10460313ad
Update StorageReplicatedMergeTree.cpp 2021-01-28 17:48:09 +03:00
Alexander Tokmakov
ffaa8e34a6 minor code improvements around ThreadStatus 2021-01-28 16:57:36 +03:00
Alexander Tokmakov
52e5c0aad7 fix thread status 2021-01-28 16:48:17 +03:00
alesapin
a7ca26fec4
Merge pull request #19708 from ClickHouse/fix_flaky_test_multiple_ttl 
Fix flaky test test_concurrent_ttl_merges
 2021-01-28 15:26:56 +03:00
alesapin
d14f4a525b Merge branch 'master' into fix_rare_bug_after_part_corruption 2021-01-28 11:09:41 +03:00
Alexey Milovidov
7086402859 Merge branch 'master' into disable-checksums-on-read 2021-01-28 11:08:13 +03:00
alesapin
402e031a22 Throw exception only in debug mode 2021-01-28 11:07:18 +03:00
Vladimir Chebotarev
579f8da573 Added SSE-C support in S3 client. 2021-01-28 09:32:41 +03:00
alexey-milovidov
7c240d0ca7
Merge pull request #17870 from excitoon-favorites/s3testiostream 
Update of AWS C++ SDK
 2021-01-28 07:12:52 +03:00
Alexey Milovidov
065d22cf2b Merge branch 'master' into disable-checksums-on-read 2021-01-28 03:07:31 +03:00
Anton Popov
e5125b8c73 add comments and test for compatibility 2021-01-28 02:39:15 +03:00
alesapin
c74631c650 Fix test and add logical error 2021-01-27 21:54:05 +03:00
Azat Khuzhin
acde56ff8a Add separate pool for message brokers (RabbitMQ and Kafka) 
background_message_broker_schedule_pool_size was introduced in #13939,
but never used actually, use it for RabbitMQ and Kafka.
 2021-01-27 21:08:41 +03:00
Anton Popov
a8f3078ce9 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-27 19:48:55 +03:00
alesapin
5622e6daa6 Fix rare max_number_of_merges_with_ttl_in_pool limit overrun for non-replicated MergeTree 2021-01-27 14:56:12 +03:00
alexey-milovidov
c3828a1fb2
Merge pull request #19660 from CurtizJ/sync-directory 
Better abstractions in disk interface
 2021-01-27 13:47:53 +03:00
alesapin
6fc39b10d3 Spelling 2021-01-27 13:11:48 +03:00
alesapin
01c8b9e1b1 Fix rare bug when some replicated operations (like mutation) cannot process some parts after data corruption 2021-01-27 13:07:18 +03:00
Vladimir Chebotarev
afdea5eff0 Changed handling of PocoHTTPClientConfiguration in order to ensure AWS SDK is initialized. 2021-01-27 09:38:35 +03:00
Alexey Milovidov
48b4d98b21 Amend 2021-01-27 04:48:41 +03:00
Alexey Milovidov
a6f5f40a65 Merge branch 'master' into disable-checksums-on-read 2021-01-27 04:09:00 +03:00
Alexander Tokmakov
f20d5e3b41 fix 2021-01-26 20:51:25 +03:00
vdimir
fc8b8d23b2
Merge pull request #18812 from excitoon-favorites/betters3endpoints 
Added prefix-based S3 endpoint settings
 2021-01-26 20:47:57 +03:00
Anton Popov
c7070da85a better abstractions in disk interface 2021-01-26 17:49:35 +03:00
Denis Glazachev
0d1c9479f8 Fix compilation/linter 2021-01-26 18:39:12 +04:00
kreuzerkrieg
29a2ef3089 Add IStoragePolicy interface 2021-01-26 10:55:28 +02:00
Vladimir Chebotarev
25037edfa6 Removed space in winking face smiley. 2021-01-26 11:28:47 +03:00
tavplubix
2d6a71fced
Merge pull request #19443 from ClickHouse/fix_virtual_parts_in_parts_to_do 
Addition to #15537
 2021-01-26 11:27:14 +03:00
Amos Bird
66fe97d8bd
Per MergeTree table query limit 2021-01-26 14:03:31 +08:00
Alexey Milovidov
8dfa933028 Amend 2021-01-25 23:48:10 +03:00
Alexey Milovidov
9ee5c1535e Allow to disable checksums on read 2021-01-25 23:29:04 +03:00
Denis Glazachev
11b53d3b9d Fix compilation/linter 2021-01-26 00:17:48 +04:00
Denis Glazachev
ff5ce1a5ae Fix compilation 2021-01-25 22:01:59 +04:00
Anton Popov
658f24dcff
Merge pull request #19358 from CurtizJ/fix-subcolumns 
Fix several cases, while reading subcolumns
 2021-01-25 20:26:07 +03:00
Denis Glazachev
4f6c880232 Pass and handle a chain of multiple prewhere infos 2021-01-25 18:31:59 +04:00
tavplubix
5f07bfb9f8
Update ReplicatedMergeTreeQueue.cpp 2021-01-25 16:15:47 +03:00
Alexander Tokmakov
3bd4d97353 Merge branch 'master' into database_replicated 2021-01-25 14:19:04 +03:00
tavplubix
a88a564aae
Update ReplicatedMergeTreeQueue.cpp 2021-01-25 12:51:06 +03:00
alesapin
fe133580ee
Merge pull request #19346 from azat/system.parts-_state-fix 
Fix system.parts _state column
 2021-01-25 10:11:19 +03:00
alexey-milovidov
ba3e064a63
Merge pull request #19371 from kssenii/test-coverage-with-factories 
test coverage with factories
 2021-01-25 07:11:09 +03:00
Vladimir Chebotarev
8ba8e8efd5 Added prefix-based S3 endpoint settings. 2021-01-25 06:40:57 +03:00
alexey-milovidov
73501102f3
Merge pull request #19528 from azat/merge_tree_min_for_concurrent_read-SIGSEGV 
Fix SIGSEGV with merge_tree_min_rows_for_concurrent_read/merge_tree_min_bytes_for_concurrent_read=0/UINT64_MAX
 2021-01-25 00:18:42 +03:00
Azat Khuzhin
1c364b6ee3 Fix SIGSEGV with merge_tree_min_rows_for_concurrent_read/merge_tree_min_bytes_for_concurrent_read=0/UINT64_MAX 
In case of 0 or too huge value it will try to read not existing marks
and got:

    Logical error: 'Trying to get non existing mark 11936128518282651045, while size is 2'.
 2021-01-24 14:39:57 +03:00
alexey-milovidov
7a10fbc9c3
Merge pull request #19479 from ClickHouse/columns-description-idiotic 
Fix wrong serde of ColumnsDescription
 2021-01-23 12:08:15 +03:00
alexey-milovidov
7f059e3765
Merge pull request #19449 from azat/dist-send-check-block 
Mark distributed batch as broken in case of empty data block in one of files
 2021-01-23 12:04:31 +03:00
Alexey Milovidov
7005e3a826 Fix wrong serde of ColumnsDescription 2021-01-23 04:11:18 +03:00
Alexey Milovidov
7d2108d4e9 Fix double whitespace 2021-01-23 02:57:35 +03:00
Azat Khuzhin
109dbe5df4 Check the stream before sending while hanlding async INSERTs into Distributed 
It is possible to get corruption (even though it is very unlikely, and
initially it wasn't corruption) just before the data block goes in the
file on disk, and in case of batching, it will break the packets, since
it will write the packet type but will not write any data after.
 2021-01-22 21:29:58 +03:00
Alexander Tokmakov
963dadae54 fix wrong parts in parts_to_do 2021-01-22 19:07:19 +03:00
tavplubix
9801bfc0ee
Merge branch 'master' into fix_18063 2021-01-22 17:28:05 +03:00
alexey-milovidov
6e1d660e36
Merge pull request #19390 from ClickHouse/http_referer 
Add http_referer to client info
 2021-01-22 17:25:09 +03:00
alexey-milovidov
9818bd319a
Merge pull request #19381 from azat/parts-types-metrics 
Add metrics for MergeTree parts (Wide/Compact/InMemory) types
 2021-01-22 17:24:50 +03:00
Alexander Kuzmenkov
b58a696b24
Merge pull request #19379 from azat/Buffer-less-lock-contention 
Reduce lock contention for multiple layers of the Buffer engine
 2021-01-22 13:17:34 +03:00
alexey-milovidov
4e11e7cfa9
Merge pull request #18772 from azat/optimize-memory-tracking-fix 
[RFC] Fix memory tracking for OPTIMIZE TABLE/merges
 2021-01-22 03:48:06 +03:00
Alexey Milovidov
afc6f2fd8e Add http_referer to client info 2021-01-22 01:55:45 +03:00
kssenii
daab2c91bb Better 2021-01-21 21:15:11 +00:00
Azat Khuzhin
b0a80af888 Reduce lock contention for multiple layers of the Buffer engine 
Otherwise you can see something like this for the following query:

    ```sql
    WITH
        arrayMap(x -> demangle(addressToSymbol(x)), s.trace) AS trace_array,
        arrayStringConcat(trace_array, '\n') AS trace_string
    SELECT
        p.thread_id,
        p.query_id,
        p.query,
        trace_string
    FROM
    (
        SELECT
            query_id,
            query,
            arrayJoin(thread_ids) AS thread_id
        FROM system.processes
    ) AS p
    INNER JOIN system.stack_trace AS s ON p.thread_id = s.thread_id
    ORDER BY p.query_id ASC
    SETTINGS enable_global_with_statement = 0, allow_introspection_functions = 1
    FORMAT PrettyCompactNoEscapes
    ```

Lots of the following:

    ```sql
    INSERT INTO buffer (...) VALUES

    __lll_lock_wait
    pthread_mutex_lock
    std::__1::mutex::lock()
    DB::StorageBuffer::reschedule()
    DB::BufferBlockOutputStream::write(DB::Block const&)
    ```

That will wait one of this:

    ```
    INSERT INTO buffer (...) VALUES

    ...
    DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
    DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
    DB::SquashingBlockOutputStream::finalize()
    DB::SquashingBlockOutputStream::writeSuffix()
    DB::PushingToViewsBlockOutputStream::writeSuffix()
    DB::StorageBuffer::writeBlockToDestination(DB::Block const&, std::__1::shared_ptr<DB::IStorage>)
    DB::StorageBuffer::flushBuffer(DB::StorageBuffer::Buffer&, bool, bool, bool)
    ```

P.S. we cannot simply unlock the buffer during flushing, see comments in
the code
 2021-01-21 22:09:24 +03:00
Azat Khuzhin
cb951c2116 Add metrics for MergeTree parts types 
- PartsWide
- PartsCompact
- PartsInMemory
 2021-01-21 21:17:00 +03:00
Azat Khuzhin
c68f7cd5b1 Measure time that spend during flush of the Buffer to the underlying 2021-01-21 21:11:39 +03:00
Pavel Kruglov
900580af02 Add parallel select when there is one part with level>0 in select final 2021-01-21 20:34:50 +03:00
kssenii
c1702f34ee Add factories info into system.query_log 2021-01-21 15:46:37 +00:00
alexey-milovidov
062f00aa5d
Merge pull request #19290 from azat/dist-broken-on-EOF-fix 
Do not mark file for distributed send as broken on EOF
 2021-01-21 17:00:36 +03:00
tavplubix
ac50b1b231
Update StorageMaterializedView.cpp 2021-01-21 15:57:18 +03:00
Anton Popov
ac3de63a71 fix several cases, while reading subcolumns 2021-01-21 15:34:11 +03:00
alesapin
fe6b964b32 Revert "Revert "Auto version update to [21.2.1.1] [54446]"" 
This reverts commit 42f63e14b5.
 2021-01-21 12:39:46 +03:00
Azat Khuzhin
5f3059555a Fix system.parts _state column 
There was LOGICAL_ERROR when querying this column, due to incorrect order:

    SELECT
        *,
        _state
    FROM system.parts

    2021.01.21 10:22:57.731556 [ 22851 ] {02a07c6d-467d-4681-9203-4dc11cc6fbee} <Fatal> : Logical error: 'Invalid Field get from type String to type UInt64'.
 2021-01-21 10:41:23 +03:00
Alexander Tokmakov
6560ec3ed5 fix segfault on aggregation when MV has unexpected structure 2021-01-20 19:36:18 +03:00
Anton Ivashkin
357d98eb36 Merge master 2021-01-20 12:23:03 +03:00
Azat Khuzhin
8a00816396 Do not mark file for distributed send as broken on EOF 
- the sender will got ATTEMPT_TO_READ_AFTER_EOF (added in
  946c275dfb) when the client just go
  away, i.e. server had been restarted, and this is incorrect to mark the
  file as broken in this case.

- since #18853 the file will be checked on the sender locally, and
  in case the file was truncated CANNOT_READ_ALL_DATA will be thrown.
  But before #18853 the sender will not receive
  ATTEMPT_TO_READ_AFTER_EOF from the client in case of file was truncated
  on the sender, since the client will just wait for more data, IOW just hang.

- and I don't see how ATTEMPT_TO_READ_AFTER_EOF can be received while
  reading local file.
 2021-01-20 01:10:17 +03:00
alesapin
70d9c7ea03
Merge pull request #19237 from ClickHouse/fix_wait_for_log_entries 
Don't wait forever for log update after table was dropped
 2021-01-19 10:22:52 +03:00
Anton Ivashkin
eba98b04b0 Zero copy replication over S3: Hybrid storage support 2021-01-18 19:16:45 +03:00
Alexander Tokmakov
7f97a11c84 Merge branch 'master' into database_replicated 2021-01-18 17:09:39 +03:00
alesapin
4ee96869a2 Don't wait forever for log update after table was dropped 2021-01-18 15:15:07 +03:00
Pavel Kovalenko
1e3a059f64 Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata 
# Conflicts:
#	src/Disks/DiskCacheWrapper.cpp
#	src/Disks/S3/DiskS3.cpp
 2021-01-18 13:39:49 +03:00
Alexander Kuzmenkov
d22c04568d
Merge pull request #13405 from excitoon-favorites/s3keepalive 
Connection pools for S3
 2021-01-18 12:52:37 +03:00
alexey-milovidov
1444ccb85e
Merge pull request #19206 from azat/dist-pull 
Tiny changes in DistributedBlockOutputStream
 2021-01-18 01:04:50 +03:00
alexey-milovidov
15f4ae26c2
Merge pull request #17310 from CurtizJ/multiple-nested 
Allow nested with multiple nesting and subcolumns of complex types
 2021-01-17 15:00:26 +03:00
Azat Khuzhin
a6631287a7 DistributedBlockOutputStream: add more comments 2021-01-17 12:50:37 +03:00
Azat Khuzhin
b725e1d131 DistributedBlockOutputStream: Remove superfluous brackets for string construction 2021-01-17 12:48:51 +03:00
Azat Khuzhin
2955e25e83 Fix inserted blocks accounting for insert_distributed_one_random_shard=1 
It is tricky due to block splitting

Refs: https://github.com/ClickHouse/ClickHouse/pull/18294
 2021-01-17 12:45:42 +03:00
Azat Khuzhin
858f07c796 Update comment for query AST cloning during inesrt into multiple local shards 
Refs: https://github.com/ClickHouse/ClickHouse/pull/18264#discussion_r558839456
 2021-01-17 12:40:41 +03:00
alexey-milovidov
7de745ce77
Merge pull request #18554 from kssenii/pg2ch 
Add PostgreSQL table function, dictionary source, database engine
 2021-01-16 23:55:05 +03:00
Alexey Milovidov
029302d766 Merge with master 2021-01-16 17:09:44 +03:00
Alexey Milovidov
24c8e53440 Merge branch 'master' into multiple-nested 2021-01-16 16:28:40 +03:00
alexey-milovidov
e67e4588e7
Update PostgreSQLConnection.h 2021-01-16 15:26:21 +03:00
alexey-milovidov
a15092eeb7
Merge pull request #18264 from ucasFL/insert-cluster 
Support insert into table function cluster
 2021-01-16 13:22:49 +03:00
alexey-milovidov
a5a19de878
Update DistributedBlockOutputStream.cpp 2021-01-16 13:22:25 +03:00
alexey-milovidov
4efc7a7dc3
Update MergeTreeIOSettings.h 2021-01-16 13:07:58 +03:00
alexey-milovidov
e32b1e3fe6
Update StorageTinyLog.cpp 2021-01-16 11:53:28 +03:00
Alexey Milovidov
4ee9c57190 Add TODO 2021-01-16 11:43:59 +03:00
Alexey Milovidov
b25e334313 Add assert 2021-01-16 11:30:35 +03:00
alexey-milovidov
2e2988e5d8
Merge pull request #19146 from azat/server-memory-limit-blocking 
MemoryTracker: Do not ignore server memory limits during blocking by default
 2021-01-16 11:09:19 +03:00
alesapin
67fd381034
Merge pull request #19123 from ClickHouse/additional_check_in_writer 
Fix max granules size in MergeTreeDataWriter
 2021-01-16 10:17:44 +03:00
alexey-milovidov
5f189c5756
Merge pull request #19122 from ClickHouse/data-part-better-code 
Add metrics for part number in MergeTree in ClickHouse
 2021-01-16 00:20:15 +03:00
alexey-milovidov
e09ec9fed5
Merge pull request #18373 from amosbird/fix-18364 
Fix 2-arg functions with constant in PK analysis
 2021-01-16 00:20:03 +03:00
Alexey Milovidov
67f207d8ca More safe connection string 2021-01-16 00:18:51 +03:00
Alexey Milovidov
52cfc1d110 Minor changes 2021-01-15 23:09:31 +03:00
Alexey Milovidov
ba1921f380 Minor changes 2021-01-15 22:59:49 +03:00
Azat Khuzhin
61b2d0ce42 MemoryTracker: Do not ignore server memory limits during blocking by default 2021-01-15 22:46:58 +03:00
Alexey Milovidov
b293d56a63 Merge branch 'master' into kssenii-pg2ch 2021-01-15 22:32:03 +03:00
alesapin
184dbedb06 Fix stupid error 2021-01-15 21:50:30 +03:00
alexey-milovidov
b97beea22a
Merge pull request #19101 from ClickHouse/check_compression_codec_read 
Fix compression codec read for empty files
 2021-01-15 20:55:58 +03:00
alexey-milovidov
971ff2ee0a
Merge pull request #19086 from ClickHouse/faster-parts-removal 
Faster parts removal, more safe and efficient interface of IDisk
 2021-01-15 20:37:35 +03:00
feng lv
9829c09720 fix 2021-01-15 15:54:35 +00:00
Alexey Milovidov
a19e7edd14 Merge branch 'master' into kssenii-pg2ch 2021-01-15 17:33:19 +03:00
alexey-milovidov
ca825f36f1
Merge pull request #18853 from azat/dist-send-checksums 
Improve checksum checks for async INSERT into Distributed on the sender
 2021-01-15 17:12:54 +03:00
alesapin
8ccaa6ede9 Additional check for huge granules in MergeTreeDataWriter 2021-01-15 15:40:37 +03:00
Alexey Milovidov
e238fd64ac Add part metrics 2021-01-15 15:28:53 +03:00
Alexey Milovidov
6a2a5e53ed Slightly better code of IMergeTreeDataPart #18955 2021-01-15 15:15:13 +03:00
alexey-milovidov
78fff6bc39
Merge branch 'master' into multiple-nested 2021-01-15 14:54:27 +03:00
Alexey Milovidov
4bae04d500 Merge branch 'master' into amosbird/fix-18364 2021-01-15 14:37:35 +03:00
alexey-milovidov
4a71971b43
Update KeyCondition.cpp 2021-01-15 14:36:07 +03:00
alexey-milovidov
8d58ce532a
Merge pull request #19064 from CurtizJ/restrict-modify-ttl 
Restrict MODIFY TTL for tables created in old syntax
 2021-01-15 14:09:47 +03:00
Alexey Milovidov
d553e46a06 Merge branch 'master' into faster-parts-removal 2021-01-15 13:25:20 +03:00
alexey-milovidov
461d370e8d
Merge pull request #19020 from ClickHouse/fix_18879 
Fix duplicate UUIDs of LiveView on server startup
 2021-01-15 13:04:08 +03:00
alesapin
d601faa669
Merge pull request #18935 from fastio/bugfix_attach_partition_does_not_reset_mutation 
Bugfix: attach partition should reset the mutation
 2021-01-15 12:21:16 +03:00
alesapin
e106df2ad0 Fix comment 2021-01-15 12:10:03 +03:00
alesapin
0662d6bd7d Fix compression codec read for empty files 2021-01-15 12:04:23 +03:00
Azat Khuzhin
ecae6c1c60 Avoid reading the distributed batch just to read the block header 
Before this patch batched mode of the DirectoryMonitor is 2x slower then
non-batched, after it should be more or less the same as non-batched.
 2021-01-14 22:38:46 +03:00
Alexey Milovidov
8276a1c8d2 Faster parts removal, more safe and efficient interface of IDisk 2021-01-14 19:24:13 +03:00
alesapin
09e825e28c
Merge pull request #19040 from ClickHouse/fix_redundant_exception_while_dropping_part 
Avoid redundant exception while dropping part
 2021-01-14 18:59:59 +03:00
Anton Popov
ac426c3da6 restrict MODIFY TTL for tables created in old syntax 2021-01-14 15:32:20 +03:00
Alexander Tokmakov
64975c08e7 fix 2021-01-14 13:33:39 +03:00
alexey-milovidov
2ca185c9f9
Merge pull request #16995 from sundy-li/alias_column_partition_prune 
Add optimize_respect_aliases
 2021-01-14 11:30:45 +03:00
Nikolai Kochetov
7b33ad5e44
Merge pull request #19010 from ClickHouse/fix-insert-lc-to-tiny-log 
Fix insert LC to TinyLog
 2021-01-14 11:25:38 +03:00
alesapin
bfc27254b2 Avoid redundant exception while dropping part 2021-01-14 11:07:13 +03:00
alexey-milovidov
3b02a3ef02
Merge pull request #18976 from ClickHouse/fix_terminate_when_not_enough_memory 
Finally get rid of allocations in ThreadPool::worker(...)
 2021-01-13 23:48:46 +03:00
Alexander Tokmakov
5f0f8ae3cd fix duplicate UUIDs of LiveView on server startup 2021-01-13 21:13:46 +03:00
Anton Popov
0e903552a0 fix TTLs with WHERE 2021-01-13 17:04:27 +03:00
Nikolai Kochetov
da0cb8d47f Added comment. 2021-01-13 16:32:27 +03:00
ygrek
8f2a830d83
add zstd long range option (#17184) 
* add zstd long compression option

* tests: add zstd long read-write test

Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com>
Co-authored-by: ip <igor@ahrefs.com>
 2021-01-13 16:22:59 +03:00
Nikolai Kochetov
12f3b22623 Do not skip streams after serializeBinaryBulkStatePrefix. 2021-01-13 15:20:10 +03:00
Alexander Tokmakov
a6510cc4b7 Merge branch 'master' into fix_terminate_when_not_enough_memory 2021-01-13 14:05:54 +03:00
alesapin
c97469773d
Merge pull request #18969 from ClickHouse/fix_drop_part_deduplication 
Fix DROP PART query break deduplication
 2021-01-13 11:50:26 +03:00
alesapin
73e536a074
Merge pull request #18928 from ClickHouse/more_checks_in_writer_wide 
More checks in merge tree writer wide
 2021-01-13 09:59:45 +03:00
Anton Popov
d7200ee2ed minor changes 2021-01-13 02:20:32 +03:00
Pavel Kovalenko
b09862b7b9 Ability to backup-restore metadata files for DiskS3 (fixes and tests) 2021-01-12 20:18:40 +03:00
Anton Popov
15ead18673 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-12 19:46:10 +03:00
Anton Popov
60b88986bf minor changes near TTL computation 2021-01-12 19:42:49 +03:00
Alexander Tokmakov
ff6c3c75c2 add protection from unsafe allocations 2021-01-12 18:41:24 +03:00
Anton Popov
aed8c78d0d better check for existence of aggregate function 2021-01-12 18:35:07 +03:00
Anton Popov
58b9ef5a10 fix TTL info serialization 2021-01-12 17:14:47 +03:00
alesapin
dead1016d6 Fix deduplication block names parsing 2021-01-12 13:55:02 +03:00
sundy-li
ad01534bd5 Merge branch 'master' into alias_column_partition_prune 2021-01-12 18:28:30 +08:00