thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
91,550 Commits 9,326 Branches 1,783 Tags 2.1 GiB
106f92dcdb
Commit Graph

8453 Commits

Author SHA1 Message Date
mergify[bot]
2d01abf871
Merge branch 'master' into revert-37647-Fix-all-CheckTriviallyCopyableMove-Errors 2022-06-07 13:32:30 +00:00
Nikita Taranov
d09ad5fcd5
Fix build 
https://s3.amazonaws.com/clickhouse-test-reports/36593/a4b9614385e0ecd1654d15f507159678758ec557/fast_test__actions_/build_log.txt
 2022-06-07 13:56:03 +02:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type" 2022-06-07 13:53:10 +02:00
Anton Popov
ef6f5a6500
Merge pull request #37570 from azat/column-ttl-expired-fix 
Do not write expired columns by TTL after subsequent merges
 2022-06-07 13:05:03 +02:00
Kseniia Sumarokova
c4b1f537cd
Merge pull request #37879 from Vxider/wv-inner-target-drop 
Do not drop the inner target table when alter WindowView
 2022-06-07 13:00:15 +02:00
Kseniia Sumarokova
edc6b68801
Merge pull request #37853 from Vxider/fire-when-inserted 
Fire and clean windows in WindowView only when data is inserted
 2022-06-07 11:24:50 +02:00
Vitaly Baranov
d199478169
Merge pull request #37303 from ClickHouse/fix_trash 
Try to fix some trash
 2022-06-07 10:17:39 +02:00
taiyang-li
73a484256e Merge branch 'master' into async_hdfs_read_buffer 2022-06-07 12:16:46 +08:00
taiyang-li
c65c56fd48 fix typo 2022-06-07 09:58:29 +08:00
mergify[bot]
e237e44b69
Merge branch 'master' into fire-when-inserted 2022-06-06 19:22:55 +00:00
mergify[bot]
dd0603a595
Merge branch 'master' into metadata_storage 2022-06-06 18:59:45 +00:00
Anton Popov
d40b23272e
Merge pull request #37755 from CurtizJ/fix-mutations-again 
Return back #37266
 2022-06-06 20:22:03 +02:00
Vxider
acda04a3d4 do not drop inner target table when alter windowview 2022-06-06 05:06:02 +00:00
taiyang-li
b36d9f8143 refactor readinto 2022-06-06 12:58:22 +08:00
Robert Schulze
2d87af2a15
Merge pull request #37647 from DevTeamBK/Fix-all-CheckTriviallyCopyableMove-Errors 
Fix errors of CheckTriviallyCopyableMove type
 2022-06-05 19:58:47 +02:00
alesapin
a25a01af16 Fix WAL 2022-06-05 16:15:11 +02:00
alesapin
38df94666b Fix merge tree wal 2022-06-05 15:09:39 +02:00
alesapin
a2704bc541 Trying to fix WAL log 2022-06-05 01:05:15 +02:00
Vxider
d4f0e8d4c2 update code style 2022-06-04 17:14:42 +00:00
Vxider
1f10276686 fire and clean only when data inserted 2022-06-04 16:55:46 +00:00
alesapin
083b15fea0 bug impl 2022-06-04 17:33:07 +02:00
Alexey Milovidov
4576b29e57
Update StorageReplicatedMergeTree.cpp 2022-06-04 14:33:29 +03:00
Bharat Nallan Chakravarthy
5fb9336159 add ssl version to build options 2022-06-04 15:31:08 +05:30
Alexey Milovidov
f860407af5
Merge pull request #37777 from ClickHouse/avx512_tail_zero 
Use multitarget framework for numZerosInTail implementation
 2022-06-04 01:46:56 +03:00
Nikolai Kochetov
3f67abc3b0 Fix some tests. 2022-06-03 18:49:12 +00:00
Kseniia Sumarokova
7632de911d
Merge pull request #37807 from Vxider/deley-cleanup 
Delay schedule of the cleaning task in WindowView
 2022-06-03 19:48:57 +02:00
Alexander Tokmakov
2358878c38 do not count covered unexpected parts 2022-06-03 16:32:02 +02:00
mergify[bot]
ddf7210ecc
Merge branch 'master' into remove-useless-code-2 2022-06-03 13:58:45 +00:00
Maksim Kita
8469e0d10d
ExecutableLambdaAdapter forward job and callback (#37830) 2022-06-03 15:54:47 +02:00
Nikita Mikhaylov
fe477355df
Added support for schema inference for hdfsCluster [2] (#35812) 2022-06-03 15:54:29 +02:00
alesapin
ac03e3af3e
Merge branch 'master' into metadata_storage 2022-06-03 14:42:21 +02:00
Anton Popov
5cf2c67410
Merge branch 'master' into fix-mutations-again 2022-06-03 14:17:05 +02:00
taiyang-li
f202c35311 Merge branch 'master' into async_hdfs_read_buffer 2022-06-03 17:52:09 +08:00
Kseniia Sumarokova
9c0e838404
Merge pull request #37770 from Vxider/fix-empty-target-id 
Fix empty target table id in WindowView
 2022-06-03 11:23:35 +02:00
Vxider
d3c907cf12 log exception when fire 2022-06-02 18:22:48 +00:00
Vxider
1282d038cc delay schedule of windowview cleanup 2022-06-02 18:21:56 +00:00
Nikolai Kochetov
8991f39412 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-06-02 17:00:08 +00:00
alesapin
46bccae078 Trying to create metadata layer 2022-06-02 18:09:40 +02:00
Nikolai Kochetov
25c8a437ea Fixing build. 2022-06-02 15:00:07 +00:00
HeenaBansal2009
e3080f2a97 Merge remote-tracking branch 'origin' into Fix-all-CheckTriviallyCopyableMove-Errors 2022-06-02 07:30:08 -07:00
Alexander Tokmakov
a893595025
Merge pull request #37596 from hanfei1991/hanfei/zk-write 
Support insert into system.zookeeper
 2022-06-02 15:09:07 +03:00
Kruglov Pavel
37a4d11c64
Merge pull request #37645 from azat/fix-write 
Fix possible "Cannot write to finalized buffer"
 2022-06-02 14:01:30 +02:00
mergify[bot]
fbfaaa9c74
Merge branch 'master' into fix-mutations-again 2022-06-02 11:57:19 +00:00
Alexander Tokmakov
45657d203a
Merge pull request #37763 from ClickHouse/fix_replace_range_again 
Fix flaky 01154_move_partition_long
 2022-06-02 14:15:16 +03:00
Alexander Gololobov
0b004b907a Added AVX2 implementation of numZerosInTail 2022-06-02 13:01:13 +02:00
Nikolai Kochetov
fc726c236f
Merge pull request #37661 from ClickHouse/no_alias_actions 
Trying to remove prewhere_info.alias_actions as they seem to always be included in prewhere_actions
 2022-06-02 12:31:04 +02:00
Nikolai Kochetov
89c5855d20 Merge branch 'master' into refactor-something-in-part-volumes 2022-06-02 12:19:07 +02:00
Nikita Mikhaylov
d34e051c69
Support for simultaneous read from local and remote parallel replica (#37204) 2022-06-02 11:46:33 +02:00
Nikolai Kochetov
edac3d6714 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-06-02 09:36:20 +00:00
Alexander Gololobov
d8e4dcf9e2 Use multitarget framework for numZerosInTail implementation 2022-06-02 11:23:44 +02:00
Igor Nikonov
8c40f05d4c
Merge pull request #37674 from xlwh/cleanup_unused 
Cleanup unused file
 2022-06-02 10:18:29 +02:00
mergify[bot]
88bed9de20
Merge branch 'master' into fix_replace_range_again 2022-06-02 07:05:44 +00:00
Alexander Gololobov
a0cf902d49
Merge pull request #37588 from yaqi-zhao/avx512_tail_zero 
add avx512 support for mergetree reader
 2022-06-02 08:51:52 +03:00
Han Fei
1424c420fa try to fill in right metadata columns 2022-06-02 13:41:37 +08:00
Vxider
8221fcd5f1 update var name 2022-06-02 04:38:45 +00:00
Vxider
df4db70bb8 fix empty target table id 2022-06-02 04:28:18 +00:00
lthaooo
6632616733
Fix TTL merge scheduling bug (#36387) 2022-06-01 21:09:53 +02:00
Azat Khuzhin
545a56ce45 Fix sinks with onException() handler 
It is possible to call onException() even after onFinish(), in case of
onFinish() throws, and in this case onException() should be no-op for
such sinks.

Also there can be caveats with PartitionedSync.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-01 21:50:30 +03:00
Azat Khuzhin
02af58f41d Fix possible "Cannot write to finalized buffer" 
It is still possible to get this error since onException does not
finalize format correctly.

Here is an example of such error, that was found by CI [1]:

<details>

    [ 2686 ] {fa01bf02-73f6-4f7f-b14f-e725de6d7f9b} <Fatal> : Logical error: 'Cannot write to finalized buffer'.
    [ 34577 ] {} <Fatal> BaseDaemon: ########################################
    [ 34577 ] {} <Fatal> BaseDaemon: (version 22.6.1.1, build id: AB8040A6769E01A0) (from thread 2686) (query_id: fa01bf02-73f6-4f7f-b14f-e725de6d7f9b) (query: insert into test_02302 select number from numbers(10) settings s3_truncate_on_insert=1;) Received signal Aborted (6)
    [ 34577 ] {} <Fatal> BaseDaemon:
    [ 34577 ] {} <Fatal> BaseDaemon: Stack trace: 0x7fcbaa5a703b 0x7fcbaa586859 0xfad9bab 0xfad9e05 0xfaf6a3b 0x24a48c7f 0x258fb9b9 0x258f2004 0x258b88f4 0x258b863b 0x2581773d 0x258177ce 0x24bb5e98 0xfad01d6 0xfad0105 0x2419b11d 0xfad01d6 0xfad0105 0x2215afbb 0x2215aa48 0xfad01d6 0xfad0105 0xfcc265d 0x225cc546 0x249a1c40 0x249bc1b6 0x2685902c 0x26859505 0x269d7767 0x269d504c 0x7fcbaa75e609 0x7fcbaa683163
    [ 34577 ] {} <Fatal> BaseDaemon: 3. raise @ 0x7fcbaa5a703b in ?
    [ 34577 ] {} <Fatal> BaseDaemon: 4. abort @ 0x7fcbaa586859 in ?
    [ 34577 ] {} <Fatal> BaseDaemon: 5. ./build_docker/../src/Common/Exception.cpp:47: DB::abortOnFailedAssertion(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfad9bab in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 6. ./build_docker/../src/Common/Exception.cpp:70: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xfad9e05 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 7. ./build_docker/../src/IO/WriteBuffer.h:0: DB::WriteBuffer::write(char const*, unsigned long) @ 0xfaf6a3b in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 8. ./build_docker/../src/Processors/Formats/Impl/ArrowBufferedStreams.cpp:47: DB::ArrowBufferedOutputStream::Write(void const*, long) @ 0x24a48c7f in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 9. long parquet::ThriftSerializer::Serialize<parquet::format::FileMetaData>(parquet::format::FileMetaData const*, arrow::io::OutputStream*, std::__1::shared_ptr<parquet::Encryptor> const&) @ 0x258fb9b9 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 10. parquet::FileMetaData::FileMetaDataImpl::WriteTo(arrow::io::OutputStream*, std::__1::shared_ptr<parquet::Encryptor> const&) const @ 0x258f2004 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 11. parquet::WriteFileMetaData(parquet::FileMetaData const&, arrow::io::OutputStream*) @ 0x258b88f4 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 12. parquet::ParquetFileWriter::~ParquetFileWriter() @ 0x258b863b in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 13. parquet::arrow::FileWriterImpl::~FileWriterImpl() @ 0x2581773d in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 14. parquet::arrow::FileWriterImpl::~FileWriterImpl() @ 0x258177ce in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 15. ./build_docker/../src/Processors/Formats/Impl/ParquetBlockOutputFormat.h:27: DB::ParquetBlockOutputFormat::~ParquetBlockOutputFormat() @ 0x24bb5e98 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 16. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:173: std::__1::__shared_count::__release_shared() @ 0xfad01d6 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 17. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:216: std::__1::__shared_weak_count::__release_shared() @ 0xfad0105 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 18.1. inlined from ./build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:312: std::__1::unique_ptr<DB::WriteBuffer, std::__1::default_delete<DB::WriteBuffer> >::reset(DB::WriteBuffer*)
    [ 34577 ] {} <Fatal> BaseDaemon: 18.2. inlined from ../contrib/libcxx/include/__memory/unique_ptr.h:269: ~unique_ptr
    [ 34577 ] {} <Fatal> BaseDaemon: 18. ../src/Storages/StorageS3.cpp:566: DB::StorageS3Sink::~StorageS3Sink() @ 0x2419b11d in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 19. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:173: std::__1::__shared_count::__release_shared() @ 0xfad01d6 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 20. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:216: std::__1::__shared_weak_count::__release_shared() @ 0xfad0105 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 21. ./build_docker/../contrib/abseil-cpp/absl/container/internal/raw_hash_set.h:1662: absl::lts_20211102::container_internal::raw_hash_set<absl::lts_20211102::container_internal::FlatHashMapPolicy<StringRef, std::__1::shared_ptr<DB::SinkToStorage> >, absl::lts_20211102::hash_internal::Hash<StringRef>, std::__1::equal_to<StringRef>, std::__1::allocator<std::__1::pair<StringRef const, std::__1::shared_ptr<DB::SinkToStorage> > > >::destroy_slots() @ 0x2215afbb in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 22.1. inlined from ./build_docker/../contrib/libcxx/include/string:1445: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long() const
    [ 34577 ] {} <Fatal> BaseDaemon: 22.2. inlined from ../contrib/libcxx/include/string:2231: ~basic_string
    [ 34577 ] {} <Fatal> BaseDaemon: 22. ../src/Storages/PartitionedSink.h:14: DB::PartitionedSink::~PartitionedSink() @ 0x2215aa48 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 23. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:173: std::__1::__shared_count::__release_shared() @ 0xfad01d6 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 24. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:216: std::__1::__shared_weak_count::__release_shared() @ 0xfad0105 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 25. ./build_docker/../contrib/libcxx/include/vector:802: std::__1::vector<std::__1::shared_ptr<DB::IProcessor>, std::__1::allocator<std::__1::shared_ptr<DB::IProcessor> > >::__base_destruct_at_end(std::__1::shared_ptr<DB::IProcessor>*) @ 0xfcc265d in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 26.1. inlined from ./build_docker/../contrib/libcxx/include/vector:402: ~vector
    [ 34577 ] {} <Fatal> BaseDaemon: 26.2. inlined from ../src/QueryPipeline/QueryPipeline.cpp:29: ~QueryPipeline
    [ 34577 ] {} <Fatal> BaseDaemon: 26. ../src/QueryPipeline/QueryPipeline.cpp:535: DB::QueryPipeline::reset() @ 0x225cc546 in /usr/bin/clickhouse
    [ 614 ] {} <Fatal> Application: Child process was terminated by signal 6.

</details>

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/37542/8a224239c1d922158b4dc9f5d6609dca836dfd06/stress_test__undefined__actions_.html

Follow-up for: #36979

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-01 21:50:30 +03:00
Alexander Tokmakov
06f80770b8 fix stuck REPALCE_RANGE 2022-06-01 20:11:53 +02:00
Yakov Olkhovskiy
e23cec01d5
Merge pull request #37581 from ClickHouse/http-named-collection 
Support for HTTP source for Data Dictionaries in Named Collections
 2022-06-01 11:55:04 -04:00
Nikolai Kochetov
6e924cdc77 Fix some more tests. 2022-06-01 15:21:47 +00:00
Sema Checherinda
2626a49616 FR: Expose what triggered the merge in system.part_log #26255 2022-06-01 16:58:06 +02:00
Alexander Tokmakov
3d346c766a better code 2022-06-01 16:49:26 +02:00
Kseniia Sumarokova
7afcfcbaaf
Merge pull request #37691 from kssenii/fix-rabbitmq-restart-with-no-settings 
Fix rabbitmq restart with empty settings
 2022-06-01 14:59:34 +02:00
Alexander Tokmakov
75f49a48e1 Merge branch 'master' into fix_trash 2022-06-01 14:20:46 +02:00
Anton Popov
011131198c Merge remote-tracking branch 'upstream/master' into fix-mutations-again 2022-06-01 12:04:18 +00:00
Anton Popov
cae2478b3f Revert "Merge pull request #37355 from ClickHouse/revert-37266-fix-mutations-with-object" 
This reverts commit a53cfa9fca, reversing
changes made to 9acb42fcdb.
 2022-06-01 10:57:20 +00:00
Han Fei
ea693dd0c2 add config and change test logic 2022-06-01 14:57:07 +08:00
Anton Popov
6cf9405f09 fix optimize_monotonous_functions_in_order_by in distributed queries 2022-06-01 00:50:28 +00:00
Alexander Gololobov
26609a1875 Style fixes 2022-05-31 21:41:10 +02:00
Nikolai Kochetov
86fbb74703 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-31 18:07:47 +00:00
Han Fei
5693e6212d add config and fix style check 2022-05-31 23:18:05 +08:00
Nikolai Kochetov
147a819221 Refactor a little bit more. 2022-05-31 14:43:38 +00:00
kssenii
c2087b3145 Fix 2022-05-31 14:38:11 +02:00
mergify[bot]
d85c3ec69e
Merge branch 'master' into turn_on_s3_tests 2022-05-31 11:58:16 +00:00
mergify[bot]
1e08046c47
Merge branch 'master' into cleanup_unused 2022-05-31 10:28:19 +00:00
taiyang-li
047387bf1c fix 2 bugs: 1. select count(1) from hive_table; 2. select _file, _path from hive_table 2022-05-31 17:39:02 +08:00
Kseniia Sumarokova
73ed9c3977
Merge pull request #37619 from Vxider/wv-fix-table-identifier 
Fix bugs in WindowView when using table identifier
 2022-05-31 11:07:11 +02:00
xlwh
ba4cdd43bd Cleanup unused file 2022-05-31 14:37:30 +08:00
yaqi-zhao
a2857491c4 add avx512 support for mergetreereader 2022-05-30 20:53:00 -04:00
mergify[bot]
55913cf8e1
Merge branch 'master' into turn_on_s3_tests 2022-05-30 20:52:40 +00:00
Nikolai Kochetov
77b07dd0a8
Merge pull request #37163 from ClickHouse/grouping-function 
Add GROUPING function
 2022-05-30 20:45:04 +02:00
HeenaBansal2009
b7eb6bbd38 Fixed clang-tidy-CheckTriviallyCopyableMove-errors 2022-05-30 11:09:03 -07:00
Alexander Gololobov
e2dd6f6249 Removed prewhere_info.alias_actions 2022-05-30 19:58:23 +02:00
Han Fei
e15cdec39c address comments 2022-05-31 01:46:31 +08:00
mergify[bot]
d4e722bbfa
Merge branch 'master' into http-named-collection 2022-05-30 16:40:18 +00:00
Han Fei
af86900c52 Merge branch 'hanfei/zk-write' of github.com:hanfei1991/ClickHouse into hanfei/zk-write 2022-05-31 00:17:38 +08:00
Han Fei
42fca8d5f0 address comments 2022-05-31 00:17:32 +08:00
Han Fei
a464b10afe
Update src/Storages/System/StorageSystemZooKeeper.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2022-05-30 23:52:37 +08:00
Han Fei
194445646a
Update src/Storages/System/StorageSystemZooKeeper.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2022-05-30 23:47:43 +08:00
Nikolai Kochetov
c71256ea38 Remove some commented code. 2022-05-30 13:18:20 +00:00
Kruglov Pavel
0615866aea
Merge pull request #37450 from Avogar/check-format-on-storage-creation 
Check format name on storage creation
 2022-05-30 14:23:20 +02:00
Nikolai Kochetov
5b4658aa5e Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-30 09:47:35 +00:00
taiyang-li
dbb8a09825 merge master and solve conflict 2022-05-30 10:47:04 +08:00
taiyang-li
51a893c8be add some metrics 2022-05-30 10:05:20 +08:00
alesapin
c32b6076fb Remove stranges from code 2022-05-30 00:12:33 +02:00
alesapin
0e8ab36913 Merge branch 'master' into turn_on_s3_tests 2022-05-29 14:37:10 +02:00
alesapin
d2cdbf3956 Fix refactoring issue 2022-05-29 14:09:49 +02:00
alesapin
9dc81e5cc8
Merge pull request #37598 from ClickHouse/revert-37545-revert-37424-fix_fetching_part_deadlock 
Revert "Revert "(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part""
 2022-05-29 13:54:34 +02:00
Alexander Tokmakov
562eec591e
Revert "Implemented changing comment to a ReplicatedMergeTree table" 2022-05-29 09:28:47 +03:00
Alexey Milovidov
97606c324c
Merge pull request #37574 from azat/mt-tiny-refactor 
Remove unused MergeTreeDataMergerMutator::chooseMergeAlgorithm()
 2022-05-29 07:59:57 +03:00
Alexey Milovidov
73e2e63414
Merge pull request #37612 from ClickHouse/clang-tidy-14 
Fix clang-tidy-14, part 1
 2022-05-29 03:16:32 +03:00
Alexander Tokmakov
4e52f45695 Merge branch 'master' into fix_trash 2022-05-28 19:43:19 +02:00
Anton Popov
1d9b3be7da
Merge pull request #37536 from CurtizJ/profile-events-for-part-types 
Add profile events for introspection of part types
 2022-05-28 14:25:21 +02:00
Han Fei
340a264a62 fix style 2022-05-28 18:26:14 +08:00
Han Fei
0a0d77bdef fix build 2022-05-28 17:57:59 +08:00
Han Fei
0f71231574 try fix flaky tests and refine code style 2022-05-28 17:25:33 +08:00
Vxider
b24346328d fix parser when using table identifer 2022-05-28 08:22:34 +00:00
Alexey Milovidov
c50791dd3b Fix clang-tidy-14, part 1 2022-05-27 22:52:14 +02:00
Nikolai Kochetov
b80b1940ce Fix some tests. 2022-05-27 20:47:35 +00:00
Yakov Olkhovskiy
41ef0044f0 endpoint is added 2022-05-27 13:43:34 -04:00
mergify[bot]
f5ee337bab
Merge branch 'master' into revert-37545-revert-37424-fix_fetching_part_deadlock 2022-05-27 16:52:00 +00:00
mergify[bot]
923ad2e905
Merge branch 'master' into turn_on_s3_tests 2022-05-27 16:31:43 +00:00
alesapin
f63fa9bcc6
Merge pull request #37416 from Enmk/fix_ReplicatedMergeTree_comments 
Implemented changing comment to a ReplicatedMergeTree table
 2022-05-27 18:29:34 +02:00
alesapin
5a296aec01 Fix build 2022-05-27 16:34:16 +02:00
alesapin
be1c3c132b Fix some trash 2022-05-27 16:08:49 +02:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
alesapin
c79600c4c8 Fix build 2022-05-27 13:44:29 +02:00
taiyang-li
73d2c889c6 fix log level 2022-05-27 19:23:58 +08:00
alesapin
841858ec30
Revert "Revert "(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part"" 2022-05-27 13:13:36 +02:00
Han Fei
2ea027ffcb Support insert into system.zookeeper 2022-05-27 18:53:12 +08:00
taiyang-li
ea450b86cb add some prefetch metric codes 2022-05-27 18:06:40 +08:00
Kseniia Sumarokova
2943d44bf1
Merge pull request #37554 from msaf1980/cleanup_hdfs 
Cleanup StorageHDFS (unused variables prevent build with clang 12)
 2022-05-27 11:57:18 +02:00
Yakov Olkhovskiy
25884c68f1 http named collection source implemented for dictionary 2022-05-26 20:46:26 -04:00
Azat Khuzhin
c6c60364ae Remove unused MergeTreeDataMergerMutator::chooseMergeAlgorithm() 
In favor of
MergeTask::ExecuteAndFinalizeHorizontalPart::chooseMergeAlgorithm()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 21:20:50 +03:00
Alexander Tokmakov
eb71dd4c78
Merge pull request #37547 from ClickHouse/followup_37398 
Follow-up to #37398
 2022-05-26 20:29:41 +03:00
Azat Khuzhin
73a99d4eee Improve error message for skipped/expired columns 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
Azat Khuzhin
f0cac5417a More optional skipping of fully expired columns (by TTL) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
Azat Khuzhin
46a94f395d Move the check into IMergedBlockOutputStream::removeEmptyColumnsFromPart() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
Azat Khuzhin
4288d09a85 Do not write expired columns by TTL after merge w/o TTL 
Usually second merge do not perform TTL, since everything is up to date,
however in this case TTLTransform is not used, and hence expired_columns
will not be filled for new part, and so those columns will be written
with default values.

Avoid this, by manually filling expired_columns.

Here is a simpler reproducer:

Simple reproducer:

    ```sql
    create table ttl_02262 (date Date, key Int, value String TTL date + interval 1 month) engine=MergeTree order by key settings min_bytes_for_wide_part=0, min_rows_for_wide_part=0;
    insert into ttl_02262 values ('2010-01-01', 2010, 'foo');
    ```

    ```sh
    # ls -l .server/data/default/ttl_02262/all_*
    .server/data/default/ttl_02262/all_1_1_0:
    total 48
    -rw-r----- 1 root root 335 May 26 14:19 checksums.txt
    -rw-r----- 1 root root  76 May 26 14:19 columns.txt
    -rw-r----- 1 root root   1 May 26 14:19 count.txt
    -rw-r----- 1 root root  28 May 26 14:19 date.bin
    -rw-r----- 1 root root  48 May 26 14:19 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:19 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:19 key.bin
    -rw-r----- 1 root root  48 May 26 14:19 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:19 primary.idx
    -rw-r----- 1 root root  99 May 26 14:19 ttl.txt
    -rw-r----- 1 root root  30 May 26 14:19 value.bin
    -rw-r----- 1 root root  48 May 26 14:19 value.mrk2
    ```

    ```sql
    optimize table ttl_02262 final;
    ```

    ```sh
    .server/data/default/ttl_02262/all_1_1_1:
    total 40
    -rw-r----- 1 root root 279 May 26 14:19 checksums.txt
    -rw-r----- 1 root root  61 May 26 14:19 columns.txt
    -rw-r----- 1 root root   1 May 26 14:19 count.txt
    -rw-r----- 1 root root  28 May 26 14:19 date.bin
    -rw-r----- 1 root root  48 May 26 14:19 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:19 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:19 key.bin
    -rw-r----- 1 root root  48 May 26 14:19 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:19 primary.idx
    -rw-r----- 1 root root  81 May 26 14:19 ttl.txt
    ```

    ```sql
    optimize table ttl_02262 final;
    ```

    ```sh
    .server/data/default/ttl_02262/all_1_1_2:
    total 48
    -rw-r----- 1 root root 349 May 26 14:20 checksums.txt
    -rw-r----- 1 root root  76 May 26 14:20 columns.txt
    -rw-r----- 1 root root   1 May 26 14:20 count.txt
    -rw-r----- 1 root root  28 May 26 14:20 date.bin
    -rw-r----- 1 root root  48 May 26 14:20 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:20 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:20 key.bin
    -rw-r----- 1 root root  48 May 26 14:20 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:20 primary.idx
    -rw-r----- 1 root root  81 May 26 14:20 ttl.txt
    -rw-r----- 1 root root  27 May 26 14:20 value.bin
    -rw-r----- 1 root root  48 May 26 14:20 value.mrk2
    ```

And now we have `value.*` for all_1_1_2, this should not happen.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
Azat Khuzhin
8328d7068b Fix updating of MergeTreeDataPartTTLInfo::finished 
Previously you cannot distinguish non-initialized finished with
initialized to false, so update() cannot do the correct thing.

Rename the field to avoid hidden usage.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
Azat Khuzhin
0de1a64436 Log empty parts in IMergedBlockOutputStream::removeEmptyColumnsFromPart() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:09 +03:00
Nikolai Kochetov
ec4e8d71b2 Fixing build 2022-05-26 15:33:21 +00:00
Alexander Tokmakov
e8f33fb0d9 fix flaky tests 2022-05-26 14:17:05 +02:00
Azat Khuzhin
dc9ca3d70c
Fix LOGICAL_ERROR in getMaxSourcePartsSizeForMerge during merges (#37413) 2022-05-26 14:14:58 +02:00
Nikolai Kochetov
84f97b53de Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-26 11:07:45 +00:00
Maksim Kita
3a92e61827
Merge pull request #37148 from kitaisreal/dictionary-get-descendants-performance-improvement 
Dictionary getDescendants performance improvement
 2022-05-26 12:29:17 +02:00
Vasily Nemkov
abe6b5d013
Reverted unnecessary modification 2022-05-26 10:09:27 +03:00
taiyang-li
561c87222d add prefetch for hive text 2022-05-26 11:04:35 +08:00
Anton Popov
f488efd27e fix tests 2022-05-26 00:03:31 +00:00
Dmitry Novik
16c6b60703 Introduce AggregationKeysInfo 2022-05-25 23:22:29 +00:00
alesapin
8f1aac0ce4 Fix merge with master 2022-05-26 00:44:45 +02:00
alesapin
1db9cf480b Merge remote-tracking branch 'origin/master' into i_object_storage 2022-05-25 22:50:22 +02:00
Alexander Tokmakov
47820c216d
Revert "(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part" 2022-05-25 23:10:33 +03:00
alesapin
c7b16065e1 Merge with master 2022-05-25 21:47:05 +02:00
Nikolai Kochetov
6d4a26afac Update ReadProgressCallback. 2022-05-25 19:45:48 +00:00
alesapin
6f5c86e55e Merge branch 'master' into i_object_storage 2022-05-25 20:49:01 +02:00
Alexander Tokmakov
779e6ea0b9 make it better, fix on cluster queries 2022-05-25 20:17:49 +02:00
Nikolai Kochetov
54d7e4139f Fix build. 2022-05-25 18:16:48 +00:00
alesapin
51868a9a4f
Merge pull request #37424 from metahys/fix_fetching_part_deadlock 
(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part
 2022-05-25 20:15:41 +02:00
alesapin
0a3597da72
Merge pull request #34915 from ianton-ru/MDB-16962 
Fix collision of S3 operation log revision
 2022-05-25 18:15:31 +02:00
Anton Popov
16e839ac71 add profile events for introspection of part types 2022-05-25 14:54:49 +00:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
msaf1980
fda6ddeffa cleanup StorageHDFS (unused variables) 2022-05-25 19:23:05 +05:00
mergify[bot]
73662b4436
Merge branch 'master' into fix_fetching_part_deadlock 2022-05-25 14:22:35 +00:00
taiyang-li
a7a816dcb6 fix build error 2022-05-25 19:55:11 +08:00
taiyang-li
1d9f65a7d4 Merge branch 'master' into async_hdfs_read_buffer 2022-05-25 17:10:22 +08:00
Kseniia Sumarokova
b50d4549c9
Merge pull request #37356 from amosbird/partition-prune-for-s3 
"Partition pruning" for s3
 2022-05-25 11:03:07 +02:00
avogar
f782fa31c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into check-format-on-storage-creation 2022-05-25 08:42:54 +00:00
Nikolai Kochetov
3d84aae0ab Better. 2022-05-24 20:06:08 +00:00
Nikolai Kochetov
333fd09dbf Fixing build. 2022-05-24 19:29:00 +00:00
Vasily Nemkov
59b4d4a643 ALTER COMMENT is now local-only operation and immediately observable 2022-05-24 21:08:30 +03:00
alesapin
9a19309e69 Slightly better fix 2022-05-24 14:46:29 +02:00
Alexander Tokmakov
229d35408b
Merge pull request #37398 from ClickHouse/fixes_for_transactions 
Fixes for transactions
 2022-05-24 15:28:01 +03:00
alesapin
3ca7a8831b Revert "fix deadlock during fetching part" 
This reverts commit 6ae8a26fae.
 2022-05-24 14:26:06 +02:00
alesapin
a5ba6bca95 Merge branch 'master' into metahys-fix_fetching_part_deadlock 2022-05-24 14:07:47 +02:00
Kruglov Pavel
6c9a524f6b
Merge pull request #37192 from Avogar/formats-with-names 
Improve performance and memory usage for select of subset of columns for some formats
 2022-05-24 13:28:14 +02:00
Amos Bird
093d315756
partition pruning for s3 2022-05-24 18:57:55 +08:00
mergify[bot]
51ff49a0ee
Merge branch 'master' into i_object_storage 2022-05-23 20:29:49 +00:00
Nikolai Kochetov
fd97a9d885 Move some resources 2022-05-23 19:47:32 +00:00
Alexander Tokmakov
6bc68c0cbc Merge branch 'master' into fixes_for_transactions 2022-05-23 18:49:21 +02:00
Nikolai Kochetov
9756b759c6 Move some resources 2022-05-23 13:46:57 +00:00
Kruglov Pavel
a0aa841909
Fix failed thread stateless tests 2022-05-23 15:15:45 +02:00
avogar
37b66c8a9e Check format name on storage creation 2022-05-23 12:48:48 +00:00
metahys
6ae8a26fae fix deadlock during fetching part 2022-05-23 20:08:04 +08:00
taiyang-li
29e2157469 change as request 2022-05-23 18:42:54 +08:00
Maksim Kita
585b86446e Added hierarchical_index_bytes_allocated column in system.dictionaries 2022-05-23 12:42:00 +02:00
taiyang-li
14f84f02d5 Merge branch 'master' into async_hdfs_read_buffer 2022-05-23 18:36:21 +08:00
Sergei Trifonov
b6bf283f4d
Merge pull request #37243 from Vxider/mv-fix-dependency 
Fix adding dependency when MaterializedView is dropped
 2022-05-23 12:33:54 +02:00
Kruglov Pavel
f539fb835d
Merge branch 'master' into formats-with-names 2022-05-23 12:14:20 +02:00
Kruglov Pavel
754e675ec3
Merge pull request #37253 from Avogar/fix-defaults 
Fix inserting defaults for missing values in columnar formats
 2022-05-23 12:10:14 +02:00
Kseniia Sumarokova
ca3feddf73
Merge pull request #37426 from Vxider/wv-simplify 
Remove unused code in WindowView
 2022-05-23 11:43:51 +02:00
mergify[bot]
747aa5575c
Merge branch 'master' into remove-useless-code-2 2022-05-22 17:41:57 +00:00
Vxider
fcd317a642 update windowview create syntax 2022-05-22 10:18:34 +00:00
alesapin
c8d92b87c8 Merge branch 'master' into i_object_storage 2022-05-22 12:16:10 +02:00
Vxider
5f84f06d6d simplify windowview 2022-05-22 10:13:16 +00:00
Robert Schulze
19d53c14fa
Merge pull request #37382 from ClickHouse/wc++98-compat-extra-semi 
Enable -Wc++98-compat-extra-semi
 2022-05-22 09:40:45 +02:00
Alexey Milovidov
4bfbb0b7ba
Merge pull request #37388 from CurtizJ/explicit-bool-operator 
Mark all `operator bool()` as explicit
 2022-05-22 00:18:19 +03:00
Alexey Milovidov
790f442362
Merge pull request #37407 from ClickHouse/certificates-mem-leak 
Fix possible memory leaks in system.certificates implementation
 2022-05-22 00:15:30 +03:00
Vasily Nemkov
c12f826d22 Implemented changing comment to a ReplicatedMergeTree table 2022-05-21 20:08:30 +03:00
Yakov Olkhovskiy
4c13b52b6a scope guard resources 2022-05-20 19:59:17 -04:00
Alexander Tokmakov
44f2d4529a better waiting, add fault injection 2022-05-20 22:10:01 +02:00
Nikolai Kochetov
56feef01e7 Move some resources 2022-05-20 19:49:31 +00:00
avogar
3bad21cea7 Try fix tests 2022-05-20 17:48:24 +00:00
Alexander Tokmakov
1027798517 handle connection loss on commit 2022-05-20 17:35:29 +02:00
Anton Popov
cb0e6c2718 mark all operators bool() as explicit 2022-05-20 15:29:54 +00:00
avogar
a4cf07708c Fix comments 2022-05-20 14:57:27 +00:00
avogar
566d1b15fd Merge branch 'master' of github.com:ClickHouse/ClickHouse into formats-with-names 2022-05-20 13:54:52 +00:00
avogar
78069765bb Try to remove code duplication 2022-05-20 13:20:52 +00:00