thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
91,550 Commits 9,326 Branches 1,783 Tags 2.1 GiB
106f92dcdb
Commit Graph

8453 Commits

Author SHA1 Message Date
Igor Nikonov
8c40f05d4c
Merge pull request #37674 from xlwh/cleanup_unused 
Cleanup unused file
 2022-06-02 10:18:29 +02:00
mergify[bot]
88bed9de20
Merge branch 'master' into fix_replace_range_again 2022-06-02 07:05:44 +00:00
Alexander Gololobov
a0cf902d49
Merge pull request #37588 from yaqi-zhao/avx512_tail_zero 
add avx512 support for mergetree reader
 2022-06-02 08:51:52 +03:00
Han Fei
1424c420fa try to fill in right metadata columns 2022-06-02 13:41:37 +08:00
Vxider
8221fcd5f1 update var name 2022-06-02 04:38:45 +00:00
Vxider
df4db70bb8 fix empty target table id 2022-06-02 04:28:18 +00:00
lthaooo
6632616733
Fix TTL merge scheduling bug (#36387) 2022-06-01 21:09:53 +02:00
Azat Khuzhin
545a56ce45 Fix sinks with onException() handler 
It is possible to call onException() even after onFinish(), in case of
onFinish() throws, and in this case onException() should be no-op for
such sinks.

Also there can be caveats with PartitionedSync.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-01 21:50:30 +03:00
Azat Khuzhin
02af58f41d Fix possible "Cannot write to finalized buffer" 
It is still possible to get this error since onException does not
finalize format correctly.

Here is an example of such error, that was found by CI [1]:

<details>

    [ 2686 ] {fa01bf02-73f6-4f7f-b14f-e725de6d7f9b} <Fatal> : Logical error: 'Cannot write to finalized buffer'.
    [ 34577 ] {} <Fatal> BaseDaemon: ########################################
    [ 34577 ] {} <Fatal> BaseDaemon: (version 22.6.1.1, build id: AB8040A6769E01A0) (from thread 2686) (query_id: fa01bf02-73f6-4f7f-b14f-e725de6d7f9b) (query: insert into test_02302 select number from numbers(10) settings s3_truncate_on_insert=1;) Received signal Aborted (6)
    [ 34577 ] {} <Fatal> BaseDaemon:
    [ 34577 ] {} <Fatal> BaseDaemon: Stack trace: 0x7fcbaa5a703b 0x7fcbaa586859 0xfad9bab 0xfad9e05 0xfaf6a3b 0x24a48c7f 0x258fb9b9 0x258f2004 0x258b88f4 0x258b863b 0x2581773d 0x258177ce 0x24bb5e98 0xfad01d6 0xfad0105 0x2419b11d 0xfad01d6 0xfad0105 0x2215afbb 0x2215aa48 0xfad01d6 0xfad0105 0xfcc265d 0x225cc546 0x249a1c40 0x249bc1b6 0x2685902c 0x26859505 0x269d7767 0x269d504c 0x7fcbaa75e609 0x7fcbaa683163
    [ 34577 ] {} <Fatal> BaseDaemon: 3. raise @ 0x7fcbaa5a703b in ?
    [ 34577 ] {} <Fatal> BaseDaemon: 4. abort @ 0x7fcbaa586859 in ?
    [ 34577 ] {} <Fatal> BaseDaemon: 5. ./build_docker/../src/Common/Exception.cpp:47: DB::abortOnFailedAssertion(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0xfad9bab in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 6. ./build_docker/../src/Common/Exception.cpp:70: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xfad9e05 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 7. ./build_docker/../src/IO/WriteBuffer.h:0: DB::WriteBuffer::write(char const*, unsigned long) @ 0xfaf6a3b in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 8. ./build_docker/../src/Processors/Formats/Impl/ArrowBufferedStreams.cpp:47: DB::ArrowBufferedOutputStream::Write(void const*, long) @ 0x24a48c7f in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 9. long parquet::ThriftSerializer::Serialize<parquet::format::FileMetaData>(parquet::format::FileMetaData const*, arrow::io::OutputStream*, std::__1::shared_ptr<parquet::Encryptor> const&) @ 0x258fb9b9 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 10. parquet::FileMetaData::FileMetaDataImpl::WriteTo(arrow::io::OutputStream*, std::__1::shared_ptr<parquet::Encryptor> const&) const @ 0x258f2004 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 11. parquet::WriteFileMetaData(parquet::FileMetaData const&, arrow::io::OutputStream*) @ 0x258b88f4 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 12. parquet::ParquetFileWriter::~ParquetFileWriter() @ 0x258b863b in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 13. parquet::arrow::FileWriterImpl::~FileWriterImpl() @ 0x2581773d in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 14. parquet::arrow::FileWriterImpl::~FileWriterImpl() @ 0x258177ce in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 15. ./build_docker/../src/Processors/Formats/Impl/ParquetBlockOutputFormat.h:27: DB::ParquetBlockOutputFormat::~ParquetBlockOutputFormat() @ 0x24bb5e98 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 16. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:173: std::__1::__shared_count::__release_shared() @ 0xfad01d6 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 17. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:216: std::__1::__shared_weak_count::__release_shared() @ 0xfad0105 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 18.1. inlined from ./build_docker/../contrib/libcxx/include/__memory/unique_ptr.h:312: std::__1::unique_ptr<DB::WriteBuffer, std::__1::default_delete<DB::WriteBuffer> >::reset(DB::WriteBuffer*)
    [ 34577 ] {} <Fatal> BaseDaemon: 18.2. inlined from ../contrib/libcxx/include/__memory/unique_ptr.h:269: ~unique_ptr
    [ 34577 ] {} <Fatal> BaseDaemon: 18. ../src/Storages/StorageS3.cpp:566: DB::StorageS3Sink::~StorageS3Sink() @ 0x2419b11d in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 19. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:173: std::__1::__shared_count::__release_shared() @ 0xfad01d6 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 20. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:216: std::__1::__shared_weak_count::__release_shared() @ 0xfad0105 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 21. ./build_docker/../contrib/abseil-cpp/absl/container/internal/raw_hash_set.h:1662: absl::lts_20211102::container_internal::raw_hash_set<absl::lts_20211102::container_internal::FlatHashMapPolicy<StringRef, std::__1::shared_ptr<DB::SinkToStorage> >, absl::lts_20211102::hash_internal::Hash<StringRef>, std::__1::equal_to<StringRef>, std::__1::allocator<std::__1::pair<StringRef const, std::__1::shared_ptr<DB::SinkToStorage> > > >::destroy_slots() @ 0x2215afbb in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 22.1. inlined from ./build_docker/../contrib/libcxx/include/string:1445: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long() const
    [ 34577 ] {} <Fatal> BaseDaemon: 22.2. inlined from ../contrib/libcxx/include/string:2231: ~basic_string
    [ 34577 ] {} <Fatal> BaseDaemon: 22. ../src/Storages/PartitionedSink.h:14: DB::PartitionedSink::~PartitionedSink() @ 0x2215aa48 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 23. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:173: std::__1::__shared_count::__release_shared() @ 0xfad01d6 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 24. ./build_docker/../contrib/libcxx/include/__memory/shared_ptr.h:216: std::__1::__shared_weak_count::__release_shared() @ 0xfad0105 in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 25. ./build_docker/../contrib/libcxx/include/vector:802: std::__1::vector<std::__1::shared_ptr<DB::IProcessor>, std::__1::allocator<std::__1::shared_ptr<DB::IProcessor> > >::__base_destruct_at_end(std::__1::shared_ptr<DB::IProcessor>*) @ 0xfcc265d in /usr/bin/clickhouse
    [ 34577 ] {} <Fatal> BaseDaemon: 26.1. inlined from ./build_docker/../contrib/libcxx/include/vector:402: ~vector
    [ 34577 ] {} <Fatal> BaseDaemon: 26.2. inlined from ../src/QueryPipeline/QueryPipeline.cpp:29: ~QueryPipeline
    [ 34577 ] {} <Fatal> BaseDaemon: 26. ../src/QueryPipeline/QueryPipeline.cpp:535: DB::QueryPipeline::reset() @ 0x225cc546 in /usr/bin/clickhouse
    [ 614 ] {} <Fatal> Application: Child process was terminated by signal 6.

</details>

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/37542/8a224239c1d922158b4dc9f5d6609dca836dfd06/stress_test__undefined__actions_.html

Follow-up for: #36979

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-01 21:50:30 +03:00
Alexander Tokmakov
06f80770b8 fix stuck REPALCE_RANGE 2022-06-01 20:11:53 +02:00
Yakov Olkhovskiy
e23cec01d5
Merge pull request #37581 from ClickHouse/http-named-collection 
Support for HTTP source for Data Dictionaries in Named Collections
 2022-06-01 11:55:04 -04:00
Nikolai Kochetov
6e924cdc77 Fix some more tests. 2022-06-01 15:21:47 +00:00
Sema Checherinda
2626a49616 FR: Expose what triggered the merge in system.part_log #26255 2022-06-01 16:58:06 +02:00
Alexander Tokmakov
3d346c766a better code 2022-06-01 16:49:26 +02:00
Kseniia Sumarokova
7afcfcbaaf
Merge pull request #37691 from kssenii/fix-rabbitmq-restart-with-no-settings 
Fix rabbitmq restart with empty settings
 2022-06-01 14:59:34 +02:00
Alexander Tokmakov
75f49a48e1 Merge branch 'master' into fix_trash 2022-06-01 14:20:46 +02:00
Anton Popov
011131198c Merge remote-tracking branch 'upstream/master' into fix-mutations-again 2022-06-01 12:04:18 +00:00
Anton Popov
cae2478b3f Revert "Merge pull request #37355 from ClickHouse/revert-37266-fix-mutations-with-object" 
This reverts commit a53cfa9fca, reversing
changes made to 9acb42fcdb.
 2022-06-01 10:57:20 +00:00
Han Fei
ea693dd0c2 add config and change test logic 2022-06-01 14:57:07 +08:00
Anton Popov
6cf9405f09 fix optimize_monotonous_functions_in_order_by in distributed queries 2022-06-01 00:50:28 +00:00
Alexander Gololobov
26609a1875 Style fixes 2022-05-31 21:41:10 +02:00
Nikolai Kochetov
86fbb74703 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-31 18:07:47 +00:00
Han Fei
5693e6212d add config and fix style check 2022-05-31 23:18:05 +08:00
Nikolai Kochetov
147a819221 Refactor a little bit more. 2022-05-31 14:43:38 +00:00
kssenii
c2087b3145 Fix 2022-05-31 14:38:11 +02:00
mergify[bot]
d85c3ec69e
Merge branch 'master' into turn_on_s3_tests 2022-05-31 11:58:16 +00:00
mergify[bot]
1e08046c47
Merge branch 'master' into cleanup_unused 2022-05-31 10:28:19 +00:00
taiyang-li
047387bf1c fix 2 bugs: 1. select count(1) from hive_table; 2. select _file, _path from hive_table 2022-05-31 17:39:02 +08:00
Kseniia Sumarokova
73ed9c3977
Merge pull request #37619 from Vxider/wv-fix-table-identifier 
Fix bugs in WindowView when using table identifier
 2022-05-31 11:07:11 +02:00
xlwh
ba4cdd43bd Cleanup unused file 2022-05-31 14:37:30 +08:00
yaqi-zhao
a2857491c4 add avx512 support for mergetreereader 2022-05-30 20:53:00 -04:00
mergify[bot]
55913cf8e1
Merge branch 'master' into turn_on_s3_tests 2022-05-30 20:52:40 +00:00
Nikolai Kochetov
77b07dd0a8
Merge pull request #37163 from ClickHouse/grouping-function 
Add GROUPING function
 2022-05-30 20:45:04 +02:00
HeenaBansal2009
b7eb6bbd38 Fixed clang-tidy-CheckTriviallyCopyableMove-errors 2022-05-30 11:09:03 -07:00
Alexander Gololobov
e2dd6f6249 Removed prewhere_info.alias_actions 2022-05-30 19:58:23 +02:00
Han Fei
e15cdec39c address comments 2022-05-31 01:46:31 +08:00
mergify[bot]
d4e722bbfa
Merge branch 'master' into http-named-collection 2022-05-30 16:40:18 +00:00
Han Fei
af86900c52 Merge branch 'hanfei/zk-write' of github.com:hanfei1991/ClickHouse into hanfei/zk-write 2022-05-31 00:17:38 +08:00
Han Fei
42fca8d5f0 address comments 2022-05-31 00:17:32 +08:00
Han Fei
a464b10afe
Update src/Storages/System/StorageSystemZooKeeper.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2022-05-30 23:52:37 +08:00
Han Fei
194445646a
Update src/Storages/System/StorageSystemZooKeeper.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2022-05-30 23:47:43 +08:00
Nikolai Kochetov
c71256ea38 Remove some commented code. 2022-05-30 13:18:20 +00:00
Kruglov Pavel
0615866aea
Merge pull request #37450 from Avogar/check-format-on-storage-creation 
Check format name on storage creation
 2022-05-30 14:23:20 +02:00
Nikolai Kochetov
5b4658aa5e Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-30 09:47:35 +00:00
taiyang-li
dbb8a09825 merge master and solve conflict 2022-05-30 10:47:04 +08:00
taiyang-li
51a893c8be add some metrics 2022-05-30 10:05:20 +08:00
alesapin
c32b6076fb Remove stranges from code 2022-05-30 00:12:33 +02:00
alesapin
0e8ab36913 Merge branch 'master' into turn_on_s3_tests 2022-05-29 14:37:10 +02:00
alesapin
d2cdbf3956 Fix refactoring issue 2022-05-29 14:09:49 +02:00
alesapin
9dc81e5cc8
Merge pull request #37598 from ClickHouse/revert-37545-revert-37424-fix_fetching_part_deadlock 
Revert "Revert "(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part""
 2022-05-29 13:54:34 +02:00
Alexander Tokmakov
562eec591e
Revert "Implemented changing comment to a ReplicatedMergeTree table" 2022-05-29 09:28:47 +03:00
Alexey Milovidov
97606c324c
Merge pull request #37574 from azat/mt-tiny-refactor 
Remove unused MergeTreeDataMergerMutator::chooseMergeAlgorithm()
 2022-05-29 07:59:57 +03:00
Alexey Milovidov
73e2e63414
Merge pull request #37612 from ClickHouse/clang-tidy-14 
Fix clang-tidy-14, part 1
 2022-05-29 03:16:32 +03:00
Alexander Tokmakov
4e52f45695 Merge branch 'master' into fix_trash 2022-05-28 19:43:19 +02:00
Anton Popov
1d9b3be7da
Merge pull request #37536 from CurtizJ/profile-events-for-part-types 
Add profile events for introspection of part types
 2022-05-28 14:25:21 +02:00
Han Fei
340a264a62 fix style 2022-05-28 18:26:14 +08:00
Han Fei
0a0d77bdef fix build 2022-05-28 17:57:59 +08:00
Han Fei
0f71231574 try fix flaky tests and refine code style 2022-05-28 17:25:33 +08:00
Vxider
b24346328d fix parser when using table identifer 2022-05-28 08:22:34 +00:00
Alexey Milovidov
c50791dd3b Fix clang-tidy-14, part 1 2022-05-27 22:52:14 +02:00
Nikolai Kochetov
b80b1940ce Fix some tests. 2022-05-27 20:47:35 +00:00
Yakov Olkhovskiy
41ef0044f0 endpoint is added 2022-05-27 13:43:34 -04:00
mergify[bot]
f5ee337bab
Merge branch 'master' into revert-37545-revert-37424-fix_fetching_part_deadlock 2022-05-27 16:52:00 +00:00
mergify[bot]
923ad2e905
Merge branch 'master' into turn_on_s3_tests 2022-05-27 16:31:43 +00:00
alesapin
f63fa9bcc6
Merge pull request #37416 from Enmk/fix_ReplicatedMergeTree_comments 
Implemented changing comment to a ReplicatedMergeTree table
 2022-05-27 18:29:34 +02:00
alesapin
5a296aec01 Fix build 2022-05-27 16:34:16 +02:00
alesapin
be1c3c132b Fix some trash 2022-05-27 16:08:49 +02:00
alesapin
6d6779f17a
Merge pull request #37139 from ClickHouse/i_object_storage 
Separate object storage operations from disks
 2022-05-27 13:59:50 +02:00
alesapin
c79600c4c8 Fix build 2022-05-27 13:44:29 +02:00
taiyang-li
73d2c889c6 fix log level 2022-05-27 19:23:58 +08:00
alesapin
841858ec30
Revert "Revert "(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part"" 2022-05-27 13:13:36 +02:00
Han Fei
2ea027ffcb Support insert into system.zookeeper 2022-05-27 18:53:12 +08:00
taiyang-li
ea450b86cb add some prefetch metric codes 2022-05-27 18:06:40 +08:00
Kseniia Sumarokova
2943d44bf1
Merge pull request #37554 from msaf1980/cleanup_hdfs 
Cleanup StorageHDFS (unused variables prevent build with clang 12)
 2022-05-27 11:57:18 +02:00
Yakov Olkhovskiy
25884c68f1 http named collection source implemented for dictionary 2022-05-26 20:46:26 -04:00
Azat Khuzhin
c6c60364ae Remove unused MergeTreeDataMergerMutator::chooseMergeAlgorithm() 
In favor of
MergeTask::ExecuteAndFinalizeHorizontalPart::chooseMergeAlgorithm()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 21:20:50 +03:00
Alexander Tokmakov
eb71dd4c78
Merge pull request #37547 from ClickHouse/followup_37398 
Follow-up to #37398
 2022-05-26 20:29:41 +03:00
Azat Khuzhin
73a99d4eee Improve error message for skipped/expired columns 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
Azat Khuzhin
f0cac5417a More optional skipping of fully expired columns (by TTL) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
Azat Khuzhin
46a94f395d Move the check into IMergedBlockOutputStream::removeEmptyColumnsFromPart() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
Azat Khuzhin
4288d09a85 Do not write expired columns by TTL after merge w/o TTL 
Usually second merge do not perform TTL, since everything is up to date,
however in this case TTLTransform is not used, and hence expired_columns
will not be filled for new part, and so those columns will be written
with default values.

Avoid this, by manually filling expired_columns.

Here is a simpler reproducer:

Simple reproducer:

    ```sql
    create table ttl_02262 (date Date, key Int, value String TTL date + interval 1 month) engine=MergeTree order by key settings min_bytes_for_wide_part=0, min_rows_for_wide_part=0;
    insert into ttl_02262 values ('2010-01-01', 2010, 'foo');
    ```

    ```sh
    # ls -l .server/data/default/ttl_02262/all_*
    .server/data/default/ttl_02262/all_1_1_0:
    total 48
    -rw-r----- 1 root root 335 May 26 14:19 checksums.txt
    -rw-r----- 1 root root  76 May 26 14:19 columns.txt
    -rw-r----- 1 root root   1 May 26 14:19 count.txt
    -rw-r----- 1 root root  28 May 26 14:19 date.bin
    -rw-r----- 1 root root  48 May 26 14:19 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:19 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:19 key.bin
    -rw-r----- 1 root root  48 May 26 14:19 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:19 primary.idx
    -rw-r----- 1 root root  99 May 26 14:19 ttl.txt
    -rw-r----- 1 root root  30 May 26 14:19 value.bin
    -rw-r----- 1 root root  48 May 26 14:19 value.mrk2
    ```

    ```sql
    optimize table ttl_02262 final;
    ```

    ```sh
    .server/data/default/ttl_02262/all_1_1_1:
    total 40
    -rw-r----- 1 root root 279 May 26 14:19 checksums.txt
    -rw-r----- 1 root root  61 May 26 14:19 columns.txt
    -rw-r----- 1 root root   1 May 26 14:19 count.txt
    -rw-r----- 1 root root  28 May 26 14:19 date.bin
    -rw-r----- 1 root root  48 May 26 14:19 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:19 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:19 key.bin
    -rw-r----- 1 root root  48 May 26 14:19 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:19 primary.idx
    -rw-r----- 1 root root  81 May 26 14:19 ttl.txt
    ```

    ```sql
    optimize table ttl_02262 final;
    ```

    ```sh
    .server/data/default/ttl_02262/all_1_1_2:
    total 48
    -rw-r----- 1 root root 349 May 26 14:20 checksums.txt
    -rw-r----- 1 root root  76 May 26 14:20 columns.txt
    -rw-r----- 1 root root   1 May 26 14:20 count.txt
    -rw-r----- 1 root root  28 May 26 14:20 date.bin
    -rw-r----- 1 root root  48 May 26 14:20 date.mrk2
    -rw-r----- 1 root root  10 May 26 14:20 default_compression_codec.txt
    -rw-r----- 1 root root  30 May 26 14:20 key.bin
    -rw-r----- 1 root root  48 May 26 14:20 key.mrk2
    -rw-r----- 1 root root   8 May 26 14:20 primary.idx
    -rw-r----- 1 root root  81 May 26 14:20 ttl.txt
    -rw-r----- 1 root root  27 May 26 14:20 value.bin
    -rw-r----- 1 root root  48 May 26 14:20 value.mrk2
    ```

And now we have `value.*` for all_1_1_2, this should not happen.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
Azat Khuzhin
8328d7068b Fix updating of MergeTreeDataPartTTLInfo::finished 
Previously you cannot distinguish non-initialized finished with
initialized to false, so update() cannot do the correct thing.

Rename the field to avoid hidden usage.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:10 +03:00
Azat Khuzhin
0de1a64436 Log empty parts in IMergedBlockOutputStream::removeEmptyColumnsFromPart() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-26 20:14:09 +03:00
Nikolai Kochetov
ec4e8d71b2 Fixing build 2022-05-26 15:33:21 +00:00
Alexander Tokmakov
e8f33fb0d9 fix flaky tests 2022-05-26 14:17:05 +02:00
Azat Khuzhin
dc9ca3d70c
Fix LOGICAL_ERROR in getMaxSourcePartsSizeForMerge during merges (#37413) 2022-05-26 14:14:58 +02:00
Nikolai Kochetov
84f97b53de Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-26 11:07:45 +00:00
Maksim Kita
3a92e61827
Merge pull request #37148 from kitaisreal/dictionary-get-descendants-performance-improvement 
Dictionary getDescendants performance improvement
 2022-05-26 12:29:17 +02:00
Vasily Nemkov
abe6b5d013
Reverted unnecessary modification 2022-05-26 10:09:27 +03:00
taiyang-li
561c87222d add prefetch for hive text 2022-05-26 11:04:35 +08:00
Anton Popov
f488efd27e fix tests 2022-05-26 00:03:31 +00:00
Dmitry Novik
16c6b60703 Introduce AggregationKeysInfo 2022-05-25 23:22:29 +00:00
alesapin
8f1aac0ce4 Fix merge with master 2022-05-26 00:44:45 +02:00
alesapin
1db9cf480b Merge remote-tracking branch 'origin/master' into i_object_storage 2022-05-25 22:50:22 +02:00
Alexander Tokmakov
47820c216d
Revert "(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part" 2022-05-25 23:10:33 +03:00
alesapin
c7b16065e1 Merge with master 2022-05-25 21:47:05 +02:00
Nikolai Kochetov
6d4a26afac Update ReadProgressCallback. 2022-05-25 19:45:48 +00:00
alesapin
6f5c86e55e Merge branch 'master' into i_object_storage 2022-05-25 20:49:01 +02:00
Alexander Tokmakov
779e6ea0b9 make it better, fix on cluster queries 2022-05-25 20:17:49 +02:00
Nikolai Kochetov
54d7e4139f Fix build. 2022-05-25 18:16:48 +00:00
alesapin
51868a9a4f
Merge pull request #37424 from metahys/fix_fetching_part_deadlock 
(only with zero-copy replication, non-production experimental feature not recommended to use) fix possible deadlock during fetching part
 2022-05-25 20:15:41 +02:00
alesapin
0a3597da72
Merge pull request #34915 from ianton-ru/MDB-16962 
Fix collision of S3 operation log revision
 2022-05-25 18:15:31 +02:00
Anton Popov
16e839ac71 add profile events for introspection of part types 2022-05-25 14:54:49 +00:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
msaf1980
fda6ddeffa cleanup StorageHDFS (unused variables) 2022-05-25 19:23:05 +05:00
mergify[bot]
73662b4436
Merge branch 'master' into fix_fetching_part_deadlock 2022-05-25 14:22:35 +00:00
taiyang-li
a7a816dcb6 fix build error 2022-05-25 19:55:11 +08:00
taiyang-li
1d9f65a7d4 Merge branch 'master' into async_hdfs_read_buffer 2022-05-25 17:10:22 +08:00
Kseniia Sumarokova
b50d4549c9
Merge pull request #37356 from amosbird/partition-prune-for-s3 
"Partition pruning" for s3
 2022-05-25 11:03:07 +02:00
avogar
f782fa31c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into check-format-on-storage-creation 2022-05-25 08:42:54 +00:00
Nikolai Kochetov
3d84aae0ab Better. 2022-05-24 20:06:08 +00:00
Nikolai Kochetov
333fd09dbf Fixing build. 2022-05-24 19:29:00 +00:00
Vasily Nemkov
59b4d4a643 ALTER COMMENT is now local-only operation and immediately observable 2022-05-24 21:08:30 +03:00
alesapin
9a19309e69 Slightly better fix 2022-05-24 14:46:29 +02:00
Alexander Tokmakov
229d35408b
Merge pull request #37398 from ClickHouse/fixes_for_transactions 
Fixes for transactions
 2022-05-24 15:28:01 +03:00
alesapin
3ca7a8831b Revert "fix deadlock during fetching part" 
This reverts commit 6ae8a26fae.
 2022-05-24 14:26:06 +02:00
alesapin
a5ba6bca95 Merge branch 'master' into metahys-fix_fetching_part_deadlock 2022-05-24 14:07:47 +02:00
Kruglov Pavel
6c9a524f6b
Merge pull request #37192 from Avogar/formats-with-names 
Improve performance and memory usage for select of subset of columns for some formats
 2022-05-24 13:28:14 +02:00
Amos Bird
093d315756
partition pruning for s3 2022-05-24 18:57:55 +08:00
mergify[bot]
51ff49a0ee
Merge branch 'master' into i_object_storage 2022-05-23 20:29:49 +00:00
Nikolai Kochetov
fd97a9d885 Move some resources 2022-05-23 19:47:32 +00:00
Alexander Tokmakov
6bc68c0cbc Merge branch 'master' into fixes_for_transactions 2022-05-23 18:49:21 +02:00
Nikolai Kochetov
9756b759c6 Move some resources 2022-05-23 13:46:57 +00:00
Kruglov Pavel
a0aa841909
Fix failed thread stateless tests 2022-05-23 15:15:45 +02:00
avogar
37b66c8a9e Check format name on storage creation 2022-05-23 12:48:48 +00:00
metahys
6ae8a26fae fix deadlock during fetching part 2022-05-23 20:08:04 +08:00
taiyang-li
29e2157469 change as request 2022-05-23 18:42:54 +08:00
Maksim Kita
585b86446e Added hierarchical_index_bytes_allocated column in system.dictionaries 2022-05-23 12:42:00 +02:00
taiyang-li
14f84f02d5 Merge branch 'master' into async_hdfs_read_buffer 2022-05-23 18:36:21 +08:00
Sergei Trifonov
b6bf283f4d
Merge pull request #37243 from Vxider/mv-fix-dependency 
Fix adding dependency when MaterializedView is dropped
 2022-05-23 12:33:54 +02:00
Kruglov Pavel
f539fb835d
Merge branch 'master' into formats-with-names 2022-05-23 12:14:20 +02:00
Kruglov Pavel
754e675ec3
Merge pull request #37253 from Avogar/fix-defaults 
Fix inserting defaults for missing values in columnar formats
 2022-05-23 12:10:14 +02:00
Kseniia Sumarokova
ca3feddf73
Merge pull request #37426 from Vxider/wv-simplify 
Remove unused code in WindowView
 2022-05-23 11:43:51 +02:00
mergify[bot]
747aa5575c
Merge branch 'master' into remove-useless-code-2 2022-05-22 17:41:57 +00:00
Vxider
fcd317a642 update windowview create syntax 2022-05-22 10:18:34 +00:00
alesapin
c8d92b87c8 Merge branch 'master' into i_object_storage 2022-05-22 12:16:10 +02:00
Vxider
5f84f06d6d simplify windowview 2022-05-22 10:13:16 +00:00
Robert Schulze
19d53c14fa
Merge pull request #37382 from ClickHouse/wc++98-compat-extra-semi 
Enable -Wc++98-compat-extra-semi
 2022-05-22 09:40:45 +02:00
Alexey Milovidov
4bfbb0b7ba
Merge pull request #37388 from CurtizJ/explicit-bool-operator 
Mark all `operator bool()` as explicit
 2022-05-22 00:18:19 +03:00
Alexey Milovidov
790f442362
Merge pull request #37407 from ClickHouse/certificates-mem-leak 
Fix possible memory leaks in system.certificates implementation
 2022-05-22 00:15:30 +03:00
Vasily Nemkov
c12f826d22 Implemented changing comment to a ReplicatedMergeTree table 2022-05-21 20:08:30 +03:00
Yakov Olkhovskiy
4c13b52b6a scope guard resources 2022-05-20 19:59:17 -04:00
Alexander Tokmakov
44f2d4529a better waiting, add fault injection 2022-05-20 22:10:01 +02:00
Nikolai Kochetov
56feef01e7 Move some resources 2022-05-20 19:49:31 +00:00
avogar
3bad21cea7 Try fix tests 2022-05-20 17:48:24 +00:00
Alexander Tokmakov
1027798517 handle connection loss on commit 2022-05-20 17:35:29 +02:00
Anton Popov
cb0e6c2718 mark all operators bool() as explicit 2022-05-20 15:29:54 +00:00
avogar
a4cf07708c Fix comments 2022-05-20 14:57:27 +00:00
avogar
566d1b15fd Merge branch 'master' of github.com:ClickHouse/ClickHouse into formats-with-names 2022-05-20 13:54:52 +00:00
avogar
78069765bb Try to remove code duplication 2022-05-20 13:20:52 +00:00
avogar
6e2df9ecc4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-defaults 2022-05-20 13:15:34 +00:00
Robert Schulze
0f6715bd91
Follow-up to PR #37300: semicolon warnings 
In PR #37300, Alexej asked why we the compiler does not warn about
unnecessary semicolons, e.g.

  f()
  {
  }; // <-- here

The answer is surprising: In C++98, above syntax was disallowed but by
most compilers accepted it regardless. C++>11 introduced "empty
declarations" which made the syntax legal.

The previous behavior can be restored using flag
-Wc++98-compat-extra-semi. This finds many useless semicolons which were
removed in this change. Unfortunately, there are also false positives
which would require #pragma-s and HAS_* logic (--> check_flags.cmake) to
suppress. In the end, -Wc++98-compat-extra-semi comes with extra effort
for little benefit. Therefore, this change only fixes some semicolons
but does not enable the flag.
 2022-05-20 15:06:34 +02:00
Vxider
956459eba2 Merged with master 2022-05-20 10:50:21 +00:00
Alexander Tokmakov
12bbb7de87 fix race on TID allocation 2022-05-20 12:41:44 +02:00
alesapin
654b27e307 Merge branch 'master' into i_object_storage 2022-05-20 11:56:13 +02:00
Kseniia Sumarokova
ee295038fa
Merge pull request #37214 from Vxider/wv-engine 
Change ENGINE behavior in WindowView
 2022-05-20 11:14:19 +02:00
Robert Schulze
b475fbc9a7
Merge pull request #37300 from ClickHouse/cmake-cleanup-pt3 
Various cmake cleanups
 2022-05-20 10:02:36 +02:00
alesapin
92c15ec97c Get rid of IDiskRemote 2022-05-19 20:07:15 +02:00
Dmitry Novik
b3ccf96c81 Merge remote-tracking branch 'origin/master' into grouping-function 2022-05-19 17:58:33 +00:00
Alexander Tokmakov
a53cfa9fca
Merge pull request #37355 from ClickHouse/revert-37266-fix-mutations-with-object 
Revert "Fix mutations in tables with columns of type `Object`"
 2022-05-19 16:27:22 +03:00
Vxider
f6cc73235b Merged with master 2022-05-19 13:03:45 +00:00
Vxider
b2c7ba7cbe update 2022-05-19 12:54:41 +00:00
Vxider
24f1a1fde7 Merge branch 'wv-alter' of https://github.com/Vxider/ClickHouse into wv-populate 2022-05-19 12:49:37 +00:00
vxider
797edb0ae1
Update src/Storages/WindowView/StorageWindowView.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-05-19 20:49:02 +08:00
Vxider
1cc347364f Merged with master 2022-05-19 12:45:46 +00:00
Kseniia Sumarokova
d4ad138a04
Merge pull request #37103 from bigo-sg/hive_partition_key_read 
optimization for reading hive file  when all columns to read are partition keys
 2022-05-19 14:24:00 +02:00
Kseniia Sumarokova
b0c7cf910a
Merge pull request #37224 from Vxider/wv-fix-dependency 
Fix addDependency in WindowView
 2022-05-19 13:51:11 +02:00
Kseniia Sumarokova
24d0899cb7
Merge pull request #36945 from Vxider/wv-populate 
POPULATE support for WindowView
 2022-05-19 13:47:52 +02:00
Kseniia Sumarokova
af2d9154e1
Merge pull request #37279 from bigo-sg/hive_local_cache_multi_disk 
Support multi disks for caching hive files
 2022-05-19 13:25:03 +02:00
Kruglov Pavel
468954cc87
Remove redundant include 2022-05-19 12:52:33 +02:00
Alexander Tokmakov
f787dc7097
Revert "Fix mutations in tables with columns of type Object" 2022-05-19 13:24:48 +03:00
Mikhail f. Shiryaev
32769caf04
Merge pull request #37346 from ClickHouse/22.6-prepare 
Update version after release
 2022-05-19 10:54:53 +02:00
Mikhail f. Shiryaev
d0fe794fe5
Update version to 22.6.1.1 2022-05-19 10:06:09 +02:00
mergify[bot]
7ff8098272
Merge branch 'master' into wv-alter 2022-05-19 01:51:50 +00:00
Vitaly Baranov
7c8956ca8d
Merge pull request #36879 from tonickkozlov/tonickkozlov/lazy-marks-loading 
Lazy marks loading
 2022-05-18 22:56:46 +02:00
Vxider
04f1b20229 init inner window names 2022-05-18 14:34:40 +00:00
avogar
3c85e8b455 Fix tests 2022-05-18 12:36:13 +00:00
Vxider
4b5eda4de9 fix cleanup 2022-05-18 11:14:08 +00:00
alesapin
e99c90118e Merge branch 'master' into i_object_storage 2022-05-18 12:36:22 +02:00
avogar
b33ad449a9 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-defaults 2022-05-18 10:26:38 +00:00
Kruglov Pavel
aaace46da2
Merge pull request #37266 from CurtizJ/fix-mutations-with-object 
Fix mutations in tables with columns of type `Object`
 2022-05-18 12:19:55 +02:00
Kseniia Sumarokova
f9b08ae5dd
Merge pull request #37281 from bigo-sg/hive_max_partition_query_check 
Check max partitions to read for each hive table
 2022-05-18 11:11:11 +02:00
Robert Schulze
6b2a72d451
Merge pull request #37228 from mrk-andreev/fix-rm-groupid-from-producer 
Remove group.id from StorageKafka::createWriteBuffer
 2022-05-18 11:04:40 +02:00
Vxider
f102497929 fix parser with join query 2022-05-18 08:05:06 +00:00
mergify[bot]
37d6da7506
Merge branch 'master' into wv-engine 2022-05-18 01:56:56 +00:00
mergify[bot]
4900d2249f
Merge branch 'master' into wv-populate 2022-05-18 01:56:52 +00:00
mergify[bot]
c33ca1f392
Merge branch 'master' into system-certificates 2022-05-17 19:52:28 +00:00
Vxider
037db32e24 set alter query internal 2022-05-17 13:50:55 +00:00
lgbo-ustc
e4bc7e9979 use splitInto instead of Poco::Tokenizer 2022-05-17 19:31:59 +08:00
lgbo-ustc
1497e08301 update exception msg 2022-05-17 19:27:43 +08:00
Kruglov Pavel
515ae986e5
Fix special build 2022-05-17 12:15:41 +02:00
Kseniia Sumarokova
8cb71d0172
Merge pull request #33332 from Michicosun/MeiliSearch 
MeiliSearch
 2022-05-17 11:47:56 +02:00
alesapin
4d4634b12d
Merge pull request #37164 from kssenii/ficx-async-reads-assertions 
fix async reads from remote fs internal setting not being always turned on
 2022-05-17 11:29:14 +02:00
taiyang-li
14ab7eb5a3 merge master and solve conflict 2022-05-17 16:28:08 +08:00
lgbo-ustc
0b3468a150 TOO_MANY_PARTITIONS 2022-05-17 15:50:03 +08:00
lgbo-ustc
f4f4a2d85b reuse setting max_partitions_to_read 2022-05-17 15:49:14 +08:00
lgbo-ustc
4d64678963 Merge remote-tracking branch 'ck/master' into hive_max_partition_query_check 2022-05-17 15:42:10 +08:00
lgbo-ustc
a161a21992 add max partitions check for each hive table 2022-05-17 15:37:32 +08:00
Vxider
90bac2f004 add shutdown check 2022-05-17 06:20:08 +00:00
lgbo-ustc
bc81302bf4 fixed code-style 2022-05-17 12:10:55 +08:00
lgbo-ustc
3a32a22f39 support using multi disks for caching hive files 2022-05-17 11:55:30 +08:00
Vxider
f74c934a61 Merged with master 2022-05-17 02:53:59 +00:00
mergify[bot]
0d0fc23170
Merge branch 'master' into wv-engine 2022-05-17 02:24:18 +00:00
Vxider
3986e82b19 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into wv-populate 2022-05-17 02:18:17 +00:00
lgbo-ustc
d8ad9ad2a6 update codes 2022-05-17 09:27:03 +08:00
Kseniia Sumarokova
94683786dc
Merge branch 'master' into MeiliSearch 2022-05-16 22:42:09 +02:00
Kseniia Sumarokova
f073efc025
Merge pull request #37225 from Vxider/wv-fix-cleanup 
Fix WindowView cleanup
 2022-05-16 22:39:58 +02:00
Vitaly Baranov
72bff5242a
Merge pull request #37168 from vitlibar/backup-improvements-5 
Backups Improvements 5
 2022-05-16 21:25:54 +02:00
Robert Schulze
43945cea1b
Fixing some warnings 2022-05-16 20:59:27 +02:00
Yakov Olkhovskiy
4f6c354c3f refactoring 2022-05-16 14:38:47 -04:00
Anton Popov
b6c5ab4fcf fix mutations in tables with columns of type Object 2022-05-16 18:26:53 +00:00
Dmitry Novik
e5b395e054 Support ROLLUP and CUBE in GROUPING function 2022-05-16 17:33:38 +00:00
Yakov Olkhovskiy
863c52f0f4 refactoring, test added 2022-05-16 13:31:28 -04:00
alesapin
ce6d5f7dcc Merge branch 'master' into i_object_storage 2022-05-16 17:42:22 +02:00
avogar
ce834b1086 Remove code duplication 2022-05-16 14:41:01 +00:00
avogar
2d4b4b9008 Fix inserting defaults for missing values in columnar formats 2022-05-16 14:19:44 +00:00
Vxider
8f271159bf check null pointer 2022-05-16 13:59:55 +00:00
Mikhail Artemenko
535b3ea686 refactor code & change exception numbers 2022-05-16 16:13:05 +03:00
Vxider
b3a731c226 fix build 2022-05-16 12:51:19 +00:00
mergify[bot]
b7d6d97e64
Merge branch 'master' into wv-alter 2022-05-16 12:44:46 +00:00
Robert Schulze
ed7df7cabd
Merge pull request #36960 from ClickHouse/clangtidies 
Enable clangtidies
 2022-05-16 14:22:50 +02:00
Vxider
6b3b744e3f fix data race 2022-05-16 11:31:49 +00:00
kssenii
f7493965b7 Remove additional check for merge tree 2022-05-16 11:19:57 +02:00
kssenii
49201f217f Revert 2022-05-16 11:17:56 +02:00
Vxider
31eee576a5 do not addDependency when table is dropped 2022-05-16 09:04:37 +00:00
Vxider
95e6e5aef7 fix window view dependency 2022-05-16 08:41:32 +00:00
Robert Schulze
e3cfec5b09
Merge remote-tracking branch 'origin/master' into clangtidies 2022-05-16 10:12:50 +02:00
Vxider
9201df7cde update code style 2022-05-16 02:00:39 +00:00
alesapin
4f1d4dde22 Disable s3 logging by default 2022-05-16 00:16:00 +02:00
Mark Andreev
fa6a5ab6e4 Remove group.id from StorageKafka::createWriteBuffer 2022-05-15 23:51:11 +04:00
Vxider
77ad48fee9 fix build 2022-05-15 17:40:42 +00:00
Vxider
15b35c1910 speed up windowview watch 2022-05-15 17:14:53 +00:00
Vxider
18ee285ce1 fix windowview cleanup 2022-05-15 16:50:34 +00:00
Vxider
93fec4ebc7 remove dependency when drop windowview 2022-05-15 16:12:24 +00:00
Robert Schulze
859f2f3391
Enclose large if block in {} 2022-05-15 14:49:20 +02:00
Vxider
033b1e7f82 add lock for cleanup query 2022-05-15 11:18:42 +00:00
Vxider
6c2d78b802 rename var 2022-05-15 11:18:23 +00:00
Vxider
65779adf8b update code style 2022-05-15 07:07:27 +00:00
Vxider
4ed79cd9e1 fix windowview watch 2022-05-15 06:45:25 +00:00
Yakov Olkhovskiy
6156047c83 some refactoring 2022-05-14 17:40:50 -04:00
Yakov Olkhovskiy
ef2d8d3d0e bugfix, refactoring 2022-05-14 15:45:07 -04:00
Vxider
f459dd5aba change windowview engine behaviour 2022-05-14 16:46:49 +00:00
Mikhail Artemenko
91edbf0f47 add subquery evaluation in getConfiguration 2022-05-14 14:44:35 +03:00
Vxider
f2ef27e5d6 rename windowview header 2022-05-14 09:21:54 +00:00
Vxider
b52cac970d fix alter on replicated engine 2022-05-14 06:35:50 +00:00
mergify[bot]
4e184306c1
Merge branch 'master' into wv-alter 2022-05-14 02:11:46 +00:00
Dmitry Novik
6fc7dfea80 Support ordinary GROUP BY 2022-05-13 23:04:12 +00:00
alesapin
cc304a3178 Merge branch 'master' into i_object_storage 2022-05-13 23:44:00 +02:00
alesapin
1c082717a4 Relax log level for some checks in check thread 2022-05-13 23:29:36 +02:00
Yakov Olkhovskiy
d2ffd0e1c2 take certificate path from poco Context 2022-05-13 16:48:34 -04:00