Commit Graph

5179 Commits

Author SHA1 Message Date
Vladimir Chebotarev
ce0850007f 1. Draw some circles. 2021-08-30 11:49:26 +03:00
alexey-milovidov
af709ab9a0
Merge pull request #28326 from DimasKovas/generate_ya_make_30_08_2021
Try to fix arcadia build (generate ya.make)
2021-08-30 11:28:02 +03:00
Dmitrii Kovalkov
9871ad70ff Exclude fuzzers 2021-08-30 11:12:25 +03:00
Dmitrii Kovalkov
8acf0fa16a run generate-ya-make 2021-08-30 10:57:40 +03:00
kssenii
378f4854da Rename 2021-08-30 05:50:08 +00:00
Maksim Kita
d88de46312 StorageExecutable added support for create 2021-08-29 23:19:05 +03:00
alexey-milovidov
77f26e8673
Merge pull request #28299 from amosbird/fixtmprowsource
Use real tmp file instead of predefined “rows_sources" for vertical merges
2021-08-29 21:49:59 +03:00
Alexey Milovidov
e4f3d97b52 Merge branch 'master' of github.com:yandex/ClickHouse into async-reads 2021-08-29 21:42:14 +03:00
alexey-milovidov
3d3afbdb35
Merge pull request #28193 from amosbird/alterprimarykey2
Reserve protocol number for ALTER PRIMARY KEY.
2021-08-29 20:37:19 +03:00
Kseniia Sumarokova
7784f4ebb0
Merge pull request #28268 from infinivision/hdfs_namenode_ha
configable LIBHDFS3_CONF, refers to #8159
2021-08-29 12:25:42 +03:00
Amos Bird
9e7ae5dd6d
Use real tmp file instead of predefined one 2021-08-29 16:54:15 +08:00
Maksim Kita
4aa2479ca1 Added support for multiple pipes 2021-08-28 22:47:59 +03:00
Amos Bird
4624bf70b0
Always monotonic for non-zero division 2021-08-28 23:33:18 +08:00
Amos Bird
f2374a6916
Better nullable primary key implementation. 2021-08-28 17:48:28 +08:00
zhongyuankai
72e05cda1f Merge branch 'master' into add-system-of-table_views 2021-08-28 11:32:58 +08:00
Amos Bird
b68857d086
Simplify projection, add minmax_count projection. 2021-08-28 11:25:37 +08:00
Zhichang Yu
ef1994f420 set env LIBHDFS3_CONF, refers to ClickHouse#8159 2021-08-28 09:16:48 +08:00
tavplubix
e3b2586a02
Merge pull request #28274 from ClickHouse/revert-28054-mysql
Revert "Add test for #13398"
2021-08-28 04:04:10 +03:00
Alexey Milovidov
79e0433ba7 Merge branch 'master' of github.com:yandex/ClickHouse into async-reads 2021-08-28 01:19:16 +03:00
alexey-milovidov
4eef445df9
Merge pull request #28266 from CurtizJ/fix-order-by-merge
Fix order by for `Merge` tables with `optimize_read_in_order`
2021-08-28 01:15:40 +03:00
tavplubix
c36fe210b8
Revert "Add test for #13398" 2021-08-27 21:07:10 +03:00
Anton Popov
89e1d1f64a fix order by for StorageMerge with optimize_read_in_order 2021-08-27 16:55:55 +03:00
Amos Bird
ce96b2c35a
bump 2021-08-27 20:44:26 +08:00
tavplubix
703101fe4d
Merge pull request #27931 from ClickHouse/wait_for_all_replicas_timeouts
Avoid too long waiting for inactive replicas
2021-08-27 14:31:36 +03:00
Nikolai Kochetov
34f4bd3e72
Merge pull request #28178 from amosbird/projection-fix14
Projection bug fixes and refactoring.
2021-08-27 11:47:46 +03:00
zhongyuankai
a7ee7621dd optimize code 2021-08-27 15:44:50 +08:00
Azat Khuzhin
19458a0c61 Fix removing of parts in a Temporary state
Parts in a temporary state is not exists in data_parts_by_info, so do
not try to search there and throw LOGICAL_ERROR in case of failure:

<details>

```
    02:45:49.037546 [ 5890 ] {} <Error> test_iy9rta.concurrent_kill_4 (526aa7c8-db2a-4f0e-926a-a7c8db2a9f0e): Code: 40. DB::Exception: Part all_0_0_0_1 from r11 has different columns hash. (CHECKSUM_DOESNT_MATCH) (version 21.10.1.7910 (official build)). Data after mutation is not byte-identical to data on another replicas. We will download merged part from replica to force byte-identical result.
    02:45:49.049422 [ 5890 ] {} <Trace> test_iy9rta.concurrent_kill_4 (526aa7c8-db2a-4f0e-926a-a7c8db2a9f0e): Trying to immediately remove part all_0_0_0_1 (state Temporary)
    02:45:49.060210 [ 5890 ] {} <Fatal> : Logical error: 'Part all_0_0_0_1 doesn't exist'.
    02:47:01.572508 [ 29208 ] {} <Fatal> BaseDaemon: (version 21.10.1.7910 (official build), build id: 9309CECED9A0D32CBB00BB8EC77B91456679868E) (from thread 5890) (no query) Received signal Aborted (6)
    ...
    02:47:06.815000 [ 29208 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeData.cpp:2593: DB::MergeTreeData::tryRemovePartImmediately(std::__1::shared_ptr<DB::IMergeTreeDataPart const>&&) @ 0x1f959e0d in /usr/bin/clickhouse
    02:47:06.816309 [ 29209 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeData.cpp:2593: DB::MergeTreeData::tryRemovePartImmediately(std::__1::shared_ptr<DB::IMergeTreeDataPart const>&&) @ 0x1f959e0d in /usr/bin/clickhouse
    02:47:09.455665 [ 29208 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1939: DB::StorageReplicatedMergeTree::tryExecutePartMutation(DB::ReplicatedMergeTreeLogEntry const&) @ 0x1f5f2bf6 in /usr/bin/clickhouse
    02:47:09.468738 [ 29209 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1939: DB::StorageReplicatedMergeTree::tryExecutePartMutation(DB::ReplicatedMergeTreeLogEntry const&) @ 0x1f5f2bf6 in /usr/bin/clickhouse
    02:47:11.776857 [ 29208 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1581: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x1f5e484c in /usr/bin/clickhouse
    02:47:11.904232 [ 29209 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1581: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x1f5e484c in /usr/bin/clickhouse
    02:47:13.941811 [ 29208 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:3176: DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18::operator() shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) const @ 0x1f65faa5 in /usr/bin/clickhouse
    02:47:14.477158 [ 29209 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:3176: DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18::operator() shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) const @ 0x1f65faa5 in /usr/bin/clickhouse
    02:47:16.475373 [ 29208 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676: decltype(std::__1::forward<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>))(std::__1::forward<std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(fp0))) std::__1::__invoke<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) @ 0x1f65fa32 in /usr/bin/clickhouse
    02:47:16.970325 [ 29209 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676: decltype(std::__1::forward<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>))(std::__1::forward<std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(fp0))) std::__1::__invoke<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) @ 0x1f65fa32 in /usr/bin/clickhouse
    02:47:18.979481 [ 29208 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:317: bool std::__1::__invoke_void_return_wrapper<bool>::__call<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplireeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogE x1f65f9f2 in /usr/bin/clickhouse
    02:47:19.450807 [ 29209 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:317: bool std::__1::__invoke_void_return_wrapper<bool>::__call<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplireeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogE x1f65f9f2 in /usr/bin/clickhouse
    02:47:21.055007 [ 29208 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608: std::__1::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::ry>)::$_18, bool (std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&)>::operator()(std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) @ 0x1f65f9b0 in /usr/bin/clickhouse
    02:47:23.546946 [ 413 ] {} <Fatal> Application: Child process was terminated by signal 6.
```

</details>

CI: https://clickhouse-test-reports.s3.yandex.net/0/4a8b82232c11512232df3ecdf4ffaec287116ad5/stress_test_(debug).html#fail1
2021-08-27 10:08:44 +03:00
zhongyuankai
306da6f414 update system views test 2021-08-27 10:41:15 +08:00
Alexey Milovidov
46732e781f Merge branch 'master' of github.com:yandex/ClickHouse into async-reads 2021-08-27 01:57:48 +03:00
Amos Bird
103e0abe1c
Reserve protocol number for ALTER PRIMARY KEY. 2021-08-26 22:08:58 +08:00
Kruglov Pavel
5bc332c40c
Merge pull request #28054 from kssenii/mysql
Add test for #13398
2021-08-26 16:39:43 +03:00
Kseniia Sumarokova
31afd7d09c
Merge pull request #28081 from kssenii/pg-conflict
Support `on conflict` for postgres engine
2021-08-26 16:30:30 +03:00
zhongyuankai
15cbd7470a StorageSystemViews bug fix 2021-08-26 20:54:04 +08:00
zhongyuankai
042d17531c system.views add columns type 2021-08-26 20:05:01 +08:00
Amos Bird
0169fce78e
Projection bug fixes and refactoring. 2021-08-26 19:09:31 +08:00
alesapin
bcaff65457
Merge pull request #28028 from ClickHouse/replicated-tree-attach-wip
Allow attaching parts with 'compatible' enum types
2021-08-26 13:50:29 +03:00
Ilya Yatsishin
5553a064be
Merge pull request #28139 from lthaooo/fix_prewhere_total_size_of_queried_columns_bug
Fix bug in MergeTreeWhereOptimizer
2021-08-26 10:42:23 +03:00
Alexey Milovidov
7dd0146223 Fix error 2021-08-26 01:56:08 +03:00
Alexey Milovidov
3cb9cf7242 Merge branch 'master' of github.com:yandex/ClickHouse into async-reads 2021-08-26 01:55:46 +03:00
kssenii
0b66581de2 Fix build 2021-08-25 21:51:43 +00:00
Nikita Mikhaylov
ae7c8df0b9
Merge pull request #27872 from nikitamikhaylov/codecs-fuzzing
Add fuzzers for codecs
2021-08-26 00:17:01 +03:00
Maksim Kita
6ccdde4a88 Updated TableFunctionExecutable 2021-08-25 22:30:22 +03:00
Maksim Kita
2403686bb9
Merge pull request #28135 from amosbird/projection-improve3
Get rid of useless projection columns during merge
2021-08-25 22:14:38 +03:00
alesapin
eaa49f56dd
Merge pull request #27019 from lthaooo/materialize_ttl_recalculate_only
Improvement of Materialize TTL
2021-08-25 19:10:20 +03:00
Nikita Mikhaylov
43fb3098ae Deleted some unneded changes 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
2c09533c83 lower a binary size for compression fuzzers 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
ad4966f6d2 better 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
0615bf27d6
Merge pull request #27990 from nikitamikhaylov/fix-clang-tidy
Build fuzzers in CI
2021-08-25 18:33:41 +03:00
lthaooo
eecd5b21c9 fix total_size_of_queried_columns bug 2021-08-25 23:02:13 +08:00
Amos Bird
a6b2a6ee36
Get rid of useless projection columns during merge 2021-08-25 20:54:27 +08:00
zhongyuankai
64efa917d8 review system.views 2021-08-25 18:11:40 +08:00
zhongyuankai
52570b1c80 Merge branch 'master' into add-system-of-table_views 2021-08-25 15:59:23 +08:00
lthaooo
314ba1e682
Merge branch 'master' into materialize_ttl_recalculate_only 2021-08-25 10:56:24 +08:00
Alexey Milovidov
848c2ec23e Tuneable prefetch 2021-08-25 02:38:08 +03:00
Alexey Milovidov
fd27a2faef Remove imperfect code 2021-08-25 02:05:55 +03:00
Alexey Milovidov
7c1d0a3baf Progress on development 2021-08-25 01:24:47 +03:00
Alexey Milovidov
f7ca674b0c Progress on development 2021-08-25 01:07:06 +03:00
Alexey Milovidov
8f57216180 Progress on development 2021-08-25 00:45:58 +03:00
Alexey Milovidov
4d9ad3725d Merge branch 'master' into async-reads 2021-08-25 00:45:40 +03:00
Maksim Kita
5545959581 Updated StorageExecutable 2021-08-24 22:38:42 +03:00
Maksim Kita
016c7c74f8
Merge branch 'executable-table-function' into add-executable-table-function 2021-08-24 21:45:43 +03:00
Dmitry Novik
2da3893b1f refactor isCompatibleEnumTypes to remove PVS warning 2021-08-24 21:11:47 +03:00
Maksim Kita
82ba24cd10 Merge branch 'master' into create-user-defined-lambda-function 2021-08-24 19:12:32 +03:00
tavplubix
0602d74a11
Merge pull request #28035 from ClickHouse/fix_replace_ranges_may_stuck
Fix race between REPLACE PARTITION and MOVE PARTITION
2021-08-24 17:49:57 +03:00
Vitaly Baranov
64f69462c6
Merge pull request #27921 from vitlibar/add-tests-for-field-init
Allow implicit cast bool to Field
2021-08-24 17:15:44 +03:00
Kseniia Sumarokova
5bbf5dacc5
Update src/Storages/StoragePostgreSQL.cpp
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
2021-08-24 16:13:57 +03:00
kssenii
0ec716292c Postgres ON CONFLICT 2021-08-24 12:37:32 +00:00
OnePiece
17ba4c620e
Merge branch 'ClickHouse:master' into add-system-of-table_views 2021-08-24 20:20:24 +08:00
OnePiece
e9edabc3cb
Update attachSystemTables.cpp 2021-08-24 20:09:36 +08:00
OnePiece
d2919e98c7
Update attachSystemTables.cpp 2021-08-24 20:09:04 +08:00
OnePiece
29aa41577f
Create StorageSystemTableViews.h 2021-08-24 20:07:48 +08:00
OnePiece
ba594f41b9
Create StorageSystemTableViews.cpp 2021-08-24 20:07:08 +08:00
Dmitry Novik
1984940eb6 Update all StorageInMemoryMetadata::check overloads 2021-08-24 14:59:35 +03:00
mergify[bot]
ae22bab2c4
Merge branch 'master' into materialize_ttl_recalculate_only 2021-08-24 08:49:19 +00:00
kssenii
0d5000f917 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into mysql 2021-08-24 07:30:42 +00:00
kssenii
14b87feb27 Fix build 2021-08-24 07:30:33 +00:00
kssenii
b536653851 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-24 07:29:50 +00:00
Vladimir C
98acccbdcc
Merge branch 'master' into s3partitionedwrite 2021-08-24 10:03:57 +03:00
kssenii
5307d31924 Throw for alter and silence for drop 2021-08-23 23:48:53 +00:00
kssenii
962f415740 Add test and settings 2021-08-23 22:18:04 +00:00
Dmitry Novik
e78f9b3d4b code cleanup 2021-08-24 00:42:03 +03:00
Vitaly Baranov
63e8bc1f20 Added new commands BACKUP and RESTORE. 2021-08-23 23:41:45 +03:00
Anton Popov
0db8b524f0
Merge pull request #27984 from BraulioVM/optimize/has-all-bloom-filter
Make `hasAll` filter condition leverage bloom filters
2021-08-23 22:19:50 +03:00
Alexander Tokmakov
d95131dd4a better check for alter partition version 2021-08-23 20:51:20 +03:00
Dmitry Novik
6dad70a1fb Add check whether enums have the same underlying type or not 2021-08-23 17:55:21 +03:00
Alexander Tokmakov
4a4a0b4823 fix 2021-08-23 16:32:07 +03:00
Dmitry Novik
ac2e9a27e6 Fixes #26672. Lower restrictions in StorageInMemoryMetadata::check. 2021-08-23 16:08:23 +03:00
Alexander Tokmakov
cc9c2fd63b make code better 2021-08-23 15:57:50 +03:00
Ilya Yatsishin
315a835c95
Merge pull request #28020 from qoega/fix-style 2021-08-23 15:48:41 +03:00
Yatsishin Ilya
17bb938541 fix style 2021-08-23 13:59:01 +03:00
mergify[bot]
95e9d517ff
Merge branch 'master' into fix-clang-tidy 2021-08-23 10:54:36 +00:00
Maksim Kita
2a6aa50d49 Merge branch 'master' into create-user-defined-lambda-function 2021-08-23 11:08:49 +03:00
alesapin
31b12b5240
Merge pull request #27787 from kafka1991/master
modify column datatype to Nullable(datatype) or Nullable(datatype) to Nullable(datatype) donot allways need mutations.
2021-08-23 10:35:39 +03:00
Vitaly Baranov
89166a655a Init SettingFieldBool with true or false, not with 1 or 0. 2021-08-23 09:47:36 +03:00
vdimir
6fe63a80bc Fix validateKey/Bucket for S3 2021-08-23 09:22:15 +03:00
vdimir
6ba6577978 Do not allow slashes in bucket formatted from PARTITION BY 2021-08-23 09:22:15 +03:00
vdimir
c4c42b61c4 Validate uft8 in partition key from PARTITION BY for s3 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
f6cb82aba5 Minor fix. 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
e203c330c2 Validation. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
6e10d28c78 Fix. 2021-08-23 09:22:14 +03:00