thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 11:22:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
77,532 Commits 9,338 Branches 1,783 Tags 2.1 GiB
56b613068b
Commit Graph

2703 Commits

Author SHA1 Message Date
Alexander Tokmakov
518f542c3e fix reordering of REPLACE_RANGE and DROP PART 2021-09-10 17:32:45 +03:00
alesapin
d1dea0e95f
Update src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp 
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
 2021-09-10 15:32:24 +03:00
Anton Popov
5cff615eca Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-10 13:40:48 +03:00
alexey-milovidov
2236a5df98
Merge pull request #28085 from myrrc/improvement/merge-tree-part-opt-parse 
optional<> semantics for parsing MergeTreePartInfo and DetachedPartInfo
 2021-09-10 00:04:03 +03:00
Nikolai Kochetov
f569a3e3f7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-09 20:30:23 +03:00
alesapin
59edf6b5f6 Better logic on part checker 2021-09-09 18:19:12 +03:00
kssenii
0c013ed91b Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2 2021-09-09 15:26:36 +03:00
Nikolai Kochetov
13eb93a9c0
Merge pull request #28700 from amosbird/projection-fix16 
Fix crash on exception with projection aggregate
 2021-09-09 15:24:05 +03:00
Anton Popov
8e5b3b2f6c Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-09 15:12:34 +03:00
Nikolai Kochetov
abfb1d20ea
Merge pull request #25734 from vdimir/join-arrayjoin-type-cast 
Move join type inference to ExpressionAnalyzer from syntax stage + small refactor
 2021-09-09 14:38:43 +03:00
Anton Popov
2ccdb6c22d fix clang-tidy 2021-09-09 12:40:26 +03:00
Nikita Mikhaylov
4db5062d6b
Merge pull request #28374 from nikitamikhaylov/global-merge-executor 
Introduced global executor for background MergeTree-related operations
 2021-09-09 11:30:21 +03:00
alesapin
407c0bbb56
Merge pull request #28734 from ClickHouse/fix_race_on_mutation_pointer 
Fix race on mutation_pointer update
 2021-09-09 10:36:09 +03:00
Amos Bird
91293c7449
Fix crash on exception with projection aggregate 2021-09-09 10:43:56 +08:00
Nikita Mikhaylov
f12504b002 Fix stupid race 2021-09-08 20:30:54 +00:00
Anton Popov
9816014f6c fix nested and scalar columns with dot, which is missed in part 2021-09-08 21:42:34 +03:00
kssenii
6108a4139c Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2 2021-09-08 20:54:07 +03:00
Nikita Mikhaylov
1416b1ccba
Update MergeTreeBackgroundExecutor.cpp 2021-09-08 20:21:25 +03:00
Nikita Mikhaylov
dae9a01b4b Fix build 2021-09-08 16:13:32 +00:00
Nikita Mikhaylov
3b7fa06540 Review fixes 2021-09-08 12:42:03 +00:00
alesapin
f8a6dfb0c4 Fix race on mutation_pointer update 2021-09-08 14:37:16 +03:00
Nikita Mikhaylov
ed37a22c37 Delete debug print 2021-09-08 09:35:18 +00:00
Nikita Mikhaylov
6062dd0021 Better 2021-09-08 00:21:21 +00:00
Azat Khuzhin
db0767a194 Implement detach_not_byte_identical_parts 
Maybe useful for further analysis of non byte-identical parts.
 2021-09-07 23:29:57 +03:00
Nikita Mikhaylov
231f64cf37 Fix unit test 2021-09-07 13:51:30 +00:00
Nikita Mikhaylov
9b38248e64 Remove scheduler thread 2021-09-07 13:13:23 +00:00
Nikita Mikhaylov
8a0765c60d Review fixes (partially) 2021-09-07 12:45:39 +00:00
mergify[bot]
e9fbdec13a
Merge branch 'master' into improvement/merge-tree-part-opt-parse 2021-09-07 09:52:24 +00:00
Nikita Mikhaylov
cfe2207be3 Better 2021-09-06 22:07:41 +00:00
Nikita Mikhaylov
324ab97f50 Fix gcc build 2021-09-06 16:44:46 +00:00
Nikita Mikhaylov
7c39f84a5e Merge upstream/master into global-merge-executor (using imerge) 2021-09-06 15:37:18 +00:00
Mike Kot
8e9aacadd1 Initial: replacing hardcoded toString for enums with magic_enum 2021-09-06 16:24:03 +02:00
Nikita Mikhaylov
17d65b3d7b Allow allocations 2021-09-06 14:04:37 +00:00
Nikita Mikhaylov
ea0fbf81af Renaming 2021-09-06 12:01:16 +00:00
Nikita Mikhaylov
0249015515 added priority queue 2021-09-06 11:37:51 +00:00
Nikolai Kochetov
2178ed21df
Merge pull request #28560 from amosbird/projection-fix15 
Fix wrong header of minmax_count projection
 2021-09-06 13:00:44 +03:00
mergify[bot]
8eba5f29a1
Merge branch 'master' into improvement/merge-tree-part-opt-parse 2021-09-06 09:53:46 +00:00
Amos Bird
8fac97692a
Fix nullable primary key with constant conversion 2021-09-06 15:11:37 +08:00
Nikita Mikhaylov
3eec8a3c2f Use boost::circular_buffer 2021-09-03 22:15:20 +00:00
kssenii
6ee94c923f Fix 2021-09-04 00:51:11 +03:00
Anton Popov
59128412ea Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-03 22:17:11 +03:00
Nikolai Kochetov
66a76ab70f Rewrite PushingToViewsBlockOutputStream part 6 2021-09-03 20:29:36 +03:00
Kruglov Pavel
f559c34113
Merge pull request #28302 from amosbird/binaryconstantwrap 
Always monotonic for non-zero division
 2021-09-03 20:02:54 +03:00
Nikita Mikhaylov
26ab6ebc0a finalize in destructor 2021-09-03 14:20:34 +00:00
Nikita Mikhaylov
b867eabe73 Use Poco::Event 2021-09-03 13:02:49 +00:00
alesapin
dd7493d871
Merge pull request #27038 from excitoon-favorites/altermaterializecolumn 
`ALTER TABLE ... MATERIALIZE COLUMN`
 2021-09-03 15:50:52 +03:00
Nikita Mikhaylov
218f0adca7 Merge upstream/master into global-merge-executor (using imerge) 2021-09-03 12:29:48 +00:00
Nikita Mikhaylov
ca7adb76b9 Owning future 2021-09-03 12:27:49 +00:00
Mike Kot
1f148719ce Review fixes 2021-09-03 13:33:40 +02:00
Amos Bird
bb87d01579
Fix wrong header of minmax_count projection 2021-09-03 18:49:01 +08:00
Nikita Mikhaylov
d777c6cd70 Better [2] 2021-09-03 00:51:12 +00:00
Nikita Mikhaylov
60765b8c6e Better 2021-09-03 00:45:19 +00:00
Nikita Mikhaylov
292a24abe8 Merge upstream/master into global-merge-executor (using imerge) 2021-09-03 00:34:24 +00:00
Nikita Mikhaylov
9257188512 Comments 2021-09-03 00:28:29 +00:00
Nikita Mikhaylov
7baad6adde Fix build 2021-09-02 22:56:03 +00:00
Nikita Mikhaylov
cc7c221fad Own PriorityQueue + prettifying the code 2021-09-02 21:31:32 +00:00
Nikita Mikhaylov
6624fa12ba deleted set 2021-09-02 18:49:37 +00:00
Nikita Mikhaylov
a6fe91ca47 get rid of half of allocations 2021-09-02 17:40:29 +00:00
alesapin
464b8167d1 Update comment 2021-09-02 18:29:26 +03:00
alesapin
ba78a737dc Fix detach/attach for ReplicatedVersionedCollapsingMergeTree after alter 2021-09-02 17:47:00 +03:00
alesapin
b592400e54
Merge pull request #28404 from ClickHouse/remove_outdated_settings 
Remove obsolete settings for replicated fetches.
 2021-09-02 14:03:12 +03:00
Nikita Mikhaylov
7f21cd7f3d save 2021-09-02 10:39:27 +00:00
Nikolai Kochetov
5f0b1bc4e3
Merge pull request #26286 from amosbird/projection-improve2 
Virtual projection for min max indices.
 2021-09-02 13:03:30 +03:00
Anton Popov
5e421ab272 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-02 02:42:50 +03:00
mergify[bot]
5d299fbdee
Merge branch 'master' into remove_outdated_settings 2021-09-01 14:07:48 +00:00
Nikita Mikhaylov
c7727f093d Merge upstream/master into global-merge-executor (using imerge) 2021-08-31 23:34:11 +00:00
Nikita Mikhaylov
ac5316ce98 Stress test + reworker executor 2021-08-31 23:20:23 +00:00
alexey-milovidov
4cc0b0298c
Merge pull request #28269 from amosbird/fixweirdcode 
Better nullable primary key implementation
 2021-09-01 00:48:45 +03:00
Nikita Mikhaylov
28eff0e369 save 2021-08-31 18:07:24 +00:00
alesapin
2c7a51d816
Merge pull request #28366 from azat/remove-Temporary-part-fix-v2 
Fix removing of parts in a Temporary state (v2 followup)
 2021-08-31 19:13:15 +03:00
Nikita Mikhaylov
dbc950caa4 added a test 2021-08-31 14:54:24 +00:00
Nikolai Kochetov
5be05c2ef9 Rewrite PushingToViewsBlockOutputStream part 3. 2021-08-31 16:50:07 +03:00
Mike Kot
4194ed2ec7 Merge remote-tracking branch 'upstream/master' into improvement/merge-tree-part-opt-parse 2021-08-31 14:35:14 +02:00
Nikita Mikhaylov
e39f9aabde delete useless file 2021-08-31 12:29:51 +00:00
alesapin
9a3bbf44e3 Better description 2021-08-31 15:28:17 +03:00
vdimir
e8e26463bf
Change signature of ASTSelectQuery::arrayJoinExpressionList 2021-08-31 15:27:44 +03:00
alesapin
921e51e061 Remove some obsolete settings for replicated fetches 2021-08-31 15:22:56 +03:00
Nikita Mikhaylov
3027efe860 after merge with master 2021-08-31 12:09:35 +00:00
Nikita Mikhaylov
f8d4f04294 Merge upstream/master into global-merge-executor (using imerge) 2021-08-31 11:52:11 +00:00
Nikita Mikhaylov
1adb9bfe23 better 2021-08-31 11:02:39 +00:00
alesapin
525999145e
Merge pull request #26791 from ClickHouse/async-reads 
Experiment with asynchronous readers
 2021-08-31 13:17:45 +03:00
Nikolai Kochetov
8143c2e2b8
Merge pull request #28353 from ClickHouse/another-try-to-fix-BackgroundPoolTask-decrement 
Another try to fix BackgroundPoolTask decrement.
 2021-08-31 11:34:26 +03:00
Alexey Milovidov
ca6f6306f3 Merge branch 'master' into async-reads 2021-08-31 02:52:58 +03:00
Nikita Mikhaylov
c4416906c8 done 2021-08-30 19:37:03 +00:00
Azat Khuzhin
2c0390f326 Fix removing of parts in a Temporary state (v2 followup) 
CI reports [1]:

    <Fatal> : Logical error: 'Deleting data part all_0_0_0_3 doesn't exist'.
    ...
    <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeData.cpp:1254: DB::MergeTreeData::removePartsFinally() @ 0x1f94dffa in /usr/bin/clickhouse

  [1]: https://clickhouse-test-reports.s3.yandex.net/0/855a53ff8160c4638fe345b0d26e062804ba790a/stress_test_(debug).html#fail1

Follow-up for: #28221
 2021-08-30 21:07:26 +03:00
Azat Khuzhin
0828548b88 Fix intersecting parts due to new part had been replaced with an empty part 
AFAICS the problem is that some parts may be replaced with empty parts
(after #25820), and removed by the cleanup thread, due to it is empty
[1] (while it should not be deleted since it can download source part):

    <details>

    ```
    2021.08.18 20:11:22.687933 [ 341 ] {} <Trace> test_dpefxp.alter_table_1 (0758ca24-90e7-452c-8758-ca2490e7252c): Created log entry for mutation -1_115_115_0_146
    ...
    2021.08.18 20:11:22.707609 [ 22825 ] {} <Trace> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Executing log entry to mutate part -1_115_115_0 to -1_115_115_0_146
    2021.08.18 20:11:22.707643 [ 22825 ] {} <Debug> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Source part -1_115_115_0 for -1_115_115_0_146 is not ready; will try to fetch it instead
    ...
    2021.08.18 20:11:22.709397 [ 333 ] {} <Trace> test_dpefxp.alter_table_6 (ReplicatedMergeTreeQueue): Not executing log entry queue-0000001579 for part -1_115_115_0 because it is covered by part -1_115_115_0_146 that is currently executing.
    2021.08.18 20:11:22.718861 [ 22825 ] {} <Information> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): DB::Exception: No active replica has part -1_115_115_0_146 or covering part
    ...
    2021.08.18 20:11:27.936829 [ 295 ] {} <Information> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Going to replace lost part -1_115_115_0_146 with empty part
    2021.08.18 20:11:27.957839 [ 295 ] {} <Information> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Created empty part -1_115_115_0_146 instead of lost part
    ...
    2021.08.18 20:11:28.731635 [ 257 ] {} <Trace> test_dpefxp.alter_table_6 (ReplicatedMergeTreeCleanupThread): Cleared 190 old blocks from ZooKeeper
    ...
    2021.08.18 20:11:28.734507 [ 257 ] {} <Trace> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Will try to insert a log entry to DROP_RANGE for part: -1_115_115_0_146
    2021.08.18 20:11:28.779373 [ 22837 ] {} <Debug> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Removed 1 parts inside -1_115_115_0_146.
    ...
    2021.08.18 20:11:28.792600 [ 273 ] {} <Trace> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Created log entry /clickhouse/tables/00993_system_parts_race_condition_drop_zookeeper_test_dpefxp/alter_table/log/log-0000003459 for merge -1_111_118_1
    ...
    2021.08.18 20:11:28.910988 [ 354 ] {} <Error> test_dpefxp.alter_table_7 (ReplicatedMergeTreeQueue): Code: 49. DB::Exception: Part -1_111_118_1 intersects next part -1_115_115_0_146. It is a bug. (LOGICAL_ERROR), Stack trace (when copying this message, always include the lines below):
    2021.08.18 20:11:31.282160 [ 305 ] {} <Error> test_dpefxp.alter_table_2 (ReplicatedMergeTreeQueue): Code: 49. DB::Exception: Part -1_111_118_1 intersects next part -1_115_115_0_146. It is a bug. (LOGICAL_ERROR), Stack trace (when copying this message, always include the lines below):
    ```

    </details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/27752/59e3cb18f4e53c453951267b5599afeb664290d8/functional_stateless_tests_(release,_wide_parts_enabled).html
 2021-08-30 21:05:46 +03:00
Nikolai Kochetov
ab45c412d0 Another try to fix BackgroundPoolTask decrement. 2021-08-30 15:45:32 +03:00
alesapin
531079c452
Merge pull request #28221 from azat/remove-Temporary-part-fix 
Fix removing of parts in a Temporary state
 2021-08-30 12:39:08 +03:00
Vladimir Chebotarev
de5daa22ab Revert "MATERIALIZE COLUMN xxx FINAL" 
This reverts commit bf1430c1ec.
 2021-08-30 11:50:26 +03:00
Vladimir Chebotarev
0d6e0181d6 MATERIALIZE COLUMN xxx FINAL 2021-08-30 11:49:59 +03:00
Vladimir Chebotarev
ce0850007f 1. Draw some circles. 2021-08-30 11:49:26 +03:00
alexey-milovidov
77f26e8673
Merge pull request #28299 from amosbird/fixtmprowsource 
Use real tmp file instead of predefined “rows_sources" for vertical merges
 2021-08-29 21:49:59 +03:00
Alexey Milovidov
e4f3d97b52 Merge branch 'master' of github.com:yandex/ClickHouse into async-reads 2021-08-29 21:42:14 +03:00
alexey-milovidov
3d3afbdb35
Merge pull request #28193 from amosbird/alterprimarykey2 
Reserve protocol number for ALTER PRIMARY KEY.
 2021-08-29 20:37:19 +03:00
Amos Bird
9e7ae5dd6d
Use real tmp file instead of predefined one 2021-08-29 16:54:15 +08:00
Amos Bird
4624bf70b0
Always monotonic for non-zero division 2021-08-28 23:33:18 +08:00
Amos Bird
f2374a6916
Better nullable primary key implementation. 2021-08-28 17:48:28 +08:00
Amos Bird
b68857d086
Simplify projection, add minmax_count projection. 2021-08-28 11:25:37 +08:00
Alexey Milovidov
79e0433ba7 Merge branch 'master' of github.com:yandex/ClickHouse into async-reads 2021-08-28 01:19:16 +03:00
Amos Bird
ce96b2c35a
bump 2021-08-27 20:44:26 +08:00
tavplubix
703101fe4d
Merge pull request #27931 from ClickHouse/wait_for_all_replicas_timeouts 
Avoid too long waiting for inactive replicas
 2021-08-27 14:31:36 +03:00
Azat Khuzhin
19458a0c61 Fix removing of parts in a Temporary state 
Parts in a temporary state is not exists in data_parts_by_info, so do
not try to search there and throw LOGICAL_ERROR in case of failure:

<details>

```
    02:45:49.037546 [ 5890 ] {} <Error> test_iy9rta.concurrent_kill_4 (526aa7c8-db2a-4f0e-926a-a7c8db2a9f0e): Code: 40. DB::Exception: Part all_0_0_0_1 from r11 has different columns hash. (CHECKSUM_DOESNT_MATCH) (version 21.10.1.7910 (official build)). Data after mutation is not byte-identical to data on another replicas. We will download merged part from replica to force byte-identical result.
    02:45:49.049422 [ 5890 ] {} <Trace> test_iy9rta.concurrent_kill_4 (526aa7c8-db2a-4f0e-926a-a7c8db2a9f0e): Trying to immediately remove part all_0_0_0_1 (state Temporary)
    02:45:49.060210 [ 5890 ] {} <Fatal> : Logical error: 'Part all_0_0_0_1 doesn't exist'.
    02:47:01.572508 [ 29208 ] {} <Fatal> BaseDaemon: (version 21.10.1.7910 (official build), build id: 9309CECED9A0D32CBB00BB8EC77B91456679868E) (from thread 5890) (no query) Received signal Aborted (6)
    ...
    02:47:06.815000 [ 29208 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeData.cpp:2593: DB::MergeTreeData::tryRemovePartImmediately(std::__1::shared_ptr<DB::IMergeTreeDataPart const>&&) @ 0x1f959e0d in /usr/bin/clickhouse
    02:47:06.816309 [ 29209 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeData.cpp:2593: DB::MergeTreeData::tryRemovePartImmediately(std::__1::shared_ptr<DB::IMergeTreeDataPart const>&&) @ 0x1f959e0d in /usr/bin/clickhouse
    02:47:09.455665 [ 29208 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1939: DB::StorageReplicatedMergeTree::tryExecutePartMutation(DB::ReplicatedMergeTreeLogEntry const&) @ 0x1f5f2bf6 in /usr/bin/clickhouse
    02:47:09.468738 [ 29209 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1939: DB::StorageReplicatedMergeTree::tryExecutePartMutation(DB::ReplicatedMergeTreeLogEntry const&) @ 0x1f5f2bf6 in /usr/bin/clickhouse
    02:47:11.776857 [ 29208 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1581: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x1f5e484c in /usr/bin/clickhouse
    02:47:11.904232 [ 29209 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1581: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x1f5e484c in /usr/bin/clickhouse
    02:47:13.941811 [ 29208 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:3176: DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18::operator() shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) const @ 0x1f65faa5 in /usr/bin/clickhouse
    02:47:14.477158 [ 29209 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:3176: DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18::operator() shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) const @ 0x1f65faa5 in /usr/bin/clickhouse
    02:47:16.475373 [ 29208 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676: decltype(std::__1::forward<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>))(std::__1::forward<std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(fp0))) std::__1::__invoke<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) @ 0x1f65fa32 in /usr/bin/clickhouse
    02:47:16.970325 [ 29209 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676: decltype(std::__1::forward<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>))(std::__1::forward<std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(fp0))) std::__1::__invoke<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) @ 0x1f65fa32 in /usr/bin/clickhouse
    02:47:18.979481 [ 29208 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:317: bool std::__1::__invoke_void_return_wrapper<bool>::__call<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplireeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogE x1f65f9f2 in /usr/bin/clickhouse
    02:47:19.450807 [ 29209 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:317: bool std::__1::__invoke_void_return_wrapper<bool>::__call<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplireeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogE x1f65f9f2 in /usr/bin/clickhouse
    02:47:21.055007 [ 29208 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608: std::__1::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::ry>)::$_18, bool (std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&)>::operator()(std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) @ 0x1f65f9b0 in /usr/bin/clickhouse
    02:47:23.546946 [ 413 ] {} <Fatal> Application: Child process was terminated by signal 6.
```

</details>

CI: https://clickhouse-test-reports.s3.yandex.net/0/4a8b82232c11512232df3ecdf4ffaec287116ad5/stress_test_(debug).html#fail1
 2021-08-27 10:08:44 +03:00
Alexey Milovidov
46732e781f Merge branch 'master' of github.com:yandex/ClickHouse into async-reads 2021-08-27 01:57:48 +03:00
Mike Kot
1f3701d485 Disk empty name fix 2021-08-26 19:28:33 +02:00
Amos Bird
103e0abe1c
Reserve protocol number for ALTER PRIMARY KEY. 2021-08-26 22:08:58 +08:00
Amos Bird
0169fce78e
Projection bug fixes and refactoring. 2021-08-26 19:09:31 +08:00
Ilya Yatsishin
5553a064be
Merge pull request #28139 from lthaooo/fix_prewhere_total_size_of_queried_columns_bug 
Fix bug in MergeTreeWhereOptimizer
 2021-08-26 10:42:23 +03:00
Anton Popov
6d3274c22c Merge remote-tracking branch 'upstream/master' into HEAD 2021-08-26 03:31:46 +03:00
Alexey Milovidov
7dd0146223 Fix error 2021-08-26 01:56:08 +03:00
Alexey Milovidov
3cb9cf7242 Merge branch 'master' of github.com:yandex/ClickHouse into async-reads 2021-08-26 01:55:46 +03:00
Maksim Kita
2403686bb9
Merge pull request #28135 from amosbird/projection-improve3 
Get rid of useless projection columns during merge
 2021-08-25 22:14:38 +03:00
Mike Kot
407af64280 Merge remote-tracking branch 'upstream/master' into improvement/merge-tree-part-opt-parse 2021-08-25 18:24:43 +02:00
lthaooo
eecd5b21c9 fix total_size_of_queried_columns bug 2021-08-25 23:02:13 +08:00
Amos Bird
a6b2a6ee36
Get rid of useless projection columns during merge 2021-08-25 20:54:27 +08:00
lthaooo
314ba1e682
Merge branch 'master' into materialize_ttl_recalculate_only 2021-08-25 10:56:24 +08:00
Alexey Milovidov
848c2ec23e Tuneable prefetch 2021-08-25 02:38:08 +03:00
Alexey Milovidov
fd27a2faef Remove imperfect code 2021-08-25 02:05:55 +03:00
Alexey Milovidov
7c1d0a3baf Progress on development 2021-08-25 01:24:47 +03:00
Alexey Milovidov
f7ca674b0c Progress on development 2021-08-25 01:07:06 +03:00
Alexey Milovidov
8f57216180 Progress on development 2021-08-25 00:45:58 +03:00
Alexey Milovidov
4d9ad3725d Merge branch 'master' into async-reads 2021-08-25 00:45:40 +03:00
Mike Kot
d621605954 Merge remote-tracking branch 'upstream/master' into improvement/merge-tree-part-opt-parse 2021-08-24 19:43:33 +02:00
Mike Kot
f85e2e027c Optional semantics for [Detached]MergeTreePartInfo 2021-08-24 19:43:09 +02:00
Vitaly Baranov
64f69462c6
Merge pull request #27921 from vitlibar/add-tests-for-field-init 
Allow implicit cast bool to Field
 2021-08-24 17:15:44 +03:00
mergify[bot]
ae22bab2c4
Merge branch 'master' into materialize_ttl_recalculate_only 2021-08-24 08:49:19 +00:00
Vitaly Baranov
63e8bc1f20 Added new commands BACKUP and RESTORE. 2021-08-23 23:41:45 +03:00
Anton Popov
0db8b524f0
Merge pull request #27984 from BraulioVM/optimize/has-all-bloom-filter 
Make `hasAll` filter condition leverage bloom filters
 2021-08-23 22:19:50 +03:00
Alexander Tokmakov
4a4a0b4823 fix 2021-08-23 16:32:07 +03:00
Alexander Tokmakov
cc9c2fd63b make code better 2021-08-23 15:57:50 +03:00
Yatsishin Ilya
17bb938541 fix style 2021-08-23 13:59:01 +03:00
Amos Bird
a2256b1307
Dedicated Mark/Uncompressed cache for skip indices 2021-08-23 15:27:31 +08:00
Vitaly Baranov
89166a655a Init SettingFieldBool with true or false, not with 1 or 0. 2021-08-23 09:47:36 +03:00
Vitaly Baranov
ff1fad8521 Use setSetting() with implicit conversion true/false -> Field. 2021-08-23 08:57:00 +03:00
Braulio Valdivielso
d48f6122e3 clean up implementation a bit 2021-08-22 10:06:30 +01:00
Braulio Valdivielso
e95563fb82 leverage bloom filter index for 'hasAll' 
Fixes #26151. For now, this is a draft implementation. There is some
cleanup that can be done, but the new test is passing
 2021-08-21 20:14:56 +01:00
Alexander Tokmakov
59eb3aa9a9 avoid too long waiting for inactive replicas 2021-08-20 15:59:57 +03:00
nvartolomei
c09c90125f
Merge branch 'master' into nv/last-queue-update-exception 2021-08-19 10:29:16 +01:00
Nikolai Kochetov
7dcff5c90e
Merge pull request #27742 from amosbird/projection-improvement3 
Improve projection analysis.
 2021-08-19 12:11:20 +03:00
alesapin
0aec151719
Merge pull request #27808 from ClickHouse/fix_replicas_may_diverge 
Fix a couple of bugs that may cause replicas to diverge
 2021-08-19 09:58:07 +03:00
Alexander Tokmakov
09ff66da0e fix a couple of bugs that may cause replicas to diverge 2021-08-18 12:50:46 +03:00
alesapin
5cc98c67a4 Fix 2021-08-18 11:56:18 +03:00
mergify[bot]
f11e396151
Merge branch 'master' into nv/last-queue-update-exception 2021-08-18 07:00:50 +00:00
Nikita Mikhaylov
cc8ad0a4e7
Merge pull request #27733 from Algunenano/i27620_take2 
Refactor arrayJoin check on partition expressions
 2021-08-17 15:57:16 +03:00
Nikolai Kochetov
98eb619b4b
Merge pull request #23367 from Avogar/short-circuit 
Implement short circuit function evaluation
 2021-08-17 12:36:07 +03:00
Amos Bird
ea7dc495cf
Better code. 2021-08-17 12:32:49 +08:00
Raúl Marín
90f224807a Style 2021-08-16 17:31:37 +02:00
Amos Bird
90881aab09
Better code style 2021-08-16 22:17:07 +08:00
Raúl Marín
3ec904a953 Only analyze if tuple has arguments 2021-08-16 14:47:41 +02:00
Amos Bird
b162a2b699
Improve projection analysis. 
Remove duplicate index analysis and avoid possible invalid limit checks
during projection analysis.
 2021-08-16 20:11:57 +08:00
Raúl Marín
34099bc196 Refactor arrayJoin check on partition expressions 2021-08-16 12:46:18 +02:00
Alexey Milovidov
d184b79bba Progress on async reads. 2021-08-16 03:00:32 +03:00
alexey-milovidov
4a36366925
Merge pull request #27648 from Algunenano/i27620 
Disable arrayJoin on partition expressions
 2021-08-16 02:53:36 +03:00
Alexey Milovidov
0cd3f25ba4 Merge remote-tracking branch 'origin/master' into async-reads 2021-08-16 02:41:01 +03:00
alexey-milovidov
76b0502482
Merge pull request #27690 from kitaisreal/remove-dense-hash-map-set 
Removed DenseHashMap, DenseHashSet
 2021-08-16 02:39:41 +03:00
Maksim Kita
5f5470c2cd Removed DenseHashMap, DenseHashSet 2021-08-15 13:32:56 +03:00
Alexey Milovidov
020beb885d Merge branch 'master' into async-reads 2021-08-15 12:08:19 +03:00
mergify[bot]
bf426cfb96
Merge branch 'master' into fix_intersecting_parts 2021-08-15 09:07:20 +00:00
mergify[bot]
353a324887
Merge branch 'master' into i27620 2021-08-15 02:32:26 +00:00
alexey-milovidov
ab4415a22b
Update MergeTreeData.cpp 2021-08-15 05:28:38 +03:00
Alexander Tokmakov
c534363abe fix intersecting parts 2021-08-13 20:33:10 +03:00
Raúl Marín
73cb7d55ec Disable arrayJoin on partition expressions 2021-08-13 16:18:46 +02:00
mergify[bot]
319e5af6eb
Merge branch 'master' into short-circuit 2021-08-13 13:01:46 +00:00
Nikolai Kochetov
66fdb859f3
Merge pull request #27528 from amosbird/projection-fix12 
Fix projection materialization with missing columns
 2021-08-13 15:20:51 +03:00
Kseniia Sumarokova
0917488cec
Merge pull request #27382 from kssenii/cast-internal 
add Cast internal function
 2021-08-13 13:13:51 +03:00
Nikolai Kochetov
5e5a6ff654 Merge branch 'master' into qoega-fix-access-gtest-in-arcadia 2021-08-12 12:02:15 +03:00
mergify[bot]
80eaea1c51
Merge branch 'master' into short-circuit 2021-08-12 08:38:45 +00:00
kssenii
dcc6f597dc Move templates from FunctionsConvertion 2021-08-11 19:09:51 +00:00
kssenii
7555fc19fd Revert "Move some templates" 
This reverts commit 7f247becca.
 2021-08-11 18:49:25 +00:00
mergify[bot]
68a52fc065
Merge branch 'master' into projection-fix12 2021-08-11 17:08:04 +00:00
Alexander Kuzmenkov
8f9845722a
Merge pull request #27250 from azat/minmax-nullable-new-format 
Add new index data skipping minmax index format for proper Nullable support
 2021-08-11 13:56:16 +03:00
kssenii
7d8b694a9f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into cast-internal 2021-08-11 07:37:21 +00:00
kssenii
7f247becca Move some templates 2021-08-11 07:32:46 +00:00
Alexey Milovidov
08f8511854 Fix Style 2021-08-11 06:52:28 +03:00
alexey-milovidov
54d8282209
Merge pull request #27248 from azat/mt-mutation-stuck-fix 
Fix mutation stuck on invalid partitions in non-replicated MergeTree
 2021-08-11 06:10:15 +03:00
alexey-milovidov
a0a5c0da32
Update BackgroundJobsExecutor.cpp 2021-08-11 06:08:30 +03:00
Amos Bird
e63c26edb7
Fix projection materialization with missing columns 2021-08-10 21:47:27 +08:00
kssenii
eb4746ac69 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into cast-internal 2021-08-10 09:38:41 +00:00
Nikolai Kochetov
a1ec7f75c5 Merge branch 'master' into qoega-fix-access-gtest-in-arcadia 2021-08-10 11:31:47 +03:00
Nikolai Kochetov
bb4c11cd27
Merge pull request #27298 from ClickHouse/fix-23515 
Fix bug from #23515.
 2021-08-09 23:25:30 +03:00
Nikolai Kochetov
8cc493a3cd Try fix build. 2021-08-09 18:09:29 +03:00
Pavel Kruglov
70b51133c1 Try to simplify code 2021-08-09 18:01:08 +03:00
Pavel Kruglov
0662df8b76 Fix performance with JIT, add arguments to function isSuitableForShortCircuitArgumentsExecution 2021-08-09 17:54:14 +03:00
Pavel Kruglov
e792fa588f Mark all Functions as sutable or not for executing as short circuit arguments 2021-08-09 17:50:09 +03:00
Nicolae Vartolomei
3f291b024a
Use plain mutex instead of MultiVersion 2021-08-09 13:58:23 +01:00
Nikolai Kochetov
4a00e402ae Fix spelling. 2021-08-09 12:09:09 +03:00
Nikolai Kochetov
372293b85d
Merge branch 'master' into fix-27179 2021-08-09 11:59:22 +03:00
Alexey Milovidov
8b9da6fe26 Merge branch 'master' into async-reads 2021-08-09 03:33:32 +03:00
kssenii
7991bb6e83 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into cast-internal 2021-08-08 21:30:17 +00:00
Azat Khuzhin
038241b6ed Add new index data skipping minmax index format for proper Nullable support 
Note, that it cannot be done w/o new extension, since index does not
have any header.

v2: use IDisk interface for existence check
v3: remove extra file existence check
v4: fix MATERIALIZE INDEX
 2021-08-08 19:30:12 +03:00
Azat Khuzhin
5139067631 Guard BackgroundJobsExecutor from thread termination in case of uncaught exception 2021-08-08 19:27:58 +03:00
alexey-milovidov
ba3fbbaab0
Update ReplicatedMergeTreeMergeStrategyPicker.cpp 2021-08-08 04:43:59 +03:00
mergify[bot]
9869c8bb10
Merge branch 'master' into zero_copy_by_default 2021-08-07 16:53:44 +00:00
kssenii
9ca422f0c5 Introduce CAST for internal usage 2021-08-07 09:03:10 +00:00
Alexander Tokmakov
5e2bfd5ba1 fix partition id validation 2021-08-06 23:56:51 +03:00
Nikolai Kochetov
3cbeab8959 Fix incorrect row-level filtering 2021-08-06 21:16:06 +03:00
Nikolai Kochetov
7320411a7f Fix bug from #23515. 2021-08-06 16:39:11 +03:00
Anton Popov
3a0d4807a5 Merge branch 'async-insert' of git://github.com/abyss7/ClickHouse into merging-20557 2021-08-06 16:20:02 +03:00
Alexey Milovidov
cea9690327 Merge branch 'master' into async-reads 2021-08-05 23:58:57 +03:00
alexey-milovidov
1a3d8cebd2
Merge pull request #27197 from azat/nullable-index-fix 
Fix on-disk format breakage for secondary indices over Nullable column
 2021-08-05 21:52:18 +03:00
Kruglov Pavel
59a94bd322
Merge pull request #27183 from ClickHouse/fix_detached_parts_name_partsing 
Fix part name parsing in system.detached_parts
 2021-08-05 18:21:03 +03:00
Alexander Kuzmenkov
d852207b0e
Merge pull request #25721 from CurtizJ/read-in-order-perf-2 
Reduce memory usage in queries with `ORDER BY primary_key
 2021-08-05 16:13:48 +03:00
terrylin
dd539f0a0d improvement of materilize ttl 2021-08-05 15:33:02 +08:00
terrylin
bd3d9a4518 materialize ttl recalculate only (optional) 2021-08-05 14:17:48 +08:00
Azat Khuzhin
dee27fcbb9 Fix on-disk format breakage for secondary indices over Nullable column 
[1] breaks on disk format (and the relevant change in the:

  [1]: https://github.com/ClickHouse/ClickHouse/pull/12455#discussion_r682830812

Too bad that I checked this patchset only for compatibility after
reverting this patch [2] (use case: I've applied it manually, then
revert it, and data skipping indexes over Nullable column had been
broken)

  [2]: https://github.com/ClickHouse/ClickHouse/pull/12455#issuecomment-823423772

But this patchset actually breaks compatibility with older versions of
clickhouse for Nullable data skipping indexes after simple upgrade:

Here is a simple reproducer:

    --
    -- run this with 21.6 or similar (i.e. w/o this patch)
    --

    CREATE TABLE data
    (
        `key` Int,
        `value` Nullable(Int),
        INDEX value_index value TYPE minmax GRANULARITY 1
    )
    ENGINE = MergeTree
    ORDER BY key;

    INSERT INTO data SELECT
        number,
        number
    FROM numbers(10000);

    SELECT * FROM data WHERE value = 20000 SETTINGS force_data_skipping_indices = 'value_index' SETTINGS force_data_skipping_indices = 'value_index', max_rows_to_read=1;

Now upgrade and run the query again:

    SELECT * FROM data WHERE value = 20000 SETTINGS force_data_skipping_indices = 'value_index' SETTINGS force_data_skipping_indices = 'value_index', max_rows_to_read=1;

And it will fail because of on disk format changes:

    $ ll --time-style=+ data/*/data/all_1_1_0/skp*.idx
    -rw-r----- 1 azat azat 36  data/with_nullable_patch/data/all_1_1_0/skp_idx_value_index.idx
    -rw-r----- 1 azat azat 37  data/without_nullable_patch/data/all_1_1_0/skp_idx_value_index.idx

    $ md5sum data/*/data/all_1_1_0/skp*.idx
    a19c95c4a14506c65665a1e30ab404bf  data/with_nullable_patch/data/all_1_1_0/skp_idx_value_index.idx
    e50e2fcfa873b232196623d56ab26105  data/without_nullable_patch/data/all_1_1_0/skp_idx_value_index.idx

Note, that there is no stable release with this patch included yet, so
no need to backport.

Also note that you may create data skipping indexes over Nullable
column even before [3].

  [3]: https://github.com/ClickHouse/ClickHouse/pull/12455

v2: break cases when granulas has Null in values due to backward
compatibility
 2021-08-05 00:19:38 +03:00
Anton Popov
9586bb7e90 fix pvs 2021-08-04 20:42:22 +03:00
terrylin
6f42ec6b9b fix compile error 2021-08-05 00:16:21 +08:00
Anton Popov
915ef50c2d optimize reading in order of key more 2021-08-04 18:18:03 +03:00
Alexander Tokmakov
23f8b3d07d fix part name parsing in system.detached_parts 2021-08-04 17:42:48 +03:00
terrylin
88b9e7f24f materialize ttl recalculate only optional 2021-08-04 22:16:13 +08:00
Anton Ivashkin
61016da2b1 Set allow_remote_fs_zero_copy_replication to true by default 2021-08-04 17:08:09 +03:00
Yatsishin Ilya
55e06ea16b fix 2021-08-04 13:06:01 +03:00
Alexey Milovidov
fd3b5680e2 Merge branch 'master' into async-reads 2021-08-03 23:13:31 +03:00
Anton Popov
fc9d72e75d fix performance of short queries with large number of columns 2021-08-02 15:03:55 +03:00
Nikolai Kochetov
6951e8147d
Merge pull request #27020 from amosbird/shardlevelconstness 
Shard-level const column
 2021-08-02 13:45:20 +03:00
Anton Popov
cc3ed12ed6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-08-02 02:56:00 +03:00
Alexey Milovidov
2a0cba2b9f Merge branch 'master' into async-reads 2021-08-01 01:25:13 +03:00
alexey-milovidov
701e2ffd79
Merge pull request #27002 from azat/mutation-stuck 
Fix possible mutation stuck due to race with DROP_RANGE
 2021-07-31 21:17:46 +03:00
Amos Bird
cd302eacc1
Fix projection match 2021-07-31 15:44:16 +08:00
tavplubix
00ac8194dc
Merge pull request #26896 from ClickHouse/small_improvement_in_queue 
Maybe fix extremely rare `intersecting parts`.
 2021-07-30 13:04:20 +03:00
terrylin
c49c2e5e18 fix compile error 2021-07-30 17:15:04 +08:00
mergify[bot]
41273ef5f3
Merge branch 'master' into mutation-stuck 2021-07-30 08:42:45 +00:00
Azat Khuzhin
bc9a4366d8 Fix possible mutation stack due to race with DROP_RANGE 
After #25884 parts that was dropped (DROP_RANGE) will not be removed
from old mutations and this will stuck the mutation.

Interesting, that this mutation may continue after some merge.
 2021-07-30 08:22:10 +03:00
terrylin
55f3e7e65a improvement of materilize ttl 2021-07-30 12:27:15 +08:00
alesapin
21599ffefc Simple validation for partition id before drop partition 2021-07-29 16:11:05 +03:00
alesapin
5799487caa Fix stupid bug 2021-07-29 14:54:36 +03:00
alesapin
bdc6858801 More correct comparsion 2021-07-28 13:19:34 +03:00
alesapin
26056cfb25 Fix extremely rare race condition on remove of unsuccessful entry from queue 2021-07-28 13:06:22 +03:00
Alexey Milovidov
9baf84224a Fix build 2021-07-28 02:54:20 +03:00
Alexey Milovidov
fd5c1561e2 Merge branch 'master' into async-reads 2021-07-27 23:58:32 +03:00
Nicolae Vartolomei
8b07a7f180 Store exception generated when we tried to update the queue last time 
The use case is to alert when queue contains broken entries. Especially
important when ClickHouse breaks backwards compatibility between
versions and log entries written by newer versions aren't parseable by
old versions.

```
Code: 27, e.displayText() = DB::Exception: Cannot parse input: expected 'quorum: ' before: 'merge_type: 2\n'
```
 2021-07-27 15:42:40 +01:00
tavplubix
168edaed73
Merge pull request #26808 from amosbird/safe2 
Make sure table is readonly when restarting fails.
 2021-07-27 17:21:08 +03:00
Nikolai Kochetov
97bc754dd8
Merge pull request #26758 from ClickHouse/output-streams-to-processors 
Remove some output streams
 2021-07-27 17:19:26 +03:00
alexey-milovidov
d7b2404119
Merge pull request #15408 from ClickHouse/random_timezone_for_stateless_tests 
Use random timezone in stateless and stateful tests
 2021-07-27 16:18:14 +03:00
Alexey Milovidov
5e2ea4e042 Whitespace 2021-07-27 05:20:03 +03:00
Nikolai Kochetov
61d8f880cd Rename some files. 2021-07-26 19:48:25 +03:00
Nikolai Kochetov
9b5a816b43 Merge branch 'master' into output-streams-to-processors 2021-07-26 18:03:11 +03:00
Nikolai Kochetov
0eb563dc1b Fix more tests. 2021-07-26 17:47:29 +03:00
Amos Bird
623faf47e4
Make sure table is readonly when restarting fails. 2021-07-26 19:21:14 +08:00
Nikolai Kochetov
fa1c223269 Fix some tests. 2021-07-26 13:08:40 +03:00
Alexey Milovidov
5c3f5cb35a Experiment with asynchronous readers 2021-07-26 03:34:36 +03:00
Nikolai Kochetov
9c92f43359 Update storages. 2021-07-23 22:33:59 +03:00
Azat Khuzhin
00e2083421 Fix event_time_microseconds for REMOVE_PART in system.part_log 2021-07-23 00:59:08 +03:00
alexey-milovidov
fcca3e3e56
Merge pull request #26423 from ClickHouse/fix-header-for-scalar-query-with-empty-result 
Fix incompatible result type for scalar queries with empty result.
 2021-07-21 02:07:04 +03:00
Anton Popov
fa6857920d fix arcadia 2021-07-19 16:45:35 +03:00
Nikolai Kochetov
7a24e72e76 Merge branch 'master' into fix-header-for-scalar-query-with-empty-result 2021-07-19 15:48:44 +03:00
Nikolai Kochetov
96e20e2641 Fix some tests. 2021-07-19 15:35:55 +03:00
Kseniia Sumarokova
b0ddc4fb30
Merge pull request #26364 from kssenii/fixing-date32 
Continuation of #25774
 2021-07-18 21:44:45 +03:00
alexey-milovidov
b52411a715
Merge pull request #12455 from amosbird/npc 
Nullable primary key with correct KeyCondition
 2021-07-18 17:52:20 +03:00
Alexey Milovidov
261a220227 Remove some code 2021-07-17 21:06:46 +03:00
Azat Khuzhin
071dc7c3f5 Improve "Not executing log entry queue-X of type MUTATE_PART" message 2021-07-17 11:50:04 +03:00
Alexey Milovidov
bd924a5b8a Whitespace 2021-07-17 01:59:39 +03:00
alexey-milovidov
c373b86743
Merge pull request #26371 from CurtizJ/improve-perf-get-columns 
Improve performance of getting list of columns
 2021-07-17 01:34:15 +03:00
Anton Popov
048e089a58 remove useless code 2021-07-16 17:47:03 +03:00
Anton Popov
03c785931a fix pushdown of limit to reading stage 2021-07-16 17:27:38 +03:00
Anton Popov
c45a9e3bd6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-07-16 17:11:34 +03:00
Anton Popov
9e3af27f40 better performance of getSampleBlockForColumns 2021-07-15 20:36:48 +03:00
Anton Popov
5f71a6f5bb improve performance of getting columns list 2021-07-15 16:29:01 +03:00
kssenii
3ba2d114d5 Revert "Merge pull request #26352 from ClickHouse/revert-25774-datatype-date32" 
This reverts commit 94a210650f, reversing
changes made to 0893b9ff8e.
 2021-07-15 11:41:52 +00:00
alexey-milovidov
9ba4064c86
Revert "Datatype Date32, support range 1925 to 2283" 2021-07-15 10:53:35 +03:00
alexey-milovidov
0893b9ff8e
Merge pull request #26218 from amosbird/projection-improve1 
Aggressive IN index analysis for projections.
 2021-07-15 08:13:45 +03:00
Filatenkov Artur
f1702d356e
Merge pull request #26120 from lthaooo/settings-merge_selecting_sleep_ms 
add merge_selecting_sleep_ms setting
 2021-07-14 18:28:47 +03:00
Amos Bird
28b567a01b
Update 2021-07-14 23:09:17 +08:00
alexey-milovidov
55fc7323fd
Update MergeTreeDataSelectExecutor.h 2021-07-14 22:56:52 +08:00
alexey-milovidov
1ef068596c
Update MergeTreeDataSelectExecutor.cpp 2021-07-14 22:56:52 +08:00
Amos Bird
0558ecdc3f
Aggressive IN index analysis for projections. 2021-07-14 22:56:52 +08:00
Anton Popov
e2249bf1e5
Merge pull request #26256 from zxc111/issue-822 
Check hash function at table creation
 2021-07-14 14:16:02 +03:00
Zhichang Yu
fe5d17680e Merge remote-tracking branch 'origin/master' into zero_copy_hdfs 2021-07-14 02:12:28 +00:00
Maksim Kita
3d05f07ece
Merge pull request #25917 from excitoon-favorites/defaultblocksizeprediction 
Using required columns for block size prediction
 2021-07-13 20:05:41 +03:00
Vitaly Baranov
3c395389b0
Merge pull request #24206 from alexelex/alexelex-encryption-1 
Data encryption on VFS level
 2021-07-13 19:50:25 +03:00
Anton Popov
cb042afa9d pushdown limit while reading in order of primary key 2021-07-13 17:24:45 +03:00
zxc111
7b2fed2951 use flag to check sample_condition 2021-07-13 21:17:22 +08:00
Anton Popov
19d0f1bbfc Merge remote-tracking branch 'upstream/master' into HEAD 2021-07-13 16:12:08 +03:00
Ivan Lezhankin
ab26aed6f9 Merge remote-tracking branch 'upstream/master' into async-insert 2021-07-13 13:54:09 +03:00
xiedeyantu
df438f4288 Modify code comments 2021-07-13 17:52:24 +08:00
Vitaly Baranov
e086e5e384
Merge branch 'master' into alexelex-encryption-1 2021-07-13 11:28:41 +03:00
Zhichang Yu
b4e6689bf9 fix test_hdfs_zero_copy_replication_move[tiered_copy-2] 2021-07-13 07:20:23 +00:00
Zhichang Yu
5047c758f4 fix per review 2021-07-13 07:20:20 +00:00
Zhichang Yu
fbd5eee8a1 hdfs zero copy 2021-07-13 07:19:12 +00:00
alexey-milovidov
4b1e38eed9
Merge branch 'master' into projection-fix11 2021-07-13 04:52:15 +03:00
alexey-milovidov
5295f7ebb2
Merge pull request #26235 from ClickHouse/check-non-deterministic-functions-in-keys 
Check for non-deterministic functions in keys, including constant expressions
 2021-07-13 02:46:44 +03:00
Kseniia Sumarokova
4a6b4d6c26
Merge pull request #25774 from liuneng1994/datatype-date32 
Datatype Date32, support range 1925 to 2283
 2021-07-12 20:33:39 +03:00
zxc111
e0261866a4 add settings && add test case 2021-07-13 01:15:35 +08:00
mergify[bot]
b7ae4e4ced
Merge branch 'master' into async-insert 2021-07-12 09:21:37 +00:00
Alexey Milovidov
2a2aa69459 Handle non-existent "detached" directories 2021-07-12 10:01:26 +03:00
Alexey Milovidov
ff5c433f10 Check for non-deterministic functions in keys, including constant expressions 2021-07-12 08:58:19 +03:00
alexey-milovidov
8f1d7e67ca
Merge pull request #26003 from ClickHouse/ditch-aio 
Remove AIO for sequential reads
 2021-07-12 03:25:23 +03:00
zxc111
cd97478c06 check sampling condition when create table 2021-07-12 00:38:20 +08:00
Alexey Milovidov
1ce40d6629 Fix bad code (default function argument) 2021-07-11 04:01:23 +03:00
Alexey Milovidov
81646f8389 Merge branch 'master' into ditch-aio 2021-07-11 03:06:45 +03:00
terrylin
faed6263bb add merge_selecting_sleep_ms setting 2021-07-09 17:29:17 +08:00
Anton Popov
9f1ffa777f remove unused code 2021-07-08 15:06:33 +03:00
alesapin
4c85dae572
Merge pull request #25743 from ClickHouse/fix_aggregation_ttl 
Fix bug in execution of TTL GROUP BY
 2021-07-07 10:49:16 +03:00
alesapin
0d8844c510
Merge pull request #25884 from ClickHouse/fix_drop_part_in_queue 
Relax `DROP PART` guarantees and turn on checks in ReplicationQueue.
 2021-07-07 10:48:48 +03:00
mergify[bot]
446474932b
Merge branch 'master' into projection-fix11 2021-07-07 05:02:44 +00:00
Amos Bird
55981cb0ae
Fix normal projection 2021-07-07 13:01:30 +08:00
alesapin
3dee74df54 Comment 2021-07-06 19:53:54 +03:00
alesapin
a8fdc41193 Fix bug and add more trash to test 2021-07-06 19:51:23 +03:00
mergify[bot]
7ff8ac51a9
Merge branch 'master' into fix-merge-tree-settings-change 2021-07-06 12:14:33 +00:00
alesapin
75e26b93d0 Review bug fixes 2021-07-06 15:05:58 +03:00
alesapin
e9540f0621 Remove debug logs 2021-07-06 14:13:35 +03:00
mergify[bot]
8dd378f7c2
Merge branch 'master' into fix_drop_part_in_queue 2021-07-06 11:00:04 +00:00
alesapin
53b23775a9 Fix drop part 2021-07-06 13:58:53 +03:00
alesapin
2600922786 Followup fix 2021-07-06 11:36:39 +03:00
Alexey Milovidov
21fb6ddea5 Remove AIO for sequential reads 2021-07-05 23:02:24 +03:00
alesapin
f7e1cfdb24 Some partially working code 2021-07-05 22:58:55 +03:00
Anton Popov
9071ecd428 fix alter of settings in MergeTree 2021-07-05 15:44:58 +03:00
NengLiu
8b88988618 Merge branch 'master' into datatype-date32 
# Conflicts:
#	src/DataStreams/PostgreSQLBlockInputStream.cpp
 2021-07-05 10:18:38 +08:00
NengLiu
dc0c65ef4f Merge remote-tracking branch 'origin/master' into datatype-date32 2021-07-05 10:15:08 +08:00
alexey-milovidov
fef7c557f6
Merge pull request #24789 from kitaisreal/compile-aggregation-functions 
Compile aggregate functions
 2021-07-05 02:57:49 +03:00
mergify[bot]
1559c09977
Merge branch 'master' into fix_drop_part_in_queue 2021-07-04 16:00:39 +00:00
alesapin
cf2fc94d9d
Merge pull request #25820 from ClickHouse/better_remove_empty_parts 
Replace lost parts with empty parts instead of hacking replication queue
 2021-07-04 18:55:49 +03:00
Vladimir Chebotarev
e5cbd82879 Took PREWHERE into account. 2021-07-04 18:06:15 +03:00
alexey-milovidov
51bf60a938
Merge pull request #17769 from xjewer/alex/remove_engine_setting 
CLICKHOUSE-784: reset merge tree setting
 2021-07-03 18:27:15 +03:00
alexey-milovidov
1dda771864
Merge pull request #25554 from CurtizJ/fix-alter-ttl-column 
Fix `ALTER MODIFY COLUMN` of columns, which participates in TTL expressions
 2021-07-03 18:26:37 +03:00
mergify[bot]
9da1c98998
Merge branch 'master' into fix_aggregation_ttl 2021-07-03 15:07:44 +00:00
alesapin
7b4a56977d
Merge pull request #25893 from ClickHouse/no_backoff_when_pool_is_full 
Reset counter if background pool is full
 2021-07-03 18:03:29 +03:00
mergify[bot]
fbe1ff0fb4
Merge branch 'master' into fix_drop_part_in_queue 2021-07-03 15:02:15 +00:00
mergify[bot]
a2183dff39
Merge branch 'master' into better_remove_empty_parts 2021-07-03 15:01:11 +00:00
mergify[bot]
02cb287dc9
Merge branch 'master' into fix-alter-ttl-column 2021-07-02 23:49:46 +00:00
alesapin
b4b54a0549 Fix bug 2021-07-02 18:25:05 +03:00
Vladimir Chebotarev
6f5a72c963 Using required columns for block size prediction. 2021-07-02 16:55:00 +03:00
alesapin
72b84141e1 Merge branch 'better_remove_empty_parts' into fix_drop_part_in_queue 2021-07-02 12:41:28 +03:00
alesapin
f8f734d3dc Better comment 2021-07-02 12:40:13 +03:00
alesapin
abc3aca19b Merge branch 'better_remove_empty_parts' into fix_drop_part_in_queue 2021-07-02 12:30:54 +03:00
alesapin
2e29dc2975 More safe empty parts creation 2021-07-02 12:30:17 +03:00
alesapin
5e79c68e8e Debug 2021-07-02 11:54:22 +03:00
Alexey Milovidov
0e621788c7 Allow constexpr parameters for aggregate functions 2021-07-02 03:53:08 +03:00
Maksim Kita
da8c957167 Aggregator added CompiledExpressionCache 2021-07-01 22:56:36 +03:00
Maksim Kita
d24d3ae992 Added second variant of compilation 2021-07-01 22:56:36 +03:00
Maksim Kita
3fe559b31f Compile aggregate functions 2021-07-01 22:56:35 +03:00
Aleksei Semiglazov
9a5365fc41
CLICKHOUSE-784: reset merge tree setting 
Add an ability to reset custom setting to default and remove it from table's metadata.
This will allow to rollback the change without knowing the system/config's default.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>
 2021-07-01 20:09:54 +01:00
alesapin
7792b09911 Reset counter if background pool is full 2021-07-01 16:10:07 +03:00
alesapin
1a4ccab8e6 Fix style 2021-07-01 15:12:27 +03:00
mergify[bot]
98fa9f7951
Merge branch 'master' into better_remove_empty_parts 2021-07-01 11:32:43 +00:00
tavplubix
afbc6bf17d
Merge pull request #25684 from ClickHouse/cancel_merges_on_drop_partition 
Cancel merges on DROP PARTITION
 2021-06-30 23:41:04 +03:00
alesapin
c845a205f9 Merge branch 'better_remove_empty_parts' into fix_drop_part_in_queue 2021-06-30 22:42:02 +03:00
alesapin
45edc8c861 Fix tidy 2021-06-30 22:11:00 +03:00
alesapin
89a401a84f Merge branch 'better_remove_empty_parts' into fix_drop_part_in_queue 2021-06-30 18:53:17 +03:00
alesapin
6a73c8b49e Review fixes 2021-06-30 18:24:51 +03:00
tavplubix
c958f876b8
Update MergeList.h 2021-06-30 15:55:26 +03:00
alesapin
21f15be990 Merge branch 'better_remove_empty_parts' into fix_drop_part_in_queue 2021-06-30 15:30:32 +03:00
alesapin
2eb27540b2 Some test version 2021-06-30 15:29:09 +03:00
alesapin
94d4d32e4c
Merge pull request #25783 from ClickHouse/followup_drop_part_fix 
Relax check during DROP PART
 2021-06-30 12:25:42 +03:00
mergify[bot]
1799804243
Merge branch 'master' into fix_aggregation_ttl 2021-06-30 09:15:46 +00:00
mergify[bot]
8182b6e890
Merge branch 'master' into better_remove_empty_parts 2021-06-30 08:42:26 +00:00
alesapin
0af53ace55 Fix comment 2021-06-29 22:49:41 +03:00
alesapin
ca1d21b663
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: tavplubix <avtokmakov@yandex-team.ru>
 2021-06-29 22:48:46 +03:00
alesapin
a6834213a1 Add tests 2021-06-29 22:47:54 +03:00
Nikolai Kochetov
d5bd2b1fa9
Merge pull request #25719 from ClickHouse/remove-PrewhereDAGInfo 
Remove PrewhereDAGInfo.
 2021-06-29 18:58:03 +03:00
alesapin
5822f0ba29 Replace lost parts with empty parts instead of hacking replication queue 2021-06-29 18:14:44 +03:00