Commit Graph

204 Commits

Author SHA1 Message Date
Azat Khuzhin
6cb2e81d48 Do not try convert columns that does not exists in the result block 2021-06-03 21:07:48 +03:00
Azat Khuzhin
01e6a4d919 Do not try convert columns that does not exists in materialized view 2021-06-03 21:07:48 +03:00
kssenii
12ea45e956 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-01 17:43:34 +00:00
tavplubix
3dd7d252a9
Merge pull request #24806 from ClickHouse/tavplubix-patch-2
Small improvement for StorageMaterializedView::getActionLock(...)
2021-06-01 11:32:16 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
tavplubix
572dcbcbe2
Update StorageMaterializedView.cpp 2021-05-31 16:38:33 +03:00
kssenii
a78c9e63e7 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-23 12:07:26 +00:00
Alexander Tokmakov
2996e7c072 fix rename matview 2021-05-19 21:53:31 +03:00
kssenii
0955e5f0be Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-12 15:11:15 +00:00
Amos Bird
cd6414639e
add metadata_snapshot to getQueryProcessingStage 2021-05-11 18:12:26 +08:00
kssenii
88b50621d4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-18 14:40:37 +00:00
tavplubix
39d55556b8
Update StorageMaterializedView.cpp 2021-04-13 23:14:05 +03:00
Alexander Tokmakov
05e4c8d3ef fix attach mv in atomic db 2021-04-13 22:13:26 +03:00
kssenii
beae1c5fa4 Use ContextPtr 2021-04-11 08:08:32 +00:00
kssenii
c1adece94f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-11 06:00:18 +00:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
kssenii
bc228f4010 Better way to drop nested table for single storage 2021-04-10 19:26:53 +00:00
tavplubix
5dfd0045a6
Merge branch 'master' into distributed_ddl_improvements 2021-03-22 18:13:19 +03:00
Nikolai Kochetov
4f133ae036 Merge branch 'master' into refactor-actions-dag 2021-03-10 12:40:55 +03:00
tavplubix
5a6cc876e4
Merge pull request #20626 from ClickHouse/fix_force_drop_materialized_view
Make force_drop_table work with MaterializedView
2021-03-09 20:05:51 +03:00
Alexander Tokmakov
2484781070 fix MV recovery in Replicated db 2021-03-08 20:26:38 +03:00
Nikolai Kochetov
a669f7d641 Merge branch 'master' into refactor-actions-dag 2021-03-05 18:21:14 +03:00
Alexander Tokmakov
e8987f799e Merge branch 'fix_force_drop_materialized_view' into just_another_fix_for_ddl_worker 2021-03-05 15:59:27 +03:00
Nikolai Kochetov
9a39459888 Refactor ActionsDAG 2021-03-04 20:38:12 +03:00
Nikolai Kochetov
15921fbfcb Remove index by name from ActionsDAG 2021-03-03 23:01:07 +03:00
Anton Popov
a4c00ab5dc
Merge pull request #21303 from ucasFL/forbid
Forbid to drop a column if it's referenced by materialized view
2021-03-03 02:55:06 +03:00
feng lv
51021c1164 forbid to drop a column if it's referenced by materialized view 2021-02-28 05:24:39 +00:00
Nikolai Kochetov
d91b8a3acb
Merge branch 'master' into filter-push-down 2021-02-26 19:33:27 +03:00
Nikolai Kochetov
d328bfa41f Review fixes. Add setting max_optimizations_to_apply. 2021-02-26 19:29:56 +03:00
alesapin
5c6c318737 Restrict mutations for engines which doesn't support them 2021-02-25 13:07:48 +03:00
tavplubix
103d569885
Merge branch 'master' into fix_force_drop_materialized_view 2021-02-20 20:41:03 +03:00
Alexander Tokmakov
2a36d6cb55 review suggestions 2021-02-20 02:41:58 +03:00
Alexander Tokmakov
13ae988eff make force_drop_table work with materialized view 2021-02-17 17:23:07 +03:00
Alexander Tokmakov
7b54b892b5 fix 2021-02-16 17:05:58 +03:00
Alexander Tokmakov
0073c87d5d fix 2021-02-02 13:32:42 +03:00
Alexander Tokmakov
f20d5e3b41 fix 2021-01-26 20:51:25 +03:00
tavplubix
ac50b1b231
Update StorageMaterializedView.cpp 2021-01-21 15:57:18 +03:00
Alexander Tokmakov
6560ec3ed5 fix segfault on aggregation when MV has unexpected structure 2021-01-20 19:36:18 +03:00
sundyli
f8815f233c
Merge branch 'master' into alias_column_partition_prune 2021-01-02 11:23:55 +08:00
sundyli
36ca21c24a
Merge branch 'master' into alias_column_partition_prune 2020-12-13 00:45:14 +08:00
sundy-li
9407028540 update 2020-12-13 00:42:15 +08:00
Vasily Nemkov
70ea507dae OPTIMIZE DEDUPLICATE BY columns
Extended OPTIMIZE ... DEDUPLICATE syntax to allow explicit (or implicit with asterisk/column transformers) list of columns to check for duplicates on.

Following syntax variants are now supported:

OPTIMIZE TABLE table DEDUPLICATE; -- the old one
OPTIMIZE TABLE table DEDUPLICATE BY *;
OPTIMIZE TABLE table DEDUPLICATE BY * EXCEPT colX;
OPTIMIZE TABLE table DEDUPLICATE BY * EXCEPT (colX, colY);
OPTIMIZE TABLE table DEDUPLICATE BY col1,col2,col3;
OPTIMIZE TABLE table DEDUPLICATE BY COLUMNS('column-matched-by-regex');
OPTIMIZE TABLE table DEDUPLICATE BY COLUMNS('column-matched-by-regex') EXCEPT colX;
OPTIMIZE TABLE table DEDUPLICATE BY COLUMNS('column-matched-by-regex') EXCEPT (colX, colY);

Note that * behaves just like in SELECT: MATERIALIZED, and ALIAS columns are not used for expansion.
Also, it is an error to specify empty list of columns, or write an expression that results in an empty list of columns, or deduplicate by an ALIAS column.
Column transformers other than EXCEPT are not supported.
2020-12-07 09:44:07 +03:00
tavplubix
88de1b052c
Merge pull request #16048 from ClickHouse/materialized-view-check-stack-size
Fix possible stack overflow in materialized views.
2020-11-16 12:02:27 +03:00
Alexander Tokmakov
5c6191f798 fix 2020-11-16 00:31:52 +03:00
Nikolai Kochetov
195c941c4e Merge branch 'master' into storage-read-query-plan 2020-11-10 15:02:22 +03:00
alesapin
72046313d4 Merge branch 'master' into nvartolomei-drop-part 2020-11-09 10:02:02 +03:00
Alexey Milovidov
5314185e25 Merge branch 'master' into azat-optimize_skip_unused_shards-optimization 2020-11-08 00:17:59 +03:00
Nikolai Kochetov
c10f733587 Merge branch 'master' into storage-read-query-plan 2020-11-06 15:43:46 +03:00
Azat Khuzhin
e7f15dff55 Do not pass StoragePtr to ReadInOrderOptimizer::getInputOrder()
Looks like this is not required anymore, since #11745
2020-11-03 21:22:46 +03:00
alesapin
67d22b3e57 Merge branch 'master' into nvartolomei-drop-part 2020-11-02 19:09:09 +03:00
Nikolai Kochetov
a42fd18390 Try fix tests. 2020-10-22 15:01:22 +03:00
Nikolai Kochetov
7fa045cff8 Merge branch 'master' into storage-read-query-plan 2020-10-22 13:31:10 +03:00
Alexey Milovidov
6c3aad34be Avoid stack overflow in materialized views, part 2: sanity check 2020-10-16 03:19:45 +03:00
Alexey Milovidov
ffd3d2519e Avoid stack overflow in materialized views, part 1 2020-10-16 03:13:17 +03:00
tavplubix
1003de75ed
Update StorageMaterializedView.cpp 2020-10-15 15:06:44 +03:00
tavplubix
6dc2f00347
Update StorageMaterializedView.cpp 2020-10-15 13:57:47 +03:00
Azat Khuzhin
bba44d0f0e Add IF EXISTS for removing inner table of the materialized view
Instead of playing with has_inner_table flag, which looks a bit icky.
2020-10-12 21:46:07 +03:00
Azat Khuzhin
d16adddb41 Fix drop of materialized view with inner table in Atomic database
Materialized view drop the inner table (if any) drop the drop() method,
and this will lead to essure recursive drop() from the drop worker
thread, which will hang:

    3  std::__1::condition_variable::wait<> ()
    4  DB::DatabaseCatalog::waitTableFinallyDropped (this=0x7ffff7831500, uuid=...) at ../src/Interpreters/DatabaseCatalog.cpp:828
    5  DB::InterpreterDropQuery::executeToTable (this=this@entry=0x7fff16ff88a0, table_id_=..., query=...) at ../src/Interpreters/InterpreterDropQuery.cpp:135
    6  DB::InterpreterDropQuery::execute (this=this@entry=0x7fff16ff88a0) at ../contrib/libcxx/include/string:1474
    7  DB::executeDropQuery (kind=kind@entry=DB::ASTDropQuery::Drop, global_context=..., target_table_id=...) at ../src/Storages/StorageMaterializedView.cpp:156
    8  DB::StorageMaterializedView::drop (this=0x7ffefc348f40) at ../src/Storages/StorageMaterializedView.cpp:169
    9  DB::DatabaseCatalog::dropTableFinally (this=this@entry=0x7ffff7831500, table=...) at ../src/Interpreters/DatabaseCatalog.cpp:775
    10 DB::DatabaseCatalog::dropTableDataTask (this=0x7ffff7831500) at ../src/Interpreters/DatabaseCatalog.cpp:745

Fix this by dropping the inner table just before scheduling job the the
drop worker thread.
2020-10-12 21:46:07 +03:00
Nikolai Kochetov
c5cb05f5f3 Try fix tests. 2020-10-07 14:26:29 +03:00
Azat Khuzhin
b838214a35 Pass non-const SelectQueryInfo (and drop mutable qualifiers) 2020-10-02 22:42:35 +03:00
Azat Khuzhin
587cde853e Avoid skipping unused shards twice (for query processing stage and read itself) 2020-10-02 22:42:09 +03:00
Nikolai Kochetov
5ac6bc071d QueryPlan for StorageBuffer and StorageMaterializedView read. 2020-09-30 15:22:57 +03:00
Nicolae Vartolomei
97d0b5ab23 Remove unused query argument 2020-09-08 10:56:10 +01:00
Nikolai Kochetov
2cca4d5fcf Refactor Pipe [part 2]. 2020-08-03 16:54:14 +03:00
alesapin
c75830e878 Alter table freeze with verbose output 2020-07-28 18:10:36 +03:00
alesapin
014bb070ec Fix tests 2020-07-14 11:19:39 +03:00
alesapin
c9fa5d2ec3 Better naming 2020-06-19 18:39:41 +03:00
alesapin
d79982f497 Better locks in Storages 2020-06-18 19:10:47 +03:00
alesapin
aab4ce6394 Truncate with metadata 2020-06-18 13:29:13 +03:00
alesapin
31abbe5dbd Select query in metadata 2020-06-17 17:06:22 +03:00
alesapin
ed8f3b2fc4 TTL in storage in memory metadata 2020-06-17 16:39:26 +03:00
alesapin
ba04d02f1e Compilable sorting key in metadata 2020-06-17 14:05:11 +03:00
alesapin
1ddeb3d149 Buildable getSampleBlock in StorageInMemoryMetadata 2020-06-16 18:51:29 +03:00
alesapin
36ba0192df Metadata in read and write methods of IStorage 2020-06-15 22:08:58 +03:00
alesapin
b47a7327fd All set methods in metadata 2020-06-15 21:08:05 +03:00
alesapin
af2fe2ba55 Compilable setColumns, setConstraints, setIndices 2020-06-15 19:55:33 +03:00
alesapin
8be957ecb5 Better checks around metadata 2020-06-10 14:16:31 +03:00
alesapin
75977bd939 Gcc fixed 2020-06-10 00:22:01 +03:00
alesapin
7b63a88120 Remove redundant error code 2020-06-08 18:19:41 +03:00
alesapin
ac67d0485c Merge branch 'master' into consistent_metadata4 2020-06-08 15:06:51 +03:00
Anton Popov
5c42408add
Merge pull request #9113 from dimarub2000/group_by_in_order_optimization
[WIP] Optimization of GROUP BY with respect to table sorting key.
2020-06-06 14:25:59 +03:00
alesapin
abaf47f0cd Make metadata single structure 2020-06-05 20:29:40 +03:00
alesapin
5bc7f67e61 Better metadata for select query and renames 2020-06-05 14:54:54 +03:00
alesapin
663e92b1c5 Rename to methods 2020-06-01 14:29:11 +03:00
alesapin
3847ea892d Merge branch 'master' into consistent_metadata3 2020-06-01 13:17:59 +03:00
Dmitry
4b0d32f026 Merge branch 'master' of github.com:yandex/ClickHouse into group_by_in_order_optimization 2020-05-31 00:21:02 +03:00
Alexander Tokmakov
c071e0a473 add context to DatabaseCatalog methods 2020-05-29 02:01:18 +03:00
alesapin
52ca6b2051 I'm able to build it 2020-05-28 15:37:05 +03:00
Dmitry
bbe0245b9d changes after review #1 2020-05-13 16:49:10 +03:00
alesapin
f981649213 Fix pushing to views stream and refactor virtuals 2020-04-28 13:38:57 +03:00
alesapin
2829774105 Merge branch 'master' into refactor_istorage 2020-04-27 15:34:21 +03:00
alesapin
b4425c2dc4 Remove getColumn and hasColumn methods from IStorage 2020-04-24 13:20:03 +03:00
Alexander Tokmakov
20e6593f96 disable by default 2020-04-12 21:52:59 +03:00
Alexander Tokmakov
5e6d4b9449 Merge branch 'master' into database_atomic 2020-04-12 16:35:44 +03:00
Alexander Kazakov
497df3086f Merge branch 'master' into timed_rwlock
Change-Id: I620bfde2121ff013773b001d514b40b1e796a58b
2020-04-10 11:38:20 +03:00
Alexander Tokmakov
50decc755d fix rename materialized view 2020-04-10 04:35:37 +03:00
Alexander Kazakov
26dd6140b2 Added new config settings to control timeouts
* "lock_acquire_timeout" controls for how long a query will continue to
acquire each lock on its argument tables
 * "lock_acquire_timeout_for_background_operations" is a per-table
setting for storages of *MergeTree family
2020-04-09 21:10:27 +03:00
Alexander Tokmakov
dd1590830b Merge branch 'master' into database_atomic 2020-04-08 22:00:46 +03:00
Alexander Tokmakov
135197aa75 fixes 2020-04-08 04:02:00 +03:00
Alexander Tokmakov
4c48b7dd80 better rename 2020-04-07 18:31:33 +03:00
Alexander Tokmakov
a2101c9b1c better code 2020-04-07 02:22:44 +03:00
Alexander Tokmakov
08bae4668d Merge branch 'master' into database_atomic 2020-04-06 16:18:07 +03:00
Azat Khuzhin
1232760f78 Fix Distributed-over-Distributed when nested table has only one shard 2020-04-04 13:47:35 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00