thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-19 16:20:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add definers for views (Attempt 2) (#60439)

Browse Source
This commit is contained in:
pufit 2024-02-27 19:00:17 -05:00 committed by GitHub
parent 90c9ae1b22
commit 330a206470
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
69 changed files with 1317 additions and 350 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
docs/en/operations/server-configuration-parameters/settings.md
View File

@ -2927,3 +2927,15 @@ If set to true, then alter operations will be surrounded by parentheses in forma
Type: Bool
Default: 0
## ignore_empty_sql_security_in_create_view_query {#ignore_empty_sql_security_in_create_view_query}
If true, ClickHouse doesn't write defaults for empty SQL security statement in CREATE VIEW queries.
:::note
This setting is only necessary for the migration period and will become obsolete in 24.4
:::
Type: Bool
Default: 1

18
docs/en/operations/settings/settings.md
View File

@ -5378,6 +5378,24 @@ SELECT map('a', range(number), 'b', number, 'c', 'str_' || toString(number)) as
Default value: `false`.
## default_normal_view_sql_security {#default_normal_view_sql_security}
Allows to set default `SQL SECURITY` option while creating a normal view. [More about SQL security](../../sql-reference/statements/create/view.md#sql_security).
The default value is `INVOKER`.
## default_materialized_view_sql_security {#default_materialized_view_sql_security}
Allows to set a default value for SQL SECURITY option when creating a materialized view. [More about SQL security](../../sql-reference/statements/create/view.md#sql_security).
The default value is `DEFINER`.
## default_view_definer {#default_view_definer}
Allows to set default `DEFINER` option while creating a view. [More about SQL security](../../sql-reference/statements/create/view.md#sql_security).
The default value is `CURRENT_USER`.
## max_partition_size_to_drop
Restriction on dropping partitions in query time. The value 0 means that you can drop partitions without any restrictions.

51
docs/en/sql-reference/statements/create/view.md
View File

@ -13,7 +13,9 @@ Creates a new view. Views can be [normal](#normal-view), [materialized](#materia
Syntax:
``` sql
CREATE [OR REPLACE] VIEW [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster_name] AS SELECT ...
CREATE [OR REPLACE] VIEW [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster_name]
[DEFINER = { user | CURRENT_USER }] [SQL SECURITY { DEFINER | INVOKER | NONE }]
AS SELECT ...
```
Normal views do not store any data. They just perform a read from another table on each access. In other words, a normal view is nothing more than a saved query. When reading from a view, this saved query is used as a subquery in the [FROM](../../../sql-reference/statements/select/from.md) clause.
@ -52,7 +54,9 @@ SELECT * FROM view(column1=value1, column2=value2 ...)
## Materialized View
``` sql
CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db.]table_name [ON CLUSTER] [TO[db.]name] [ENGINE = engine] [POPULATE] AS SELECT ...
CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db.]table_name [ON CLUSTER] [TO[db.]name] [ENGINE = engine] [POPULATE]
[DEFINER = { user | CURRENT_USER }] [SQL SECURITY { DEFINER | INVOKER | NONE }]
AS SELECT ...
```
:::tip
@ -91,6 +95,49 @@ Views look the same as normal tables. For example, they are listed in the result
To delete a view, use [DROP VIEW](../../../sql-reference/statements/drop.md#drop-view). Although `DROP TABLE` works for VIEWs as well.
## SQL security {#sql_security}
`DEFINER` and `SQL SECURITY` allow you to specify which ClickHouse user to use when executing the view's underlying query.
`SQL SECURITY` has three legal values: `DEFINER`, `INVOKER`, or `NONE`. You can specify any existing user or `CURRENT_USER` in the `DEFINER` clause.
The following table will explain which rights are required for which user in order to select from view.
Note that regardless of the SQL security option, in every case it is still required to have `GRANT SELECT ON <view>` in order to read from it.
| SQL security option | View | Materialized View |
|---------------------|-----------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------|
| `DEFINER alice` | `alice` must have a `SELECT` grant for the view's source table. | `alice` must have a `SELECT` grant for the view's source table and an `INSERT` grant for the view's target table. |
| `INVOKER` | User must have a `SELECT` grant for the view's source table. | `SQL SECURITY INVOKER` can't be specified for materialized views. |
| `NONE` | - | - |
:::note
`SQL SECURITY NONE` is a deprecated option. Any user with the rights to create views with `SQL SECURITY NONE` will be able to execute any arbitrary query.
Thus, it is required to have `GRANT ALLOW SQL SECURITY NONE TO <user>` in order to create a view with this option.
:::
If `DEFINER`/`SQL SECURITY` aren't specified, the default values are used:
- `SQL SECURITY`: `INVOKER` for normal views and `DEFINER` for materialized views ([configurable by settings](../../../operations/settings/settings.md#default_normal_view_sql_security))
- `DEFINER`: `CURRENT_USER` ([configurable by settings](../../../operations/settings/settings.md#default_view_definer))
If a view is attached without `DEFINER`/`SQL SECURITY` specified, the default value is `SQL SECURITY NONE` for the materialized view and `SQL SECURITY INVOKER` for the normal view.
To change SQL security for an existing view, use
```sql
ALTER TABLE MODIFY SQL SECURITY { DEFINER | INVOKER | NONE } [DEFINER = { user | CURRENT_USER }]
```
### Examples sql security
```sql
CREATE test_view
DEFINER = alice SQL SECURITY DEFINER
AS SELECT ...
```
```sql
CREATE test_view
SQL SECURITY INVOKER
AS SELECT ...
```
## Live View [Deprecated]
This feature is deprecated and will be removed in the future.

3
docs/en/sql-reference/statements/grant.md
View File

@ -114,6 +114,7 @@ Hierarchy of privileges:
- `ALTER VIEW`
- `ALTER VIEW REFRESH`
- `ALTER VIEW MODIFY QUERY`
- `ALTER VIEW MODIFY SQL SECURITY`
- [CREATE](#grant-create)
- `CREATE DATABASE`
- `CREATE TABLE`
@ -307,6 +308,7 @@ Allows executing [ALTER](../../sql-reference/statements/alter/index.md) queries
- `ALTER VIEW` Level: `GROUP`
- `ALTER VIEW REFRESH`. Level: `VIEW`. Aliases: `ALTER LIVE VIEW REFRESH`, `REFRESH VIEW`
- `ALTER VIEW MODIFY QUERY`. Level: `VIEW`. Aliases: `ALTER TABLE MODIFY QUERY`
- `ALTER VIEW MODIFY SQL SECURITY`. Level: `VIEW`. Aliases: `ALTER TABLE MODIFY SQL SECURITY`
Examples of how this hierarchy is treated:
@ -409,6 +411,7 @@ Allows a user to execute queries that manage users, roles and row policies.
- `SHOW_ROW_POLICIES`. Level: `GLOBAL`. Aliases: `SHOW POLICIES`, `SHOW CREATE ROW POLICY`, `SHOW CREATE POLICY`
- `SHOW_QUOTAS`. Level: `GLOBAL`. Aliases: `SHOW CREATE QUOTA`
- `SHOW_SETTINGS_PROFILES`. Level: `GLOBAL`. Aliases: `SHOW PROFILES`, `SHOW CREATE SETTINGS PROFILE`, `SHOW CREATE PROFILE`
- `ALLOW SQL SECURITY NONE`. Level: `GLOBAL`. Aliases: `CREATE SQL SECURITY NONE`, `SQL SECURITY NONE`, `SECURITY NONE`
The `ROLE ADMIN` privilege allows a user to assign and revoke any roles including those which are not assigned to the user with the admin option.

54
docs/ru/sql-reference/statements/create/view.md
View File

@ -11,7 +11,9 @@ sidebar_label: "Представление"
## Обычные представления {#normal}
``` sql
CREATE [OR REPLACE] VIEW [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster_name] AS SELECT ...
CREATE [OR REPLACE] VIEW [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster_name]
[DEFINER = { user | CURRENT_USER }] [SQL SECURITY { DEFINER | INVOKER | NONE }]
AS SELECT ...
```
Обычные представления не хранят никаких данных, они выполняют чтение данных из другой таблицы при каждом доступе. Другими словами, обычное представление — это не что иное, как сохраненный запрос. При чтении данных из представления этот сохраненный запрос используется как подзапрос в секции [FROM](../../../sql-reference/statements/select/from.md).
@ -37,7 +39,9 @@ SELECT a, b, c FROM (SELECT ...)
## Материализованные представления {#materialized}
``` sql
CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db.]table_name [ON CLUSTER] [TO[db.]name] [ENGINE = engine] [POPULATE] AS SELECT ...
CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db.]table_name [ON CLUSTER] [TO[db.]name] [ENGINE = engine] [POPULATE]
[DEFINER = { user | CURRENT_USER }] [SQL SECURITY { DEFINER | INVOKER | NONE }]
AS SELECT ...
```
Материализованные (MATERIALIZED) представления хранят данные, преобразованные соответствующим запросом [SELECT](../../../sql-reference/statements/select/index.md).
@ -66,6 +70,52 @@ CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db.]table_name [ON CLUSTER] [TO[db.]na
Чтобы удалить представление, следует использовать [DROP VIEW](../../../sql-reference/statements/drop.md#drop-view). Впрочем, `DROP TABLE` тоже работает для представлений.
## SQL безопасность {#sql_security}
Параметры `DEFINER` и `SQL SECURITY` позволяют задать правило от имени какого пользователя будут выполняться запросы к таблицам, на которые ссылается представление.
Для `SQL SECURITY` допустимо три значения: `DEFINER`, `INVOKER`, или `NONE`.
Для `DEFINER` можно указать имя любого существующего пользователя или же `CURRENT_USER`.
Далее приведена таблица, объясняющая какие права необходимы каким пользователям при заданных параметрах SQL безопасности.
Обратите внимание, что, в независимости от заданных параметров SQL безопасности,
у пользователя должно быть право `GRANT SELECT ON <view>` для чтения из представления.
| SQL security option | View | Materialized View |
|---------------------|----------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------|
| `DEFINER alice` | У `alice` должно быть право `SELECT` на таблицу-источник. | У `alice` должны быть права `SELECT` на таблицу-источник и `INSERT` на таблицу-назначение. |
| `INVOKER` | У пользователя выполняющего запрос к представлению должно быть право `SELECT` на таблицу-источник. | Тип `SQL SECURITY INVOKER` не может быть указан для материализованных представлений. |
| `NONE` | - | - |
:::note
Тип `SQL SECURITY NONE` не безопасен для использования. Любой пользователь с правом создавать представления с `SQL SECURITY NONE` сможет исполнять любые запросы без проверки прав.
По умолчанию, у пользователей нет прав указывать `SQL SECURITY NONE`, однако, при необходимости, это право можно выдать с помощью `GRANT ALLOW SQL SECURITY NONE TO <user>`.
:::
Если `DEFINER`/`SQL SECURITY` не указан, будут использованы значения по умолчанию:
- `SQL SECURITY`: `INVOKER` для обычных представлений и `DEFINER` для материализованных ([изменяется в настройках](../../../operations/settings/settings.md#default_normal_view_sql_security))
- `DEFINER`: `CURRENT_USER` ([изменяется в настройках](../../../operations/settings/settings.md#default_view_definer))
Если представление подключается с помощью ключевого слова `ATTACH` и настройки SQL безопасности не были заданы,
то по умолчанию будет использоваться `SQL SECURITY NONE` для материализованных представлений и `SQL SECURITY INVOKER` для обычных.
Изменить параметры SQL безопасности возможно с помощью следующего запроса:
```sql
ALTER TABLE MODIFY SQL SECURITY { DEFINER | INVOKER | NONE } [DEFINER = { user | CURRENT_USER }]
```
### Примеры представлений с SQL безопасностью
```sql
CREATE test_view
DEFINER = alice SQL SECURITY DEFINER
AS SELECT ...
```
```sql
CREATE test_view
SQL SECURITY INVOKER
AS SELECT ...
```
## LIVE-представления [экспериментальный функционал] {#live-view}
:::note Важно

16
src/Access/Common/AccessFlags.cpp
View File

@ -103,6 +103,7 @@ namespace
const Flags & getColumnFlags() const { return all_flags_for_target[COLUMN]; }
const Flags & getDictionaryFlags() const { return all_flags_for_target[DICTIONARY]; }
const Flags & getNamedCollectionFlags() const { return all_flags_for_target[NAMED_COLLECTION]; }
const Flags & getUserNameFlags() const { return all_flags_for_target[USER_NAME]; }
const Flags & getAllFlagsGrantableOnGlobalLevel() const { return getAllFlags(); }
const Flags & getAllFlagsGrantableOnGlobalWithParameterLevel() const { return getGlobalWithParameterFlags(); }
const Flags & getAllFlagsGrantableOnDatabaseLevel() const { return all_flags_grantable_on_database_level; }
@ -121,6 +122,7 @@ namespace
COLUMN,
DICTIONARY,
NAMED_COLLECTION,
USER_NAME,
};
struct Node;
@ -300,7 +302,7 @@ namespace
collectAllFlags(child.get());
all_flags_grantable_on_table_level = all_flags_for_target[TABLE] | all_flags_for_target[DICTIONARY] | all_flags_for_target[COLUMN];
all_flags_grantable_on_global_with_parameter_level = all_flags_for_target[NAMED_COLLECTION];
all_flags_grantable_on_global_with_parameter_level = all_flags_for_target[NAMED_COLLECTION] | all_flags_for_target[USER_NAME];
all_flags_grantable_on_database_level = all_flags_for_target[DATABASE] | all_flags_grantable_on_table_level;
}
@ -351,7 +353,7 @@ namespace
std::unordered_map<std::string_view, Flags> keyword_to_flags_map;
std::vector<Flags> access_type_to_flags_mapping;
Flags all_flags;
Flags all_flags_for_target[static_cast<size_t>(NAMED_COLLECTION) + 1];
Flags all_flags_for_target[static_cast<size_t>(USER_NAME) + 1];
Flags all_flags_grantable_on_database_level;
Flags all_flags_grantable_on_table_level;
Flags all_flags_grantable_on_global_with_parameter_level;
@ -371,7 +373,11 @@ std::unordered_map<AccessFlags::ParameterType, AccessFlags> AccessFlags::splitIn
if (named_collection_flags)
result.emplace(ParameterType::NAMED_COLLECTION, named_collection_flags);
auto other_flags = (~AccessFlags::allNamedCollectionFlags()) & *this;
auto user_flags = AccessFlags::allUserNameFlags() & *this;
if (user_flags)
result.emplace(ParameterType::USER_NAME, user_flags);
auto other_flags = (~named_collection_flags & ~user_flags) & *this;
if (other_flags)
result.emplace(ParameterType::NONE, other_flags);
@ -387,6 +393,9 @@ AccessFlags::ParameterType AccessFlags::getParameterType() const
if (AccessFlags::allNamedCollectionFlags().contains(*this))
return AccessFlags::NAMED_COLLECTION;
if (AccessFlags::allUserNameFlags().contains(*this))
return AccessFlags::USER_NAME;
throw Exception(ErrorCodes::MIXED_ACCESS_PARAMETER_TYPES, "Having mixed parameter types: {}", toString());
}
@ -405,6 +414,7 @@ AccessFlags AccessFlags::allTableFlags() { return Helper::instance().getTableFla
AccessFlags AccessFlags::allColumnFlags() { return Helper::instance().getColumnFlags(); }
AccessFlags AccessFlags::allDictionaryFlags() { return Helper::instance().getDictionaryFlags(); }
AccessFlags AccessFlags::allNamedCollectionFlags() { return Helper::instance().getNamedCollectionFlags(); }
AccessFlags AccessFlags::allUserNameFlags() { return Helper::instance().getUserNameFlags(); }
AccessFlags AccessFlags::allFlagsGrantableOnGlobalLevel() { return Helper::instance().getAllFlagsGrantableOnGlobalLevel(); }
AccessFlags AccessFlags::allFlagsGrantableOnGlobalWithParameterLevel() { return Helper::instance().getAllFlagsGrantableOnGlobalWithParameterLevel(); }
AccessFlags AccessFlags::allFlagsGrantableOnDatabaseLevel() { return Helper::instance().getAllFlagsGrantableOnDatabaseLevel(); }

4
src/Access/Common/AccessFlags.h
View File

@ -57,6 +57,7 @@ public:
{
NONE,
NAMED_COLLECTION,
USER_NAME,
};
ParameterType getParameterType() const;
std::unordered_map<ParameterType, AccessFlags> splitIntoParameterTypes() const;
@ -103,6 +104,9 @@ public:
/// Returns all the flags related to a named collection.
static AccessFlags allNamedCollectionFlags();
/// Returns all the flags related to a user.
static AccessFlags allUserNameFlags();
/// Returns all the flags which could be granted on the global level.
/// The same as allFlags().
static AccessFlags allFlagsGrantableOnGlobalLevel();

5
src/Access/Common/AccessType.h
View File

@ -12,7 +12,7 @@ enum class AccessType
/// Macro M should be defined as M(name, aliases, node_type, parent_group_name)
/// where name is identifier with underscores (instead of spaces);
/// aliases is a string containing comma-separated list;
/// node_type either specifies access type's level (GLOBAL/NAMED_COLLECTION/DATABASE/TABLE/DICTIONARY/VIEW/COLUMNS),
/// node_type either specifies access type's level (GLOBAL/NAMED_COLLECTION/USER_NAME/DATABASE/TABLE/DICTIONARY/VIEW/COLUMNS),
/// or specifies that the access type is a GROUP of other access types;
/// parent_group_name is the name of the group containing this access type (or NONE if there is no such group).
/// NOTE A parent group must be declared AFTER all its children.
@ -82,6 +82,7 @@ enum class AccessType
\
M(ALTER_VIEW_MODIFY_QUERY, "ALTER TABLE MODIFY QUERY", VIEW, ALTER_VIEW) \
M(ALTER_VIEW_MODIFY_REFRESH, "ALTER TABLE MODIFY QUERY", VIEW, ALTER_VIEW) \
M(ALTER_VIEW_MODIFY_SQL_SECURITY, "ALTER TABLE MODIFY SQL SECURITY", VIEW, ALTER_VIEW) \
M(ALTER_VIEW, "", GROUP, ALTER) /* allows to execute ALTER VIEW REFRESH, ALTER VIEW MODIFY QUERY, ALTER VIEW MODIFY REFRESH;
implicitly enabled by the grant ALTER_TABLE */\
\
@ -138,6 +139,7 @@ enum class AccessType
M(CREATE_SETTINGS_PROFILE, "CREATE PROFILE", GLOBAL, ACCESS_MANAGEMENT) \
M(ALTER_SETTINGS_PROFILE, "ALTER PROFILE", GLOBAL, ACCESS_MANAGEMENT) \
M(DROP_SETTINGS_PROFILE, "DROP PROFILE", GLOBAL, ACCESS_MANAGEMENT) \
M(ALLOW_SQL_SECURITY_NONE, "CREATE SQL SECURITY NONE, ALLOW SQL SECURITY NONE, SQL SECURITY NONE, SECURITY NONE", GLOBAL, ACCESS_MANAGEMENT) \
M(SHOW_USERS, "SHOW CREATE USER", GLOBAL, SHOW_ACCESS) \
M(SHOW_ROLES, "SHOW CREATE ROLE", GLOBAL, SHOW_ACCESS) \
M(SHOW_ROW_POLICIES, "SHOW POLICIES, SHOW CREATE ROW POLICY, SHOW CREATE POLICY", TABLE, SHOW_ACCESS) \
@ -149,6 +151,7 @@ enum class AccessType
M(SHOW_NAMED_COLLECTIONS_SECRETS, "SHOW NAMED COLLECTIONS SECRETS", NAMED_COLLECTION, NAMED_COLLECTION_ADMIN) \
M(NAMED_COLLECTION, "NAMED COLLECTION USAGE, USE NAMED COLLECTION", NAMED_COLLECTION, NAMED_COLLECTION_ADMIN) \
M(NAMED_COLLECTION_ADMIN, "NAMED COLLECTION CONTROL", NAMED_COLLECTION, ALL) \
M(SET_DEFINER, "", USER_NAME, ALL) \
\
M(SYSTEM_SHUTDOWN, "SYSTEM KILL, SHUTDOWN", GLOBAL, SYSTEM) \
M(SYSTEM_DROP_DNS_CACHE, "SYSTEM DROP DNS, DROP DNS CACHE, DROP DNS", GLOBAL, SYSTEM_DROP_CACHE) \

11
src/Access/Common/SQLSecurityDefs.h Normal file
View File

@ -0,0 +1,11 @@
#pragma once
#include <Core/Types.h>
/// SQL security enum. Used in ASTSQLSecurity::type. For more info, please refer to the docs/sql-reference/statements/create/view.md#sql_security
enum class SQLSecurityType : uint8_t
{
INVOKER, /// All queries will be executed with the current user's context.
DEFINER, /// All queries will be executed with the specified user's context.
NONE, /// All queries will be executed with the global context.
};

3
src/Access/tests/gtest_access_rights_ops.cpp
View File

@ -53,7 +53,8 @@ TEST(AccessRights, Union)
"SHOW ROW POLICIES, SYSTEM MERGES, SYSTEM TTL MERGES, SYSTEM FETCHES, "
"SYSTEM MOVES, SYSTEM PULLING REPLICATION LOG, SYSTEM CLEANUP, SYSTEM VIEWS, SYSTEM SENDS, SYSTEM REPLICATION QUEUES, "
"SYSTEM DROP REPLICA, SYSTEM SYNC REPLICA, SYSTEM RESTART REPLICA, "
"SYSTEM RESTORE REPLICA, SYSTEM WAIT LOADING PARTS, SYSTEM SYNC DATABASE REPLICA, SYSTEM FLUSH DISTRIBUTED, dictGet ON db1.*, GRANT NAMED COLLECTION ADMIN ON db1");
"SYSTEM RESTORE REPLICA, SYSTEM WAIT LOADING PARTS, SYSTEM SYNC DATABASE REPLICA, SYSTEM FLUSH DISTRIBUTED, dictGet ON db1.*, "
"GRANT SET DEFINER ON db1, GRANT NAMED COLLECTION ADMIN ON db1");
}

1
src/Core/ServerSettings.h
View File

@ -59,6 +59,7 @@ namespace DB
M(Double, cgroup_memory_watcher_soft_limit_ratio, 0.9, "Sort memory limit ratio limit for cgroup memory usage observer", 0) \
M(UInt64, async_insert_threads, 16, "Maximum number of threads to actually parse and insert data in background. Zero means asynchronous mode is disabled", 0) \
M(Bool, async_insert_queue_flush_on_shutdown, true, "If true queue of asynchronous inserts is flushed on graceful shutdown", 0) \
M(Bool, ignore_empty_sql_security_in_create_view_query, true, "If true, ClickHouse doesn't write defaults for empty SQL security statement in CREATE VIEW queries. This setting is only necessary for the migration period and will become obsolete in 24.4", 0) \
\
M(UInt64, max_concurrent_queries, 0, "Maximum number of concurrently executed queries. Zero means unlimited.", 0) \
M(UInt64, max_concurrent_insert_queries, 0, "Maximum number of concurrently INSERT queries. Zero means unlimited.", 0) \

3
src/Core/Settings.h
View File

@ -870,6 +870,9 @@ class IColumn;
M(Bool, print_pretty_type_names, true, "Print pretty type names in DESCRIBE query and toTypeName() function", 0) \
M(Bool, create_table_empty_primary_key_by_default, false, "Allow to create *MergeTree tables with empty primary key when ORDER BY and PRIMARY KEY not specified", 0) \
M(Bool, allow_named_collection_override_by_default, true, "Allow named collections' fields override by default.", 0)\
M(SQLSecurityType, default_normal_view_sql_security, SQLSecurityType::INVOKER, "Allows to set a default value for SQL SECURITY option when creating a normal view.", 0) \
M(SQLSecurityType, default_materialized_view_sql_security, SQLSecurityType::DEFINER, "Allows to set a default value for SQL SECURITY option when creating a materialized view.", 0) \
M(String, default_view_definer, "CURRENT_USER", "Allows to set a default value for DEFINER option when creating view.", 0) \
M(Bool, allow_experimental_shared_merge_tree, false, "Only available in ClickHouse Cloud", 0) \
M(UInt64, cache_warmer_threads, 4, "Only available in ClickHouse Cloud", 0) \
M(Int64, ignore_cold_parts_seconds, 0, "Only available in ClickHouse Cloud", 0) \

3
src/Core/SettingsChangesHistory.h
View File

@ -105,6 +105,9 @@ static std::map<ClickHouseVersion, SettingsChangesHistory::SettingsChanges> sett
{"min_external_table_block_size_bytes", DEFAULT_INSERT_BLOCK_SIZE * 256, DEFAULT_INSERT_BLOCK_SIZE * 256, "Squash blocks passed to external table to specified size in bytes, if blocks are not big enough."},
{"parallel_replicas_prefer_local_join", true, true, "If true, and JOIN can be executed with parallel replicas algorithm, and all storages of right JOIN part are *MergeTree, local JOIN will be used instead of GLOBAL JOIN."},
{"extract_key_value_pairs_max_pairs_per_row", 0, 0, "Max number of pairs that can be produced by the `extractKeyValuePairs` function. Used as a safeguard against consuming too much memory."},
{"default_view_definer", "CURRENT_USER", "CURRENT_USER", "Allows to set default `DEFINER` option while creating a view"},
{"default_materialized_view_sql_security", "DEFINER", "DEFINER", "Allows to set a default value for SQL SECURITY option when creating a materialized view"},
{"default_normal_view_sql_security", "INVOKER", "INVOKER", "Allows to set default `SQL SECURITY` option while creating a normal view"},
{"mysql_map_string_to_text_in_show_columns", false, true, "Reduce the configuration effort to connect ClickHouse with BI tools."},
{"mysql_map_fixed_string_to_text_in_show_columns", false, true, "Reduce the configuration effort to connect ClickHouse with BI tools."},
}},

6
src/Core/SettingsEnums.cpp
View File

@ -1,5 +1,6 @@
#include <Core/SettingsEnums.h>
#include <magic_enum.hpp>
#include <Access/Common/SQLSecurityDefs.h>
namespace DB
@ -206,4 +207,9 @@ IMPLEMENT_SETTING_ENUM(DateTimeOverflowBehavior, ErrorCodes::BAD_ARGUMENTS,
{{"throw", FormatSettings::DateTimeOverflowBehavior::Throw},
{"ignore", FormatSettings::DateTimeOverflowBehavior::Ignore},
{"saturate", FormatSettings::DateTimeOverflowBehavior::Saturate}})
IMPLEMENT_SETTING_ENUM(SQLSecurityType, ErrorCodes::BAD_ARGUMENTS,
{{"DEFINER", SQLSecurityType::DEFINER},
{"INVOKER", SQLSecurityType::INVOKER},
{"NONE", SQLSecurityType::NONE}})
}

2
src/Core/SettingsEnums.h
View File

@ -6,6 +6,7 @@
#include <Formats/FormatSettings.h>
#include <IO/ReadSettings.h>
#include <Common/ShellCommandSettings.h>
#include <Parsers/ASTSQLSecurity.h>
namespace DB
@ -266,4 +267,5 @@ DECLARE_SETTING_ENUM(SchemaInferenceMode)
DECLARE_SETTING_ENUM_WITH_RENAME(DateTimeOverflowBehavior, FormatSettings::DateTimeOverflowBehavior)
DECLARE_SETTING_ENUM(SQLSecurityType)
}

2
src/Interpreters/AsynchronousInsertQueue.cpp
View File

@ -261,7 +261,7 @@ void AsynchronousInsertQueue::preprocessInsertQuery(const ASTPtr & query, const
InterpreterInsertQuery interpreter(query, query_context, query_context->getSettingsRef().insert_allow_materialized_columns);
auto table = interpreter.getTable(insert_query);
auto sample_block = interpreter.getSampleBlock(insert_query, table, table->getInMemoryMetadataPtr());
auto sample_block = interpreter.getSampleBlock(insert_query, table, table->getInMemoryMetadataPtr(), query_context);
if (!FormatFactory::instance().isInputFormat(insert_query.format))
throw Exception(ErrorCodes::UNKNOWN_FORMAT, "Unknown input format {}", insert_query.format);

28
src/Interpreters/Context.cpp
View File

@ -797,6 +797,7 @@ ContextMutablePtr Context::createGlobal(ContextSharedPart * shared_part)
{
auto res = std::shared_ptr<Context>(new Context);
res->shared = shared_part;
res->query_access_info = std::make_shared<QueryAccessInfo>();
return res;
}
@ -816,7 +817,9 @@ SharedContextHolder Context::createShared()
ContextMutablePtr Context::createCopy(const ContextPtr & other)
{
SharedLockGuard lock(other->mutex);
return std::shared_ptr<Context>(new Context(*other));
auto new_context = std::shared_ptr<Context>(new Context(*other));
new_context->query_access_info = std::make_shared<QueryAccessInfo>(*other->query_access_info);
return new_context;
}
ContextMutablePtr Context::createCopy(const ContextWeakPtr & other)
@ -1610,12 +1613,12 @@ void Context::addQueryAccessInfo(
if (isGlobalContext())
throw Exception(ErrorCodes::LOGICAL_ERROR, "Global context cannot have query access info");
std::lock_guard lock(query_access_info.mutex);
query_access_info.databases.emplace(quoted_database_name);
query_access_info.tables.emplace(full_quoted_table_name);
std::lock_guard lock(query_access_info->mutex);
query_access_info->databases.emplace(quoted_database_name);
query_access_info->tables.emplace(full_quoted_table_name);
for (const auto & column_name : column_names)
query_access_info.columns.emplace(full_quoted_table_name + "." + backQuoteIfNeed(column_name));
query_access_info->columns.emplace(full_quoted_table_name + "." + backQuoteIfNeed(column_name));
}
void Context::addQueryAccessInfo(const Names & partition_names)
@ -1623,9 +1626,9 @@ void Context::addQueryAccessInfo(const Names & partition_names)
if (isGlobalContext())
throw Exception(ErrorCodes::LOGICAL_ERROR, "Global context cannot have query access info");
std::lock_guard<std::mutex> lock(query_access_info.mutex);
std::lock_guard<std::mutex> lock(query_access_info->mutex);
for (const auto & partition_name : partition_names)
query_access_info.partitions.emplace(partition_name);
query_access_info->partitions.emplace(partition_name);
}
void Context::addViewAccessInfo(const String & view_name)
@ -1633,8 +1636,8 @@ void Context::addViewAccessInfo(const String & view_name)
if (isGlobalContext())
throw Exception(ErrorCodes::LOGICAL_ERROR, "Global context cannot have query access info");
std::lock_guard<std::mutex> lock(query_access_info.mutex);
query_access_info.views.emplace(view_name);
std::lock_guard<std::mutex> lock(query_access_info->mutex);
query_access_info->views.emplace(view_name);
}
void Context::addQueryAccessInfo(const QualifiedProjectionName & qualified_projection_name)
@ -1645,8 +1648,8 @@ void Context::addQueryAccessInfo(const QualifiedProjectionName & qualified_proje
if (isGlobalContext())
throw Exception(ErrorCodes::LOGICAL_ERROR, "Global context cannot have query access info");
std::lock_guard<std::mutex> lock(query_access_info.mutex);
query_access_info.projections.emplace(fmt::format(
std::lock_guard<std::mutex> lock(query_access_info->mutex);
query_access_info->projections.emplace(fmt::format(
"{}.{}", qualified_projection_name.storage_id.getFullTableName(), backQuoteIfNeed(qualified_projection_name.projection_name)));
}
@ -2297,7 +2300,8 @@ void Context::setMacros(std::unique_ptr<Macros> && macros)
ContextMutablePtr Context::getQueryContext() const
{
auto ptr = query_context.lock();
if (!ptr) throw Exception(ErrorCodes::THERE_IS_NO_QUERY, "There is no query or query context has expired");
if (!ptr)
throw Exception(ErrorCodes::THERE_IS_NO_QUERY, "There is no query or query context has expired");
return ptr;
}

9
src/Interpreters/Context.h
View File

@ -351,8 +351,11 @@ protected:
std::set<std::string> projections{};
std::set<std::string> views{};
};
using QueryAccessInfoPtr = std::shared_ptr<QueryAccessInfo>;
QueryAccessInfo query_access_info;
/// In some situations, we want to be able to transfer the access info from children back to parents (e.g. definers context).
/// Therefore, query_access_info must be a pointer.
QueryAccessInfoPtr query_access_info;
/// Record names of created objects of factories (for testing, etc)
struct QueryFactoriesInfo
@ -677,7 +680,9 @@ public:
const Block * tryGetSpecialScalar(const String & name) const;
void addSpecialScalar(const String & name, const Block & block);
const QueryAccessInfo & getQueryAccessInfo() const { return query_access_info; }
const QueryAccessInfo & getQueryAccessInfo() const { return *getQueryAccessInfoPtr(); }
const QueryAccessInfoPtr getQueryAccessInfoPtr() const { return query_access_info; }
void setQueryAccessInfo(QueryAccessInfoPtr other) { query_access_info = other; }
void addQueryAccessInfo(
const String & quoted_database_name,

13
src/Interpreters/InterpreterAlterQuery.cpp
View File

@ -8,6 +8,7 @@
#include <Interpreters/AddDefaultDatabaseVisitor.h>
#include <Interpreters/Context.h>
#include <Interpreters/FunctionNameNormalizer.h>
#include <Interpreters/InterpreterCreateQuery.h>
#include <Interpreters/MutationsInterpreter.h>
#include <Interpreters/MutationsNonDeterministicHelpers.h>
#include <Interpreters/QueryLog.h>
@ -70,6 +71,13 @@ BlockIO InterpreterAlterQuery::execute()
BlockIO InterpreterAlterQuery::executeToTable(const ASTAlterQuery & alter)
{
for (auto & child : alter.command_list->children)
{
auto * command_ast = child->as<ASTAlterCommand>();
if (command_ast->sql_security)
InterpreterCreateQuery::processSQLSecurityOption(getContext(), command_ast->sql_security->as<ASTSQLSecurity &>());
}
BlockIO res;
if (!UserDefinedSQLFunctionFactory::instance().empty())
@ -482,6 +490,11 @@ AccessRightsElements InterpreterAlterQuery::getRequiredAccessForCommand(const AS
required_access.emplace_back(AccessType::ALTER_MODIFY_COMMENT, database, table);
break;
}
case ASTAlterCommand::MODIFY_SQL_SECURITY:
{
required_access.emplace_back(AccessType::ALTER_VIEW_MODIFY_SQL_SECURITY, database, table);
break;
}
}
return required_access;

64
src/Interpreters/InterpreterCreateQuery.cpp
View File

@ -2,6 +2,9 @@
#include <filesystem>
#include <Access/AccessControl.h>
#include <Access/User.h>
#include "Common/Exception.h"
#include <Common/StringUtils/StringUtils.h>
#include <Common/escapeForFileName.h>
@ -1095,6 +1098,12 @@ BlockIO InterpreterCreateQuery::createTable(ASTCreateQuery & create)
String current_database = getContext()->getCurrentDatabase();
auto database_name = create.database ? create.getDatabase() : current_database;
if (!create.sql_security && !getContext()->getServerSettings().ignore_empty_sql_security_in_create_view_query)
create.sql_security = std::make_shared<ASTSQLSecurity>();
if (create.sql_security)
processSQLSecurityOption(getContext(), create.sql_security->as<ASTSQLSecurity &>(), create.attach, create.is_materialized_view);
DDLGuardPtr ddl_guard;
// If this is a stub ATTACH query, read the query definition from the database
@ -1883,6 +1892,61 @@ void InterpreterCreateQuery::addColumnsDescriptionToCreateQueryIfNecessary(ASTCr
}
}
void InterpreterCreateQuery::processSQLSecurityOption(ContextPtr context_, ASTSQLSecurity & sql_security, bool is_attach, bool is_materialized_view)
{
/// If no SQL security is specified, apply default from default_*_view_sql_security setting.
if (!sql_security.type.has_value())
{
SQLSecurityType default_security;
if (is_materialized_view)
default_security = context_->getSettingsRef().default_materialized_view_sql_security;
else
default_security = context_->getSettingsRef().default_normal_view_sql_security;
if (default_security == SQLSecurityType::DEFINER)
{
String default_definer = context_->getSettingsRef().default_view_definer;
if (default_definer == "CURRENT_USER")
sql_security.is_definer_current_user = true;
else
sql_security.definer = std::make_shared<ASTUserNameWithHost>(default_definer);
}
sql_security.type = default_security;
}
/// Resolves `DEFINER = CURRENT_USER`. Can change the SQL security type if we try to resolve the user during the attachment.
const auto current_user_name = context_->getUserName();
if (sql_security.is_definer_current_user)
{
if (current_user_name.empty())
/// This can happen only when attaching a view for the first time after migration and with `CURRENT_USER` default.
if (is_materialized_view)
sql_security.type = SQLSecurityType::NONE;
else
sql_security.type = SQLSecurityType::INVOKER;
else if (sql_security.definer)
sql_security.definer->replace(current_user_name);
else
sql_security.definer = std::make_shared<ASTUserNameWithHost>(current_user_name);
}
/// Checks the permissions for the specified definer user.
if (sql_security.definer && !sql_security.is_definer_current_user && !is_attach)
{
const auto definer_name = sql_security.definer->toString();
/// Validate that the user exists.
context_->getAccessControl().getID<User>(definer_name);
if (definer_name != current_user_name)
context_->checkAccess(AccessType::SET_DEFINER, definer_name);
}
if (sql_security.type == SQLSecurityType::NONE && !is_attach)
context_->checkAccess(AccessType::ALLOW_SQL_SECURITY_NONE);
}
void registerInterpreterCreateQuery(InterpreterFactory & factory)
{
auto create_fn = [] (const InterpreterFactory::Arguments & args)

3
src/Interpreters/InterpreterCreateQuery.h
View File

@ -80,6 +80,9 @@ public:
void extendQueryLogElemImpl(QueryLogElement & elem, const ASTPtr & ast, ContextPtr) const override;
/// Check access right, validate definer statement and replace `CURRENT USER` with actual name.
static void processSQLSecurityOption(ContextPtr context_, ASTSQLSecurity & sql_security, bool is_attach = false, bool is_materialized_view = false);
private:
struct TableProperties
{

21
src/Interpreters/InterpreterInsertQuery.cpp
View File

@ -125,7 +125,10 @@ StoragePtr InterpreterInsertQuery::getTable(ASTInsertQuery & query)
Block InterpreterInsertQuery::getSampleBlock(
const ASTInsertQuery & query,
const StoragePtr & table,
const StorageMetadataPtr & metadata_snapshot) const
const StorageMetadataPtr & metadata_snapshot,
ContextPtr context_,
bool no_destination,
bool allow_materialized)
{
/// If the query does not include information about columns
if (!query.columns)
@ -139,7 +142,7 @@ Block InterpreterInsertQuery::getSampleBlock(
}
/// Form the block based on the column names from the query
const auto columns_ast = processColumnTransformers(getContext()->getCurrentDatabase(), table, metadata_snapshot, query.columns);
const auto columns_ast = processColumnTransformers(context_->getCurrentDatabase(), table, metadata_snapshot, query.columns);
Names names;
names.reserve(columns_ast->children.size());
for (const auto & identifier : columns_ast->children)
@ -148,7 +151,7 @@ Block InterpreterInsertQuery::getSampleBlock(
names.emplace_back(std::move(current_name));
}
return getSampleBlock(names, table, metadata_snapshot);
return getSampleBlock(names, table, metadata_snapshot, allow_materialized);
}
std::optional<Names> InterpreterInsertQuery::getInsertColumnNames() const
@ -173,7 +176,8 @@ std::optional<Names> InterpreterInsertQuery::getInsertColumnNames() const
Block InterpreterInsertQuery::getSampleBlock(
const Names & names,
const StoragePtr & table,
const StorageMetadataPtr & metadata_snapshot) const
const StorageMetadataPtr & metadata_snapshot,
bool allow_materialized)
{
Block table_sample_physical = metadata_snapshot->getSampleBlock();
Block table_sample_insertable = metadata_snapshot->getSampleBlockInsertable();
@ -260,7 +264,8 @@ Chain InterpreterInsertQuery::buildChain(
const StorageMetadataPtr & metadata_snapshot,
const Names & columns,
ThreadStatusesHolderPtr thread_status_holder,
std::atomic_uint64_t * elapsed_counter_ms)
std::atomic_uint64_t * elapsed_counter_ms,
bool check_access)
{
ProfileEvents::increment(ProfileEvents::InsertQueriesWithSubqueries);
ProfileEvents::increment(ProfileEvents::QueriesWithSubqueries);
@ -271,7 +276,9 @@ Chain InterpreterInsertQuery::buildChain(
if (!running_group)
running_group = std::make_shared<ThreadGroup>(getContext());
auto sample = getSampleBlock(columns, table, metadata_snapshot);
auto sample = getSampleBlock(columns, table, metadata_snapshot, allow_materialized);
if (check_access)
getContext()->checkAccess(AccessType::INSERT, table->getStorageID(), sample.getNames());
Chain sink = buildSink(table, metadata_snapshot, thread_status_holder, running_group, elapsed_counter_ms);
Chain chain = buildPreSinkChain(sink.getInputHeader(), table, metadata_snapshot, sample);
@ -397,7 +404,7 @@ BlockIO InterpreterInsertQuery::execute()
auto table_lock = table->lockForShare(getContext()->getInitialQueryId(), settings.lock_acquire_timeout);
auto metadata_snapshot = table->getInMemoryMetadataPtr();
auto query_sample_block = getSampleBlock(query, table, metadata_snapshot);
auto query_sample_block = getSampleBlock(query, table, metadata_snapshot, getContext(), no_destination, allow_materialized);
/// For table functions we check access while executing
/// getTable() -> ITableFunction::execute().

13
src/Interpreters/InterpreterInsertQuery.h
View File

@ -46,14 +46,21 @@ public:
const StorageMetadataPtr & metadata_snapshot,
const Names & columns,
ThreadStatusesHolderPtr thread_status_holder = {},
std::atomic_uint64_t * elapsed_counter_ms = nullptr);
std::atomic_uint64_t * elapsed_counter_ms = nullptr,
bool check_access = false);
static void extendQueryLogElemImpl(QueryLogElement & elem, ContextPtr context_);
void extendQueryLogElemImpl(QueryLogElement & elem, const ASTPtr & ast, ContextPtr context_) const override;
StoragePtr getTable(ASTInsertQuery & query);
Block getSampleBlock(const ASTInsertQuery & query, const StoragePtr & table, const StorageMetadataPtr & metadata_snapshot) const;
static Block getSampleBlock(
const ASTInsertQuery & query,
const StoragePtr & table,
const StorageMetadataPtr & metadata_snapshot,
ContextPtr context_,
bool no_destination = false,
bool allow_materialized = false);
bool supportsTransactions() const override { return true; }
@ -62,7 +69,7 @@ public:
bool shouldAddSquashingFroStorage(const StoragePtr & table) const;
private:
Block getSampleBlock(const Names & names, const StoragePtr & table, const StorageMetadataPtr & metadata_snapshot) const;
static Block getSampleBlock(const Names & names, const StoragePtr & table, const StorageMetadataPtr & metadata_snapshot, bool allow_materialized);
ASTPtr query_ptr;
const bool allow_materialized;

2
src/Interpreters/InterpreterSelectQuery.cpp
View File

@ -834,7 +834,7 @@ InterpreterSelectQuery::InterpreterSelectQuery(
if (query.prewhere() && !query.where())
analysis_result.prewhere_info->need_filter = true;
if (table_id && got_storage_from_query && !joined_tables.isLeftTableFunction())
if (table_id && got_storage_from_query && !joined_tables.isLeftTableFunction() && !options.ignore_access_check)
{
/// The current user should have the SELECT privilege. If this table_id is for a table
/// function we don't check access rights here because in this case they have been already

10
src/Interpreters/SelectQueryOptions.h
View File

@ -46,6 +46,10 @@ struct SelectQueryOptions
/// Bypass setting constraints for some internal queries such as projection ASTs.
bool ignore_setting_constraints = false;
/// Bypass access check for select query.
/// This allows to skip double access check in some specific cases (e.g. insert into table with materialized view)
bool ignore_access_check = false;
/// These two fields are used to evaluate shardNum() and shardCount() function when
/// prefer_localhost_replica == 1 and local instance is selected. They are needed because local
/// instance might have multiple shards and scalars can only hold one value.
@ -129,6 +133,12 @@ struct SelectQueryOptions
return *this;
}
SelectQueryOptions & ignoreAccessCheck(bool value = true)
{
ignore_access_check = value;
return *this;
}
SelectQueryOptions & setInternal(bool value = false)
{
is_internal = value;

5
src/Parsers/ASTAlterQuery.cpp
View File

@ -475,6 +475,11 @@ void ASTAlterCommand::formatImpl(const FormatSettings & settings, FormatState &
settings.ostr << (settings.hilite ? hilite_keyword : "") << " TO ";
rename_to->formatImpl(settings, state, frame);
}
else if (type == ASTAlterCommand::MODIFY_SQL_SECURITY)
{
settings.ostr << (settings.hilite ? hilite_keyword : "") << "MODIFY " << (settings.hilite ? hilite_none : "");
sql_security->formatImpl(settings, state, frame);
}
else if (type == ASTAlterCommand::APPLY_DELETED_MASK)
{
settings.ostr << (settings.hilite ? hilite_keyword : "") << "APPLY DELETED MASK" << (settings.hilite ? hilite_none : "");

4
src/Parsers/ASTAlterQuery.h
View File

@ -80,6 +80,7 @@ public:
MODIFY_DATABASE_SETTING,
MODIFY_COMMENT,
MODIFY_SQL_SECURITY,
};
Type type = NO_TYPE;
@ -162,6 +163,9 @@ public:
/// For MODIFY_QUERY
IAST * select = nullptr;
/// For MODIFY_SQL_SECURITY
IAST * sql_security = nullptr;
/// In ALTER CHANNEL, ADD, DROP, SUSPEND, RESUME, REFRESH, MODIFY queries, the list of live views is stored here
IAST * values = nullptr;

48
src/Parsers/ASTCreateQuery.cpp
View File

@ -12,6 +12,37 @@
namespace DB
{
void ASTSQLSecurity::formatImpl(const FormatSettings & settings, FormatState & state, FormatStateStacked frame) const
{
if (!type.has_value())
return;
if (definer || is_definer_current_user)
{
settings.ostr << (settings.hilite ? hilite_keyword : "") << "DEFINER" << (settings.hilite ? hilite_none : "");
settings.ostr << " = ";
if (definer)
definer->formatImpl(settings, state, frame);
else
settings.ostr << "CURRENT_USER";
settings.ostr << " ";
}
settings.ostr << (settings.hilite ? hilite_keyword : "") << "SQL SECURITY" << (settings.hilite ? hilite_none : "");
switch (*type)
{
case SQLSecurityType::INVOKER:
settings.ostr << " INVOKER";
break;
case SQLSecurityType::DEFINER:
settings.ostr << " DEFINER";
break;
case SQLSecurityType::NONE:
settings.ostr << " NONE";
break;
}
}
ASTPtr ASTStorage::clone() const
{
auto res = std::make_shared<ASTStorage>(*this);
@ -292,10 +323,9 @@ void ASTCreateQuery::formatQueryImpl(const FormatSettings & settings, FormatStat
else if (is_window_view)
what = "WINDOW VIEW";
settings.ostr
<< (settings.hilite ? hilite_keyword : "")
<< action << " "
<< (temporary ? "TEMPORARY " : "")
settings.ostr << (settings.hilite ? hilite_keyword : "") << action << (settings.hilite ? hilite_none : "");
settings.ostr << " ";
settings.ostr << (settings.hilite ? hilite_keyword : "") << (temporary ? "TEMPORARY " : "")
<< what << " "
<< (if_not_exists ? "IF NOT EXISTS " : "")
<< (settings.hilite ? hilite_none : "")
@ -444,10 +474,16 @@ void ASTCreateQuery::formatQueryImpl(const FormatSettings & settings, FormatStat
else if (is_create_empty)
settings.ostr << (settings.hilite ? hilite_keyword : "") << " EMPTY" << (settings.hilite ? hilite_none : "");
if (sql_security && sql_security->as<ASTSQLSecurity &>().type.has_value())
{
settings.ostr << settings.nl_or_ws;
sql_security->formatImpl(settings, state, frame);
}
if (select)
{
settings.ostr << (settings.hilite ? hilite_keyword : "") << " AS"
<< settings.nl_or_ws
settings.ostr << settings.nl_or_ws;
settings.ostr << (settings.hilite ? hilite_keyword : "") << "AS "
<< (comment ? "(" : "") << (settings.hilite ? hilite_none : "");
select->formatImpl(settings, state, frame);
settings.ostr << (settings.hilite ? hilite_keyword : "") << (comment ? ")" : "") << (settings.hilite ? hilite_none : "");

3
src/Parsers/ASTCreateQuery.h
View File

@ -5,6 +5,7 @@
#include <Parsers/ASTDictionary.h>
#include <Parsers/ASTDictionaryAttributeDeclaration.h>
#include <Parsers/ASTTableOverrides.h>
#include <Parsers/ASTSQLSecurity.h>
#include <Parsers/ASTRefreshStrategy.h>
#include <Interpreters/StorageID.h>
@ -15,6 +16,7 @@ class ASTFunction;
class ASTSetQuery;
class ASTSelectWithUnionQuery;
class ASTStorage : public IAST
{
public:
@ -111,6 +113,7 @@ public:
IAST * as_table_function = nullptr;
ASTSelectWithUnionQuery * select = nullptr;
IAST * comment = nullptr;
ASTPtr sql_security = nullptr;
ASTTableOverrideList * table_overrides = nullptr; /// For CREATE DATABASE with engines that automatically create tables

39
src/Parsers/ASTSQLSecurity.cpp Normal file
View File

@ -0,0 +1,39 @@
#include <Parsers/ASTSQLSecurity.h>
#include <IO/Operators.h>
namespace DB
{
void ASTSQLSecurity::formatImpl(const FormatSettings & settings, FormatState & state, FormatStateStacked frame) const
{
if (!type.has_value())
return;
if (definer || is_definer_current_user)
{
settings.ostr << (settings.hilite ? hilite_keyword : "") << "DEFINER" << (settings.hilite ? hilite_none : "");
settings.ostr << " = ";
if (definer)
definer->formatImpl(settings, state, frame);
else
settings.ostr << "CURRENT_USER";
settings.ostr << " ";
}
settings.ostr << (settings.hilite ? hilite_keyword : "") << "SQL SECURITY" << (settings.hilite ? hilite_none : "");
switch (*type)
{
case SQLSecurityType::INVOKER:
settings.ostr << " INVOKER";
break;
case SQLSecurityType::DEFINER:
settings.ostr << " DEFINER";
break;
case SQLSecurityType::NONE:
settings.ostr << " NONE";
break;
}
}
}

26
src/Parsers/ASTSQLSecurity.h Normal file
View File

@ -0,0 +1,26 @@
#pragma once
#include <Parsers/Access/ASTUserNameWithHost.h>
#include <Access/Common/SQLSecurityDefs.h>
namespace DB
{
/// DEFINER = <user_name | CURRENT_USER> SQL SECURITY <DEFINER | INVOKER | NONE>
/// If type was not set during parsing, the default type from settings will be used.
/// Currently supports only views.
class ASTSQLSecurity : public IAST
{
public:
bool is_definer_current_user{false};
std::shared_ptr<ASTUserNameWithHost> definer = nullptr;
std::optional<SQLSecurityType> type = std::nullopt;
String getID(char) const override { return "View SQL Security"; }
ASTPtr clone() const override { return std::make_shared<ASTSQLSecurity>(*this); }
void formatImpl(const FormatSettings & s, FormatState & state, FormatStateStacked frame) const override;
};
}

6
src/Parsers/Access/ASTUserNameWithHost.cpp
View File

@ -28,6 +28,12 @@ void ASTUserNameWithHost::concatParts()
host_pattern.clear();
}
void ASTUserNameWithHost::replace(const String name_)
{
base_name = name_;
host_pattern.clear();
}
void ASTUserNamesWithHost::formatImpl(const FormatSettings & settings, FormatState &, FormatStateStacked) const
{

1
src/Parsers/Access/ASTUserNameWithHost.h
View File

@ -27,6 +27,7 @@ public:
String getID(char) const override { return "UserNameWithHost"; }
ASTPtr clone() const override { return std::make_shared<ASTUserNameWithHost>(*this); }
void formatImpl(const FormatSettings & settings, FormatState &, FormatStateStacked) const override;
void replace(const String name_);
};

13
src/Parsers/ParserAlterQuery.cpp
View File

@ -40,6 +40,7 @@ bool ParserAlterCommand::parseImpl(Pos & pos, ASTPtr & node, Expected & expected
ParserKeyword s_modify_setting("MODIFY SETTING");
ParserKeyword s_reset_setting("RESET SETTING");
ParserKeyword s_modify_query("MODIFY QUERY");
ParserKeyword s_modify_sql_security("MODIFY SQL SECURITY");
ParserKeyword s_modify_refresh("MODIFY REFRESH");
ParserKeyword s_add_index("ADD INDEX");
@ -139,6 +140,7 @@ bool ParserAlterCommand::parseImpl(Pos & pos, ASTPtr & node, Expected & expected
/* allow_empty = */ false);
ParserNameList values_p;
ParserSelectWithUnionQuery select_p;
ParserSQLSecurity sql_security_p;
ParserRefreshStrategy refresh_p;
ParserTTLExpressionList parser_ttl_list;
@ -163,6 +165,7 @@ bool ParserAlterCommand::parseImpl(Pos & pos, ASTPtr & node, Expected & expected
ASTPtr command_select;
ASTPtr command_values;
ASTPtr command_rename_to;
ASTPtr command_sql_security;
if (with_round_bracket)
{
@ -861,6 +864,14 @@ bool ParserAlterCommand::parseImpl(Pos & pos, ASTPtr & node, Expected & expected
return false;
command->type = ASTAlterCommand::MODIFY_QUERY;
}
else if (s_modify_sql_security.ignore(pos, expected))
{
/// This is a hack so we can reuse parser from create and don't have to write `MODIFY SQL SECURITY SQL SECURITY INVOKER`
pos -= 2;
if (!sql_security_p.parse(pos, command_sql_security, expected))
return false;
command->type = ASTAlterCommand::MODIFY_SQL_SECURITY;
}
else if (s_modify_refresh.ignore(pos, expected))
{
if (!refresh_p.parse(pos, command->refresh, expected))
@ -935,6 +946,8 @@ bool ParserAlterCommand::parseImpl(Pos & pos, ASTPtr & node, Expected & expected
command->select = command->children.emplace_back(std::move(command_select)).get();
if (command_values)
command->values = command->children.emplace_back(std::move(command_values)).get();
if (command_sql_security)
command->sql_security = command->children.emplace_back(std::move(command_sql_security)).get();
if (command_rename_to)
command->rename_to = command->children.emplace_back(std::move(command_rename_to)).get();

79
src/Parsers/ParserCreateQuery.cpp
View File

@ -1,4 +1,5 @@
#include <IO/ReadHelpers.h>
#include <Parsers/Access/ParserUserNameWithHost.h>
#include <Parsers/ASTConstraintDeclaration.h>
#include <Parsers/ASTCreateQuery.h>
#include <Parsers/ASTExpressionList.h>
@ -84,6 +85,65 @@ bool ParserNestedTable::parseImpl(Pos & pos, ASTPtr & node, Expected & expected)
return true;
}
bool ParserSQLSecurity::parseImpl(Pos & pos, ASTPtr & node, Expected & expected)
{
ParserToken s_eq(TokenType::Equals);
ParserKeyword s_definer("DEFINER");
bool is_definer_current_user = false;
ASTPtr definer;
std::optional<SQLSecurityType> type;
while (true)
{
if (!definer && s_definer.ignore(pos, expected))
{
s_eq.ignore(pos, expected);
if (ParserKeyword{"CURRENT_USER"}.ignore(pos, expected))
is_definer_current_user = true;
else if (!ParserUserNameWithHost{}.parse(pos, definer, expected))
return false;
continue;
}
if (!type && ParserKeyword{"SQL SECURITY"}.ignore(pos, expected))
{
if (s_definer.ignore(pos, expected))
type = SQLSecurityType::DEFINER;
else if (ParserKeyword{"INVOKER"}.ignore(pos, expected))
type = SQLSecurityType::INVOKER;
else if (ParserKeyword{"NONE"}.ignore(pos, expected))
type = SQLSecurityType::NONE;
else
return false;
continue;
}
break;
}
if (!type)
{
if (is_definer_current_user || definer)
type = SQLSecurityType::DEFINER;
else
return false;
}
else if (type == SQLSecurityType::DEFINER && !definer)
is_definer_current_user = true;
auto result = std::make_shared<ASTSQLSecurity>();
result->is_definer_current_user = is_definer_current_user;
result->type = type;
if (definer)
result->definer = typeid_cast<std::shared_ptr<ASTUserNameWithHost>>(definer);
node = std::move(result);
return true;
}
bool ParserIdentifierWithParameters::parseImpl(Pos & pos, ASTPtr & node, Expected & expected)
{
@ -849,6 +909,7 @@ bool ParserCreateLiveViewQuery::parseImpl(Pos & pos, ASTPtr & node, Expected & e
ParserStorage storage_inner{ParserStorage::TABLE_ENGINE};
ParserTablePropertiesDeclarationList table_properties_p;
ParserSelectWithUnionQuery select_p;
ParserSQLSecurity sql_security_p;
ASTPtr table;
ASTPtr to_table;
@ -857,6 +918,7 @@ bool ParserCreateLiveViewQuery::parseImpl(Pos & pos, ASTPtr & node, Expected & e
ASTPtr as_table;
ASTPtr select;
ASTPtr live_view_periodic_refresh;
ASTPtr sql_security;
String cluster_str;
bool attach = false;
@ -873,6 +935,8 @@ bool ParserCreateLiveViewQuery::parseImpl(Pos & pos, ASTPtr & node, Expected & e
return false;
}
sql_security_p.parse(pos, sql_security, expected);
if (!s_live.ignore(pos, expected))
return false;
@ -925,6 +989,9 @@ bool ParserCreateLiveViewQuery::parseImpl(Pos & pos, ASTPtr & node, Expected & e
return false;
}
if (!sql_security && !sql_security_p.parse(pos, sql_security, expected))
sql_security = std::make_shared<ASTSQLSecurity>();
/// AS SELECT ...
if (!s_as.ignore(pos, expected))
return false;
@ -967,6 +1034,9 @@ bool ParserCreateLiveViewQuery::parseImpl(Pos & pos, ASTPtr & node, Expected & e
if (comment)
query->set(query->comment, comment);
if (sql_security)
query->sql_security = typeid_cast<std::shared_ptr<ASTSQLSecurity>>(sql_security);
return true;
}
@ -1384,6 +1454,7 @@ bool ParserCreateViewQuery::parseImpl(Pos & pos, ASTPtr & node, Expected & expec
ParserTablePropertiesDeclarationList table_properties_p;
ParserSelectWithUnionQuery select_p;
ParserNameList names_p;
ParserSQLSecurity sql_security_p;
ASTPtr table;
ASTPtr to_table;
@ -1393,6 +1464,7 @@ bool ParserCreateViewQuery::parseImpl(Pos & pos, ASTPtr & node, Expected & expec
ASTPtr as_database;
ASTPtr as_table;
ASTPtr select;
ASTPtr sql_security;
ASTPtr refresh_strategy;
String cluster_str;
@ -1418,6 +1490,8 @@ bool ParserCreateViewQuery::parseImpl(Pos & pos, ASTPtr & node, Expected & expec
replace_view = true;
}
sql_security_p.parse(pos, sql_security, expected);
if (!replace_view && s_materialized.ignore(pos, expected))
{
is_materialized_view = true;
@ -1510,6 +1584,9 @@ bool ParserCreateViewQuery::parseImpl(Pos & pos, ASTPtr & node, Expected & expec
}
}
if (!sql_security)
sql_security_p.parse(pos, sql_security, expected);
/// AS SELECT ...
if (!s_as.ignore(pos, expected))
return false;
@ -1552,6 +1629,8 @@ bool ParserCreateViewQuery::parseImpl(Pos & pos, ASTPtr & node, Expected & expec
query->set(query->refresh_strategy, refresh_strategy);
if (comment)
query->set(query->comment, comment);
if (sql_security)
query->sql_security = typeid_cast<std::shared_ptr<ASTSQLSecurity>>(sql_security);
tryGetIdentifierNameInto(as_database, query->as_database);
tryGetIdentifierNameInto(as_table, query->as_table);

8
src/Parsers/ParserCreateQuery.h
View File

@ -25,6 +25,14 @@ protected:
bool parseImpl(Pos & pos, ASTPtr & node, Expected & expected) override;
};
/** Parses sql security option. DEFINER = user_name SQL SECURITY DEFINER
*/
class ParserSQLSecurity : public IParserBase
{
protected:
const char * getName() const override { return "sql security"; }
bool parseImpl(Pos & pos, ASTPtr & node, Expected & expected) override;
};
/** Storage engine or Codec. For example:
* Memory()

12
src/Parsers/TokenIterator.h
View File

@ -62,6 +62,18 @@ public:
return *this;
}
ALWAYS_INLINE TokenIterator & operator-=(int value)
{
index -= value;
return *this;
}
ALWAYS_INLINE TokenIterator & operator+=(int value)
{
index += value;
return *this;
}
ALWAYS_INLINE bool operator<(const TokenIterator & rhs) const { return index < rhs.index; }
ALWAYS_INLINE bool operator<=(const TokenIterator & rhs) const { return index <= rhs.index; }
ALWAYS_INLINE bool operator==(const TokenIterator & rhs) const { return index == rhs.index; }

496
src/Processors/Transforms/buildPushingToViewsChain.cpp
View File

@ -187,6 +187,244 @@ private:
std::exception_ptr any_exception;
};
/// Generates one chain part for every view in buildPushingToViewsChain
std::optional<Chain> generateViewChain(
ContextPtr context,
const StorageID & view_id,
ThreadGroupPtr running_group,
Chain & result_chain,
ViewsDataPtr views_data,
ThreadStatusesHolderPtr thread_status_holder,
bool async_insert,
const Block & storage_header,
bool disable_deduplication_for_children)
{
auto view = DatabaseCatalog::instance().tryGetTable(view_id, context);
if (view == nullptr)
{
LOG_WARNING(
getLogger("PushingToViews"), "Trying to access table {} but it doesn't exist", view_id.getFullTableName());
return std::nullopt;
}
auto view_metadata_snapshot = view->getInMemoryMetadataPtr();
auto select_context = view_metadata_snapshot->getSQLSecurityOverriddenContext(context);
select_context->setQueryAccessInfo(context->getQueryAccessInfoPtr());
auto insert_context = Context::createCopy(select_context);
const auto & insert_settings = insert_context->getSettingsRef();
// Do not deduplicate insertions into MV if the main insertion is Ok
if (disable_deduplication_for_children)
{
insert_context->setSetting("insert_deduplicate", Field{false});
}
else if (insert_settings.update_insert_deduplication_token_in_dependent_materialized_views &&
!insert_settings.insert_deduplication_token.value.empty())
{
/** Update deduplication token passed to dependent MV with current view id. So it is possible to properly handle
* deduplication in complex INSERT flows.
*
* Example:
*
* landing -┬--> mv_1_1 ---> ds_1_1 ---> mv_2_1 --┬-> ds_2_1 ---> mv_3_1 ---> ds_3_1
* | |
* └--> mv_1_2 ---> ds_1_2 ---> mv_2_2 --┘
*
* Here we want to avoid deduplication for two different blocks generated from `mv_2_1` and `mv_2_2` that will
* be inserted into `ds_2_1`.
*
* We are forced to use view id instead of table id because there are some possible INSERT flows where no tables
* are involved.
*
* Example:
*
* landing -┬--> mv_1_1 --┬-> ds_1_1
* | |
* └--> mv_1_2 --┘
*
*/
auto insert_deduplication_token = insert_settings.insert_deduplication_token.value;
if (view_id.hasUUID())
insert_deduplication_token += "_" + toString(view_id.uuid);
else
insert_deduplication_token += "_" + view_id.getFullNameNotQuoted();
insert_context->setSetting("insert_deduplication_token", insert_deduplication_token);
}
// Processing of blocks for MVs is done block by block, and there will
// be no parallel reading after (plus it is not a costless operation)
select_context->setSetting("parallelize_output_from_storages", Field{false});
// Separate min_insert_block_size_rows/min_insert_block_size_bytes for children
if (insert_settings.min_insert_block_size_rows_for_materialized_views)
insert_context->setSetting("min_insert_block_size_rows", insert_settings.min_insert_block_size_rows_for_materialized_views.value);
if (insert_settings.min_insert_block_size_bytes_for_materialized_views)
insert_context->setSetting("min_insert_block_size_bytes", insert_settings.min_insert_block_size_bytes_for_materialized_views.value);
ASTPtr query;
Chain out;
/// We are creating a ThreadStatus per view to store its metrics individually
/// Since calling ThreadStatus() changes current_thread we save it and restore it after the calls
/// Later on, before doing any task related to a view, we'll switch to its ThreadStatus, do the work,
/// and switch back to the original thread_status.
auto * original_thread = current_thread;
SCOPE_EXIT({ current_thread = original_thread; });
current_thread = nullptr;
std::unique_ptr<ThreadStatus> view_thread_status_ptr = std::make_unique<ThreadStatus>(/*check_current_thread_on_destruction=*/ false);
/// Copy of a ThreadStatus should be internal.
view_thread_status_ptr->setInternalThread();
view_thread_status_ptr->attachToGroup(running_group);
auto * view_thread_status = view_thread_status_ptr.get();
views_data->thread_status_holder->thread_statuses.push_front(std::move(view_thread_status_ptr));
auto runtime_stats = std::make_unique<QueryViewsLogElement::ViewRuntimeStats>();
runtime_stats->target_name = view_id.getFullTableName();
runtime_stats->thread_status = view_thread_status;
runtime_stats->event_time = std::chrono::system_clock::now();
runtime_stats->event_status = QueryViewsLogElement::ViewStatus::EXCEPTION_BEFORE_START;
auto & type = runtime_stats->type;
auto & target_name = runtime_stats->target_name;
auto * view_counter_ms = &runtime_stats->elapsed_ms;
if (auto * materialized_view = dynamic_cast<StorageMaterializedView *>(view.get()))
{
auto lock = materialized_view->tryLockForShare(context->getInitialQueryId(), context->getSettingsRef().lock_acquire_timeout);
if (lock == nullptr)
{
// In case the materialized view is dropped/detached at this point, we register a warning and ignore it
assert(materialized_view->is_dropped || materialized_view->is_detached);
LOG_WARNING(
getLogger("PushingToViews"), "Trying to access table {} but it doesn't exist", view_id.getFullTableName());
return std::nullopt;
}
type = QueryViewsLogElement::ViewType::MATERIALIZED;
result_chain.addTableLock(lock);
StoragePtr inner_table = materialized_view->tryGetTargetTable();
/// If target table was dropped, ignore this materialized view.
if (!inner_table)
{
if (context->getSettingsRef().ignore_materialized_views_with_dropped_target_table)
return std::nullopt;
throw Exception(
ErrorCodes::UNKNOWN_TABLE,
"Target table '{}' of view '{}' doesn't exists. To ignore this view use setting "
"ignore_materialized_views_with_dropped_target_table",
materialized_view->getTargetTableId().getFullTableName(),
view_id.getFullTableName());
}
auto inner_table_id = inner_table->getStorageID();
auto inner_metadata_snapshot = inner_table->getInMemoryMetadataPtr();
const auto & select_query = view_metadata_snapshot->getSelectQuery();
if (select_query.select_table_id != views_data->source_storage_id)
{
/// It may happen if materialize view query was changed and it doesn't depend on this source table anymore.
/// See setting `allow_experimental_alter_materialized_view_structure`
LOG_DEBUG(
getLogger("PushingToViews"), "Table '{}' is not a source for view '{}' anymore, current source is '{}'",
select_query.select_table_id.getFullTableName(), view_id.getFullTableName(), views_data->source_storage_id);
return std::nullopt;
}
query = select_query.inner_query;
target_name = inner_table_id.getFullTableName();
Block header;
/// Get list of columns we get from select query.
if (select_context->getSettingsRef().allow_experimental_analyzer)
header = InterpreterSelectQueryAnalyzer::getSampleBlock(query, select_context);
else
header = InterpreterSelectQuery(query, select_context, SelectQueryOptions()).getSampleBlock();
/// Insert only columns returned by select.
Names insert_columns;
const auto & inner_table_columns = inner_metadata_snapshot->getColumns();
for (const auto & column : header)
{
/// But skip columns which storage doesn't have.
if (inner_table_columns.hasNotAlias(column.name))
insert_columns.emplace_back(column.name);
}
InterpreterInsertQuery interpreter(nullptr, insert_context, false, false, false);
out = interpreter.buildChain(inner_table, inner_metadata_snapshot, insert_columns, thread_status_holder, view_counter_ms, !materialized_view->hasInnerTable());
if (interpreter.shouldAddSquashingFroStorage(inner_table))
{
bool table_prefers_large_blocks = inner_table->prefersLargeBlocks();
const auto & settings = insert_context->getSettingsRef();
out.addSource(std::make_shared<SquashingChunksTransform>(
out.getInputHeader(),
table_prefers_large_blocks ? settings.min_insert_block_size_rows : settings.max_block_size,
table_prefers_large_blocks ? settings.min_insert_block_size_bytes : 0ULL));
}
auto counting = std::make_shared<CountingTransform>(out.getInputHeader(), current_thread, insert_context->getQuota());
counting->setProcessListElement(insert_context->getProcessListElement());
counting->setProgressCallback(insert_context->getProgressCallback());
out.addSource(std::move(counting));
out.addStorageHolder(view);
out.addStorageHolder(inner_table);
}
else if (auto * live_view = dynamic_cast<StorageLiveView *>(view.get()))
{
runtime_stats->type = QueryViewsLogElement::ViewType::LIVE;
query = live_view->getInnerQuery();
out = buildPushingToViewsChain(
view, view_metadata_snapshot, insert_context, ASTPtr(),
/* no_destination= */ true,
thread_status_holder, running_group, view_counter_ms, async_insert, storage_header);
}
else if (auto * window_view = dynamic_cast<StorageWindowView *>(view.get()))
{
runtime_stats->type = QueryViewsLogElement::ViewType::WINDOW;
query = window_view->getMergeableQuery();
out = buildPushingToViewsChain(
view, view_metadata_snapshot, insert_context, ASTPtr(),
/* no_destination= */ true,
thread_status_holder, running_group, view_counter_ms, async_insert);
}
else
out = buildPushingToViewsChain(
view, view_metadata_snapshot, insert_context, ASTPtr(),
/* no_destination= */ false,
thread_status_holder, running_group, view_counter_ms, async_insert);
views_data->views.emplace_back(ViewRuntimeData{
std::move(query),
out.getInputHeader(),
view_id,
nullptr,
std::move(runtime_stats)});
if (type == QueryViewsLogElement::ViewType::MATERIALIZED)
{
auto executing_inner_query = std::make_shared<ExecutingInnerQueryFromViewTransform>(
storage_header, views_data->views.back(), views_data);
executing_inner_query->setRuntimeData(view_thread_status, view_counter_ms);
out.addSource(std::move(executing_inner_query));
}
return out;
}
Chain buildPushingToViewsChain(
const StoragePtr & storage,
@ -231,259 +469,45 @@ Chain buildPushingToViewsChain(
auto table_id = storage->getStorageID();
auto views = DatabaseCatalog::instance().getDependentViews(table_id);
/// We need special context for materialized views insertions
ContextMutablePtr select_context;
ContextMutablePtr insert_context;
ViewsDataPtr views_data;
if (!views.empty())
{
select_context = Context::createCopy(context);
insert_context = Context::createCopy(context);
const auto & insert_settings = insert_context->getSettingsRef();
// Do not deduplicate insertions into MV if the main insertion is Ok
if (disable_deduplication_for_children)
{
insert_context->setSetting("insert_deduplicate", Field{false});
}
// Processing of blocks for MVs is done block by block, and there will
// be no parallel reading after (plus it is not a costless operation)
select_context->setSetting("parallelize_output_from_storages", Field{false});
// Separate min_insert_block_size_rows/min_insert_block_size_bytes for children
if (insert_settings.min_insert_block_size_rows_for_materialized_views)
insert_context->setSetting("min_insert_block_size_rows", insert_settings.min_insert_block_size_rows_for_materialized_views.value);
if (insert_settings.min_insert_block_size_bytes_for_materialized_views)
insert_context->setSetting("min_insert_block_size_bytes", insert_settings.min_insert_block_size_bytes_for_materialized_views.value);
views_data = std::make_shared<ViewsData>(thread_status_holder, select_context, table_id, metadata_snapshot, storage);
auto process_context = Context::createCopy(context); /// This context will be used in `process` function
views_data = std::make_shared<ViewsData>(thread_status_holder, process_context, table_id, metadata_snapshot, storage);
}
std::vector<Chain> chains;
for (const auto & view_id : views)
{
auto view = DatabaseCatalog::instance().tryGetTable(view_id, context);
if (view == nullptr)
try
{
LOG_WARNING(
getLogger("PushingToViews"), "Trying to access table {} but it doesn't exist", view_id.getFullTableName());
continue;
}
auto out = generateViewChain(
context, view_id, running_group, result_chain,
views_data, thread_status_holder, async_insert, storage_header, disable_deduplication_for_children);
auto view_metadata_snapshot = view->getInMemoryMetadataPtr();
ASTPtr query;
Chain out;
/// We are creating a ThreadStatus per view to store its metrics individually
/// Since calling ThreadStatus() changes current_thread we save it and restore it after the calls
/// Later on, before doing any task related to a view, we'll switch to its ThreadStatus, do the work,
/// and switch back to the original thread_status.
auto * original_thread = current_thread;
SCOPE_EXIT({ current_thread = original_thread; });
current_thread = nullptr;
std::unique_ptr<ThreadStatus> view_thread_status_ptr = std::make_unique<ThreadStatus>(/*check_current_thread_on_destruction=*/ false);
/// Copy of a ThreadStatus should be internal.
view_thread_status_ptr->setInternalThread();
view_thread_status_ptr->attachToGroup(running_group);
auto * view_thread_status = view_thread_status_ptr.get();
views_data->thread_status_holder->thread_statuses.push_front(std::move(view_thread_status_ptr));
auto runtime_stats = std::make_unique<QueryViewsLogElement::ViewRuntimeStats>();
runtime_stats->target_name = view_id.getFullTableName();
runtime_stats->thread_status = view_thread_status;
runtime_stats->event_time = std::chrono::system_clock::now();
runtime_stats->event_status = QueryViewsLogElement::ViewStatus::EXCEPTION_BEFORE_START;
auto & type = runtime_stats->type;
auto & target_name = runtime_stats->target_name;
auto * view_counter_ms = &runtime_stats->elapsed_ms;
const auto & insert_settings = insert_context->getSettingsRef();
ContextMutablePtr view_insert_context = insert_context;
if (!disable_deduplication_for_children &&
insert_settings.update_insert_deduplication_token_in_dependent_materialized_views &&
!insert_settings.insert_deduplication_token.value.empty())
{
/** Update deduplication token passed to dependent MV with current view id. So it is possible to properly handle
* deduplication in complex INSERT flows.
*
* Example:
*
* landing -┬--> mv_1_1 ---> ds_1_1 ---> mv_2_1 --┬-> ds_2_1 ---> mv_3_1 ---> ds_3_1
* | |
* └--> mv_1_2 ---> ds_1_2 ---> mv_2_2 --┘
*
* Here we want to avoid deduplication for two different blocks generated from `mv_2_1` and `mv_2_2` that will
* be inserted into `ds_2_1`.
*
* We are forced to use view id instead of table id because there are some possible INSERT flows where no tables
* are involved.
*
* Example:
*
* landing -┬--> mv_1_1 --┬-> ds_1_1
* | |
* └--> mv_1_2 --┘
*
*/
auto insert_deduplication_token = insert_settings.insert_deduplication_token.value;
if (view_id.hasUUID())
insert_deduplication_token += "_" + toString(view_id.uuid);
else
insert_deduplication_token += "_" + view_id.getFullNameNotQuoted();
view_insert_context = Context::createCopy(insert_context);
view_insert_context->setSetting("insert_deduplication_token", insert_deduplication_token);
}
if (auto * materialized_view = dynamic_cast<StorageMaterializedView *>(view.get()))
{
auto lock = materialized_view->tryLockForShare(context->getInitialQueryId(), context->getSettingsRef().lock_acquire_timeout);
if (lock == nullptr)
{
// In case the materialized view is dropped/detached at this point, we register a warning and ignore it
assert(materialized_view->is_dropped || materialized_view->is_detached);
LOG_WARNING(
getLogger("PushingToViews"), "Trying to access table {} but it doesn't exist", view_id.getFullTableName());
if (!out.has_value())
continue;
}
type = QueryViewsLogElement::ViewType::MATERIALIZED;
result_chain.addTableLock(lock);
chains.emplace_back(std::move(*out));
StoragePtr inner_table = materialized_view->tryGetTargetTable();
/// If target table was dropped, ignore this materialized view.
if (!inner_table)
/// Add the view to the query access info so it can appear in system.query_log
/// hasQueryContext - for materialized tables with background replication process query context is not added
if (!no_destination && context->hasQueryContext())
{
if (context->getSettingsRef().ignore_materialized_views_with_dropped_target_table)
continue;
context->getQueryContext()->addQueryAccessInfo(
backQuoteIfNeed(view_id.getDatabaseName()),
views_data->views.back().runtime_stats->target_name,
/*column_names=*/ {});
throw Exception(
ErrorCodes::UNKNOWN_TABLE,
"Target table '{}' of view '{}' doesn't exists. To ignore this view use setting "
"ignore_materialized_views_with_dropped_target_table",
materialized_view->getTargetTableId().getFullTableName(),
view_id.getFullTableName());
context->getQueryContext()->addViewAccessInfo(view_id.getFullTableName());
}
auto inner_table_id = inner_table->getStorageID();
auto inner_metadata_snapshot = inner_table->getInMemoryMetadataPtr();
const auto & select_query = view_metadata_snapshot->getSelectQuery();
if (select_query.select_table_id != table_id)
{
/// It may happen if materialize view query was changed and it doesn't depend on this source table anymore.
/// See setting `allow_experimental_alter_materialized_view_structure`
LOG_DEBUG(
getLogger("PushingToViews"), "Table '{}' is not a source for view '{}' anymore, current source is '{}'",
select_query.select_table_id.getFullTableName(), view_id.getFullTableName(), table_id);
continue;
}
query = select_query.inner_query;
target_name = inner_table_id.getFullTableName();
Block header;
/// Get list of columns we get from select query.
if (select_context->getSettingsRef().allow_experimental_analyzer)
header = InterpreterSelectQueryAnalyzer::getSampleBlock(query, select_context);
else
header = InterpreterSelectQuery(query, select_context, SelectQueryOptions()).getSampleBlock();
/// Insert only columns returned by select.
Names insert_columns;
const auto & inner_table_columns = inner_metadata_snapshot->getColumns();
for (const auto & column : header)
{
/// But skip columns which storage doesn't have.
if (inner_table_columns.hasNotAlias(column.name))
insert_columns.emplace_back(column.name);
}
InterpreterInsertQuery interpreter(nullptr, view_insert_context, false, false, false);
out = interpreter.buildChain(inner_table, inner_metadata_snapshot, insert_columns, thread_status_holder, view_counter_ms);
if (interpreter.shouldAddSquashingFroStorage(inner_table))
{
bool table_prefers_large_blocks = inner_table->prefersLargeBlocks();
const auto & settings = view_insert_context->getSettingsRef();
out.addSource(std::make_shared<SquashingChunksTransform>(
out.getInputHeader(),
table_prefers_large_blocks ? settings.min_insert_block_size_rows : settings.max_block_size,
table_prefers_large_blocks ? settings.min_insert_block_size_bytes : 0ULL));
}
auto counting = std::make_shared<CountingTransform>(out.getInputHeader(), current_thread, view_insert_context->getQuota());
counting->setProcessListElement(view_insert_context->getProcessListElement());
counting->setProgressCallback(view_insert_context->getProgressCallback());
out.addSource(std::move(counting));
out.addStorageHolder(view);
out.addStorageHolder(inner_table);
}
else if (auto * live_view = dynamic_cast<StorageLiveView *>(view.get()))
catch (const Exception & e)
{
runtime_stats->type = QueryViewsLogElement::ViewType::LIVE;
query = live_view->getInnerQuery(); // Used only to log in system.query_views_log
out = buildPushingToViewsChain(
view, view_metadata_snapshot, view_insert_context, ASTPtr(),
/* no_destination= */ true,
thread_status_holder, running_group, view_counter_ms, async_insert, storage_header);
}
else if (auto * window_view = dynamic_cast<StorageWindowView *>(view.get()))
{
runtime_stats->type = QueryViewsLogElement::ViewType::WINDOW;
query = window_view->getMergeableQuery(); // Used only to log in system.query_views_log
out = buildPushingToViewsChain(
view, view_metadata_snapshot, view_insert_context, ASTPtr(),
/* no_destination= */ true,
thread_status_holder, running_group, view_counter_ms, async_insert);
}
else
out = buildPushingToViewsChain(
view, view_metadata_snapshot, view_insert_context, ASTPtr(),
/* no_destination= */ false,
thread_status_holder, running_group, view_counter_ms, async_insert);
views_data->views.emplace_back(ViewRuntimeData{
std::move(query),
out.getInputHeader(),
view_id,
nullptr,
std::move(runtime_stats)});
if (type == QueryViewsLogElement::ViewType::MATERIALIZED)
{
auto executing_inner_query = std::make_shared<ExecutingInnerQueryFromViewTransform>(
storage_header, views_data->views.back(), views_data);
executing_inner_query->setRuntimeData(view_thread_status, view_counter_ms);
out.addSource(std::move(executing_inner_query));
}
chains.emplace_back(std::move(out));
/// Add the view to the query access info so it can appear in system.query_log
/// hasQueryContext - for materialized tables with background replication process query context is not added
if (!no_destination && context->hasQueryContext())
{
context->getQueryContext()->addQueryAccessInfo(
backQuoteIfNeed(view_id.getDatabaseName()),
views_data->views.back().runtime_stats->target_name,
/*column_names=*/ {});
context->getQueryContext()->addViewAccessInfo(view_id.getFullTableName());
LOG_ERROR(&Poco::Logger::get("PushingToViews"), "Failed to push block to view {}, {}", view_id, e.message());
if (!context->getSettingsRef().materialized_views_ignore_errors)
throw;
}
}
@ -579,12 +603,12 @@ static QueryPipeline process(Block block, ViewRuntimeData & view, const ViewsDat
if (local_context->getSettingsRef().allow_experimental_analyzer)
{
InterpreterSelectQueryAnalyzer interpreter(view.query, local_context, local_context->getViewSource(), SelectQueryOptions());
InterpreterSelectQueryAnalyzer interpreter(view.query, local_context, local_context->getViewSource(), SelectQueryOptions().ignoreAccessCheck());
pipeline = interpreter.buildQueryPipeline();
}
else
{
InterpreterSelectQuery interpreter(view.query, local_context, SelectQueryOptions());
InterpreterSelectQuery interpreter(view.query, local_context, SelectQueryOptions().ignoreAccessCheck());
pipeline = interpreter.buildQueryPipeline();
}

10
src/Storages/AlterCommands.cpp
View File

@ -442,6 +442,14 @@ std::optional<AlterCommand> AlterCommand::parse(const ASTAlterCommand * command_
command.if_exists = command_ast->if_exists;
return command;
}
else if (command_ast->type == ASTAlterCommand::MODIFY_SQL_SECURITY)
{
AlterCommand command;
command.ast = command_ast->clone();
command.type = AlterCommand::MODIFY_SQL_SECURITY;
command.sql_security = command_ast->sql_security->clone();
return command;
}
else
return {};
}
@ -854,6 +862,8 @@ void AlterCommand::apply(StorageInMemoryMetadata & metadata, ContextPtr context)
for (auto & index : metadata.secondary_indices)
rename_visitor.visit(index.definition_ast);
}
else if (type == MODIFY_SQL_SECURITY)
metadata.setSQLSecurity(sql_security->as<ASTSQLSecurity &>());
else
throw Exception(ErrorCodes::LOGICAL_ERROR, "Wrong parameter type in ALTER query");
}

4
src/Storages/AlterCommands.h
View File

@ -50,6 +50,7 @@ struct AlterCommand
MODIFY_DATABASE_SETTING,
COMMENT_TABLE,
REMOVE_SAMPLE_BY,
MODIFY_SQL_SECURITY,
};
/// Which property user wants to remove from column
@ -147,6 +148,9 @@ struct AlterCommand
/// For MODIFY_QUERY
ASTPtr select = nullptr;
/// For MODIFY_SQL_SECURITY
ASTPtr sql_security = nullptr;
/// For MODIFY_REFRESH
ASTPtr refresh = nullptr;

72
src/Storages/StorageInMemoryMetadata.cpp
View File

@ -1,5 +1,8 @@
#include <Storages/StorageInMemoryMetadata.h>
#include <Access/AccessControl.h>
#include <Access/User.h>
#include <Common/HashTable/HashMap.h>
#include <Common/HashTable/HashSet.h>
#include <Common/quoteString.h>
@ -7,6 +10,7 @@
#include <Core/ColumnWithTypeAndName.h>
#include <DataTypes/NestedUtils.h>
#include <DataTypes/DataTypeEnum.h>
#include <Interpreters/Context.h>
#include <IO/ReadBufferFromString.h>
#include <IO/ReadHelpers.h>
#include <IO/Operators.h>
@ -23,6 +27,7 @@ namespace ErrorCodes
extern const int NOT_FOUND_COLUMN_IN_BLOCK;
extern const int TYPE_MISMATCH;
extern const int EMPTY_LIST_OF_COLUMNS_PASSED;
extern const int LOGICAL_ERROR;
}
StorageInMemoryMetadata::StorageInMemoryMetadata(const StorageInMemoryMetadata & other)
@ -41,6 +46,8 @@ StorageInMemoryMetadata::StorageInMemoryMetadata(const StorageInMemoryMetadata &
, settings_changes(other.settings_changes ? other.settings_changes->clone() : nullptr)
, select(other.select)
, refresh(other.refresh ? other.refresh->clone() : nullptr)
, definer(other.definer)
, sql_security_type(other.sql_security_type)
, comment(other.comment)
, metadata_version(other.metadata_version)
{
@ -71,6 +78,8 @@ StorageInMemoryMetadata & StorageInMemoryMetadata::operator=(const StorageInMemo
settings_changes.reset();
select = other.select;
refresh = other.refresh ? other.refresh->clone() : nullptr;
definer = other.definer;
sql_security_type = other.sql_security_type;
comment = other.comment;
metadata_version = other.metadata_version;
return *this;
@ -81,6 +90,69 @@ void StorageInMemoryMetadata::setComment(const String & comment_)
comment = comment_;
}
void StorageInMemoryMetadata::setSQLSecurity(const ASTSQLSecurity & sql_security)
{
if (sql_security.definer)
definer = sql_security.definer->toString();
sql_security_type = sql_security.type;
}
UUID StorageInMemoryMetadata::getDefinerID(DB::ContextPtr context) const
{
if (!definer)
{
if (const auto definer_id = context->getUserID())
return *definer_id;
throw Exception(ErrorCodes::LOGICAL_ERROR, "No user in context for sub query execution.");
}
const auto & access_control = context->getAccessControl();
return access_control.getID<User>(*definer);
}
ContextMutablePtr StorageInMemoryMetadata::getSQLSecurityOverriddenContext(ContextPtr context) const
{
if (!sql_security_type)
return Context::createCopy(context);
if (sql_security_type == SQLSecurityType::INVOKER)
return Context::createCopy(context);
auto new_context = Context::createCopy(context->getGlobalContext());
new_context->setClientInfo(context->getClientInfo());
new_context->makeQueryContext();
const auto & database = context->getCurrentDatabase();
if (!database.empty())
new_context->setCurrentDatabase(database);
new_context->setInsertionTable(context->getInsertionTable(), context->getInsertionTableColumnNames());
new_context->setProgressCallback(context->getProgressCallback());
new_context->setProcessListElement(context->getProcessListElement());
if (context->getCurrentTransaction())
new_context->setCurrentTransaction(context->getCurrentTransaction());
if (context->getZooKeeperMetadataTransaction())
new_context->initZooKeeperMetadataTransaction(context->getZooKeeperMetadataTransaction());
if (sql_security_type == SQLSecurityType::NONE)
{
new_context->applySettingsChanges(context->getSettingsRef().changes());
return new_context;
}
new_context->setUser(getDefinerID(context));
auto changed_settings = context->getSettingsRef().changes();
new_context->clampToSettingsConstraints(changed_settings, SettingSource::QUERY);
new_context->applySettingsChanges(changed_settings);
return new_context;
}
void StorageInMemoryMetadata::setColumns(ColumnsDescription columns_)
{
if (columns_.getAllPhysical().empty())

19
src/Storages/StorageInMemoryMetadata.h
View File

@ -1,5 +1,7 @@
#pragma once
#include <Parsers/Access/ASTUserNameWithHost.h>
#include <Parsers/ASTCreateQuery.h>
#include <Parsers/IAST_fwd.h>
#include <Storages/ColumnDependency.h>
#include <Storages/ColumnsDescription.h>
@ -51,6 +53,14 @@ struct StorageInMemoryMetadata
/// Materialized view REFRESH parameters.
ASTPtr refresh;
/// DEFINER <user_name>. Allows to specify a definer of the table.
/// Supported for MaterializedView and View.
std::optional<String> definer;
/// SQL SECURITY <DEFINER | INVOKER | NONE>
/// Supported for MaterializedView and View.
std::optional<SQLSecurityType> sql_security_type;
String comment;
/// Version of metadata. Managed properly by ReplicatedMergeTree only
@ -105,6 +115,15 @@ struct StorageInMemoryMetadata
/// Get copy of current metadata with metadata_version_
StorageInMemoryMetadata withMetadataVersion(int32_t metadata_version_) const;
/// Sets SQL security for the storage.
void setSQLSecurity(const ASTSQLSecurity & sql_security);
UUID getDefinerID(ContextPtr context) const;
/// Returns a copy of the context with the correct user from SQL security options.
/// If the SQL security wasn't set, this is equivalent to `Context::createCopy(context)`.
/// The context from this function must be used every time whenever views execute any read/write operations or subqueries.
ContextMutablePtr getSQLSecurityOverriddenContext(ContextPtr context) const;
/// Returns combined set of columns
const ColumnsDescription & getColumns() const;

59
src/Storages/StorageMaterializedView.cpp
View File

@ -39,6 +39,7 @@ namespace ErrorCodes
extern const int BAD_ARGUMENTS;
extern const int NOT_IMPLEMENTED;
extern const int INCORRECT_QUERY;
extern const int QUERY_IS_NOT_SUPPORTED_IN_MATERIALIZED_VIEW;
extern const int TOO_MANY_MATERIALIZED_VIEWS;
}
@ -77,6 +78,11 @@ StorageMaterializedView::StorageMaterializedView(
{
StorageInMemoryMetadata storage_metadata;
storage_metadata.setColumns(columns_);
if (query.sql_security)
storage_metadata.setSQLSecurity(query.sql_security->as<ASTSQLSecurity &>());
if (storage_metadata.sql_security_type == SQLSecurityType::INVOKER)
throw Exception(ErrorCodes::QUERY_IS_NOT_SUPPORTED_IN_MATERIALIZED_VIEW, "SQL SECURITY INVOKER can't be specified for MATERIALIZED VIEW");
if (!query.select)
throw Exception(ErrorCodes::INCORRECT_QUERY, "SELECT query is not specified for {}", getName());
@ -175,19 +181,28 @@ void StorageMaterializedView::read(
const size_t max_block_size,
const size_t num_streams)
{
auto context = getInMemoryMetadataPtr()->getSQLSecurityOverriddenContext(local_context);
auto storage = getTargetTable();
auto lock = storage->lockForShare(local_context->getCurrentQueryId(), local_context->getSettingsRef().lock_acquire_timeout);
auto lock = storage->lockForShare(context->getCurrentQueryId(), context->getSettingsRef().lock_acquire_timeout);
auto target_metadata_snapshot = storage->getInMemoryMetadataPtr();
auto target_storage_snapshot = storage->getStorageSnapshot(target_metadata_snapshot, local_context);
auto target_storage_snapshot = storage->getStorageSnapshot(target_metadata_snapshot, context);
if (query_info.order_optimizer)
query_info.input_order_info = query_info.order_optimizer->getInputOrder(target_metadata_snapshot, local_context);
query_info.input_order_info = query_info.order_optimizer->getInputOrder(target_metadata_snapshot, context);
storage->read(query_plan, column_names, target_storage_snapshot, query_info, local_context, processed_stage, max_block_size, num_streams);
if (!getInMemoryMetadataPtr()->select.select_table_id.empty())
context->checkAccess(AccessType::SELECT, getInMemoryMetadataPtr()->select.select_table_id, column_names);
auto storage_id = storage->getStorageID();
/// We don't need to check access if the inner table was created automatically.
if (!has_inner_table && !storage_id.empty())
context->checkAccess(AccessType::SELECT, storage_id, column_names);
storage->read(query_plan, column_names, target_storage_snapshot, query_info, context, processed_stage, max_block_size, num_streams);
if (query_plan.isInitialized())
{
auto mv_header = getHeaderForProcessingStage(column_names, storage_snapshot, query_info, local_context, processed_stage);
auto mv_header = getHeaderForProcessingStage(column_names, storage_snapshot, query_info, context, processed_stage);
auto target_header = query_plan.getCurrentDataStream().header;
/// No need to convert columns that does not exists in MV
@ -222,11 +237,20 @@ void StorageMaterializedView::read(
SinkToStoragePtr StorageMaterializedView::write(const ASTPtr & query, const StorageMetadataPtr & /*metadata_snapshot*/, ContextPtr local_context, bool async_insert)
{
auto context = getInMemoryMetadataPtr()->getSQLSecurityOverriddenContext(local_context);
auto storage = getTargetTable();
auto lock = storage->lockForShare(local_context->getCurrentQueryId(), local_context->getSettingsRef().lock_acquire_timeout);
auto lock = storage->lockForShare(context->getCurrentQueryId(), context->getSettingsRef().lock_acquire_timeout);
auto metadata_snapshot = storage->getInMemoryMetadataPtr();
auto sink = storage->write(query, metadata_snapshot, local_context, async_insert);
auto storage_id = storage->getStorageID();
/// We don't need to check access if the inner table was created automatically.
if (!has_inner_table && !storage_id.empty())
{
auto query_sample_block = InterpreterInsertQuery::getSampleBlock(query->as<ASTInsertQuery &>(), storage, metadata_snapshot, context);
context->checkAccess(AccessType::INSERT, storage_id, query_sample_block.getNames());
}
auto sink = storage->write(query, metadata_snapshot, context, async_insert);
sink->addTableLock(lock);
return sink;
@ -297,7 +321,7 @@ bool StorageMaterializedView::optimize(
std::tuple<ContextMutablePtr, std::shared_ptr<ASTInsertQuery>> StorageMaterializedView::prepareRefresh() const
{
auto refresh_context = Context::createCopy(getContext());
auto refresh_context = getInMemoryMetadataPtr()->getSQLSecurityOverriddenContext(getContext());
/// Generate a random query id.
refresh_context->setCurrentQueryId("");
@ -378,15 +402,24 @@ void StorageMaterializedView::checkAlterIsPossible(const AlterCommands & command
{
for (const auto & command : commands)
{
if (command.isCommentAlter())
if (command.type == AlterCommand::MODIFY_SQL_SECURITY)
{
if (command.sql_security->as<ASTSQLSecurity &>().type == SQLSecurityType::INVOKER)
throw Exception(ErrorCodes::QUERY_IS_NOT_SUPPORTED_IN_MATERIALIZED_VIEW, "SQL SECURITY INVOKER can't be specified for MATERIALIZED VIEW");
continue;
if (command.type == AlterCommand::MODIFY_QUERY)
}
else if (command.isCommentAlter())
continue;
if (command.type == AlterCommand::MODIFY_REFRESH && refresher)
else if (command.type == AlterCommand::MODIFY_QUERY)
continue;
else if (command.type == AlterCommand::MODIFY_REFRESH && refresher)
continue;
throw Exception(ErrorCodes::NOT_IMPLEMENTED, "Alter of type '{}' is not supported by storage {}",
command.type, getName());
command.type, getName());
}
}
void StorageMaterializedView::checkMutationIsPossible(const MutationCommands & commands, const Settings & settings) const

23
src/Storages/StorageView.cpp
View File

@ -12,6 +12,7 @@
#include <Parsers/ASTSubquery.h>
#include <Parsers/ASTTablesInSelectQuery.h>
#include <Storages/AlterCommands.h>
#include <Storages/StorageView.h>
#include <Storages/StorageFactory.h>
#include <Storages/SelectQueryDescription.h>
@ -35,6 +36,7 @@ namespace ErrorCodes
{
extern const int INCORRECT_QUERY;
extern const int LOGICAL_ERROR;
extern const int NOT_IMPLEMENTED;
}
@ -90,10 +92,10 @@ bool hasJoin(const ASTSelectWithUnionQuery & ast)
/** There are no limits on the maximum size of the result for the view.
* Since the result of the view is not the result of the entire query.
*/
ContextPtr getViewContext(ContextPtr context)
ContextPtr getViewContext(ContextPtr context, const StorageSnapshotPtr & storage_snapshot)
{
auto view_context = Context::createCopy(context);
Settings view_settings = context->getSettings();
auto view_context = storage_snapshot->metadata->getSQLSecurityOverriddenContext(context);
Settings view_settings = view_context->getSettings();
view_settings.max_result_rows = 0;
view_settings.max_result_bytes = 0;
view_settings.extremes = false;
@ -122,6 +124,8 @@ StorageView::StorageView(
storage_metadata.setColumns(columns_);
storage_metadata.setComment(comment);
if (query.sql_security)
storage_metadata.setSQLSecurity(query.sql_security->as<ASTSQLSecurity &>());
if (!query.select)
throw Exception(ErrorCodes::INCORRECT_QUERY, "SELECT query is not specified for {}", getName());
@ -160,13 +164,13 @@ void StorageView::read(
if (context->getSettingsRef().allow_experimental_analyzer)
{
InterpreterSelectQueryAnalyzer interpreter(current_inner_query, getViewContext(context), options);
InterpreterSelectQueryAnalyzer interpreter(current_inner_query, getViewContext(context, storage_snapshot), options);
interpreter.addStorageLimits(*query_info.storage_limits);
query_plan = std::move(interpreter).extractQueryPlan();
}
else
{
InterpreterSelectWithUnionQuery interpreter(current_inner_query, getViewContext(context), options, column_names);
InterpreterSelectWithUnionQuery interpreter(current_inner_query, getViewContext(context, storage_snapshot), options, column_names);
interpreter.addStorageLimits(*query_info.storage_limits);
interpreter.buildQueryPlan(query_plan);
}
@ -282,6 +286,15 @@ ASTPtr StorageView::restoreViewName(ASTSelectQuery & select_query, const ASTPtr
return subquery->children[0];
}
void StorageView::checkAlterIsPossible(const AlterCommands & commands, ContextPtr /* local_context */) const
{
for (const auto & command : commands)
{
if (!command.isCommentAlter() && command.type != AlterCommand::MODIFY_SQL_SECURITY)
throw Exception(ErrorCodes::NOT_IMPLEMENTED, "Alter of type '{}' is not supported by storage {}", command.type, getName());
}
}
void registerStorageView(StorageFactory & factory)
{
factory.registerStorage("View", [](const StorageFactory::Arguments & args)

2
src/Storages/StorageView.h
View File

@ -26,6 +26,8 @@ public:
bool supportsSampling() const override { return true; }
bool supportsFinal() const override { return true; }
void checkAlterIsPossible(const AlterCommands & commands, ContextPtr local_context) const override;
void read(
QueryPlan & query_plan,
const Names & column_names,

2
src/Storages/System/StorageSystemPrivileges.cpp
View File

@ -29,6 +29,7 @@ namespace
VIEW,
COLUMN,
NAMED_COLLECTION,
USER_NAME,
};
DataTypeEnum8::Values getLevelEnumValues()
@ -41,6 +42,7 @@ namespace
enum_values.emplace_back("VIEW", static_cast<Int8>(VIEW));
enum_values.emplace_back("COLUMN", static_cast<Int8>(COLUMN));
enum_values.emplace_back("NAMED_COLLECTION", static_cast<Int8>(NAMED_COLLECTION));
enum_values.emplace_back("USER_NAME", static_cast<Int8>(USER_NAME));
return enum_values;
}
}

35
src/Storages/System/attachInformationSchemaTables.cpp
View File

@ -35,8 +35,9 @@ static constexpr std::string_view schemata = R"(
`DEFAULT_CHARACTER_SET_SCHEMA` Nullable(String),
`DEFAULT_CHARACTER_SET_NAME` Nullable(String),
`SQL_PATH` Nullable(String)
) AS
SELECT
)
SQL SECURITY INVOKER
AS SELECT
name AS catalog_name,
name AS schema_name,
'default' AS schema_owner,
@ -73,8 +74,9 @@ static constexpr std::string_view tables = R"(
`DATA_LENGTH` Nullable(UInt64),
`TABLE_COLLATION` Nullable(String),
`TABLE_COMMENT` Nullable(String)
) AS
SELECT
)
SQL SECURITY INVOKER
AS SELECT
database AS table_catalog,
database AS table_schema,
name AS table_name,
@ -122,8 +124,9 @@ static constexpr std::string_view views = R"(
`IS_TRIGGER_UPDATABLE` Enum8('NO' = 0, 'YES' = 1),
`IS_TRIGGER_DELETABLE` Enum8('NO' = 0, 'YES' = 1),
`IS_TRIGGER_INSERTABLE_INTO` Enum8('NO' = 0, 'YES' = 1)
) AS
SELECT
)
SQL SECURITY INVOKER
AS SELECT
database AS table_catalog,
database AS table_schema,
name AS table_name,
@ -203,8 +206,9 @@ static constexpr std::string_view columns = R"(
`EXTRA` Nullable(String),
`COLUMN_COMMENT` String,
`COLUMN_TYPE` String
) AS
SELECT
)
SQL SECURITY INVOKER
AS SELECT
database AS table_catalog,
database AS table_schema,
table AS table_name,
@ -291,8 +295,9 @@ static constexpr std::string_view key_column_usage = R"(
`REFERENCED_TABLE_SCHEMA` Nullable(String),
`REFERENCED_TABLE_NAME` Nullable(String),
`REFERENCED_COLUMN_NAME` Nullable(String)
) AS
SELECT
)
SQL SECURITY INVOKER
AS SELECT
'def' AS constraint_catalog,
database AS constraint_schema,
'PRIMARY' AS constraint_name,
@ -346,8 +351,9 @@ static constexpr std::string_view referential_constraints = R"(
`DELETE_RULE` String,
`TABLE_NAME` String,
`REFERENCED_TABLE_NAME` String
) AS
SELECT
)
SQL SECURITY INVOKER
AS SELECT
'' AS constraint_catalog,
NULL AS constraint_name,
'' AS constraint_schema,
@ -412,8 +418,9 @@ static constexpr std::string_view statistics = R"(
`INDEX_COMMENT` String,
`IS_VISIBLE` String,
`EXPRESSION` Nullable(String)
) AS
SELECT
)
SQL SECURITY INVOKER
AS SELECT
'' AS table_catalog,
'' AS table_schema,
'' AS table_name,

1
tests/integration/test_postgresql_replica_database_engine_2/test.py
View File

@ -723,6 +723,7 @@ def test_materialized_view(started_cluster):
pg_manager.execute(f"INSERT INTO test_table SELECT 3, 4")
check_tables_are_synchronized(instance, "test_table")
assert "1\t2\n3\t4" == instance.query("SELECT * FROM mv ORDER BY 1, 2").strip()
instance.query("DROP VIEW mv")
pg_manager.drop_materialized_db()

2
tests/queries/0_stateless/00599_create_view_with_subquery.reference
View File

@ -1 +1 @@
CREATE VIEW default.test_view_00599\n(\n `id` UInt64\n) AS\nSELECT *\nFROM default.test_00599\nWHERE id = (\n SELECT 1\n)
CREATE VIEW default.test_view_00599\n(\n `id` UInt64\n)\nAS SELECT *\nFROM default.test_00599\nWHERE id = (\n SELECT 1\n)

4
tests/queries/0_stateless/00751_default_databasename_for_view.reference
View File

@ -6,8 +6,8 @@ CREATE MATERIALIZED VIEW default.t_mv_00751
)
ENGINE = MergeTree
ORDER BY date
SETTINGS index_granularity = 8192 AS
SELECT
SETTINGS index_granularity = 8192
AS SELECT
date,
platform,
app

4
tests/queries/0_stateless/00916_create_or_replace_view.reference
View File

@ -1,2 +1,2 @@
CREATE VIEW default.t\n(\n `number` UInt64\n) AS\nSELECT number\nFROM system.numbers
CREATE VIEW default.t\n(\n `next_number` UInt64\n) AS\nSELECT number + 1 AS next_number\nFROM system.numbers
CREATE VIEW default.t\n(\n `number` UInt64\n)\nAS SELECT number\nFROM system.numbers
CREATE VIEW default.t\n(\n `next_number` UInt64\n)\nAS SELECT number + 1 AS next_number\nFROM system.numbers

2
tests/queries/0_stateless/01083_expressions_in_engine_arguments.reference
View File

@ -6,6 +6,6 @@ CREATE TABLE default.distributed\n(\n `n` Int8\n)\nENGINE = Distributed(\'tes
CREATE TABLE default.distributed_tf\n(\n `n` Int8\n) AS cluster(\'test_shard_localhost\', \'default\', \'buffer\')
CREATE TABLE default.url\n(\n `n` UInt64,\n `col` String\n)\nENGINE = URL(\'https://localhost:8443/?query=select+n,+_table+from+default.merge+format+CSV\', \'CSV\')
CREATE TABLE default.rich_syntax\n(\n `n` Int64\n) AS remote(\'localhos{x|y|t}\', cluster(\'test_shard_localhost\', remote(\'127.0.0.{1..4}\', \'default\', \'view\')))
CREATE VIEW default.view\n(\n `n` Int64\n) AS\nSELECT toInt64(n) AS n\nFROM\n(\n SELECT toString(n) AS n\n FROM default.merge\n WHERE _table != \'qwerty\'\n ORDER BY _table ASC\n)\nUNION ALL\nSELECT *\nFROM default.file
CREATE VIEW default.view\n(\n `n` Int64\n)\nAS SELECT toInt64(n) AS n\nFROM\n(\n SELECT toString(n) AS n\n FROM default.merge\n WHERE _table != \'qwerty\'\n ORDER BY _table ASC\n)\nUNION ALL\nSELECT *\nFROM default.file
CREATE DICTIONARY default.dict\n(\n `n` UInt64,\n `col` String DEFAULT \'42\'\n)\nPRIMARY KEY n\nSOURCE(CLICKHOUSE(HOST \'localhost\' PORT 9440 SECURE 1 USER \'default\' TABLE \'url\'))\nLIFETIME(MIN 0 MAX 1)\nLAYOUT(CACHE(SIZE_IN_CELLS 1))
16

8
tests/queries/0_stateless/01153_attach_mv_uuid.reference
View File

@ -4,18 +4,18 @@
2 4
3 9
4 16
CREATE MATERIALIZED VIEW default.mv UUID \'e15f3ab5-6cae-4df3-b879-f40deafd82c2\'\n(\n `n` Int32,\n `n2` Int64\n)\nENGINE = MergeTree\nPARTITION BY n % 10\nORDER BY n AS\nSELECT\n n,\n n * n AS n2\nFROM default.src
CREATE MATERIALIZED VIEW default.mv UUID \'e15f3ab5-6cae-4df3-b879-f40deafd82c2\'\n(\n `n` Int32,\n `n2` Int64\n)\nENGINE = MergeTree\nPARTITION BY n % 10\nORDER BY n\nAS SELECT\n n,\n n * n AS n2\nFROM default.src
1 1
2 4
CREATE MATERIALIZED VIEW default.mv UUID \'e15f3ab5-6cae-4df3-b879-f40deafd82c2\'\n(\n `n` Int32,\n `n2` Int64\n)\nENGINE = MergeTree\nPARTITION BY n % 10\nORDER BY n AS\nSELECT\n n,\n n * n AS n2\nFROM default.src
CREATE MATERIALIZED VIEW default.mv UUID \'e15f3ab5-6cae-4df3-b879-f40deafd82c2\'\n(\n `n` Int32,\n `n2` Int64\n)\nENGINE = MergeTree\nPARTITION BY n % 10\nORDER BY n\nAS SELECT\n n,\n n * n AS n2\nFROM default.src
1 1
2 4
3 9
4 16
CREATE MATERIALIZED VIEW default.mv UUID \'e15f3ab5-6cae-4df3-b879-f40deafd82c2\' TO INNER UUID \'3bd68e3c-2693-4352-ad66-a66eba9e345e\'\n(\n `n` Int32,\n `n2` Int64\n)\nENGINE = MergeTree\nPARTITION BY n % 10\nORDER BY n AS\nSELECT\n n,\n n * n AS n2\nFROM default.src
CREATE MATERIALIZED VIEW default.mv UUID \'e15f3ab5-6cae-4df3-b879-f40deafd82c2\' TO INNER UUID \'3bd68e3c-2693-4352-ad66-a66eba9e345e\'\n(\n `n` Int32,\n `n2` Int64\n)\nENGINE = MergeTree\nPARTITION BY n % 10\nORDER BY n\nAS SELECT\n n,\n n * n AS n2\nFROM default.src
1 1
2 4
CREATE MATERIALIZED VIEW default.mv UUID \'e15f3ab5-6cae-4df3-b879-f40deafd82c2\' TO INNER UUID \'3bd68e3c-2693-4352-ad66-a66eba9e345e\'\n(\n `n` Int32,\n `n2` Int64\n)\nENGINE = MergeTree\nPARTITION BY n % 10\nORDER BY n AS\nSELECT\n n,\n n * n AS n2\nFROM default.src
CREATE MATERIALIZED VIEW default.mv UUID \'e15f3ab5-6cae-4df3-b879-f40deafd82c2\' TO INNER UUID \'3bd68e3c-2693-4352-ad66-a66eba9e345e\'\n(\n `n` Int32,\n `n2` Int64\n)\nENGINE = MergeTree\nPARTITION BY n % 10\nORDER BY n\nAS SELECT\n n,\n n * n AS n2\nFROM default.src
1 1
2 4
3 9

3
tests/queries/0_stateless/01271_show_privileges.reference
View File

@ -48,6 +48,7 @@ ALTER TABLE [] \N ALTER
ALTER DATABASE [] \N ALTER
ALTER VIEW MODIFY QUERY ['ALTER TABLE MODIFY QUERY'] VIEW ALTER VIEW
ALTER VIEW MODIFY REFRESH ['ALTER TABLE MODIFY QUERY'] VIEW ALTER VIEW
ALTER VIEW MODIFY SQL SECURITY ['ALTER TABLE MODIFY SQL SECURITY'] VIEW ALTER VIEW
ALTER VIEW [] \N ALTER
ALTER [] \N ALL
CREATE DATABASE [] DATABASE CREATE
@ -89,6 +90,7 @@ DROP QUOTA [] GLOBAL ACCESS MANAGEMENT
CREATE SETTINGS PROFILE ['CREATE PROFILE'] GLOBAL ACCESS MANAGEMENT
ALTER SETTINGS PROFILE ['ALTER PROFILE'] GLOBAL ACCESS MANAGEMENT
DROP SETTINGS PROFILE ['DROP PROFILE'] GLOBAL ACCESS MANAGEMENT
ALLOW SQL SECURITY NONE ['CREATE SQL SECURITY NONE','ALLOW SQL SECURITY NONE','SQL SECURITY NONE','SECURITY NONE'] GLOBAL ACCESS MANAGEMENT
SHOW USERS ['SHOW CREATE USER'] GLOBAL SHOW ACCESS
SHOW ROLES ['SHOW CREATE ROLE'] GLOBAL SHOW ACCESS
SHOW ROW POLICIES ['SHOW POLICIES','SHOW CREATE ROW POLICY','SHOW CREATE POLICY'] TABLE SHOW ACCESS
@ -100,6 +102,7 @@ SHOW NAMED COLLECTIONS ['SHOW NAMED COLLECTIONS'] NAMED_COLLECTION NAMED COLLECT
SHOW NAMED COLLECTIONS SECRETS ['SHOW NAMED COLLECTIONS SECRETS'] NAMED_COLLECTION NAMED COLLECTION ADMIN
NAMED COLLECTION ['NAMED COLLECTION USAGE','USE NAMED COLLECTION'] NAMED_COLLECTION NAMED COLLECTION ADMIN
NAMED COLLECTION ADMIN ['NAMED COLLECTION CONTROL'] NAMED_COLLECTION ALL
SET DEFINER [] USER_NAME ALL
SYSTEM SHUTDOWN ['SYSTEM KILL','SHUTDOWN'] GLOBAL SYSTEM
SYSTEM DROP DNS CACHE ['SYSTEM DROP DNS','DROP DNS CACHE','DROP DNS'] GLOBAL SYSTEM DROP CACHE
SYSTEM DROP MARK CACHE ['SYSTEM DROP MARK','DROP MARK CACHE','DROP MARKS'] GLOBAL SYSTEM DROP CACHE

12
tests/queries/0_stateless/01602_show_create_view.reference
View File

@ -1,6 +1,6 @@
CREATE VIEW test_1602.v\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n) AS\nSELECT *\nFROM test_1602.tbl
CREATE MATERIALIZED VIEW test_1602.vv\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n)\nENGINE = MergeTree\nPARTITION BY toYYYYMM(EventDate)\nORDER BY (CounterID, EventDate, intHash32(UserID))\nSETTINGS index_granularity = 8192 AS\nSELECT *\nFROM test_1602.tbl
CREATE VIEW test_1602.VIEW\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n) AS\nSELECT *\nFROM test_1602.tbl
CREATE VIEW test_1602.DATABASE\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n) AS\nSELECT *\nFROM test_1602.tbl
CREATE VIEW test_1602.DICTIONARY\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n) AS\nSELECT *\nFROM test_1602.tbl
CREATE VIEW test_1602.TABLE\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n) AS\nSELECT *\nFROM test_1602.tbl
CREATE VIEW test_1602.v\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n)\nAS SELECT *\nFROM test_1602.tbl
CREATE MATERIALIZED VIEW test_1602.vv\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n)\nENGINE = MergeTree\nPARTITION BY toYYYYMM(EventDate)\nORDER BY (CounterID, EventDate, intHash32(UserID))\nSETTINGS index_granularity = 8192\nAS SELECT *\nFROM test_1602.tbl
CREATE VIEW test_1602.VIEW\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n)\nAS SELECT *\nFROM test_1602.tbl
CREATE VIEW test_1602.DATABASE\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n)\nAS SELECT *\nFROM test_1602.tbl
CREATE VIEW test_1602.DICTIONARY\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n)\nAS SELECT *\nFROM test_1602.tbl
CREATE VIEW test_1602.TABLE\n(\n `EventDate` DateTime,\n `CounterID` UInt32,\n `UserID` UInt32\n)\nAS SELECT *\nFROM test_1602.tbl

2
tests/queries/0_stateless/01890_materialized_distributed_join.sh
View File

@ -20,7 +20,7 @@ $CLICKHOUSE_CLIENT -nm -q "
insert into test_shard values (1, 1);
insert into test_local values (1, 2);
create materialized view test_distributed engine Distributed('test_cluster_two_shards', $CLICKHOUSE_DATABASE, 'test_shard', k) as select k, v from test_source;
create materialized view $CLICKHOUSE_DATABASE.test_distributed engine Distributed('test_cluster_two_shards', $CLICKHOUSE_DATABASE, 'test_shard', k) as select k, v from test_source;
select * from test_distributed td asof join $CLICKHOUSE_DATABASE.test_local tl on td.k = tl.k and td.v < tl.v;
select td.v, td.k, td.v, tl.v, tl.k, td.v from test_distributed td asof join $CLICKHOUSE_DATABASE.test_local tl on td.k = tl.k and td.v < tl.v FORMAT TSVWithNamesAndTypes;

2
tests/queries/0_stateless/01913_fix_column_transformer_replace_format.reference
View File

@ -1 +1 @@
CREATE VIEW default.my_view\n(\n `Id` UInt32,\n `Object.Key` Array(UInt16),\n `Object.Value` Array(String)\n) AS\nSELECT * REPLACE arrayMap(x -> (x + 1), `Object.Key`) AS `Object.Key`\nFROM default.my_table
CREATE VIEW default.my_view\n(\n `Id` UInt32,\n `Object.Key` Array(UInt16),\n `Object.Value` Array(String)\n)\nAS SELECT * REPLACE arrayMap(x -> (x + 1), `Object.Key`) AS `Object.Key`\nFROM default.my_table

2
tests/queries/0_stateless/02184_default_table_engine.reference
View File

@ -9,7 +9,7 @@ CREATE TABLE default.numbers1\n(\n `number` UInt64\n)\nENGINE = Memory
CREATE TABLE default.numbers2\n(\n `number` UInt64\n)\nENGINE = MergeTree\nORDER BY intHash32(number)\nSAMPLE BY intHash32(number)\nSETTINGS index_granularity = 8192
45
CREATE TABLE default.numbers3\n(\n `number` UInt64\n)\nENGINE = Log
CREATE MATERIALIZED VIEW default.test_view_filtered\n(\n `EventDate` Date,\n `CounterID` UInt32\n)\nENGINE = Memory AS\nSELECT\n CounterID,\n EventDate\nFROM default.test_table\nWHERE EventDate < \'2013-01-01\'
CREATE MATERIALIZED VIEW default.test_view_filtered\n(\n `EventDate` Date,\n `CounterID` UInt32\n)\nENGINE = Memory\nAS SELECT\n CounterID,\n EventDate\nFROM default.test_table\nWHERE EventDate < \'2013-01-01\'
2014-01-02 0 0 1969-12-31 16:00:00 2014-01-02 03:04:06
1 2014-01-01 19:04:06
CREATE TABLE default.t1\n(\n `Rows` UInt64,\n `MaxHitTime` DateTime(\'UTC\')\n)\nENGINE = MergeTree\nORDER BY Rows\nSETTINGS index_granularity = 8192

10
tests/queries/0_stateless/02206_information_schema_show_database.reference
View File

@ -1,6 +1,6 @@
CREATE DATABASE INFORMATION_SCHEMA\nENGINE = Memory
CREATE VIEW INFORMATION_SCHEMA.COLUMNS\n(\n `table_catalog` String,\n `table_schema` String,\n `table_name` String,\n `column_name` String,\n `ordinal_position` UInt64,\n `column_default` String,\n `is_nullable` String,\n `data_type` String,\n `character_maximum_length` Nullable(UInt64),\n `character_octet_length` Nullable(UInt64),\n `numeric_precision` Nullable(UInt64),\n `numeric_precision_radix` Nullable(UInt64),\n `numeric_scale` Nullable(UInt64),\n `datetime_precision` Nullable(UInt64),\n `character_set_catalog` Nullable(String),\n `character_set_schema` Nullable(String),\n `character_set_name` Nullable(String),\n `collation_catalog` Nullable(String),\n `collation_schema` Nullable(String),\n `collation_name` Nullable(String),\n `domain_catalog` Nullable(String),\n `domain_schema` Nullable(String),\n `domain_name` Nullable(String),\n `extra` Nullable(String),\n `column_comment` String,\n `column_type` String,\n `TABLE_CATALOG` String,\n `TABLE_SCHEMA` String,\n `TABLE_NAME` String,\n `COLUMN_NAME` String,\n `ORDINAL_POSITION` UInt64,\n `COLUMN_DEFAULT` String,\n `IS_NULLABLE` String,\n `DATA_TYPE` String,\n `CHARACTER_MAXIMUM_LENGTH` Nullable(UInt64),\n `CHARACTER_OCTET_LENGTH` Nullable(UInt64),\n `NUMERIC_PRECISION` Nullable(UInt64),\n `NUMERIC_PRECISION_RADIX` Nullable(UInt64),\n `NUMERIC_SCALE` Nullable(UInt64),\n `DATETIME_PRECISION` Nullable(UInt64),\n `CHARACTER_SET_CATALOG` Nullable(String),\n `CHARACTER_SET_SCHEMA` Nullable(String),\n `CHARACTER_SET_NAME` Nullable(String),\n `COLLATION_CATALOG` Nullable(String),\n `COLLATION_SCHEMA` Nullable(String),\n `COLLATION_NAME` Nullable(String),\n `DOMAIN_CATALOG` Nullable(String),\n `DOMAIN_SCHEMA` Nullable(String),\n `DOMAIN_NAME` Nullable(String),\n `EXTRA` Nullable(String),\n `COLUMN_COMMENT` String,\n `COLUMN_TYPE` String\n) AS\nSELECT\n database AS table_catalog,\n database AS table_schema,\n table AS table_name,\n name AS column_name,\n position AS ordinal_position,\n default_expression AS column_default,\n type LIKE \'Nullable(%)\' AS is_nullable,\n type AS data_type,\n character_octet_length AS character_maximum_length,\n character_octet_length,\n numeric_precision,\n numeric_precision_radix,\n numeric_scale,\n datetime_precision,\n NULL AS character_set_catalog,\n NULL AS character_set_schema,\n NULL AS character_set_name,\n NULL AS collation_catalog,\n NULL AS collation_schema,\n NULL AS collation_name,\n NULL AS domain_catalog,\n NULL AS domain_schema,\n NULL AS domain_name,\n multiIf(default_kind = \'DEFAULT\', \'DEFAULT_GENERATED\', default_kind = \'MATERIALIZED\', \'STORED GENERATED\', default_kind = \'ALIAS\', \'VIRTUAL GENERATED\', \'\') AS extra,\n comment AS column_comment,\n type AS column_type,\n table_catalog AS TABLE_CATALOG,\n table_schema AS TABLE_SCHEMA,\n table_name AS TABLE_NAME,\n column_name AS COLUMN_NAME,\n ordinal_position AS ORDINAL_POSITION,\n column_default AS COLUMN_DEFAULT,\n is_nullable AS IS_NULLABLE,\n data_type AS DATA_TYPE,\n character_maximum_length AS CHARACTER_MAXIMUM_LENGTH,\n character_octet_length AS CHARACTER_OCTET_LENGTH,\n numeric_precision AS NUMERIC_PRECISION,\n numeric_precision_radix AS NUMERIC_PRECISION_RADIX,\n numeric_scale AS NUMERIC_SCALE,\n datetime_precision AS DATETIME_PRECISION,\n character_set_catalog AS CHARACTER_SET_CATALOG,\n character_set_schema AS CHARACTER_SET_SCHEMA,\n character_set_name AS CHARACTER_SET_NAME,\n collation_catalog AS COLLATION_CATALOG,\n collation_schema AS COLLATION_SCHEMA,\n collation_name AS COLLATION_NAME,\n domain_catalog AS DOMAIN_CATALOG,\n domain_schema AS DOMAIN_SCHEMA,\n domain_name AS DOMAIN_NAME,\n extra AS EXTRA,\n column_comment AS COLUMN_COMMENT,\n column_type AS COLUMN_TYPE\nFROM system.columns
CREATE VIEW INFORMATION_SCHEMA.TABLES (`table_catalog` String, `table_schema` String, `table_name` String, `table_type` String, `table_rows` Nullable(UInt64), `data_length` Nullable(UInt64), `table_collation` Nullable(String), `table_comment` Nullable(String), `TABLE_CATALOG` String, `TABLE_SCHEMA` String, `TABLE_NAME` String, `TABLE_TYPE` String, `TABLE_ROWS` Nullable(UInt64), `DATA_LENGTH` Nullable(UInt64), `TABLE_COLLATION` Nullable(String), `TABLE_COMMENT` Nullable(String)) AS SELECT database AS table_catalog, database AS table_schema, name AS table_name, multiIf(is_temporary, \'LOCAL TEMPORARY\', engine LIKE \'%View\', \'VIEW\', engine LIKE \'System%\', \'SYSTEM VIEW\', has_own_data = 0, \'FOREIGN TABLE\', \'BASE TABLE\') AS table_type, total_rows AS table_rows, total_bytes AS data_length, \'utf8mb4_0900_ai_ci\' AS table_collation, comment AS table_comment, table_catalog AS TABLE_CATALOG, table_schema AS TABLE_SCHEMA, table_name AS TABLE_NAME, table_type AS TABLE_TYPE, table_rows AS TABLE_ROWS, data_length AS DATA_LENGTH, table_collation AS TABLE_COLLATION, table_comment AS TABLE_COMMENT FROM system.tables
CREATE VIEW INFORMATION_SCHEMA.tables (`table_catalog` String, `table_schema` String, `table_name` String, `table_type` String, `table_rows` Nullable(UInt64), `data_length` Nullable(UInt64), `table_collation` Nullable(String), `table_comment` Nullable(String), `TABLE_CATALOG` String, `TABLE_SCHEMA` String, `TABLE_NAME` String, `TABLE_TYPE` String, `TABLE_ROWS` Nullable(UInt64), `DATA_LENGTH` Nullable(UInt64), `TABLE_COLLATION` Nullable(String), `TABLE_COMMENT` Nullable(String)) AS SELECT database AS table_catalog, database AS table_schema, name AS table_name, multiIf(is_temporary, \'LOCAL TEMPORARY\', engine LIKE \'%View\', \'VIEW\', engine LIKE \'System%\', \'SYSTEM VIEW\', has_own_data = 0, \'FOREIGN TABLE\', \'BASE TABLE\') AS table_type, total_rows AS table_rows, total_bytes AS data_length, \'utf8mb4_0900_ai_ci\' AS table_collation, comment AS table_comment, table_catalog AS TABLE_CATALOG, table_schema AS TABLE_SCHEMA, table_name AS TABLE_NAME, table_type AS TABLE_TYPE, table_rows AS TABLE_ROWS, data_length AS DATA_LENGTH, table_collation AS TABLE_COLLATION, table_comment AS TABLE_COMMENT FROM system.tables
CREATE VIEW information_schema.TABLES (`table_catalog` String, `table_schema` String, `table_name` String, `table_type` String, `table_rows` Nullable(UInt64), `data_length` Nullable(UInt64), `table_collation` Nullable(String), `table_comment` Nullable(String), `TABLE_CATALOG` String, `TABLE_SCHEMA` String, `TABLE_NAME` String, `TABLE_TYPE` String, `TABLE_ROWS` Nullable(UInt64), `DATA_LENGTH` Nullable(UInt64), `TABLE_COLLATION` Nullable(String), `TABLE_COMMENT` Nullable(String)) AS SELECT database AS table_catalog, database AS table_schema, name AS table_name, multiIf(is_temporary, \'LOCAL TEMPORARY\', engine LIKE \'%View\', \'VIEW\', engine LIKE \'System%\', \'SYSTEM VIEW\', has_own_data = 0, \'FOREIGN TABLE\', \'BASE TABLE\') AS table_type, total_rows AS table_rows, total_bytes AS data_length, \'utf8mb4_0900_ai_ci\' AS table_collation, comment AS table_comment, table_catalog AS TABLE_CATALOG, table_schema AS TABLE_SCHEMA, table_name AS TABLE_NAME, table_type AS TABLE_TYPE, table_rows AS TABLE_ROWS, data_length AS DATA_LENGTH, table_collation AS TABLE_COLLATION, table_comment AS TABLE_COMMENT FROM system.tables
CREATE VIEW information_schema.tables (`table_catalog` String, `table_schema` String, `table_name` String, `table_type` String, `table_rows` Nullable(UInt64), `data_length` Nullable(UInt64), `table_collation` Nullable(String), `table_comment` Nullable(String), `TABLE_CATALOG` String, `TABLE_SCHEMA` String, `TABLE_NAME` String, `TABLE_TYPE` String, `TABLE_ROWS` Nullable(UInt64), `DATA_LENGTH` Nullable(UInt64), `TABLE_COLLATION` Nullable(String), `TABLE_COMMENT` Nullable(String)) AS SELECT database AS table_catalog, database AS table_schema, name AS table_name, multiIf(is_temporary, \'LOCAL TEMPORARY\', engine LIKE \'%View\', \'VIEW\', engine LIKE \'System%\', \'SYSTEM VIEW\', has_own_data = 0, \'FOREIGN TABLE\', \'BASE TABLE\') AS table_type, total_rows AS table_rows, total_bytes AS data_length, \'utf8mb4_0900_ai_ci\' AS table_collation, comment AS table_comment, table_catalog AS TABLE_CATALOG, table_schema AS TABLE_SCHEMA, table_name AS TABLE_NAME, table_type AS TABLE_TYPE, table_rows AS TABLE_ROWS, data_length AS DATA_LENGTH, table_collation AS TABLE_COLLATION, table_comment AS TABLE_COMMENT FROM system.tables
CREATE VIEW INFORMATION_SCHEMA.COLUMNS\n(\n `table_catalog` String,\n `table_schema` String,\n `table_name` String,\n `column_name` String,\n `ordinal_position` UInt64,\n `column_default` String,\n `is_nullable` String,\n `data_type` String,\n `character_maximum_length` Nullable(UInt64),\n `character_octet_length` Nullable(UInt64),\n `numeric_precision` Nullable(UInt64),\n `numeric_precision_radix` Nullable(UInt64),\n `numeric_scale` Nullable(UInt64),\n `datetime_precision` Nullable(UInt64),\n `character_set_catalog` Nullable(String),\n `character_set_schema` Nullable(String),\n `character_set_name` Nullable(String),\n `collation_catalog` Nullable(String),\n `collation_schema` Nullable(String),\n `collation_name` Nullable(String),\n `domain_catalog` Nullable(String),\n `domain_schema` Nullable(String),\n `domain_name` Nullable(String),\n `extra` Nullable(String),\n `column_comment` String,\n `column_type` String,\n `TABLE_CATALOG` String,\n `TABLE_SCHEMA` String,\n `TABLE_NAME` String,\n `COLUMN_NAME` String,\n `ORDINAL_POSITION` UInt64,\n `COLUMN_DEFAULT` String,\n `IS_NULLABLE` String,\n `DATA_TYPE` String,\n `CHARACTER_MAXIMUM_LENGTH` Nullable(UInt64),\n `CHARACTER_OCTET_LENGTH` Nullable(UInt64),\n `NUMERIC_PRECISION` Nullable(UInt64),\n `NUMERIC_PRECISION_RADIX` Nullable(UInt64),\n `NUMERIC_SCALE` Nullable(UInt64),\n `DATETIME_PRECISION` Nullable(UInt64),\n `CHARACTER_SET_CATALOG` Nullable(String),\n `CHARACTER_SET_SCHEMA` Nullable(String),\n `CHARACTER_SET_NAME` Nullable(String),\n `COLLATION_CATALOG` Nullable(String),\n `COLLATION_SCHEMA` Nullable(String),\n `COLLATION_NAME` Nullable(String),\n `DOMAIN_CATALOG` Nullable(String),\n `DOMAIN_SCHEMA` Nullable(String),\n `DOMAIN_NAME` Nullable(String),\n `EXTRA` Nullable(String),\n `COLUMN_COMMENT` String,\n `COLUMN_TYPE` String\n)\nSQL SECURITY INVOKER\nAS SELECT\n database AS table_catalog,\n database AS table_schema,\n table AS table_name,\n name AS column_name,\n position AS ordinal_position,\n default_expression AS column_default,\n type LIKE \'Nullable(%)\' AS is_nullable,\n type AS data_type,\n character_octet_length AS character_maximum_length,\n character_octet_length,\n numeric_precision,\n numeric_precision_radix,\n numeric_scale,\n datetime_precision,\n NULL AS character_set_catalog,\n NULL AS character_set_schema,\n NULL AS character_set_name,\n NULL AS collation_catalog,\n NULL AS collation_schema,\n NULL AS collation_name,\n NULL AS domain_catalog,\n NULL AS domain_schema,\n NULL AS domain_name,\n multiIf(default_kind = \'DEFAULT\', \'DEFAULT_GENERATED\', default_kind = \'MATERIALIZED\', \'STORED GENERATED\', default_kind = \'ALIAS\', \'VIRTUAL GENERATED\', \'\') AS extra,\n comment AS column_comment,\n type AS column_type,\n table_catalog AS TABLE_CATALOG,\n table_schema AS TABLE_SCHEMA,\n table_name AS TABLE_NAME,\n column_name AS COLUMN_NAME,\n ordinal_position AS ORDINAL_POSITION,\n column_default AS COLUMN_DEFAULT,\n is_nullable AS IS_NULLABLE,\n data_type AS DATA_TYPE,\n character_maximum_length AS CHARACTER_MAXIMUM_LENGTH,\n character_octet_length AS CHARACTER_OCTET_LENGTH,\n numeric_precision AS NUMERIC_PRECISION,\n numeric_precision_radix AS NUMERIC_PRECISION_RADIX,\n numeric_scale AS NUMERIC_SCALE,\n datetime_precision AS DATETIME_PRECISION,\n character_set_catalog AS CHARACTER_SET_CATALOG,\n character_set_schema AS CHARACTER_SET_SCHEMA,\n character_set_name AS CHARACTER_SET_NAME,\n collation_catalog AS COLLATION_CATALOG,\n collation_schema AS COLLATION_SCHEMA,\n collation_name AS COLLATION_NAME,\n domain_catalog AS DOMAIN_CATALOG,\n domain_schema AS DOMAIN_SCHEMA,\n domain_name AS DOMAIN_NAME,\n extra AS EXTRA,\n column_comment AS COLUMN_COMMENT,\n column_type AS COLUMN_TYPE\nFROM system.columns
CREATE VIEW INFORMATION_SCHEMA.TABLES (`table_catalog` String, `table_schema` String, `table_name` String, `table_type` String, `table_rows` Nullable(UInt64), `data_length` Nullable(UInt64), `table_collation` Nullable(String), `table_comment` Nullable(String), `TABLE_CATALOG` String, `TABLE_SCHEMA` String, `TABLE_NAME` String, `TABLE_TYPE` String, `TABLE_ROWS` Nullable(UInt64), `DATA_LENGTH` Nullable(UInt64), `TABLE_COLLATION` Nullable(String), `TABLE_COMMENT` Nullable(String)) SQL SECURITY INVOKER AS SELECT database AS table_catalog, database AS table_schema, name AS table_name, multiIf(is_temporary, \'LOCAL TEMPORARY\', engine LIKE \'%View\', \'VIEW\', engine LIKE \'System%\', \'SYSTEM VIEW\', has_own_data = 0, \'FOREIGN TABLE\', \'BASE TABLE\') AS table_type, total_rows AS table_rows, total_bytes AS data_length, \'utf8mb4_0900_ai_ci\' AS table_collation, comment AS table_comment, table_catalog AS TABLE_CATALOG, table_schema AS TABLE_SCHEMA, table_name AS TABLE_NAME, table_type AS TABLE_TYPE, table_rows AS TABLE_ROWS, data_length AS DATA_LENGTH, table_collation AS TABLE_COLLATION, table_comment AS TABLE_COMMENT FROM system.tables
CREATE VIEW INFORMATION_SCHEMA.tables (`table_catalog` String, `table_schema` String, `table_name` String, `table_type` String, `table_rows` Nullable(UInt64), `data_length` Nullable(UInt64), `table_collation` Nullable(String), `table_comment` Nullable(String), `TABLE_CATALOG` String, `TABLE_SCHEMA` String, `TABLE_NAME` String, `TABLE_TYPE` String, `TABLE_ROWS` Nullable(UInt64), `DATA_LENGTH` Nullable(UInt64), `TABLE_COLLATION` Nullable(String), `TABLE_COMMENT` Nullable(String)) SQL SECURITY INVOKER AS SELECT database AS table_catalog, database AS table_schema, name AS table_name, multiIf(is_temporary, \'LOCAL TEMPORARY\', engine LIKE \'%View\', \'VIEW\', engine LIKE \'System%\', \'SYSTEM VIEW\', has_own_data = 0, \'FOREIGN TABLE\', \'BASE TABLE\') AS table_type, total_rows AS table_rows, total_bytes AS data_length, \'utf8mb4_0900_ai_ci\' AS table_collation, comment AS table_comment, table_catalog AS TABLE_CATALOG, table_schema AS TABLE_SCHEMA, table_name AS TABLE_NAME, table_type AS TABLE_TYPE, table_rows AS TABLE_ROWS, data_length AS DATA_LENGTH, table_collation AS TABLE_COLLATION, table_comment AS TABLE_COMMENT FROM system.tables
CREATE VIEW information_schema.TABLES (`table_catalog` String, `table_schema` String, `table_name` String, `table_type` String, `table_rows` Nullable(UInt64), `data_length` Nullable(UInt64), `table_collation` Nullable(String), `table_comment` Nullable(String), `TABLE_CATALOG` String, `TABLE_SCHEMA` String, `TABLE_NAME` String, `TABLE_TYPE` String, `TABLE_ROWS` Nullable(UInt64), `DATA_LENGTH` Nullable(UInt64), `TABLE_COLLATION` Nullable(String), `TABLE_COMMENT` Nullable(String)) SQL SECURITY INVOKER AS SELECT database AS table_catalog, database AS table_schema, name AS table_name, multiIf(is_temporary, \'LOCAL TEMPORARY\', engine LIKE \'%View\', \'VIEW\', engine LIKE \'System%\', \'SYSTEM VIEW\', has_own_data = 0, \'FOREIGN TABLE\', \'BASE TABLE\') AS table_type, total_rows AS table_rows, total_bytes AS data_length, \'utf8mb4_0900_ai_ci\' AS table_collation, comment AS table_comment, table_catalog AS TABLE_CATALOG, table_schema AS TABLE_SCHEMA, table_name AS TABLE_NAME, table_type AS TABLE_TYPE, table_rows AS TABLE_ROWS, data_length AS DATA_LENGTH, table_collation AS TABLE_COLLATION, table_comment AS TABLE_COMMENT FROM system.tables
CREATE VIEW information_schema.tables (`table_catalog` String, `table_schema` String, `table_name` String, `table_type` String, `table_rows` Nullable(UInt64), `data_length` Nullable(UInt64), `table_collation` Nullable(String), `table_comment` Nullable(String), `TABLE_CATALOG` String, `TABLE_SCHEMA` String, `TABLE_NAME` String, `TABLE_TYPE` String, `TABLE_ROWS` Nullable(UInt64), `DATA_LENGTH` Nullable(UInt64), `TABLE_COLLATION` Nullable(String), `TABLE_COMMENT` Nullable(String)) SQL SECURITY INVOKER AS SELECT database AS table_catalog, database AS table_schema, name AS table_name, multiIf(is_temporary, \'LOCAL TEMPORARY\', engine LIKE \'%View\', \'VIEW\', engine LIKE \'System%\', \'SYSTEM VIEW\', has_own_data = 0, \'FOREIGN TABLE\', \'BASE TABLE\') AS table_type, total_rows AS table_rows, total_bytes AS data_length, \'utf8mb4_0900_ai_ci\' AS table_collation, comment AS table_comment, table_catalog AS TABLE_CATALOG, table_schema AS TABLE_SCHEMA, table_name AS TABLE_NAME, table_type AS TABLE_TYPE, table_rows AS TABLE_ROWS, data_length AS DATA_LENGTH, table_collation AS TABLE_COLLATION, table_comment AS TABLE_COMMENT FROM system.tables

2
tests/queries/0_stateless/02343_create_empty_as_select.reference
View File

@ -1,4 +1,4 @@
CREATE TABLE default.t\n(\n `1` UInt8\n)\nENGINE = Memory
0
CREATE MATERIALIZED VIEW default.mv\n(\n `1` UInt8\n)\nENGINE = Memory AS\nSELECT 1
CREATE MATERIALIZED VIEW default.mv\n(\n `1` UInt8\n)\nENGINE = Memory\nAS SELECT 1
0

2
tests/queries/0_stateless/02539_settings_alias.reference
View File

@ -18,7 +18,7 @@ Using HTTP with query params
Using client options
0
2
CREATE VIEW default.`02539_settings_alias_view`\n(\n `1` UInt8\n) AS\nSELECT 1\nSETTINGS replication_alter_partitions_sync = 2
CREATE VIEW default.`02539_settings_alias_view`\n(\n `1` UInt8\n)\nAS SELECT 1\nSETTINGS replication_alter_partitions_sync = 2
replication_alter_partitions_sync 0 1 alter_sync
replication_alter_partitions_sync 2 1 alter_sync
alter_sync 0 1

2
tests/queries/0_stateless/02765_queries_with_subqueries_profile_events.sh
View File

@ -11,7 +11,7 @@ $CLICKHOUSE_CLIENT -n -q "
CREATE TABLE input (key Int) Engine=Null;
CREATE TABLE output AS input Engine=Null;
CREATE MATERIALIZED VIEW mv TO output AS SELECT * FROM input;
CREATE MATERIALIZED VIEW mv TO output SQL SECURITY NONE AS SELECT * FROM input;
"
for allow_experimental_analyzer in 0 1; do

2
tests/queries/0_stateless/02868_select_support_from_keywords.reference
View File

@ -1 +1 @@
CREATE VIEW default.test_view\n(\n `date` Date,\n `__sign` Int8,\n `from` Float64,\n `to` Float64\n) AS\nWITH cte AS\n (\n SELECT\n date,\n __sign,\n from,\n to\n FROM default.test_table\n FINAL\n )\nSELECT\n date,\n __sign,\n from,\n to\nFROM cte
CREATE VIEW default.test_view\n(\n `date` Date,\n `__sign` Int8,\n `from` Float64,\n `to` Float64\n)\nAS WITH cte AS\n (\n SELECT\n date,\n __sign,\n from,\n to\n FROM default.test_table\n FINAL\n )\nSELECT\n date,\n __sign,\n from,\n to\nFROM cte

32
tests/queries/0_stateless/02884_create_view_with_sql_security_option.reference Normal file
View File

@ -0,0 +1,32 @@
===== StorageView =====
OK
OK
OK
2
2
OK
OK
2
2
OK
2
2
OK
===== MaterializedView =====
OK
0
0
OK
OK
OK
2
OK
OK
===== TestGrants =====
OK
OK
===== TestRowPolicy =====
1 1
2 2
6 6
9 9

226
tests/queries/0_stateless/02884_create_view_with_sql_security_option.sh Executable file
View File

@ -0,0 +1,226 @@
#!/usr/bin/env bash
CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
. "$CURDIR"/../shell_config.sh
user1="user02884_1_$RANDOM$RANDOM"
user2="user02884_2_$RANDOM$RANDOM"
user3="user02884_3_$RANDOM$RANDOM"
db="db02884_$RANDOM$RANDOM"
${CLICKHOUSE_CLIENT} --multiquery <<EOF
DROP DATABASE IF EXISTS $db;
CREATE DATABASE $db;
CREATE TABLE $db.test_table (s String) ENGINE = MergeTree ORDER BY s;
DROP USER IF EXISTS $user1, $user2, $user3;
CREATE USER $user1, $user2, $user3;
GRANT SELECT ON $db.* TO $user1;
EOF
echo "===== StorageView ====="
${CLICKHOUSE_CLIENT} --multiquery <<EOF
CREATE VIEW $db.test_view_1 (s String)
AS SELECT * FROM $db.test_table;
CREATE DEFINER $user1 VIEW $db.test_view_2 (s String)
AS SELECT * FROM $db.test_table;
CREATE VIEW $db.test_view_3 (s String)
DEFINER = $user1 SQL SECURITY DEFINER
AS SELECT * FROM $db.test_table;
CREATE VIEW $db.test_view_4 (s String)
DEFINER = $user1 SQL SECURITY INVOKER
AS SELECT * FROM $db.test_table;
CREATE VIEW $db.test_view_5 (s String)
SQL SECURITY INVOKER
AS SELECT * FROM $db.test_table;
CREATE VIEW $db.test_view_6 (s String)
SQL SECURITY DEFINER
AS SELECT * FROM $db.test_table;
CREATE VIEW $db.test_view_7 (s String)
DEFINER CURRENT_USER
AS SELECT * FROM $db.test_table;
CREATE VIEW $db.test_view_8 (s String)
DEFINER $user3
AS SELECT * FROM $db.test_table;
CREATE VIEW $db.test_view_9 (s String)
SQL SECURITY NONE
AS SELECT * FROM $db.test_table;
CREATE VIEW $db.test_view_10 (s String)
SQL SECURITY DEFINER
AS SELECT * FROM $db.test_table;
EOF
(( $(${CLICKHOUSE_CLIENT} --query "SHOW TABLE $db.test_view_5" 2>&1 | grep -c "INVOKER") >= 1 )) && echo "OK" || echo "UNEXPECTED"
(( $(${CLICKHOUSE_CLIENT} --query "SHOW TABLE $db.test_view_2" 2>&1 | grep -c "DEFINER = $user1") >= 1 )) && echo "OK" || echo "UNEXPECTED"
${CLICKHOUSE_CLIENT} --multiquery <<EOF
GRANT SELECT ON $db.test_view_1 TO $user2;
GRANT SELECT ON $db.test_view_2 TO $user2;
GRANT SELECT ON $db.test_view_3 TO $user2;
GRANT SELECT ON $db.test_view_4 TO $user2;
GRANT SELECT ON $db.test_view_5 TO $user2;
GRANT SELECT ON $db.test_view_6 TO $user2;
GRANT SELECT ON $db.test_view_7 TO $user2;
GRANT SELECT ON $db.test_view_8 TO $user2;
GRANT SELECT ON $db.test_view_9 TO $user2;
GRANT SELECT ON $db.test_view_10 TO $user2;
EOF
${CLICKHOUSE_CLIENT} --query "INSERT INTO $db.test_table VALUES ('foo'), ('bar');"
(( $(${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT * FROM $db.test_view_1" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT count() FROM $db.test_view_2"
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT count() FROM $db.test_view_3"
(( $(${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT * FROM $db.test_view_4" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
(( $(${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT * FROM $db.test_view_5" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT count() FROM $db.test_view_6"
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT count() FROM $db.test_view_7"
(( $(${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT * FROM $db.test_view_8" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT count() FROM $db.test_view_9"
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT count() FROM $db.test_view_10"
${CLICKHOUSE_CLIENT} --query "ALTER TABLE $db.test_view_10 MODIFY SQL SECURITY INVOKER"
(( $(${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT * FROM $db.test_view_10" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
echo "===== MaterializedView ====="
${CLICKHOUSE_CLIENT} --query "
CREATE MATERIALIZED VIEW $db.test_mv_1 (s String)
ENGINE = MergeTree ORDER BY s
DEFINER = $user1 SQL SECURITY DEFINER
AS SELECT * FROM $db.test_table;
"
(( $(${CLICKHOUSE_CLIENT} --query "
CREATE MATERIALIZED VIEW $db.test_mv_2 (s String)
ENGINE = MergeTree ORDER BY s
SQL SECURITY INVOKER
AS SELECT * FROM $db.test_table;
" 2>&1 | grep -c "SQL SECURITY INVOKER can't be specified for MATERIALIZED VIEW") >= 1 )) && echo "OK" || echo "UNEXPECTED"
${CLICKHOUSE_CLIENT} --query "
CREATE MATERIALIZED VIEW $db.test_mv_3 (s String)
ENGINE = MergeTree ORDER BY s
SQL SECURITY NONE
AS SELECT * FROM $db.test_table;
"
${CLICKHOUSE_CLIENT} --query "CREATE TABLE $db.test_mv_data (s String) ENGINE = MergeTree ORDER BY s;"
${CLICKHOUSE_CLIENT} --query "
CREATE MATERIALIZED VIEW $db.test_mv_4
TO $db.test_mv_data
DEFINER = $user1 SQL SECURITY DEFINER
AS SELECT * FROM $db.test_table;
"
${CLICKHOUSE_CLIENT} --query "
CREATE MATERIALIZED VIEW $db.test_mv_5 (s String)
ENGINE = MergeTree ORDER BY s
DEFINER = $user2 SQL SECURITY DEFINER
AS SELECT * FROM $db.test_table;
"
${CLICKHOUSE_CLIENT} --query "GRANT SELECT ON $db.test_mv_5 TO $user2"
${CLICKHOUSE_CLIENT} --query "ALTER TABLE $db.test_mv_5 MODIFY SQL SECURITY NONE"
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT * FROM $db.test_mv_5"
${CLICKHOUSE_CLIENT} --query "GRANT SELECT ON $db.test_mv_1 TO $user2"
${CLICKHOUSE_CLIENT} --query "GRANT SELECT ON $db.test_mv_3 TO $user2"
${CLICKHOUSE_CLIENT} --query "GRANT SELECT ON $db.test_mv_4 TO $user2"
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT count() FROM $db.test_mv_1"
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT count() FROM $db.test_mv_3"
${CLICKHOUSE_CLIENT} --query "REVOKE SELECT ON $db.test_mv_data FROM $user1"
(( $(${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT * FROM $db.test_mv_4" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
(( $(${CLICKHOUSE_CLIENT} --query "INSERT INTO $db.test_table VALUES ('foo'), ('bar');" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
(( $(${CLICKHOUSE_CLIENT} --materialized_views_ignore_errors 1 --query "INSERT INTO $db.test_table VALUES ('foo'), ('bar');" 2>&1 | grep -c "Failed to push block to view") >= 1 )) && echo "OK" || echo "UNEXPECTED"
${CLICKHOUSE_CLIENT} --query "GRANT INSERT ON $db.test_mv_data TO $user1"
${CLICKHOUSE_CLIENT} --query "GRANT SELECT ON $db.test_mv_data TO $user1"
${CLICKHOUSE_CLIENT} --query "INSERT INTO $db.test_table VALUES ('foo'), ('bar');"
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT count() FROM $db.test_mv_4"
${CLICKHOUSE_CLIENT} --query "REVOKE SELECT ON $db.test_table FROM $user1"
(( $(${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT * FROM $db.test_mv_4" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
(( $(${CLICKHOUSE_CLIENT} --query "INSERT INTO $db.test_table VALUES ('foo'), ('bar');" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
echo "===== TestGrants ====="
${CLICKHOUSE_CLIENT} --query "GRANT CREATE ON *.* TO $user1"
${CLICKHOUSE_CLIENT} --query "GRANT SELECT ON $db.test_table TO $user1, $user2"
${CLICKHOUSE_CLIENT} --user $user1 --query "
CREATE VIEW $db.test_view_g_1
DEFINER = CURRENT_USER SQL SECURITY DEFINER
AS SELECT * FROM $db.test_table;
"
(( $(${CLICKHOUSE_CLIENT} --user $user1 --query "
CREATE VIEW $db.test_view_g_2
DEFINER = $user2
AS SELECT * FROM $db.test_table;
" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
${CLICKHOUSE_CLIENT} --query "GRANT SET DEFINER ON $user2 TO $user1"
${CLICKHOUSE_CLIENT} --user $user1 --query "
CREATE VIEW $db.test_view_g_2
DEFINER = $user2
AS SELECT * FROM $db.test_table;
"
(( $(${CLICKHOUSE_CLIENT} --user $user1 --query "
CREATE VIEW $db.test_view_g_3
SQL SECURITY NONE
AS SELECT * FROM $db.test_table;
" 2>&1 | grep -c "Not enough privileges") >= 1 )) && echo "OK" || echo "UNEXPECTED"
${CLICKHOUSE_CLIENT} --query "GRANT SET DEFINER ON $user2 TO $user1"
echo "===== TestRowPolicy ====="
${CLICKHOUSE_CLIENT} --multiquery <<EOF
CREATE TABLE $db.test_row_t (x Int32, y Int32) ENGINE = MergeTree ORDER BY x;
CREATE VIEW $db.test_view_row_1 DEFINER = $user1 SQL SECURITY DEFINER AS SELECT x, y AS z FROM $db.test_row_t;
CREATE ROW POLICY r1 ON $db.test_row_t FOR SELECT USING x <= y TO $user1;
CREATE ROW POLICY r2 ON $db.test_view_row_1 FOR SELECT USING x >= z TO $user2;
INSERT INTO $db.test_row_t VALUES (1, 2), (1, 1), (2, 2), (3, 2), (4, 0);
GRANT SELECT ON $db.test_view_row_1 to $user2;
EOF
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT * FROM $db.test_view_row_1"
${CLICKHOUSE_CLIENT} --multiquery <<EOF
CREATE TABLE $db.test_row_t2 (x Int32, y Int32) ENGINE = MergeTree ORDER BY x;
CREATE VIEW $db.test_mv_row_2 DEFINER = $user1 SQL SECURITY DEFINER AS SELECT x, y AS z FROM $db.test_row_t2;
CREATE ROW POLICY r1 ON $db.test_row_t2 FOR SELECT USING x <= y TO $user1;
CREATE ROW POLICY r2 ON $db.test_mv_row_2 FOR SELECT USING x >= z TO $user2;
INSERT INTO $db.test_row_t2 VALUES (5, 6), (6, 5), (6, 6), (8, 7), (9, 9);
GRANT SELECT ON $db.test_mv_row_2 to $user2;
EOF
${CLICKHOUSE_CLIENT} --user $user2 --query "SELECT * FROM $db.test_mv_row_2"
${CLICKHOUSE_CLIENT} --query "DROP DATABASE IF EXISTS $db;"
${CLICKHOUSE_CLIENT} --query "DROP USER IF EXISTS $user1, $user2, $user3";

4
tests/queries/0_stateless/02916_set_formatting.reference
View File

@ -5,7 +5,7 @@ Row 1:
statement: CREATE VIEW default.v1
(
`v` UInt64
) AS
SELECT v
)
AS SELECT v
FROM default.t1
SETTINGS additional_table_filters = {'default.t1':'s != \'s1%\''}

2
tests/queries/0_stateless/02931_alter_materialized_view_query_inconsistent.reference
View File

@ -1,3 +1,3 @@
v UInt64
v2 UInt8
CREATE MATERIALIZED VIEW default.pipe TO default.dest\n(\n `v` UInt64,\n `v2` UInt8\n) AS\nSELECT\n v * 2 AS v,\n 1 AS v2\nFROM default.src
CREATE MATERIALIZED VIEW default.pipe TO default.dest\n(\n `v` UInt64,\n `v2` UInt8\n)\nAS SELECT\n v * 2 AS v,\n 1 AS v2\nFROM default.src

12
tests/queries/0_stateless/02932_refreshable_materialized_views.reference
View File

@ -1,14 +1,14 @@
<1: created view> a [] 1
CREATE MATERIALIZED VIEW default.a\nREFRESH AFTER 1 SECOND\n(\n `x` UInt64\n)\nENGINE = Memory AS\nSELECT number AS x\nFROM numbers(2)\nUNION ALL\nSELECT rand64() AS x
CREATE MATERIALIZED VIEW default.a\nREFRESH AFTER 1 SECOND\n(\n `x` UInt64\n)\nENGINE = Memory\nAS SELECT number AS x\nFROM numbers(2)\nUNION ALL\nSELECT rand64() AS x
<2: refreshed> 3 1 1
<3: time difference at least> 500
<4: next refresh in> 1
<4.5: altered> Scheduled Finished 2052-01-01 00:00:00
CREATE MATERIALIZED VIEW default.a\nREFRESH EVERY 2 YEAR\n(\n `x` Int16\n)\nENGINE = Memory AS\nSELECT x * 2 AS x\nFROM default.src
CREATE MATERIALIZED VIEW default.a\nREFRESH EVERY 2 YEAR\n(\n `x` Int16\n)\nENGINE = Memory\nAS SELECT x * 2 AS x\nFROM default.src
<5: no refresh> 3
<6: refreshed> 2
<7: refreshed> Scheduled Finished 2054-01-01 00:00:00
CREATE MATERIALIZED VIEW default.b\nREFRESH EVERY 2 YEAR DEPENDS ON default.a\n(\n `y` Int32\n)\nENGINE = MergeTree\nORDER BY y\nSETTINGS index_granularity = 8192 AS\nSELECT x * 10 AS y\nFROM default.a
CREATE MATERIALIZED VIEW default.b\nREFRESH EVERY 2 YEAR DEPENDS ON default.a\n(\n `y` Int32\n)\nENGINE = MergeTree\nORDER BY y\nSETTINGS index_granularity = 8192\nAS SELECT x * 10 AS y\nFROM default.a
<8: refreshed> 20
<9: refreshed> a Scheduled Finished 2054-01-01 00:00:00
<9: refreshed> b Scheduled Finished 2054-01-01 00:00:00
@ -25,7 +25,7 @@ CREATE MATERIALIZED VIEW default.b\nREFRESH EVERY 2 YEAR DEPENDS ON default.a\n(
<17: chain-refreshed> a Scheduled 2062-01-01 00:00:00
<17: chain-refreshed> b Scheduled 2062-01-01 00:00:00
<18: removed dependency> b Scheduled [] 2062-03-03 03:03:03 2064-01-01 00:00:00 5
CREATE MATERIALIZED VIEW default.b\nREFRESH EVERY 2 YEAR\n(\n `y` Int32\n)\nENGINE = MergeTree\nORDER BY y\nSETTINGS index_granularity = 8192 AS\nSELECT x * 10 AS y\nFROM default.a
CREATE MATERIALIZED VIEW default.b\nREFRESH EVERY 2 YEAR\n(\n `y` Int32\n)\nENGINE = MergeTree\nORDER BY y\nSETTINGS index_granularity = 8192\nAS SELECT x * 10 AS y\nFROM default.a
<19: exception> 1
<20: unexception> 1
<21: rename> 1
@ -34,9 +34,9 @@ CREATE MATERIALIZED VIEW default.b\nREFRESH EVERY 2 YEAR\n(\n `y` Int32\n)\nE
<24: rename during refresh> 1
<25: rename during refresh> f Running
<27: cancelled> f Scheduled
CREATE MATERIALIZED VIEW default.g\nREFRESH EVERY 1 WEEK OFFSET 3 DAY 4 HOUR RANDOMIZE FOR 4 DAY 1 HOUR\n(\n `x` Int64\n)\nENGINE = Memory AS\nSELECT 42
CREATE MATERIALIZED VIEW default.g\nREFRESH EVERY 1 WEEK OFFSET 3 DAY 4 HOUR RANDOMIZE FOR 4 DAY 1 HOUR\n(\n `x` Int64\n)\nENGINE = Memory\nAS SELECT 42
<29: randomize> 1 1
CREATE MATERIALIZED VIEW default.h\nREFRESH EVERY 1 SECOND TO default.dest\n(\n `x` Int64\n) AS\nSELECT x * 10 AS x\nFROM default.src
CREATE MATERIALIZED VIEW default.h\nREFRESH EVERY 1 SECOND TO default.dest\n(\n `x` Int64\n)\nAS SELECT x * 10 AS x\nFROM default.src
<30: to existing table> 10
<31: to existing table> 10
<31: to existing table> 20

8
tests/queries/0_stateless/02968_url_args.reference
View File

@ -2,7 +2,7 @@ CREATE TABLE default.a\n(\n `x` Int64\n)\nENGINE = URL(\'https://example.com/
CREATE TABLE default.b\n(\n `x` Int64\n)\nENGINE = URL(\'https://example.com/\', \'CSV\', headers())
CREATE TABLE default.c\n(\n `x` Int64\n)\nENGINE = S3(\'https://example.s3.amazonaws.com/a.csv\', \'NOSIGN\', \'CSV\', headers(\'foo\' = \'[HIDDEN]\'))
CREATE TABLE default.d\n(\n `x` Int64\n)\nENGINE = S3(\'https://example.s3.amazonaws.com/a.csv\', \'NOSIGN\', headers(\'foo\' = \'[HIDDEN]\'))
CREATE VIEW default.e\n(\n `x` Int64\n) AS\nSELECT count()\nFROM url(\'https://example.com/\', CSV, headers(\'foo\' = \'[HIDDEN]\', \'a\' = \'[HIDDEN]\'))
CREATE VIEW default.f\n(\n `x` Int64\n) AS\nSELECT count()\nFROM url(\'https://example.com/\', CSV, headers())
CREATE VIEW default.g\n(\n `x` Int64\n) AS\nSELECT count()\nFROM s3(\'https://example.s3.amazonaws.com/a.csv\', CSV, headers(\'foo\' = \'[HIDDEN]\'))
CREATE VIEW default.h\n(\n `x` Int64\n) AS\nSELECT count()\nFROM s3(\'https://example.s3.amazonaws.com/a.csv\', headers(\'foo\' = \'[HIDDEN]\'))
CREATE VIEW default.e\n(\n `x` Int64\n)\nAS SELECT count()\nFROM url(\'https://example.com/\', CSV, headers(\'foo\' = \'[HIDDEN]\', \'a\' = \'[HIDDEN]\'))
CREATE VIEW default.f\n(\n `x` Int64\n)\nAS SELECT count()\nFROM url(\'https://example.com/\', CSV, headers())
CREATE VIEW default.g\n(\n `x` Int64\n)\nAS SELECT count()\nFROM s3(\'https://example.s3.amazonaws.com/a.csv\', CSV, headers(\'foo\' = \'[HIDDEN]\'))
CREATE VIEW default.h\n(\n `x` Int64\n)\nAS SELECT count()\nFROM s3(\'https://example.s3.amazonaws.com/a.csv\', headers(\'foo\' = \'[HIDDEN]\'))