thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-21 23:21:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Rename some variables in keeper

Browse Source
This commit is contained in:
alesapin 2022-03-17 11:55:15 +01:00
parent 6ab309b25d
commit bf5b3a856d
13 changed files with 40 additions and 35 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/en/operations/clickhouse-keeper.md
View File

@ -55,7 +55,7 @@ Internal coordination settings are located in `<keeper_server>.<coordination_set
- `auto_forwarding` — Allow to forward write requests from followers to the leader (default: true).
- `shutdown_timeout` — Wait to finish internal connections and shutdown (ms) (default: 5000).
- `startup_timeout` — If the server doesn't connect to other quorum participants in the specified timeout it will terminate (ms) (default: 30000).
- `four_letter_word_white_list` — White list of 4lw commands (default: "conf,cons,crst,envi,ruok,srst,srvr,stat,wchs,dirs,mntr,isro").
- `four_letter_word_allow_list` — Allow list of 4lw commands (default: "conf,cons,crst,envi,ruok,srst,srvr,stat,wchs,dirs,mntr,isro").
Quorum configuration is located in `<keeper_server>.<raft_configuration>` section and contain servers description.
@ -121,7 +121,7 @@ clickhouse keeper --config /etc/your_path_to_config/config.xml
ClickHouse Keeper also provides 4lw commands which are almost the same with Zookeeper. Each command is composed of four letters such as `mntr`, `stat` etc. There are some more interesting commands: `stat` gives some general information about the server and connected clients, while `srvr` and `cons` give extended details on server and connections respectively.
The 4lw commands has a white list configuration `four_letter_word_white_list` which has default value "conf,cons,crst,envi,ruok,srst,srvr,stat,wchs,dirs,mntr,isro".
The 4lw commands has a allow list configuration `four_letter_word_allow_list` which has default value "conf,cons,crst,envi,ruok,srst,srvr,stat,wchs,dirs,mntr,isro".
You can issue the commands to ClickHouse Keeper via telnet or nc, at the client port.
@ -201,7 +201,7 @@ Server stats reset.
```
server_id=1
tcp_port=2181
four_letter_word_white_list=*
four_letter_word_allow_list=*
log_storage_path=./coordination/logs
snapshot_storage_path=./coordination/snapshots
max_requests_batch_size=100

6
docs/ru/operations/clickhouse-keeper.md
View File

@ -54,7 +54,7 @@ ClickHouse Keeper может использоваться как равноце
- `auto_forwarding` — разрешить пересылку запросов на запись от последователей лидеру (по умолчанию: true).
- `shutdown_timeout` — время ожидания завершения внутренних подключений и выключения, в миллисекундах (по умолчанию: 5000).
- `startup_timeout` — время отключения сервера, если он не подключается к другим участникам кворума, в миллисекундах (по умолчанию: 30000).
- `four_letter_word_white_list` — список разрешенных 4-х буквенных команд (по умолчанию: "conf,cons,crst,envi,ruok,srst,srvr,stat,wchc,wchs,dirs,mntr,isro").
- `four_letter_word_allow_list` — список разрешенных 4-х буквенных команд (по умолчанию: "conf,cons,crst,envi,ruok,srst,srvr,stat,wchs,dirs,mntr,isro").
Конфигурация кворума находится в `<keeper_server>.<raft_configuration>` и содержит описание серверов.
@ -114,7 +114,7 @@ clickhouse-keeper --config /etc/your_path_to_config/config.xml --daemon
ClickHouse Keeper также поддерживает 4-х буквенные команды, почти такие же, как у Zookeeper. Каждая команда состоит из 4-х символов, например, `mntr`, `stat` и т. д. Несколько интересных команд: `stat` предоставляет общую информацию о сервере и подключенных клиентах, а `srvr` и `cons` предоставляют расширенные сведения о сервере и подключениях соответственно.
У 4-х буквенных команд есть параметр для настройки разрешенного списка `four_letter_word_white_list`, который имеет значение по умолчанию "conf,cons,crst,envi,ruok,srst,srvr,stat, wchc,wchs,dirs,mntr,isro".
У 4-х буквенных команд есть параметр для настройки разрешенного списка `four_letter_word_allow_list`, который имеет значение по умолчанию "conf,cons,crst,envi,ruok,srst,srvr,stat,wchs,dirs,mntr,isro".
Вы можете отправлять команды в ClickHouse Keeper через telnet или nc на порт для клиента.
@ -194,7 +194,7 @@ Server stats reset.
```
server_id=1
tcp_port=2181
four_letter_word_white_list=*
four_letter_word_allow_list=*
log_storage_path=./coordination/logs
snapshot_storage_path=./coordination/snapshots
max_requests_batch_size=100

11
src/Coordination/CoordinationSettings.cpp
View File

@ -82,8 +82,8 @@ void KeeperConfigurationAndSettings::dump(WriteBufferFromOwnString & buf) const
write_int(tcp_port_secure);
}
writeText("four_letter_word_white_list=", buf);
writeText(four_letter_word_white_list, buf);
writeText("four_letter_word_allow_list=", buf);
writeText(four_letter_word_allow_list, buf);
buf.write('\n');
writeText("log_storage_path=", buf);
@ -177,7 +177,12 @@ KeeperConfigurationAndSettings::loadFromConfig(const Poco::Util::AbstractConfigu
ret->super_digest = config.getString("keeper_server.superdigest");
}
ret->four_letter_word_white_list = config.getString("keeper_server.four_letter_word_white_list", DEFAULT_FOUR_LETTER_WORD_CMD);
if (config.has("keeper_server.four_letter_word_allow_list"))
ret->four_letter_word_allow_list = config.getString("keeper_server.four_letter_word_allow_list");
if (config.has("keeper_server.four_letter_word_white_list"))
ret->four_letter_word_allow_list = config.getString("keeper_server.four_letter_word_white_list", DEFAULT_FOUR_LETTER_WORD_CMD);
ret->log_storage_path = getLogsPathFromConfig(config, standalone_keeper_);
ret->snapshot_storage_path = getSnapshotsPathFromConfig(config, standalone_keeper_);

2
src/Coordination/CoordinationSettings.h
View File

@ -68,7 +68,7 @@ struct KeeperConfigurationAndSettings
int tcp_port;
int tcp_port_secure;
String four_letter_word_white_list;
String four_letter_word_allow_list;
String super_digest;

16
src/Coordination/FourLetterCommand.cpp
View File

@ -129,7 +129,7 @@ void FourLetterCommandFactory::registerCommands(KeeperDispatcher & keeper_dispat
FourLetterCommandPtr watch_command = std::make_shared<WatchCommand>(keeper_dispatcher);
factory.registerCommand(watch_command);
factory.initializeWhiteList(keeper_dispatcher);
factory.initializeAllowList(keeper_dispatcher);
factory.setInitialize(true);
}
}
@ -137,17 +137,17 @@ void FourLetterCommandFactory::registerCommands(KeeperDispatcher & keeper_dispat
bool FourLetterCommandFactory::isEnabled(int32_t code)
{
checkInitialization();
if (!white_list.empty() && *white_list.cbegin() == WHITE_LIST_ALL)
if (!allow_list.empty() && *allow_list.cbegin() == ALLOW_LIST_ALL)
return true;
return std::find(white_list.begin(), white_list.end(), code) != white_list.end();
return std::find(allow_list.begin(), allow_list.end(), code) != allow_list.end();
}
void FourLetterCommandFactory::initializeWhiteList(KeeperDispatcher & keeper_dispatcher)
void FourLetterCommandFactory::initializeAllowList(KeeperDispatcher & keeper_dispatcher)
{
const auto & keeper_settings = keeper_dispatcher.getKeeperConfigurationAndSettings();
String list_str = keeper_settings->four_letter_word_white_list;
String list_str = keeper_settings->four_letter_word_allow_list;
Strings tokens;
splitInto<','>(tokens, list_str);
@ -157,15 +157,15 @@ void FourLetterCommandFactory::initializeWhiteList(KeeperDispatcher & keeper_dis
if (token == "*")
{
white_list.clear();
white_list.push_back(WHITE_LIST_ALL);
allow_list.clear();
allow_list.push_back(ALLOW_LIST_ALL);
return;
}
else
{
if (commands.contains(IFourLetterCommand::toCode(token)))
{
white_list.push_back(IFourLetterCommand::toCode(token));
allow_list.push_back(IFourLetterCommand::toCode(token));
}
else
{

12
src/Coordination/FourLetterCommand.h
View File

@ -40,10 +40,10 @@ struct FourLetterCommandFactory : private boost::noncopyable
{
public:
using Commands = std::unordered_map<int32_t, FourLetterCommandPtr>;
using WhiteList = std::vector<int32_t>;
using AllowList = std::vector<int32_t>;
///represent '*' which is used in white list
static constexpr int32_t WHITE_LIST_ALL = 0;
///represent '*' which is used in allow list
static constexpr int32_t ALLOW_LIST_ALL = 0;
bool isKnown(int32_t code);
bool isEnabled(int32_t code);
@ -52,7 +52,7 @@ public:
/// There is no need to make it thread safe, because registration is no initialization and get is after startup.
void registerCommand(FourLetterCommandPtr & command);
void initializeWhiteList(KeeperDispatcher & keeper_dispatcher);
void initializeAllowList(KeeperDispatcher & keeper_dispatcher);
void checkInitialization() const;
bool isInitialized() const { return initialized; }
@ -64,7 +64,7 @@ public:
private:
std::atomic<bool> initialized = false;
Commands commands;
WhiteList white_list;
AllowList allow_list;
};
/**Tests if server is running in a non-error state. The server will respond with imok if it is running.
@ -130,7 +130,7 @@ struct StatResetCommand : public IFourLetterCommand
};
/// A command that does not do anything except reply to client with predefined message.
///It is used to inform clients who execute none white listed four letter word commands.
///It is used to inform clients who execute none allow listed four letter word commands.
struct NopCommand : public IFourLetterCommand
{
explicit NopCommand(KeeperDispatcher & keeper_dispatcher_)

2
tests/integration/test_keeper_four_word_command/configs/enable_keeper1.xml
View File

@ -4,7 +4,7 @@
<server_id>1</server_id>
<log_storage_path>/var/lib/clickhouse/coordination/log</log_storage_path>
<snapshot_storage_path>/var/lib/clickhouse/coordination/snapshots</snapshot_storage_path>
<four_letter_word_white_list>*</four_letter_word_white_list>
<four_letter_word_allow_list>*</four_letter_word_allow_list>
<coordination_settings>
<operation_timeout_ms>5000</operation_timeout_ms>

2
tests/integration/test_keeper_four_word_command/configs/enable_keeper2.xml
View File

@ -4,7 +4,7 @@
<server_id>2</server_id>
<log_storage_path>/var/lib/clickhouse/coordination/log</log_storage_path>
<snapshot_storage_path>/var/lib/clickhouse/coordination/snapshots</snapshot_storage_path>
<four_letter_word_white_list>*</four_letter_word_white_list>
<four_letter_word_allow_list>*</four_letter_word_allow_list>
<coordination_settings>
<operation_timeout_ms>5000</operation_timeout_ms>

2
tests/integration/test_keeper_four_word_command/configs/enable_keeper3.xml
View File

@ -4,7 +4,7 @@
<server_id>3</server_id>
<log_storage_path>/var/lib/clickhouse/coordination/log</log_storage_path>
<snapshot_storage_path>/var/lib/clickhouse/coordination/snapshots</snapshot_storage_path>
<four_letter_word_white_list>*</four_letter_word_white_list>
<four_letter_word_allow_list>*</four_letter_word_allow_list>
<coordination_settings>
<operation_timeout_ms>5000</operation_timeout_ms>

2
tests/integration/test_keeper_four_word_command/configs/keeper_config_with_white_list.xml
View File

@ -2,7 +2,7 @@
<keeper_server>
<tcp_port>9181</tcp_port>
<server_id>1</server_id>
<four_letter_word_white_list>ruok, conf</four_letter_word_white_list>
<four_letter_word_allow_list>ruok, conf</four_letter_word_allow_list>
<raft_configuration>
<server>
<id>1</id>

2
tests/integration/test_keeper_four_word_command/configs/keeper_config_with_white_list_all.xml
View File

@ -2,7 +2,7 @@
<keeper_server>
<tcp_port>9181</tcp_port>
<server_id>3</server_id>
<four_letter_word_white_list>*</four_letter_word_white_list>
<four_letter_word_allow_list>*</four_letter_word_allow_list>
<raft_configuration>
<server>
<id>1</id>

2
tests/integration/test_keeper_four_word_command/test.py
View File

@ -281,7 +281,7 @@ def test_cmd_conf(started_cluster):
assert "tcp_port_secure" not in result
assert "superdigest" not in result
assert result["four_letter_word_white_list"] == "*"
assert result["four_letter_word_allow_list"] == "*"
assert result["log_storage_path"] == "/var/lib/clickhouse/coordination/log"
assert result["snapshot_storage_path"] == "/var/lib/clickhouse/coordination/snapshots"

10
tests/integration/test_keeper_four_word_command/test_white_list.py
View File

@ -3,10 +3,10 @@ import pytest
from helpers.cluster import ClickHouseCluster
import time
cluster = ClickHouseCluster(__file__, name="test_keeper_4lw_white_list")
node1 = cluster.add_instance('node1', main_configs=['configs/keeper_config_with_white_list.xml'], stay_alive=True)
node2 = cluster.add_instance('node2', main_configs=['configs/keeper_config_without_white_list.xml'], stay_alive=True)
node3 = cluster.add_instance('node3', main_configs=['configs/keeper_config_with_white_list_all.xml'], stay_alive=True)
cluster = ClickHouseCluster(__file__, name="test_keeper_4lw_allow_list")
node1 = cluster.add_instance('node1', main_configs=['configs/keeper_config_with_allow_list.xml'], stay_alive=True)
node2 = cluster.add_instance('node2', main_configs=['configs/keeper_config_without_allow_list.xml'], stay_alive=True)
node3 = cluster.add_instance('node3', main_configs=['configs/keeper_config_with_allow_list_all.xml'], stay_alive=True)
from kazoo.client import KazooClient, KazooState
@ -85,7 +85,7 @@ def send_cmd(node_name, command = "ruok"):
close_keeper_socket(client)
def test_white_list(started_cluster):
def test_allow_list(started_cluster):
client = None
try:
wait_nodes()