thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-14 03:25:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
0f30513652
ClickHouse/tests/integration/test_allowed_url_from_config/test.py

132 lines
8.7 KiB
Python
Raw Normal View History

Add integration tests.
2019-10-06 14:52:41 +00:00
import pytest
from helpers.cluster import ClickHouseCluster
cluster = ClickHouseCluster(__file__)
node1 = cluster.add_instance('node1', main_configs=['configs/config_with_hosts.xml'])
node2 = cluster.add_instance('node2', main_configs=['configs/config_with_only_primary_hosts.xml'])
node3 = cluster.add_instance('node3', main_configs=['configs/config_with_only_regexp_hosts.xml'])
Minor fix.
2021-02-04 07:41:47 +00:00
node4 = cluster.add_instance('node4', main_configs=[]) # No `remote_url_allow_hosts` at all.
node5 = cluster.add_instance('node5', main_configs=['configs/config_without_allowed_hosts.xml'])
Created class StorageOfAllowedURL Created test for table function remote and remoteSecure Created check allowed url in remote and remoteSecure Fixed tests
2019-10-09 20:29:41 +00:00
node6 = cluster.add_instance('node6', main_configs=['configs/config_for_remote.xml'])
Fixed bugs
2019-10-23 11:58:35 +00:00
node7 = cluster.add_instance('node7', main_configs=['configs/config_for_redirect.xml'], with_hdfs=True)
Add integration tests.
2019-10-06 14:52:41 +00:00
Created test for HDFS Storage and hdfs table function Deleted extra tests
2019-11-05 15:30:35 +00:00
Add integration tests.
2019-10-06 14:52:41 +00:00
@pytest.fixture(scope="module")
def start_cluster():
try:
cluster.start()
yield cluster
finally:
cluster.shutdown()
Format and optimize imports in integration test files This PR formats all the `*.py` files found under the `tests/integration` folder. It also reorders the imports and cleans up a bunch of unused imports. The formatting also takes care of other things like wrapping lines and fixing spaces and indents such that the tests look more readable.
2020-09-16 04:26:10 +00:00
Add integration tests.
2019-10-06 14:52:41 +00:00
def test_config_with_hosts(start_cluster):
Created class StorageOfAllowedURL Created test for table function remote and remoteSecure Created check allowed url in remote and remoteSecure Fixed tests
2019-10-09 20:29:41 +00:00
assert node1.query("CREATE TABLE table_test_1_1 (word String) Engine=URL('http://host:80', HDFS)") == ""
Add integration tests.
2019-10-06 14:52:41 +00:00
assert node1.query("CREATE TABLE table_test_1_2 (word String) Engine=URL('https://yandex.ru', CSV)") == ""
Format and optimize imports in integration test files This PR formats all the `*.py` files found under the `tests/integration` folder. It also reorders the imports and cleans up a bunch of unused imports. The formatting also takes care of other things like wrapping lines and fixing spaces and indents such that the tests look more readable.
2020-09-16 04:26:10 +00:00
assert "not allowed" in node1.query_and_get_error(
"CREATE TABLE table_test_1_4 (word String) Engine=URL('https://host:123', S3)")
assert "not allowed" in node1.query_and_get_error(
"CREATE TABLE table_test_1_4 (word String) Engine=URL('https://yandex2.ru', CSV)")
Add integration tests.
2019-10-06 14:52:41 +00:00
def test_config_with_only_primary_hosts(start_cluster):
assert node2.query("CREATE TABLE table_test_2_1 (word String) Engine=URL('https://host:80', CSV)") == ""
Created class StorageOfAllowedURL Created test for table function remote and remoteSecure Created check allowed url in remote and remoteSecure Fixed tests
2019-10-09 20:29:41 +00:00
assert node2.query("CREATE TABLE table_test_2_2 (word String) Engine=URL('https://host:123', S3)") == ""
Add integration tests.
2019-10-06 14:52:41 +00:00
assert node2.query("CREATE TABLE table_test_2_3 (word String) Engine=URL('https://yandex.ru', CSV)") == ""
Created class StorageOfAllowedURL Created test for table function remote and remoteSecure Created check allowed url in remote and remoteSecure Fixed tests
2019-10-09 20:29:41 +00:00
assert node2.query("CREATE TABLE table_test_2_4 (word String) Engine=URL('https://yandex.ru:87', HDFS)") == ""
Format and optimize imports in integration test files This PR formats all the `*.py` files found under the `tests/integration` folder. It also reorders the imports and cleans up a bunch of unused imports. The formatting also takes care of other things like wrapping lines and fixing spaces and indents such that the tests look more readable.
2020-09-16 04:26:10 +00:00
assert "not allowed" in node2.query_and_get_error(
"CREATE TABLE table_test_2_5 (word String) Engine=URL('https://host', HDFS)")
assert "not allowed" in node2.query_and_get_error(
"CREATE TABLE table_test_2_5 (word String) Engine=URL('https://host:234', CSV)")
assert "not allowed" in node2.query_and_get_error(
"CREATE TABLE table_test_2_6 (word String) Engine=URL('https://yandex2.ru', S3)")
Add integration tests.
2019-10-06 14:52:41 +00:00
def test_config_with_only_regexp_hosts(start_cluster):
Created class StorageOfAllowedURL Created test for table function remote and remoteSecure Created check allowed url in remote and remoteSecure Fixed tests
2019-10-09 20:29:41 +00:00
assert node3.query("CREATE TABLE table_test_3_1 (word String) Engine=URL('https://host:80', HDFS)") == ""
Add integration tests.
2019-10-06 14:52:41 +00:00
assert node3.query("CREATE TABLE table_test_3_2 (word String) Engine=URL('https://yandex.ru', CSV)") == ""
Format and optimize imports in integration test files This PR formats all the `*.py` files found under the `tests/integration` folder. It also reorders the imports and cleans up a bunch of unused imports. The formatting also takes care of other things like wrapping lines and fixing spaces and indents such that the tests look more readable.
2020-09-16 04:26:10 +00:00
assert "not allowed" in node3.query_and_get_error(
"CREATE TABLE table_test_3_3 (word String) Engine=URL('https://host', CSV)")
assert "not allowed" in node3.query_and_get_error(
"CREATE TABLE table_test_3_4 (word String) Engine=URL('https://yandex2.ru', S3)")
Add integration tests.
2019-10-06 14:52:41 +00:00
Minor fix.
2021-02-04 07:41:47 +00:00
def test_config_without_allowed_hosts_section(start_cluster):
Add integration tests.
2019-10-06 14:52:41 +00:00
assert node4.query("CREATE TABLE table_test_4_1 (word String) Engine=URL('https://host:80', CSV)") == ""
Added test for S3 table function.
2021-02-08 06:22:00 +00:00
assert node4.query("CREATE TABLE table_test_4_2 (word String) Engine=S3('https://host:80/bucket/key', CSV)") == ""
assert node4.query("CREATE TABLE table_test_4_3 (word String) Engine=URL('https://host', HDFS)") == ""
assert node4.query("CREATE TABLE table_test_4_4 (word String) Engine=URL('https://yandex.ru', CSV)") == ""
assert node4.query("CREATE TABLE table_test_4_5 (word String) Engine=URL('ftp://something.com', S3)") == ""
Fixed open behavior of remote host filter in case when there is remote_url_allow_hosts section in configuration but no entries there.
2021-02-04 06:21:05 +00:00
Minor fix.
2021-02-04 07:41:47 +00:00
def test_config_without_allowed_hosts(start_cluster):
assert "not allowed" in node5.query_and_get_error(
"CREATE TABLE table_test_5_1 (word String) Engine=URL('https://host:80', CSV)")
assert "not allowed" in node5.query_and_get_error(
Added test for S3 table function.
2021-02-08 06:22:00 +00:00
"CREATE TABLE table_test_5_2 (word String) Engine=S3('https://host:80/bucket/key', CSV)")
Minor fix.
2021-02-04 07:41:47 +00:00
assert "not allowed" in node5.query_and_get_error(
Added test for S3 table function.
2021-02-08 06:22:00 +00:00
"CREATE TABLE table_test_5_3 (word String) Engine=URL('https://host', HDFS)")
Minor fix.
2021-02-04 07:41:47 +00:00
assert "not allowed" in node5.query_and_get_error(
Added test for S3 table function.
2021-02-08 06:22:00 +00:00
"CREATE TABLE table_test_5_4 (word String) Engine=URL('https://yandex.ru', CSV)")
assert "not allowed" in node5.query_and_get_error(
"CREATE TABLE table_test_5_5 (word String) Engine=URL('ftp://something.com', S3)")
Add integration tests.
2019-10-06 14:52:41 +00:00
Format and optimize imports in integration test files This PR formats all the `*.py` files found under the `tests/integration` folder. It also reorders the imports and cleans up a bunch of unused imports. The formatting also takes care of other things like wrapping lines and fixing spaces and indents such that the tests look more readable.
2020-09-16 04:26:10 +00:00
Created test for HDFS Storage and hdfs table function Deleted extra tests
2019-11-05 15:30:35 +00:00
def test_table_function_remote(start_cluster):
Format and optimize imports in integration test files This PR formats all the `*.py` files found under the `tests/integration` folder. It also reorders the imports and cleans up a bunch of unused imports. The formatting also takes care of other things like wrapping lines and fixing spaces and indents such that the tests look more readable.
2020-09-16 04:26:10 +00:00
assert "not allowed in config.xml" not in node6.query_and_get_error(
"SELECT * FROM remoteSecure('example01-01-{1|2}', system, events)",
settings={"connections_with_failover_max_tries": 1, "connect_timeout_with_failover_ms": 1000,
"connect_timeout_with_failover_secure_ms": 1000, "connect_timeout": 1, "send_timeout": 1})
assert "not allowed in config.xml" not in node6.query_and_get_error(
"SELECT * FROM remoteSecure('example01-01-1,example01-02-1', system, events)",
settings={"connections_with_failover_max_tries": 1, "connect_timeout_with_failover_ms": 1000,
"connect_timeout_with_failover_secure_ms": 1000, "connect_timeout": 1, "send_timeout": 1})
assert "not allowed in config.xml" not in node6.query_and_get_error(
"SELECT * FROM remote('example01-0{1,2}-1', system, events",
settings={"connections_with_failover_max_tries": 1, "connect_timeout_with_failover_ms": 1000,
"connect_timeout_with_failover_secure_ms": 1000, "connect_timeout": 1, "send_timeout": 1})
assert "not allowed in config.xml" not in node6.query_and_get_error(
"SELECT * FROM remote('example01-0{1,2}-{1|2}', system, events)",
settings={"connections_with_failover_max_tries": 1, "connect_timeout_with_failover_ms": 1000,
"connect_timeout_with_failover_secure_ms": 1000, "connect_timeout": 1, "send_timeout": 1})
assert "not allowed in config.xml" not in node6.query_and_get_error(
"SELECT * FROM remoteSecure('example01-{01..02}-{1|2}', system, events)",
settings={"connections_with_failover_max_tries": 1, "connect_timeout_with_failover_ms": 1000,
"connect_timeout_with_failover_secure_ms": 1000, "connect_timeout": 1, "send_timeout": 1})
assert "not allowed" in node6.query_and_get_error(
"SELECT * FROM remoteSecure('example01-01-1,example01-03-1', system, events)",
settings={"connections_with_failover_max_tries": 1, "connect_timeout_with_failover_ms": 1000,
"connect_timeout_with_failover_secure_ms": 1000, "connect_timeout": 1, "send_timeout": 1})
assert "not allowed" in node6.query_and_get_error("SELECT * FROM remote('example01-01-{1|3}', system, events)",
settings={"connections_with_failover_max_tries": 1,
"connect_timeout_with_failover_ms": 1000,
"connect_timeout_with_failover_secure_ms": 1000,
"connect_timeout": 1, "send_timeout": 1})
assert "not allowed" in node6.query_and_get_error(
"SELECT * FROM remoteSecure('example01-0{1,3}-1', system, metrics)",
settings={"connections_with_failover_max_tries": 1, "connect_timeout_with_failover_ms": 1000,
"connect_timeout_with_failover_secure_ms": 1000, "connect_timeout": 1, "send_timeout": 1})
Created class StorageOfAllowedURL Created test for table function remote and remoteSecure Created check allowed url in remote and remoteSecure Fixed tests
2019-10-09 20:29:41 +00:00
assert node6.query("SELECT * FROM remote('localhost', system, events)") != ""
assert node6.query("SELECT * FROM remoteSecure('localhost', system, metrics)") != ""
Format and optimize imports in integration test files This PR formats all the `*.py` files found under the `tests/integration` folder. It also reorders the imports and cleans up a bunch of unused imports. The formatting also takes care of other things like wrapping lines and fixing spaces and indents such that the tests look more readable.
2020-09-16 04:26:10 +00:00
assert "URL \"localhost:800\" is not allowed in config.xml" in node6.query_and_get_error(
"SELECT * FROM remoteSecure('localhost:800', system, events)")
assert "URL \"localhost:800\" is not allowed in config.xml" in node6.query_and_get_error(
"SELECT * FROM remote('localhost:800', system, metrics)")
Fixed bugs
2019-10-23 11:58:35 +00:00
def test_redirect(start_cluster):
wip
2021-06-09 09:23:02 +00:00
hdfs_api = start_cluster.hdfs_api
wip
2021-02-19 12:58:11 +00:00
hdfs_api.write_data("/simple_storage", "1\t\n")
assert hdfs_api.read_data("/simple_storage") == "1\t\n"
Format and optimize imports in integration test files This PR formats all the `*.py` files found under the `tests/integration` folder. It also reorders the imports and cleans up a bunch of unused imports. The formatting also takes care of other things like wrapping lines and fixing spaces and indents such that the tests look more readable.
2020-09-16 04:26:10 +00:00
node7.query(
"CREATE TABLE table_test_7_1 (word String) ENGINE=URL('http://hdfs1:50070/webhdfs/v1/simple_storage?op=OPEN&namenoderpcaddress=hdfs1:9000&offset=0', CSV)")
Created test for HDFS Storage and hdfs table function Deleted extra tests
2019-11-05 15:30:35 +00:00
assert "not allowed" in node7.query_and_get_error("SET max_http_get_redirects=1; SELECT * from table_test_7_1")
Format and optimize imports in integration test files This PR formats all the `*.py` files found under the `tests/integration` folder. It also reorders the imports and cleans up a bunch of unused imports. The formatting also takes care of other things like wrapping lines and fixing spaces and indents such that the tests look more readable.
2020-09-16 04:26:10 +00:00
Created test for HDFS Storage and hdfs table function Deleted extra tests
2019-11-05 15:30:35 +00:00
def test_HDFS(start_cluster):
Format and optimize imports in integration test files This PR formats all the `*.py` files found under the `tests/integration` folder. It also reorders the imports and cleans up a bunch of unused imports. The formatting also takes care of other things like wrapping lines and fixing spaces and indents such that the tests look more readable.
2020-09-16 04:26:10 +00:00
assert "not allowed" in node7.query_and_get_error(
"CREATE TABLE table_test_7_2 (word String) ENGINE=HDFS('http://hdfs1:50075/webhdfs/v1/simple_storage?op=OPEN&namenoderpcaddress=hdfs1:9000&offset=0', 'CSV')")
assert "not allowed" in node7.query_and_get_error(
"SELECT * FROM hdfs('http://hdfs1:50075/webhdfs/v1/simple_storage?op=OPEN&namenoderpcaddress=hdfs1:9000&offset=0', 'TSV', 'word String')")
Reference in New Issue Copy Permalink