Commit Graph

21 Commits

Author SHA1 Message Date
Azat Khuzhin
c25d6cd624
Rename directory monitor concept into background INSERT (#55978)
* Limit log frequence for "Skipping send data over distributed table" message

After SYSTEM STOP DISTRIBUTED SENDS it will constantly print this
message.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename directory monitor concept into async INSERT

Rename the following query settings (with preserving backward
compatiblity, by keeping old name as an alias):
- distributed_directory_monitor_sleep_time_ms -> distributed_async_insert_sleep_time_ms
- distributed_directory_monitor_max_sleep_time_ms -> distributed_async_insert_max_sleep_time_ms
- distributed_directory_monitor_batch -> distributed_async_insert_batch_inserts
- distributed_directory_monitor_split_batch_on_failure -> distributed_async_insert_split_batch_on_failure

Rename the following table settings (with preserving backward
compatiblity, by keeping old name as an alias):
- monitor_batch_inserts -> async_insert_batch
- monitor_split_batch_on_failure -> async_insert_split_batch_on_failure
- directory_monitor_sleep_time_ms -> async_insert_sleep_time_ms
- directory_monitor_max_sleep_time_ms -> async_insert_max_sleep_time_ms

And also update all the references:

    $ gg -e directory_monitor_ -e monitor_ tests docs | cut -d: -f1 | sort -u | xargs sed -e 's/distributed_directory_monitor_sleep_time_ms/distributed_async_insert_sleep_time_ms/g' -e 's/distributed_directory_monitor_max_sleep_time_ms/distributed_async_insert_max_sleep_time_ms/g' -e 's/distributed_directory_monitor_batch_inserts/distributed_async_insert_batch/g' -e 's/distributed_directory_monitor_split_batch_on_failure/distributed_async_insert_split_batch_on_failure/g' -e 's/monitor_batch_inserts/async_insert_batch/g' -e 's/monitor_split_batch_on_failure/async_insert_split_batch_on_failure/g' -e 's/monitor_sleep_time_ms/async_insert_sleep_time_ms/g' -e 's/monitor_max_sleep_time_ms/async_insert_max_sleep_time_ms/g' -i

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Rename async_insert for Distributed into background_insert

This will avoid amigibuity between general async INSERT's and INSERT
into Distributed, which are indeed background, so new term express it
even better.

Mostly done with:

    $ git di HEAD^ --name-only | xargs sed -i -e 's/distributed_async_insert/distributed_background_insert/g' -e 's/async_insert_batch/background_insert_batch/g' -e 's/async_insert_split_batch_on_failure/background_insert_split_batch_on_failure/g' -e 's/async_insert_sleep_time_ms/background_insert_sleep_time_ms/g' -e 's/async_insert_max_sleep_time_ms/background_insert_max_sleep_time_ms/g'

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Mark 02417_opentelemetry_insert_on_distributed_table as long

CI: https://s3.amazonaws.com/clickhouse-test-reports/55978/7a6abb03a0b507e29e999cb7e04f246a119c6f28/stateless_tests_flaky_check__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-11-01 15:09:39 +01:00
Yarik Briukhovetskyi
a6f89c0546
Added suggestions for mistyped names for db and tables with different… (#49801)
* Added suggestions for mistyped names for db and tables with different scenarios commented

* fixed bugs

* fixed style check

* fixed errors

* fixed errors

* fixed error with exceptions

* fixed exceptions

* fixed exceptions

* fixed exceptions

* added test and fixed bugs

* fixed style check

* fixed style check

* fixed style check

* fixed check black

* Update test.py

* Fixed server crash

* Fixed server crash and style check

* Fixed style check

* Fixed style check

* Fixed style check

* Fixed bugs with drop_db

* fixed fast test

* added tests

* fixed style check

* fixed style check

* fixed bug with lock_db

* fixed bug with lock_db and fixed reviews

* fixed bug with lock_db and fixed reviews

* fixed style check

* fixed fast test

* fixed fast test

* revert to fd582a2

* revert to fd582a2

* Removed unused parameters

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* Remove unused parameters

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* resolved arguments issue in assertDatabaseExists

* fixing fast test

* fixed fast test

* fixed stateless test for default db

* Update src/Interpreters/DatabaseCatalog.cpp

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* Update src/Interpreters/DatabaseCatalog.cpp

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* Update src/Interpreters/DatabaseCatalog.cpp

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* Fixing tests.

* resolved problem with mutex

* Fixed mutex in assertDatabaseExists

* changes about assertDatabaseExists

* fixed bugs with file types

* fixed string types

* fixed fast test

* fixed mutex

* fixed mutex

* fixed style check

* Update src/Interpreters/DatabaseCatalog.cpp

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* Update src/Interpreters/DatabaseCatalog.cpp

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* Update src/Interpreters/DatabaseCatalog.cpp

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* Update src/Interpreters/DatabaseCatalog.cpp

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* Update src/Interpreters/DatabaseCatalog.cpp

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* Update src/Interpreters/DatabaseCatalog.cpp

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* fixed build

* added -unlocked versions of functions

* Revert "fixed build"

This reverts commit 8ce961be21.

* Revert "fixed build"

This reverts commit 8ce961be21.

* changed usage of assertDatabaseExistsUnlocked()

* fixed style check

* style check

* style check

* Revert "style check"

This reverts commit 28a9ee85a0.

* Merge branch 'master' into hints-for-wrong-db-or-table-name

* Changed AssertDatabaseExists and unified exception output

* resolved proposed changes and modified tests

* Revert "resolved proposed changes and modified tests"

This reverts commit d45337d65c.

* resolved requested changes

* fixed tests

* fixed tests

* fixed check black

* Update include brackets

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>

* Update suggested changes

* Update suggested changes

* Fixed style check

* Added test to analyzer_integration_broken

* Update DatabaseCatalog.cpp

* Update test.py

* fixed test

* Revert "fixed test"

This reverts commit ca6d4c17c8.

* fixed test

* Revert "fixed test"

This reverts commit fe6d0d9c86, reversing
changes made to 22f4496704.

* Update test.py

* fixed black check

* Update test.py

* fixed long_log_tinylog_deadlock_race

* Update DatabaseCatalog.cpp

* Update test.py

* style

* Update DatabaseCatalog.cpp

* Fixed test

* implemented for IDatabase

* Style check

* removed const_cast

* Update DatabaseCatalog.h

* Update DatabaseCatalog.h

* Update DatabaseCatalog.cpp

* Update DatabaseCatalog.cpp

* Added db name to hints

* Update 00002_log_and_exception_messages_formatting.sql

* Update 00002_log_and_exception_messages_formatting.sql

---------

Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
2023-08-11 13:24:16 +03:00
Alexander Tokmakov
cad1e0b768 fix 2023-02-25 01:18:34 +01:00
Alexander Tokmakov
94dd80e900 fix tests 2022-06-24 19:10:33 +02:00
Alexander Tokmakov
f00e6b5a7a deprecate old MergeTree syntax 2022-06-23 11:24:54 +02:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
Yatsishin Ilya
9d159072ef mERGe remote-tracking branch 'origin' into integration-2 2021-05-11 13:32:44 +03:00
Azat Khuzhin
a958f4178b Fix test_insert_into_distributed flaps
In [1] the order of the tests was different:

    Running pytest container as: 'docker run --net=host  --rm --name clickhouse_integration_tests --privileged         --volume=/place/sandbox-data/tasks/7/4/962738347/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/place/sandbox-data/tasks/7/4/962738347/clickhouse:/clickhouse         --volume=/place/sandbox-data/tasks/7/4/962738347/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/place/sandbox-data/tasks/7/4/962738347/clickhouse:/clickhouse         --volume=/place/sandbox-data/tasks/7/4/962738347/ClickHouse/programs/server:/clickhouse-config --volume=/place/sandbox-data/tasks/7/4/962738347/ClickHouse/tests/integration:/ClickHouse/tests/integration         --volume=/place/sandbox-data/tasks/7/4/962738347/ClickHouse/src/Server/grpc_protos:/ClickHouse/src/Server/grpc_protos         --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=latest -e DOCKER_MYSQL_JAVA_CLIENT_TAG=latest -e DOCKER_MYSQL_JS_CLIENT_TAG=latest -e DOCKER_MYSQL_PHP_CLIENT_TAG=latest -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=latest -e DOCKER_BASE_TAG=latest -e DOCKER_KERBEROS_KDC_TAG=latest -e PYTEST_OPTS='-ss test_insert_into_distributed/test.py::test_inserts_batching test_insert_into_distributed/test.py::test_inserts_local test_insert_into_distributed/test.py::test_inserts_low_cardinality test_insert_into_distributed/test.py::test_inserts_single_replica_internal_replication test_insert_into_distributed/test.py::test_inserts_single_replica_local_internal_replication test_insert_into_distributed/test.py::test_inserts_single_replica_no_internal_replication test_insert_into_distributed/test.py::test_prefer_localhost_replica test_insert_into_distributed/test.py::test_reconnect test_insert_into_distributed/test.py::test_table_function -rfEp --color=no --durations=0 ' yandex/clickhouse-integration-tests-runner:latest '.

  [1]: https://clickhouse-test-reports.s3.yandex.net/23874/80065b08f8182936ac57408bd84492d95f2322c1/integration_tests_(asan).html#fail1

So the table should be TRUNCATEd to avoid failures in this case.
2021-05-05 21:18:43 +03:00
Yatsishin Ilya
744fb93945 Merge remote-tracking branch 'origin' into integration-2 2021-04-30 12:18:24 +03:00
Yatsishin Ilya
a6c7e03aba merge remote-tracking branch 'origin' into integration-2 2021-04-29 17:30:54 +03:00
Azat Khuzhin
c4a1aa175c Improve test_insert_into_distributed 2021-04-29 10:35:09 +03:00
Amos Bird
fd3aef0feb
Update tests/integration/test_insert_into_distributed/test.py
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2021-04-28 12:16:27 +08:00
Amos Bird
3d125e64a0
Try fix test 2021-04-27 18:33:37 +08:00
Amos Bird
a3b8681a5f
add test 2021-04-21 11:25:59 +08:00
Yatsishin Ilya
06bb4e2c71 wip 2021-03-16 13:00:49 +03:00
Azat Khuzhin
471deab63a Rename fsync_tmp_directory to fsync_directories for Distributed engine 2021-01-09 17:51:30 +03:00
Azat Khuzhin
9cdb1d4ae6 Simple coverage of fsync_after_insert/fsync_tmp_directory with batching 2021-01-09 16:31:52 +03:00
bharatnc
b75963d370 Format and optimize imports in integration test files
This PR formats all the `*.py` files found under the `tests/integration`
folder. It also reorders the imports and cleans up a bunch of unused
imports.

The formatting also takes care of other things like wrapping lines and
fixing spaces and indents such that the tests look more readable.
2020-09-15 23:10:54 -07:00
alesapin
eb2236fe8b Fix configs 2020-08-03 14:13:43 +03:00
Alexey Milovidov
e06dc573b4 Fix flacky integration test 2020-04-20 06:15:29 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up (#9974)
* Move some code outside dbms/src folder
* Fix paths
2020-04-02 02:51:21 +03:00