Commit Graph

2786 Commits

Author SHA1 Message Date
Azat Khuzhin
8ef677b15f Add ability to set any rocksdb option via config.xml
v2: Cover rocksdb options in ClickHouse config
v3: add missing __init__.py
v4: Rework rocksdb options from config
v5: add column_family_options support
2021-07-27 08:40:50 +03:00
Alexander Tokmakov
a7e644f774 support old name 2021-07-26 21:24:32 +03:00
Alexander Tokmakov
63ab38ee09 rename MaterializeMySQL to MaterializedMySQL 2021-07-26 21:17:28 +03:00
Vitaly Baranov
4862e9f80d
Merge pull request #26733 from vitlibar/support-multiple-keys
Support multiple keys for encrypted disk
2021-07-26 10:19:50 +03:00
Vitaly Baranov
51f882d8cc Support multiple keys for encrypted disk. Display error message if the key is probably wrong. 2021-07-24 23:45:12 +03:00
Vitaly Baranov
22728cbca0 Add new functions currentRoles(), enabledRoles(), defaultRoles(). 2021-07-24 23:20:40 +03:00
Vitaly Baranov
7b4e5f8e21
Merge pull request #26759 from vitlibar/changing-default-roles-affects-new-sessions
Changing default roles affects new sessions only.
2021-07-24 23:15:14 +03:00
alexey-milovidov
967fe89917
Merge pull request #26730 from vitlibar/set-profile-applies-constraints-too
SET PROFILE applies constraints too.
2021-07-24 18:53:36 +03:00
Vitaly Baranov
249ccd879e SET PROFILE applies constraints too. 2021-07-23 23:28:55 +03:00
Vitaly Baranov
db97921b5b Changing default roles affects new sessions only. 2021-07-23 23:23:14 +03:00
Vitaly Baranov
c68c74634d
Merge pull request #26707 from vitlibar/fix-set-role
Fix SET ROLE
2021-07-23 23:16:15 +03:00
Vladimir
817ed354ff
Merge pull request #26673 from vdimir/fix_test_materialize_mysql_database 2021-07-23 11:26:19 +03:00
ANDREI STAROVEROV
263c3f9ff3 Add __init__.py for test_access_for_functions 2021-07-23 10:27:35 +03:00
Vitaly Baranov
19d5a6ab2f
Merge pull request #26714 from vitlibar/new-function-current-profiles
New functions currentProfiles(), enabledProfiles(), defaultProfiles().
2021-07-23 09:10:29 +03:00
Vitaly Baranov
7afcc65060 Add new functions currentProfiles(), enabledProfiles(), defaultProfiles(). 2021-07-22 22:20:53 +03:00
ANDREI STAROVEROV
21c577d337 Rename __main__ to __init__ in tests 2021-07-22 20:13:19 +03:00
Vitaly Baranov
d0412994a2 Fix SET ROLE. 2021-07-22 17:41:52 +03:00
Vladimir
ecebd5c132
Merge pull request #26691 from vdimir/fix-ci-runner-listing-error 2021-07-22 15:43:06 +03:00
vdimir
45233666b6
Relax filter for information_schema.processlist in mysql_kill_sync_thread_restore_test 2021-07-22 14:05:23 +03:00
vdimir
f61ecee989
Remove tests/integration/test_foo 2021-07-22 13:33:43 +03:00
vdimir
d1b6274deb
Strip lines in runner output in _get_all_tests 2021-07-22 13:33:05 +03:00
vdimir
a8f986350c
Log errors on integration test listing error in ci-runner 2021-07-22 12:31:15 +03:00
Nikolai Kochetov
5ffd99dfd4
Merge pull request #25674 from amosbird/distributedreturnconnection
Drain connection asynchronously
2021-07-22 11:04:49 +03:00
vdimir
1d0ceff2dc
Fix mysql_kill_sync_thread_restore_test 2021-07-21 18:17:45 +03:00
ANDREI STAROVEROV
c3461ce99a Fix integration tests 2021-07-21 16:04:52 +03:00
ANDREI STAROVEROV
1c99253c6a Fix integration tests 2021-07-21 14:10:37 +03:00
ANDREI STAROVEROV
503d62cce1 Fix integration tests 2021-07-21 12:41:17 +03:00
Realist007
31e26862b5
Merge branch 'master' into feature/create-simple-lambda-function 2021-07-21 10:33:53 +03:00
ANDREI STAROVEROV
31f4fe9a4d Fix tests 2021-07-21 10:29:26 +03:00
ANDREI STAROVEROV
44e1c2bcf5 Fix test for check access rights for functions 2021-07-20 23:42:48 +03:00
ANDREI STAROVEROV
f4e2e552eb Add integration test for persistence 2021-07-20 23:42:23 +03:00
alexey-milovidov
3bcef76a9d
Merge pull request #26529 from kssenii/fixing-rabbitmq
Fix rabbitmq shutdown in case rabbitmq setup was not started
2021-07-20 16:41:25 +03:00
Nikita Mikhaylov
47c1bb3416
Merge pull request #25822 from ClickHouse/fix_no_column_materialized_mysql
Fix 'Not found column' error in MaterializeMySQL
2021-07-20 15:03:54 +03:00
ANDREI STAROVEROV
6ce203504b Add access rights 2021-07-20 02:34:04 +03:00
kssenii
63c05e6e14 Fix rabbitmq shutdown in case rabbitmq setup was not started 2021-07-19 21:06:10 +00:00
Amos Bird
dbfb699690
Asynchronously drain connections. 2021-07-19 21:53:29 +08:00
mergify[bot]
39f71f4a08
Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-19 12:27:38 +00:00
Yatsishin Ilya
cdc8f2f743 fix new test 2021-07-19 11:32:20 +03:00
Yatsishin Ilya
2ae7d72dc9 merge remote-tracking branch 'origin' into integration-6 2021-07-19 09:49:17 +03:00
Vitaly Baranov
1fc822b19d
Merge pull request #26465 from vitlibar/explicit-encryption-mode
Set encryption algorithm explicitly.
2021-07-19 09:38:02 +03:00
Vitaly Baranov
81ea8521b1 Set encryption algorithm explicitly. 2021-07-18 10:29:42 +03:00
alexey-milovidov
7dda0da027
Merge pull request #26461 from azat/integration-tests-fixes-v21.9.1.7477
Fix flaky test_replicated_mutations (due to lack of threads in pool)
2021-07-17 19:22:30 +03:00
Azat Khuzhin
bb22615fad Fix flaky test_replicated_mutations (number_of_free_entries_in_pool_to_execute_mutation=0)
CI reports [1]:

    2021.07.16 09:09:21.040944 [ 172 ] {} <Debug> DiskLocal: Reserving 1.00 MiB on disk `default`, having unreserved 18.00 GiB.
    ...
    2021.07.16 09:09:21.041200 [ 28 ] {} <Debug> default.test_mutations (ReplicatedMergeTreeQueue): Not executing log entry queue-0000000288 of type MUTATE_PART for part 200201_3_3_0_16 because source parts size (217.00 B) is greater than the current maximum (0.00 B).
    2021.07.16 09:09:21.041529 [ 171 ] {} <Debug> DiskLocal: Reserving 1.00 MiB on disk `default`, having unreserved 18.00 GiB.
    2021.07.16 09:09:21.041838 [ 34 ] {} <Debug> default.test_mutations (ReplicatedMergeTreeQueue): Not executing log entry queue-0000000288 of type MUTATE_PART for part 200201_3_3_0_16 because source parts size (217.00 B) is greater than the current maximum (0.00 B).
    2021.07.16 09:09:21.042359 [ 34 ] {} <Debug> default.test_mutations (ReplicatedMergeTreeQueue): Not executing log entry queue-0000000290 of type MUTATE_PART for part 200201_12_12_0_16 because source parts size (237.00 B) is greater than the current maximum (0.00 B).

  [1]: https://clickhouse-test-reports.s3.yandex.net/26336/a3653bd665bd59c7c05b0fcd58ea2246e0b69915/integration_tests_(thread).html#fail1
2021-07-17 11:50:04 +03:00
mergify[bot]
89634ae224
Merge branch 'master' into estimates_for_select_query 2021-07-16 23:15:30 +00:00
alexey-milovidov
38dfe1fc0a
Merge pull request #26172 from ClickHouse/exception_id
Add error id to exceptions
2021-07-17 02:07:17 +03:00
Vitaly Baranov
958494b98e
Merge pull request #26377 from vitlibar/encrypted-disk-comments-and-improvements
Add comments for encrypted disks; some improvements in code.
2021-07-16 21:24:51 +03:00
Yatsishin Ilya
d11ec777c9 Merge remote-tracking branch 'origin' into integration-6 2021-07-16 16:28:02 +03:00
Yatsishin Ilya
e6b724ad3f postgres rework 2021-07-16 16:26:35 +03:00
alexey-milovidov
72ed5e0dfb
Merge pull request #26031 from nvartolomei/nv/test-cte-distributed
Test CTE with aggregation over distributed table
2021-07-16 12:24:29 +03:00
Vitaly Baranov
9cc258327e Add comments for encrypted disks; some improvements in code. 2021-07-16 12:04:25 +03:00
Vitaly Baranov
427813071d
Merge pull request #26363 from vitlibar/fix-crash-login-as-dropped-user
Fix possible crash when login as dropped user
2021-07-16 09:47:35 +03:00
Yatsishin Ilya
aa698f6c98 better 2021-07-15 15:49:52 +03:00
Yatsishin Ilya
be9a0a9ca2 better test_storage_s3, test_storage_s3_get_unstable still not OK 2021-07-15 15:49:25 +03:00
Yatsishin Ilya
51fcaa906c add sync for on cluster ddl 2021-07-15 15:48:09 +03:00
Vitaly Baranov
c2bb0a02a8 Add test. 2021-07-15 14:36:56 +03:00
Alexey Milovidov
d984fc4301 Fix flaky integration test 2021-07-15 13:35:37 +03:00
vdimir
dd3e1a8a4d
Adjust exception message in test_replicated_fetches_timeouts 2021-07-15 11:58:21 +03:00
alexey-milovidov
21f2b006cd
Update test_cte_distributed.py 2021-07-15 08:17:22 +03:00
Kseniia Sumarokova
e42a26a585
Merge pull request #26234 from jasine/master
convert timestamp and  timestamptz data types to DateTime64 in postgres engine
2021-07-14 23:42:51 +03:00
Peng Jian
eed01abf65 fix test case 2021-07-14 20:42:02 +08:00
vdimir
55d12a497c
Adjust integration test to exception messages 2021-07-14 14:50:27 +03:00
Zhichang Yu
fe5d17680e Merge remote-tracking branch 'origin/master' into zero_copy_hdfs 2021-07-14 02:12:28 +00:00
jasine
bd59ca22df
fix: style and MaterializedPostgreSQL test (#3)
fix: style and MaterializedPostgreSQL test (#3)
2021-07-14 08:51:14 +08:00
alexey-milovidov
c01497a80a
Merge pull request #26028 from nvartolomei/nv/aggregate-alias-column
Add test case for aggregation w/ alias columns
2021-07-14 03:30:14 +03:00
mergify[bot]
d5898ecba6
Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-13 14:03:05 +00:00
alexey-milovidov
2dafd9d4a8
Update test_select_aggregate_alias_column.py 2021-07-13 11:39:15 +03:00
Vitaly Baranov
e086e5e384
Merge branch 'master' into alexelex-encryption-1 2021-07-13 11:28:41 +03:00
jasine
9832c48c33
fix postgres test and array of DateTime64 (#1)
fix postgres test and array of DateTime64
2021-07-13 15:28:15 +08:00
Zhichang Yu
a9028dbd7d try fix test_replicated_merge_tree_hdfs_zero_copy/test.py failures under tsan 2021-07-13 07:20:23 +00:00
Zhichang Yu
b4e6689bf9 fix test_hdfs_zero_copy_replication_move[tiered_copy-2] 2021-07-13 07:20:23 +00:00
Zhichang Yu
5047c758f4 fix per review 2021-07-13 07:20:20 +00:00
Zhichang Yu
fbd5eee8a1 hdfs zero copy 2021-07-13 07:19:12 +00:00
Anton Popov
313cd0077d
Fix output of TSV in integration tests 2021-07-12 20:23:19 +03:00
Yatsishin Ilya
a4a960328e Merge remote-tracking branch 'origin' into integration-6 2021-07-12 11:34:23 +03:00
Yatsishin Ilya
143368d8d6 wip 2021-07-12 11:32:20 +03:00
Alexey Milovidov
57b5c0e1e5 Adjust integration tests 2021-07-12 08:59:01 +03:00
alexey-milovidov
e0c095397d
Update test_select_aggregate_alias_column.py 2021-07-12 00:36:27 +03:00
Alexey Milovidov
9c0097cd9b Review fixes 2021-07-11 02:14:03 +03:00
Alexey Milovidov
78cde85ea8 Merge branch 'master' into estimates_for_select_query 2021-07-11 02:11:16 +03:00
alexey-milovidov
7a1a184c48
Merge pull request #24404 from nvartolomei/nv/config-in-place-include
Support for multiple includes in XML based configuration
2021-07-10 10:10:18 +03:00
Peng Jian
24f0c738df test case 2021-07-09 22:26:08 +08:00
Nikita Mikhaylov
334d5439c8 done 2021-07-08 20:29:25 +00:00
Maksim Kita
42e1844135
Merge pull request #26023 from vitlibar/replace-print-with-logging-in-integration-tests
Replace print() with logging.debug() in integration tests.
2021-07-07 21:00:15 +03:00
alesapin
aa86fac4e7 Fix unbundled build 2021-07-07 14:07:20 +03:00
alesapin
f0fc12d0dd Merge branch 'master' into evillique-nlp 2021-07-07 12:14:49 +03:00
alesapin
4c85dae572
Merge pull request #25743 from ClickHouse/fix_aggregation_ttl
Fix bug in execution of TTL GROUP BY
2021-07-07 10:49:16 +03:00
Vitaly Baranov
b0a3a7180f Replace print() with logging.debug() in integration tests 2021-07-07 10:35:11 +03:00
Kseniia Sumarokova
7a7b774ba3
Merge pull request #25550 from kssenii/truncate
Allow truncate in storages s3/hdfs
2021-07-06 22:07:53 +03:00
Nicolae Vartolomei
3b5c41fc6d Test CTE with aggregation over distributed table 2021-07-06 14:04:49 +01:00
Nicolae Vartolomei
a5c195fefd Add test case for aggregation w/ alias columns 2021-07-06 13:26:58 +01:00
Vitaly Baranov
e7b1e9a2fb
Merge pull request #25986 from vitlibar/fix-flaky-test_dictionaries_all_layouts_separate_sources-test_https
Fix parallel execution of integration tests
2021-07-06 09:22:06 +03:00
Vitaly Baranov
cdcc735d83
Merge pull request #25899 from vitlibar/improve-logging-in-integration-tests
Improve logging in integration tests.
2021-07-06 08:22:21 +03:00
Vitaly Baranov
87f59ba670 Fix parallel execution of integration tests. 2021-07-05 23:50:04 +03:00
Alexander Tokmakov
0f377fc4d8 Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-05 22:07:56 +03:00
Kseniia Sumarokova
8b075e34e7
Merge pull request #25984 from kssenii/add-drops-in-tests
Correct test
2021-07-05 18:35:53 +03:00
Kseniia Sumarokova
c72a1be0be
Merge pull request #25064 from ucasfl/merge
Database argument for StorageMerge support regular expression.
2021-07-05 14:14:44 +03:00
Vitaly Baranov
72d751dc32 Add process id to pytest logs. 2021-07-05 11:25:58 +03:00
kssenii
c713ebe9c1 Add drops in tests 2021-07-05 11:20:58 +03:00
feng lv
b293f701b7 fix test 2021-07-05 03:34:19 +00:00
kssenii
32b7d7b750 More correct startup on create query 2021-07-04 19:53:05 +00:00
mergify[bot]
27d6a69f6a
Merge branch 'master' into truncate 2021-07-04 19:37:13 +00:00
Vitaly Baranov
618a77fafa Improve logging in integration tests. 2021-07-04 22:24:54 +03:00
Kseniia Sumarokova
8351f1db99
Merge pull request #20470 from kssenii/pg-ch-replica
MaterializePostgreSQL table engine and database engine
2021-07-04 19:34:54 +03:00
kssenii
b39888f199 Do not allow inconsistency in publication 2021-07-04 08:54:41 +00:00
mergify[bot]
9da1c98998
Merge branch 'master' into fix_aggregation_ttl 2021-07-03 15:07:44 +00:00
mergify[bot]
a2183dff39
Merge branch 'master' into better_remove_empty_parts 2021-07-03 15:01:11 +00:00
kssenii
a0623ddb08 Correct test 2021-07-03 13:35:11 +00:00
kssenii
0ce4c565d1 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-07-03 11:48:15 +00:00
Ilya Yatsishin
523155a020
Merge pull request #25908 from arenadata/ADQM-320 2021-07-02 13:25:22 +03:00
Ilya Golshtein
63c71a7b4b kerberized HDFS test fix if run in parallel 2021-07-02 02:12:31 +03:00
alexey-milovidov
919cc10a3c
Merge pull request #25823 from azat/integration-tests-fixes
tests/integration: use iptables --wait
2021-07-02 00:58:33 +03:00
mergify[bot]
98fa9f7951
Merge branch 'master' into better_remove_empty_parts 2021-07-01 11:32:43 +00:00
kssenii
4ad0f45f0e Some more tests 2021-07-01 08:35:28 +00:00
kssenii
f01c8edbff Fix access to tables after restart 2021-07-01 07:33:58 +00:00
Kseniia Sumarokova
360f772762
Merge pull request #25795 from sand6255/MySQL-column-comments-support
MySQL Engine column comments support
2021-06-30 20:18:47 +03:00
Kseniia Sumarokova
ed687f45a1
Merge branch 'master' into truncate 2021-06-30 20:08:42 +03:00
Vitaly Baranov
27bc0caf53
Merge pull request #25861 from vitlibar/correct-messages-in-integration-tests
Correct messages in integration tests.
2021-06-30 14:45:02 +03:00
Vitaly Baranov
92ace627d2 Correct messages in integration tests. 2021-06-30 14:16:37 +03:00
alesapin
19aeb14dfd Small tuning in test 2021-06-30 12:39:09 +03:00
alesapin
b33391f4ec
Merge pull request #25816 from vitlibar/collect-stderr-in-integration-tests
Collect stderr.log and stdout.log in all integration tests by default.
2021-06-30 12:16:49 +03:00
mergify[bot]
1799804243
Merge branch 'master' into fix_aggregation_ttl 2021-06-30 09:15:46 +00:00
mergify[bot]
8182b6e890
Merge branch 'master' into better_remove_empty_parts 2021-06-30 08:42:26 +00:00
Kseniia Sumarokova
2f38b25ba6
Merge pull request #25834 from kssenii/fix-test
Fix odbc test
2021-06-30 10:29:42 +03:00
Storozhuk Kostiantyn
a279cf721d
Update test.py 2021-06-30 10:03:03 +03:00
kssenii
da70f85d4e More tests 2021-06-29 23:11:46 +00:00
kssenii
9d5847a1a8 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-29 21:30:31 +00:00
kssenii
a8067fb3dd Update test 2021-06-29 20:29:21 +00:00
alexey-milovidov
3b5468d4a4
Merge pull request #25810 from ClickHouse/fix_version_update
Better test_version_update_after_mutation
2021-06-29 23:11:57 +03:00
alesapin
a6834213a1 Add tests 2021-06-29 22:47:54 +03:00
Azat Khuzhin
283ec54cbe tests/integration: use iptables --wait
Since integration tests runs in parallel, this should fix:

  test_keeper_persistent_log_multinode/test.py::test_restart_multinode Another app is currently holding the xtables lock. Perhaps you want to use the -w option?

And similar errors.
2021-06-29 21:07:39 +03:00
Alexander Tokmakov
fd914f3d97 fix 'Not found column' error in MaterializeMySQL 2021-06-29 20:47:41 +03:00
Vitaly Baranov
3a3cbeb7aa Collect stderr.log and stdout.log in all integration tests by default. 2021-06-29 17:30:47 +03:00
Kostiantyn Storozhuk
a9fad56b7a Flaky test fixed 2021-06-29 18:07:18 +08:00
alesapin
cdc95fa98a
Merge pull request #25428 from ClickHouse/zookeeper_snapshots
Tool for conversion of ZooKeeper data into clickhouse-keeper snapshot
2021-06-29 12:58:45 +03:00
alesapin
44f1a96ad6 Better test_version_update_after_mutation 2021-06-29 12:49:40 +03:00
Kostiantyn Storozhuk
a9510f25f9 Style fix 2021-06-29 15:22:08 +08:00
alexey-milovidov
878a159538
Merge pull request #25773 from ClickHouse/system-query-log-map-type-merge
Merging #23934
2021-06-29 04:19:53 +03:00
Kseniia Sumarokova
d9c46003cb
Merge pull request #25781 from sand6255/MaterializeMySQL-Engine-column-comments-improvement
MaterializeMySQL: Improved column comments support
2021-06-28 21:57:24 +03:00
Kseniia Sumarokova
9d02af7d8b
Merge pull request #25676 from sand6255/MaterializeMySQL-Support-Enum-Data-Type
MaterializeMySQL: support ENUM data type
2021-06-28 19:15:00 +03:00
Kostiantyn Storozhuk
c2c78929cb Implemented MySQL column comments support 2021-06-28 22:44:38 +08:00
alesapin
f90cfae368
Merge pull request #25619 from vitlibar/new-tests-for-select-privilege
Add tests for checking access rights for SELECT.
2021-06-28 17:23:16 +03:00
mergify[bot]
04814114e7
Merge branch 'master' into zookeeper_snapshots 2021-06-28 14:17:32 +00:00
alexey-milovidov
05f4bb45a5
Merge pull request #25756 from kssenii/rabbit-test
Try fix flacky rabbitmq test
2021-06-28 14:27:07 +03:00
alesapin
b11254d191
Merge branch 'master' into fix_aggregation_ttl 2021-06-28 13:31:29 +03:00
alesapin
7e73762b48 Fix flaky test and wrong message 2021-06-28 11:28:45 +03:00
Maksim Kita
294b937a39
Merge branch 'system-query-log-map-type-merge' into system-querylog-map-continue 2021-06-28 11:27:59 +03:00
kssenii
aa7133c239 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-27 20:08:00 +00:00
kssenii
c115c25e9c Some change 2021-06-27 22:52:24 +03:00
kssenii
c3a2fc0584 Review fixes 2021-06-27 19:17:17 +00:00
alexey-milovidov
fb89505656
Merge pull request #25538 from kssenii/pg-dix
Fix postrges array conversion
2021-06-27 19:35:24 +03:00
alexey-milovidov
0a26687115
Merge pull request #23864 from azat/dist-split-batch-and-retry
Add ability to split distributed batch on failures (i.e. due to memory limits)
2021-06-27 19:28:26 +03:00
kssenii
db998c3f6c More robust 2021-06-27 16:28:10 +00:00
alesapin
c977c33d6d Fix bug in execution of TTL GROUP BY 2021-06-27 19:18:15 +03:00
kssenii
159de92197 Uncomment test 2021-06-26 22:05:20 +00:00
kssenii
3d90f0710c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-26 22:04:47 +00:00
alexey-milovidov
420839a4c0
Merge pull request #25697 from sundy-li/mysql-fix
[mysql] fix mysql select user() return empty
2021-06-26 03:31:46 +03:00
Kostiantyn Storozhuk
ca04b077ef Improved column comments support 2021-06-25 22:30:11 +08:00
Anton Popov
4a250dc14e
Merge pull request #25371 from CurtizJ/add-compatibility-setting
Add compatibility setting for name of tuple literals
2021-06-25 15:08:13 +03:00
Storozhuk Kostiantyn
4a3145f586 Materialize my sql support enum data type
* Implemented Enum for MaterializeMySQL
2021-06-25 16:42:25 +08:00
sundy-li
d423d07a87 [mysql] fix mysql select user() return empty 2021-06-25 10:20:22 +08:00
Anton Popov
a15ee72d0f Merge remote-tracking branch 'upstream/master' into HEAD 2021-06-23 13:53:01 +03:00
Vitaly Baranov
c6d877d10b Add tests for checking access rights for SELECT. 2021-06-23 12:54:50 +03:00
Azat Khuzhin
353a770a87 Extend test_insert_distributed_async_send for distributed_directory_monitor_split_batch_on_failure 2021-06-23 02:48:47 +03:00
Azat Khuzhin
a0209178cc Add ability to split distributed batch on failures
Add distributed_directory_monitor_split_batch_on_failure setting (OFF by
default), that will split the batch and send files one by one in case of
retriable errors.

v2: more error codes
2021-06-23 02:48:47 +03:00
Azat Khuzhin
e148ef739d Drop replicas from dirname for internal_replication=true
Under use_compact_format_in_distributed_parts_names=1 and
internal_replication=true the server encodes all replicas for the
directory name for async INSERT into Distributed, and the directory name
looks like:

    shard1_replica1,shard1_replica2,shard3_replica3

This is required for creating connections (to specific replicas only),
but in case of internal_replication=true, this can be avoided, since
this path will always includes all replicas.

This patch replaces all replicas with "_all_replicas" marker.

Note, that initial problem was that this path may overflow the NAME_MAX
if you will have more then 15 replicas, and the server will fail to
create the directory.

Also note, that changed directory name should not be a problem, since:
- empty directories will be removed since #16729
- and replicas encoded in the directory name is also supported anyway.
2021-06-23 02:47:38 +03:00
alesapin
788e61f80e Snapshots test 2021-06-22 14:43:26 +03:00
alesapin
6b815b4f17 Merge branch 'master' into zookeeper_snapshots 2021-06-22 13:50:27 +03:00
alesapin
bf0a4864ac Add support for set/get ACL commands 2021-06-22 13:49:35 +03:00
alesapin
730554589f fix superdigest 2021-06-22 11:14:54 +03:00
alesapin
66b7df1d03
Merge pull request #25525 from ClickHouse/fix_some_tests
Fix some bugs in integration tests
2021-06-22 10:52:07 +03:00
Ilya Yatsishin
d6bdde54d9
Merge pull request #25212 from qoega/integration-6
More integration tests improvements
2021-06-22 10:19:22 +03:00
Kseniia Sumarokova
155bb03239
Merge branch 'master' into pg-dix 2021-06-22 10:14:52 +03:00
alesapin
70f7959de2 Merge branch 'update_buffer_size_in_nuraft' into zookeeper_snapshots 2021-06-21 23:02:42 +03:00
alesapin
8bbe65b4f4 Remove timeouts 2021-06-21 23:02:00 +03:00
alesapin
a356a9ca3d Merge branch 'fix_some_tests' into zookeeper_snapshots 2021-06-21 23:00:07 +03:00
Kseniia Sumarokova
41c65a9965
Merge pull request #25528 from kssenii/something
Fix convertion of datetime with timezone for mysql, odbc, etc..
2021-06-21 22:55:22 +03:00
alesapin
c9c22996d4 One more fixed test 2021-06-21 22:52:44 +03:00
Kseniia Sumarokova
f47dd116c4
Update test.py 2021-06-21 19:07:17 +03:00
alesapin
426710f8e3 Followup fix 2021-06-21 18:53:47 +03:00
alesapin
731edc9a6d Fixes in ACLs 2021-06-21 18:45:45 +03:00
kssenii
ac0f86cdbf Truncate for s3 2021-06-21 15:44:36 +00:00
Yatsishin Ilya
e04957ce82 more 2021-06-21 18:42:40 +03:00
kssenii
4688f9e038 hdfs truncate table 2021-06-21 13:59:57 +00:00
alesapin
76cee4e3cf Debugging 2021-06-21 16:58:39 +03:00
kssenii
0f9fc33a4e Fix postgres arrays 2021-06-21 11:27:10 +00:00
alesapin
743600a359 Merge branch 'update_buffer_size_in_nuraft' into zookeeper_snapshots 2021-06-21 13:46:22 +03:00
alesapin
8cf5f8e748 Followup fix 2021-06-21 13:30:39 +03:00
Anton Popov
c2754c34f4 Merge remote-tracking branch 'upstream/master' into HEAD 2021-06-21 12:42:48 +03:00
kssenii
c0732ddc12 Fix datetime with timezone 2021-06-21 08:22:12 +00:00
alesapin
a63da94ae4 Revert "Merge pull request #25526 from ClickHouse/revert-25459-add_run_id_option"
This reverts commit 7c7e96f26c, reversing
changes made to a54cbef5b7.
2021-06-21 11:14:26 +03:00
alesapin
da0a3997f4 Merge branch 'master' into fix_some_tests 2021-06-21 11:14:22 +03:00
alesapin
feecf398fb
Revert "Add run-id option to integration tests" 2021-06-21 11:10:44 +03:00
alesapin
c7781e24d5 Fix some flaky tests 2021-06-21 11:02:27 +03:00
alesapin
65ce3929b8
Merge pull request #25459 from ClickHouse/add_run_id_option
Add run-id option to integration tests
2021-06-21 10:21:24 +03:00
Nikolay Degterinsky
8cb25c5757 Fixed lemmatize integration test 2021-06-20 11:28:35 +00:00
Mike Kot
4c391f8e99
SYSTEM RESTORE REPLICA replica [ON CLUSTER cluster] (#13652)
* initial commit: add setting and stub

* typo

* added test stub

* fix

* wip merging new integration test and code proto

* adding steps interpreters

* adding firstly proposed solution (moving parts etc)

* added checking zookeeper path existence

* fixing the include

* fixing and sorting includes

* fixing outdated struct

* fix the name

* added ast ptr as level of indirection

* fix ref

* updating the changes

* working on test stub

* fix iterator -> reference

* revert rocksdb submodule update

* fixed show privileges test

* updated the test stub

* replaced rand() with thread_local_rng(), updated the tests

updated the test

fixed test config path

test fix

removed error messages

fixed the test

updated the test

fixed string literal

fixed literal

typo: =

* fixed the empty replica error message

* updated the test and the code with logs

* updated the possible test cases, updated

* added the code/test milestone comments

* updated the test (added more testcases)

* replaced native assert with CH one

* individual replicas recursive delete fix

* updated the AS db.name AST

* two small logging fixes

* manually generated AST fixes

* Updated the test, added the possible algo change

* Some thoughts about optimizing the solution:

ALTER MOVE PARTITION .. TO TABLE -> move to detached/ + ALTER ... ATTACH

* fix

* Removed the replica sync in test as it's invalid

* Some test tweaks

* tmp

* Rewrote the algo by using the executeQuery instead of

hand-crafting the ASTPtr.

Two questions still active.

* tr: logging active parts

* Extracted the parts moving algo into a separate helper function

* Fixed the test data and the queries slightly

* Replaced query to system.parts to direct invocation,

started building the test that breaks on various parts.

* Added the case for tables when at least one replica is alive

* Updated the test to test replicas restoration by detaching/attaching

* Altered the test to check restoration without replica restart

* Added the tables swap in the start if the server failed last time

* Hotfix when only /replicas/replica... path was deleted

* Restore ZK paths while creating a replicated MergeTree table

* Updated the docs, fixed the algo for individual replicas restoration case

* Initial parts table storage fix, tests sync fix

* Reverted individual replica restoration to general algo

* Slightly optimised getDataParts

* Trying another solution with parts detaching

* Rewrote algo without any steps, added ON CLUSTER support

* Attaching parts from other replica on restoration

* Getting part checksums from ZK

* Removed ON CLUSTER, finished working solution

* Multiple small changes after review

* Fixing parallel test

* Supporting rewritten form on cluster

* Test fix

* Moar logging

* Using source replica as checksum provider

* improve test, remove some code from parser

* Trying solution with move to detached + forget

* Moving all parts (not only Committed) to detached

* Edited docs for RESTORE REPLICA

* Re-merging

* minor fixes

Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru>
2021-06-20 11:24:43 +03:00
Nikolay Degterinsky
b1c15d9014 Fixed integration test 2021-06-19 20:27:26 +00:00
alesapin
49cd16e74f Better 2021-06-18 22:08:12 +03:00
alesapin
b5dae909dd Add some tests 2021-06-18 21:36:19 +03:00
alesapin
5a014cb1e1 Add sometest 2021-06-18 14:02:15 +03:00