kssenii
dd6a2c2d37
Fix style check
2023-01-06 16:53:24 +01:00
kssenii
2a030c1dc0
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2023-01-06 15:58:27 +01:00
Vitaly Baranov
92be50bd68
Add test for IMDS session token.
2023-01-06 15:55:26 +01:00
Vitaly Baranov
0d12d90fb5
Move common code to helpers/mock_servers.py
2023-01-06 15:55:21 +01:00
kssenii
c980e16345
Merge remote-tracking branch 'upstream/master' into storage-rabbitmq-fix-flush
2023-01-06 15:51:25 +01:00
Anton Popov
b25f875674
Merge pull request #44875 from ClickHouse/fix-another-one-cannot-read-all-data-for-lc-dict-error
...
Fix right offset for reading LowCardinality dictionary from remote fs
2023-01-06 15:24:36 +01:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2
...
Replace old named collections code with new (from #43147 ) part 2
2023-01-06 13:06:26 +01:00
Nikolai Kochetov
12192ef099
Merge pull request #44938 from ClickHouse/try-fix-test_multiple_disks-test_freeze
...
Stop merges to avoid a race between merge and freeze.
2023-01-06 12:45:10 +01:00
Azat Khuzhin
7859346e40
tests: fix test_replicated_users flakiness
...
Alter for users/quotas/... does not wait until all replicas will be up
to date, so use *_with_retries() helperx.
CI: https://s3.amazonaws.com/clickhouse-test-reports/44922/bd885be9229cf47752c5c98392f09129261550f9/integration_tests__tsan__[6/6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-06 12:45:03 +01:00
Azat Khuzhin
c03510cf6b
tests/integration: add query_and_get_error_with_retry() helper
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-06 12:45:03 +01:00
Alexey Milovidov
08ebdc9b82
Merge pull request #44903 from ClickHouse/less-flaky-test-concurrent-queries
...
Fix flaky `test_concurrent_queries_restriction_by_query_kind`
2023-01-06 03:18:28 +03:00
robot-clickhouse
62a9b33bbb
Automatic style fix
2023-01-05 16:29:58 +00:00
Ilya Yatsishin
a28d6fb490
Merge pull request #44848 from qoega/green-ci-2
...
Pre-pulling images for integration tests to detect broken infrastructure earlier
2023-01-05 17:29:39 +01:00
Nikolai Kochetov
90f3f5789f
Speedup a bit.
2023-01-05 16:23:49 +00:00
Nikolai Kochetov
dc5009d401
Stop merges to aviod a race between merge and freeze.
2023-01-05 15:28:56 +00:00
Anton Popov
ed00399eb6
Merge remote-tracking branch 'upstream/master' into HEAD
2023-01-05 14:12:02 +00:00
Yatsishin Ilya
1b03a60e43
Do not report status if we failed to pull images.
2023-01-05 13:53:45 +00:00
Ilya Yatsishin
c5f01a653c
Merge pull request #44880 from qoega/green-ci-4
...
Rewrite test_postgres_protocol test
2023-01-05 12:34:33 +01:00
Antonio Andelic
7dd8ac717a
Merge branch 'master' into less-flaky-test-concurrent-queries
2023-01-05 12:23:30 +01:00
Ilya Yatsishin
5d0f3e7819
Merge pull request #44921 from ClickHouse/fix-flaky-test-5
...
Fix flaky test `test_lost_part`
2023-01-05 10:28:07 +01:00
xiedeyantu
dfaa16b7a0
fix
2023-01-05 09:09:42 +08:00
xiedeyantu
110f95bae6
fix
2023-01-05 09:00:53 +08:00
Nikita Mikhaylov
e3ca9fb3e9
Blakc
2023-01-04 20:30:52 +00:00
Nikita Mikhaylov
ea7c338a7b
Done
2023-01-04 20:28:03 +00:00
Yatsishin Ilya
ccb6226d22
install CH before test
2023-01-04 20:07:52 +00:00
Yakov Olkhovskiy
dadd96ba6f
Merge pull request #44135 from ClickHouse/feature-protocol-proxy
...
Improve support for PROXYv1 protocol.
2023-01-04 14:06:17 -05:00
Yatsishin Ilya
daf2ccab81
Merge remote-tracking branch 'origin/master' into green-ci-2
2023-01-04 18:27:19 +00:00
xiedeyantu
2f59b1d002
better
2023-01-05 00:27:07 +08:00
Yakov Olkhovskiy
7a5a36cbed
Merge branch 'master' into refactoring-ip-types
2023-01-04 11:11:06 -05:00
Antonio Andelic
8ee589c5c1
Merge pull request #44898 from ClickHouse/try-fix-test-storage-kafka
...
Try to fix flaky `test_storage_kafka::test_kafka_produce_key_timestamp`
2023-01-04 16:54:47 +01:00
Yatsishin Ilya
03582f4b87
move logic to runner
2023-01-04 15:52:34 +00:00
Yatsishin Ilya
9cf599e58b
Merge remote-tracking branch 'origin/master' into green-ci-2
2023-01-04 15:49:22 +00:00
Yatsishin Ilya
d8d012523d
add pre-pull to runner
2023-01-04 15:47:50 +00:00
alesapin
92255e5d4b
Merge pull request #43020 from vitlibar/update-aws-c-submodules
...
Update aws-c* submodules
2023-01-04 16:25:22 +01:00
Antonio Andelic
1a1afc6a74
Make test_concurrent_queries_restriction_by_query_kind less flaky
2023-01-04 14:18:35 +00:00
Nikita Mikhaylov
e9c7555365
Use TablesDependencyGraph in DatabaseReplicated recovery process ( #44697 )
2023-01-04 14:49:39 +01:00
Yatsishin Ilya
4d5131ed7d
debug quotes
2023-01-04 13:16:31 +00:00
Antonio Andelic
890f094829
Set retention policy for Kafka topic
2023-01-04 12:04:00 +00:00
Yatsishin Ilya
08fbf6e5b2
again
2023-01-04 11:48:28 +00:00
alesapin
c09f83654d
Merge pull request #44533 from ClickHouse/40951_Allow_macro_substitution_in_endpoint
...
Allow macro substitution in endpoint of disks
2023-01-04 12:29:57 +01:00
Azat Khuzhin
0ab0f76737
Convert integration test_dictionaries_update_field to a stateless
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-04 11:05:00 +01:00
Antonio Andelic
9b58c97563
Store ZK generated data in archive
2023-01-04 09:10:16 +00:00
Alexey Milovidov
66e6f53698
Merge pull request #44839 from ClickHouse/move-odbc-test
...
Move `test_dies_with_parent` to another module
2023-01-04 05:17:19 +03:00
Alexey Milovidov
3a027d285f
Merge pull request #44828 from ClickHouse/remove-two-lines-of-code
...
What if I will remove two lines of code?
2023-01-04 04:50:52 +03:00
Ilya Yatsishin
49155d9614
Merge pull request #44711 from qoega/green-ci-1
...
Improve some integration tests and add pytest-random
2023-01-03 22:39:54 +01:00
Yatsishin Ilya
bb288c1f14
how many more?
2023-01-03 21:38:59 +00:00
Yatsishin Ilya
bbc02c0762
black
2023-01-03 21:28:54 +00:00
Yatsishin Ilya
14274d650b
Rewrite test_postgres_protocol test
2023-01-03 19:30:14 +00:00
Yatsishin Ilya
683df2dea5
fix
2023-01-03 18:56:24 +00:00
Robert Schulze
27f5aad49e
What happens if I remove 156 lines of code?
2023-01-03 18:51:16 +00:00
robot-clickhouse
d6c090b530
Automatic style fix
2023-01-03 18:29:09 +00:00
Nikolai Kochetov
da26f62a9b
Fix right offset for reading LowCardinality dictionary from remote fs in case if right mark was in the middle of compressed block.
2023-01-03 18:19:51 +00:00
Alexey Milovidov
a06d03e097
Merge pull request #44851 from qoega/green-ci-3
...
No more retries in integration tests
2023-01-03 19:40:34 +03:00
Yatsishin Ilya
3deada0c29
fix
2023-01-03 16:05:20 +00:00
kssenii
67509aa2d5
Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2
2023-01-03 16:41:30 +01:00
Antonio Andelic
a88171b594
Add timeout for ZK stop
2023-01-03 09:59:06 +00:00
Antonio Andelic
4ccd828a44
Wait for ZK process to stop
2023-01-03 09:39:14 +00:00
Nikita Mikhaylov
b34fc8650c
Merge branch 'master' into move-odbc-test
2023-01-02 21:22:50 +01:00
SmitaRKulkarni
16dd6ca02f
Merge branch 'master' into 40951_Allow_macro_substitution_in_endpoint
2023-01-02 20:34:24 +01:00
Alexey Milovidov
e855d3519a
Merge branch 'master' into refactoring-ip-types
2023-01-02 21:58:53 +03:00
Ilya Yatsishin
8a9bb9b107
No more retries in integration tests
2023-01-02 18:56:24 +01:00
Yatsishin Ilya
3251d9278c
black
2023-01-02 17:44:04 +00:00
Yatsishin Ilya
9ab5cf7d62
Download inside runner
2023-01-02 16:46:24 +00:00
Antonio Andelic
509fd873ec
Try restarting zookeeper if connection failed ( #44834 )
2023-01-02 16:24:57 +01:00
Yatsishin Ilya
b40f6c0b10
Pre-pull docker image and fail check completely if it failed
2023-01-02 15:04:14 +00:00
Kruglov Pavel
966f57ef68
Merge pull request #42777 from Avogar/improve-streaming-engines
...
Refactor and Improve streaming engines Kafka/RabbitMQ/NATS and data formats
2023-01-02 15:59:06 +01:00
Alexey Milovidov
17e792ca0b
Merge pull request #44825 from azat/ci/integration-fixes
...
Fix misleading integration tests reports for parametrized tests
2023-01-02 17:49:57 +03:00
robot-clickhouse
675fe5cd91
Automatic style fix
2023-01-02 13:38:18 +00:00
Nikita Mikhaylov
ec5e8ee9cf
Done
2023-01-02 13:31:19 +00:00
Smita Kulkarni
5592d30798
Updated to skip access check for hdfs , as done for other tests - Allow macro substitution in endpoint
2023-01-02 14:03:51 +01:00
Yatsishin Ilya
54e2df724f
fix test
2023-01-02 12:43:16 +01:00
Yatsishin Ilya
6ec3bf40b1
black
2023-01-02 12:43:16 +01:00
Yatsishin Ilya
f779fe3c0a
Improve some integration tests
2023-01-02 12:43:16 +01:00
Azat Khuzhin
c3b3ae886f
tests/integration: ignore exit code for a running group
...
tee is useful there, otherwise logs are empty on CI
This reverts commit 92fd7def39
.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-02 12:08:44 +01:00
Azat Khuzhin
f2c069b852
tests/integration: allow to split parametrized tests between parallel groups
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-02 12:06:56 +01:00
Smita Kulkarni
2f73bf48b8
Added config to integration test test_endpoint_macro_substitution - Allow macro substitution in endpoint
2023-01-02 11:43:19 +01:00
Alexey Milovidov
b89513e789
Merge pull request #44823 from azat/tests/fix-flaky-test_multiple_disks
...
Fix flaky test_multiple_disks::test_jbod_overflow
2023-01-02 05:25:27 +03:00
Azat Khuzhin
4b71b481c7
tests/integration: do not split paramaterized tests into multiple groups
...
There is also one assumption the runner already, that leads to
misleading reports like in [1].
[1]: https://s3.amazonaws.com/clickhouse-test-reports/44762/89c071e291980e355f2c2be42dc15047caabc9bc/integration_tests__asan__%5B4/6%5D.html
Here the test test_multiple_disks/test.py::test_jbod_overflow had been
splitted into multiple groups, however because of that one assumption
it had been runned as a whole in both groups and in one it failed but
succeeded in another.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:35:21 +01:00
Azat Khuzhin
700943a2b0
tests/integration: remove logging message for undone thing
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:13:00 +01:00
Azat Khuzhin
fa78569c39
tests/integration: fix logging message
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:12:25 +01:00
Azat Khuzhin
92fd7def39
tests/integration: fix detecting exit code for a running group
...
Because of tee the exit code is the result of tee, not of the pytest.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 23:09:12 +01:00
Azat Khuzhin
3123761980
Fix flaky test_multiple_disks::test_jbod_overflow
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-01 22:21:24 +01:00
Alexey Milovidov
f5fc9b2825
Fix tests for bridges
2023-01-01 22:00:08 +01:00
Vitaly Baranov
426e2a9da1
Add test for getting credentials from AWS EC2 metadata.
2022-12-31 18:56:40 +01:00
alesapin
4948a8c17b
Remove disk restart proxy and disk decorator ( #44647 )
...
* Remove disk restart proxy and disk decorator
* Automatic style fix
* Returned some trash back
* Fix build again
* Fix failing test
Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
2022-12-30 14:47:30 +01:00
SmitaRKulkarni
f9a3aeb172
Merge branch 'master' into 40951_Allow_macro_substitution_in_endpoint
2022-12-30 10:43:14 +01:00
Kruglov Pavel
894726bd8f
Merge branch 'master' into improve-streaming-engines
2022-12-29 22:59:45 +01:00
robot-clickhouse
c09d1bc758
Automatic style fix
2022-12-29 15:16:50 +00:00
Smita Kulkarni
8cc3b2e26c
Fixed integration flaky test issue by specifying hdfs client in each test- Allow macro substitution in endpoint
2022-12-29 16:10:06 +01:00
Nikita Mikhaylov
ffaff7167b
Fix flaky tests ( #44690 )
2022-12-29 14:32:10 +01:00
Alexey Milovidov
9378194320
Merge pull request #44688 from ClickHouse/improve-odbc-test
...
Improve odbc test
2022-12-29 15:40:47 +03:00
Azat Khuzhin
90b707d130
tests: more fixes for test_keeper_auth
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-29 09:29:24 +01:00
Alexey Milovidov
f7b2b961f7
Merge pull request #44667 from azat/tests/integration/dmesg-fix
...
Move dmesg dumping out from runner to ci-runner.py
2022-12-29 01:56:34 +03:00
Alexey Milovidov
23ea273f73
Edit another test
2022-12-28 19:49:57 +01:00
Alexey Milovidov
edd78da0cb
ODBC test: avoid poisoning other tests
2022-12-28 19:45:01 +01:00
Alexey Milovidov
0263499b0c
Merge pull request #44666 from azat/tests/integration/kazoo-stop
...
tests/integration: add missing kazoo client termination
2022-12-28 21:20:14 +03:00
Anton Popov
bc122d368a
Merge remote-tracking branch 'upstream/master' into HEAD
2022-12-28 17:45:04 +00:00
Azat Khuzhin
03f80bae90
Move dmesg dumping out from runner to ci-runner.py
...
runner is used by developers to run tests, while ci-runner.py is used
only by CI scripts, and to avoid requiring CAP_SYSLOG for manual dmesg
clear/dump had been moved.
Also for manual runs this can be done manually.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-28 15:47:46 +01:00
Vitaly Baranov
8a2fbbe88c
Merge pull request #43947 from azat/backups/dedup
...
RFC: Add ability to disable deduplication for BACKUP
2022-12-28 15:24:53 +01:00
Alexey Milovidov
1b4121459d
Merge pull request #44645 from ClickHouse/keeper-test-fix
...
Fix flaky test `test_keeper_multinode_simple`
2022-12-28 16:38:17 +03:00
Azat Khuzhin
bc6adc88c8
tests/integration: add workaround for a bug in kazoo driver
...
Refs: python-zk/kazoo#688
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-28 10:52:49 +01:00
Vitaly Baranov
b211dff013
Increase the create table timeout in test "test_backup_restore_on_cluster".
2022-12-28 00:17:19 +01:00
Nikita Mikhaylov
00ecf6dd05
Better
2022-12-27 17:04:47 +00:00
Nikita Mikhaylov
ce6264b684
Done
2022-12-27 17:03:57 +00:00
Azat Khuzhin
e1ad5cee7e
tests/integration: add missing kazoo client termination
...
pytest play games with logging output [1].
[1]: https://github.com/pytest-dev/pytest/issues/5502
But this does not actually affect ClickHouse tests, the only reason is
that the kazoo client is not stopped correctly without calling
kazoo.client.KazooClient.stop(), and that's why you can see the
following messages:
<details>
test_storage_rabbitmq/test.py::test_rabbitmq_big_message --- Logging error ---
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 179, in _socket_error_handling
yield
File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 659, in _connect
self._socket = self.handler.create_connection(
File "/usr/local/lib/python3.8/dist-packages/kazoo/handlers/threading.py", line 178, in create_connection
return utils.create_tcp_connection(socket, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/kazoo/handlers/utils.py", line 265, in create_tcp_connection
sock = module.create_connection(address, timeout_at)
File "/usr/lib/python3.8/socket.py", line 808, in create_connection
raise err
File "/usr/lib/python3.8/socket.py", line 796, in create_connection
sock.connect(sa)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 579, in _connect_attempt
read_timeout, connect_timeout = self._connect(host, hostip, port)
File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 659, in _connect
self._socket = self.handler.create_connection(
File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 182, in _socket_error_handling
raise ConnectionDropped("socket connection error: %s" % (err,))
kazoo.exceptions.ConnectionDropped: socket connection error: None
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/logging/__init__.py", line 1088, in emit
stream.write(msg + self.terminator)
ValueError: I/O operation on closed file.
Call stack:
File "/usr/lib/python3.8/threading.py", line 890, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 512, in zk_loop
if retry(self._connect_loop, retry) is STOP_CONNECTING:
File "/usr/local/lib/python3.8/dist-packages/kazoo/retry.py", line 126, in __call__
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 552, in _connect_loop
status = self._connect_attempt(host, hostip, port, retry)
File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 622, in _connect_attempt
self.logger.warning('Connection dropped: %s', e)
Message: 'Connection dropped: %s'
Arguments: (ConnectionDropped('socket connection error: None'),)
</details>
Which eventually leads to incorrectly parsed report.
This patch is an addition to the `logging.raiseExceptions=False` - #44618
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-27 17:01:06 +01:00
Kruglov Pavel
c3405be272
Increase the number of attempts in tests
2022-12-27 16:49:32 +01:00
Azat Khuzhin
c9c590071d
Add ability to disable deduplication for BACKUP
...
Right now BACKUP omit similar files, and will not allow to use this
backup as a regular table, and usually those similar files are quite
small (i.e. columns.txt).
So by using `BACKUP TO S3() deduplicate_files=0` you will be possible to
use `ATTACH TABLE` directly from S3.
P.S. right now it is possible only for the table with one part, since,
usually, there is nothing to deduplicate (if the columns are different).
v2: Add deduplicate_files into metadata
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-27 15:56:06 +01:00
Alexey Milovidov
7e447592b1
Merge pull request #44618 from azat/tests/integration/pytest-logging-fix
...
tests/integration: suppress exceptions during logging (due to pytest)
2022-12-27 14:33:37 +03:00
Alexey Milovidov
79d43c836a
Merge pull request #44572 from ClickHouse/fix-hdfs-test
...
Fix HDFS test
2022-12-27 14:31:00 +03:00
Alexey Milovidov
41fc65b47b
Merge pull request #44573 from ClickHouse/fix-distributed-queries-stress
...
Fix test_distributed_queries_stress
2022-12-27 14:30:28 +03:00
Azat Khuzhin
8b2ed0e3cd
tests/integration: disable logging.raiseExceptions
...
Due to pytest play games with logging [1] it is better to ignore it.
[1]: https://github.com/pytest-dev/pytest/issues/5502
This is due to incorrect kazoo client usage (not all clients calls
stop())
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-26 22:57:54 +01:00
Alexey Milovidov
a92a9b4d55
Merge pull request #44535 from azat/tests/integration-dmesg
...
tests: capture dmesg in integration tests
2022-12-26 23:49:02 +03:00
Roman Vasin
465cc36526
Update Exception messages in test.py
2022-12-26 13:17:22 +00:00
Roman Vasin
483e26cf2f
Merge branch 'master' into ADQM-634
2022-12-26 15:34:38 +03:00
Roman Vasin
3cdc9b3f81
Remove -vvv from kinit call in tests
2022-12-26 11:59:37 +00:00
Roman Vasin
a70e3c20fa
Make calls of test from Ubuntu client
2022-12-26 11:56:45 +00:00
Alexey Milovidov
a06c8268ea
Remove the offender
2022-12-26 11:28:26 +01:00
Alexey Milovidov
f9f0f9446d
Fix distributed_queries_stress
2022-12-25 09:46:51 +01:00
robot-clickhouse
19f05d4fb4
Automatic style fix
2022-12-25 08:38:02 +00:00
Alexey Milovidov
cbf140633b
Fix HDFS test
2022-12-25 09:30:20 +01:00
Anton Popov
0722020cae
Merge remote-tracking branch 'upstream/master' into HEAD
2022-12-23 18:15:55 +00:00
Smita Kulkarni
84362461fb
Updated test to use macro substitution for hdfs disk - Allow macro substitution in endpoint
2022-12-23 16:07:56 +01:00
Azat Khuzhin
69fadd2193
tests: capture dmesg in integration tests
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-23 15:50:10 +01:00
Smita Kulkarni
3d62bc2635
Allow macro substitution in endpoint of disks
...
Implementation:
* Updated to substitute macros in endpoint for all disks (s3, hfs, web).
Testing:
* Added an integration test test/integration/test_endpoint_macro_substitution.
2022-12-23 15:09:24 +01:00
Mikhail f. Shiryaev
6d0cd35cd0
Ignore exit code 1 for tar in integration tests
2022-12-23 13:08:59 +01:00
kssenii
8308177361
Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2
2022-12-23 12:14:46 +01:00
Vladimir C
7482ea54ab
Merge pull request #43972 from ClickHouse/vdimir/tmp-data-in-fs-cache-2
2022-12-23 11:59:27 +01:00
kssenii
853f2ea123
Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2
2022-12-23 11:49:02 +01:00
kssenii
8f1145f6e4
Fix tests
2022-12-23 11:47:51 +01:00
kssenii
11c0aa189a
Better error message
2022-12-22 15:29:35 +01:00
Nikolai Kochetov
773de7a8a6
Merge pull request #44472 from CurtizJ/fix-mutations
...
Fix mutations with setting `max_streams_for_merge_tree_reading`
2022-12-22 14:05:20 +01:00
kssenii
a58b8b8e6c
Fix tests
2022-12-22 12:27:42 +01:00
kssenii
ab458e68cd
Fix tests
2022-12-22 12:24:20 +01:00
vdimir
182b34c11e
Fixes
2022-12-22 10:22:57 +00:00
vdimir
511e8d698d
Rename config param tmp_cache -> temporary_data_in_cache
2022-12-22 10:22:56 +00:00
vdimir
d30d25dbbe
Temporary files evict fs cache
2022-12-22 10:22:49 +00:00
Roman Vasin
2d9f3c242b
Add test_bad_path_to_keytab test
2022-12-22 08:07:16 +00:00
Yakov Olkhovskiy
cea5b92536
binary backward compatibility test is added
2022-12-22 07:38:49 +00:00
kssenii
d3db1dd6a7
Fix tests
2022-12-22 00:27:22 +01:00
Anton Popov
5d6064cd28
better config in test
2022-12-21 11:35:11 +00:00
Kruglov Pavel
5e01a3d74e
Merge branch 'master' into improve-streaming-engines
2022-12-21 10:51:50 +01:00
Anton Popov
4b7f1e0cf2
fix mutations with setting 'max_streams_for_merge_tree_reading'
2022-12-21 01:59:53 +00:00
kssenii
1d75f740d7
Fix tests
2022-12-20 22:33:54 +01:00
kssenii
499565a604
Fix tests
2022-12-20 20:02:18 +01:00
Yakov Olkhovskiy
8462b724ca
Merge branch 'master' into feature-protocol-proxy
2022-12-20 13:12:07 -05:00
robot-clickhouse
82203e629c
Automatic style fix
2022-12-20 17:46:17 +00:00
Yakov Olkhovskiy
9092134544
auth_use_forwarded_address config setting added, using forwarded address for proxied TCP and HTTP connection, likely bug is fixed in AccessControl, relevant test is appended
2022-12-20 17:39:40 +00:00
alesapin
80cae1aa3d
Better
2022-12-20 17:37:03 +01:00
alesapin
33389f01e6
Disable tsan assertion for test
2022-12-20 17:33:08 +01:00
Konstantin Bogdanov
93375fc3e5
Merge pull request #44411 from thevar1able/fix-settings-constraints
...
Fix settings constraints
2022-12-20 10:19:07 +01:00
Konstantin Bogdanov
a094af9a49
Lint
2022-12-20 04:00:45 +01:00
Konstantin Bogdanov
b07001e7d7
Add a test
2022-12-20 03:26:31 +01:00
kssenii
a0c8d34775
RabbitMQ: respect flushing data by flush interval and by block size
2022-12-19 21:07:44 +01:00
Roman Vasin
8a692432de
Remove unneeded headers from test.py
2022-12-19 11:48:10 +00:00
Roman Vasin
c26c257c57
Refactor test.py
2022-12-19 11:39:30 +00:00
Roman Vasin
64c0c6ae99
Refactor test_kerberos_auth test
2022-12-19 11:28:49 +00:00
Antonio Andelic
55dd591392
Try restarting ZK cluster on failed connection
2022-12-19 09:28:59 +00:00
Antonio Andelic
0162632275
Merge pull request #44306 from ClickHouse/another-fix-for-test-server-reload
...
Another fix `test_server_reload`
2022-12-19 09:14:03 +01:00
Nikolay Degterinsky
7a29bf0e4c
Merge pull request #43993 from evillique/fix_settings_constraints
...
Fix CREATE query constraints
2022-12-16 18:01:04 +01:00
Anton Popov
8b9b8b083c
Merge pull request #43726 from CurtizJ/optimize-storage-s3
...
Improve performance of storage `S3` with large number of small files
2022-12-16 14:38:10 +01:00
Antonio Andelic
372c187773
Correctly compare get result
2022-12-16 09:48:47 +00:00
Roman Vasin
775c700c24
Add second test for the case when there is no keytab
2022-12-16 08:20:01 +00:00
Nikolay Degterinsky
1861e670e9
Merge branch 'master' into fix_settings_constraints
2022-12-15 21:54:02 +01:00
Kruglov Pavel
c5b2e4cc23
Merge branch 'master' into improve-streaming-engines
2022-12-15 18:44:35 +01:00
Roman Vasin
5491aa9627
Implement wait_kerberos_kdc_is_available
2022-12-15 11:50:23 +00:00
Anton Popov
c4bcfe54cd
fix test
2022-12-15 04:19:16 +00:00
Vitaly Baranov
fb8aca8319
Merge pull request #44158 from vitlibar/improve-referential-deps
...
Improve referential dependencies
2022-12-14 21:17:02 +01:00
Roman Vasin
e2ced517dd
Add integration test test_kerberos_auth
2022-12-14 13:39:23 +00:00
kssenii
ea5b06023a
Merge remote-tracking branch 'refs/remotes/upstream/master' into allow-single-disk-instead-of-storage-policy
2022-12-14 13:59:51 +01:00
Sema Checherinda
fdecb85dc4
Merge pull request #44156 from CheSema/fix_flaky_test_attach_detach
...
make calls to be sure that parts are deleted
2022-12-14 12:39:39 +01:00
Antonio Andelic
b2cce09004
Merge pull request #44184 from ClickHouse/close-grpc-channel-in-tests
...
Close GRPC channels in tests
2022-12-14 09:47:36 +01:00
Anton Popov
3008e701ea
fix tests
2022-12-14 01:36:03 +00:00
Anton Popov
cce3257f39
Merge branch 'master' into optimize-storage-s3
2022-12-13 21:35:12 +01:00
Nikolay Degterinsky
809f15ac81
Merge branch 'master' into fix_settings_constraints
2022-12-13 20:33:16 +01:00
Nikolay Degterinsky
443bc3e656
Merge pull request #43719 from evillique/password_complexity_rules
...
Add password complexity rules
2022-12-13 17:56:45 +01:00
Anton Popov
0c87031e80
Merge remote-tracking branch 'upstream/master' into HEAD
2022-12-13 16:33:21 +00:00
Mikhail f. Shiryaev
9879d03bc4
Use contextmanager for grpc_channels
2022-12-13 12:43:48 +01:00
Antonio Andelic
b8f76ce0e5
Close GRPC channels in tests
2022-12-13 10:25:54 +00:00
Vitaly Baranov
5c1f490b3a
Implement referential dependencies for table engine "Distributed"
...
and for functions cluster() and clusterAllReplicas().
2022-12-13 11:09:42 +01:00
Nikolay Degterinsky
f352403af2
Fix style
2022-12-13 09:41:31 +00:00
Nikolay Degterinsky
034269087f
Fix tests
2022-12-13 09:16:23 +00:00
Anton Popov
0b396bab14
Merge remote-tracking branch 'upstream/master' into HEAD
2022-12-12 18:08:27 +00:00
Vitaly Baranov
5aaff60650
Fix referential dependencies when host & post in a clickHouse dictionary source are set by default.
2022-12-12 18:22:14 +01:00
Vitaly Baranov
76ba8ab3d4
Add new tests.
2022-12-12 18:22:09 +01:00
Anton Popov
a2fee110b8
add query for waiting of loading parts and fix test_transactions
2022-12-12 14:33:42 +00:00
Sema Checherinda
aecc509f80
make calls to be sure that parts are deleted
2022-12-12 12:37:52 +01:00
Nikolay Degterinsky
19e95e8cd8
Better implementation of password rules
2022-12-11 23:59:04 +00:00
Sema Checherinda
9a1152339c
Merge branch 'master' into do-not-run-clear-old-parts-at-shutdown
2022-12-10 09:44:56 +01:00
Nikolay Degterinsky
a8545a29a4
Better test
2022-12-10 01:32:15 +00:00
Sergei Trifonov
7c37e48092
Merge pull request #43903 from ClickHouse/merge_tree_settings_constraints
...
Constraints on merge tree settings
2022-12-09 14:16:14 +01:00
Kseniia Sumarokova
e21d4a93b8
Merge pull request #44050 from ClickHouse/fix_clickhouse_disk
...
Slightly better clickhouse disks and remove DiskMemory
2022-12-09 13:09:59 +01:00
Alexey Milovidov
68e506d48f
Unify __init__.py files
2022-12-08 23:04:56 +01:00
Alexey Milovidov
f880644d3a
Merge branch 'master' into nv/fix-mutate-fetch
2022-12-09 00:51:54 +03:00
Sergei Trifonov
bae1519970
Merge branch 'master' into merge_tree_settings_constraints
2022-12-08 22:25:16 +01:00
robot-clickhouse
98ce0dbb12
Automatic style fix
2022-12-08 17:27:23 +00:00
alesapin
7a223f4956
Slightly better clickhouse disks and remove DiskMemory
2022-12-08 18:20:54 +01:00
serxa
6428b8a1bb
check one more case in test
2022-12-07 20:36:04 +00:00
Alexey Milovidov
9c81fd7afc
Merge pull request #43517 from ClickHouse/password-reset-message
...
Move password reset message from client to server
2022-12-07 23:26:58 +03:00
serxa
05788245dd
add test
2022-12-07 20:16:10 +00:00
Nikolay Degterinsky
8a53f5364b
Fix style
2022-12-07 10:36:16 +01:00
Nikolay Degterinsky
bb9fdc0839
Fix CREATE query constraints
2022-12-07 00:53:27 +00:00
Sema Checherinda
0b2a3195c8
adjust integration/test_partition/test_detached_part_dir_exists
2022-12-05 17:39:27 +01:00
Sema Checherinda
1f96ef57d2
remove integration/test_parts_removal
2022-12-05 17:24:54 +01:00
kssenii
d278a0dbfb
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2022-12-05 12:59:44 +01:00
kssenii
dca3444283
Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands
2022-12-04 15:56:02 +01:00
Vitaly Baranov
e1f7f04752
Referential dependencies for RESTORE ( #43834 )
...
* Rename DDLDependencyVisitor -> DDLLoadingDependencyVisitor.
* Move building a loading graph to TablesLoader.
* Implement referential dependencies for tables and use them
when restoring tables from a backup.
* Remove StorageID::operator < (because of its inconsistency with ==).
* Add new tests.
* Fix test.
* Fix memory leak.
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2022-12-02 15:05:46 +01:00
Vladimir C
7d6950d397
Revert "Temporary files evict fs cache"
2022-12-02 14:50:56 +01:00
Alexander Tokmakov
edbd460997
Merge pull request #43848 from elenatorro/fix_reset_setting_on_cluster
...
Add RESET_SETTING to check if it is a settings alter
2022-12-02 15:06:28 +03:00
Alexander Tokmakov
c2a19fc354
Update test.py
2022-12-01 19:41:29 +03:00
Nikita Mikhaylov
9ef54c6ecc
Try to use exchanges during database replicated replica recovery process ( #43628 )
2022-12-01 13:50:56 +01:00
vdimir
08b8d335f0
add integration/test_temporary_data_in_cache
2022-12-01 11:49:25 +00:00
vdimir
7a814d2854
wip: temporary files evict fs cache
2022-12-01 11:49:24 +00:00
Elena Torró
5630412944
Merge branch 'master' into fix_reset_setting_on_cluster
2022-11-30 21:47:43 +01:00
Elena
0a35a072d9
Add RESET_SETTING to check if is a settings alter
2022-11-30 21:44:19 +01:00
Robert Schulze
6f564c59bd
Merge pull request #42641 from Enmk/fix_profile_names_mismatch
...
Fixed exception when user tries to log in
2022-11-30 13:01:57 +01:00
Anton Popov
df6d2f2163
Merge remote-tracking branch 'upstream/master' into HEAD
2022-11-29 23:21:23 +00:00
Alexey Milovidov
72bf1115e9
Merge branch 'master' into password-reset-message
2022-11-29 23:51:46 +01:00
Vasily Nemkov
3d0c07ac5b
Updated tests to match improved behaviour of currentProfiles()
2022-11-30 02:32:27 +04:00
Anton Popov
2a1fd48e91
fix tests
2022-11-29 17:33:35 +00:00
robot-clickhouse
ad66e01ace
Automatic style fix
2022-11-29 14:48:05 +00:00
Alexander Tokmakov
12a7d64ef5
Update test.py
2022-11-29 17:33:02 +03:00
Antonio Andelic
db46842c8a
Merge branch 'master' into more-aggresive-batching-keeper
2022-11-29 12:58:33 +00:00
alesapin
a800a80ab6
Merge pull request #43770 from ClickHouse/fix_flaky_store_cleanup2
...
Fix flaky `test_store_cleanup`
2022-11-29 13:11:57 +01:00
Antonio Andelic
e39f97a255
Properly close connections and add retries
2022-11-29 09:13:23 +00:00
Anton Popov
486da48ae7
fix tests
2022-11-28 21:15:41 +00:00
Alexander Tokmakov
40d28e3489
fix flaky test
2022-11-28 18:31:22 +01:00
Alexander Tokmakov
06e6844ec0
Update test.py
2022-11-28 18:22:03 +03:00
Antonio Andelic
af7f0257c3
Merge pull request #43658 from ClickHouse/try-fix-flaky-keeper-map
...
Fix flaky `KeeperMap` integration tests
2022-11-28 12:04:04 +01:00
Nikita Taranov
8ed5cfc265
Memory bound merging for distributed aggregation in order ( #40879 )
...
* impl
* fix style
* make executeQueryWithParallelReplicas similar to executeQuery
* impl for parallel replicas
* cleaner code for remote sorting properties
* update test
* fix
* handle when nodes of old versions participate
* small fixes
* temporary enable for testing
* fix after merge
* Revert "temporary enable for testing"
This reverts commit cce7f8884c
.
* review fixes
* add bc test
* Update src/Core/Settings.h
2022-11-28 00:41:31 +01:00
Antonio Andelic
cade21b76e
Fix 4LW test
2022-11-26 19:24:10 +00:00
Antonio Andelic
65dba31c3a
Merge branch 'master' into try-fix-flaky-keeper-map
2022-11-26 19:00:30 +01:00
Sema Checherinda
15a6ce254b
Merge pull request #41145 from CheSema/lock-free-drop-partition
...
Delete parts by replacing them with empty parts
2022-11-25 12:54:30 +01:00
Antonio Andelic
f3b01d455b
Use dummy pool
2022-11-25 11:12:01 +00:00
Anton Popov
c65be55da1
Merge remote-tracking branch 'upstream/master' into HEAD
2022-11-24 14:30:23 +00:00
Alexander Tokmakov
52d33fff00
Merge pull request #43366 from zzsmdfj/issue_15357_MaterializeMySQL_support_drop_mulit_table
...
MaterializeMySQL_support_drop_mulit_table
2022-11-24 14:54:24 +03:00
Sema Checherinda
6fd7dcf26f
Merge branch 'master' into lock-free-drop-partition
2022-11-24 12:34:19 +01:00
alesapin
f64f0461fe
Merge pull request #43288 from ClickHouse/Enable_compress_marks_and_primary_key
...
Enable compress marks and primary key for CI
2022-11-24 12:24:04 +01:00
Sema Checherinda
4f592bc208
fix isolation breach, droping outdated part should't be ignored
2022-11-23 15:16:09 +00:00
Sema Checherinda
f0294cd6ed
fix test_merge_tree_s3_restore
2022-11-23 15:16:09 +00:00
Sema Checherinda
597421014e
fix style check
2022-11-23 15:16:08 +00:00
Sema Checherinda
db86cd0f0a
fix several critical races, fix tests
2022-11-23 15:16:08 +00:00
Sema Checherinda
49539e6862
fix test_partition
2022-11-23 15:16:08 +00:00
Sema Checherinda
62d3e9da4d
fix test_merge_tree_hdfs
2022-11-23 15:16:08 +00:00
Sema Checherinda
d43d7ed6ea
fix test_merge_tree_empty_parts and test_ttl_replicated
2022-11-23 15:16:08 +00:00
Sema Checherinda
dd93d331c1
fix 01168_mutations_isolation
2022-11-23 15:16:08 +00:00
Sema Checherinda
9edad4a078
make test_detached_parts_metrics wait delete inactive and empty parts
2022-11-23 15:16:08 +00:00
Sema Checherinda
44a8f2f460
work with test_merge_tree_s3 and test_merge_tree_s3_restore
2022-11-23 15:16:08 +00:00
Sema Checherinda
cddf44034d
address test_transactions
2022-11-23 15:16:08 +00:00
Sema Checherinda
dc19141d84
fix test_detached_parts_metrics
2022-11-23 15:16:08 +00:00
Sema Checherinda
15012c7070
address the review comments
2022-11-23 15:16:08 +00:00
Sema Checherinda
fb86019db9
Delete parts over replacing them with empty parts
2022-11-23 15:16:08 +00:00
Alexander Tokmakov
f44eb4e021
Update test.py
2022-11-23 14:10:49 +03:00
kssenii
6044a9257c
Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands
2022-11-23 12:00:55 +01:00
Alexey Milovidov
bedfe68503
Merge branch 'master' into nv/fix-mutate-fetch
2022-11-22 20:06:56 +01:00
kssenii
5f649baffe
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2022-11-22 14:19:51 +01:00
Dmitry Novik
e67bda65a0
Merge pull request #43453 from ClickHouse/fix-hard-limit-in-test
...
Do not reuse jemalloc memory in test_global_overcommit
2022-11-22 13:24:34 +01:00
Alexander Tokmakov
cab33c02af
Merge branch 'master' into revert_SingleValueDataString
2022-11-22 14:27:58 +03:00
Antonio Andelic
843401c3a3
Merge pull request #43323 from ClickHouse/fix-describe-delta-lake-hudi
...
Fix `DESCRIBE` for `deltaLake` and `hudi` table functions
2022-11-22 09:09:39 +01:00
kssenii
a279bac767
Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands
2022-11-21 21:49:34 +01:00
Kruglov Pavel
581e57be9f
Merge pull request #43236 from xiedeyantu/fix_hdfscluster_connection_fail
...
fix skip_unavailable_shards does not work using hdfsCluster table function
2022-11-21 19:22:16 +01:00
Igor Nikonov
47ce029cbf
Merge pull request #43326 from ClickHouse/igor/test_read_only_fast_fix
...
Fix: make test_read_only_table more stable
2022-11-21 18:25:14 +01:00
Dmitry Novik
b17dc24a94
Do not reuse jemalloc memory in test
2022-11-21 16:10:47 +00:00
Raúl Marín
35e9e169a1
Merge branch 'master' into revert_SingleValueDataString
2022-11-21 15:11:13 +01:00
Kseniia Sumarokova
a59dac6337
Merge pull request #43143 from azat/disks/s3-check-fix
...
Add server UUID for disks access checks (read/read-by-offset/write/delete) to avoid possible races
2022-11-21 15:03:46 +01:00
alesapin
b78b602ea1
Merge branch 'master' into Enable_compress_marks_and_primary_key
2022-11-21 12:05:04 +01:00
Antonio Andelic
3cb202a63b
Merge pull request #43026 from JackyWoo/keeper_manually_assign_leader
...
Keeper support manually assigning leader
2022-11-21 09:30:00 +01:00
Antonio Andelic
9a8d82f0fc
Merge branch 'master' into fix-describe-delta-lake-hudi
2022-11-21 08:23:33 +00:00
zzsmdfj
a0f391745d
to 15357_MaterializeMySQL_support_drop_mulit_table-fix style-black
2022-11-21 11:48:57 +08:00
Azat Khuzhin
3e42ffd372
tests: fix hdfs disks (chicken and an egg problem with current cluster.py)
...
Right now cluster.py first creates HDFS and then creates clickhouse in
one API call, so you cannot interract and add missing mkdirs for the
clickhouse, fix this by using root dir where it is possible.
2022-11-20 16:28:35 +01:00
Azat Khuzhin
0678fba3d1
tests: fix test_disk_types (by adding mkdir for HDFS)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
Kseniia Sumarokova
f0dbfbb0f4
Merge pull request #42800 from azat/disks/web-fix
...
Do not suppress exceptions in web disk (and fix retries for requests from web disk)
2022-11-20 16:07:45 +01:00
Alexander Tokmakov
9464c32bb8
Merge branch 'master' into revert_SingleValueDataString
2022-11-19 18:00:04 +01:00
kssenii
08e4805c99
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2022-11-19 13:26:56 +01:00
Azat Khuzhin
c029549859
Allow to drop tables from s3_plain disk (so as from web disk)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-19 10:10:27 +01:00
zzsmdfj
e177927efb
to 15357_MaterializeMySQL_support_drop_mulit_table-fix code-style
2022-11-19 12:05:27 +08:00
alesapin
c7d648174b
Merge pull request #43328 from ClickHouse/better-format-version-check
...
Check content of `format_version` file in `MergeTreeData`
2022-11-18 22:34:37 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info
...
Improve masking sensitive info
2022-11-18 15:37:50 +01:00
zzsmdfj
9239f50a87
Merge branch 'master' into issue_15357_MaterializeMySQL_support_drop_mulit_table
2022-11-18 21:33:31 +08:00
zzsmdfj
6a998917b0
to MaterializeMySQL_support_drop_mulit_table
2022-11-18 21:29:13 +08:00
alesapin
06894534e8
Followup fix
2022-11-17 18:52:18 +01:00
robot-clickhouse
8c55679b01
Automatic style fix
2022-11-17 17:13:23 +00:00
alesapin
cecf032b9e
Disable compressed marks for some tests
2022-11-17 18:06:59 +01:00
kssenii
5cdf216c8f
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2022-11-17 16:39:36 +01:00
kssenii
69d71a8e5f
Mask collection values in logs
2022-11-17 16:01:42 +01:00
robot-clickhouse
926b35b7c4
Automatic style fix
2022-11-17 14:46:25 +00:00
kssenii
5d9b34b59d
More tests
2022-11-17 15:34:35 +01:00
robot-clickhouse
13e051a5de
Automatic style fix
2022-11-17 13:46:21 +00:00
Antonio Andelic
d41af8b741
Check and compare content of format_version file
2022-11-17 13:41:36 +00:00
Igor Nikonov
d2d6e75d97
Fix: make test_read_only_table more stable
...
+ add retries to INSERT queries after keeper node restart
2022-11-17 13:26:32 +00:00
Nikita Taranov
7beb58b0cf
Optimize merge of uniqExact without_key ( #43072 )
...
* impl for uniqExact
* rm unused (read|write)Text methods
* fix style
* small fixes
* impl for variadic uniqExact
* refactor
* fix style
* more agressive inlining
* disable if max_threads=1
* small improvements
* review fixes
* Revert "rm unused (read|write)Text methods"
This reverts commit a7e7480584
.
* encapsulate is_able_to_parallelize_merge in Data
* encapsulate is_exact & argument_is_tuple in Data
2022-11-17 13:19:02 +01:00
robot-clickhouse
1ad7362db7
Automatic style fix
2022-11-17 11:54:13 +00:00
Antonio Andelic
d5848d53cc
Fix DESCRIBE for deltaLake and hudi table functions
2022-11-17 11:46:17 +00:00
kssenii
1964291fba
Add tests
2022-11-17 12:35:43 +01:00
JackyWoo
5a09de279c
fix test
2022-11-17 10:35:01 +08:00
chen
cb83337586
Update test.py
2022-11-17 06:41:43 +08:00
xiedeyantu
fc1d06f096
fix
2022-11-16 23:43:23 +08:00
Raúl Marín
45a611bc1d
Fix style with black
2022-11-16 13:28:04 +01:00
Raúl Marín
4e7ccd1c06
Merge remote-tracking branch 'blessed/master' into revert_SingleValueDataString
2022-11-16 12:59:47 +01:00
xiedeyantu
7dc941cacd
fix
2022-11-16 19:47:57 +08:00
Alexander Tokmakov
7e6fe94a8e
Merge pull request #43266 from ClickHouse/global-overcommit-test
...
Make test_global_overcommit_tracker non-parallel
2022-11-16 14:28:56 +03:00
alesapin
32358d2180
Merge pull request #43256 from ClickHouse/vdimir/msg_in_wait_zookeeper_to_start
...
Better message in wait_zookeeper_to_start
2022-11-16 11:58:53 +01:00
JackyWoo
01b63fd98a
fix style
2022-11-16 12:51:51 +08:00
Anton Popov
f94182384a
Merge remote-tracking branch 'upstream/master' into HEAD
2022-11-16 03:30:29 +00:00
xiedeyantu
c23cd091a3
fix
2022-11-16 09:59:44 +08:00
JackyWoo
f8d71f60a2
Merge branch 'master' into keeper_manually_assign_leader
2022-11-16 09:34:40 +08:00
xiedeyantu
8144516e53
fix
2022-11-16 09:33:46 +08:00
JackyWoo
ace40a2538
fix tests
2022-11-16 09:33:46 +08:00
xiedeyantu
3d218795be
fix
2022-11-16 09:18:45 +08:00
Vitaly Baranov
2e242fa991
Add tests.
2022-11-16 01:35:11 +01:00
Dmitry Novik
fcdc9dfaac
Make test_global_overcommit_tracker non-parallel
2022-11-15 20:52:18 +00:00
Vitaly Baranov
22abe9a4fb
Merge pull request #42950 from azat/disks/s3_plain/wide-part-fix
...
Fixes for s3_plain disk that will allow to attach Wide parts
2022-11-15 17:56:16 +01:00
robot-clickhouse
aaeeeaf1fc
Automatic style fix
2022-11-15 16:40:01 +00:00
vdimir
03968eb694
Better message in wait_zookeeper_to_start
2022-11-15 16:17:12 +00:00
Antonio Andelic
6beab87610
Merge pull request #43080 from ucasfl/hudi-udtf
...
Add table function hudi and deltaLake
2022-11-15 15:42:25 +01:00
Kseniia Sumarokova
9fc96ab7fd
Merge branch 'master' into disks/web-fix
2022-11-15 12:16:32 +01:00
xiedeyantu
09c749fbd0
update
2022-11-15 13:30:29 +08:00
xiedeyantu
ec6698395e
fix skip_unavailable_shards does not work using hdfsCluster table function
2022-11-15 13:25:15 +08:00
Kruglov Pavel
386b69acae
Merge pull request #43131 from xiedeyantu/fix_s3cluster_connection_fail
...
fix skip_unavailable_shards does not work using s3Cluster table function
2022-11-14 20:48:00 +01:00
Raúl Marín
790cfe2b7e
Allow backward incompatible change for toDate32 since it was declared as such in 22.8
2022-11-14 18:10:36 +01:00
Kruglov Pavel
eacd7e1876
Merge pull request #42931 from zzsmdfj/MaterializedMySQL_improvement_bug_fix
...
Fix issue 42856 and MaterializedMySQL improvement
2022-11-14 16:54:12 +01:00
flynn
5bd8e2338b
Merge branch 'master' into hudi-udtf
2022-11-14 22:28:04 +08:00
Raúl Marín
58da0398a0
Merge remote-tracking branch 'blessed/master' into revert_SingleValueDataString
2022-11-14 12:46:00 +01:00
Antonio Andelic
41c0100dd7
Merge pull request #43115 from ClickHouse/fix-4lw-command-test
...
Fix `test_keeper_four_word_command::test_cmd_stat`
2022-11-14 08:33:11 +01:00
Azat Khuzhin
f2260ab724
Add a test for unavalable web server and web disk
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-12 13:25:10 +01:00
Azat Khuzhin
de5d777ca2
tests/integration: add ability not to retry server start
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-12 13:20:51 +01:00
Igor Nikonov
ee5756b5b4
Merge pull request #43122 from ClickHouse/revert-43116-revert-42607-igor/insert_zk_retries_retry
...
Revert "Revert " Keeper retries during insert (clean)""
2022-11-11 18:22:38 +01:00
Raúl Marín
5a3f324311
Try adding compatibility on write instead of on read
2022-11-11 18:12:34 +01:00
Dmitry Novik
31794284cb
Merge pull request #43105 from ClickHouse/overcommit-test
...
Rewrite tests for memory overcommit
2022-11-11 17:34:53 +01:00
xiedeyantu
5504f3af9b
fix skip_unavailable_shards does not work using s3Cluster table function
2022-11-12 00:03:36 +08:00
Anton Kozlov
ccede70b5c
Merge branch 'master' into tonickkozlov/add-interserver-dns-retries
2022-11-11 14:54:30 +00:00
Anton Kozlov
024d9382c7
CLICKHOUSE-2375 Add interserver DNS retries
2022-11-11 14:47:26 +00:00
Kruglov Pavel
4ecbcbf5e1
Merge branch 'master' into MaterializedMySQL_improvement_bug_fix
2022-11-11 12:54:02 +01:00
zzsmdfj
a9a208c3cd
to MaterializedMySQL_improvement_bug_fix-fix code style
2022-11-11 11:46:22 +08:00
zzsmdfj
227dcd68ba
Merge branch 'MaterializedMySQL_improvement_bug_fix' of github.com:zzsmdfj/ClickHouse into MaterializedMySQL_improvement_bug_fix
2022-11-11 11:25:03 +08:00
zzsmdfj
0e614aec12
to MaterializedMySQL_improvement_bug_fix-fix code sytle
2022-11-11 11:24:29 +08:00
zzsmdfj
b01eea5c10
Merge branch 'master' into MaterializedMySQL_improvement_bug_fix
2022-11-11 10:45:36 +08:00
zzsmdfj
d4e270b522
to MaterializedMySQL_improvement_bug_fix-fix codestyle
2022-11-11 10:44:16 +08:00
Stig Bakken
9020631826
Ignore SAVEPOINT queries in MaterializedMySQL
2022-11-10 23:27:21 +01:00
Azat Khuzhin
afc6c8bb73
tests: add ATTACH FROM s3_plain for wide part
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-10 17:42:32 +01:00
robot-clickhouse
fe283af82b
Automatic style fix
2022-11-10 16:08:05 +00:00
Dmitry Novik
4e15d648e7
Add test_global_overcommit_tracker/__init__.py
2022-11-10 16:02:45 +00:00
Dmitry Novik
b1e942e14c
Do not run global test with sanitizers
2022-11-10 16:00:10 +00:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)""
2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)"
2022-11-10 14:12:24 +03:00
Antonio Andelic
c50e795788
Fix expected ZXID
2022-11-10 11:04:51 +00:00
Yakov Olkhovskiy
c08826d174
Merge branch 'master' into tests-connection-error
2022-11-10 02:57:10 -05:00
JackyWoo
8e881e6009
Merge branch 'master' into keeper_manually_assign_leader
2022-11-10 14:38:04 +08:00
flynn
da77199bf5
fix style
...
fix style
2022-11-10 03:35:38 +00:00
flynn
6cbfab4e95
fix test
2022-11-10 03:10:25 +00:00
robot-clickhouse
eb612a699d
Automatic style fix
2022-11-10 01:08:12 +00:00
Dmitry Novik
981ece08ed
Rewrite tests for memory overcommit
2022-11-10 00:55:39 +00:00
flynn
0ffdf3e0c7
fix style
2022-11-09 12:42:13 +00:00
flynn
c226a3cc9a
fix style
2022-11-09 12:32:12 +00:00
JackyWoo
6a2957c269
fix test
2022-11-09 19:15:56 +08:00
flynn
592f6b31d9
fix and add test
2022-11-09 10:04:53 +00:00
Igor Nikonov
345304abe0
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-11-08 20:26:50 +00:00
kssenii
50c4beeef6
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2022-11-08 14:45:42 +01:00
Mikhail f. Shiryaev
cca2620c44
Merge pull request #42891 from ClickHouse/sync-integration-logs
...
Wait for all files are in sync before archiving them
2022-11-08 13:30:32 +01:00
JackyWoo
8f57ecf595
fix style
2022-11-08 19:56:20 +08:00
JackyWoo
a9d59a7e3f
Simplify keeper_utils.is_leader method
2022-11-08 19:43:36 +08:00
Kseniia Sumarokova
d2b777c059
Merge pull request #41054 from rubin-do/hudi
...
Add Hudi and DeltaLake table engines
2022-11-08 12:03:06 +01:00
JackyWoo
9060141f16
revert last change
2022-11-08 18:44:43 +08:00
JackyWoo
eeedd7d9ba
Keeper support manually assigning leader
2022-11-08 12:01:53 +08:00
Kseniia Sumarokova
4fc7926c16
Merge branch 'master' into system_table_add_column
2022-11-07 23:36:57 +01:00
alesapin
e6c3ea7ab8
Merge pull request #42890 from AVMusorin/s3_plain_system_name
...
changed type name for s3 plain storage
2022-11-07 22:46:25 +01:00
alesapin
257e766e0e
Trying to fix two flaky tests
2022-11-07 20:49:41 +01:00
kssenii
b5b25c2676
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2022-11-07 17:02:41 +01:00
kssenii
819be6bb32
Merge remote-tracking branch 'upstream/master' into hudi
2022-11-07 11:30:53 +01:00
Antonio Andelic
9ac829d4c4
Merge pull request #41766 from JackyWoo/manual_snapshot_for_keeper
...
Manually snapshot creation for keeper
2022-11-07 09:18:58 +01:00
Antonio Andelic
e08f94d0f8
Merge pull request #42659 from ClickHouse/optimize-entire-partition
...
Support for optimizing old parts for entire partition only
2022-11-07 09:06:11 +01:00
Yakov Olkhovskiy
11bc712c8e
add retries on ConnectionError
2022-11-07 07:12:55 +00:00
Igor Nikonov
503649a638
Merge branch 'master' into igor/insert_zk_retries_retry
2022-11-06 22:55:13 +01:00
Kseniia Sumarokova
cd5c6acbd2
Merge pull request #42960 from kssenii/fix-pg-db-datetime-table-attach
...
Fix postgres db engine attaching table with datetime
2022-11-06 22:39:15 +01:00
robot-clickhouse
f44e4fe7c6
Automatic style fix
2022-11-05 22:36:32 +00:00
Igor Nikonov
e94b9cd003
Fixes
...
+ enable retries in integration test
+ typo in fault injection traces
2022-11-05 22:28:32 +00:00
JackyWoo
9404e089fc
Merge branch 'master' into manual_snapshot_for_keeper
2022-11-05 17:37:36 +08:00
Igor Nikonov
859022660e
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-11-04 23:31:51 +00:00
Aleksandr Musorin
5cb69d8a22
changed type name for S3_Plain storage
...
renamed a disk for S3PlainObjectStorage in system.disks table from s3 to s3_plain
2022-11-04 17:35:51 +01:00
Vitaly Baranov
52b1f4aed9
Merge pull request #42484 from vitlibar/mask-sensitive-info-in-logs
...
Mask some information in logs
2022-11-04 14:09:38 +01:00
Kruglov Pavel
a7a5a70189
Merge pull request #42814 from kgoralski/failed_metrics_for_async_inserts
...
Add `FailedAsyncInsertQuery` event metric for async inserts
2022-11-04 13:41:39 +01:00
alesapin
82729a88c8
Merge pull request #42833 from vitlibar/backup-s3-increase-upload-part-size-exponentially
...
Increase the size of upload part exponentially for backup to S3.
2022-11-04 13:41:10 +01:00
kssenii
c9b4bc66b5
Fiux
2022-11-04 12:19:30 +01:00
Krzysztof Góralski
f526ddba68
Update tests/integration/test_failed_async_inserts/test.py
...
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
2022-11-03 22:26:24 +01:00
Krzysztof Góralski
14e922aca1
style check fixes test.py
2022-11-03 22:26:24 +01:00
Krzysztof Góralski
69f56d8d68
style check fixes
2022-11-03 22:26:24 +01:00
Krzysztof Góralski
be53a3596c
Review: Failed queries metrics for Async Inserts, added integration test instead of stateless one
2022-11-03 22:26:24 +01:00
Daniil Rubin
99d57325bb
Merge branch 'master' into hudi
2022-11-03 19:29:23 +00:00
Kruglov Pavel
b124875257
Merge branch 'master' into improve-streaming-engines
2022-11-03 13:22:06 +01:00
zzsmdfj
eeb9c57c59
to fix issue 42856 and MaterializedMySQL improvement
2022-11-03 20:21:02 +08:00
Antonio Andelic
66ffdda973
Merge branch 'master' into optimize-entire-partition
2022-11-03 09:52:13 +01:00
Antonio Andelic
25b19e6ba0
Merge branch 'master' into manual_snapshot_for_keeper
2022-11-03 08:55:01 +01:00
Igor Nikonov
9a315b7899
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-11-02 15:32:54 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs
2022-11-02 16:26:13 +01:00
avogar
9f39a6a049
Fix possible heap-use-after-free
2022-11-02 14:17:48 +00:00
Mikhail f. Shiryaev
550c776ffe
Wait for all files are in sync before archiving them
2022-11-02 13:16:27 +01:00
Vitaly Baranov
b9f2f17331
Add test and logging.
2022-11-01 12:23:20 +01:00
Vitaly Baranov
73f5664109
Fix tests.
2022-11-01 11:10:58 +01:00
Igor Nikonov
9cc64a0fcc
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-31 18:21:21 +00:00
Vitaly Baranov
d1c69a0753
Avoid changing query if it doesn't contain password.
2022-10-31 17:04:44 +01:00
Azat Khuzhin
8345c5f881
tests: cover ATTACH of BACKUP from s3_plain disk
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-31 12:02:26 +01:00
Vitaly Baranov
901ae126e1
Improve test test_on_cluster.
2022-10-31 10:50:33 +01:00
Vitaly Baranov
aa5cfd6980
Improve tests.
2022-10-31 10:50:33 +01:00
Vitaly Baranov
854e1e918f
Use assert_cast() instead of as() in some places.
2022-10-31 10:50:33 +01:00
Vitaly Baranov
43efbad642
Wipe passwords from distributed queries too.
2022-10-31 10:50:33 +01:00
Vitaly Baranov
dcf8724adf
Move prepareQueryForLogging() to a separate header.
2022-10-31 10:50:33 +01:00
Vitaly Baranov
5d2a222fe4
Mask sensitive information in logs.
2022-10-31 10:50:33 +01:00
avogar
8e13d1f1ec
Improve and refactor Kafka/StorageMQ/NATS and data formats
2022-10-28 16:41:10 +00:00
Azat Khuzhin
7a5432feaa
Fix possible SIGSEGV for web disks when file does not exists
...
It can be triggered in multiple ways, either when file does not exists
and you are trying to create MergeTree table from web (that has special
code for UUID handling) or by simply OPTIMIZE TABLE FINAL for MergeTree
table that is located on web disk, in both cases you will get the
following:
<details>
<summary>stacktrace</summary>
2022.10.28 14:08:40.631226 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> ContextAccess (default): Access granted: OPTIMIZE ON default.data_from_web
2022.10.28 14:08:40.632017 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Debug> default.data_from_web (a3e65e1f-5fd4-47ed-9dbd-307f2586b52d) (MergerMutator): Selected 1 parts from all_1_1_0 to all_1_1_0
2022.10.28 14:08:40.632496 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> default.data_from_web (a3e65e1f-5fd4-47ed-9dbd-307f2586b52d): Trying to reserve 1.00 MiB using storage policy from min volume index 0
2022.10.28 14:08:40.632752 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> DiskObjectStorage(DiskWebServer): Reserved 1.00 MiB on remote disk `web_disk`, having unreserved 16.00 EiB.
2022.10.28 14:08:40.634155 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Debug> MergeTask::PrepareStage: Merging 1 parts: from all_1_1_0 to all_1_1_0 into Compact
2022.10.28 14:08:40.634498 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> WebObjectStorage: Loading metadata for directory: http://127.0.0.1:8080/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1
2022.10.28 14:08:40.635025 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> DiskWeb: Adding directory: store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/
2022.10.28 14:08:40.635355 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> ReadWriteBufferFromHTTP: Sending request to http://127.0.0.1:8080/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index
2022.10.28 14:08:40.639618 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> DiskWeb: Cannot load disk metadata. Error: Code: 86. DB::Exception: Received error from remote server /store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index. HTTP status code: 404 Not Found, body: <!doctype html><html><head><title>404 Not Found</title><style>
body { background-color: #fcfcfc; color: #333333 ; margin: 0; padding:0; }
h1 { font-size: 1.5em; font-weight: normal; background-color: #9999cc; min-height:2em; line-height:2em; border-bottom: 1px inset black; margin: 0; }
h1, p { padding-left: 10px; }
code.url { background-color: #eeeeee; font-family:monospace; padding:0 2px;}
</style>
</head><body><h1>Not Found</h1><p>The requested resource <code class="url">/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index</code> was not found on this server.</p></body></html>: while loading disk metadata. (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER) (version 22.11.1.1)
2022.10.28 14:08:40.640527 [ 5488 ] {} <Trace> BaseDaemon: Received signal 11
2022.10.28 14:08:40.641529 [ 9027 ] {} <Fatal> BaseDaemon: ########################################
2022.10.28 14:08:40.642759 [ 9027 ] {} <Fatal> BaseDaemon: (version 22.11.1.1, build id: 12145DA78CE5E9EBB10A034177FAE5967EF81A4A) (from thread 6043) (query_id: a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1) (query: optimize table data_from_web final) Received signal Segmentation fault (11)
2022.10.28 14:08:40.643260 [ 9027 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Unknown si_code.
2022.10.28 14:08:40.643769 [ 9027 ] {} <Fatal> BaseDaemon: Stack trace: 0x7ffff416c0f2 0x7ffff7cd1ca8 0x7ffff679ae5e 0x7fffd52e7906 0x7fffd50c65aa 0x7fffca7a0d42 0x7fffcaee79ec 0x7fffcaf242f8 0x7fffcaf242b5 0x7fffcaf2427d 0x7fffcaf24255 0x7fffcaf2421d 0x7ffff65c3686 0x7ffff65c2295 0x7fffcaeee2a9 0x7fffcaef2c43 0x7fffcaee3c0e 0x7fffcc4a7851 0x7fffcc4a768f 0x7fffcc4abb2d 0x7fffcfdce828 0x7fffd03e3eaa 0x7fffd03dfe3b 0x7fffc8ec42d4 0x7fffc8ed51d2 0x7ffff4bdd839 0x7ffff4bde0a8 0x7ffff48ab261 0x7ffff48a769a 0x7ffff48a6335 0x7ffff409f8fd 0x7ffff4121a60
2022.10.28 14:08:40.644411 [ 9027 ] {} <Fatal> BaseDaemon: 4. ? @ 0x7ffff416c0f2 in ?
2022.10.28 14:08:40.676390 [ 9027 ] {} <Fatal> BaseDaemon: 5. /src/ch/clickhouse/src/Common/StringUtils/StringUtils.cpp:9: detail::startsWith(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, char const*, unsigned long) @ 0x1ca8 in /src/ch/clickhouse/.cmake/src/Common/StringUtils/libstring_utilsd.so
2022.10.28 14:08:40.730727 [ 9027 ] {} <Fatal> BaseDaemon: 6. /src/ch/clickhouse/src/Common/StringUtils/StringUtils.h:19: startsWith(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) @ 0x59ae5e in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
2022.10.28 14:08:40.923955 [ 9027 ] {} <Fatal> BaseDaemon: 7. /src/ch/clickhouse/src/Disks/ObjectStorages/Web/MetadataStorageFromStaticFilesWebServer.cpp:58: DB::MetadataStorageFromStaticFilesWebServer::exists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const @ 0x6e7906 in /src/ch/clickhouse/.cmake/src/libdbmsd.so
2022.10.28 14:08:41.291996 [ 9027 ] {} <Fatal> BaseDaemon: 8. /src/ch/clickhouse/src/Disks/ObjectStorages/DiskObjectStorage.cpp:181: DB::DiskObjectStorage::exists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const @ 0x4c65aa in /src/ch/clickhouse/.cmake/src/libdbmsd.so
2022.10.28 14:08:41.704697 [ 9027 ] {} <Fatal> BaseDaemon: 9. /src/ch/clickhouse/src/Storages/MergeTree/DataPartStorageOnDisk.cpp:74: DB::DataPartStorageOnDisk::exists() const @ 0xda0d42 in /src/ch/clickhouse/.cmake/src/libclickhouse_storages_mergetreed.so
2022.10.28 14:08:43.032459 [ 9027 ] {} <Fatal> BaseDaemon: 10. /src/ch/clickhouse/src/Storages/MergeTree/MergeTask.cpp:147: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x14e79ec in /src/ch/clickhouse/.cmake/src/libclickhouse_storages_mergetreed.so
...
Segmentation fault (core dumped)
</details>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-28 15:15:49 +02:00
Antonio Andelic
5fae8b0cc9
Merge branch 'master' into optimize-entire-partition
2022-10-28 11:44:10 +00:00
Igor Nikonov
3682913172
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-27 22:01:53 +00:00
Alexander Tokmakov
86b81a2b6f
Merge pull request #42234 from arthurpassos/fix_ares_fd_process_race_condition
...
Fix c-ares crash
2022-10-27 17:17:22 +03:00
Igor Nikonov
649e8d302c
Sync with retiring branch igor/insert_zk_retries
2022-10-26 17:41:44 +00:00
Igor Nikonov
8435cba352
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-26 17:35:47 +00:00
Antonio Andelic
35a9672704
Merge branch 'master' into optimize-entire-partition
2022-10-26 11:09:25 +00:00
Antonio Andelic
97aaebfa18
Address PR comments
2022-10-26 10:06:56 +00:00
Antonio Andelic
611c2e2bd7
Support for optimizing old parts for entire partition only
2022-10-25 13:40:18 +00:00
Nikolai Kochetov
5c4444237e
Merge pull request #42617 from ClickHouse/revert-revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3
...
Revert revert 41268 disable s3 parallel write for part moves to disk s3
2022-10-25 14:29:27 +02:00
Nikita Taranov
49f6692a2e
Adapt internal data structures to 512-bit era ( #42564 )
...
* impl
* update tests
* fix tests
2022-10-25 13:56:28 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections ( #42541 )
...
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2022-10-25 13:26:29 +02:00
JackyWoo
9a36a509fe
fix test
2022-10-25 17:15:49 +08:00
Alexey Milovidov
9ee7131f67
Merge pull request #42423 from ClickHouse/optimize-partitions-in-background
...
Optimize partitions in background
2022-10-24 19:41:50 +02:00
Nikolai Kochetov
beb40e4579
Split tests.
2022-10-24 14:22:06 +00:00
Vitaly Baranov
cd158b021f
Merge pull request #42562 from vitlibar/subqueries-in-row-policy-filters
...
Fix using subqueries in row policy filters
2022-10-24 13:38:15 +02:00
Igor Nikonov
54013009d3
Code migration from #41734 to align diff and git blame.
...
Initial implementation was different and it changed the entire
ReplicatedMergeTreeSink::commitPart() which change history provided by git blame.
Then RetriesControl.retryLoop() was introduced later which significantly reduces
the diff since it's like while() used before.
So, check outing the current version will keep more original history in
git blame, which is useful here
2022-10-24 08:20:11 +00:00
JackyWoo
42e391a019
fix test
2022-10-22 22:47:03 +08:00
JackyWoo
39c88c74e8
check whether last manually created snapshot is done
2022-10-22 22:31:17 +08:00
JackyWoo
bf291790b7
Merge branch 'master' into manual_snapshot_for_keeper
2022-10-22 16:57:50 +08:00
Azat Khuzhin
dbb4f8216f
Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32
...
Conflicts:
- src/Functions/GregorianDate.h
2022-10-22 08:20:05 +02:00
Nikolay Degterinsky
22f87e2bf9
Merge pull request #42549 from evillique/disks-better-logs
...
Better logs in clickhouse-disks
2022-10-22 04:31:44 +02:00
Azat Khuzhin
4eeb7337ed
Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32
...
Conflicts:
- src/Common/intExp.h
- src/Functions/DateTimeTransforms.h
- src/Functions/FunctionsConversion.h
2022-10-21 22:51:56 +02:00
Alexander Tokmakov
7d11054134
Merge branch 'master' into fix_ares_fd_process_race_condition
2022-10-21 18:27:22 +03:00
Vitaly Baranov
2033ea1216
Support subqueries in row policy filters.
2022-10-21 15:42:29 +02:00
Alexander Tokmakov
5ffd026261
Merge pull request #42529 from azat/tests/integration/fix-sanitizer-reports
...
Fix sanitizer reports in integration tests
2022-10-21 16:01:43 +03:00
Alexander Tokmakov
1343f62e8f
Update test.py
2022-10-21 15:59:27 +03:00
Azat Khuzhin
0365bc56aa
Add backward compatiblity check for some functions
...
* State of aggregate functions
This test goes though all aggregate functions that:
- has only one argument
- support string as an argument
And do a simple check by creating the aggregate state with one string.
Yes this is not covers everything (does not cover functions with
different number of arguments, types, different states in case of
multiple values - uniqCombined, but as for uniqCombined it will be
checked via uniqHLL12), but at least something.
And about type, String had been selected, since it more likely that
there will be used some hash function internally.
* Function(String)
Check every function which has only one argument, and it's type is
String.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:43 +02:00
Antonio Andelic
3091d609fa
Merge branch 'master' into optimize-partitions-in-background
2022-10-21 11:01:41 +02:00
Nikolay Degterinsky
d7d0794d2f
Better logs in clickhouse-disks
2022-10-20 23:17:29 +00:00
Nikolai Kochetov
57ffcb7ed0
Revert "Merge pull request #41681 from ClickHouse/revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3"
...
This reverts commit 7956c2becf
, reversing
changes made to 57be648984
.
2022-10-20 19:51:27 +02:00
Arthur Passos
50f9acb221
Merge branch 'master' into fix_ares_fd_process_race_condition
2022-10-20 13:35:58 -03:00
Arthur Passos
2fe59e7e8f
lower thread and iterations number
2022-10-20 13:30:20 -03:00
Alexander Tokmakov
905a4ded42
Merge pull request #42535 from ClickHouse/tavplubix-patch-7
...
Disable `test_storage_nats`, because it's permanently broken
2022-10-20 16:58:46 +03:00
Alexander Tokmakov
5b03bbce2e
Merge pull request #41981 from ClickHouse/fixes_for_replicated_merge_tree2
...
Some fixes for ReplicatedMergeTree
2022-10-20 16:57:51 +03:00
Alexander Tokmakov
89ff3d4731
Update test.py
2022-10-20 16:54:52 +03:00
Azat Khuzhin
c399b021cf
Fix sanitizer reports in integration tests
...
Before they was ignored because first there was a check for a sign of
sanitizer (==================), but it was done by
clickhouse-server.log, while sanitizer write to stderr.log.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-20 15:45:40 +02:00
Antonio Andelic
9d7e44409a
Merge pull request #41342 from ClickHouse/keeper-upload-snapshot-to-s3
...
Upload Keeper snapshots to S3
2022-10-20 09:14:59 +02:00
Antonio Andelic
77eb353839
Merge pull request #42243 from ClickHouse/use-multiread-more
...
Use MultiRead where possible
2022-10-20 09:04:54 +02:00
Alexander Tokmakov
f958c155f3
Merge branch 'master' into fixes_for_replicated_merge_tree2
2022-10-19 17:25:24 +03:00
Antonio Andelic
05e6b94b54
Polishing
2022-10-19 12:59:28 +00:00
Antonio Andelic
e5408aac99
Simplify logic
2022-10-19 12:34:20 +00:00
Arthur Passos
7f0a2d32c8
fix test errors
2022-10-18 20:15:58 -03:00
Vitaly Baranov
1365105bc4
Implement backup to S3
2022-10-19 00:04:41 +02:00
Alexander Tokmakov
ea77ff9022
Merge branch 'master' into fixes_for_replicated_merge_tree2
2022-10-18 17:25:25 +02:00
Antonio Andelic
e828dce331
Disable auto optimize by default
2022-10-18 12:11:23 +00:00
Antonio Andelic
bd2522fdd0
Merge branch 'master' into optimize-partitions-in-background
2022-10-18 09:13:59 +00:00
Antonio Andelic
ee2f5977d5
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-10-18 10:02:50 +02:00
chen
163b3c9f52
fix test
...
fix test
2022-10-18 14:53:21 +08:00
xiedeyantu
10acf43db4
fix test
2022-10-18 09:20:59 +08:00
Vitaly Baranov
eb637a6f81
Merge pull request #42232 from azat/backups/s3
...
Support BACKUP to S3 with as-is path/data structure
2022-10-18 02:37:07 +02:00
Yakov Olkhovskiy
3ce1fa09d6
Merge pull request #41198 from ClickHouse/composable-protocol
...
Composable protocol
2022-10-17 11:23:12 -04:00
Antonio Andelic
7c7395575f
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-10-17 11:02:36 +00:00
Antonio Andelic
878dee40b1
Use ZooKeeper in a single test
2022-10-17 07:36:49 +00:00
xiedeyantu
2a356715da
fix test
2022-10-16 16:46:00 +08:00
Arthur Passos
f414edebae
fix black style
2022-10-14 15:54:48 -03:00
Arthur Passos
299d848473
style fix?
2022-10-14 15:23:10 -03:00
Arthur Passos
78202f9660
Merge branch 'master' into fix_ares_fd_process_race_condition
2022-10-14 12:56:39 -03:00
Arthur Passos
6d22bb78e0
custom py script in integ tests
2022-10-14 12:39:08 -03:00
Anton Popov
95fdb0a877
remove outdated parts asynchronously
2022-10-14 14:52:57 +00:00
Azat Khuzhin
8830f0608d
Support BACKUP to S3 with as-is path/data structure
...
Right now backup to S3 does not make a lot of sense, since:
- it has random names, and to decoding them
- requires metadata from local disk (/var/lib/disks/DISK/BACKUP_NAME)
- or send_metadata (but it is also tricky even with it)
So this patch adds simpler interface for S3, it is only suitable for
BACKUP/RESTORE, so don't try to use it for MergeTree engine.
It is done by adding separate disk - `s3_plain` for this, that:
- does not support any extended features, like renames/hardlinks/attrs/...
(so basically everything that MergeTree requires)
- only write/read/unlink/list files
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-14 12:59:25 +02:00
nvartolomei
ea47ab1400
assert unused value in test_replicated_merge_tree_compatibility
2022-10-12 18:12:25 +01:00
Yakov Olkhovskiy
6c8e52eb67
Merge branch 'master' into composable-protocol
2022-10-08 20:33:32 -04:00
Anton Popov
9f1104c2ad
do not load inactive parts
2022-10-08 01:39:46 +00:00
robot-clickhouse
3bed015a1f
Automatic style fix
2022-10-08 00:09:24 +00:00
Yakov Olkhovskiy
6bb62d4d03
add test for PROXYv1
2022-10-08 00:01:58 +00:00
Vitaly Baranov
69ebf12dab
Merge pull request #42146 from azat/backups/metadata-overflow-fix
...
Fix reusing of files > 4GB from base backup
2022-10-08 00:22:28 +02:00
Alexander Tokmakov
b8d9066004
Revert "Resurrect parallel distributed insert select with s3Cluster ( #41535 )"
...
This reverts commit 860e34e760
.
2022-10-07 15:53:30 +02:00
Azat Khuzhin
94566abda9
Fix reusing of files > 4GB from base backup
...
Previosly u64 numbers was truncated to u32 numbers during writing to the
mdatadata xml file, and further incremental backup cannot reuse them,
since the file in base backup is smaller.
P.S. There can be other places, I thought about enabling
-Wshorten-64-to-32, but there are lots of warnings right now.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-07 14:56:09 +02:00
Yakov Olkhovskiy
2229087896
Merge branch 'master' into composable-protocol
2022-10-07 02:23:26 -04:00
Yakov Olkhovskiy
997fa5e295
review suggestions
2022-10-07 01:16:22 +00:00
Nikita Mikhaylov
860e34e760
Resurrect parallel distributed insert select with s3Cluster ( #41535 )
2022-10-06 13:47:32 +02:00
Nikita Mikhaylov
766107df0a
Fixed the restriction on maximum size of replicated fetches ( #42090 )
2022-10-06 13:18:46 +02:00
Kseniia Sumarokova
316824c02a
Merge pull request #42085 from elenatorro/use_send_metadata_setting_on_storage_config
...
Use send_metadata instead of send_object_metadata
2022-10-06 12:33:25 +02:00
Kseniia Sumarokova
3cff776109
Merge pull request #41254 from filimonov/system_disks_unreserved_space
...
Add unreserved_space column to system.disks
2022-10-06 12:28:23 +02:00
Elena
3914b8afda
Remove 'send_metadata' setting
2022-10-05 14:54:08 +02:00
Elena
5bbd2d1729
Use send_metadata instead of send_object_metadata
2022-10-05 13:43:59 +02:00
Kseniia Sumarokova
6775c2c0a8
Update test.py
2022-10-04 16:52:53 +02:00
Kseniia Sumarokova
0b0c7e359f
Update test.py
2022-10-04 16:04:57 +02:00
Kseniia Sumarokova
39d5dfb3f6
Fix integration test
2022-10-04 16:03:11 +02:00
Kseniia Sumarokova
1ffa3291d6
Update storage_conf.xml
2022-10-04 11:18:02 +02:00
kssenii
2ab884359e
More efficient WriteBufferFromAzureBlobStorage
2022-10-03 22:15:04 +02:00
kssenii
c6d7cd5820
Fix write into azure blob storage
2022-10-03 18:54:07 +02:00
Kseniia Sumarokova
0bfbb5c38a
Merge branch 'master' into system_disks_unreserved_space
2022-10-03 14:25:47 +02:00
Alexander Tokmakov
a2c29076d6
avoid detached broken part duplicates
2022-10-03 12:27:17 +02:00
Alexander Tokmakov
7b5b7e4963
Merge pull request #41945 from ClickHouse/enable_more_s3_tests
...
Enable some disabled S3 tests
2022-10-03 13:25:50 +03:00
Alexander Tokmakov
035be1feae
fix
2022-10-02 14:58:33 +02:00
robot-clickhouse
4cd2dc8d0d
Automatic style fix
2022-10-02 12:57:04 +00:00
Alexander Tokmakov
4ce8f47400
fix
2022-10-02 14:50:24 +02:00
Vitaly Baranov
baf48ec400
Merge pull request #41562 from vitlibar/refactor-wiping-info-from-logs
...
Refactor wiping sensitive information from logs.
2022-10-02 12:58:58 +02:00
Alexander Tokmakov
e40917a9c5
fix
2022-10-02 00:44:46 +02:00
Kseniia Sumarokova
fe6de361c6
Merge pull request #41933 from ClickHouse/kssenii-patch-5
...
Fix broken test_disks_app_func
2022-10-02 00:17:28 +02:00
Vitaly Baranov
a5b2a49ab1
Refactor wiping sensitive information from logs.
2022-10-01 17:17:45 +02:00
Alexey Milovidov
645dd997d3
Update test.py
2022-10-01 18:16:25 +03:00
robot-clickhouse
26ad8b2f13
Automatic style fix
2022-10-01 11:55:58 +00:00
Kseniia Sumarokova
3afec69021
Update test.py
2022-10-01 13:49:50 +02:00
kssenii
7fbc77eacc
Avoid incorrect warning during update of DiskSelector disks
2022-09-30 17:49:18 +02:00
kssenii
229414ed09
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2022-09-30 17:22:23 +02:00
kssenii
d538b5f3b8
Allow custom disk configuration
2022-09-30 17:20:10 +02:00
kssenii
fb3bcd2fe9
Fix use_environment_credentials
2022-09-30 12:25:39 +02:00
Alexander Tokmakov
bd305960e7
Merge pull request #41898 from ClickHouse/fix-tests-sync-replica
...
Another sync replicas in test_recovery_replica
2022-09-29 19:36:20 +03:00
alesapin
0097f15ee7
Merge pull request #41832 from ClickHouse/make_copy_instead_of_hardlink
...
Use copy for some operations instead of hardlinks
2022-09-29 17:12:08 +02:00
Alexander Tokmakov
4e422b8046
Merge pull request #41741 from ClickHouse/fix_intersecting_parts
...
Fix intersecting parts
2022-09-29 18:00:22 +03:00
robot-clickhouse
acdc3504aa
Automatic style fix
2022-09-29 11:21:41 +00:00
Kseniia Sumarokova
6270f513fc
Update test.py
2022-09-29 13:03:03 +02:00
Yakov Olkhovskiy
028af59702
Update tests/integration/test_recovery_replica/test.py
...
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2022-09-28 12:33:38 -04:00
alesapin
905d4585ac
Merge branch 'master' into make_copy_instead_of_hardlink
2022-09-28 16:47:45 +02:00
Yakov Olkhovskiy
10f6ba90e0
add sync_replicas
2022-09-28 10:00:59 -04:00
alesapin
4b4c63749a
Correct test version
2022-09-28 14:38:11 +02:00
Daniil Rubin
95b5d2d29d
Merge branch 'master' into hudi
2022-09-28 14:24:55 +03:00
alesapin
2eef384807
Add comments and test
2022-09-28 13:09:48 +02:00
Alexey Milovidov
e5ffd74a56
Merge branch 'master' into trying_to_fix_compressed_marks
2022-09-28 02:30:10 +03:00
Alexey Milovidov
12dc33f4fd
Merge branch 'master' into nv/fix-mutate-fetch
2022-09-28 01:28:19 +03:00
kssenii
01c4299481
Support single disk instead of storage policy
2022-09-27 22:34:31 +02:00
Alexander Tokmakov
0915bfdb17
fix
2022-09-27 17:42:36 +02:00
Alexander Tokmakov
b4a10c330c
fix
2022-09-27 17:34:24 +02:00
Antonio Andelic
036d1c8cbc
Merge pull request #40991 from ClickHouse/start-embedded-keeper-async
...
Always start embedded Keeper in async mode
2022-09-27 14:44:19 +02:00
robot-clickhouse
08af2a32bf
Automatic style fix
2022-09-27 09:53:26 +00:00
alesapin
4d2a5d752d
Add failing test
2022-09-27 11:36:20 +02:00
Antonio Andelic
655bde8e3f
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-09-27 07:28:26 +00:00
JackyWoo
d1c85f68e8
manually snapshot creation for keeper
2022-09-26 18:29:15 +08:00
robot-clickhouse
7a6386c732
Automatic style fix
2022-09-25 16:13:17 +00:00
Yakov Olkhovskiy
1407f6bcda
add test
2022-09-25 15:46:12 +00:00
kssenii
3725397040
Try fix azure tests
2022-09-23 15:24:10 +02:00
Alexander Tokmakov
3eebceaa38
Update test.py
2022-09-23 14:01:00 +03:00
Kseniia Sumarokova
ea43cb5648
Merge pull request #41261 from kssenii/s3-header-auth
...
Support s3 authorisation headers from ast arguments
2022-09-23 12:48:08 +02:00
Kseniia Sumarokova
a0c5da3459
Merge pull request #41652 from kssenii/fix-disk-web
...
Fixes for disk web
2022-09-23 12:43:50 +02:00
kssenii
0e8343e6f3
Revert "Better config for cache"
...
This reverts commit 0ca6aadbdb
.
2022-09-22 17:57:44 +02:00
kssenii
109d2ad022
Better
2022-09-22 16:22:05 +02:00
Antonio Andelic
44a3d6babe
Address PR comments
2022-09-22 13:03:27 +00:00
Kseniia Sumarokova
79304422b3
Update test.py
2022-09-22 13:11:35 +02:00
kssenii
cb442a3ad7
Fixes for disk web
2022-09-21 21:26:55 +02:00
kssenii
92c2219e82
Fix test
2022-09-21 20:51:09 +02:00
Antonio Andelic
6798b500e9
Wait on startup for Keeper
2022-09-21 15:12:16 +00:00
Antonio Andelic
aa700836b8
Merge branch 'master' into start-embedded-keeper-async
2022-09-21 13:46:54 +00:00
Kseniia Sumarokova
1e7308832c
Merge branch 'master' into system_disks_unreserved_space
2022-09-21 13:04:39 +02:00
kssenii
f917b268b3
Merge remote-tracking branch 'upstream/master' into s3-header-auth
2022-09-21 12:56:25 +02:00
nvartolomei
8d55637af8
Update test.py
2022-09-21 10:10:21 +01:00
Antonio Andelic
fc6c1faa57
Merge branch 'master' into keeper-upload-snapshot-to-s3
2022-09-21 06:57:58 +00:00
nvartolomei
842b0bb44b
Explain why hostName function is used in test mutation
2022-09-20 22:41:14 +01:00
nvartolomei
3d6545cea1
Reformat test.py to match style
2022-09-20 22:04:20 +01:00
Nicolae Vartolomei
45f1f572f0
Get rid of scary setting for testing fetching byte identical parts
2022-09-20 19:27:49 +00:00
Nicolae Vartolomei
90666a43a8
CLICKHOUSE-2153: Move new_data_part shared pointer to allow Outdated part to be deleted immediately on mutation failure
2022-09-20 18:23:19 +00:00