thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 08:32:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
107,257 Commits 9,168 Branches 1,781 Tags 2.1 GiB
65cbb0fcfc
Commit Graph

5686 Commits

Author SHA1 Message Date
Azat Khuzhin
03f80bae90 Move dmesg dumping out from runner to ci-runner.py 
runner is used by developers to run tests, while ci-runner.py is used
only by CI scripts, and to avoid requiring CAP_SYSLOG for manual dmesg
clear/dump had been moved.

Also for manual runs this can be done manually.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-28 15:47:46 +01:00
Vitaly Baranov
8a2fbbe88c
Merge pull request #43947 from azat/backups/dedup 
RFC: Add ability to disable deduplication for BACKUP
 2022-12-28 15:24:53 +01:00
Alexey Milovidov
1b4121459d
Merge pull request #44645 from ClickHouse/keeper-test-fix 
Fix flaky test `test_keeper_multinode_simple`
 2022-12-28 16:38:17 +03:00
Azat Khuzhin
bc6adc88c8 tests/integration: add workaround for a bug in kazoo driver 
Refs: python-zk/kazoo#688
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-28 10:52:49 +01:00
Vitaly Baranov
b211dff013 Increase the create table timeout in test "test_backup_restore_on_cluster". 2022-12-28 00:17:19 +01:00
Nikita Mikhaylov
00ecf6dd05 Better 2022-12-27 17:04:47 +00:00
Nikita Mikhaylov
ce6264b684 Done 2022-12-27 17:03:57 +00:00
Azat Khuzhin
e1ad5cee7e tests/integration: add missing kazoo client termination 
pytest play games with logging output [1].

  [1]: https://github.com/pytest-dev/pytest/issues/5502

But this does not actually affect ClickHouse tests, the only reason is
that the kazoo client is not stopped correctly without calling
kazoo.client.KazooClient.stop(), and that's why you can see the
following messages:

<details>

    test_storage_rabbitmq/test.py::test_rabbitmq_big_message --- Logging error ---
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 179, in _socket_error_handling
        yield
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 659, in _connect
        self._socket = self.handler.create_connection(
      File "/usr/local/lib/python3.8/dist-packages/kazoo/handlers/threading.py", line 178, in create_connection
        return utils.create_tcp_connection(socket, *args, **kwargs)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/handlers/utils.py", line 265, in create_tcp_connection
        sock = module.create_connection(address, timeout_at)
      File "/usr/lib/python3.8/socket.py", line 808, in create_connection
        raise err
      File "/usr/lib/python3.8/socket.py", line 796, in create_connection
        sock.connect(sa)
    socket.timeout: timed out

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 579, in _connect_attempt
        read_timeout, connect_timeout = self._connect(host, hostip, port)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 659, in _connect
        self._socket = self.handler.create_connection(
      File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 182, in _socket_error_handling
        raise ConnectionDropped("socket connection error: %s" % (err,))
    kazoo.exceptions.ConnectionDropped: socket connection error: None

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib/python3.8/logging/__init__.py", line 1088, in emit
        stream.write(msg + self.terminator)
    ValueError: I/O operation on closed file.
    Call stack:
      File "/usr/lib/python3.8/threading.py", line 890, in _bootstrap
        self._bootstrap_inner()
      File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.8/threading.py", line 870, in run
        self._target(*self._args, **self._kwargs)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 512, in zk_loop
        if retry(self._connect_loop, retry) is STOP_CONNECTING:
      File "/usr/local/lib/python3.8/dist-packages/kazoo/retry.py", line 126, in __call__
        return func(*args, **kwargs)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 552, in _connect_loop
        status = self._connect_attempt(host, hostip, port, retry)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 622, in _connect_attempt
        self.logger.warning('Connection dropped: %s', e)
    Message: 'Connection dropped: %s'
    Arguments: (ConnectionDropped('socket connection error: None'),)

</details>

Which eventually leads to incorrectly parsed report.

This patch is an addition to the `logging.raiseExceptions=False` - #44618

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-27 17:01:06 +01:00
Kruglov Pavel
c3405be272
Increase the number of attempts in tests 2022-12-27 16:49:32 +01:00
Azat Khuzhin
c9c590071d Add ability to disable deduplication for BACKUP 
Right now BACKUP omit similar files, and will not allow to use this
backup as a regular table, and usually those similar files are quite
small (i.e. columns.txt).

So by using `BACKUP TO S3() deduplicate_files=0` you will be possible to
use `ATTACH TABLE` directly from S3.

P.S. right now it is possible only for the table with one part, since,
usually, there is nothing to deduplicate (if the columns are different).

v2: Add deduplicate_files into metadata
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-27 15:56:06 +01:00
Alexey Milovidov
7e447592b1
Merge pull request #44618 from azat/tests/integration/pytest-logging-fix 
tests/integration: suppress exceptions during logging (due to pytest)
 2022-12-27 14:33:37 +03:00
Alexey Milovidov
79d43c836a
Merge pull request #44572 from ClickHouse/fix-hdfs-test 
Fix HDFS test
 2022-12-27 14:31:00 +03:00
Alexey Milovidov
41fc65b47b
Merge pull request #44573 from ClickHouse/fix-distributed-queries-stress 
Fix test_distributed_queries_stress
 2022-12-27 14:30:28 +03:00
Azat Khuzhin
8b2ed0e3cd tests/integration: disable logging.raiseExceptions 
Due to pytest play games with logging [1] it is better to ignore it.

  [1]: https://github.com/pytest-dev/pytest/issues/5502

This is due to incorrect kazoo client usage (not all clients calls
stop())

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-26 22:57:54 +01:00
Alexey Milovidov
a92a9b4d55
Merge pull request #44535 from azat/tests/integration-dmesg 
tests: capture dmesg in integration tests
 2022-12-26 23:49:02 +03:00
Roman Vasin
465cc36526 Update Exception messages in test.py 2022-12-26 13:17:22 +00:00
Roman Vasin
483e26cf2f
Merge branch 'master' into ADQM-634 2022-12-26 15:34:38 +03:00
Roman Vasin
3cdc9b3f81 Remove -vvv from kinit call in tests 2022-12-26 11:59:37 +00:00
Roman Vasin
a70e3c20fa Make calls of test from Ubuntu client 2022-12-26 11:56:45 +00:00
Alexey Milovidov
a06c8268ea Remove the offender 2022-12-26 11:28:26 +01:00
Alexey Milovidov
f9f0f9446d Fix distributed_queries_stress 2022-12-25 09:46:51 +01:00
robot-clickhouse
19f05d4fb4 Automatic style fix 2022-12-25 08:38:02 +00:00
Alexey Milovidov
cbf140633b Fix HDFS test 2022-12-25 09:30:20 +01:00
Anton Popov
0722020cae Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-23 18:15:55 +00:00
Smita Kulkarni
84362461fb Updated test to use macro substitution for hdfs disk - Allow macro substitution in endpoint 2022-12-23 16:07:56 +01:00
Azat Khuzhin
69fadd2193 tests: capture dmesg in integration tests 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-12-23 15:50:10 +01:00
Smita Kulkarni
3d62bc2635 Allow macro substitution in endpoint of disks 
Implementation:
* Updated to substitute macros in endpoint for all disks (s3, hfs, web).
Testing:
* Added an integration test test/integration/test_endpoint_macro_substitution.
 2022-12-23 15:09:24 +01:00
Mikhail f. Shiryaev
6d0cd35cd0
Ignore exit code 1 for tar in integration tests 2022-12-23 13:08:59 +01:00
kssenii
8308177361 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 12:14:46 +01:00
Vladimir C
7482ea54ab
Merge pull request #43972 from ClickHouse/vdimir/tmp-data-in-fs-cache-2 2022-12-23 11:59:27 +01:00
kssenii
853f2ea123 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 11:49:02 +01:00
kssenii
8f1145f6e4 Fix tests 2022-12-23 11:47:51 +01:00
kssenii
11c0aa189a Better error message 2022-12-22 15:29:35 +01:00
Nikolai Kochetov
773de7a8a6
Merge pull request #44472 from CurtizJ/fix-mutations 
Fix mutations with setting `max_streams_for_merge_tree_reading`
 2022-12-22 14:05:20 +01:00
kssenii
a58b8b8e6c Fix tests 2022-12-22 12:27:42 +01:00
kssenii
ab458e68cd Fix tests 2022-12-22 12:24:20 +01:00
vdimir
182b34c11e
Fixes 2022-12-22 10:22:57 +00:00
vdimir
511e8d698d
Rename config param tmp_cache -> temporary_data_in_cache 2022-12-22 10:22:56 +00:00
vdimir
d30d25dbbe
Temporary files evict fs cache 2022-12-22 10:22:49 +00:00
Roman Vasin
2d9f3c242b Add test_bad_path_to_keytab test 2022-12-22 08:07:16 +00:00
Yakov Olkhovskiy
cea5b92536 binary backward compatibility test is added 2022-12-22 07:38:49 +00:00
kssenii
d3db1dd6a7 Fix tests 2022-12-22 00:27:22 +01:00
Anton Popov
5d6064cd28 better config in test 2022-12-21 11:35:11 +00:00
Kruglov Pavel
5e01a3d74e
Merge branch 'master' into improve-streaming-engines 2022-12-21 10:51:50 +01:00
Anton Popov
4b7f1e0cf2 fix mutations with setting 'max_streams_for_merge_tree_reading' 2022-12-21 01:59:53 +00:00
kssenii
1d75f740d7 Fix tests 2022-12-20 22:33:54 +01:00
kssenii
499565a604 Fix tests 2022-12-20 20:02:18 +01:00
Yakov Olkhovskiy
8462b724ca
Merge branch 'master' into feature-protocol-proxy 2022-12-20 13:12:07 -05:00
robot-clickhouse
82203e629c Automatic style fix 2022-12-20 17:46:17 +00:00
Yakov Olkhovskiy
9092134544 auth_use_forwarded_address config setting added, using forwarded address for proxied TCP and HTTP connection, likely bug is fixed in AccessControl, relevant test is appended 2022-12-20 17:39:40 +00:00
alesapin
80cae1aa3d Better 2022-12-20 17:37:03 +01:00
alesapin
33389f01e6 Disable tsan assertion for test 2022-12-20 17:33:08 +01:00
Konstantin Bogdanov
93375fc3e5
Merge pull request #44411 from thevar1able/fix-settings-constraints 
Fix settings constraints
 2022-12-20 10:19:07 +01:00
Konstantin Bogdanov
a094af9a49
Lint 2022-12-20 04:00:45 +01:00
Konstantin Bogdanov
b07001e7d7
Add a test 2022-12-20 03:26:31 +01:00
kssenii
a0c8d34775 RabbitMQ: respect flushing data by flush interval and by block size 2022-12-19 21:07:44 +01:00
Roman Vasin
8a692432de Remove unneeded headers from test.py 2022-12-19 11:48:10 +00:00
Roman Vasin
c26c257c57 Refactor test.py 2022-12-19 11:39:30 +00:00
Roman Vasin
64c0c6ae99 Refactor test_kerberos_auth test 2022-12-19 11:28:49 +00:00
Antonio Andelic
55dd591392 Try restarting ZK cluster on failed connection 2022-12-19 09:28:59 +00:00
Antonio Andelic
0162632275
Merge pull request #44306 from ClickHouse/another-fix-for-test-server-reload 
Another fix `test_server_reload`
 2022-12-19 09:14:03 +01:00
Nikolay Degterinsky
7a29bf0e4c
Merge pull request #43993 from evillique/fix_settings_constraints 
Fix CREATE query constraints
 2022-12-16 18:01:04 +01:00
Anton Popov
8b9b8b083c
Merge pull request #43726 from CurtizJ/optimize-storage-s3 
Improve performance of storage `S3` with large number of small files
 2022-12-16 14:38:10 +01:00
Antonio Andelic
372c187773 Correctly compare get result 2022-12-16 09:48:47 +00:00
Roman Vasin
775c700c24 Add second test for the case when there is no keytab 2022-12-16 08:20:01 +00:00
Nikolay Degterinsky
1861e670e9
Merge branch 'master' into fix_settings_constraints 2022-12-15 21:54:02 +01:00
Kruglov Pavel
c5b2e4cc23
Merge branch 'master' into improve-streaming-engines 2022-12-15 18:44:35 +01:00
Roman Vasin
5491aa9627 Implement wait_kerberos_kdc_is_available 2022-12-15 11:50:23 +00:00
Anton Popov
c4bcfe54cd fix test 2022-12-15 04:19:16 +00:00
Vitaly Baranov
fb8aca8319
Merge pull request #44158 from vitlibar/improve-referential-deps 
Improve referential dependencies
 2022-12-14 21:17:02 +01:00
Roman Vasin
e2ced517dd Add integration test test_kerberos_auth 2022-12-14 13:39:23 +00:00
Sema Checherinda
fdecb85dc4
Merge pull request #44156 from CheSema/fix_flaky_test_attach_detach 
make calls to be sure that parts are deleted
 2022-12-14 12:39:39 +01:00
Antonio Andelic
b2cce09004
Merge pull request #44184 from ClickHouse/close-grpc-channel-in-tests 
Close GRPC channels in tests
 2022-12-14 09:47:36 +01:00
Anton Popov
3008e701ea fix tests 2022-12-14 01:36:03 +00:00
Anton Popov
cce3257f39
Merge branch 'master' into optimize-storage-s3 2022-12-13 21:35:12 +01:00
Nikolay Degterinsky
809f15ac81
Merge branch 'master' into fix_settings_constraints 2022-12-13 20:33:16 +01:00
Nikolay Degterinsky
443bc3e656
Merge pull request #43719 from evillique/password_complexity_rules 
Add password complexity rules
 2022-12-13 17:56:45 +01:00
Anton Popov
0c87031e80 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-13 16:33:21 +00:00
Mikhail f. Shiryaev
9879d03bc4
Use contextmanager for grpc_channels 2022-12-13 12:43:48 +01:00
Antonio Andelic
b8f76ce0e5 Close GRPC channels in tests 2022-12-13 10:25:54 +00:00
Vitaly Baranov
5c1f490b3a Implement referential dependencies for table engine "Distributed" 
and for functions cluster() and clusterAllReplicas().
 2022-12-13 11:09:42 +01:00
Nikolay Degterinsky
f352403af2 Fix style 2022-12-13 09:41:31 +00:00
Nikolay Degterinsky
034269087f Fix tests 2022-12-13 09:16:23 +00:00
Anton Popov
0b396bab14 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-12 18:08:27 +00:00
Vitaly Baranov
5aaff60650 Fix referential dependencies when host & post in a clickHouse dictionary source are set by default. 2022-12-12 18:22:14 +01:00
Vitaly Baranov
76ba8ab3d4 Add new tests. 2022-12-12 18:22:09 +01:00
Anton Popov
a2fee110b8 add query for waiting of loading parts and fix test_transactions 2022-12-12 14:33:42 +00:00
Sema Checherinda
aecc509f80 make calls to be sure that parts are deleted 2022-12-12 12:37:52 +01:00
Nikolay Degterinsky
19e95e8cd8 Better implementation of password rules 2022-12-11 23:59:04 +00:00
Sema Checherinda
9a1152339c
Merge branch 'master' into do-not-run-clear-old-parts-at-shutdown 2022-12-10 09:44:56 +01:00
Nikolay Degterinsky
a8545a29a4 Better test 2022-12-10 01:32:15 +00:00
Sergei Trifonov
7c37e48092
Merge pull request #43903 from ClickHouse/merge_tree_settings_constraints 
Constraints on merge tree settings
 2022-12-09 14:16:14 +01:00
Kseniia Sumarokova
e21d4a93b8
Merge pull request #44050 from ClickHouse/fix_clickhouse_disk 
Slightly better clickhouse disks and remove DiskMemory
 2022-12-09 13:09:59 +01:00
Alexey Milovidov
68e506d48f Unify __init__.py files 2022-12-08 23:04:56 +01:00
Alexey Milovidov
f880644d3a
Merge branch 'master' into nv/fix-mutate-fetch 2022-12-09 00:51:54 +03:00
Sergei Trifonov
bae1519970
Merge branch 'master' into merge_tree_settings_constraints 2022-12-08 22:25:16 +01:00
robot-clickhouse
98ce0dbb12 Automatic style fix 2022-12-08 17:27:23 +00:00
alesapin
7a223f4956 Slightly better clickhouse disks and remove DiskMemory 2022-12-08 18:20:54 +01:00
serxa
6428b8a1bb check one more case in test 2022-12-07 20:36:04 +00:00
Alexey Milovidov
9c81fd7afc
Merge pull request #43517 from ClickHouse/password-reset-message 
Move password reset message from client to server
 2022-12-07 23:26:58 +03:00
serxa
05788245dd add test 2022-12-07 20:16:10 +00:00
Nikolay Degterinsky
8a53f5364b
Fix style 2022-12-07 10:36:16 +01:00
Nikolay Degterinsky
bb9fdc0839 Fix CREATE query constraints 2022-12-07 00:53:27 +00:00
Sema Checherinda
0b2a3195c8 adjust integration/test_partition/test_detached_part_dir_exists 2022-12-05 17:39:27 +01:00
Sema Checherinda
1f96ef57d2 remove integration/test_parts_removal 2022-12-05 17:24:54 +01:00
kssenii
dca3444283 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-12-04 15:56:02 +01:00
Vitaly Baranov
e1f7f04752
Referential dependencies for RESTORE (#43834) 
* Rename DDLDependencyVisitor -> DDLLoadingDependencyVisitor.

* Move building a loading graph to TablesLoader.

* Implement referential dependencies for tables and use them
when restoring tables from a backup.

* Remove StorageID::operator < (because of its inconsistency with ==).

* Add new tests.

* Fix test.

* Fix memory leak.

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2022-12-02 15:05:46 +01:00
Vladimir C
7d6950d397
Revert "Temporary files evict fs cache" 2022-12-02 14:50:56 +01:00
Alexander Tokmakov
edbd460997
Merge pull request #43848 from elenatorro/fix_reset_setting_on_cluster 
Add RESET_SETTING to check if it is a settings alter
 2022-12-02 15:06:28 +03:00
Alexander Tokmakov
c2a19fc354
Update test.py 2022-12-01 19:41:29 +03:00
Nikita Mikhaylov
9ef54c6ecc
Try to use exchanges during database replicated replica recovery process (#43628) 2022-12-01 13:50:56 +01:00
vdimir
08b8d335f0
add integration/test_temporary_data_in_cache 2022-12-01 11:49:25 +00:00
vdimir
7a814d2854
wip: temporary files evict fs cache 2022-12-01 11:49:24 +00:00
Elena Torró
5630412944
Merge branch 'master' into fix_reset_setting_on_cluster 2022-11-30 21:47:43 +01:00
Elena
0a35a072d9 Add RESET_SETTING to check if is a settings alter 2022-11-30 21:44:19 +01:00
Robert Schulze
6f564c59bd
Merge pull request #42641 from Enmk/fix_profile_names_mismatch 
Fixed exception when user tries to log in
 2022-11-30 13:01:57 +01:00
Anton Popov
df6d2f2163 Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-29 23:21:23 +00:00
Alexey Milovidov
72bf1115e9
Merge branch 'master' into password-reset-message 2022-11-29 23:51:46 +01:00
Vasily Nemkov
3d0c07ac5b Updated tests to match improved behaviour of currentProfiles() 2022-11-30 02:32:27 +04:00
Anton Popov
2a1fd48e91 fix tests 2022-11-29 17:33:35 +00:00
robot-clickhouse
ad66e01ace Automatic style fix 2022-11-29 14:48:05 +00:00
Alexander Tokmakov
12a7d64ef5
Update test.py 2022-11-29 17:33:02 +03:00
Antonio Andelic
db46842c8a Merge branch 'master' into more-aggresive-batching-keeper 2022-11-29 12:58:33 +00:00
alesapin
a800a80ab6
Merge pull request #43770 from ClickHouse/fix_flaky_store_cleanup2 
Fix flaky `test_store_cleanup`
 2022-11-29 13:11:57 +01:00
Antonio Andelic
e39f97a255 Properly close connections and add retries 2022-11-29 09:13:23 +00:00
Anton Popov
486da48ae7 fix tests 2022-11-28 21:15:41 +00:00
Alexander Tokmakov
40d28e3489 fix flaky test 2022-11-28 18:31:22 +01:00
Alexander Tokmakov
06e6844ec0
Update test.py 2022-11-28 18:22:03 +03:00
Antonio Andelic
af7f0257c3
Merge pull request #43658 from ClickHouse/try-fix-flaky-keeper-map 
Fix flaky `KeeperMap` integration tests
 2022-11-28 12:04:04 +01:00
Nikita Taranov
8ed5cfc265
Memory bound merging for distributed aggregation in order (#40879) 
* impl

* fix style

* make executeQueryWithParallelReplicas similar to executeQuery

* impl for parallel replicas

* cleaner code for remote sorting properties

* update test

* fix

* handle when nodes of old versions participate

* small fixes

* temporary enable for testing

* fix after merge

* Revert "temporary enable for testing"

This reverts commit cce7f8884c.

* review fixes

* add bc test

* Update src/Core/Settings.h
 2022-11-28 00:41:31 +01:00
Antonio Andelic
cade21b76e Fix 4LW test 2022-11-26 19:24:10 +00:00
Antonio Andelic
65dba31c3a
Merge branch 'master' into try-fix-flaky-keeper-map 2022-11-26 19:00:30 +01:00
Sema Checherinda
15a6ce254b
Merge pull request #41145 from CheSema/lock-free-drop-partition 
Delete parts by replacing them with empty parts
 2022-11-25 12:54:30 +01:00
Antonio Andelic
f3b01d455b Use dummy pool 2022-11-25 11:12:01 +00:00
Anton Popov
c65be55da1 Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-24 14:30:23 +00:00
Alexander Tokmakov
52d33fff00
Merge pull request #43366 from zzsmdfj/issue_15357_MaterializeMySQL_support_drop_mulit_table 
MaterializeMySQL_support_drop_mulit_table
 2022-11-24 14:54:24 +03:00
Sema Checherinda
6fd7dcf26f
Merge branch 'master' into lock-free-drop-partition 2022-11-24 12:34:19 +01:00
alesapin
f64f0461fe
Merge pull request #43288 from ClickHouse/Enable_compress_marks_and_primary_key 
Enable compress marks and primary key for CI
 2022-11-24 12:24:04 +01:00
Sema Checherinda
4f592bc208 fix isolation breach, droping outdated part should't be ignored 2022-11-23 15:16:09 +00:00
Sema Checherinda
f0294cd6ed fix test_merge_tree_s3_restore 2022-11-23 15:16:09 +00:00
Sema Checherinda
597421014e fix style check 2022-11-23 15:16:08 +00:00
Sema Checherinda
db86cd0f0a fix several critical races, fix tests 2022-11-23 15:16:08 +00:00
Sema Checherinda
49539e6862 fix test_partition 2022-11-23 15:16:08 +00:00
Sema Checherinda
62d3e9da4d fix test_merge_tree_hdfs 2022-11-23 15:16:08 +00:00
Sema Checherinda
d43d7ed6ea fix test_merge_tree_empty_parts and test_ttl_replicated 2022-11-23 15:16:08 +00:00
Sema Checherinda
dd93d331c1 fix 01168_mutations_isolation 2022-11-23 15:16:08 +00:00
Sema Checherinda
9edad4a078 make test_detached_parts_metrics wait delete inactive and empty parts 2022-11-23 15:16:08 +00:00
Sema Checherinda
44a8f2f460 work with test_merge_tree_s3 and test_merge_tree_s3_restore 2022-11-23 15:16:08 +00:00
Sema Checherinda
cddf44034d address test_transactions 2022-11-23 15:16:08 +00:00
Sema Checherinda
dc19141d84 fix test_detached_parts_metrics 2022-11-23 15:16:08 +00:00
Sema Checherinda
15012c7070 address the review comments 2022-11-23 15:16:08 +00:00
Sema Checherinda
fb86019db9 Delete parts over replacing them with empty parts 2022-11-23 15:16:08 +00:00
Alexander Tokmakov
f44eb4e021
Update test.py 2022-11-23 14:10:49 +03:00
kssenii
6044a9257c Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-23 12:00:55 +01:00
Alexey Milovidov
bedfe68503
Merge branch 'master' into nv/fix-mutate-fetch 2022-11-22 20:06:56 +01:00
Dmitry Novik
e67bda65a0
Merge pull request #43453 from ClickHouse/fix-hard-limit-in-test 
Do not reuse jemalloc memory in test_global_overcommit
 2022-11-22 13:24:34 +01:00
Alexander Tokmakov
cab33c02af
Merge branch 'master' into revert_SingleValueDataString 2022-11-22 14:27:58 +03:00
Antonio Andelic
843401c3a3
Merge pull request #43323 from ClickHouse/fix-describe-delta-lake-hudi 
Fix `DESCRIBE` for `deltaLake` and `hudi` table functions
 2022-11-22 09:09:39 +01:00
kssenii
a279bac767 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-21 21:49:34 +01:00
Kruglov Pavel
581e57be9f
Merge pull request #43236 from xiedeyantu/fix_hdfscluster_connection_fail 
fix skip_unavailable_shards does not work using hdfsCluster table function
 2022-11-21 19:22:16 +01:00
Igor Nikonov
47ce029cbf
Merge pull request #43326 from ClickHouse/igor/test_read_only_fast_fix 
Fix: make test_read_only_table more stable
 2022-11-21 18:25:14 +01:00
Dmitry Novik
b17dc24a94 Do not reuse jemalloc memory in test 2022-11-21 16:10:47 +00:00
Raúl Marín
35e9e169a1
Merge branch 'master' into revert_SingleValueDataString 2022-11-21 15:11:13 +01:00
Kseniia Sumarokova
a59dac6337
Merge pull request #43143 from azat/disks/s3-check-fix 
Add server UUID for disks access checks (read/read-by-offset/write/delete) to avoid possible races
 2022-11-21 15:03:46 +01:00
alesapin
b78b602ea1
Merge branch 'master' into Enable_compress_marks_and_primary_key 2022-11-21 12:05:04 +01:00
Antonio Andelic
3cb202a63b
Merge pull request #43026 from JackyWoo/keeper_manually_assign_leader 
Keeper support manually assigning leader
 2022-11-21 09:30:00 +01:00
Antonio Andelic
9a8d82f0fc Merge branch 'master' into fix-describe-delta-lake-hudi 2022-11-21 08:23:33 +00:00
zzsmdfj
a0f391745d to 15357_MaterializeMySQL_support_drop_mulit_table-fix style-black 2022-11-21 11:48:57 +08:00
Azat Khuzhin
3e42ffd372 tests: fix hdfs disks (chicken and an egg problem with current cluster.py) 
Right now cluster.py first creates HDFS and then creates clickhouse in
one API call, so you cannot interract and add missing mkdirs for the
clickhouse, fix this by using root dir where it is possible.
 2022-11-20 16:28:35 +01:00
Azat Khuzhin
0678fba3d1 tests: fix test_disk_types (by adding mkdir for HDFS) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-20 16:28:35 +01:00
Kseniia Sumarokova
f0dbfbb0f4
Merge pull request #42800 from azat/disks/web-fix 
Do not suppress exceptions in web disk (and fix retries for requests from web disk)
 2022-11-20 16:07:45 +01:00
Alexander Tokmakov
9464c32bb8 Merge branch 'master' into revert_SingleValueDataString 2022-11-19 18:00:04 +01:00
Azat Khuzhin
c029549859 Allow to drop tables from s3_plain disk (so as from web disk) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-19 10:10:27 +01:00
zzsmdfj
e177927efb to 15357_MaterializeMySQL_support_drop_mulit_table-fix code-style 2022-11-19 12:05:27 +08:00
alesapin
c7d648174b
Merge pull request #43328 from ClickHouse/better-format-version-check 
Check content of `format_version` file in `MergeTreeData`
 2022-11-18 22:34:37 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info 
Improve masking sensitive info
 2022-11-18 15:37:50 +01:00
zzsmdfj
9239f50a87
Merge branch 'master' into issue_15357_MaterializeMySQL_support_drop_mulit_table 2022-11-18 21:33:31 +08:00
zzsmdfj
6a998917b0 to MaterializeMySQL_support_drop_mulit_table 2022-11-18 21:29:13 +08:00
alesapin
06894534e8 Followup fix 2022-11-17 18:52:18 +01:00
robot-clickhouse
8c55679b01 Automatic style fix 2022-11-17 17:13:23 +00:00
alesapin
cecf032b9e Disable compressed marks for some tests 2022-11-17 18:06:59 +01:00
kssenii
69d71a8e5f Mask collection values in logs 2022-11-17 16:01:42 +01:00
robot-clickhouse
926b35b7c4 Automatic style fix 2022-11-17 14:46:25 +00:00
kssenii
5d9b34b59d More tests 2022-11-17 15:34:35 +01:00
robot-clickhouse
13e051a5de Automatic style fix 2022-11-17 13:46:21 +00:00
Antonio Andelic
d41af8b741 Check and compare content of format_version file 2022-11-17 13:41:36 +00:00
Igor Nikonov
d2d6e75d97 Fix: make test_read_only_table more stable 
+ add retries to INSERT queries after keeper node restart
 2022-11-17 13:26:32 +00:00
Nikita Taranov
7beb58b0cf
Optimize merge of uniqExact without_key (#43072) 
* impl for uniqExact

* rm unused (read|write)Text methods

* fix style

* small fixes

* impl for variadic uniqExact

* refactor

* fix style

* more agressive inlining

* disable if max_threads=1

* small improvements

* review fixes

* Revert "rm unused (read|write)Text methods"

This reverts commit a7e7480584.

* encapsulate is_able_to_parallelize_merge in Data

* encapsulate is_exact & argument_is_tuple in Data
 2022-11-17 13:19:02 +01:00
robot-clickhouse
1ad7362db7 Automatic style fix 2022-11-17 11:54:13 +00:00
Antonio Andelic
d5848d53cc Fix DESCRIBE for deltaLake and hudi table functions 2022-11-17 11:46:17 +00:00
kssenii
1964291fba Add tests 2022-11-17 12:35:43 +01:00
JackyWoo
5a09de279c fix test 2022-11-17 10:35:01 +08:00
chen
cb83337586
Update test.py 2022-11-17 06:41:43 +08:00
xiedeyantu
fc1d06f096 fix 2022-11-16 23:43:23 +08:00
Raúl Marín
45a611bc1d Fix style with black 2022-11-16 13:28:04 +01:00
Raúl Marín
4e7ccd1c06 Merge remote-tracking branch 'blessed/master' into revert_SingleValueDataString 2022-11-16 12:59:47 +01:00
xiedeyantu
7dc941cacd fix 2022-11-16 19:47:57 +08:00
Alexander Tokmakov
7e6fe94a8e
Merge pull request #43266 from ClickHouse/global-overcommit-test 
Make test_global_overcommit_tracker non-parallel
 2022-11-16 14:28:56 +03:00
alesapin
32358d2180
Merge pull request #43256 from ClickHouse/vdimir/msg_in_wait_zookeeper_to_start 
Better message in wait_zookeeper_to_start
 2022-11-16 11:58:53 +01:00
JackyWoo
01b63fd98a fix style 2022-11-16 12:51:51 +08:00
Anton Popov
f94182384a Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-16 03:30:29 +00:00
xiedeyantu
c23cd091a3 fix 2022-11-16 09:59:44 +08:00
JackyWoo
f8d71f60a2
Merge branch 'master' into keeper_manually_assign_leader 2022-11-16 09:34:40 +08:00
xiedeyantu
8144516e53 fix 2022-11-16 09:33:46 +08:00
JackyWoo
ace40a2538 fix tests 2022-11-16 09:33:46 +08:00
xiedeyantu
3d218795be fix 2022-11-16 09:18:45 +08:00
Vitaly Baranov
2e242fa991 Add tests. 2022-11-16 01:35:11 +01:00
Dmitry Novik
fcdc9dfaac Make test_global_overcommit_tracker non-parallel 2022-11-15 20:52:18 +00:00
Vitaly Baranov
22abe9a4fb
Merge pull request #42950 from azat/disks/s3_plain/wide-part-fix 
Fixes for s3_plain disk that will allow to attach Wide parts
 2022-11-15 17:56:16 +01:00
robot-clickhouse
aaeeeaf1fc Automatic style fix 2022-11-15 16:40:01 +00:00
vdimir
03968eb694
Better message in wait_zookeeper_to_start 2022-11-15 16:17:12 +00:00
Antonio Andelic
6beab87610
Merge pull request #43080 from ucasfl/hudi-udtf 
Add table function hudi and deltaLake
 2022-11-15 15:42:25 +01:00
Kseniia Sumarokova
9fc96ab7fd
Merge branch 'master' into disks/web-fix 2022-11-15 12:16:32 +01:00
xiedeyantu
09c749fbd0 update 2022-11-15 13:30:29 +08:00
xiedeyantu
ec6698395e fix skip_unavailable_shards does not work using hdfsCluster table function 2022-11-15 13:25:15 +08:00
Kruglov Pavel
386b69acae
Merge pull request #43131 from xiedeyantu/fix_s3cluster_connection_fail 
fix skip_unavailable_shards does not work using s3Cluster table function
 2022-11-14 20:48:00 +01:00
Raúl Marín
790cfe2b7e Allow backward incompatible change for toDate32 since it was declared as such in 22.8 2022-11-14 18:10:36 +01:00
Kruglov Pavel
eacd7e1876
Merge pull request #42931 from zzsmdfj/MaterializedMySQL_improvement_bug_fix 
Fix issue 42856 and MaterializedMySQL improvement
 2022-11-14 16:54:12 +01:00
flynn
5bd8e2338b
Merge branch 'master' into hudi-udtf 2022-11-14 22:28:04 +08:00
Raúl Marín
58da0398a0 Merge remote-tracking branch 'blessed/master' into revert_SingleValueDataString 2022-11-14 12:46:00 +01:00
Antonio Andelic
41c0100dd7
Merge pull request #43115 from ClickHouse/fix-4lw-command-test 
Fix `test_keeper_four_word_command::test_cmd_stat`
 2022-11-14 08:33:11 +01:00
Azat Khuzhin
f2260ab724 Add a test for unavalable web server and web disk 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-12 13:25:10 +01:00
Azat Khuzhin
de5d777ca2 tests/integration: add ability not to retry server start 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-12 13:20:51 +01:00
Igor Nikonov
ee5756b5b4
Merge pull request #43122 from ClickHouse/revert-43116-revert-42607-igor/insert_zk_retries_retry 
Revert "Revert " Keeper retries during insert (clean)""
 2022-11-11 18:22:38 +01:00
Raúl Marín
5a3f324311 Try adding compatibility on write instead of on read 2022-11-11 18:12:34 +01:00
Dmitry Novik
31794284cb
Merge pull request #43105 from ClickHouse/overcommit-test 
Rewrite tests for memory overcommit
 2022-11-11 17:34:53 +01:00
xiedeyantu
5504f3af9b fix skip_unavailable_shards does not work using s3Cluster table function 2022-11-12 00:03:36 +08:00
Kruglov Pavel
4ecbcbf5e1
Merge branch 'master' into MaterializedMySQL_improvement_bug_fix 2022-11-11 12:54:02 +01:00
zzsmdfj
a9a208c3cd to MaterializedMySQL_improvement_bug_fix-fix code style 2022-11-11 11:46:22 +08:00
zzsmdfj
227dcd68ba Merge branch 'MaterializedMySQL_improvement_bug_fix' of github.com:zzsmdfj/ClickHouse into MaterializedMySQL_improvement_bug_fix 2022-11-11 11:25:03 +08:00
zzsmdfj
0e614aec12 to MaterializedMySQL_improvement_bug_fix-fix code sytle 2022-11-11 11:24:29 +08:00
zzsmdfj
b01eea5c10
Merge branch 'master' into MaterializedMySQL_improvement_bug_fix 2022-11-11 10:45:36 +08:00
zzsmdfj
d4e270b522 to MaterializedMySQL_improvement_bug_fix-fix codestyle 2022-11-11 10:44:16 +08:00
Stig Bakken
9020631826 Ignore SAVEPOINT queries in MaterializedMySQL 2022-11-10 23:27:21 +01:00
Azat Khuzhin
afc6c8bb73 tests: add ATTACH FROM s3_plain for wide part 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-10 17:42:32 +01:00
robot-clickhouse
fe283af82b Automatic style fix 2022-11-10 16:08:05 +00:00
Dmitry Novik
4e15d648e7 Add test_global_overcommit_tracker/__init__.py 2022-11-10 16:02:45 +00:00
Dmitry Novik
b1e942e14c Do not run global test with sanitizers 2022-11-10 16:00:10 +00:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)"" 2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)" 2022-11-10 14:12:24 +03:00
Antonio Andelic
c50e795788 Fix expected ZXID 2022-11-10 11:04:51 +00:00
Yakov Olkhovskiy
c08826d174
Merge branch 'master' into tests-connection-error 2022-11-10 02:57:10 -05:00
JackyWoo
8e881e6009
Merge branch 'master' into keeper_manually_assign_leader 2022-11-10 14:38:04 +08:00
flynn
da77199bf5 fix style 
fix style
 2022-11-10 03:35:38 +00:00
flynn
6cbfab4e95 fix test 2022-11-10 03:10:25 +00:00
robot-clickhouse
eb612a699d Automatic style fix 2022-11-10 01:08:12 +00:00
Dmitry Novik
981ece08ed Rewrite tests for memory overcommit 2022-11-10 00:55:39 +00:00
flynn
0ffdf3e0c7 fix style 2022-11-09 12:42:13 +00:00
flynn
c226a3cc9a fix style 2022-11-09 12:32:12 +00:00
JackyWoo
6a2957c269 fix test 2022-11-09 19:15:56 +08:00
flynn
592f6b31d9 fix and add test 2022-11-09 10:04:53 +00:00
Igor Nikonov
345304abe0 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-08 20:26:50 +00:00
Mikhail f. Shiryaev
cca2620c44
Merge pull request #42891 from ClickHouse/sync-integration-logs 
Wait for all files are in sync before archiving them
 2022-11-08 13:30:32 +01:00
JackyWoo
8f57ecf595 fix style 2022-11-08 19:56:20 +08:00
JackyWoo
a9d59a7e3f Simplify keeper_utils.is_leader method 2022-11-08 19:43:36 +08:00
Kseniia Sumarokova
d2b777c059
Merge pull request #41054 from rubin-do/hudi 
Add Hudi and DeltaLake table engines
 2022-11-08 12:03:06 +01:00
JackyWoo
9060141f16 revert last change 2022-11-08 18:44:43 +08:00
JackyWoo
eeedd7d9ba Keeper support manually assigning leader 2022-11-08 12:01:53 +08:00
Kseniia Sumarokova
4fc7926c16
Merge branch 'master' into system_table_add_column 2022-11-07 23:36:57 +01:00
alesapin
e6c3ea7ab8
Merge pull request #42890 from AVMusorin/s3_plain_system_name 
changed type name for s3 plain storage
 2022-11-07 22:46:25 +01:00
alesapin
257e766e0e Trying to fix two flaky tests 2022-11-07 20:49:41 +01:00
kssenii
819be6bb32 Merge remote-tracking branch 'upstream/master' into hudi 2022-11-07 11:30:53 +01:00
Antonio Andelic
9ac829d4c4
Merge pull request #41766 from JackyWoo/manual_snapshot_for_keeper 
Manually snapshot creation for keeper
 2022-11-07 09:18:58 +01:00
Antonio Andelic
e08f94d0f8
Merge pull request #42659 from ClickHouse/optimize-entire-partition 
Support for optimizing old parts for entire partition only
 2022-11-07 09:06:11 +01:00
Yakov Olkhovskiy
11bc712c8e add retries on ConnectionError 2022-11-07 07:12:55 +00:00
Igor Nikonov
503649a638
Merge branch 'master' into igor/insert_zk_retries_retry 2022-11-06 22:55:13 +01:00
Kseniia Sumarokova
cd5c6acbd2
Merge pull request #42960 from kssenii/fix-pg-db-datetime-table-attach 
Fix postgres db engine attaching table with datetime
 2022-11-06 22:39:15 +01:00
robot-clickhouse
f44e4fe7c6 Automatic style fix 2022-11-05 22:36:32 +00:00
Igor Nikonov
e94b9cd003 Fixes 
+ enable retries in integration test
+ typo in fault injection traces
 2022-11-05 22:28:32 +00:00
JackyWoo
9404e089fc
Merge branch 'master' into manual_snapshot_for_keeper 2022-11-05 17:37:36 +08:00
Igor Nikonov
859022660e Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-04 23:31:51 +00:00
Aleksandr Musorin
5cb69d8a22 changed type name for S3_Plain storage 
renamed a disk for S3PlainObjectStorage in system.disks table from s3 to s3_plain
 2022-11-04 17:35:51 +01:00
Vitaly Baranov
52b1f4aed9
Merge pull request #42484 from vitlibar/mask-sensitive-info-in-logs 
Mask some information in logs
 2022-11-04 14:09:38 +01:00
Kruglov Pavel
a7a5a70189
Merge pull request #42814 from kgoralski/failed_metrics_for_async_inserts 
Add `FailedAsyncInsertQuery` event metric for async inserts
 2022-11-04 13:41:39 +01:00
alesapin
82729a88c8
Merge pull request #42833 from vitlibar/backup-s3-increase-upload-part-size-exponentially 
Increase the size of upload part exponentially for backup to S3.
 2022-11-04 13:41:10 +01:00
kssenii
c9b4bc66b5 Fiux 2022-11-04 12:19:30 +01:00
Krzysztof Góralski
f526ddba68 Update tests/integration/test_failed_async_inserts/test.py 
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
14e922aca1 style check fixes test.py 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
69f56d8d68 style check fixes 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
be53a3596c Review: Failed queries metrics for Async Inserts, added integration test instead of stateless one 2022-11-03 22:26:24 +01:00
Daniil Rubin
99d57325bb Merge branch 'master' into hudi 2022-11-03 19:29:23 +00:00
Kruglov Pavel
b124875257
Merge branch 'master' into improve-streaming-engines 2022-11-03 13:22:06 +01:00
zzsmdfj
eeb9c57c59 to fix issue 42856 and MaterializedMySQL improvement 2022-11-03 20:21:02 +08:00
Antonio Andelic
66ffdda973
Merge branch 'master' into optimize-entire-partition 2022-11-03 09:52:13 +01:00
Antonio Andelic
25b19e6ba0
Merge branch 'master' into manual_snapshot_for_keeper 2022-11-03 08:55:01 +01:00
Igor Nikonov
9a315b7899 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-02 15:32:54 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs 2022-11-02 16:26:13 +01:00
avogar
9f39a6a049 Fix possible heap-use-after-free 2022-11-02 14:17:48 +00:00
Mikhail f. Shiryaev
550c776ffe
Wait for all files are in sync before archiving them 2022-11-02 13:16:27 +01:00
Vitaly Baranov
b9f2f17331 Add test and logging. 2022-11-01 12:23:20 +01:00
Vitaly Baranov
73f5664109 Fix tests. 2022-11-01 11:10:58 +01:00
Igor Nikonov
9cc64a0fcc Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-31 18:21:21 +00:00
Vitaly Baranov
d1c69a0753 Avoid changing query if it doesn't contain password. 2022-10-31 17:04:44 +01:00
Azat Khuzhin
8345c5f881 tests: cover ATTACH of BACKUP from s3_plain disk 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Vitaly Baranov
901ae126e1 Improve test test_on_cluster. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
aa5cfd6980 Improve tests. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
854e1e918f Use assert_cast() instead of as() in some places. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
43efbad642 Wipe passwords from distributed queries too. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
dcf8724adf Move prepareQueryForLogging() to a separate header. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
5d2a222fe4 Mask sensitive information in logs. 2022-10-31 10:50:33 +01:00