何李夫
e4c8c4cecf
Add zookeeper name in endpoint id ( #49780 )
...
* Add zookeeper name in endpoint id
When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.
Here is some infomation:
1.old table:
CREATE TABLE a1 (`id` UInt64)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/default/a1/{shard}', '{replica}')
ORDER BY (id);
2.new table:
CREATE TABLE a2 (`id` UInt64)
ENGINE = ReplicatedMergeTree('aux1:/clickhouse/tables/default/a1/{shard}', '{replica}')
ORDER BY (id);
3.error info:
<Error> executeQuery: Code: 220. DB::Exception: Duplicate interserver IO endpoint:
DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
(DUPLICATE_INTERSERVER_IO_ENDPOINT)
<Error> InterserverIOHTTPHandler: Code: 221. DB::Exception: No interserver IO endpoint
named DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
(NO_SUCH_INTERSERVER_IO_ENDPOINT)
* Revert "Add zookeeper name in endpoint id"
This reverts commit 9deb75b249619b7abdd38e3949ca8b3a76c9df8e.
* Add zookeeper name in endpoint id
When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.
* Fix incompatible with a new setting
* add a test, fix other issues
* Update 02442_auxiliary_zookeeper_endpoint_id.sql
* Update 02735_system_zookeeper_connection.reference
* Update 02735_system_zookeeper_connection.sql
* Update run.sh
* Remove the 'no-fasttest' tag
* Update 02442_auxiliary_zookeeper_endpoint_id.sql
---------
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-05-25 12:50:14 +03:00
Alexander Tokmakov
b8305503d8
more flexible cleanup thread scheduling
2023-05-22 19:07:18 +02:00
alekseygolub
70b9077bb2
Merge branch 'ClickHouse:master' into localfilefunction
2023-05-21 17:49:29 +03:00
Alexey Milovidov
4dbe5b8329
Support them in tests
2023-05-19 11:13:28 +02:00
Kseniia Sumarokova
8c0b634a64
Update storage_conf.xml
2023-05-07 14:38:27 +02:00
Kseniia Sumarokova
431b2e94a9
Update storage_conf.xml
2023-05-05 19:10:12 +02:00
kssenii
9032e23f87
Clean up storage_conf.xml, use dynamic disks for tests
2023-05-05 15:43:23 +02:00
alekseygolub
963d6be120
Added configurations for DatabaseS3
2023-05-04 16:44:08 +00:00
alesapin
412b161104
Merge pull request #48791 from kssenii/better-local-object-storage
...
Make local object storage work consistently with s3 object storage, fix problem with append, make it configurable as independent storage
2023-05-04 11:47:43 +02:00
Kseniia Sumarokova
b00c859838
Merge branch 'master' into kssenii-patch-6
2023-05-01 12:24:39 +02:00
Alexey Milovidov
911d4e9977
Merge branch 'master' into fix_some_tests3
2023-04-28 17:23:46 +03:00
kssenii
298ed454ce
Merge remote-tracking branch 'upstream/master' into better-local-object-storage
2023-04-28 13:26:55 +02:00
kssenii
9374666c0c
Deprecetae local object storage on fake metadata storage
2023-04-15 13:53:11 +02:00
Alexander Tokmakov
70c1000c22
fix some tests
2023-04-14 19:42:17 +02:00
Kseniia Sumarokova
d04a19e160
Merge branch 'master' into kssenii-patch-6
2023-04-14 13:15:34 +02:00
kssenii
09c23287aa
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-13 13:25:45 +02:00
kssenii
8b98701edc
Remove unfixably flaky test
2023-04-06 22:04:43 +02:00
kssenii
a405d06e8a
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-04-05 18:46:52 +02:00
Kseniia Sumarokova
b3854b09dc
Fix outdated cache configuration in tests s3_storage_policy_by_default.xml
2023-04-05 14:33:34 +02:00
Azat Khuzhin
218b1f9c29
Add ability to throttle BACKUPs on per-server/backup basis
...
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server
Query settings:
- backup_read_bandwidth
- backup_write_bandwidth
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:48 +02:00
Antonio Andelic
f31f11dd67
Disable env credentials for stateless tests
2023-03-31 09:12:41 +00:00
kssenii
9acbcb5d50
Make max_size less in CI (for more pressure on cache)
2023-03-29 15:34:00 +02:00
kssenii
4cf2862a0e
Merge remote-tracking branch 'upstream/master' into cache-better-locks
2023-03-29 12:50:36 +02:00
kssenii
88f631b297
Review fixes, perform delayed removal in background
2023-03-26 16:53:33 +02:00
Raúl Marín
bf7b0c5cfe
Different approach to inject timeouts
2023-03-24 17:12:14 +01:00
Antonio Andelic
0ad436aa2c
Revert debug changes
2023-03-09 08:07:59 +00:00
Antonio Andelic
37b09ce46a
add back trace log
2023-03-07 12:16:01 +00:00
Antonio Andelic
cda4ff0df7
Remove trace logs
2023-03-01 19:54:22 +00:00
Antonio Andelic
32c198be45
trace raft logs
2023-03-01 12:47:31 +00:00
alesapin
3f2afa1fa0
Temporary enable force_sync for keeper in CI
2023-02-28 15:28:17 +01:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy
2023-02-09 13:16:55 +01:00
kssenii
a391de18a1
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2023-02-02 20:49:41 +01:00
Arthur Passos
7b1ceaa2c5
Merge branch 'master' into fix_cares_crash
2023-01-30 12:05:56 -03:00
Arthur Passos
513f430334
Add setting to enable/disable reverseDNSQuery function
2023-01-30 12:04:37 -03:00
Nikita Taranov
fdea042991
Add delete by ttl for zookeeper_log ( #45419 )
2023-01-19 13:42:16 +01:00
kssenii
2a030c1dc0
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2023-01-06 15:58:27 +01:00
kssenii
67509aa2d5
Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2
2023-01-03 16:41:30 +01:00
alesapin
93b9d0a8a6
Revert strange change
2022-12-27 12:52:03 +01:00
Alexey Milovidov
91c1f9de8f
Faster server startup after stress test
2022-12-26 18:13:50 +01:00
kssenii
853f2ea123
Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2
2022-12-23 11:49:02 +01:00
kssenii
4cd4d2cc07
Fix test
2022-12-18 16:14:56 +01:00
kssenii
30547d2dcd
Replace old named collections code for url
2022-12-17 00:24:05 +01:00
Kseniia Sumarokova
d3fed3b96a
Update storage_conf.xml
2022-12-15 22:40:04 +01:00
kssenii
dfefd8dfcd
Fix write-throw-cache possible write finish
2022-12-15 19:58:54 +01:00
kssenii
ea5b06023a
Merge remote-tracking branch 'refs/remotes/upstream/master' into allow-single-disk-instead-of-storage-policy
2022-12-14 13:59:51 +01:00
alesapin
d2f076768f
Remove leftovers
2022-12-08 23:21:44 +01:00
kssenii
d278a0dbfb
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2022-12-05 12:59:44 +01:00
Nikita Taranov
8ed5cfc265
Memory bound merging for distributed aggregation in order ( #40879 )
...
* impl
* fix style
* make executeQueryWithParallelReplicas similar to executeQuery
* impl for parallel replicas
* cleaner code for remote sorting properties
* update test
* fix
* handle when nodes of old versions participate
* small fixes
* temporary enable for testing
* fix after merge
* Revert "temporary enable for testing"
This reverts commit cce7f8884c
2022-11-28 00:41:31 +01:00
alesapin
e7717d7407
Enable compressed marks in CI
2022-11-18 15:54:22 +01:00
kssenii
50c4beeef6
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2022-11-08 14:45:42 +01:00
kssenii
b5b25c2676
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2022-11-07 17:02:41 +01:00
Vladimir C
0481999e35
Fix displayName test, add doc
2022-11-07 14:34:47 +01:00
Vladimir C
136f589343
Set display_name in functional tests
2022-11-07 14:34:47 +01:00
kssenii
301c7c90ef
Add local disk path restriction
2022-10-21 23:59:53 +02:00
KinderRiven
32cf2c7623
add test for bypass_cache_threshold
2022-10-19 06:54:09 +00:00
Alexey Milovidov
59b37bf4a0
Merge pull request #40684 from ClickHouse/add-missing-tests-geobase
...
Add missing tests for legacy geobase
2022-09-28 14:08:30 +03:00
Alexey Milovidov
fcd443f0e3
Merge branch 'master' into add-missing-tests-geobase
2022-09-28 01:58:02 +03:00
Kseniia Sumarokova
134157df3d
Update storage_conf.xml
2022-09-27 22:42:39 +02:00
alesapin
34d9794ab7
Merge pull request #41653 from ClickHouse/investigating_more_bugs
...
Add very explicit logging on disk choice for fetch
2022-09-24 16:15:02 +02:00
alesapin
06e0f554d8
Fix fetch to local disk
2022-09-23 16:46:53 +02:00
Kseniia Sumarokova
133446a404
Update storage_conf.xml
2022-09-22 17:00:48 +02:00
kssenii
f038c7ea00
Better config for cache
2022-09-22 14:50:29 +02:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow
2022-09-19 12:52:36 +02:00
Antonio Andelic
baf7255cff
Merge pull request #39976 from ClickHouse/keeper-storage
...
KeeperMap storage engine
2022-09-13 14:15:09 +02:00
Alexander Tokmakov
4d146b05a9
Merge pull request #38262 from PolyProgrammist/fix-ordinary-system-unfreeze
...
Fix SYSTEM UNFREEZE for ordinary database
2022-09-13 14:55:11 +03:00
Sergei Trifonov
fe88a7991d
Merge branch 'master' into readonly-settings-allow
2022-09-12 16:20:40 +02:00
Antonio Andelic
1cd43d8e5c
Merge branch 'master' into keeper-storage
2022-09-12 13:42:51 +00:00
Alexey Milovidov
61278c81e4
Merge branch 'master' into remove-useless-line
2022-09-10 05:07:10 +02:00
Antonio Andelic
32483aeec8
Merge branch 'master' into keeper-storage
2022-09-09 12:24:01 +00:00
Sergei Trifonov
6217559175
Merge branch 'master' into readonly-settings-allow
2022-09-06 16:10:48 +02:00
ianton-ru
39e1fc7a0f
Merge branch 'master' into fix-ordinary-system-unfreeze
2022-09-05 17:10:59 +03:00
Alexey Milovidov
0e5c94ce80
Merge branch 'master' into add-missing-tests-geobase
2022-09-04 03:32:29 +02:00
Alexey Milovidov
13a129bee7
Merge branch 'master' into remove-useless-line
2022-09-04 03:24:40 +02:00
Antonio Andelic
41b431c8b4
Enable keeper_map in tests
2022-09-02 07:41:25 +00:00
Sergei Trifonov
7fec55eea4
work in progress
2022-09-02 04:12:05 +02:00
Alexander Tokmakov
8bdb589c2b
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-29 13:34:57 +02:00
Alexey Milovidov
f462160bd1
Merge remote-tracking branch 'origin/master' into remove-useless-line
2022-08-28 21:00:29 +02:00
alesapin
bcc8106182
Merge branch 'master' into enable_zero_copy_replication_in_ci
2022-08-28 17:04:08 +02:00
alesapin
0b827207db
Enable zero-copy replication in CI
2022-08-28 15:46:11 +02:00
Alexey Milovidov
b42d26acfe
Remove one line from XML, because we do not care
2022-08-28 02:44:02 +02:00
Alexey Milovidov
632867dbff
Fix error
2022-08-28 02:40:43 +02:00
Alexey Milovidov
a66710ca9d
Update legacy_geobase.xml
2022-08-28 03:36:02 +03:00
Alexey Milovidov
b14bd43590
Fix error
2022-08-27 20:25:06 +02:00
Alexey Milovidov
b70373d429
Fix error
2022-08-27 20:23:15 +02:00
Vadim Volodin
f140cedd88
fix xml
2022-08-19 10:41:19 +03:00
Vadim Volodin
4c60921da4
Move config
2022-08-19 08:59:13 +03:00
Alexander Tokmakov
3d253ec51b
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-18 21:23:50 +03:00
Alexander Tokmakov
ae000e9125
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-17 12:48:54 +03:00
kssenii
eb26b219b9
Merge master
2022-08-16 00:56:27 +02:00
Alexander Tokmakov
46aaf842f8
Merge pull request #40031 from ClickHouse/fix_old_tmp_dirs_cleanup
...
Fix old temporary directories cleanup
2022-08-12 20:31:20 +03:00
kssenii
768a207176
Merge master
2022-08-11 22:25:53 +02:00
Alexander Tokmakov
b9d18182f2
fix
2022-08-11 15:27:26 +02:00
kssenii
3d61baff72
Fix checks
2022-08-10 21:09:18 +02:00
kssenii
0ff5609738
Finish support for persistent files
2022-08-10 16:54:59 +02:00
Alexander Tokmakov
c0763050a7
more aggressive tests
2022-08-10 16:01:09 +02:00
Alexander Tokmakov
fff903ee81
fix
2022-08-10 15:48:56 +02:00
Alexander Tokmakov
fe572104aa
fix old tmp dirs cleanup
2022-08-09 18:44:51 +02:00
kssenii
e81f85a79e
Fix checks
2022-08-07 16:44:02 +02:00
Azat Khuzhin
479ea9e6a6
tests: enable prometheus exporter
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-06 12:58:08 +03:00
kssenii
91f4143b6e
Merge master
2022-08-05 12:14:08 +02:00
Antonio Andelic
71e1b1916c
Randomize snapshot on exit in tests
2022-08-03 11:43:30 +00:00
Alexander Tokmakov
ecf7ce1f74
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-01 20:49:01 +02:00
Kseniia Sumarokova
6b60fb2067
Update storage_conf.xml
2022-07-20 22:35:33 +02:00
Kseniia Sumarokova
11c0a477a9
Update storage_conf.xml
2022-07-20 14:29:42 +02:00
kssenii
df17df67c2
Fix checks
2022-07-19 10:45:54 +02:00
kssenii
274457e5cf
Allow cache over azure blob storage, add tests
2022-07-18 16:15:04 +02:00
kssenii
8ad26b3111
Merge master
2022-07-16 22:50:03 +02:00
Vitaly Baranov
6bf7bffbeb
Correct the list of always accessible system tables.
2022-07-15 15:44:29 +02:00
Vitaly Baranov
c85b2b5732
Add option enabling that SELECT from the system database requires grant.
2022-07-15 15:44:25 +02:00
kssenii
9aa7e8aed8
Test for multi-layer cache
2022-07-14 17:31:41 +02:00
kssenii
31f0abe672
Test cache over WebObjectStorage
2022-07-14 15:48:07 +02:00
kssenii
b52084265c
Merge master
2022-07-04 21:37:43 +02:00
Alexander Tokmakov
ba0fcec993
add background cleanup of store/ subdirs
2022-06-21 12:35:47 +02:00
kssenii
e974ab61e6
Fix
2022-06-15 11:31:45 +02:00
kssenii
4813b90281
Merge master
2022-06-13 21:32:00 +02:00
kssenii
aaf9a46f07
Merge master
2022-06-11 15:56:05 +02:00
Kseniia Sumarokova
580a30c6ae
Merge pull request #37859 from KinderRiven/support_max_request_cache_size
...
Support to set max cache size for per query in local cache
2022-06-10 16:17:18 +02:00
KinderRiven
6c76be9db1
fix config
2022-06-08 16:11:12 +08:00
KinderRiven
e0819f346a
fix test
2022-06-08 16:04:47 +08:00
KinderRiven
b853f6e68c
fix unit_test
2022-06-08 14:19:59 +08:00
Vitaly Baranov
d199478169
Merge pull request #37303 from ClickHouse/fix_trash
...
Try to fix some trash
2022-06-07 10:17:39 +02:00
Alexander Tokmakov
a893595025
Merge pull request #37596 from hanfei1991/hanfei/zk-write
...
Support insert into system.zookeeper
2022-06-02 15:09:07 +03:00
kssenii
e3647585aa
Fix tests
2022-06-01 19:58:28 +02:00
kssenii
aa60de8f63
Merge master
2022-06-01 15:32:01 +02:00
Alexander Tokmakov
75f49a48e1
Merge branch 'master' into fix_trash
2022-06-01 14:20:46 +02:00
Han Fei
dd7e609585
add zk_write.xml
2022-06-01 17:08:19 +08:00
kssenii
53b6b76218
Better, add test
2022-05-31 15:33:56 +02:00
alesapin
59a070778d
More quite logging for S3 tests
2022-05-29 14:48:04 +02:00
Alexander Tokmakov
4e52f45695
Merge branch 'master' into fix_trash
2022-05-28 19:43:19 +02:00
kssenii
b9618ea71c
Fix checks
2022-05-28 01:17:08 +02:00
kssenii
0b350ce391
Fix checks
2022-05-27 20:52:30 +02:00
kssenii
25b1ed526d
Merge master
2022-05-27 16:50:45 +02:00
Kseniia Sumarokova
f5d69506b4
Merge pull request #37516 from KinderRiven/improve_local_cache
...
Control cache downloads to avoid negative optimization of local caches
2022-05-27 11:53:17 +02:00
KinderRiven
55379f0d03
fix config
2022-05-26 20:59:31 +08:00
KinderRiven
ab4614dee5
fix
2022-05-26 16:39:50 +08:00
KinderRiven
f5a186c774
add test for local cache download
2022-05-26 16:36:26 +08:00
kssenii
b89a4edc04
Merge master
2022-05-25 20:55:04 +02:00
Alexander Tokmakov
4618429201
fixes
2022-05-24 18:53:52 +02:00
Alexander Tokmakov
bb5f04efaf
Merge branch 'master' into fix_trash
2022-05-24 17:45:51 +02:00
Alexander Tokmakov
d0f998fb88
Merge branch 'master' into fix_trash
2022-05-23 21:25:56 +02:00
kssenii
9aa20193d5
Merge master
2022-05-22 00:56:20 +02:00
Alexander Tokmakov
c48410b574
fix trash in my code
2022-05-21 02:05:02 +02:00
Alexander Tokmakov
44f2d4529a
better waiting, add fault injection
2022-05-20 22:10:01 +02:00
kssenii
e501a5a106
Update tests config
2022-05-19 15:55:07 +02:00
Alexander Tokmakov
538b5cd536
fix more trash
2022-05-18 14:06:52 +02:00
Alexander Tokmakov
5590adeffa
fix more trash
2022-05-17 23:04:24 +02:00
Azat Khuzhin
01e1c5345a
Add separate CLUSTER grant
...
In case you have different roles for the same user on multiple clusters,
ON CLUSTER query can help to overcome some limitations.
Consider the following example:
- cluster_with_data, dev_user (readonly=2)
- stage_cluster, dev_user (readonly=0)
So when you will execute the following query from stage_cluster, it will
be successfully executed, since ON CLUSTER queries has different system
profile:
DROP DATABASE default ON CLUSTER cluster_with_data
This is not 100% safe, but at least something.
Note, that right now only ON CLUSTER query it self is supported, but
separate clusters are not (i.e. GRANT CLUSTER some_cluster_name TO
default), since right now grants sticked to database+.
v2: on_cluster_queries_require_cluster_grant
v3: fix test and process flags as bit mask
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-05-16 13:57:45 +03:00
kssenii
47ff7b6132
Fixes after merge
2022-05-13 19:22:22 +02:00
kssenii
58faaf0ea3
Merge master
2022-05-13 18:51:07 +02:00
Alexander Tokmakov
c23bc01196
Merge branch 'master' into zookeeper_client_fault_injection
2022-05-11 17:20:42 +02:00
