Commit Graph

399 Commits

Author SHA1 Message Date
Antonio Andelic
4ac090b122 Fix tests 2023-06-13 11:01:44 +00:00
Alexander Tokmakov
d2aa1779ed Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-06-07 14:58:50 +02:00
Alexander Tokmakov
4209ccfc08 fix 2023-06-02 17:42:04 +02:00
Alexander Tokmakov
d8f936987e Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-05-25 12:36:41 +02:00
何李夫
e4c8c4cecf
Add zookeeper name in endpoint id (#49780)
* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

Here is some infomation:
1.old table:
  CREATE TABLE a1 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
2.new table:
  CREATE TABLE a2 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('aux1:/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
3.error info:
  <Error> executeQuery: Code: 220. DB::Exception: Duplicate interserver IO endpoint:
          DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (DUPLICATE_INTERSERVER_IO_ENDPOINT)
  <Error> InterserverIOHTTPHandler: Code: 221. DB::Exception: No interserver IO endpoint
          named DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (NO_SUCH_INTERSERVER_IO_ENDPOINT)

* Revert "Add zookeeper name in endpoint id"

This reverts commit 9deb75b249619b7abdd38e3949ca8b3a76c9df8e.

* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

* Fix incompatible with a new setting

* add a test, fix other issues

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

* Update 02735_system_zookeeper_connection.reference

* Update 02735_system_zookeeper_connection.sql

* Update run.sh

* Remove the 'no-fasttest' tag

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-05-25 12:50:14 +03:00
Alexander Tokmakov
b8305503d8 more flexible cleanup thread scheduling 2023-05-22 19:07:18 +02:00
alekseygolub
70b9077bb2
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-21 17:49:29 +03:00
Alexey Milovidov
4dbe5b8329 Support them in tests 2023-05-19 11:13:28 +02:00
Kseniia Sumarokova
8c0b634a64
Update storage_conf.xml 2023-05-07 14:38:27 +02:00
Kseniia Sumarokova
431b2e94a9
Update storage_conf.xml 2023-05-05 19:10:12 +02:00
kssenii
9032e23f87 Clean up storage_conf.xml, use dynamic disks for tests 2023-05-05 15:43:23 +02:00
alekseygolub
963d6be120 Added configurations for DatabaseS3 2023-05-04 16:44:08 +00:00
alesapin
412b161104
Merge pull request #48791 from kssenii/better-local-object-storage
Make local object storage work consistently with s3 object storage, fix problem with append, make it configurable as independent storage
2023-05-04 11:47:43 +02:00
Kseniia Sumarokova
b00c859838
Merge branch 'master' into kssenii-patch-6 2023-05-01 12:24:39 +02:00
Alexey Milovidov
911d4e9977
Merge branch 'master' into fix_some_tests3 2023-04-28 17:23:46 +03:00
kssenii
298ed454ce Merge remote-tracking branch 'upstream/master' into better-local-object-storage 2023-04-28 13:26:55 +02:00
kssenii
9374666c0c Deprecetae local object storage on fake metadata storage 2023-04-15 13:53:11 +02:00
Alexander Tokmakov
70c1000c22 fix some tests 2023-04-14 19:42:17 +02:00
Kseniia Sumarokova
d04a19e160
Merge branch 'master' into kssenii-patch-6 2023-04-14 13:15:34 +02:00
kssenii
09c23287aa Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-04-13 13:25:45 +02:00
kssenii
8b98701edc Remove unfixably flaky test 2023-04-06 22:04:43 +02:00
kssenii
a405d06e8a Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-04-05 18:46:52 +02:00
Kseniia Sumarokova
b3854b09dc
Fix outdated cache configuration in tests s3_storage_policy_by_default.xml 2023-04-05 14:33:34 +02:00
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:48 +02:00
Antonio Andelic
f31f11dd67 Disable env credentials for stateless tests 2023-03-31 09:12:41 +00:00
kssenii
9acbcb5d50 Make max_size less in CI (for more pressure on cache) 2023-03-29 15:34:00 +02:00
kssenii
4cf2862a0e Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-03-29 12:50:36 +02:00
kssenii
88f631b297 Review fixes, perform delayed removal in background 2023-03-26 16:53:33 +02:00
Raúl Marín
bf7b0c5cfe Different approach to inject timeouts 2023-03-24 17:12:14 +01:00
Antonio Andelic
0ad436aa2c Revert debug changes 2023-03-09 08:07:59 +00:00
Antonio Andelic
37b09ce46a add back trace log 2023-03-07 12:16:01 +00:00
Antonio Andelic
cda4ff0df7 Remove trace logs 2023-03-01 19:54:22 +00:00
Antonio Andelic
32c198be45 trace raft logs 2023-03-01 12:47:31 +00:00
alesapin
3f2afa1fa0 Temporary enable force_sync for keeper in CI 2023-02-28 15:28:17 +01:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy 2023-02-09 13:16:55 +01:00
kssenii
a391de18a1 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-02-02 20:49:41 +01:00
Arthur Passos
7b1ceaa2c5
Merge branch 'master' into fix_cares_crash 2023-01-30 12:05:56 -03:00
Arthur Passos
513f430334 Add setting to enable/disable reverseDNSQuery function 2023-01-30 12:04:37 -03:00
Nikita Taranov
fdea042991
Add delete by ttl for zookeeper_log (#45419) 2023-01-19 13:42:16 +01:00
kssenii
2a030c1dc0 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-01-06 15:58:27 +01:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
alesapin
93b9d0a8a6
Revert strange change 2022-12-27 12:52:03 +01:00
Alexey Milovidov
91c1f9de8f Faster server startup after stress test 2022-12-26 18:13:50 +01:00
kssenii
853f2ea123 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 11:49:02 +01:00
kssenii
4cd4d2cc07 Fix test 2022-12-18 16:14:56 +01:00
kssenii
30547d2dcd Replace old named collections code for url 2022-12-17 00:24:05 +01:00
Kseniia Sumarokova
d3fed3b96a
Update storage_conf.xml 2022-12-15 22:40:04 +01:00
kssenii
dfefd8dfcd Fix write-throw-cache possible write finish 2022-12-15 19:58:54 +01:00
kssenii
ea5b06023a Merge remote-tracking branch 'refs/remotes/upstream/master' into allow-single-disk-instead-of-storage-policy 2022-12-14 13:59:51 +01:00
alesapin
d2f076768f Remove leftovers 2022-12-08 23:21:44 +01:00
kssenii
d278a0dbfb Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-12-05 12:59:44 +01:00
Nikita Taranov
8ed5cfc265
Memory bound merging for distributed aggregation in order (#40879)
* impl

* fix style

* make executeQueryWithParallelReplicas similar to executeQuery

* impl for parallel replicas

* cleaner code for remote sorting properties

* update test

* fix

* handle when nodes of old versions participate

* small fixes

* temporary enable for testing

* fix after merge

* Revert "temporary enable for testing"

This reverts commit cce7f8884c.

* review fixes

* add bc test

* Update src/Core/Settings.h
2022-11-28 00:41:31 +01:00
alesapin
e7717d7407 Enable compressed marks in CI 2022-11-18 15:54:22 +01:00
kssenii
50c4beeef6 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-08 14:45:42 +01:00
kssenii
b5b25c2676 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-07 17:02:41 +01:00
Vladimir C
0481999e35 Fix displayName test, add doc 2022-11-07 14:34:47 +01:00
Vladimir C
136f589343 Set display_name in functional tests 2022-11-07 14:34:47 +01:00
kssenii
301c7c90ef Add local disk path restriction 2022-10-21 23:59:53 +02:00
KinderRiven
32cf2c7623 add test for bypass_cache_threshold 2022-10-19 06:54:09 +00:00
Alexey Milovidov
59b37bf4a0
Merge pull request #40684 from ClickHouse/add-missing-tests-geobase
Add missing tests for legacy geobase
2022-09-28 14:08:30 +03:00
Alexey Milovidov
fcd443f0e3
Merge branch 'master' into add-missing-tests-geobase 2022-09-28 01:58:02 +03:00
Kseniia Sumarokova
134157df3d
Update storage_conf.xml 2022-09-27 22:42:39 +02:00
alesapin
34d9794ab7
Merge pull request #41653 from ClickHouse/investigating_more_bugs
Add very explicit logging on disk choice for fetch
2022-09-24 16:15:02 +02:00
alesapin
06e0f554d8 Fix fetch to local disk 2022-09-23 16:46:53 +02:00
Kseniia Sumarokova
133446a404
Update storage_conf.xml 2022-09-22 17:00:48 +02:00
kssenii
f038c7ea00 Better config for cache 2022-09-22 14:50:29 +02:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow 2022-09-19 12:52:36 +02:00
Antonio Andelic
baf7255cff
Merge pull request #39976 from ClickHouse/keeper-storage
KeeperMap storage engine
2022-09-13 14:15:09 +02:00
Alexander Tokmakov
4d146b05a9
Merge pull request #38262 from PolyProgrammist/fix-ordinary-system-unfreeze
Fix SYSTEM UNFREEZE for ordinary database
2022-09-13 14:55:11 +03:00
Sergei Trifonov
fe88a7991d
Merge branch 'master' into readonly-settings-allow 2022-09-12 16:20:40 +02:00
Antonio Andelic
1cd43d8e5c Merge branch 'master' into keeper-storage 2022-09-12 13:42:51 +00:00
Alexey Milovidov
61278c81e4 Merge branch 'master' into remove-useless-line 2022-09-10 05:07:10 +02:00
Antonio Andelic
32483aeec8 Merge branch 'master' into keeper-storage 2022-09-09 12:24:01 +00:00
Sergei Trifonov
6217559175
Merge branch 'master' into readonly-settings-allow 2022-09-06 16:10:48 +02:00
ianton-ru
39e1fc7a0f
Merge branch 'master' into fix-ordinary-system-unfreeze 2022-09-05 17:10:59 +03:00
Alexey Milovidov
0e5c94ce80 Merge branch 'master' into add-missing-tests-geobase 2022-09-04 03:32:29 +02:00
Alexey Milovidov
13a129bee7 Merge branch 'master' into remove-useless-line 2022-09-04 03:24:40 +02:00
Antonio Andelic
41b431c8b4 Enable keeper_map in tests 2022-09-02 07:41:25 +00:00
Sergei Trifonov
7fec55eea4 work in progress 2022-09-02 04:12:05 +02:00
Alexander Tokmakov
8bdb589c2b Merge branch 'master' into zookeeper_client_fault_injection 2022-08-29 13:34:57 +02:00
Alexey Milovidov
f462160bd1 Merge remote-tracking branch 'origin/master' into remove-useless-line 2022-08-28 21:00:29 +02:00
alesapin
bcc8106182 Merge branch 'master' into enable_zero_copy_replication_in_ci 2022-08-28 17:04:08 +02:00
alesapin
0b827207db Enable zero-copy replication in CI 2022-08-28 15:46:11 +02:00
Alexey Milovidov
b42d26acfe Remove one line from XML, because we do not care 2022-08-28 02:44:02 +02:00
Alexey Milovidov
632867dbff Fix error 2022-08-28 02:40:43 +02:00
Alexey Milovidov
a66710ca9d
Update legacy_geobase.xml 2022-08-28 03:36:02 +03:00
Alexey Milovidov
b14bd43590 Fix error 2022-08-27 20:25:06 +02:00
Alexey Milovidov
b70373d429 Fix error 2022-08-27 20:23:15 +02:00
Vadim Volodin
f140cedd88 fix xml 2022-08-19 10:41:19 +03:00
Vadim Volodin
4c60921da4 Move config 2022-08-19 08:59:13 +03:00
Alexander Tokmakov
3d253ec51b
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-18 21:23:50 +03:00
Alexander Tokmakov
ae000e9125
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-17 12:48:54 +03:00
kssenii
eb26b219b9 Merge master 2022-08-16 00:56:27 +02:00
Alexander Tokmakov
46aaf842f8
Merge pull request #40031 from ClickHouse/fix_old_tmp_dirs_cleanup
Fix old temporary directories cleanup
2022-08-12 20:31:20 +03:00
kssenii
768a207176 Merge master 2022-08-11 22:25:53 +02:00
Alexander Tokmakov
b9d18182f2 fix 2022-08-11 15:27:26 +02:00
kssenii
3d61baff72 Fix checks 2022-08-10 21:09:18 +02:00
kssenii
0ff5609738 Finish support for persistent files 2022-08-10 16:54:59 +02:00
Alexander Tokmakov
c0763050a7 more aggressive tests 2022-08-10 16:01:09 +02:00
Alexander Tokmakov
fff903ee81 fix 2022-08-10 15:48:56 +02:00
Alexander Tokmakov
fe572104aa fix old tmp dirs cleanup 2022-08-09 18:44:51 +02:00
kssenii
e81f85a79e Fix checks 2022-08-07 16:44:02 +02:00
Azat Khuzhin
479ea9e6a6 tests: enable prometheus exporter
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-06 12:58:08 +03:00
kssenii
91f4143b6e Merge master 2022-08-05 12:14:08 +02:00
Antonio Andelic
71e1b1916c Randomize snapshot on exit in tests 2022-08-03 11:43:30 +00:00
Alexander Tokmakov
ecf7ce1f74 Merge branch 'master' into zookeeper_client_fault_injection 2022-08-01 20:49:01 +02:00
Kseniia Sumarokova
6b60fb2067
Update storage_conf.xml 2022-07-20 22:35:33 +02:00
Kseniia Sumarokova
11c0a477a9
Update storage_conf.xml 2022-07-20 14:29:42 +02:00
kssenii
df17df67c2 Fix checks 2022-07-19 10:45:54 +02:00
kssenii
274457e5cf Allow cache over azure blob storage, add tests 2022-07-18 16:15:04 +02:00
kssenii
8ad26b3111 Merge master 2022-07-16 22:50:03 +02:00
Vitaly Baranov
6bf7bffbeb Correct the list of always accessible system tables. 2022-07-15 15:44:29 +02:00
Vitaly Baranov
c85b2b5732 Add option enabling that SELECT from the system database requires grant. 2022-07-15 15:44:25 +02:00
kssenii
9aa7e8aed8 Test for multi-layer cache 2022-07-14 17:31:41 +02:00
kssenii
31f0abe672 Test cache over WebObjectStorage 2022-07-14 15:48:07 +02:00
kssenii
b52084265c Merge master 2022-07-04 21:37:43 +02:00
Alexander Tokmakov
ba0fcec993 add background cleanup of store/ subdirs 2022-06-21 12:35:47 +02:00
kssenii
e974ab61e6 Fix 2022-06-15 11:31:45 +02:00
kssenii
4813b90281 Merge master 2022-06-13 21:32:00 +02:00
kssenii
aaf9a46f07 Merge master 2022-06-11 15:56:05 +02:00
Kseniia Sumarokova
580a30c6ae
Merge pull request #37859 from KinderRiven/support_max_request_cache_size
Support to set max cache size for per query in local cache
2022-06-10 16:17:18 +02:00
KinderRiven
6c76be9db1 fix config 2022-06-08 16:11:12 +08:00
KinderRiven
e0819f346a fix test 2022-06-08 16:04:47 +08:00
KinderRiven
b853f6e68c fix unit_test 2022-06-08 14:19:59 +08:00
Vitaly Baranov
d199478169
Merge pull request #37303 from ClickHouse/fix_trash
Try to fix some trash
2022-06-07 10:17:39 +02:00
Alexander Tokmakov
a893595025
Merge pull request #37596 from hanfei1991/hanfei/zk-write
Support insert into system.zookeeper
2022-06-02 15:09:07 +03:00
kssenii
e3647585aa Fix tests 2022-06-01 19:58:28 +02:00
kssenii
aa60de8f63 Merge master 2022-06-01 15:32:01 +02:00
Alexander Tokmakov
75f49a48e1 Merge branch 'master' into fix_trash 2022-06-01 14:20:46 +02:00
Han Fei
dd7e609585 add zk_write.xml 2022-06-01 17:08:19 +08:00
kssenii
53b6b76218 Better, add test 2022-05-31 15:33:56 +02:00
alesapin
59a070778d More quite logging for S3 tests 2022-05-29 14:48:04 +02:00
Alexander Tokmakov
4e52f45695 Merge branch 'master' into fix_trash 2022-05-28 19:43:19 +02:00
kssenii
b9618ea71c Fix checks 2022-05-28 01:17:08 +02:00
kssenii
0b350ce391 Fix checks 2022-05-27 20:52:30 +02:00
kssenii
25b1ed526d Merge master 2022-05-27 16:50:45 +02:00
Kseniia Sumarokova
f5d69506b4
Merge pull request #37516 from KinderRiven/improve_local_cache
Control cache downloads to avoid negative optimization of local caches
2022-05-27 11:53:17 +02:00
KinderRiven
55379f0d03 fix config 2022-05-26 20:59:31 +08:00
KinderRiven
ab4614dee5 fix 2022-05-26 16:39:50 +08:00
KinderRiven
f5a186c774 add test for local cache download 2022-05-26 16:36:26 +08:00
kssenii
b89a4edc04 Merge master 2022-05-25 20:55:04 +02:00
Alexander Tokmakov
4618429201 fixes 2022-05-24 18:53:52 +02:00
Alexander Tokmakov
bb5f04efaf Merge branch 'master' into fix_trash 2022-05-24 17:45:51 +02:00
Alexander Tokmakov
d0f998fb88 Merge branch 'master' into fix_trash 2022-05-23 21:25:56 +02:00
kssenii
9aa20193d5 Merge master 2022-05-22 00:56:20 +02:00
Alexander Tokmakov
c48410b574 fix trash in my code 2022-05-21 02:05:02 +02:00
Alexander Tokmakov
44f2d4529a better waiting, add fault injection 2022-05-20 22:10:01 +02:00
kssenii
e501a5a106 Update tests config 2022-05-19 15:55:07 +02:00
Alexander Tokmakov
538b5cd536 fix more trash 2022-05-18 14:06:52 +02:00
Alexander Tokmakov
5590adeffa fix more trash 2022-05-17 23:04:24 +02:00