Commit Graph

570 Commits

Author SHA1 Message Date
Alexander Tokmakov
4209ccfc08 fix 2023-06-02 17:42:04 +02:00
Alexander Tokmakov
d8f936987e Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-05-25 12:36:41 +02:00
何李夫
e4c8c4cecf
Add zookeeper name in endpoint id (#49780)
* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

Here is some infomation:
1.old table:
  CREATE TABLE a1 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
2.new table:
  CREATE TABLE a2 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('aux1:/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
3.error info:
  <Error> executeQuery: Code: 220. DB::Exception: Duplicate interserver IO endpoint:
          DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (DUPLICATE_INTERSERVER_IO_ENDPOINT)
  <Error> InterserverIOHTTPHandler: Code: 221. DB::Exception: No interserver IO endpoint
          named DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (NO_SUCH_INTERSERVER_IO_ENDPOINT)

* Revert "Add zookeeper name in endpoint id"

This reverts commit 9deb75b249619b7abdd38e3949ca8b3a76c9df8e.

* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

* Fix incompatible with a new setting

* add a test, fix other issues

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

* Update 02735_system_zookeeper_connection.reference

* Update 02735_system_zookeeper_connection.sql

* Update run.sh

* Remove the 'no-fasttest' tag

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2023-05-25 12:50:14 +03:00
Alexander Tokmakov
b8305503d8 more flexible cleanup thread scheduling 2023-05-22 19:07:18 +02:00
alekseygolub
70b9077bb2
Merge branch 'ClickHouse:master' into localfilefunction 2023-05-21 17:49:29 +03:00
Alexey Milovidov
4dbe5b8329 Support them in tests 2023-05-19 11:13:28 +02:00
Kseniia Sumarokova
8c0b634a64
Update storage_conf.xml 2023-05-07 14:38:27 +02:00
Kseniia Sumarokova
431b2e94a9
Update storage_conf.xml 2023-05-05 19:10:12 +02:00
kssenii
9032e23f87 Clean up storage_conf.xml, use dynamic disks for tests 2023-05-05 15:43:23 +02:00
alekseygolub
963d6be120 Added configurations for DatabaseS3 2023-05-04 16:44:08 +00:00
alesapin
412b161104
Merge pull request #48791 from kssenii/better-local-object-storage
Make local object storage work consistently with s3 object storage, fix problem with append, make it configurable as independent storage
2023-05-04 11:47:43 +02:00
Kseniia Sumarokova
b00c859838
Merge branch 'master' into kssenii-patch-6 2023-05-01 12:24:39 +02:00
Alexey Milovidov
911d4e9977
Merge branch 'master' into fix_some_tests3 2023-04-28 17:23:46 +03:00
kssenii
298ed454ce Merge remote-tracking branch 'upstream/master' into better-local-object-storage 2023-04-28 13:26:55 +02:00
Kseniia Sumarokova
45e2d296f9
Merge branch 'master' into better-local-object-storage 2023-04-27 14:54:04 +02:00
Alexey Milovidov
3fa19c5ee2
Merge branch 'master' into fix_some_tests3 2023-04-27 15:04:15 +03:00
kssenii
cbf0b98102 Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-04-25 15:15:22 +02:00
kssenii
9374666c0c Deprecetae local object storage on fake metadata storage 2023-04-15 13:53:11 +02:00
Alexander Tokmakov
70c1000c22 fix some tests 2023-04-14 19:42:17 +02:00
Kseniia Sumarokova
d04a19e160
Merge branch 'master' into kssenii-patch-6 2023-04-14 13:15:34 +02:00
kssenii
09c23287aa Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-04-13 13:25:45 +02:00
Dmitry Novik
f8e4d6bb41 Add CI run with new analyzer enabled 2023-04-12 15:18:28 +00:00
kssenii
8b98701edc Remove unfixably flaky test 2023-04-06 22:04:43 +02:00
kssenii
a405d06e8a Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-04-05 18:46:52 +02:00
Kseniia Sumarokova
b3854b09dc
Fix outdated cache configuration in tests s3_storage_policy_by_default.xml 2023-04-05 14:33:34 +02:00
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:48 +02:00
Antonio Andelic
f31f11dd67 Disable env credentials for stateless tests 2023-03-31 09:12:41 +00:00
kssenii
9acbcb5d50 Make max_size less in CI (for more pressure on cache) 2023-03-29 15:34:00 +02:00
kssenii
4cf2862a0e Merge remote-tracking branch 'upstream/master' into cache-better-locks 2023-03-29 12:50:36 +02:00
kssenii
88f631b297 Review fixes, perform delayed removal in background 2023-03-26 16:53:33 +02:00
Raúl Marín
bf7b0c5cfe Different approach to inject timeouts 2023-03-24 17:12:14 +01:00
kssenii
75624c52f7 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-03-20 16:38:14 +01:00
Antonio Andelic
0ad436aa2c Revert debug changes 2023-03-09 08:07:59 +00:00
Antonio Andelic
37b09ce46a add back trace log 2023-03-07 12:16:01 +00:00
kssenii
0c4bb3c80f Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-03-02 19:08:29 +01:00
Antonio Andelic
cda4ff0df7 Remove trace logs 2023-03-01 19:54:22 +00:00
Antonio Andelic
32c198be45 trace raft logs 2023-03-01 12:47:31 +00:00
alesapin
3f2afa1fa0 Temporary enable force_sync for keeper in CI 2023-02-28 15:28:17 +01:00
kssenii
353fca74f0 Fix config 2023-02-27 20:58:39 +01:00
Kseniia Sumarokova
04d3d01bbe
Update prefetch_settings.xml 2023-02-23 19:23:02 +01:00
kssenii
ceff5f41d1 Fix tests 2023-02-22 12:27:07 +01:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy 2023-02-09 13:16:55 +01:00
kssenii
2a48a96f83 Fix tests 2023-02-09 12:26:15 +01:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
kssenii
a391de18a1 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-02-02 20:49:41 +01:00
Arthur Passos
3b1f37e1aa agora 2023-01-30 16:25:06 -03:00
Arthur Passos
7b1ceaa2c5
Merge branch 'master' into fix_cares_crash 2023-01-30 12:05:56 -03:00
Arthur Passos
513f430334 Add setting to enable/disable reverseDNSQuery function 2023-01-30 12:04:37 -03:00
Nikita Taranov
fdea042991
Add delete by ttl for zookeeper_log (#45419) 2023-01-19 13:42:16 +01:00
kssenii
2a030c1dc0 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-01-06 15:58:27 +01:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
alesapin
3eda778707 Better 2022-12-28 14:29:27 +01:00
alesapin
93b9d0a8a6
Revert strange change 2022-12-27 12:52:03 +01:00
Alexey Milovidov
91c1f9de8f Faster server startup after stress test 2022-12-26 18:13:50 +01:00
kssenii
853f2ea123 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 11:49:02 +01:00
kssenii
4cd4d2cc07 Fix test 2022-12-18 16:14:56 +01:00
kssenii
30547d2dcd Replace old named collections code for url 2022-12-17 00:24:05 +01:00
Kseniia Sumarokova
d3fed3b96a
Update storage_conf.xml 2022-12-15 22:40:04 +01:00
kssenii
dfefd8dfcd Fix write-throw-cache possible write finish 2022-12-15 19:58:54 +01:00
kssenii
ea5b06023a Merge remote-tracking branch 'refs/remotes/upstream/master' into allow-single-disk-instead-of-storage-policy 2022-12-14 13:59:51 +01:00
alesapin
d2f076768f Remove leftovers 2022-12-08 23:21:44 +01:00
kssenii
d278a0dbfb Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-12-05 12:59:44 +01:00
kssenii
d91f3808bd Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-12-05 12:54:32 +01:00
Nikita Taranov
8ed5cfc265
Memory bound merging for distributed aggregation in order (#40879)
* impl

* fix style

* make executeQueryWithParallelReplicas similar to executeQuery

* impl for parallel replicas

* cleaner code for remote sorting properties

* update test

* fix

* handle when nodes of old versions participate

* small fixes

* temporary enable for testing

* fix after merge

* Revert "temporary enable for testing"

This reverts commit cce7f8884c.

* review fixes

* add bc test

* Update src/Core/Settings.h
2022-11-28 00:41:31 +01:00
alesapin
f64f0461fe
Merge pull request #43288 from ClickHouse/Enable_compress_marks_and_primary_key
Enable compress marks and primary key for CI
2022-11-24 12:24:04 +01:00
alesapin
b78b602ea1
Merge branch 'master' into Enable_compress_marks_and_primary_key 2022-11-21 12:05:04 +01:00
kssenii
98f5cc10bb Merge remote-tracking branch 'upstream/master' into named-collections-access-fix 2022-11-18 19:51:28 +01:00
alesapin
e7717d7407 Enable compressed marks in CI 2022-11-18 15:54:22 +01:00
kssenii
5cdf216c8f Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-17 16:39:36 +01:00
Kseniia Sumarokova
494d0b575d
Fix tests 2022-11-17 14:33:30 +01:00
Igor Nikonov
b12ebab65c Enable keeper fault injection and retries for insert queries in
funcntional tests
2022-11-11 18:07:15 +00:00
Igor Nikonov
c00f71327a Fix: backward-compatility check
- remove unused config for now
2022-11-10 12:31:22 +00:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)"" 2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)" 2022-11-10 14:12:24 +03:00
Igor Nikonov
345304abe0 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-08 20:26:50 +00:00
kssenii
50c4beeef6 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-08 14:45:42 +01:00
Igor Nikonov
2c7a7fa406 Disable fault injection and retries function tests settings 2022-11-07 18:38:41 +00:00
kssenii
b5b25c2676 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-07 17:02:41 +01:00
vdimir
ebb51a2023 Use config.d/display_name.xml in tests 2022-11-07 14:34:47 +01:00
Vladimir C
0481999e35 Fix displayName test, add doc 2022-11-07 14:34:47 +01:00
Vladimir C
136f589343 Set display_name in functional tests 2022-11-07 14:34:47 +01:00
Igor Nikonov
dd598fdd4a Added missed insert_keeper_max_retries setting in test config 2022-11-05 17:57:59 +00:00
Igor Nikonov
5336addd0f Disable fault injections by default
+ enable fault injection for functional tests
2022-10-24 09:45:24 +00:00
kssenii
301c7c90ef Add local disk path restriction 2022-10-21 23:59:53 +02:00
KinderRiven
32cf2c7623 add test for bypass_cache_threshold 2022-10-19 06:54:09 +00:00
Alexander Tokmakov
4e422b8046
Merge pull request #41741 from ClickHouse/fix_intersecting_parts
Fix intersecting parts
2022-09-29 18:00:22 +03:00
Alexey Milovidov
59b37bf4a0
Merge pull request #40684 from ClickHouse/add-missing-tests-geobase
Add missing tests for legacy geobase
2022-09-28 14:08:30 +03:00
Alexey Milovidov
fcd443f0e3
Merge branch 'master' into add-missing-tests-geobase 2022-09-28 01:58:02 +03:00
Alexey Milovidov
2c4a0a11a8
Update install.sh 2022-09-28 01:57:47 +03:00
Kseniia Sumarokova
134157df3d
Update storage_conf.xml 2022-09-27 22:42:39 +02:00
Alexander Tokmakov
49983dafeb fix 2022-09-26 17:13:10 +02:00
alesapin
34d9794ab7
Merge pull request #41653 from ClickHouse/investigating_more_bugs
Add very explicit logging on disk choice for fetch
2022-09-24 16:15:02 +02:00
alesapin
06e0f554d8 Fix fetch to local disk 2022-09-23 16:46:53 +02:00
Kseniia Sumarokova
133446a404
Update storage_conf.xml 2022-09-22 17:00:48 +02:00
kssenii
f038c7ea00 Better config for cache 2022-09-22 14:50:29 +02:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow 2022-09-19 12:52:36 +02:00
Antonio Andelic
baf7255cff
Merge pull request #39976 from ClickHouse/keeper-storage
KeeperMap storage engine
2022-09-13 14:15:09 +02:00
Alexander Tokmakov
4d146b05a9
Merge pull request #38262 from PolyProgrammist/fix-ordinary-system-unfreeze
Fix SYSTEM UNFREEZE for ordinary database
2022-09-13 14:55:11 +03:00
Kseniia Sumarokova
b2c9c04c7b
Merge pull request #40821 from kssenii/improve-marks-cache-loading
Allow to load marks in threadpool in advance
2022-09-13 12:31:58 +02:00
Sergei Trifonov
fe88a7991d
Merge branch 'master' into readonly-settings-allow 2022-09-12 16:20:40 +02:00
Antonio Andelic
1cd43d8e5c Merge branch 'master' into keeper-storage 2022-09-12 13:42:51 +00:00
kssenii
88b589917c Update defaults 2022-09-12 12:48:47 +02:00
Alexey Milovidov
61278c81e4 Merge branch 'master' into remove-useless-line 2022-09-10 05:07:10 +02:00
Antonio Andelic
32483aeec8 Merge branch 'master' into keeper-storage 2022-09-09 12:24:01 +00:00
Sergei Trifonov
6217559175
Merge branch 'master' into readonly-settings-allow 2022-09-06 16:10:48 +02:00
ianton-ru
39e1fc7a0f
Merge branch 'master' into fix-ordinary-system-unfreeze 2022-09-05 17:10:59 +03:00
Alexey Milovidov
ad19aa303b Merge branch 'add-missing-tests-geobase' of github.com:ClickHouse/ClickHouse into add-missing-tests-geobase 2022-09-04 03:33:29 +02:00
Alexey Milovidov
0e5c94ce80 Merge branch 'master' into add-missing-tests-geobase 2022-09-04 03:32:29 +02:00
Alexey Milovidov
13a129bee7 Merge branch 'master' into remove-useless-line 2022-09-04 03:24:40 +02:00
Antonio Andelic
a326450ddf Install enable_keeper_map config 2022-09-02 13:12:02 +00:00
Antonio Andelic
41b431c8b4 Enable keeper_map in tests 2022-09-02 07:41:25 +00:00
Sergei Trifonov
7fec55eea4 work in progress 2022-09-02 04:12:05 +02:00
Alexander Tokmakov
8bdb589c2b Merge branch 'master' into zookeeper_client_fault_injection 2022-08-29 13:34:57 +02:00
Alexey Milovidov
f462160bd1 Merge remote-tracking branch 'origin/master' into remove-useless-line 2022-08-28 21:00:29 +02:00
alesapin
bcc8106182 Merge branch 'master' into enable_zero_copy_replication_in_ci 2022-08-28 17:04:08 +02:00
alesapin
0b827207db Enable zero-copy replication in CI 2022-08-28 15:46:11 +02:00
Alexey Milovidov
93605bd88a
Update install.sh 2022-08-28 04:56:35 +03:00
Alexey Milovidov
b42d26acfe Remove one line from XML, because we do not care 2022-08-28 02:44:02 +02:00
Alexey Milovidov
632867dbff Fix error 2022-08-28 02:40:43 +02:00
Alexey Milovidov
a66710ca9d
Update legacy_geobase.xml 2022-08-28 03:36:02 +03:00
Alexey Milovidov
b14bd43590 Fix error 2022-08-27 20:25:06 +02:00
Alexey Milovidov
ef191efdc8 Merge branch 'nlp-functional-test' of github.com:ClickHouse/ClickHouse into nlp-functional-test 2022-08-27 20:23:31 +02:00
Alexey Milovidov
b70373d429 Fix error 2022-08-27 20:23:15 +02:00
Alexey Milovidov
66bee79d8b
Update install.sh 2022-08-27 02:56:56 +03:00
Alexey Milovidov
57a984b6a1
Update install.sh 2022-08-27 02:54:45 +03:00
Alexey Milovidov
86fc10af66 Add missing file 2022-08-26 23:42:36 +02:00
Alexey Milovidov
6414a01987 Rewrite NLP tests from integration to functional 2022-08-26 22:36:34 +02:00
Alexey Milovidov
77e15a24d0 Add missing tests for legacy geobase 2022-08-26 22:17:49 +02:00
alesapin
07286dd042 Add paranoid option to check blobs to S3 after upload 2022-08-23 15:12:09 +02:00
Vadim Volodin
f140cedd88 fix xml 2022-08-19 10:41:19 +03:00
Vadim Volodin
4c60921da4 Move config 2022-08-19 08:59:13 +03:00
Alexander Tokmakov
3d253ec51b
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-18 21:23:50 +03:00
kssenii
d63d34f7bb Merge master 2022-08-17 15:49:32 +02:00
Alexander Tokmakov
ae000e9125
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-17 12:48:54 +03:00
Alexander Tokmakov
832d7c6e7c Merge branch 'master' into replicated_database_improvements 2022-08-16 15:49:13 +02:00
kssenii
52832ea788 May be fix database replicated check 2022-08-16 11:51:57 +02:00
kssenii
eb26b219b9 Merge master 2022-08-16 00:56:27 +02:00
Alexander Tokmakov
a9c6a883dc
Merge pull request #40158 from ClickHouse/make_tests_less_flaky
Set sync_request_timeout to 10 to avoid reconnections in tests
2022-08-12 22:08:53 +03:00
Alexander Tokmakov
586682e116
Update tests/config/client_config.xml
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-08-12 22:01:45 +03:00
Alexander Tokmakov
46aaf842f8
Merge pull request #40031 from ClickHouse/fix_old_tmp_dirs_cleanup
Fix old temporary directories cleanup
2022-08-12 20:31:20 +03:00
Alexander Tokmakov
0395dca7be set sync_request_timeout to 10 to avoid reconnections in tests 2022-08-12 15:30:46 +02:00
kssenii
768a207176 Merge master 2022-08-11 22:25:53 +02:00
Alexander Tokmakov
b9d18182f2 fix 2022-08-11 15:27:26 +02:00
kssenii
3d61baff72 Fix checks 2022-08-10 21:09:18 +02:00
kssenii
0ff5609738 Finish support for persistent files 2022-08-10 16:54:59 +02:00
Alexander Tokmakov
c0763050a7 more aggressive tests 2022-08-10 16:01:09 +02:00
Alexander Tokmakov
fff903ee81 fix 2022-08-10 15:48:56 +02:00
Alexander Tokmakov
fe572104aa fix old tmp dirs cleanup 2022-08-09 18:44:51 +02:00
Alexander Tokmakov
2c5da1466d Merge branch 'master' into replicated_database_improvements 2022-08-08 16:08:27 +02:00