Commit Graph

62 Commits

Author SHA1 Message Date
Azat Khuzhin
2a12fb4246 Initialize weight/slot_to_shards for cluster not from xml correcty
This is:
- clusterAllReplicas
- copier
- some distributed cases

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-06-28 19:07:14 +02:00
Azat Khuzhin
c9adfe1efd Prohibit cluster with zero weight across all shards
Before it leads to SIGSEGV, due to either divizion by zero or an
a check in libdivide.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-06-28 19:07:14 +02:00
serxa
2df0e0c669 Unify priorities for connections 2023-06-07 16:25:52 +00:00
serxa
35e77f8e2a fix comment 2023-05-19 17:53:22 +00:00
Alexander Tokmakov
9ec91acef3 add some columns to system.clusters 2023-04-07 18:26:23 +02:00
Aleksei Filatov
0ac9dcd723 Add allow_distributed_ddl_queries option to the cluster config 2023-03-29 18:15:46 +03:00
Antonio Andelic
2ae0bceb2b Address PR comments 2023-03-07 10:56:11 +00:00
Antonio Andelic
737cf8e149 Better 2023-03-03 15:14:49 +00:00
Antonio Andelic
95853af459 Merge branch 'master' into custom-key-parallel-replicas 2023-01-24 10:49:40 +00:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
Antonio Andelic
6394a004fb Refactor 2023-01-19 14:47:58 +00:00
Antonio Andelic
d8adff35c3 Fix style 2023-01-19 12:08:32 +00:00
Antonio Andelic
53b53a1ec9 Add replcia shuffling 2023-01-19 12:04:07 +00:00
Antonio Andelic
7a75144ce3 Refactor 2023-01-19 09:42:54 +00:00
Antonio Andelic
fbe4808b6b Add support for custom key in parallel replicas 2023-01-10 13:13:43 +00:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Yakov Olkhovskiy
2e34b384c1 update tcp protocol, add quota_key 2022-08-03 15:44:08 -04:00
Vitaly Baranov
01aaaf7395 More accurate access checking for RESTORE. 2022-06-19 11:26:41 +02:00
Nikita Mikhaylov
d34e051c69
Support for simultaneous read from local and remote parallel replica (#37204) 2022-06-02 11:46:33 +02:00
Kruglov Pavel
af3db5a992
Merge pull request #36726 from PolyProgrammist/s3clustermacro
Handle cluster macro in s3Cluster table function
2022-05-11 18:00:57 +02:00
Vitaly Baranov
2c754f44fc Make calculation of shard_num & replica_num not dependant on match of the cluster's definitions on nodes. 2022-05-03 11:01:44 +02:00
Vadim Volodin
233f0b4b52 Handle cluster macro in s3Cluster table function 2022-04-28 18:38:17 +03:00
Nikita Mikhaylov
64f79f0c69
Added an ability to specify cluster secret in replicated database (#35333) 2022-03-25 00:14:26 +01:00
Alexey Milovidov
522c1c1397 Allow to use default port in cluster configuration 2022-02-20 19:27:14 +01:00
Maksim Kita
5ef83deaa6 Update sort to pdqsort 2022-01-30 19:49:48 +00:00
cmsxbc
c28fcd5a2f
support <secure/> for cluster 2021-12-30 23:51:05 +08:00
vdimir
f47ce534f2
preserve auto clusters on config update 2021-12-27 15:26:10 +03:00
vdimir
ceec643f90
update config structure: remote_servers_discovery -> remote_servers.discovery 2021-12-27 15:26:09 +03:00
vdimir
94bb7cba62
cluster discovery wip 2021-12-27 15:26:05 +03:00
Mike Kot
65e6e211b4 Merge remote-tracking branch 'upstream/master' into improvement/fn-traits 2021-10-02 21:37:50 +02:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Mike Kot
7670ea50c1 Replacing std::function to Fn in some places
to avoid dynamic allocating while keeping desired function signature
clear.
Simplifying SimpleCache (CachedFn) using C++20
2021-09-29 13:09:39 +02:00
Alexander Kuzmenkov
ece7d000dd
Merge pull request #26736 from Algunenano/ch_local_remote_localhost
CH local: Treat localhost:port as a remote database
2021-07-27 15:30:54 +03:00
Raúl Marín
383c982715 CH local: Treat localhost:port as a remote database 2021-07-23 13:16:35 +02:00
Amos Bird
3f05211b09
Connection pool factory. 2021-07-15 00:01:45 +08:00
Azat Khuzhin
e148ef739d Drop replicas from dirname for internal_replication=true
Under use_compact_format_in_distributed_parts_names=1 and
internal_replication=true the server encodes all replicas for the
directory name for async INSERT into Distributed, and the directory name
looks like:

    shard1_replica1,shard1_replica2,shard3_replica3

This is required for creating connections (to specific replicas only),
but in case of internal_replication=true, this can be avoided, since
this path will always includes all replicas.

This patch replaces all replicas with "_all_replicas" marker.

Note, that initial problem was that this path may overflow the NAME_MAX
if you will have more then 15 replicas, and the server will fail to
create the directory.

Also note, that changed directory name should not be a problem, since:
- empty directories will be removed since #16729
- and replicas encoded in the directory name is also supported anyway.
2021-06-23 02:47:38 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Maksim Kita
0d0a14a925 Poco::Timespan reference fix 2021-04-29 19:11:20 +03:00
Alexey Milovidov
6f56c3280f Uncompress data in Distributed sends if needed 2021-04-14 00:53:39 +03:00
Alexey Milovidov
e08c8a3d2c Merge branch 'master' into protocol-compression-auto 2021-04-13 23:43:08 +03:00
Pavel Kruglov
cc65a1db97 Update paths to the model configs in config reloading 2021-04-01 13:30:05 +03:00
Alexey Milovidov
21a909bde0 Merge branch 'master' into protocol-compression-auto 2021-03-30 04:15:02 +03:00
flynn
858d1fe593
Fix table function clusterAllReplicas return wrong _shard_num (#21498)
* Fix deadlock in delete mutations when predicate contains table itself

* Fix table function clusterAllReplicas return wrong _shard_num

fix build on gcc-10

better

update test

fix

fix

fix

fix

fix

fix cluster config

Co-authored-by: alesapin <alesapin@gmail.com>
2021-03-29 23:21:08 +03:00
Alexey Milovidov
39bb7c39c0 Autodetect compression #22234 2021-03-29 04:04:52 +03:00
tavplubix
5dfd0045a6
Merge branch 'master' into distributed_ddl_improvements 2021-03-22 18:13:19 +03:00
Pavel Kruglov
58e415c7d4 Update clusters only if their configs were updated 2021-03-13 00:17:19 +03:00
Alexander Tokmakov
83b3e4e0f5 allow to use Cluster from Replicated db 2021-03-09 20:05:24 +03:00
Azat Khuzhin
8b58dba094 Drop unused HexWriteBuffer 2021-02-28 10:52:09 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
2020-12-12 17:43:10 +03:00
Azat Khuzhin
ecf10817de Fix clang-tidy checks in Cluster.cpp 2020-11-08 11:21:07 +03:00