There are three types of cluster network configurations in VMmanager: "Switching", "Routing" and "IP-fabric". They differ in the order in which IP addresses are assigned to virtual machines and in the network topology.
"Switching" configuration type
A pool is allocated for the cluster, combining blocks of IP addresses from physical networks. You can manage IP addresses of this pool in the Networks section. Migration of virtual machines (VMs) between cluster nodes is available for this type of IP address assigning.
Example of network settings
A pool of IP addresses 192.168.10.0/24 is allocated for the cluster. VMmanager assigns addresses from this pool for VMs on cluster nodes.
"Routing" configuration type
Use this configuration type if the cluster nodes are located in Hetzner or OVH data centers.
A cluster node in a data center is assigned the primary IP address and a subnet of additional IP addresses. Virtual machines on this node are allocated only the IP addresses from the specified subnet. When connecting a node to the cluster, a range of allowable IP addresses for the virtual machine is specified. The newly created virtual machine on the node is assigned the first available IP address from the range. You can manage IP addresses in the node dashboard: Nodes → node name → IP addresses .
VM routing is performed through the virtual interfaces vnet and the interface of the network bridge vmbr0. This subnet routing type does not allow migrating virtual machines to other cluster nodes.
The Port Security function is enabled on the data center equipment. This function prevents from sending Ethernet frames if the sender MAC-address is not specified as allowed. Since a KVM virtual machine has a separate MAC-address, frames from the virtual machine will be blocked on the network equipment. To avoid this, VMmanager modifies network settings on the node so that the MAC-address in the VM frames will be changed into the MAC-address of the cluster node network interface.
Example of network settings
The server is assigned the primary IP address 220.127.116.11 and the subnet 203.0.113.40/29. As the subnet IP addresses will be used as additional, the virtual machine can be allocated all the addresses from this subnet: 203.0.113.40, 203.0.113.41, ..., 203.0.113.47. The primary IP address of the server on the network interface eno1 is used as the subnet gateway.
How to configure IPv6 on cluster nodes in the Hetzner data center
To configure IPv6 routing, add IPv6 address to the cluster node: Nodes → node name → IP addresses → IPv6 → Add an IP address . For correct operation of IPv6 network, specify the address ::2 and prefix length 64. Example: 2a01:4f8:13b:9a7::2/64. VMmanager will configure the network in the node automatically.
"IP-fabric" configuration type
The "IP-fabric" configuration type allows you to create virtual machines with public IP addresses in a private network. A pool is allocated for the cluster, combining blocks of IP addresses from physical networks. You can manage IP addresses from this pool in the Networks section. Migration of VMs between cluster nodes is available for this type of configuration.
Creating an "IP-fabric" cluster is only possible on a network that supports iBGP routing. Route Reflector equipment is used to transmit information about iBGP routes. Physical or virtual routers and/or servers can be used as such equipment. When creating a cluster, add the addresses of its nodes and BGP autonomous systems numbers to the Route Reflector configuration.
This type of configuration is only possible on KVM cluster nodes running AlmaLinux 8 and LXD cluster nodes running Ubuntu 20.
When creating a cluster, you specify:
- Gateway IP address for the VM. VMmanager will assign this address of the gateway to all virtual interfaces on the cluster node and use it only to route the VM inside the node. As IP address you can specify any address that will not match with those used in the cluster node.
- BGP community.
- IP addresses and numbers of autonomous systems of Route Reflector equipment.
When receiving the task to create a VM in the cluster "IP-fabric", VMmanager will:
Create <vmname>_net0 virtual interface on the cluster node.Comments
<vmname> — VM name. For example, for a VM with the name vm12, the interface will be called vm12_net0.
Create a VM with an IP address from the dedicated pool and the network mask of 255.255.255.255 (/32).
Set <vmname>_net0 virtual interface as gateway for the VM. Routing between the VM and the gateway is performed through a point-to-point connection (PtP).
Transmit the information about the created route to the Route Reflector equipment through the bird service.
Example of network configuration
The pool of public IP addresses 198.51.100.0/24 is allocated for the cluster. VMmanager assigns addresses from this pool to VMs on cluster nodes. The names of created VMs are vm1, vm2, vm3, etc. The interfaces vm1_net0, vm2_net0, vm3_net0, etc with IP addresses 10.0.0.1 are used as gateway. The external interfaces of the cluster nodes have private IP addresses in the 172.16.0.0/24 network.
routes — transmission of information about network routes
Node 1, Node 2 — cluster nodes
vm1, vm2, vm3... — virtual machines
vm1_net0, vm2_net0... — virtual network interfaces
data center core — equipment that provides access to the external network (Internet)
bird — service on the cluster node that implements the iBGP protocol
CORE — LInux subsystem core
route reflector — server or network equipment that receives route information from nodes and transmits it to data center core
backup route reflector — backup set of Route Reflector equipment