Join Brandon Neill for an in-depth discussion in this video Manage routing and TCP/IP stacks, part of VMware vSphere: Configure and Manage Networking.
- [Voiceover] VMkernel Routing functions much the same way as routing in other operating systems. As each interface is created, a routing entry is created for the subnet that that interface is on, and a default route can be created for any traffic that is on a non-local subnet. When the VMkernel has to route traffic, it will look through the table for a matching subnet and then use that interface. In this example, it was use vmk1 for the traffic if it was on the 172.17.199 subnet. If there is no matching subnet, then the traffic will be sent to the default route using vmk0 in this case.
Notice that vmk2 is not in the list. vmk2 is on the same subnet as vmk1. Only one interface is going to show up in the routing table, which means the traffic destined for that subnet will go out this interface, never vmk2, even if all of the physical adapters that vmk1 is connected to fail. Routing entries are not changed dynamically based off of the physical adapter status. In addition, all those little check boxes when we create a VMkernel interface only determine what IP address is used by vCenter Server when another server wants to contact this EXSi host.
It doesn't affect the interface that this EXSi host will use for outgoing traffic. For instance, if the vMotion interface is 172.16.199.11, that will be passed off to vCenter Server. When another host attempts to contact this host for vMotion, that is the IP address that it will use. If there are two interfaces on that subnet and one of them is not selected for vMotion, there is still a possibility that an interface will be used if it is the first one created.
Routing is based off destination IP, not traffic type. Like any good rule, however, there are a couple of exceptions. The first one is iSCSI port binding. Any time we bind network adapters to the iSCSI initator, then the iSCSI initiator is going to use the multipathing plugin to determine which adapter to use, not the default routing table. The second exception is Multi-NIC vMotion. If we have more than one interface configured for vMotion, then the VMkernel is going to select which one to use.
As of vSphere 6.0, I can now create multiple TCP/IP stacks. This was first introduced in 5.5 but it wasn't available on the web client. Multiple TCP/IP stacks help solve complex routing issues. For instance, if we were communicating with a vMotion server on the 10.10.30 subnet, then the VMkernel is going to go through the routing table looking for a match and then this is the interface that it is going to use. Presumably, this is also the one we've configured for vMotion.
However, as of 6.0, we can now route vMotion traffic and this creates an issue for us. If I want to sent vMotion traffic to a subnet other than 10.10.30, then that traffic is going to fall through all of my local routes and go to the default router, which is going to use vmk0. vmk0 might not be configured with physical interfaces that are designed to support the throughput necessary for vMotion. So to resolve this issue, I can now create a new PCP/IP stack.
It'll have it's own default route and potentially it's own DNS servers if necessary. I can now move the VMkernel to interface from the default stack to the vMotion stack and now all traffic destined for vMotion is going to go through the 10.10.30 interface if it's local and it if it's routed, then it's going to go to the new default route that we've selected, which is also on the 10.10.30 interface so it's going to go out our expected interface for vMotion traffic.
I can also create additional VMkernel interfaces in each of my new TCP/IP stacks. There are three TCP/IP stacks already created in vSphere 6.0. There's the default stack, for most of our traffic. There's a vMotion stack, obviously for vMotion traffic. And there's a provisioning stack for cold migrations, snapshots, and cloning operations. Each one of these can have its own default route and it will have its own interfaces in that stack. Next, let's take a look at configuring routing in the lab.
- Overview of the vSphere networking basics
- Connecting ESXi networking components
- Creating a vSwitch
- Configuring VLANs, security, and traffic shaping
- Getting network performance data
- Configuring VMkernel networks and firewalls
- Managing VMkernel routing and TCP/IP stacks