GoKawiilOne of the capabilities customers expect from cloud-integrated Kubernetes is support for Service objects of type LoadBalancer . When a user creates one, Kubernetes asks the cloud provider’s service controller to provision the necessary infrastructure and publish its address in the Service status. There was just one problem: Oxide did not yet offer a native load balancer.
Oxide did, however, have floating IPs. Floating IPs are addresses from a rack’s external IP pools that can be attached to and detached from instances, making those instances reachable from outside their VPCs. Using floating IPs offered a way to unblock LoadBalancer services. A floating IP would deliver traffic to a single Kubernetes node, and the Kubernetes Service dataplane could distribute that traffic to the appropriate pods.
Making that work required accounting for how Oxide floating IPs appear to an instance. They are transparent to the guest in two important ways. First, Oxide translates the destination address of inbound traffic to the instance’s internal IP before sending the traffic to the instance. Second, the instance has no network interface configured with the floating IP.
The resulting traffic flow looks like this:
Traffic flow to a LoadBalancer service using floating IPs. ┌────────────────────────────────────────────────────────────┐
│ Client │
│ Request to floating IP: 45.154.216.233:80 │
└────────────────────────────────────────────────────────────┘
│
▼
... continue reading