What is a Kubernetes LoadBalancer On-Prem

Question

I understand that, in Cloud scenarios, a LoadBalancer resource refers to and provisions an external layer 4 load balancer. There is some proprietary integration by which the cluster configures this load balancing. OK.

On-prem we have no such integration so we create our own load balancer outside of the cluster which distributes traffic to all nodes - effectively, in our case to the ingress.

I see no need for the cluster to know anything about this external load balancer.

What is a LoadBalancer* then in an on-prem scenario? Why do I have them? Why do I need one? What does it do? What happens when I create one? What role does the LoadBalancer resource play in ingress traffic. What effect does the LoadBalancer have outside the cluster? Is a LoadBalancer just a way to get a new IP address? How/why/ where does the IP point to and where does the IP come from?

• All questions refer to the Service of type “LoadBalancer” inside cluster and not my load balancer outside the cluster of which the cluster has no knowledge.

Answer 1

As pointed out in the comments a kubernetes service of type LoadBalancer can not be used by default with on-prem setups. You can use metallb to setup a service of that type in an on prem environment.

Kubernetes does not offer an implementation of network load balancers (Services of type LoadBalancer) for bare-metal clusters. [...] If you’re not running on a supported IaaS platform (GCP, AWS, Azure…), LoadBalancers will remain in the “pending” state indefinitely when created. [...] MetalLB aims to redress this imbalance by offering a network load balancer implementation that integrates with standard network equipment, so that external services on bare-metal clusters also “just work” as much as possible.

You can for example use the BGP mode to advertise the service's IP to your router, read more on that in the docs.

The project is still in beta but is promoted as production ready and used by several bigger companies.

Edit

Regarding your question in the comments:

Can I just broadcast the MAC address of my node and manually add the IP I am broadcasting to the LoadBalancer service via kubectl edit?

Yes that would work too. That's basically what metallb does, announcing the IP and updating the service.

Why need a software then? Imaging having 500 hosts that come and go with thousends of services of type LoadBalancer that come and go. You need an automation here.

Why does Kubernetes need to know this IP?

It doesn't. If you don't use an external ip, the service is still usable via it's NodePort, see for example the istio docs (with a little more details added by me):

$ kubectl get svc istio-ingressgateway -n istio-system
NAME                   TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)
istio-ingressgateway   LoadBalancer   192.12.129.119   <Pending>   [...],80:32123/TCP,443:30994/TCP,[...]

Here the external IP is not set and stays in <Pending>. You can still use the service by pointing your traffic to <Node-IP>:32123 for plain http and to <Node-IP>:30994 for https. As you can see above those ports are mapped to 80 and 443.

If the external ip is set you can direct traffic directly to port 80 and 443 on the external load balancer. Kube-Proxy will create an iptables chain with the destination of you external ip, that basically leads from the external IP over the service ip with a load balancer configuration to a pod ip.

To investigate that set up a service of type LoadBalancer, make sure it has an external ip, connect to the host and run the iptables-save command (e.g. iptables-save | less). Search for the external ip and follow the chain until you end up at the pod.