A k8s LB using ARP⌗

Metallb now supports ARM LBs. This is great, because it allows you to create an working LB in your (shitty) home network with off-the-shelve, cheap, router hardware. Running k8s at home (on ARM), became a whole lot more interesting!

Note: this will all be different in metallb 0.3.0, but if you want to play with this now you can follow this little guide.

We follow some of steps in the tutorial.

This guide assumes a 192.168.1/24 (home) network. The load balanced IP pool is set to 192.168.1.240/28, so you’ve got 14 IP addresses to play LB with.

This assumes a cluster is up and running, i.e. minikube might work, but I haven’t used that.

Install metallb⌗

Install metallb with ARP support and get some logs from the daemonset.

% kubectl apply -f https://raw.githubusercontent.com/google/metallb/master/manifests/metallb-arp.yaml
% kubectl logs -l app=arp-speaker -n metallb-system

A log line like:

1216 09:16:17.048864       1 arp.go:199] Address found 192.168.1.109/24 for interface: {2 1500 eth0 b8:27:eb:06:2a:72 up|broadcast|multicast}

Shows the arp-speaker found the interface of the node and will use that to send and receive ARP packets.

Apply the config map with the config for ARP:

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: my-ip-space
      cidr:
      - 192.168.1.240/28

The only thing we need is the address-pools where we tell it assign from 192.168.1.240/28. Apply this configuration:

% kubectl apply -f https://raw.githubusercontent.com/google/metallb/master/manifests/arp-config.yaml

Create a load-balanced service⌗

% kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.2.0/manifests/tutorial-2.yaml
deployment "nginx" created
service "nginx" created

The arp-speaker logs should have something like this:

I1216 09:22:33.433858       1 main.go:93] default/nginx: announcable, making advertisement
I1216 09:22:33.434890       1 main.go:103] default/nginx: end update
I1216 09:22:33.775673       1 arp.go:117] Request: who-has 192.168.1.240?  tell 192.168.1.1 (b4:75:0e:63:b2:20). reply: 192.168.1.240 is-at b8:27:eb:66:21:54
I1216 09:22:33.846087       1 arp.go:117] Request: who-has 192.168.1.240?  tell 192.168.1.1 (b4:75:0e:63:b2:20). reply: 192.168.1.240 is-at b8:27:eb:66:21:54
I1216 09:22:33.985725       1 arp.go:117] Request: who-has 192.168.1.240?  tell 192.168.1.1 (b4:75:0e:63:b2:20). reply: 192.168.1.240 is-at b8:27:eb:66:21:54
I1216 09:22:34.095799       1 arp.go:117] Request: who-has 192.168.1.240?  tell 192.168.1.1 (b4:75:0e:63:b2:20). reply: 192.168.1.240 is-at b8:27:eb:66:21:54
I1216 09:22:34.215941       1 arp.go:117] Request: who-has 192.168.1.240?  tell 192.168.1.1 (b4:75:0e:63:b2:20). reply: 192.168.1.240 is-at b8:27:eb:66:21:54

And looking at the service, we indeed see an EXTERNAL-IP:

% kubectl get svc
NAME         TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)        AGE
kubernetes   ClusterIP      10.96.0.1       <none>          443/TCP        2d
nginx        LoadBalancer   10.102.30.250   192.168.1.240   80:31517/TCP   1m

From my laptop I can now curl that IP!

% curl http://192.168.1.240
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

This whole thing just feels like magic.