Skip to content

Cluster Mastery: Multi-Node K3s Cluster on Raspberry Pi 5

Running a containerized infrastructure across multiple nodes provides redundancy, load-balancing, and resource scaling. While a single-node setup is great, a multi-node Kubernetes cluster is the ultimate endpoint for a serious home lab.

Using K3s (a highly optimized, lightweight Kubernetes distribution by Rancher) on Raspberry Pi 5, we can build a production-grade cluster with very low overhead.

This guide walks you through: 1. System-level prerequisites (configuring kernel cgroups). 2. Setting up the K3s Control Plane (Master Node). 3. Joining Worker Nodes (Agent Nodes). 4. Setting up ingress routing with Traefik and mapping it to a local DNS domain.

Hardware Architecture

graph TD A["Local User / Browser"] -->|Requests app.local| B["Local DNS (AdGuard/Pi-hole)"] B -->|Resolves to Virtual IP| C["K3s Traefik Ingress Controller"] C -->|Routes Traffic| D{"K3s Cluster Core"} D -->|Node 1 (Master)| E["Control Plane Pods"] D -->|Node 2 (Worker)| F["Nginx Web App Pod"] D -->|Node 3 (Worker)| G["Database Pod"] style C fill:#f72585,stroke:#b5179e,color:#fff style E fill:#4cc9f0,stroke:#4361ee,color:#000 style F fill:#4cc9f0,stroke:#4361ee,color:#000 style G fill:#4cc9f0,stroke:#4361ee,color:#000

Step 1: System Configuration (Execute on ALL Nodes)

Kubernetes requires kernel-level resource control mechanisms (cgroups) to restrict CPU and memory allocation for individual containers. Raspberry Pi OS disables these by default.

1. Enable Cgroups in the Boot Command Line

On every Raspberry Pi that will be part of the cluster, edit the boot command-line parameters: 

1
sudo nano /boot/firmware/cmdline.txt

Append the following parameters to the end of the existing single line (do not create a new line!): 

1
cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1

2. Disable Swap

Kubernetes schedulers perform optimally when swap space is disabled entirely, avoiding performance degradation when nodes run out of memory. 

1
2
3
sudo dphys-swapfile swapoff
sudo dphys-swapfile uninstall
sudo systemctl disable dphys-swapfile
Note: Reboot all nodes after configuring cgroups and swap. 
1
sudo reboot

Step 2: Set Up the Control Plane (Master Node)

Select your primary Raspberry Pi 5 to act as the master node. This node will host the Kubernetes control plane API database (kine/SQLite) and scheduler.

1. Install K3s Control Plane

Run the K3s installation script. We will tell it to install with Traefik enabled, but we will bind the local storage resolver: 

1
2
3
curl -sfL https://get.k3s.io | sh -s - \
  --write-kubeconfig-mode 644 \
  --disable servicelb
Note: We disable servicelb because we will routing traffic using Traefik and host ports, or you can later add MetalLB for actual IP management.

2. Retrieve the Cluster Token and Node IP

Worker nodes require a secure token generated by the master node to join the cluster.

Print the token: 

1
sudo cat /var/lib/rancher/k3s/server/node-token
(Copy this token value, e.g., K10a7216a...::server:a45e...)

Find the IP address of the master node (e.g., 192.168.1.100): 

1
hostname -I

Step 3: Join the Worker Nodes (Agent Nodes)

SSH into your second (and third) Raspberry Pi 5 and execute the join command.

Replace <MASTER_IP> with your master node's IP, and <NODE_TOKEN> with the token value you copied in the previous step: 

1
curl -sfL https://get.k3s.io | K3S_URL=https://<MASTER_IP>:6443 K3S_TOKEN=<NODE_TOKEN> sh -

Verify Cluster Status

Go back to the master node and verify that all nodes have successfully joined and are in the Ready state: 

1
kubectl get nodes
Expected Output: 
1
2
3
4
NAME           STATUS   ROLES                  AGE     VERSION
pi5-master     Ready    control-plane,master   5m22s   v1.28.2+k3s1
pi5-worker1    Ready    <none>                 2m10s   v1.28.2+k3s1
pi5-worker2    Ready    <none>                 1m45s   v1.28.2+k3s1

Step 4: Deploying a Test Application with Traefik Ingress

Now that the cluster is healthy, let's deploy a simple web application, expose it via a Kubernetes Service, and configure Traefik Ingress to route requests to it based on a custom local domain: webapp.local.

Create a deployment manifest: 

1
nano test-app.yaml

Add the following Kubernetes configuration containing a Deployment, a Service, and an Ingress: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp-deployment
  labels:
    app: webapp
spec:
  replicas: 2
  selector:
    matchLabels:
      app: webapp
  template:
    metadata:
      labels:
        app: webapp
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: webapp-service
spec:
  selector:
    app: webapp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: webapp-ingress
  annotations:
    kubernetes.io/ingress.class: "traefik"
spec:
  rules:
  - host: webapp.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: webapp-service
            port:
              number: 80

Deploy the application: 

1
kubectl apply -f test-app.yaml

Check the status of the pods: 

1
kubectl get pods -o wide
(You will notice the replicas are distributed across the worker nodes automatically by the Kubernetes scheduler).

Step 5: Local DNS Mapping

To access http://webapp.local from your local devices:

Option A: Local DNS Server (AdGuard Home / Pi-hole)

If you run a local DNS server (e.g., our AdGuard Home Guide): 1. Navigate to the DNS server dashboard. 2. Go to Filters -> DNS Rewrites. 3. Map webapp.local to the IP address of any node in your K3s cluster (Traefik listens on host port 80 across all nodes and automatically routes the traffic inside the cluster).

Option B: Local Hosts File

On your desktop machine, edit your /etc/hosts (macOS/Linux) or C:\Windows\System32\drivers\etc\hosts (Windows): 

1
<ANY_K3s_NODE_IP>  webapp.local

Open your browser and load http://webapp.local. You will see the default Nginx welcome page, served directly from your Raspberry Pi 5 Kubernetes cluster!

By implementing this configuration, you have built a powerful, resilient multi-node K3s cluster on Raspberry Pi 5, ready for production homelab application deployment.