1 - Retiring kube-proxy IPVS on MicroK8s: An Upgrade Readiness Runbook
A controlled pre-upgrade runbook for MicroK8s clusters that still use kube-proxy IPVS mode: inventory, migrate to iptables, verify Service traffic, and retain a tested rollback path.
A focused runbook for checking whether a MicroK8s cluster needs an IPVS migration before its next Kubernetes minor upgrade.
Scope and outcome
This is not a general CNI replacement guide and it does not change Traefik, IngressRoutes, or application manifests. It covers one question: does this cluster run kube-proxy in ipvs mode, and if so, can it move safely to iptables before the next MicroK8s minor upgrade?
Kubernetes has deprecated kube-proxy IPVS mode. The upstream project expects it to be disabled by default by v1.40 and removed by v1.43. It also notes that IPVS mode still relies on iptables for part of Kubernetes Service behaviour, so it is not a complete replacement for iptables rules.[1]
If the audit finds mode: iptables, leave the proxy mode alone and record the clean result. A migration is only needed when the active kube-proxy configuration explicitly selects ipvs.
Change boundary: run the inventory and verification commands first. Perform the mode switch only in an approved maintenance window, with a tested recovery decision. The examples use placeholders and must be adapted to the cluster under review.
Why this change deserves a Service-focused test plan
Unless a cluster has deliberately installed an alternative, every node runs kube-proxy. It watches Services and EndpointSlices, then programs the node to capture a Service clusterIP and redirect traffic to an endpoint.[2]
That makes this a node-level Service-forwarding change. A successful Traefik dashboard or a healthy Deployment does not prove that ClusterIP, cross-namespace traffic, endpoint churn, or NodePorts still work after kube-proxy reconciles rules.
1. Capture the current state
Start with a read-only evidence bundle. Save command output with the maintenance record; do not copy private addresses, tokens, or real application hostnames into a public runbook.
# Run on the MicroK8s host.sudo microk8s status --wait-ready
sudo snap list microk8s
sudo snap info microk8s
# Confirm the active kube-proxy mode.sudo microk8s kubectl -n kube-system get configmap kube-proxy \
-o jsonpath='{.data.config\.conf}'| grep -E '^mode:'# Capture the full effective config for the change record.sudo microk8s kubectl -n kube-system get configmap kube-proxy \
-o yaml > kube-proxy-configmap.before.yaml
# Record kube-proxy placement, logs, Services, and EndpointSlices.sudo microk8s kubectl -n kube-system get daemonset kube-proxy -o wide
sudo microk8s kubectl -n kube-system get pods -l k8s-app=kube-proxy -o wide
sudo microk8s kubectl -n kube-system logs daemonset/kube-proxy --tail=200sudo microk8s kubectl get svc -A -o wide
sudo microk8s kubectl get endpointslice -A
On an IPVS-based node, also capture the kernel-side evidence. These commands are diagnostic only; lack of ipvsadm does not prove the cluster is not using IPVS.
Before touching kube-proxy, identify the resources whose traffic depends on it:
# Service types and their selectors.sudo microk8s kubectl get svc -A \
-o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,TYPE:.spec.type,CLUSTER-IP:.spec.clusterIP,PORTS:.spec.ports[*].port'# Workloads and the CNI configuration currently in use.sudo microk8s kubectl get deploy,daemonset,statefulset -A
sudo microk8s kubectl -n kube-system get configmap -o name
sudo microk8s kubectl get crd
Document separately:
Services of type ClusterIP, NodePort, and LoadBalancer.
Any MetalLB, external load balancer, or NodePort consumer.
CNI customisations and NetworkPolicies.
Ingress controllers and one representative route per controller.
Stateful workloads, PVCs, and recovery requirements.
MicroK8s uses snap channels: patch releases stay within the installed channel, while a minor upgrade is an intentional refresh to another channel. Its documentation also warns that add-ons and the CNI are not upgraded automatically with the cluster and need independent release-note review.[3]
2. Establish recovery controls
Do not make the proxy mode change during a general application deployment. Use a maintenance window and decide in advance what restores service if verification fails.
Minimum controls:
Confirm recent workload/PVC backups and a restore test appropriate to the applications involved.
Save kube-proxy-configmap.before.yaml, current Service and EndpointSlice listings, and the node rule snapshots.
Record the current MicroK8s channel and whether a snap refresh is held.
Name an operator and a rollback decision point.
Define representative Service probes before the maintenance starts.
For a single-node cluster, a practical rollback decision is: if DNS, a same-namespace ClusterIP, a cross-namespace ClusterIP, or an externally used NodePort fails after the normal reconciliation window, restore the saved kube-proxy ConfigMap and restart the DaemonSet before investigating unrelated add-ons.
3. Prove the tests before the change
Create a temporary, disposable probe namespace. It gives the runbook a repeatable DNS and Service test without depending on an application-specific image.
After the approved apply, verify endpoints and in-cluster access:
sudo microk8s kubectl -n proxy-migration-test rollout status deploy/echo
sudo microk8s kubectl -n proxy-migration-test get svc,endpoints,endpointslice
sudo microk8s kubectl -n proxy-migration-test run dns-client \
--rm -i --restart=Never --image=busybox:1.36 -- \
sh -ec 'nslookup echo && wget -qO- http://echo'
For a cross-namespace check, run the same request from a short-lived Pod in an existing non-production namespace, targeting http://echo.proxy-migration-test.svc.cluster.local.
4. Change only the proxy mode
First inspect the stored config.conf. Do not replace the ConfigMap with a minimal hand-written one: it may contain cluster-specific settings that must remain intact.
Then update the existing ConfigMap from the reviewed configuration. The exact key should be confirmed from the inventory command; the common key is config.conf.
# Replace only the existing config.conf value after peer review.sudo microk8s kubectl -n kube-system create configmap kube-proxy \
--from-file=config.conf=./kube-proxy.config.after.yaml \
--dry-run=client -o yaml > kube-proxy-configmap.after.yaml
# Inspect the generated object before applying it.sudo microk8s kubectl apply --dry-run=server -f kube-proxy-configmap.after.yaml
sudo microk8s kubectl apply -f kube-proxy-configmap.after.yaml
# Ask each kube-proxy Pod to reload from the updated configuration.sudo microk8s kubectl -n kube-system rollout restart daemonset/kube-proxy
sudo microk8s kubectl -n kube-system rollout status daemonset/kube-proxy --timeout=5m
Important:kubectl create configmap ... -o yaml produces an object definition. Before applying it, compare it with kube-proxy-configmap.before.yaml and preserve metadata and non-config.conf data required by the installed MicroK8s version. If that comparison is not clean, stop and use the platform’s supported configuration workflow instead.
5. Verify in layers
Run every check below after kube-proxy is ready. Keep the output with the maintenance record.
Look for ready Pods, the expected mode: iptables, and no repeated proxy initialisation or rule-programming errors.
Kubernetes DNS and ClusterIP traffic
# Same namespace: DNS plus Service forwarding.sudo microk8s kubectl -n proxy-migration-test run same-ns \
--rm -i --restart=Never --image=busybox:1.36 -- \
sh -ec 'nslookup echo && wget -qO- http://echo'# Cross namespace: fully-qualified service name.sudo microk8s kubectl -n <probe-namespace> run cross-ns \
--rm -i --restart=Never --image=busybox:1.36 -- \
sh -ec 'nslookup echo.proxy-migration-test.svc.cluster.local && wget -qO- http://echo.proxy-migration-test.svc.cluster.local'
Endpoint churn, NodePort, and ingress paths
Scale the disposable echo Deployment from two replicas to one and back to two; confirm the EndpointSlice updates and requests keep succeeding.
For every NodePort consumed outside the cluster, test it from the expected LAN client and record the result.
Request one representative ingress hostname for every controller in use; verify the controller reaches its backing Service rather than only returning its own health page.
Run each application’s existing health check. A Service has no useful traffic path when it has no ready endpoints, regardless of kube-proxy mode.
Kubernetes documents that kube-proxy keeps node rules synchronized with Service and EndpointSlice state. That is why endpoint churn belongs in the verification set rather than being treated as an optional load test.[2]
6. Troubleshooting and rollback
Symptom
Check first
Controlled response
kube-proxy Pods are not ready
DaemonSet events and Pod logs
Restore the saved ConfigMap if the mode change is implicated; do not edit the CNI first.
Service DNS works but requests fail
EndpointSlices, ready endpoints, and target ports
Confirm the Service selector and endpoint readiness before treating it as a proxy failure.
Compare node rules and firewall policy with the pre-change snapshots.
Intermittent traffic after the switch
kube-proxy logs, endpoint churn, stale node state
Wait for normal reconciliation once; if the symptom persists, roll back and collect node-level evidence.
Mode remains IPVS after an edit
Active ConfigMap key and DaemonSet arguments
Re-read the live ConfigMap; do not assume an edited local file was applied.
CNI or add-on failures appear during the maintenance
Component versions and release notes
Isolate the failure. MicroK8s add-ons and CNI updates are separate upgrade work, not a reason to widen this change.[3][4]
Rollback
# Restore the captured object only after confirming its namespace and contents.sudo microk8s kubectl apply -f kube-proxy-configmap.before.yaml
sudo microk8s kubectl -n kube-system rollout restart daemonset/kube-proxy
sudo microk8s kubectl -n kube-system rollout status daemonset/kube-proxy --timeout=5m
Repeat the same DNS, ClusterIP, NodePort, ingress, and application probes. Record both the failed condition and the successful rollback result before scheduling another attempt.
7. Continue with the MicroK8s minor upgrade
Only after the proxy mode is recorded as iptables and the Service checks pass should the cluster move to a new MicroK8s minor channel. Review the target channel and the release notes, then use an approved change plan to refresh MicroK8s to that channel. MicroK8s explicitly separates the snap-channel upgrade from add-on and CNI lifecycle work, so assess those components before and after the refresh rather than assuming they update automatically.[3][4]
2 - MicroK8s with Traefik: Public and Private Application Deployment
Complete tutorial for setting up a MicroK8s cluster with Traefik ingress controller, exposing public apps via Cloudflare and private apps locally. Includes security hardening with IP allowlists and automatic image updates with Keel.
MicroK8s Home Lab with Traefik Ingress
Summary
This guide walks through setting up a complete MicroK8s cluster on Ubuntu with Traefik as the ingress controller. The architecture supports two types of applications: public apps exposed to the internet via Cloudflare DNS and TLS, and private apps accessible only from the local network. We’ll deploy containerized workloads running inside Kubernetes pods, proxy traffic to external hosts, and set up monitoring dashboards—all managed through declarative YAML manifests.
Key features of this setup:
Traefik v3 as the ingress controller with native Kubernetes CRD support
Cloudflare integration with origin certificates for secure public access
Namespace isolation between public and private workloads
IP allowlist middleware restricting public endpoints to Cloudflare IPs only
Keel for automatic container image updates
Kubernetes Dashboard for cluster monitoring
Architecture Overview
MicroK8s Architecture: Public and Private Application Deployment (Click to expand)
The architecture consists of:
Internet traffic flows through Cloudflare (DNS + proxy + TLS termination)
Cloudflare connects to the origin server using Cloudflare Origin Certificates
Traefik receives traffic on ports 80/443 and routes based on hostname
Public Namespace contains internet-facing applications
Fresh Ubuntu server (20.04 LTS or newer recommended)
Domain name with DNS managed by Cloudflare
Network access to ports 80/443 from the internet (for public apps)
Cloudflare account (free tier works)
Hardware Requirements
Resource
Minimum
Recommended
CPU
2 cores
4+ cores
RAM
4 GB
8+ GB
Storage
20 GB
50+ GB
Step 1: Install MicroK8s
MicroK8s is a lightweight, production-grade Kubernetes distribution from Canonical. It runs as a snap package on Ubuntu.
Install MicroK8s:
# Install MicroK8ssudo snap install microk8s --classic
# Add current user to microk8s group (avoids sudo for kubectl)sudo usermod -aG microk8s $USER# Apply group changesnewgrp microk8s
# Verify installationmicrok8s status --wait-ready
Enable required addons:
# DNS - Required for service discovery within the clustermicrok8s enable dns
# Storage - Required for PersistentVolumeClaimsmicrok8s enable hostpath-storage
# Metrics Server - Required for Kubernetes Dashboard resource metricsmicrok8s enable metrics-server
Verify cluster is ready:
# Check node statusmicrok8s kubectl get nodes
# Expected output:# NAME STATUS ROLES AGE VERSION# your-hostname Ready <none> 1m v1.28.x
Step 2: Prepare Cloudflare Origin Certificate
Cloudflare Origin Certificates encrypt traffic between Cloudflare and your origin server. They’re free and valid for up to 15 years.
Cloudflare occasionally updates their IP ranges. Create a script to fetch the latest:
File: scripts/update-cloudflare-ips.sh
#!/bin/bash
set -e
echo"Fetching latest Cloudflare IP ranges..."IPV4=$(curl -s https://www.cloudflare.com/ips-v4)IPV6=$(curl -s https://www.cloudflare.com/ips-v6)cat <<EOF > traefik/cloudflare-ipallowlist.yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: cloudflare-only
namespace: traefik-system
spec:
ipAllowList:
sourceRange:
EOFfor ip in $IPV4;doecho" - $ip" >> traefik/cloudflare-ipallowlist.yaml;donefor ip in $IPV6;doecho" - $ip" >> traefik/cloudflare-ipallowlist.yaml;doneecho"Updated traefik/cloudflare-ipallowlist.yaml"echo"Apply with: microk8s kubectl apply -f traefik/cloudflare-ipallowlist.yaml"
Step 9: Automatic Image Updates with Keel
Keel monitors container registries and automatically updates deployments when new images are pushed.
File: apps/private/keel.yaml
---apiVersion:v1kind:Namespacemetadata:name:keel---apiVersion:v1kind:ServiceAccountmetadata:name:keelnamespace:keel---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRolemetadata:name:keelrules:- apiGroups:["","apps","extensions","batch"]resources:["deployments","daemonsets","statefulsets","pods","services","secrets","configmaps"]verbs:["get","list","watch","update","patch"]---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRoleBindingmetadata:name:keelroleRef:apiGroup:rbac.authorization.k8s.iokind:ClusterRolename:keelsubjects:- kind:ServiceAccountname:keelnamespace:keel---apiVersion:apps/v1kind:Deploymentmetadata:name:keelnamespace:keelspec:replicas:1selector:matchLabels:app:keeltemplate:metadata:labels:app:keelspec:serviceAccountName:keelcontainers:- name:keelimage:keelhq/keel:latestcommand:["/bin/keel"]env:- name:POLLvalue:"1"- name:BASIC_AUTH_USERvalue:"admin"- name:BASIC_AUTH_PASSWORDvalue:"admin"# Change in production!ports:- containerPort:9300hostPort:9300# UI accessible on host
Keel annotations for deployments:
Annotation
Description
Example
keel.sh/policy
Update policy
all, major, minor, patch
keel.sh/trigger
How to detect updates
poll, default
keel.sh/pollSchedule
Poll frequency
@every 30m, @daily
Access Keel UI:http://PRIVATE_IP:9300
Step 10: Kubernetes Dashboard
The Kubernetes Dashboard provides a web UI for cluster monitoring.
File: apps/private/kubernetes-dashboard.yaml
---apiVersion:v1kind:Namespacemetadata:name:kubernetes-dashboard---apiVersion:v1kind:ServiceAccountmetadata:name:kubernetes-dashboardnamespace:kubernetes-dashboard---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRoleBindingmetadata:name:kubernetes-dashboardroleRef:apiGroup:rbac.authorization.k8s.iokind:ClusterRolename:cluster-admin # Full access - restrict in production!subjects:- kind:ServiceAccountname:kubernetes-dashboardnamespace:kubernetes-dashboard---apiVersion:v1kind:Secretmetadata:name:kubernetes-dashboard-tokennamespace:kubernetes-dashboardannotations:kubernetes.io/service-account.name:kubernetes-dashboardtype:kubernetes.io/service-account-token---apiVersion:apps/v1kind:Deploymentmetadata:name:kubernetes-dashboardnamespace:kubernetes-dashboardspec:replicas:1selector:matchLabels:app:kubernetes-dashboardtemplate:metadata:labels:app:kubernetes-dashboardspec:serviceAccountName:kubernetes-dashboardcontainers:- name:kubernetes-dashboardimage:kubernetesui/dashboard:v2.7.0ports:- containerPort:8443hostPort:9443# Dashboard on host port 9443args:- --auto-generate-certificates- --namespace=kubernetes-dashboard
Note: This setup provides a solid foundation for running containerized workloads on a single node. For high availability, consider multi-node MicroK8s clusters or managed Kubernetes services.