Upgrading a k3s Cluster from v1.34.6 to v1.35.3.

I recently upgraded my homelab k3s cluster from v1.34.6 to v1.35.3 so here is the write up that covers it.

Kubernetes only supports three minor versions at a time. When 1.37 ships, 1.34 goes end-of-life  no more patches, no more security fixes. 

Upgrades also have to be incremental. You can’t jump from 1.34 to 1.36 directly. It’s 1.34 → 1.35, then 1.35 → 1.36, and so on. One minor version at a time.

The control plane upgrades first, workers follow. During the window between CP and worker upgrades, you’ll have a mixed-version cluster — and that’s completely fine and by design. The rules about which components can be on different versions at the same time.

In k3s this matters less than in a kubeadm cluster because everything is packaged into a single binary. But the policy still applies.

Check what you’re running

kubectl get nodes

NAME            STATUS   ROLES           AGE   VERSION
k3s-cp-01       Ready    control-plane   18d   v1.34.6+k3s1
k3s-worker-01   Ready    worker          18d   v1.34.6+k3s1
k3s-worker-02   Ready    worker          18d   v1.34.6+k3s1

Back up the datastore

k3s uses SQLite by default on single control plane setups and not etcd. If you try k3s etcd-snapshot save you’ll get etcd datastore disabled. The right way to back up is a straight file copy:

sudo cp /var/lib/rancher/k3s/server/db/state.db ~/k3s-state-backup-$(date +%F).db

Also export your resource configs:

kubectl get all --all-namespaces -o yaml > ~/k8s-backup-$(date +%F).yaml

Grab your node token

You’ll need this to rejoin workers after upgrading. It lives on the control plane only:

sudo cat /var/lib/rancher/k3s/server/node-token

Save it somewhere. It does not exist on worker nodes.

The Upgrade

Step 1 — Control Plane first

SSH to the CP and run:

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.35.3+k3s1 sh -

The install script downloads the new binary, stops the service, replaces the binary, and starts it back up. In k3s there’s no separate kubeadm, no kubelet upgrade, no kubectl upgrade — it’s all one binary.

Output looks like this:

[INFO] Using v1.35.3+k3s1 as release
[INFO] Downloading hash ...sha256sum-amd64.txt
[INFO] Downloading binary ...k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Starting k3s

Once it’s back up, check the nodes:

kubectl get nodes

NAME            STATUS   ROLES           AGE   VERSION
k3s-cp-01       Ready    control-plane   18d   v1.35.3+k3s1   ← upgraded
k3s-worker-01   Ready    worker          18d   v1.34.6+k3s1   ← n-1, valid
k3s-worker-02   Ready    worker          18d   v1.34.6+k3s1   ← n-1, valid

The workers are still on 1.34 — that’s expected and valid. Pods on the workers keep running throughout. No scheduling happens during the upgrade and while the CP is restarting, but nothing crashes either. Think of it like vCenter going down briefly during patching 

Step 2 — Workers, one at a time

For each worker: drain, upgrade, uncordon. Never do both workers at the same time.

Drain

kubectl drain k3s-worker-01 --ignore-daemonsets --delete-emptydir-data

This does two things — cordons the node (marks it unschedulable) then evicts all pods off it. DaemonSet pods are skipped, they’ll stay.

node/k3s-worker-01 cordoned
evicting pod cert-manager/cert-manager-5f885f9c55-grfgc
evicting pod database/my-postgresql-0
...
node/k3s-worker-01 drained

Evicted pods reschedule onto the remaining nodes

Upgrade the worker

SSH directly to the worker and run:

sudo curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.35.3+k3s1 
  K3S_URL=https://172.16.77.10:6443 
  K3S_TOKEN=<your-node-token> 
  sh -s - agent

Two things to note here compared to the CP command:

• K3S_URL and K3S_TOKEN tell the installer this is a worker joining an existing cluster
• sh -s - agent at the end explicitly installs it as an agent, not a server

Successful output will reference k3s-agent:

[INFO] Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[INFO] systemd: Creating service file /etc/systemd/system/k3s-agent.service
[INFO] systemd: Enabling k3s-agent unit

Verify the version on the worker:

k3s --version
# k3s version v1.35.3+k3s1 (be38e884)

Uncordon

Back on your PC/CP:

kubectl uncordon k3s-worker-01

Pods reschedule back. Repeat the whole process for worker-02.

Once all three nodes are done:

kubectl get nodes -o wide

NAME            STATUS   ROLES           VERSION        CONTAINER-RUNTIME
k3s-cp-01       Ready    control-plane   v1.35.3+k3s1   containerd://2.2.2-k3s1
k3s-worker-01   Ready    worker          v1.35.3+k3s1   containerd://2.2.2-k3s1
k3s-worker-02   Ready    worker          v1.35.3+k3s1   containerd://2.2.2-k3s1

All nodes on the same version, all Ready, containerd changed to 2.2.2 as part of the k3s bundle.