Safely Upgrade Kubeadm Kubernetes Cluster: A Step-by-Step Guide

Last Updated:
Safely Upgrade Kubeadm Kubernetes Cluster: A Step-by-Step Guide

In this guide, we’ll walk you through the step-by-step process of how to safely upgrade kubeadm Kubernetes cluster. Upgrading a Kubernetes cluster that was originally deployed using kubeadm requires careful planning and execution to ensure minimal downtime and smooth operation.

Upgrading Kubeadm Kubernetes Cluster

Important Cluster Upgrade Considerations

Before you even begin to think about the kubeadm Kubernetes cluster upgrade, there are important considerations you need to look into. Some of these are:

  • Cluster Backup: Backup is Crucial. Murphy is always around the corner and you never know what might go wrong with the upgrade. You need to be able to recover your Kubernetes cluster data in case of any unforeseen circumstances. This is where backups become crucial. You can check how to backup and restore etcd in a Kubernetes cluster.
  • Review Kubernetes Release Notes: Always review the official Kubernetes release notes for the version you’re targeting. This can help identify potential upgrade issues or breaking changes specific to that release.
  • Static Control Plane Required: kubeadm upgrades only work with clusters using a static control plane and etcd pods or external etcd. If your setup differs, you might need to explore alternative upgrade methods.
  • Disable Swap: Ensure swap is disabled on all cluster nodes. Check how to disable swap on Kubernetes nodes.
  • Upgrade path: It is recommended to upgrade from a minor version to a next higher one (e.g., from 1.29.0 to 1.30.0), or from a patch release version to a higher one (e.g., from 1.30.1 to 1.30.2).
  • It is recommended to match kubelet and kubeadm versions. However, in some cases, it is okay to use a kubelet that is three versions older than kubeadm.

Safely Upgrade Kubeadm Kubernetes Cluster

Kubernetes releases regular updates with new features, bug fixes, and security patches. Upgrading your cluster ensures you can leverage these improvements and maintain a secure environment for your applications.

What is the Order of Kubernetes Nodes Upgrade?

You need to start upgrading the control plane node before upgrading the worker nodes. If you have multiple control plane nodes, you need to start with the primary one, basically the one that is running as etcd cluster leader, first, then proceed with the rest of the control planes and the worker nodes in any order. This is summarized as:

  1. Upgrade primary control plane node
  2. Upgrade other control plane nodes
  3. Upgrade worker nodes

Determine the version of Kubernetes to upgrade to

To begin with, you need to check your current version of the cluster packages and of course the version you are upgrading to.

Current installed versions of kubeadm, kubectl, kubelet.

kubeadm version -o json
  "clientVersion": {
    "major": "1",
    "minor": "30",
    "gitVersion": "v1.30.1",
    "gitCommit": "6911225c3f747e1cd9d109c305436d08b668f086",
    "gitTreeState": "clean",
    "buildDate": "2024-05-14T10:49:05Z",
    "goVersion": "go1.22.2",
    "compiler": "gc",
    "platform": "linux/amd64"
kubectl version
Client Version: v1.30.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.1
kubelet --version
Kubernetes v1.30.1

Cluster nodes;

kubectl get nodes
NAME        STATUS   ROLES           AGE    VERSION
master-01   Ready    control-plane   2d1h   v1.30.1
master-02   Ready    control-plane   2d1h   v1.30.1
master-03   Ready    control-plane   2d1h   v1.30.1
worker-01   Ready    <none>          2d1h   v1.30.1
worker-02   Ready    <none>          2d1h   v1.30.1
worker-03   Ready    <none>          2d1h   v1.30.1

As you can see, we are on the release version v1.30.1 of Kubernetes. As per the release notes, the current latest release as of this writing is v1.30.2 which is one patch release version higher than our current running version.

As such, we will be upgrading our cluster to a patch release version.

Change the Kubernetes Package Repository

Chances are you are using the correct Kubernetes package repository and not the legacy ones ( and which have since deprecated. Instead, ensure you are using either of:


You can check which repository you are using;

grep -irlE "kubernetes|k8s" /etc/apt/

This will print a file containing any of the kubernetes or k8s keywords.

Sample output;


Therefore, our K8S package repository file is /etc/apt/sources.list.d/kurbenetes.list.

Get the contents of the file to confirm the repository address;

cat /etc/apt/sources.list.d/kurbenetes.list
deb /

Based on the release version we are intending to upgrade to as seen in determining the version section above, the repository we have is enough to upgrade to the patch release v1.30.2.

Also, if you are using legacy repositories, be sure to update them before you can proceed.

Drain the Nodes Before Upgrade

Draining the nodes is not really necessary. However, If you are performing a minor version upgrade for any kubelet, you must first drain the node(s) that you are upgrading.

How to Drain Kubernetes Nodes

Upgrade Kubernetes Control Plane Nodes

As stated, you need to start the upgrade with control plane nodes. If you have multiple control plane nodes, start with the primary node. I would check which one is currently the etcd cluster leader to know the primary control plane.

sudo etcdctl endpoint status --cacert=/etc/kubernetes/pki/etcd/ca.crt \
	 --cert=/etc/kubernetes/pki/etcd/server.crt \
	 --key=/etc/kubernetes/pki/etcd/server.key \
	 --cluster \
	 -w table
| | 71adc069c6babb75 |  3.5.12 |   13 MB |     false |      false |         5 |      69810 |              69810 |        |
| | 7f56434149e2cc7f |  3.5.12 |   13 MB |     false |      false |         5 |      69810 |              69810 |        |
| | db4091f30b21595e |  3.5.12 |   13 MB |      true |      false |         5 |      69810 |              69810 |        |

As you can see, the node with the IP, is the current etcd leader and hence, we will treat this as the primary control plane.

Hence, login to your primary control plane. For example;

ssh [email protected]

Now that we have confirmed that the package repository is all good, let’s check the current versions of kubeadm, kubelet and kubectl available.

apt-cache madison kubeadm kubelet kubectl

Sample output;

   kubeadm | 1.30.1-1.1 |  Packages
   kubeadm | 1.30.0-1.1 |  Packages
   kubelet | 1.30.1-1.1 |  Packages
   kubelet | 1.30.0-1.1 |  Packages
   kubectl | 1.30.1-1.1 |  Packages
   kubectl | 1.30.0-1.1 |  Packages

As you can see, the latest versions provided by the Kubernetes repositories for each of the package is v1.30.1-1.1

To get the latest release versions, run system package cache update.

sudo apt update
Hit:1 noble InRelease
Get:2 noble InRelease [48.8 kB]     
Get:3 noble-updates InRelease [126 kB]   
Hit:4 noble-backports InRelease                     
Get:5 noble-security InRelease [126 kB]
Get:6 noble-updates/main amd64 Packages [177 kB]
Get:7 noble-updates/main Translation-en [49.1 kB]                 
Get:8 noble-updates/restricted amd64 Packages [70.1 kB]     
Get:9 noble-updates/restricted Translation-en [14.3 kB]                            
Get:10 noble-updates/universe amd64 Packages [66.7 kB]                  
Get:11 noble-updates/universe Translation-en [25.1 kB]                         
Get:12 noble-updates/multiverse amd64 Packages [2,968 B]                             
Get:13 noble-updates/multiverse Translation-en [968 B]                               
Get:15 noble-security/main amd64 Packages [158 kB]                                     
Get:16 noble-security/main Translation-en [41.5 kB]
Get:17 noble-security/restricted amd64 Packages [70.1 kB]
Get:18 noble-security/restricted Translation-en [14.3 kB]               
Get:19 noble-security/universe amd64 Packages [44.4 kB]                  
Get:20 noble-security/universe Translation-en [17.0 kB]            
Get:14  InRelease [1,186 B]
Get:21  Packages [5,226 B]
Fetched 1,059 kB in 1s (1,025 kB/s)    
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
30 packages can be upgraded. Run 'apt list --upgradable' to see them.

Next, confirm the available versions;

apt-cache madison kubeadm kubelet kubectl
   kubeadm | 1.30.2-1.1 |  Packages
   kubeadm | 1.30.1-1.1 |  Packages
   kubeadm | 1.30.0-1.1 |  Packages
   kubelet | 1.30.2-1.1 |  Packages
   kubelet | 1.30.1-1.1 |  Packages
   kubelet | 1.30.0-1.1 |  Packages
   kubectl | 1.30.2-1.1 |  Packages
   kubectl | 1.30.1-1.1 |  Packages
   kubectl | 1.30.0-1.1 |  Packages

So, we now have the correct patch release version available and we ready to run the upgrades.

It is usually recommended to mark hold the Kubernetes packages to avoid automatic updates and keep a controlled version of the cluster. Check if these packages are hold;

sudo apt-mark showhold

Thus, to upgrade any hold package, unhold it;

sudo apt-mark unhold kubeadm kubelet kubectl

You are now ready to upgrade.

Upgrade kubeadm on primary control plane node;

sudo apt update
sudo apt install kubeadm

The confirm;

kubeadm version -o yaml
  buildDate: "2024-06-11T20:27:59Z"
  compiler: gc
  gitCommit: 39683505b630ff2121012f3c5b16215a1449d5ed
  gitTreeState: clean
  gitVersion: v1.30.2
  goVersion: go1.22.4
  major: "1"
  minor: "30"
  platform: linux/amd64

Upgrade the package on the other control plane nodes.

Once kubeadm package is upgraded across all control plane nodes, verify that the cluster can be upgraded and fetch the version to upgrade to on the primary control plane node;

sudo kubeadm upgrade plan
[preflight] Running pre-flight checks.
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[upgrade] Running cluster health checks
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: 1.30.1
[upgrade/versions] kubeadm version: v1.30.2
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.30.2

If the cluster is upgrade-able, then you will get the command to use to apply the upgrade.

For example;

sudo kubeadm upgrade apply v1.30.2

When prompted to confirm, do confirm and proceed. It may take a while to complete the upgrade.

After upgrade;

[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.30.2". Enjoy!

[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.

While it is not really required, but the for the sake of consistency, run the command on other control plane node just to ensure everything is upgraded.

sudo kubeadm upgrade node

Next, upgrade the kubelet and the kubectl tool to the same version on all nodes (you can start with the leader):

As mentioned before, for minor release upgrade, you would have to drain other control plane nodes before proceeding with kubelet/kubectl upgrade.

Since we are doing a patch release, just proceed.

sudo apt install kubelet kubectl

The services, kubelet, will be restarted during upgrade.

After you are done with upgrade, hold the packages on all control plane nodes.

sudo apt-mark hold kubeadm kubelet kubectl

(Again, if you had drained the other nodes, uncordon them)

Confirm the nodes versions;

kubectl get nodes
NAME        STATUS   ROLES           AGE    VERSION
master-01   Ready    control-plane   2d3h   v1.30.2
master-02   Ready    control-plane   2d2h   v1.30.2
master-03   Ready    control-plane   2d2h   v1.30.2
worker-01   Ready    <none>          2d2h   v1.30.1
worker-02   Ready    <none>          2d2h   v1.30.1
worker-03   Ready    <none>          2d2h   v1.30.1

Upgrade Worker Nodes

Next, upgrade the worker nodes one by one.

Unhold the packages on all worker nodes;

sudo apt-mark unhold kubeadm kubelet kubectl

Upgrade kubeadm;

sudo apt update;sudo apt install kubeadm

Upgrade the Kubelet configuration;

sudo kubeadm upgrade node

Sample output;

[upgrade] Reading configuration from the cluster...
[upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks
[preflight] Skipping prepull. Not a control plane node.
[upgrade] Skipping phase. Not a control plane node.
[upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config2070136212/config.yaml
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[upgrade] The configuration for this node was successfully updated!
[upgrade] Now you should go ahead and upgrade the kubelet package using your package manager.

Again the node drain condition applies here! So proceed accordingly.

Upgrade kubelet and kubectl;

sudo apt update;sudo apt install kubectl kubelet

Kubelet service will be restarted during upgrade.

If you had drained the node, it is now time to uncordon it.

Upgrade the next worker nodes using the same approach.

Mark and hold the packages;

sudo apt-mark hold kubeadm kubelet kubectl

Verify Cluster Upgrade

You can now check the status of the cluster upgrade by listing the nodes;

kubectl get nodes
NAME        STATUS   ROLES           AGE    VERSION
master-01   Ready    control-plane   2d3h   v1.30.2
master-02   Ready    control-plane   2d3h   v1.30.2
master-03   Ready    control-plane   2d3h   v1.30.2
worker-01   Ready    <none>          2d3h   v1.30.2
worker-02   Ready    <none>          2d3h   v1.30.2
worker-03   Ready    <none>          2d3h   v1.30.2

And all nodes are of the same version. Ensure that all nodes are in a ready state.

After a successful upgrade, I would recommended that you review cluster logs for any errors or warnings. Additionally, consider testing your applications to ensure they function as expected on the upgraded cluster.


Upgrading a Kubernetes cluster created with kubeadm involves several critical steps to maintain the reliability and security of your infrastructure. By following these steps, you can successfully upgrade your cluster while minimizing downtime and ensuring a smooth transition. Remember, planning, attention to detail, and following best practices are key to a seamless upgrade experience.

Read more on upgrading the cluster documentation.


We're passionate about sharing our knowledge and experiences with you through our blog. If you appreciate our efforts, consider buying us a virtual coffee. Your support keeps us motivated and enables us to continually improve, ensuring that we can provide you with the best content possible. Thank you for being a coffee-fueled champion of our work!

Photo of author
Linux Certified Engineer, with a passion for open-source technology and a strong understanding of Linux systems. With experience in system administration, troubleshooting, and automation, I am skilled in maintaining and optimizing Linux infrastructure.

Leave a Comment