Upgrade a Kubernetes cluster on Creodias OpenStack Magnum
You can upgrade a Creodias OpenStack Magnum cluster to the next minor Kubernetes version when compatible templates are available.
This article shows how to upgrade a Magnum Kubernetes cluster from 1.31 to 1.32.
How the Magnum rolling upgrade works
The target cluster template specifies the Kubernetes version for the upgrade. After you select the template, Magnum upgrades the cluster nodes through its rolling upgrade mechanism. For a supported pair of consecutive Kubernetes versions, Magnum:
Upgrades the control plane nodes first.
Upgrades worker nodes one by one to minimize downtime.
Adds an extra node before upgrading each node (in addition to the nodes you specified in the cluster configuration).
Maintains API compatibility during the upgrade and thus ensures that existing workloads continue running smoothly.
It is not possible to skip minor versions when upgrading.
Prerequisites
No. 1 Hosting
You need a Creodias hosting account with Horizon interface https://horizon.cloudferro.com/auth/login/?next=/.
No. 2 Access to the cloud and cluster
Commands openstack and kubectl must be up and running:
How To Install OpenStack and Magnum Clients for Command Line Interface to Creodias Horizon
How To Access Kubernetes Cluster Post Deployment Using Kubectl On Creodias OpenStack Magnum
No. 3 Availability of upgradeable cluster templates
There is a direct correspondence between cluster templates available on Creodias and the Kubernetes versions with the identical minor version numbers:
Upgradeable Kubernetes versions |
calico cluster template |
cilium cluster template |
|---|---|---|
1.30 |
k8s-v1.30.14-1.0.0 |
k8s-v1.30.14-1.0.0-cilium |
1.31 |
k8s-v1.31.14-1.0.0 |
k8s-v1.31.14-1.0.0-cilium |
1.32 |
k8s-v1.32.13-1.0.0 |
k8s-v1.32.13-1.0.0-cilium |
Note
Upgradeable cluster templates are available on Creodias WAW4-1 region only at the moment of this writing.
The available consecutive upgrade paths are 1.30 to 1.31 and 1.31 to 1.32.
To test the waters, you can create a new cluster with the Cilium template for Kubernetes 1.31. The command may look like this:
openstack coe cluster create \
--cluster-template k8s-v1.31.14-1.0.0-cilium \
--docker-volume-size 50 \
--labels eodata_access_enabled=false,floating-ip-enabled=true \
--merge-labels \
--keypair sshkey \
--master-count 3 \
--node-count 5 \
--timeout 190 \
--master-flavor eo2a.large \
--flavor eo2a.medium \
cilium-131
No. 4 Ensuring apps compatibility with target K8s version
In this particular case, check Kubernetes 1.32 Release Notes to ensure your applications are compatible. Be sure to always check release notes for the target Kubernetes version you are upgrading the cluster to.
Backup and observe the state of the cluster before the upgrade
Before the upgrade, compare the state of the cluster
before,
during and
after the upgrade.
Ideally, everything should work right out of the box, however, it is preferable to check and verify.
Kubernetes cluster backup
It is highly recommended to make a backup of your 1.31 cluster before the update. See Backup of Kubernetes Cluster using Velero.
This backup will serve as the state “before” the update.
Monitor the upgrade process with cluster dashboard
The simplest way to observe a cluster is through cluster dashboard. See Using Dashboard To Access Kubernetes Cluster Post Deployment On Creodias OpenStack Magnum.
To see node versions, select option Nodes in the left side menu, click on a node name and scroll down a bit to find kubelet version.
Other tools for comparisons of clusters might include CI/CD tests, observing cluster with Prometheus and Grafana, storing cluster statistics in a database as a time series and so on.
Prepare the upgrade
Verify cluster version
The following command prints cluster version:
echo $(openstack coe cluster show cilium-131 | awk '/ coe_version /{print $4}')
In this article, the version before the update is 1.31.14.
Identify cluster ID
Retrieve the ID of the Kubernetes cluster you want to upgrade:
openstack coe cluster list
Record the cluster ID shown in the uuid column. In commands that follow, you can use either the cluster name or its uuid.
Trigger the upgrade
The command to initiate the upgrade is:
openstack coe cluster upgrade <cluster-id> <template-version>
For this example:
openstack coe cluster upgrade cilium-131 k8s-v1.32.13-1.0.0-cilium
Because the cilium-131 cluster uses the k8s-v1.31.14-1.0.0-cilium template, you must upgrade it with the k8s-v1.32.13-1.0.0-cilium template listed in Prerequisites No. 3.
The command should confirm that the request to upgrade the cluster has been accepted. Conversely, an invalid template name results in an HTTP 404 error.
Commands to monitor the upgrade progress
A GUI way of monitoring the upgrade process would be using the dashboard for the cluster, as mentioned above. A CLI way would be to issue the specific commands, for example:
Show only the version of the cluster
echo $(openstack coe cluster show cilium-131 | awk '/ coe_version /{print $4}')
During the upgrade, health_status field might temporarily get into UNHEALTHY status, until the upgrade stops. Regardless of that, the cluster remains responsive throughout the entire upgrade process.
Monitor the nodes
kubectl get nodes -o wide
When all of the nodes are in version 1.32.13, the upgrade is finished.
As a rule of thumb, you can assume that each node will take a few minutes to upgrade. The actual time will depend on way too many factors to list and that consideration is out of scope of this article.
Verify the upgrade
If you performed both
the backup of the old cluster and
upgraded to the next minor Kubernetes version,
you will have two clusters and be able to compare them.
Use the dashboard to compare the results “before” and “after”. Hunt down the eventual differences, learn what they mean and, if needed, resolve them.
If something is still not right, examine whether there are some breaking changes in code from version 1.31 to version 1.32.