Before I explain what Cast Off is, I like to quickly explain where the idea came from. While reading the last chapter of “Cilium: Up and Running” from O’Reilly about “Operations”, it was discussed in the book to upgrade Cilium or replace the entire cluster to roll out a new Cilium version.
-> https://www.oreilly.com/library/view/cilium-up-and/9798341622982/
Pros, cons, and pitfalls are briefly discussed. Then came the topic of Gateway API, Ingress, and layer 7 policies that depend on the Cilium Envoy DaemonSet regarding the version upgrade topic. On the same page, the tip/suggestion box caught my attention: this is solved in the enterprise version of Cilium.
Long story short, this was the point where Cast Off as an idea was born to handle a Cilium rollout in a safe manner for the Cilium agent and the Cilium Envoy DaemonSet.
Some hours and days later, with the help of GitHub Copilot on how Cast Off should look and what it should do, I have version 0.0.1 ready.
Now let us dive into Cast Off’s functionality and how to install it on a Kubernetes cluster.
Cast Off overview
Cast Off’s goal is to roll out the Cilium agent and Cilium Envoy DaemonSet in a safe manner, without disrupting your network traffic, whenever a new Cilium version rollout, Cilium Helm release, or Cilium ConfigMap change is detected.
The approach of Cast Off is straightforward: iterate through all the nodes within the Kubernetes cluster by draining the node, restarting the Cilium agent and Cilium Envoy pods on the node, waiting for the new ones to be healthy, uncordoning the node, and repeating it till all nodes have been processed.
We will discuss the details in depth shortly.
However, this requires a specific Cilium configuration to work correctly, as Cast Off will handle the mechanisms internally.
envoy:
rollOutPods: false
updateStrategy:
type: OnDelete
rollOutCiliumPods: false
updateStrategy:
type: OnDelete
As seen above, the update strategy must be changed for both DaemonSets to ensure that Kubernetes does nothing when changes to the DaemonSets are rolled out, like a new Cilium version.
-> https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/#daemonset-update-strategy
The other one instructs Cilium not to restart the DaemonSets when configuration changes happen via a Helm release.
Furthermore, you need to change your Helm release configuration for Cilium by either dismissing the –wait parameter or having a long timeout configured. Why? Because the change to the update strategy will not result in a healthy Helm release till Cast Off has done its job.
How Cast Off works
Let us now dive a bit deeper into Cast Off’s functionality. First, we look at the table below to see what scenarios are covered by Cast Off.
| Scenario | Cilium agent | Cilium Envoy |
|---|---|---|
| New Cilium version | ✅ | ✅ |
| Config change for Cilium agent or Cilium Envoy without ConfigMap changes | ✅ | ✅ |
| ConfigMap changes for Cilium agent or Cilium Envoy | ✅ | ✅ |
Cast Off runs as a Kubernetes deployment with two replicas and uses a ConfigMap to store its state. During startup, a leader is elected; the leader then executes Cast Off’s routines. After the leader election, the ConfigMap is created and will contain the following key:value pairs.
| ConfigMap Key | ConfigMap Value |
|---|---|
| cast_off | “true” |
| cilium_config_hash | <CALCULATED_HASH> |
| cilium_envoy_config_hash | <CALCULATED_HASH> |
| config_map_changed_detected | “false” |
| deployed_release_version | “1” |
| deployed_version | 1.95.5 |
| helm_release_detected | “false” |
| nodes_to_update | ‘[]’ |
| update_run_id | “” |
Cast Off then checks the Cilium Helm release every 5 minutes, when not configured otherwise, for changes. Once a change has been detected, the key:value pairs are updated accordingly; especially cast_off is set to false, indicating that the update process needs to run. Besides that, the current nodes in the Kubernetes cluster are queried and stored in the nodes_to_update array.
A node only gets removed from the array once the Cilium agent and Cilium Envoy pods have been updated successfully.
When the nodes_to_update array is empty, certain key:value pairs are reset, and cast_off is set to true, meaning the Cilium changes have been rolled out to the Kubernetes cluster.
During the update stage, it happens that the Cast Off leader gets evicted. Then a new leader is elected and starts the initialization process. As the ConfigMap already exists, the new leader detects that cast_off is set to false and resumes the ongoing update stage after a 10-second waiting period.
For alerting or log comparison, the update_run_id is provided. Other than that, Cast Off returns to its constant operational state, checking every 5 minutes the Cilium Helm release for changes.
Install Cast Off on Azure Kubernetes Service – BYOCNI
In this example, we install Cast Off on an Azure Kubernetes Service cluster with Cilium in BYOCNI. The installed Cilium version is 1.19.5, and we update Cilium after Cast Off’s installation to version 1.19.6 to observe the safe Cilium rollout process by Cast Off.
Cast Off provides a Helm Chart and can be installed using the following commands.
❯ helm upgrade --install cast-off oci://quay.io/neumanndaniel/cast-off --version 0.0.1 \
--namespace cast-off \
--create-namespace
Afterwards, we check the Cast Off logs and should see a successful initialization.
❯ kubectl get -n cast-off leases.coordination.k8s.io NAME HOLDER AGE cast-off-leader-election cast-off-7cb5f67df9-b7lc4 2m50s ❯ kubectl logs -n cast-off cast-off-7cb5f67df9-b7lc4 I0729 19:57:24.495590 1 leaderelection.go:258] "Attempting to acquire leader lease..." lock="cast-off/cast-off-leader-election" I0729 19:57:24.534534 1 leaderelection.go:272] "Successfully acquired lease" lock="cast-off/cast-off-leader-election" time=2026-07-29T19:57:24.534Z level=INFO msg="leader unchanged" identity=cast-off-7cb5f67df9-b7lc4 time=2026-07-29T19:57:24.534Z level=INFO msg="leader acquired" time=2026-07-29T19:57:24.534Z level=INFO msg="initialization stage started" time=2026-07-29T19:57:25.111Z level=INFO msg="created state configmap" deployed_version=1.19.4 deployed_release_revision=5 cilium_config_hash=c47b538d9e9092af92cad9b94932704af5cdc5050df9cae337333ee0577f23a4 cilium_envoy_config_hash=db2e931644e488331146dbcb22b827c9cc8f886f7e54b734680ead2eb16fdd8e time=2026-07-29T19:57:47.832Z level=INFO msg="controller shutdown requested"
Now we update Cilium to version 1.19.6, and we should see, after Cast Off’s next check interval, how the safe rollout is kicked off.
Once Cast Off finished its job, we ran cilium status to confirm the successful Cilium version update.
Summary
As this is the initial release of Cast Off and my little side project, let me know if you find it useful or what else is missing from the already outlined roadmap items.
You can find the entire project on GitHub under my username.