Helm pre-upgrade runtime gate
When the chart requires every application Deployment to be stopped before an upgrade, and how to configure it.
The chart runs a pre-upgrade Job named <release>-opsknight-auth-cutover before
the migration Job. It fails the upgrade unless every application Deployment
(integrated or split roles) is at zero replicas and no application pods remain.
This keeps an older runtime from overlapping a newer database schema.
When the gate runs
upgradeGates.authenticationCutover.mode controls it:
| Mode | Gate runs |
|---|---|
auto (default) |
Only when the upgrade changes the application image |
always |
On every upgrade, including values-only changes (behaviour up to 2.1.2) |
never |
Never; you are responsible for stopping the runtime before image changes |
In auto mode the chart reads the live Deployments and compares the image of
the opsknight container with the target image. A values-only upgrade (same
image) skips the gate, so configuration changes no longer need downtime. If no
Deployment can be read, for example under helm template or --dry-run, the
gate runs.
Up to 2.1.2 the gate ran on every upgrade. Any values change then needed all Deployments at zero, and GitOps controllers such as Flux retried and rolled back the release in a loop.
Upgrade to a new image
When the image changes, the gate still runs. Stop the runtime first:
APP='app.kubernetes.io/instance=opsknight,app.kubernetes.io/component!=pgbouncer'
kubectl -n opsknight scale deployment -l "$APP" --replicas=0
kubectl -n opsknight wait pod -l "$APP" --for=delete --timeout=5m
helm upgrade opsknight deploy/kubernetes/helm/opsknight \
--namespace opsknight --values values.production.yaml \
--set image.digest=sha256:<tested digest>
PgBouncer can keep running; the gate ignores it. The runbook agent, when
enabled, is stopped with the application roles.
An image upgrade still requires an explicit image.tag or image.digest.
GitOps controllers
With auto, configuration changes reconcile without downtime. For image
upgrades, stop the runtime as part of the change, or set mode: never and run
your own stop step before the new image rolls out. Limit upgrade remediation
retries in the controller so a failed gate is reported instead of retried in a
loop.
For the full upgrade procedure see the Helm upgrade guide.
Last updated for v2.2.0
Edit this page on GitHub