How a deployment switches traffic

Understand the blue-green deployment: Tenkai starts a candidate next to the active release, checks it, then moves the routes to it.

Tenkai deploys a new version of a web application next to the one that runs, checks it, then moves the routes to it and stops the old version. This is a blue-green deployment: blue is the Active release that serves your visitors, green is the Candidate that will replace it.

The steps of a deployment

Prepare the image. Tenkai builds your Git repository on the edge node, or pulls your container image, and records its digest.
Run the pre-deploy commands, if any, in a container of the new image. A failing command stops the deployment. See run a command before a deployment or a rollback.
Start the candidate as a separate service, without a route.
Check its health. Each replica must run within a minute once its node holds the image; pulling a large image may take up to ten more minutes. A Docker HEALTHCHECK in your image must pass within that first minute. Then Tenkai calls the health check path, if you set one, and only a 2xx answer passes. Without one, Tenkai waits up to two minutes for the port of the candidate to answer HTTP with any status: that shows the server listens, not that the application works.
Switch the traffic of the default route and every custom domain to the candidate.
Retire the old version. The candidate becomes the active release. The old one scales to zero and keeps its image for a rollback.

The health check

A health check tells Tenkai when the candidate can take the traffic: Tenkai calls one path on the port of the candidate and expects 2xx. The deployment then shows Health verified, and without one Health not verified.

Add one in the Configuration tab under Health check, or with Add a health check… next to Health not verified. Choose:

the path, such as /health. Answer 2xx only when your application can serve requests, for example once it reached its database;
the timeout of one call, from 0.1 to 30 seconds;
the number of attempts, from 1 to 10, and the interval between them, up to 30 seconds;
the headers to send, such as a token. Tenkai never shows their values again: leave a value blank to keep the saved one.

An app of the marketplace comes with a health check you can change or remove.

A change takes effect at the next deployment. A rollback uses the health check its release was deployed with. Remove goes back to checking the port alone. Tenkai calls the health check only during a deployment, not afterwards.

What visitors see during the switch

The active release serves every request until the switch, and the candidate after it. Tenkai never splits traffic between the two versions.

The switch leaves no gap: the active release keeps its routes until Traefik sends the requests to the candidate. A web application that keeps files is the exception: it does not answer from the stop of its active release to the switch.

When a deployment fails

If a deployment fails before the switch, for example the build, a pre-deploy command, a replica or the health check fails, Tenkai removes the candidate and your visitors see nothing.

If a step fails after the switch began, Tenkai gives the routes back to the previous active release. If it cannot tell which version receives the traffic, the hostnames may route to no version, and Tenkai refuses new deployments until you choose Reconcile routes on the page of the web application.

The troubleshooting guide tells you what to check for each failure.

What your application must accept

For a short time, two versions run at once.

Database migrations. The old version keeps serving while the candidate starts, so it must still work with the new schema. Add columns and tables first, and remove the old ones in a later deployment.
Capacity. The cluster must have room for the candidate next to the active release. Otherwise the deployment fails with "No node can run this web application".
Stopping. The old version receives SIGTERM. Finish the requests in progress, then exit.

Start the HTTP server only once your application can work, or add a health check. A candidate that exits when it cannot work stops the deployment instead of taking the traffic.

The same switch for every change

A Deployment, a Rollback, Apply configuration, and a Relocation to another node, during a drain or after you move the edge node, all follow these steps and switch the same way. A rollback starts a previous release with the current variables, after the rollback commands if any. A web application runs one of them at a time: meanwhile its older releases say Rollback unavailable.

An application that keeps files

A web application that keeps files in a volume, such as an app of the marketplace like WordPress or Uptime Kuma, or one where you keep a directory, never runs two versions at once:

Tenkai prepares the image while the active release still serves.
It stops the active release. From here the application does not answer.
When the image changes, it copies the files: this copy is the Deployment snapshot.
It runs the pre-deploy commands, if any.
It starts the candidate on the files, checks its health, then gives it the routes.

The application is down from the stop to the switch: usually under a minute, plus the time to copy the files and run the commands. If a command fails or the candidate does not start, Tenkai puts the files back and starts the previous version again.

Once the candidate has the routes, it writes to the files and Tenkai no longer puts the copy back, even if the application then answers with an error. To return to the previous version and its files, use Rollback. While Tenkai cannot see the routes on the candidate, the deployment stays Started. See A deployment never ends.

Roll back an application that keeps files

A new version may change its files, such as an SQLite schema, so Tenkai rolls back only to a version whose files it holds:

To a release with the image of the active release: the files stay as they are.
To the release that the last change of image replaced: Tenkai puts back the deployment snapshot and asks you to confirm, because what the application wrote since is no longer served. It keeps those newer files as the next deployment snapshot, so rolling forward again brings them back.
To any other release: Rollback unavailable.

Tenkai keeps one deployment snapshot per web application. Each change of image replaces it, and a restore of a backup removes it. A directory you started keeping after the snapshot makes the rollback unavailable until the next change of image. To go back further, restore a backup taken on that version.