Volumes on a cluster: why an application that keeps files stays on one node
See how Tenkai keeps the files of a web application in a volume on one node, what that means for its replicas, its deployments and a drain, and when object storage suits better.
A web application that keeps files in Tenkai runs one replica on one node. Its files live in a volume on that node's disk, not on the other nodes. Each deployment stops the running version before starting a new version that uses the same files.
Scene 1 of 7: The files live on one node
wiki keeps /app/data in a volume on node-2. Its one replica runs there because the other nodes do not have its files.
How Tenkai keeps a directory
Without a volume, a web application starts each deployment from its image and loses files it wrote inside its container. To keep uploads or an SQLite database, keep a directory such as /app/data in its Configuration tab. Marketplace apps such as WordPress and Uptime Kuma already declare their directories.
Tenkai chooses a node that accepts workloads and has enough CPU and memory for the application. This is its data node. On the next deployment, Tenkai creates a Docker volume on that node for each kept directory, up to eight directories. It makes each volume available at its configured path inside the application.
Swarm then runs the application only on its data node. Tenkai refuses a second replica to prevent two replicas from writing to the same files. Another node has no copy of them.
What changes for the application
A deployment stops the running version before the new one starts. The application cannot answer until the new version is ready and receives traffic. The application stays offline while Tenkai copies files, runs pre-deploy commands and starts the new version. This can take more than a minute.
When an image change replaces a running version, Tenkai copies its files on the same node before starting the new version. It keeps this deployment snapshot for a rollback. A deployment that keeps the image but runs commands takes a temporary copy too, then deletes it on success.
A backup also stops the application while Tenkai archives its files, backs up its databases and sends the copies to a storage bucket. A live backup avoids the stop, but may copy files while they change. An SQLite database copied that way may not open.
Tenkai refuses to drain its data node. On a manager, Stop accepting workloads moves applications without volumes and leaves this one running. Restarting the node interrupts it. It can start there again when the node returns with its files intact, but cannot start on another node.
What the volume does not protect against
A volume survives container restarts and deployments, not a lost disk. Tenkai does not copy it to another node. The deployment snapshot is on the same disk, so it cannot protect against that loss either.
Use a backup in a storage bucket at a provider to protect the files. Tenkai restores it on the same node only. If that node is lost, read the backup without Tenkai.
When object storage suits better
If your application can keep all its persistent files through the S3 API, it can run without a volume. It can then have several replicas, start a new version beside the running one and move during a drain. The old version keeps serving until the new version takes traffic.
Use a hosted bucket that Tenkai runs on your cluster with Garage, or a bucket at a provider. A hosted bucket still depends on one node. Keep a copy outside the cluster for files you cannot lose. That copy also copies deletions, so it does not replace a versioned backup.
A volume is the right choice for an application that needs a local filesystem, such as SQLite. An S3 bucket does not provide its file locks.
Limits
- You cannot choose the data node, or move the files to another node.
- You cannot remove a kept directory or change its path yet.
- Deleting the web application deletes its volumes.