Load balancing and replicas

A replica is a running copy of your application. Tenkai can run 1 to 10 replicas of a web application. The edge router spreads incoming requests across them. Two replicas get about half each, but they do not make every part of your application highly available.

Choose the count in Change configuration, then use Apply configuration or Deploy. Saving the count alone does not change the running application.

How the edge router spreads the requests

Public requests enter through one edge node. The edge router, Traefik, asks Docker Swarm which replicas are running. It forwards requests to them through their environment's network.

Traefik uses round robin, taking replicas in turn. It does not send fewer requests to a busy replica. An open WebSocket stays on the replica that accepted it.

The next request from a visitor may reach another replica. Replicas do not share memory or container files. Store shared sessions in a database or cache, such as PostgreSQL or Redis®. Store uploads in object storage, such as a hosted bucket or an external S3-compatible bucket. A storage bucket registered for backups does not give an application access to it.

Where the replicas run

Each replica reserves the CPU and memory of its size. Tenkai checks capacity before starting a deployment. Swarm places replicas on nodes that accept workloads and have room. Several replicas may share a node, so their count is not a count of independent servers.

On one server, more replicas can help a single-threaded application use more CPUs. They cannot survive the loss of that server.

Applying a new count starts a complete new set beside the old one. The cluster needs room for both. Before switching traffic, Tenkai waits for the new replicas to run and probes their configured HTTP port, 8080 by default. Without a configured health check, any HTTP status passes. That proves the server listens, not that the application works.

The old version keeps its routes until Traefik sends requests to the new one. This leaves no routing gap. See What visitors see during the switch.

When a replica stops

A process can exit, a node can fail, or Docker can stop a container after its image's HEALTHCHECK fails. Swarm tries to replace the replica without waiting for Tenkai. It needs a majority of the managers, enough capacity and access to the image.

Traefik polls Swarm every 15 seconds by default. Until Swarm reports the stopped replica and Traefik updates its routes, requests sent there can fail. This is not a 15-second recovery guarantee. With two replicas, roughly half the requests may fail during that interval.

After the router removes it, the remaining replicas take the full request rate. They need enough capacity to handle that extra work. A replacement receives traffic once Swarm reports it running and Traefik discovers it. Without an image health check, that does not prove it can answer HTTP yet.

What replicas do not protect against

• A lost edge node stops public traffic, even when replicas run elsewhere.
• Docker Swarm cannot replace replicas unless more than half the manager nodes can communicate. Containers on surviving nodes keep running.
• Application replicas do not copy the database. If its node stops, requests that need it fail.
• An application that keeps files runs one replica on its data node. It stops while that node is down.

When another option suits better

• Keep one replica if your application needs in-memory sessions and cannot share them. Those sessions still disappear when it restarts.
• If requests vary greatly in cost, test whether round robin suits them. Tenkai offers no choice of balancing algorithm.
• Use a CDN to serve static files closer to distant visitors. Adding replicas does not move your servers closer to them.

Go further

• How traffic reaches your applicationFollow a request from DNS to the edge node, through HTTPS and the edge router, to a replica on any node of the cluster.
• Clusters and node rolesUnderstand the edge node, manager nodes, worker nodes and data nodes, how many failures a cluster survives, and how to keep replicas on separate nodes.
• How a deployment switches trafficUnderstand the blue-green deployment: Tenkai starts a candidate next to the active release, checks it, then moves the routes to it.
• Keep the files of a web applicationAsk a web application to keep a directory between two deployments, in a volume on one node of your cluster, and know what that changes for it.
• Read the traffic of a web applicationRead the traffic tab of a web application: its requests by status class over time, what counts as a request, and what a gap in the chart means.

Redis is a registered trademark of Redis Ltd. Any rights therein are reserved to Redis Ltd.