🎢 Rolling Updates

What Is a Rolling Update?

A rolling update upgrades a fleet of identical instances by replacing them incrementally — one small batch at a time — within a single environment. At any moment only a fraction of instances are being swapped; the rest keep handling requests, so the service never goes fully offline.

Compared with its cousins, the rolling update's appeal is efficiency: no duplicate environment (blue-green) and no weighted-routing machinery (canary). The cost is that two versions run at once during the rollout, so they must be compatible — a theme we'll return to.

maxSurge & maxUnavailable

Two knobs control the pace and safety of a Kubernetes rolling update. Together they answer: "during the rollout, how many extra pods may I create, and how many may be missing?"

# A capacity-preserving rolling strategy: surge up, never drop below desired.
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1          # create at most 1 extra pod at a time
      maxUnavailable: 0    # never let serving capacity fall below 4

Readiness Probes Gate Traffic

Here's the single most important safety mechanism in a rolling update: Kubernetes will not send traffic to a new pod until its readiness probe passes. Without a good probe, the load balancer would route users to a pod that has started but isn't actually ready — no database connection, caches cold — and they'd get errors.

# Probes turn a rolling update from "hope it works" into "verified each step".
spec:
  containers:
    - name: my-app
      image: my-app:v2
      ports:
        - containerPort: 8080
      readinessProbe:            # GATE: no traffic until this returns 200
        httpGet: { path: /health/ready, port: 8080 }
        initialDelaySeconds: 5
        periodSeconds: 5
      livenessProbe:             # restart the pod if it wedges
        httpGet: { path: /health/live, port: 8080 }
        initialDelaySeconds: 15
        periodSeconds: 20

A meaningful readiness endpoint checks the dependencies real requests need, not just that the process is alive:

// A readiness check that actually reflects "can serve traffic".
app.get('/health/ready', async (req, res) => {
  try {
    await db.query('SELECT 1');            // database reachable?
    await redis.ping();                    // cache reachable?
    if (!global.appInitialized) throw new Error('warming up');
    res.status(200).json({ status: 'UP', version: process.env.APP_VERSION });
  } catch (err) {
    // 503 keeps this pod OUT of the load balancer until it recovers.
    res.status(503).json({ status: 'DOWN', reason: err.message });
  }
});

// Liveness is deliberately shallow: is the process responsive at all?
app.get('/health/live', (req, res) => res.status(200).json({ status: 'UP' }));

Driving & Rolling Back

With a Deployment in place, triggering a rolling update is one command: change the image. Kubernetes handles the batch-by-batch replacement, honoring your surge/unavailable settings and readiness gates.

# Trigger a rolling update by pointing at the new image.
kubectl set image deployment/my-app my-app=my-app:v2

# Watch it progress batch by batch (blocks until done or failed).
kubectl rollout status deployment/my-app

# See something wrong mid-rollout? Pause it where it stands.
kubectl rollout pause deployment/my-app

# ...investigate, then resume when you're satisfied.
kubectl rollout resume deployment/my-app

# Roll back to the previous revision — fast and built in.
kubectl rollout undo deployment/my-app

# Inspect the history of revisions.
kubectl rollout history deployment/my-app

That automation is straightforward: watch a metric after the rollout and call undo if it breaches a threshold.

#!/usr/bin/env bash
set -euo pipefail
DEPLOY="my-app"; NS="production"
ERROR_MAX=5          # percent
WINDOW=600           # watch for 10 minutes
PROM="http://prometheus:9090"

deadline=$(( $(date +%s) + WINDOW ))
while [ "$(date +%s)" -lt "$deadline" ]; do
  err=$(curl -s "$PROM/api/v1/query" --data-urlencode \
    'query=sum(rate(http_requests_total{status=~"5.."}[1m]))/sum(rate(http_requests_total[1m]))*100' \
    | jq -r '.data.result[0].value[1] // "0"')
  echo "error rate: ${err}%"

  if (( $(echo "$err > $ERROR_MAX" | bc -l) )); then
    echo "Error rate over threshold — rolling back."
    kubectl rollout undo "deployment/$DEPLOY" -n "$NS"
    exit 1
  fi
  sleep 15
done
echo "Rollout stable."

Version Compatibility

Because old and new instances run simultaneously during a rolling update, they must get along. Two versions of your code talk to the same database, and possibly to each other, at the same time.

The rules mirror the expand-contract pattern from the blue-green lesson, and they matter here for the same reason: the two versions overlap.

• Deploy schema changes first, additively. Add the new column before the code that uses it. The old version simply ignores it.
• Never remove a column, table, or API field until every instance is on the new version and any data migration is done.
• Provide defaults for new columns so old code that inserts a row without them still succeeds.

// New code must still read rows written by the OLD version.
function getCustomerPhone(customer) {
  // New schema populated? Use it. Otherwise fall back to the legacy field.
  return customer.phone_number ?? customer.legacy_phone ?? null;
}

// And old code must survive rows/fields the NEW version adds:
function handleRequest(body) {
  const name = body.name;
  // Unknown future fields are simply ignored — forward compatibility.
  const settings = body.settings ?? getDefaultSettings();
  return process(name, settings);
}

Connection Draining

When a pod is about to be replaced, it may still be in the middle of serving requests. Connection draining (graceful shutdown) means: stop accepting new requests, finish the ones already in flight, then exit. Skip this and users get dropped mid-request during every deploy.

In Node, handle SIGTERM: stop the server from taking new connections, let existing ones complete, and close your resource pools before exiting.

const server = app.listen(8080);

// Kubernetes sends SIGTERM before removing the pod. Drain gracefully.
process.on('SIGTERM', async () => {
  console.log('SIGTERM received — draining connections.');

  // 1. Stop accepting new connections; finish in-flight requests.
  server.close(async () => {
    // 2. Release shared resources once requests are done.
    await redisClient.quit();
    await db.end();
    console.log('Drain complete — exiting.');
    process.exit(0);
  });

  // 3. Safety net: force-exit if draining takes too long.
  setTimeout(() => process.exit(1), 30_000).unref();
});

Practice & Quiz

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxSurge: 2          # spin up 2 new pods at a time (uses spare quota)
    maxUnavailable: 0    # never drop below the 8 desired

Best Practices & Pitfalls

✅ Do

• Always define a readiness probe that checks real dependencies
• Start conservative: small maxSurge, maxUnavailable: 0 for capacity-sensitive services
• Keep API and schema changes backward-compatible (additive first)
• Implement graceful shutdown on SIGTERM and set a generous grace period
• Monitor error rate and latency during the rollout, and automate rollback
• Use kubectl rollout pause to freeze a suspicious deploy while you investigate

❌ Don't

• Don't ship without a readiness probe — traffic will hit half-ready pods
• Don't put dependency checks in the liveness probe (restart storms)
• Don't rename or drop columns in the same release as the code change
• Don't set both maxSurge and maxUnavailable to 0 — the rollout can't progress
• Don't ignore in-flight requests; skipping drain drops users mid-request

Summary

📚 Additional Resources

• Kubernetes — Rolling update Deployments
• Kubernetes — Liveness, readiness & startup probes
• AWS ECS — Rolling update deployments

🚀 What's Next?

You've now seen the three core deployment strategies. All of them assume infrastructure already exists to deploy onto. The next lesson steps back to ask how that infrastructure itself is created and versioned, reproducibly, from code: Infrastructure as Code (IaC) concepts.