Skip to main content

πŸ”΅πŸŸ’ Blue-Green Deployment

Your app is live and thousands of people are using it right now. You need to ship a new version β€” but "taking the site down for maintenance" is not an option. Blue-green deployment solves this the way a stage crew changes sets between acts: build the new scene completely off to the side, then flip one switch so the audience never sees the dark stage.

Week 13 · Day 2 (Tuesday: Deployment Strategies) · Lecture 1

🎯 Learning Objectives

By the end of this lesson, you will be able to:

  • Explain what a blue-green deployment is and how it delivers zero-downtime releases
  • Describe how a load balancer or DNS record switches all traffic from one environment to the other in seconds
  • Perform an instant rollback by pointing traffic back at the previous environment
  • Apply the expand-contract pattern to keep database migrations backward-compatible across a switch
  • Externalize session state so users are not logged out when the active environment changes
  • Implement a real Kubernetes blue-green switch with a health-gated cutover script

Estimated Time: 70 minutes

Practice: Design a blue-green setup for a three-tier app and write the traffic-switch + rollback logic.

In This Lesson

Why Deployment Strategies Matter

A deployment strategy is simply how you replace the running version of your app with a new one. The naΓ―ve approach β€” stop the old version, start the new one β€” leaves a window where nobody can use the site. For a hobby project that might be fine. For an e-commerce checkout, every minute of downtime is lost revenue and lost trust.

Modern strategies exist to shrink that window to zero. The three you'll study this week each make a different trade-off between speed, cost, and risk:

graph TD A["Deployment Strategies"] --> B["Recreate
stop old, start new β€” has downtime"] A --> C["Blue-Green
switch between two full environments"] A --> D["Canary
send a small percent of traffic to the new version"] A --> E["Rolling Update
replace instances batch by batch"]

πŸ›£οΈ The highway construction analogy

Picture renovating a busy highway. Recreate closes the whole road β€” fast work, furious drivers. Rolling update closes one lane at a time β€” traffic slows but keeps moving. Blue-green builds a brand-new highway alongside the old one and redirects everyone at once when it's ready. Canary opens a single lane of the new highway to a trickle of cars first, then widens it as confidence grows.

This lesson focuses on blue-green: the cleanest mental model and the easiest to reason about, because at any moment every user sees exactly one version.

How Blue-Green Works

You run two identical production environments, named "Blue" and "Green." They are identical in every way β€” same infrastructure, same configuration, same scale β€” except for the version of the application code they run. Only one is live (serving real users) at a time; the other is idle, ready to become the next live environment.

A load balancer routing all live traffic to the Blue environment, with Green idle and ready Load Balancer the traffic switch πŸ”΅ Blue (LIVE) app v1.0 β€” 100% traffic 🟒 Green (idle) app v1.1 β€” 0% traffic
All live traffic flows to Blue. Green runs the new version but receives nothing β€” until you flip the switch, at which point the solid and dashed arrows swap.

The lifecycle of one release looks like this:

sequenceDiagram participant User participant LB as Load Balancer participant Blue as Blue Environment participant Green as Green Environment Note over Blue,Green: Blue is live, Green is idle User->>LB: Request LB->>Blue: Route to the live environment Blue->>User: Response from v1.0 Note over Green: Deploy v1.1 to Green and warm it up Note over Green: Run smoke tests against Green privately Note over LB: Flip the switch to Green User->>LB: Request LB->>Green: Route to the new live environment Green->>User: Response from v1.1 Note over Blue: Blue is now idle and kept ready for rollback

The key ideas

  • Identical environments. The only difference is the app version. Keep them in sync with the same infrastructure-as-code so there is no "works on Blue but not Green" surprise.
  • A switch in front. A load balancer, router, or DNS record decides which environment is live. Changing that one setting redirects everyone.
  • Atomic cutover. Traffic moves all at once. No user ever gets a mix of old and new β€” a huge simplification compared to strategies where versions overlap.
  • Instant rollback. The old environment is still running. If the new one misbehaves, flip the switch back.
  • Shared backing services. Databases, queues, and caches are usually shared, which is exactly where the hard parts live (covered below).

The Traffic Switch & Rollback

"Flip the switch" can mean different things depending on where the switch lives. Each level trades simplicity for speed.

Switch levelMechanismCutover speedWatch out for
DNSRepoint a DNS record to the new environmentSlow (minutes to hours)TTL caching means some clients keep hitting the old IP
Load balancerChange the listener's target group / backend poolInstant (seconds)Provider-specific, but the recommended default
OrchestratorUpdate a Service selector (Kubernetes, ECS)InstantIntegrated with CI/CD; more moving parts to learn

⚠️ Why DNS is a trap for the cutover

DNS records are cached by resolvers and browsers for the duration of their TTL. Even with a 60-second TTL, some clients ignore it and keep the old address for far longer. That means a period of mixed traffic where you can't cleanly say "everyone is on Green now." Prefer a load balancer switch, whose effect is immediate and total.

Rollback is the whole point

The reason blue-green feels safe is that a rollback is the same operation as the deploy, just aimed the other way. There is no rebuild, no re-deploy, no waiting β€” the previous version is still warm and running.

# Cut over to Green (make it live)
aws elbv2 modify-listener \
  --listener-arn "$LISTENER_ARN" \
  --default-actions Type=forward,TargetGroupArn="$GREEN_TG_ARN"

# Something's wrong? Roll back instantly by pointing at Blue again.
aws elbv2 modify-listener \
  --listener-arn "$LISTENER_ARN" \
  --default-actions Type=forward,TargetGroupArn="$BLUE_TG_ARN"

βœ… Keep the old environment warm

Do not tear down Blue the instant Green goes live. Keep it running for a "bake" period (often a few hours to a day) so that if a subtle bug surfaces under real load, rollback is one command away. Only scale Blue down once you're confident.

Database Migrations: The Hard Part

Here's the catch that trips up every newcomer: Blue and Green usually share one database. You can flip application code atomically, but you cannot keep two copies of your users' live data in sync. So the schema must work with both versions of the app during the switch and rollback window.

graph LR A["Add a nullable column"] --> B["Low risk
old code ignores it"] C["Add a new table"] --> B D["Change a column type"] --> E["Medium risk
needs a transition"] F["Drop a column or table"] --> G["High risk
breaks rollback if old code needs it"]

The expand-contract pattern

Never make a breaking change in one step. Split it across multiple deploys so that at every moment, the live schema is compatible with the code that could be running:

  1. Expand. Add the new column/table alongside the old. Both versions of the app can now read the database.
  2. Migrate & deploy. Ship the new code that writes to both old and new, and backfill existing rows in the background.
  3. Contract. Only after the new version is proven stable (and rollback is no longer needed) do you drop the old column in a later release.
-- STEP 1 (Expand): additive change, safe for both app versions
ALTER TABLE users ADD COLUMN phone_number VARCHAR(20);
-- Old code never touches this column, so it keeps working.

-- STEP 3 (Contract): only in a FUTURE release, after v1.1 is trusted
-- ALTER TABLE users DROP COLUMN legacy_phone;

During the transition, the application writes to both fields so a rollback to the old code still finds the data it expects:

// Transitional write path (runs in the new version)
async function updateUserPhone(userId, phone) {
  const updates = { legacy_phone: phone };   // keep the old column populated...

  // ...and also fill the new column when the schema has it.
  if (await columnExists('users', 'phone_number')) {
    updates.phone_number = phone;
  }

  await db.update('users', userId, updates);
}

πŸ’‘ Decouple "deploy" from "release"

A feature flag lets you ship code that is dormant until you turn it on. Combined with blue-green, this means the risky new feature can be enabled after the switch, and disabled instantly without touching the deployment. Deploy strategies move code; feature flags control behavior. Use both.

Handling State & Sessions

If your app keeps user sessions in the memory of a specific server, switching environments logs everyone out β€” their session lived on Blue, and Blue is no longer serving them. The fix is to externalize state so it survives the cutover.

Kind of stateWhere it should live
User sessions / cartsA shared store like Redis, not server memory
Cached dataA shared cache (Redis/Memcached) both environments read
Background jobsA durable queue, so work isn't lost when a worker stops
WebSocket connectionsDrain gracefully; ask clients to reconnect to the new environment
// Sessions in Redis survive the blue-green switch because BOTH
// environments read from the same store β€” not from local memory.
import express from 'express';
import session from 'express-session';
import { RedisStore } from 'connect-redis';
import { createClient } from 'redis';

const app = express();
const redisClient = createClient({ url: process.env.REDIS_URL });
await redisClient.connect();

app.use(session({
  store: new RedisStore({ client: redisClient }),
  secret: process.env.SESSION_SECRET,
  resave: false,
  saveUninitialized: false,
  cookie: { secure: process.env.NODE_ENV === 'production', maxAge: 86400000 } // 1 day
}));

// A user logged in on Blue stays logged in on Green β€” the cookie
// points at a session that lives in shared Redis, not on any one box.
app.get('/profile', (req, res) => {
  if (!req.session.user) return res.redirect('/login');
  res.json({ user: req.session.user });
});

βœ… The golden rule: keep app servers stateless

If any two requests from the same user could safely hit different servers, blue-green (and every other zero-downtime strategy) becomes dramatically simpler. Push all durable state to databases, caches, and queues.

A Real Kubernetes Switch

In Kubernetes the "switch" is a Service selector. A Service routes to whichever Pods match its label selector, so changing one label β€” version: blue to version: green β€” instantly moves all traffic.

# The Service is the switch. Its selector decides which Deployment is live.
apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  selector:
    app: my-app
    version: blue      # flip to "green" to cut over
  ports:
    - port: 80
      targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-green
spec:
  replicas: 3
  selector:
    matchLabels: { app: my-app, version: green }
  template:
    metadata:
      labels: { app: my-app, version: green }
    spec:
      containers:
        - name: my-app
          image: my-app:1.1
          ports:
            - containerPort: 8080
          readinessProbe:          # gate: no traffic until this passes
            httpGet: { path: /health, port: 8080 }
            initialDelaySeconds: 5
            periodSeconds: 5

A safe cutover script deploys Green, waits for it to be healthy, smoke-tests it, then flips the selector β€” with a built-in rollback if the switch doesn't take:

#!/usr/bin/env bash
set -euo pipefail

# 1. Roll out the green version and wait until every pod is ready.
kubectl apply -f deployment-green.yaml
kubectl rollout status deployment/my-app-green --timeout=120s

# 2. Smoke-test green privately BEFORE it takes live traffic.
if ! kubectl run smoke --rm -i --restart=Never --image=curlimages/curl -- \
      -sf http://my-app-green/health; then
  echo "Green failed its health check β€” aborting, blue stays live."
  exit 1
fi

# 3. Flip the switch: point the Service at green.
kubectl patch service my-app -p '{"spec":{"selector":{"version":"green"}}}'
echo "Traffic switched to green."

# 4. Verify, and roll back automatically if green isn't serving v1.1.
sleep 10
if ! curl -sf http://my-app/version | grep -q '1.1'; then
  echo "Live traffic not reaching green β€” rolling back to blue."
  kubectl patch service my-app -p '{"spec":{"selector":{"version":"blue"}}}'
  exit 1
fi

echo "Deployment healthy. Keeping blue warm for the bake period."

⚠️ A health check must mean "ready for real traffic"

A check that only returns 200 OK because the process started is worthless β€” the pod might not have a database connection yet. Make readiness probes verify the dependencies that requests actually need, so the switch never sends users to a half-initialized environment.

Practice & Quiz

πŸ‹οΈ Exercise 1: Design the switch and rollback

Goal: For a three-tier app (React SPA, Node API, PostgreSQL) with Redis-backed sessions, sketch a blue-green plan. Decide: where does the switch live? How is the database change made safe? How do sessions survive the cutover? What is the exact rollback step?

πŸ’‘ Hint

Put the switch at the load balancer (not DNS). Use expand-contract for the schema. Sessions already live in Redis, which both environments share. The rollback is a single load-balancer change back to the previous target group.

βœ… Solution
  • Switch: ALB listener forwards to Blue or Green target group. Cutover = one modify-listener call.
  • Database: Apply additive migrations first (expand). New API writes to both old and new columns. Drop old columns only in a later release (contract).
  • Sessions: Already externalized to Redis, so a user on Blue stays logged in on Green.
  • Rollback: Point the listener back at the Blue target group β€” instant, because Blue is still running.

πŸ‹οΈ Exercise 2: Order the migration steps

Goal: You must rename users.email_addr to users.email during a blue-green release without breaking rollback. Put these steps in the correct order: (a) drop email_addr, (b) add email and copy data, (c) deploy code reading email and writing both, (d) confirm stability for a day.

βœ… Solution

Order: b β†’ c β†’ d β†’ a. Add the new column and backfill (expand), deploy code that writes both columns, bake to confirm the new version is trusted, then drop the old column in a later release (contract). A one-step rename would break the old code the moment you rolled back.

🎯 Quick Quiz

Question 1: What is the main advantage of switching traffic at the load balancer instead of via DNS?

Question 2: Why must you avoid dropping a column in the same release that switches to the new version?

Question 3: Where should user sessions live so they survive a blue-green switch?

Best Practices & Pitfalls

βœ… Do

  • Keep Blue and Green truly identical using infrastructure-as-code
  • Gate the switch on a deep health check that verifies real dependencies
  • Use expand-contract so every migration is backward-compatible
  • Externalize sessions, cache, and jobs so app servers stay stateless
  • Keep the old environment warm for a bake period before scaling it down
  • Automate both the switch and the rollback β€” never do the cutover by hand

❌ Don't

  • Don't rely on DNS for an instant cutover β€” TTL caching causes mixed traffic
  • Don't drop columns or make breaking schema changes in the switch release
  • Don't tear down Blue the moment Green goes live β€” you lose your instant rollback
  • Don't let configuration drift between the two environments
  • Don't ship a "started = healthy" probe that lets traffic hit a half-ready app

⚠️ The cost trade-off

Running two full production environments roughly doubles infrastructure cost during a deploy. Many teams accept this only around releases (spin Green up, cut over, scale Blue down) rather than 24/7. If double capacity is unacceptable, a rolling update β€” covered later this week β€” needs far less headroom.

Summary

πŸŽ‰ Key Takeaways

  • Blue-green runs two identical environments; only one is live at a time
  • A load-balancer switch moves all traffic atomically in seconds β€” no mixed versions
  • Rollback is instant because the previous environment is still warm
  • The shared database is the hard part: use the expand-contract pattern to stay backward-compatible
  • Externalize state (Redis sessions, durable queues) so app servers are stateless and survive the switch
  • Gate the cutover on deep health checks, and keep old capacity warm during a bake period

πŸ“š Additional Resources

πŸš€ What's Next?

Blue-green flips everyone at once. But what if you want to test a new version on just a sliver of real users first, watching the metrics before committing? That gradual, risk-minimizing approach is the next lesson: Canary Releases.

πŸŽ‰ Great work!

You can now ship a new version with zero downtime and an instant escape hatch. Next up, we make the rollout even more cautious.