Integration Testing Strategies

Understanding Integration Testing

If unit tests are about examining individual puzzle pieces, integration tests are about making sure the pieces fit together correctly. Integration testing verifies that different modules or services in your application work well together.

Why Integration Testing Matters

• Catch Interface Mismatches: Detect when components have incompatible interfaces
• Verify Data Flow: Ensure data correctly flows between components
• Test Configuration: Verify components are correctly configured to work together
• Bridge Unit and E2E Tests: Provide a middle ground between isolated unit tests and comprehensive E2E tests

Integration vs. Unit Testing

Types of Integration Tests

Top-Down Integration Testing

Start with high-level components and gradually integrate lower-level components.

Benefits: Early detection of interface issues, rapid feedback on system behavior.

Drawbacks: Lower-level components need stubs, which can be complex to create.

Bottom-Up Integration Testing

Start with low-level components and gradually move up to higher-level components.

Benefits: Low-level components are thoroughly tested before higher-level ones are introduced.

Drawbacks: High-level behavior isn't validated until late in the process.

Sandwich (Hybrid) Integration Testing

Combine top-down and bottom-up approaches, testing from both ends toward the middle.

Benefits: Combines advantages of both approaches, often more efficient.

Drawbacks: More complex to plan and organize.

Big Bang Integration Testing

Integrate all components at once and test the entire system.

Benefits: Simplest approach, no need for stubs or drivers.

Drawbacks: Hard to locate issues when tests fail, slow feedback cycle.

Integration Test Strategies for Web Applications

API Integration Tests

Test the RESTful API endpoints to ensure they correctly integrate with backend services and databases.

// Example API integration test using Supertest
const request = require('supertest');
const app = require('../app');
const db = require('../db');

describe('User API', () => {
  beforeAll(async () => {
    await db.connect();
  });

  afterAll(async () => {
    await db.disconnect();
  });

  beforeEach(async () => {
    await db.clearUsers();
  });

  test('GET /api/users returns empty array when no users exist', async () => {
    const response = await request(app)
      .get('/api/users')
      .expect('Content-Type', /json/)
      .expect(200);
    
    expect(response.body).toEqual([]);
  });

  test('POST /api/users creates a new user', async () => {
    const userData = {
      username: 'testuser',
      email: 'test@example.com',
      password: 'Password123!'
    };
    
    const response = await request(app)
      .post('/api/users')
      .send(userData)
      .expect('Content-Type', /json/)
      .expect(201);
    
    expect(response.body.username).toBe(userData.username);
    expect(response.body.email).toBe(userData.email);
    expect(response.body.id).toBeDefined();
    
    // Verify user was actually saved to database
    const savedUser = await db.findUserByEmail(userData.email);
    expect(savedUser).toBeTruthy();
    expect(savedUser.username).toBe(userData.username);
  });

  test('GET /api/users/:id returns 404 for non-existent user', async () => {
    await request(app)
      .get('/api/users/nonexistentid')
      .expect(404);
  });
});

Database Integration Tests

Test interactions between your application and the database to ensure correct data persistence and retrieval.

// Example database integration test
const UserRepository = require('../repositories/userRepository');
const db = require('../db');

describe('UserRepository', () => {
  let userRepository;
  
  beforeAll(async () => {
    await db.connect();
    userRepository = new UserRepository(db);
  });
  
  afterAll(async () => {
    await db.disconnect();
  });
  
  beforeEach(async () => {
    await db.clearUsers();
  });
  
  test('creates and retrieves a user', async () => {
    const userData = {
      username: 'testuser',
      email: 'test@example.com',
      password: 'hashedpassword'
    };
    
    // Create user
    const createdUser = await userRepository.create(userData);
    expect(createdUser.id).toBeDefined();
    
    // Retrieve user
    const retrievedUser = await userRepository.findById(createdUser.id);
    expect(retrievedUser).toEqual(createdUser);
  });
  
  test('updates user data', async () => {
    // Create user
    const user = await userRepository.create({
      username: 'original',
      email: 'test@example.com',
      password: 'hashedpassword'
    });
    
    // Update user
    const updatedData = { username: 'updated' };
    await userRepository.update(user.id, updatedData);
    
    // Verify update
    const updatedUser = await userRepository.findById(user.id);
    expect(updatedUser.username).toBe('updated');
    expect(updatedUser.email).toBe(user.email); // Unchanged
  });
  
  test('deletes a user', async () => {
    // Create user
    const user = await userRepository.create({
      username: 'testuser',
      email: 'test@example.com',
      password: 'hashedpassword'
    });
    
    // Delete user
    await userRepository.delete(user.id);
    
    // Verify deletion
    const retrievedUser = await userRepository.findById(user.id);
    expect(retrievedUser).toBeNull();
  });
});

Service Integration Tests

Test how different services in your application work together, often mocking external dependencies.

// Example service integration test
const UserService = require('../services/userService');
const EmailService = require('../services/emailService');
const UserRepository = require('../repositories/userRepository');

describe('UserService with EmailService integration', () => {
  let userService;
  let emailService;
  let userRepository;
  
  beforeEach(() => {
    // Create real instances but mock certain methods
    userRepository = {
      create: jest.fn(),
      findByEmail: jest.fn(),
      findById: jest.fn(),
      update: jest.fn()
    };
    
    emailService = {
      sendWelcomeEmail: jest.fn().mockResolvedValue(true),
      sendPasswordResetEmail: jest.fn().mockResolvedValue(true)
    };
    
    userService = new UserService(userRepository, emailService);
  });
  
  test('registers user and sends welcome email', async () => {
    // Setup
    const userData = {
      username: 'testuser',
      email: 'test@example.com',
      password: 'Password123!'
    };
    
    const createdUser = { 
      id: 'user-123',
      ...userData,
      password: 'hashed-password'
    };
    
    userRepository.create.mockResolvedValue(createdUser);
    
    // Execute
    const result = await userService.registerUser(userData);
    
    // Verify
    expect(userRepository.create).toHaveBeenCalledWith(expect.objectContaining({
      username: userData.username,
      email: userData.email
      // Password should be hashed, not the original
    }));
    
    // Verify email service integration
    expect(emailService.sendWelcomeEmail).toHaveBeenCalledWith(
      userData.email,
      userData.username
    );
    
    expect(result).toEqual({
      id: createdUser.id,
      username: createdUser.username,
      email: createdUser.email
    });
  });
  
  test('handles email sending failure gracefully', async () => {
    // Setup
    const userData = {
      username: 'testuser',
      email: 'test@example.com',
      password: 'Password123!'
    };
    
    const createdUser = { 
      id: 'user-123',
      ...userData,
      password: 'hashed-password'
    };
    
    userRepository.create.mockResolvedValue(createdUser);
    emailService.sendWelcomeEmail.mockRejectedValue(new Error('Email failure'));
    
    // Execute
    const result = await userService.registerUser(userData);
    
    // Verify user is still created despite email failure
    expect(userRepository.create).toHaveBeenCalled();
    expect(result).toBeDefined();
    expect(result.id).toBe(createdUser.id);
  });
});

UI Integration Tests

Test how UI components interact with each other and with services, often using React Testing Library or similar tools.

// Example UI integration test using React Testing Library
import React from 'react';
import { render, screen, fireEvent, waitFor } from '@testing-library/react';
import UserRegistrationForm from '../components/UserRegistrationForm';
import { UserService } from '../services/userService';

// Mock the user service
jest.mock('../services/userService');

describe('UserRegistrationForm', () => {
  beforeEach(() => {
    UserService.registerUser = jest.fn();
  });
  
  test('submits form data to user service', async () => {
    // Setup
    UserService.registerUser.mockResolvedValue({
      id: 'new-user-123',
      username: 'testuser',
      email: 'test@example.com'
    });
    
    // Render the component
    render(<UserRegistrationForm />);
    
    // Fill out the form
    fireEvent.change(screen.getByLabelText(/username/i), {
      target: { value: 'testuser' }
    });
    
    fireEvent.change(screen.getByLabelText(/email/i), {
      target: { value: 'test@example.com' }
    });
    
    fireEvent.change(screen.getByLabelText(/password/i), {
      target: { value: 'Password123!' }
    });
    
    // Submit the form
    fireEvent.click(screen.getByRole('button', { name: /register/i }));
    
    // Verify service was called with correct data
    expect(UserService.registerUser).toHaveBeenCalledWith({
      username: 'testuser',
      email: 'test@example.com',
      password: 'Password123!'
    });
    
    // Wait for success message
    await waitFor(() => {
      expect(screen.getByText(/registration successful/i)).toBeInTheDocument();
    });
  });
  
  test('displays validation errors', async () => {
    // Render the component
    render(<UserRegistrationForm />);
    
    // Submit without filling out the form
    fireEvent.click(screen.getByRole('button', { name: /register/i }));
    
    // Verify validation errors
    await waitFor(() => {
      expect(screen.getByText(/username is required/i)).toBeInTheDocument();
      expect(screen.getByText(/email is required/i)).toBeInTheDocument();
      expect(screen.getByText(/password is required/i)).toBeInTheDocument();
    });
    
    // UserService should not be called
    expect(UserService.registerUser).not.toHaveBeenCalled();
  });
  
  test('handles registration errors', async () => {
    // Setup error response
    UserService.registerUser.mockRejectedValue(
      new Error('Email already in use')
    );
    
    // Render and fill form
    render(<UserRegistrationForm />);
    
    fireEvent.change(screen.getByLabelText(/username/i), {
      target: { value: 'testuser' }
    });
    
    fireEvent.change(screen.getByLabelText(/email/i), {
      target: { value: 'existing@example.com' }
    });
    
    fireEvent.change(screen.getByLabelText(/password/i), {
      target: { value: 'Password123!' }
    });
    
    // Submit form
    fireEvent.click(screen.getByRole('button', { name: /register/i }));
    
    // Verify error is displayed
    await waitFor(() => {
      expect(screen.getByText(/email already in use/i)).toBeInTheDocument();
    });
  });
});

Testing API Endpoints

API endpoints are one of the most common integration points in modern web applications. Let's explore how to effectively test them.

Testing Tools for API Integration

• Supertest: Simple HTTP assertion library that works well with Express
• Jest: Testing framework that works well with Supertest
• Nock: HTTP server mocking and expectations library
• Postman/Newman: API platform with testing capabilities

Setting Up Supertest with Express

// app.js
const express = require('express');
const bodyParser = require('body-parser');
const usersRouter = require('./routes/users');

const app = express();

app.use(bodyParser.json());
app.use('/api/users', usersRouter);

// Error handler
app.use((err, req, res, next) => {
  console.error(err.stack);
  res.status(500).json({ error: 'Something went wrong!' });
});

module.exports = app;

// server.js
const app = require('./app');
const port = process.env.PORT || 3000;

app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

// test/api.test.js
const request = require('supertest');
const app = require('../app');
const db = require('../db');

// Global setup/teardown
beforeAll(async () => {
  await db.connect();
});

afterAll(async () => {
  await db.disconnect();
});

// Test suite
describe('API Endpoints', () => {
  beforeEach(async () => {
    await db.clear(); // Reset database before each test
  });
  
  describe('POST /api/users', () => {
    test('creates a new user with valid data', async () => {
      const userData = {
        name: 'John Doe',
        email: 'john@example.com',
        password: 'password123'
      };
      
      const response = await request(app)
        .post('/api/users')
        .send(userData)
        .expect('Content-Type', /json/)
        .expect(201);
      
      // Check response body
      expect(response.body.id).toBeDefined();
      expect(response.body.name).toBe(userData.name);
      expect(response.body.email).toBe(userData.email);
      
      // Password should not be returned
      expect(response.body.password).toBeUndefined();
      
      // Verify user was actually created in the database
      const user = await db.findUserByEmail(userData.email);
      expect(user).toBeTruthy();
      expect(user.name).toBe(userData.name);
    });
    
    test('returns 400 with invalid data', async () => {
      // Missing required field
      const invalidData = {
        name: 'John Doe',
        // email is missing
        password: 'password123'
      };
      
      const response = await request(app)
        .post('/api/users')
        .send(invalidData)
        .expect('Content-Type', /json/)
        .expect(400);
      
      expect(response.body.error).toBeDefined();
    });
    
    test('returns 409 if email already exists', async () => {
      // First create a user
      await db.createUser({
        name: 'Existing User',
        email: 'existing@example.com',
        password: 'hashedpassword'
      });
      
      // Try to create another user with the same email
      const duplicateData = {
        name: 'New User',
        email: 'existing@example.com',
        password: 'newpassword'
      };
      
      await request(app)
        .post('/api/users')
        .send(duplicateData)
        .expect('Content-Type', /json/)
        .expect(409);
    });
  });
  
  describe('GET /api/users', () => {
    test('returns all users', async () => {
      // Add some test users
      await db.createUser({ name: 'User 1', email: 'user1@example.com', password: 'pass1' });
      await db.createUser({ name: 'User 2', email: 'user2@example.com', password: 'pass2' });
      
      const response = await request(app)
        .get('/api/users')
        .expect('Content-Type', /json/)
        .expect(200);
      
      expect(response.body).toBeInstanceOf(Array);
      expect(response.body.length).toBe(2);
      
      // Verify user properties
      const user1 = response.body.find(u => u.email === 'user1@example.com');
      expect(user1).toBeDefined();
      expect(user1.name).toBe('User 1');
      
      // Passwords should not be returned
      expect(user1.password).toBeUndefined();
    });
    
    test('returns empty array when no users exist', async () => {
      const response = await request(app)
        .get('/api/users')
        .expect('Content-Type', /json/)
        .expect(200);
      
      expect(response.body).toEqual([]);
    });
  });
  
  describe('GET /api/users/:id', () => {
    test('returns user by id', async () => {
      // Create test user
      const user = await db.createUser({
        name: 'Test User',
        email: 'test@example.com',
        password: 'password'
      });
      
      const response = await request(app)
        .get(`/api/users/${user.id}`)
        .expect('Content-Type', /json/)
        .expect(200);
      
      expect(response.body.id).toBe(user.id);
      expect(response.body.name).toBe(user.name);
      expect(response.body.email).toBe(user.email);
      expect(response.body.password).toBeUndefined();
    });
    
    test('returns 404 for non-existent user', async () => {
      await request(app)
        .get('/api/users/nonexistentid')
        .expect(404);
    });
  });
  
  describe('PUT /api/users/:id', () => {
    test('updates user with valid data', async () => {
      // Create test user
      const user = await db.createUser({
        name: 'Original Name',
        email: 'original@example.com',
        password: 'password'
      });
      
      const updateData = {
        name: 'Updated Name',
        email: 'updated@example.com'
      };
      
      const response = await request(app)
        .put(`/api/users/${user.id}`)
        .send(updateData)
        .expect('Content-Type', /json/)
        .expect(200);
      
      expect(response.body.name).toBe(updateData.name);
      expect(response.body.email).toBe(updateData.email);
      
      // Verify database was updated
      const updatedUser = await db.findUserById(user.id);
      expect(updatedUser.name).toBe(updateData.name);
      expect(updatedUser.email).toBe(updateData.email);
    });
    
    test('returns 404 for non-existent user', async () => {
      await request(app)
        .put('/api/users/nonexistentid')
        .send({ name: 'New Name' })
        .expect(404);
    });
  });
  
  describe('DELETE /api/users/:id', () => {
    test('deletes user', async () => {
      // Create test user
      const user = await db.createUser({
        name: 'To Delete',
        email: 'delete@example.com',
        password: 'password'
      });
      
      await request(app)
        .delete(`/api/users/${user.id}`)
        .expect(204);
      
      // Verify user was deleted from database
      const deletedUser = await db.findUserById(user.id);
      expect(deletedUser).toBeNull();
    });
    
    test('returns 404 for non-existent user', async () => {
      await request(app)
        .delete('/api/users/nonexistentid')
        .expect(404);
    });
  });
});

test('returns 401 for unauthenticated request', async () => {
  await request(app)
    .get('/api/protected-resource')
    .expect(401);
});

test('returns data for authenticated request', async () => {
  // Create a user and generate a token
  const user = await db.createUser({ /* user data */ });
  const token = generateToken(user);
  
  const response = await request(app)
    .get('/api/protected-resource')
    .set('Authorization', `Bearer ${token}`)
    .expect(200);
  
  expect(response.body).toBeDefined();
});

Database Integration Testing

Database integration tests verify that your application correctly interacts with the database. This includes testing repositories, ORMs, and database-specific code.

Approaches to Database Testing

1. Test against a real database: Provides the highest confidence but can be slower and requires setup
2. Test against an in-memory database: Faster but may have subtle differences from your production database
3. Mock the database layer: Fastest but provides less confidence in real database interactions

Setting Up a Test Database

// db.js
const { Pool } = require('pg');

let pool;

module.exports = {
  connect: async () => {
    // Use a different database for testing
    const dbName = process.env.NODE_ENV === 'test' ? 'testdb' : 'appdb';
    
    pool = new Pool({
      host: process.env.DB_HOST || 'localhost',
      user: process.env.DB_USER || 'postgres',
      password: process.env.DB_PASSWORD || 'postgres',
      database: dbName,
      port: process.env.DB_PORT || 5432
    });
    
    // Test the connection
    await pool.query('SELECT NOW()');
    console.log(`Connected to ${dbName} database`);
  },
  
  disconnect: async () => {
    if (pool) {
      await pool.end();
      console.log('Database connection closed');
    }
  },
  
  // Clear all tables for testing
  clear: async () => {
    if (process.env.NODE_ENV !== 'test') {
      throw new Error('Cannot clear database in non-test environment');
    }
    
    await pool.query('TRUNCATE users, posts, comments RESTART IDENTITY CASCADE');
  },
  
  // User operations
  createUser: async (userData) => {
    const { name, email, password } = userData;
    const result = await pool.query(
      'INSERT INTO users (name, email, password) VALUES ($1, $2, $3) RETURNING *',
      [name, email, password]
    );
    return result.rows[0];
  },
  
  findUserById: async (id) => {
    const result = await pool.query('SELECT * FROM users WHERE id = $1', [id]);
    return result.rows[0] || null;
  },
  
  findUserByEmail: async (email) => {
    const result = await pool.query('SELECT * FROM users WHERE email = $1', [email]);
    return result.rows[0] || null;
  },
  
  // Add more database operations as needed
};

Testing with an In-Memory Database

For faster tests, you might use an in-memory database like SQLite:

// db.test.js - In-memory SQLite example
const sqlite3 = require('sqlite3');
const { open } = require('sqlite');

let db;

module.exports = {
  connect: async () => {
    db = await open({
      filename: ':memory:', // In-memory database
      driver: sqlite3.Database
    });
    
    // Create tables
    await db.exec(`
      CREATE TABLE users (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        name TEXT NOT NULL,
        email TEXT UNIQUE NOT NULL,
        password TEXT NOT NULL
      );
    `);
  },
  
  disconnect: async () => {
    if (db) {
      await db.close();
    }
  },
  
  clear: async () => {
    await db.exec('DELETE FROM users');
  },
  
  // User operations
  createUser: async (userData) => {
    const { name, email, password } = userData;
    const result = await db.run(
      'INSERT INTO users (name, email, password) VALUES (?, ?, ?)',
      [name, email, password]
    );
    return {
      id: result.lastID,
      name,
      email,
      password
    };
  },
  
  findUserById: async (id) => {
    return await db.get('SELECT * FROM users WHERE id = ?', [id]);
  },
  
  findUserByEmail: async (email) => {
    return await db.get('SELECT * FROM users WHERE email = ?', [email]);
  }
};

Testing a Repository Pattern

// userRepository.js
class UserRepository {
  constructor(db) {
    this.db = db;
  }
  
  async findAll() {
    return await this.db.query('SELECT id, name, email FROM users');
  }
  
  async findById(id) {
    return await this.db.findUserById(id);
  }
  
  async create(userData) {
    return await this.db.createUser(userData);
  }
  
  async update(id, userData) {
    const { name, email } = userData;
    const result = await this.db.query(
      'UPDATE users SET name = $1, email = $2 WHERE id = $3 RETURNING id, name, email',
      [name, email, id]
    );
    return result.rows[0];
  }
  
  async delete(id) {
    await this.db.query('DELETE FROM users WHERE id = $1', [id]);
  }
}

module.exports = UserRepository;

// userRepository.test.js
const UserRepository = require('../repositories/userRepository');
const db = require('../db');

describe('UserRepository', () => {
  let userRepository;
  
  beforeAll(async () => {
    await db.connect();
  });
  
  afterAll(async () => {
    await db.disconnect();
  });
  
  beforeEach(async () => {
    await db.clear();
    userRepository = new UserRepository(db);
  });
  
  test('creates and finds a user', async () => {
    // Create a user
    const userData = {
      name: 'Test User',
      email: 'test@example.com',
      password: 'password123'
    };
    
    const createdUser = await userRepository.create(userData);
    expect(createdUser.id).toBeDefined();
    expect(createdUser.name).toBe(userData.name);
    
    // Find the user by ID
    const foundUser = await userRepository.findById(createdUser.id);
    expect(foundUser).toBeTruthy();
    expect(foundUser.id).toBe(createdUser.id);
    expect(foundUser.email).toBe(userData.email);
  });
  
  test('updates a user', async () => {
    // Create a user
    const user = await userRepository.create({
      name: 'Original Name',
      email: 'original@example.com',
      password: 'password'
    });
    
    // Update the user
    const updateData = {
      name: 'Updated Name',
      email: 'updated@example.com'
    };
    
    const updatedUser = await userRepository.update(user.id, updateData);
    expect(updatedUser.name).toBe(updateData.name);
    expect(updatedUser.email).toBe(updateData.email);
    
    // Verify the update persisted
    const foundUser = await userRepository.findById(user.id);
    expect(foundUser.name).toBe(updateData.name);
  });
  
  test('deletes a user', async () => {
    // Create a user
    const user = await userRepository.create({
      name: 'To Delete',
      email: 'delete@example.com',
      password: 'password'
    });
    
    // Verify user exists
    let foundUser = await userRepository.findById(user.id);
    expect(foundUser).toBeTruthy();
    
    // Delete the user
    await userRepository.delete(user.id);
    
    // Verify user no longer exists
    foundUser = await userRepository.findById(user.id);
    expect(foundUser).toBeNull();
  });
  
  test('finds all users', async () => {
    // Create some users
    await userRepository.create({
      name: 'User 1',
      email: 'user1@example.com',
      password: 'password1'
    });
    
    await userRepository.create({
      name: 'User 2',
      email: 'user2@example.com',
      password: 'password2'
    });
    
    // Find all users
    const users = await userRepository.findAll();
    expect(users).toBeInstanceOf(Array);
    expect(users.length).toBe(2);
    
    // Verify user properties
    expect(users.some(u => u.email === 'user1@example.com')).toBe(true);
    expect(users.some(u => u.email === 'user2@example.com')).toBe(true);
  });
});

// With PostgreSQL
beforeEach(async () => {
  await db.query('BEGIN');
});

afterEach(async () => {
  await db.query('ROLLBACK');
});

Testing External Service Integrations

Many applications rely on external services like payment processors, email delivery services, or third-party APIs. Testing these integrations requires special techniques.

Approaches to Testing External Services

1. Use real services in test environments: Most realistic but can be costly and slower
2. Use sandbox/test environments provided by the service: Good balance of realism and control
3. Mock the external service: Fastest and most controlled, but less realistic

Mocking External HTTP Services with Nock

// weatherService.js
const axios = require('axios');

class WeatherService {
  constructor(apiKey) {
    this.apiKey = apiKey;
    this.baseUrl = 'https://api.weatherapi.com/v1';
  }
  
  async getCurrentWeather(city) {
    try {
      const response = await axios.get(`${this.baseUrl}/current.json`, {
        params: {
          key: this.apiKey,
          q: city
        }
      });
      
      const { current, location } = response.data;
      
      return {
        city: location.name,
        country: location.country,
        temperature: current.temp_c,
        condition: current.condition.text,
        humidity: current.humidity,
        windSpeed: current.wind_kph
      };
    } catch (error) {
      if (error.response && error.response.status === 400) {
        throw new Error(`Invalid city: ${city}`);
      }
      throw new Error(`Failed to fetch weather data: ${error.message}`);
    }
  }
  
  async getForecast(city, days = 3) {
    try {
      const response = await axios.get(`${this.baseUrl}/forecast.json`, {
        params: {
          key: this.apiKey,
          q: city,
          days
        }
      });
      
      const { forecast, location } = response.data;
      
      return {
        city: location.name,
        country: location.country,
        forecast: forecast.forecastday.map(day => ({
          date: day.date,
          maxTemp: day.day.maxtemp_c,
          minTemp: day.day.mintemp_c,
          condition: day.day.condition.text
        }))
      };
    } catch (error) {
      if (error.response && error.response.status === 400) {
        throw new Error(`Invalid city: ${city}`);
      }
      throw new Error(`Failed to fetch forecast data: ${error.message}`);
    }
  }
}

module.exports = WeatherService;

// weatherService.test.js
const nock = require('nock');
const WeatherService = require('../services/weatherService');

describe('WeatherService', () => {
  const apiKey = 'test-api-key';
  let weatherService;
  
  beforeEach(() => {
    weatherService = new WeatherService(apiKey);
    
    // Disable real HTTP requests
    nock.disableNetConnect();
  });
  
  afterEach(() => {
    // Clean up nock
    nock.cleanAll();
    
    // Re-enable real HTTP requests
    nock.enableNetConnect();
  });
  
  describe('getCurrentWeather', () => {
    test('fetches current weather for a valid city', async () => {
      // Mock API response
      nock('https://api.weatherapi.com')
        .get('/v1/current.json')
        .query({
          key: apiKey,
          q: 'London'
        })
        .reply(200, {
          location: {
            name: 'London',
            country: 'United Kingdom'
          },
          current: {
            temp_c: 15.5,
            condition: {
              text: 'Partly cloudy'
            },
            humidity: 76,
            wind_kph: 10.8
          }
        });
      
      const weather = await weatherService.getCurrentWeather('London');
      
      expect(weather).toEqual({
        city: 'London',
        country: 'United Kingdom',
        temperature: 15.5,
        condition: 'Partly cloudy',
        humidity: 76,
        windSpeed: 10.8
      });
    });
    
    test('handles API error for invalid city', async () => {
      // Mock API error response
      nock('https://api.weatherapi.com')
        .get('/v1/current.json')
        .query({
          key: apiKey,
          q: 'NonExistentCity'
        })
        .reply(400, {
          error: {
            code: 1006,
            message: 'No matching location found.'
          }
        });
      
      await expect(weatherService.getCurrentWeather('NonExistentCity'))
        .rejects.toThrow('Invalid city: NonExistentCity');
    });
    
    test('handles network failures', async () => {
      // Mock network failure
      nock('https://api.weatherapi.com')
        .get('/v1/current.json')
        .query({
          key: apiKey,
          q: 'London'
        })
        .replyWithError('Network error');
      
      await expect(weatherService.getCurrentWeather('London'))
        .rejects.toThrow('Failed to fetch weather data');
    });
  });
  
  describe('getForecast', () => {
    test('fetches forecast for a valid city and days', async () => {
      // Mock API response
      nock('https://api.weatherapi.com')
        .get('/v1/forecast.json')
        .query({
          key: apiKey,
          q: 'London',
          days: 3
        })
        .reply(200, {
          location: {
            name: 'London',
            country: 'United Kingdom'
          },
          forecast: {
            forecastday: [
              {
                date: '2023-05-01',
                day: {
                  maxtemp_c: 18.5,
                  mintemp_c: 10.2,
                  condition: {
                    text: 'Sunny'
                  }
                }
              },
              {
                date: '2023-05-02',
                day: {
                  maxtemp_c: 17.8,
                  mintemp_c: 9.7,
                  condition: {
                    text: 'Partly cloudy'
                  }
                }
              },
              {
                date: '2023-05-03',
                day: {
                  maxtemp_c: 16.5,
                  mintemp_c: 11.0,
                  condition: {
                    text: 'Light rain'
                  }
                }
              }
            ]
          }
        });
      
      const forecast = await weatherService.getForecast('London');
      
      expect(forecast).toEqual({
        city: 'London',
        country: 'United Kingdom',
        forecast: [
          {
            date: '2023-05-01',
            maxTemp: 18.5,
            minTemp: 10.2,
            condition: 'Sunny'
          },
          {
            date: '2023-05-02',
            maxTemp: 17.8,
            minTemp: 9.7,
            condition: 'Partly cloudy'
          },
          {
            date: '2023-05-03',
            maxTemp: 16.5,
            minTemp: 11.0,
            condition: 'Light rain'
          }
        ]
      });
    });
    
    test('uses the specified number of days', async () => {
      // Mock API response
      nock('https://api.weatherapi.com')
        .get('/v1/forecast.json')
        .query({
          key: apiKey,
          q: 'London',
          days: 5
        })
        .reply(200, {
          location: {
            name: 'London',
            country: 'United Kingdom'
          },
          forecast: {
            forecastday: Array(5).fill().map((_, i) => ({
              date: `2023-05-0${i+1}`,
              day: {
                maxtemp_c: 18.0,
                mintemp_c: 10.0,
                condition: {
                  text: 'Sunny'
                }
              }
            }))
          }
        });
      
      const forecast = await weatherService.getForecast('London', 5);
      
      expect(forecast.forecast.length).toBe(5);
    });
  });
});

Testing with Real External Services

For some critical integrations, you might want to test against real services:

// paymentService.integration.test.js
const PaymentService = require('../services/paymentService');
const stripeTestKey = process.env.STRIPE_TEST_KEY;

// Skip these tests if no test key is provided
const itWithStripe = stripeTestKey ? it : it.skip;

describe('PaymentService Integration', () => {
  let paymentService;
  
  beforeEach(() => {
    // Use Stripe test key
    paymentService = new PaymentService(stripeTestKey);
  });
  
  itWithStripe('creates a charge for a valid card', async () => {
    const charge = await paymentService.createCharge({
      amount: 2000, // $20.00
      currency: 'usd',
      source: 'tok_visa', // Test token for a valid Visa card
      description: 'Test charge'
    });
    
    expect(charge.id).toBeDefined();
    expect(charge.amount).toBe(2000);
    expect(charge.status).toBe('succeeded');
  });
  
  itWithStripe('handles declined cards', async () => {
    await expect(paymentService.createCharge({
      amount: 2000,
      currency: 'usd',
      source: 'tok_chargeDeclined', // Test token for a decline
      description: 'Test charge'
    })).rejects.toThrow('Your card was declined');
  });
  
  itWithStripe('handles expired cards', async () => {
    await expect(paymentService.createCharge({
      amount: 2000,
      currency: 'usd',
      source: 'tok_chargeDeclinedExpiredCard', // Test token for expired card
      description: 'Test charge'
    })).rejects.toThrow('Your card has expired');
  });
});

// Using nock's recording feature
const nock = require('nock');
const fs = require('fs');
const path = require('path');

// Enable recording
nock.recorder.rec({
  dont_print: true,
  output_objects: true
});

// Run test with real API
test('fetches weather data', async () => {
  const weatherService = new WeatherService(realApiKey);
  const result = await weatherService.getCurrentWeather('London');
  expect(result).toBeDefined();
});

// Save recorded API calls
afterAll(() => {
  const nockCalls = nock.recorder.play();
  fs.writeFileSync(
    path.join(__dirname, 'fixtures', 'weather-api-calls.json'),
    JSON.stringify(nockCalls, null, 2)
  );
});

// Load recorded API calls
beforeAll(() => {
  const nockCalls = JSON.parse(
    fs.readFileSync(path.join(__dirname, 'fixtures', 'weather-api-calls.json'))
  );
  
  nockCalls.forEach(call => {
    nock(call.scope)
      .get(call.path)
      .reply(call.status, call.response);
  });
});

Testing Middleware and Authentication

Middleware, especially authentication and authorization middleware, are critical integration points in web applications.

Testing Express Middleware

// auth.middleware.js
const jwt = require('jsonwebtoken');

function authMiddleware(req, res, next) {
  const authHeader = req.headers.authorization;
  
  if (!authHeader || !authHeader.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'Unauthorized: Missing or invalid token format' });
  }
  
  const token = authHeader.split(' ')[1];
  
  try {
    const decoded = jwt.verify(token, process.env.JWT_SECRET);
    req.user = decoded;
    next();
  } catch (error) {
    return res.status(401).json({ error: 'Unauthorized: Invalid token' });
  }
}

module.exports = authMiddleware;

// auth.middleware.test.js
const authMiddleware = require('../middleware/auth.middleware');
const jwt = require('jsonwebtoken');

// Mock environment variable
process.env.JWT_SECRET = 'test-secret';

describe('Auth Middleware', () => {
  let req, res, next;
  
  beforeEach(() => {
    // Reset mocks for each test
    req = {
      headers: {}
    };
    
    res = {
      status: jest.fn().mockReturnThis(),
      json: jest.fn()
    };
    
    next = jest.fn();
  });
  
  test('allows request with valid token', () => {
    // Create a valid token
    const user = { id: 123, email: 'test@example.com' };
    const token = jwt.sign(user, process.env.JWT_SECRET);
    
    req.headers.authorization = `Bearer ${token}`;
    
    authMiddleware(req, res, next);
    
    // Middleware should call next() and add user to req
    expect(next).toHaveBeenCalled();
    expect(req.user).toBeDefined();
    expect(req.user.id).toBe(user.id);
    expect(req.user.email).toBe(user.email);
  });
  
  test('rejects request with missing authorization header', () => {
    authMiddleware(req, res, next);
    
    expect(res.status).toHaveBeenCalledWith(401);
    expect(res.json).toHaveBeenCalledWith(
      expect.objectContaining({ error: expect.stringContaining('Unauthorized') })
    );
    expect(next).not.toHaveBeenCalled();
  });
  
  test('rejects request with malformed authorization header', () => {
    req.headers.authorization = 'InvalidFormat';
    
    authMiddleware(req, res, next);
    
    expect(res.status).toHaveBeenCalledWith(401);
    expect(next).not.toHaveBeenCalled();
  });
  
  test('rejects request with invalid token', () => {
    req.headers.authorization = 'Bearer invalid.token.here';
    
    authMiddleware(req, res, next);
    
    expect(res.status).toHaveBeenCalledWith(401);
    expect(next).not.toHaveBeenCalled();
  });
  
  test('rejects request with expired token', () => {
    // Create a token that expired in the past
    const user = { id: 123, email: 'test@example.com' };
    const token = jwt.sign(user, process.env.JWT_SECRET, { expiresIn: '-10s' });
    
    req.headers.authorization = `Bearer ${token}`;
    
    authMiddleware(req, res, next);
    
    expect(res.status).toHaveBeenCalledWith(401);
    expect(next).not.toHaveBeenCalled();
  });
});

Testing Authentication Flow with Supertest

// auth.routes.test.js
const request = require('supertest');
const app = require('../app');
const jwt = require('jsonwebtoken');
const db = require('../db');
const bcrypt = require('bcrypt');

describe('Authentication Routes', () => {
  beforeAll(async () => {
    await db.connect();
  });
  
  afterAll(async () => {
    await db.disconnect();
  });
  
  beforeEach(async () => {
    await db.clear();
  });
  
  describe('POST /api/auth/register', () => {
    test('registers a new user', async () => {
      const userData = {
        name: 'Test User',
        email: 'test@example.com',
        password: 'password123'
      };
      
      const response = await request(app)
        .post('/api/auth/register')
        .send(userData)
        .expect('Content-Type', /json/)
        .expect(201);
      
      expect(response.body.user).toBeDefined();
      expect(response.body.user.name).toBe(userData.name);
      expect(response.body.user.email).toBe(userData.email);
      expect(response.body.token).toBeDefined();
      
      // Verify user was saved to database
      const user = await db.findUserByEmail(userData.email);
      expect(user).toBeTruthy();
      
      // Password should be hashed
      expect(user.password).not.toBe(userData.password);
      const isValidPassword = await bcrypt.compare(userData.password, user.password);
      expect(isValidPassword).toBe(true);
    });
    
    test('prevents duplicate email registration', async () => {
      // Create a user first
      await request(app)
        .post('/api/auth/register')
        .send({
          name: 'Existing User',
          email: 'existing@example.com',
          password: 'password123'
        });
      
      // Try to register with the same email
      const response = await request(app)
        .post('/api/auth/register')
        .send({
          name: 'New User',
          email: 'existing@example.com',
          password: 'newpassword'
        })
        .expect(400);
      
      expect(response.body.error).toBeDefined();
      expect(response.body.error).toContain('Email already in use');
    });
  });
  
  describe('POST /api/auth/login', () => {
    beforeEach(async () => {
      // Create a test user
      const hashedPassword = await bcrypt.hash('password123', 10);
      await db.createUser({
        name: 'Test User',
        email: 'test@example.com',
        password: hashedPassword
      });
    });
    
    test('logs in with valid credentials', async () => {
      const response = await request(app)
        .post('/api/auth/login')
        .send({
          email: 'test@example.com',
          password: 'password123'
        })
        .expect('Content-Type', /json/)
        .expect(200);
      
      expect(response.body.user).toBeDefined();
      expect(response.body.user.name).toBe('Test User');
      expect(response.body.user.email).toBe('test@example.com');
      expect(response.body.token).toBeDefined();
      
      // Verify token is valid
      const decoded = jwt.verify(response.body.token, process.env.JWT_SECRET);
      expect(decoded.email).toBe('test@example.com');
    });
    
    test('rejects invalid password', async () => {
      const response = await request(app)
        .post('/api/auth/login')
        .send({
          email: 'test@example.com',
          password: 'wrongpassword'
        })
        .expect(401);
      
      expect(response.body.error).toBeDefined();
      expect(response.body.token).toBeUndefined();
    });
    
    test('rejects non-existent user', async () => {
      const response = await request(app)
        .post('/api/auth/login')
        .send({
          email: 'nonexistent@example.com',
          password: 'password123'
        })
        .expect(401);
      
      expect(response.body.error).toBeDefined();
      expect(response.body.token).toBeUndefined();
    });
  });
  
  describe('GET /api/auth/me', () => {
    let token;
    let userId;
    
    beforeEach(async () => {
      // Create a test user
      const hashedPassword = await bcrypt.hash('password123', 10);
      const user = await db.createUser({
        name: 'Test User',
        email: 'test@example.com',
        password: hashedPassword
      });
      
      userId = user.id;
      
      // Create a valid token
      token = jwt.sign(
        { id: user.id, email: user.email },
        process.env.JWT_SECRET
      );
    });
    
    test('returns user data for authenticated request', async () => {
      const response = await request(app)
        .get('/api/auth/me')
        .set('Authorization', `Bearer ${token}`)
        .expect('Content-Type', /json/)
        .expect(200);
      
      expect(response.body.id).toBe(userId);
      expect(response.body.name).toBe('Test User');
      expect(response.body.email).toBe('test@example.com');
      expect(response.body.password).toBeUndefined(); // Password should not be returned
    });
    
    test('rejects unauthenticated request', async () => {
      await request(app)
        .get('/api/auth/me')
        .expect(401);
    });
    
    test('rejects request with invalid token', async () => {
      await request(app)
        .get('/api/auth/me')
        .set('Authorization', 'Bearer invalid.token.here')
        .expect(401);
    });
  });
});

Integration Testing Best Practices

Test Real Integration Points

• Focus on testing the actual interfaces between components
• Don't test implementation details that should be covered by unit tests
• Test the contracts and promises made between components

Use Realistic Test Data

• Test data should reflect real-world scenarios
• Consider using data factories or fixtures to generate consistent test data
• Include edge cases in your test data (empty sets, large datasets)

Control External Dependencies

• Use predictable test environments for external services
• Consider using test doubles for unreliable or slow external services
• Document any assumptions about external systems

Clean Up After Tests

• Always reset the system to a known state after each test
• Use beforeEach/afterEach hooks to clean up databases, files, and other resources
• Consider using transactions for database tests to improve performance

Group Related Tests

• Organize tests by feature or component being tested
• Use describe blocks to create logical test hierarchies
• Keep setup code close to the tests that use it

Test Error Paths

• Test how components handle failures from their dependencies
• Verify error handling and recovery mechanisms
• Include tests for timeout handling, network failures, and other edge cases

Verify State Changes

• Check that operations have the expected side effects
• Verify data persistence and retrieval work correctly
• Test that events and messages are properly propagated

// Good test names
test('returns 404 when product does not exist');
test('saves order to database when payment succeeds');
test('sends confirmation email after order completion');

// Poor test names
test('product API');
test('order test');
test('test email functionality');

Integration Testing Challenges and Solutions

Slow Tests

Challenge: Integration tests are typically slower than unit tests, which can slow down your development cycle.

Solutions:

• Run integration tests less frequently (e.g., only before commits or pushes)
• Parallelize tests when possible
• Use faster alternatives where appropriate (in-memory databases, mocked services)
• Keep your test suite organized so you can run specific subsets of tests

Flaky Tests

Challenge: Tests that sometimes pass and sometimes fail without code changes.

Solutions:

• Eliminate timing dependencies with proper waiting mechanisms
• Control randomness by setting fixed seeds
• Isolate tests from each other
• Avoid depending on external systems that may change or be unavailable
• Use retry mechanisms for genuinely non-deterministic scenarios

// Example of handling non-deterministic behaviors
test('message is processed within a reasonable time', async () => {
  // Send message
  await messageBus.send({ id: '123', payload: 'test' });
  
  // Poll for results with timeout
  const checkInterval = 100; // ms
  const maxAttempts = 10;
  
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    const result = await db.getProcessedMessage('123');
    
    if (result) {
      expect(result.status).toBe('completed');
      return; // Test passes
    }
    
    // Wait before trying again
    await new Promise(resolve => setTimeout(resolve, checkInterval));
  }
  
  // If we get here, the message wasn't processed in time
  throw new Error('Message was not processed within the time limit');
});

Test Data Management

Challenge: Creating and managing realistic test data for integration tests.

Solutions:

• Use factories or fixtures to generate consistent test data
• Consider using snapshot data from production (scrubbed of sensitive information)
• Seed the database with a known state before tests
• Use database transactions to reset state between tests

// Example user factory
function createTestUser(overrides = {}) {
  return {
    name: 'Test User',
    email: `test-${Math.random().toString(36).substring(2)}@example.com`,
    password: 'password123',
    role: 'user',
    ...overrides
  };
}

// Example usage
test('user can change their email', async () => {
  // Create a test user with factory
  const user = await userRepository.create(createTestUser());
  
  // Test email change
  await userService.updateEmail(user.id, 'newemail@example.com');
  
  // Verify change
  const updatedUser = await userRepository.findById(user.id);
  expect(updatedUser.email).toBe('newemail@example.com');
});

Environment Dependencies

Challenge: Tests may depend on specific environment configurations or variables.

Solutions:

• Use configuration files specific to testing
• Set up environment variables programmatically in tests
• Document environment requirements for running tests
• Use containers or VMs to provide consistent test environments

// Example environment setup for tests
// test/setup.js
process.env.NODE_ENV = 'test';
process.env.DB_CONNECTION = 'mongodb://localhost:27017/test-db';
process.env.JWT_SECRET = 'test-secret-key';
process.env.API_RATE_LIMIT = '100';

// In your test file
beforeAll(() => {
  // Save original env
  originalEnv = { ...process.env };
  
  // Set env for this test
  process.env.FEATURE_FLAG_NEW_UI = 'true';
});

afterAll(() => {
  // Restore original env
  process.env = originalEnv;
});

Continuous Integration for Integration Tests

Integration tests are a vital part of your continuous integration (CI) pipeline, helping catch integration issues early.

Setting Up CI for Integration Tests

• Run integration tests on every significant code change
• Use separate database instances for CI tests
• Consider using Docker or similar tools to create isolated test environments
• Set up proper cleanup processes for test resources

Example GitHub Actions CI Configuration

name: Integration Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest

    services:
      # Set up PostgreSQL service for tests
      postgres:
        image: postgres:13
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test_db
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
      
      # Set up Redis for tests
      redis:
        image: redis:6
        ports:
          - 6379:6379
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v2

      - name: Set up Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '16'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Set up database schema
        run: npm run db:migrate

      - name: Run unit tests
        run: npm run test:unit

      - name: Run integration tests
        run: npm run test:integration
        env:
          NODE_ENV: test
          DB_HOST: localhost
          DB_PORT: 5432
          DB_USER: postgres
          DB_PASSWORD: postgres
          DB_NAME: test_db
          REDIS_URL: redis://localhost:6379
          JWT_SECRET: test-jwt-secret

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v2
        with:
          name: test-results
          path: test-results/

Practice Activities

Further Reading

• Integration Tests by Martin Fowler
• Supertest Documentation
• Nock Documentation
• React Testing Library
• Database Integration Testing with Prisma
• Jest MongoDB Documentation