Handling JSON Data in Express.js

What is JSON?

JSON (JavaScript Object Notation) is a lightweight data-interchange format that's easy for humans to read and write, and easy for machines to parse and generate. It's become the standard format for data exchange in web applications due to its simplicity and compatibility with JavaScript.

{
  "name": "Jane Doe",
  "age": 28,
  "isActive": true,
  "skills": ["JavaScript", "Node.js", "Express"],
  "address": {
    "city": "San Francisco",
    "state": "CA",
    "zipCode": "94105"
  }
}

JSON uses a simple structure of key-value pairs, similar to JavaScript objects, but with some important differences:

• Property names must be in double quotes
• Values can only be strings, numbers, booleans, arrays, objects, or null
• No functions, dates (they're stored as strings), undefined, or comments

JSON in Modern Web Development

JSON has become the dominant data format for web APIs for several compelling reasons:

Think of JSON as the universal translator in the Star Trek universe - enabling different systems to communicate seamlessly with each other regardless of their internal implementation.

Real-World JSON Use Cases

• API Communication: Most RESTful and GraphQL APIs use JSON for data exchange
• Configuration Files: package.json, tsconfig.json, etc.
• Database Storage: MongoDB and other NoSQL databases store data in JSON-like formats
• Browser Storage: localStorage and sessionStorage work well with JSON
• Mobile Applications: Communication between native apps and web servers

Express.js and JSON

Express makes it easy to handle JSON data in your applications through built-in middleware and response methods.

Built-in Express Middleware for JSON

Express provides the express.json() middleware which parses incoming requests with JSON payloads. It sets req.body to the parsed object so you can easily access the data in your route handlers.

const express = require('express');
const app = express();

// Add JSON parsing middleware
app.use(express.json());

// Now req.body will be populated for JSON requests
app.post('/api/users', (req, res) => {
  console.log(req.body); // Access the parsed JSON data
  // Process the data...
  res.json({ success: true });
});

app.listen(3000, () => {
  console.log('Server running on port 3000');
});

app.use(express.json({
  limit: '1mb',        // Limit payload size (default: '100kb')
  strict: true,        // Only accept arrays and objects (default: true)
  reviver: null,       // Function for transforming parsed values
  type: 'application/json'  // Content-Type to parse (default)
}));

Sending JSON Responses

Express provides an elegant way to send JSON responses with the res.json() method.

// Simple object response
app.get('/api/user', (req, res) => {
  res.json({
    id: 123,
    name: 'Alice Smith',
    role: 'Developer'
  });
});

// Array response
app.get('/api/users', (req, res) => {
  res.json([
    { id: 123, name: 'Alice Smith' },
    { id: 456, name: 'Bob Johnson' }
  ]);
});

// With status code
app.post('/api/login', (req, res) => {
  // Authentication logic...
  if (authenticated) {
    res.status(200).json({ success: true, token: 'abc123' });
  } else {
    res.status(401).json({ success: false, message: 'Invalid credentials' });
  }
});

The res.json() method automatically:

• Sets Content-Type: application/json header
• Calls JSON.stringify() on your data
• Sends the response with appropriate encoding

Working with Complex JSON Data

Real-world applications often require handling complex nested JSON structures and arrays.

app.post('/api/orders', (req, res) => {
  const { customer, items, shipping } = req.body;
  
  // Access nested properties
  const customerName = customer.name;
  const shippingZip = shipping.address.zipCode;
  
  // Process array of items
  const totalPrice = items.reduce((sum, item) => {
    return sum + (item.price * item.quantity);
  }, 0);
  
  // Create new order...
  res.json({
    orderId: 'ORD-12345',
    totalPrice,
    estimatedDelivery: '3-5 business days'
  });
});

Common JSON Operations in Express

Data Validation

Always validate incoming JSON data to ensure it has the expected structure and values.

app.post('/api/users', (req, res) => {
  const { name, email, age } = req.body;
  
  // Check required fields
  if (!name || !email) {
    return res.status(400).json({ 
      error: 'Name and email are required fields' 
    });
  }
  
  // Validate email format
  const emailRegex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
  if (!emailRegex.test(email)) {
    return res.status(400).json({ 
      error: 'Invalid email format' 
    });
  }
  
  // Validate age if provided
  if (age !== undefined && (typeof age !== 'number' || age < 0)) {
    return res.status(400).json({ 
      error: 'Age must be a positive number' 
    });
  }
  
  // Process valid data...
  res.json({ success: true });
});

JSON Transformation and Modification

Often you'll need to transform, filter, or restructure JSON data before sending responses.

app.get('/api/products', (req, res) => {
  // Fetch product data from database
  const products = [
    { id: 1, name: 'Product 1', price: 99.99, inventory: 50, 
      internal: { sku: 'ABC123', cost: 75.00 } },
    { id: 2, name: 'Product 2', price: 149.99, inventory: 30, 
      internal: { sku: 'DEF456', cost: 100.00 } }
  ];
  
  // Transform data: filter out internal data, add calculated fields
  const transformedProducts = products.map(product => ({
    id: product.id,
    name: product.name,
    price: product.price,
    available: product.inventory > 0,
    // Format price as currency string
    formattedPrice: `$${product.price.toFixed(2)}`
  }));
  
  res.json(transformedProducts);
});

Error Handling with JSON

Providing consistent and helpful JSON error responses improves developer experience for API consumers.

// Error handling middleware
app.use((err, req, res, next) => {
  console.error(err);
  
  // Set appropriate status code
  const statusCode = err.statusCode || 500;
  
  // Construct error response
  const errorResponse = {
    success: false,
    error: {
      message: err.message || 'Internal Server Error',
      code: err.code || 'INTERNAL_ERROR'
    }
  };
  
  // Include stack trace in development only
  if (process.env.NODE_ENV === 'development') {
    errorResponse.error.stack = err.stack;
  }
  
  res.status(statusCode).json(errorResponse);
});

For consistent error handling, many developers create custom error classes:

class ApiError extends Error {
  constructor(message, statusCode, code) {
    super(message);
    this.statusCode = statusCode;
    this.code = code;
    this.name = this.constructor.name;
    Error.captureStackTrace(this, this.constructor);
  }
}

// Usage example:
app.get('/api/users/:id', (req, res, next) => {
  try {
    const user = findUser(req.params.id);
    if (!user) {
      throw new ApiError('User not found', 404, 'USER_NOT_FOUND');
    }
    res.json(user);
  } catch (err) {
    next(err);
  }
});

Working with JSON in Real-world Projects

E-commerce API Example

Let's examine a more comprehensive e-commerce product API endpoint that handles a variety of JSON operations:

const express = require('express');
const router = express.Router();

// Get all products with filtering and pagination
router.get('/products', (req, res) => {
  try {
    // Parse query parameters from URL
    const { 
      category, 
      minPrice, 
      maxPrice, 
      sort = 'price',
      order = 'asc',
      page = 1,
      limit = 20
    } = req.query;
    
    // Convert numeric strings to numbers
    const parsedMinPrice = minPrice ? parseFloat(minPrice) : undefined;
    const parsedMaxPrice = maxPrice ? parseFloat(maxPrice) : undefined;
    const parsedPage = parseInt(page);
    const parsedLimit = parseInt(limit);
    
    // Validate pagination parameters
    if (isNaN(parsedPage) || parsedPage < 1) {
      return res.status(400).json({
        success: false,
        error: 'Page must be a positive number'
      });
    }
    
    // Retrieve products from database (mock data here)
    let products = [
      // Simulated database results...
      { id: 1, name: 'Smartphone', category: 'electronics', price: 699.99 },
      { id: 2, name: 'Laptop', category: 'electronics', price: 1299.99 },
      { id: 3, name: 'Running Shoes', category: 'clothing', price: 89.99 }
    ];
    
    // Apply filters
    if (category) {
      products = products.filter(p => p.category === category);
    }
    
    if (parsedMinPrice !== undefined) {
      products = products.filter(p => p.price >= parsedMinPrice);
    }
    
    if (parsedMaxPrice !== undefined) {
      products = products.filter(p => p.price <= parsedMaxPrice);
    }
    
    // Apply sorting
    products.sort((a, b) => {
      let valueA = a[sort];
      let valueB = b[sort];
      
      if (typeof valueA === 'string') {
        valueA = valueA.toLowerCase();
        valueB = valueB.toLowerCase();
      }
      
      if (order.toLowerCase() === 'asc') {
        return valueA > valueB ? 1 : -1;
      } else {
        return valueA < valueB ? 1 : -1;
      }
    });
    
    // Apply pagination
    const startIndex = (parsedPage - 1) * parsedLimit;
    const endIndex = startIndex + parsedLimit;
    const paginatedProducts = products.slice(startIndex, endIndex);
    
    // Compose response with metadata
    res.json({
      success: true,
      pagination: {
        total: products.length,
        page: parsedPage,
        limit: parsedLimit,
        pages: Math.ceil(products.length / parsedLimit)
      },
      data: paginatedProducts
    });
  } catch (error) {
    console.error('Error fetching products:', error);
    res.status(500).json({
      success: false,
      error: 'Failed to retrieve products'
    });
  }
});

module.exports = router;

Advanced JSON Techniques

JSON Streaming for Large Datasets

When dealing with very large JSON datasets, streaming can prevent memory issues:

const { Transform } = require('stream');
const express = require('express');
const app = express();

app.get('/api/large-dataset', (req, res) => {
  // Set appropriate headers
  res.setHeader('Content-Type', 'application/json');
  
  // Start JSON array
  res.write('[\n');
  
  // Initialize counter and track if it's the first item
  let count = 0;
  let isFirst = true;
  
  // Create transform stream for data
  const jsonTransform = new Transform({
    objectMode: true,
    transform(chunk, encoding, callback) {
      // Add comma if not the first item
      const prefix = isFirst ? '' : ',\n';
      isFirst = false;
      
      // Convert chunk to JSON and push to stream
      this.push(prefix + JSON.stringify(chunk));
      callback();
    }
  });
  
  // Pipe through transform stream to response
  jsonTransform.pipe(res, { end: false });
  
  // Simulate streaming data source
  const interval = setInterval(() => {
    if (count >= 1000) {
      clearInterval(interval);
      jsonTransform.end();
      
      // End JSON array and response
      res.end('\n]');
      return;
    }
    
    // Generate next data item
    const item = {
      id: count,
      name: `Item ${count}`,
      timestamp: new Date().toISOString()
    };
    
    // Push to transform stream
    jsonTransform.write(item);
    count++;
  }, 5);
});

This technique allows you to send potentially millions of records without loading them all into memory at once.

JSON Performance Considerations

Compression

Always use compression for JSON responses to reduce bandwidth and improve load times.

const express = require('express');
const compression = require('compression');
const app = express();

// Add compression middleware
app.use(compression());

// Your routes here...

app.listen(3000);

Size Optimization

Only include necessary data in your JSON responses:

• Remove unnecessary fields
• Use short, but clear property names
• Consider pagination for large data sets
• Use proper error responses to prevent sending large error stacks

JSON Security Considerations

Input Validation

Always validate user input to prevent injection attacks and other security vulnerabilities.

Size Limits

Set appropriate size limits to prevent DoS attacks:

app.use(express.json({ limit: '1mb' }));

Content-Type Verification

Verify the Content-Type header is correct before processing JSON:

app.use((req, res, next) => {
  if (req.method === 'POST' && !req.is('application/json')) {
    return res.status(415).json({ 
      error: 'Content-Type must be application/json' 
    });
  }
  next();
});

Practical Exercises

Additional Resources

Documentation

• Express.js documentation on express.json()
• MDN JSON documentation
• JSON.org - The official JSON syntax

Libraries

• Joi - Schema validation
• Ajv - JSON Schema validation
• Winston - Logging for Node.js