Understanding Request Data in Express
When building web applications, understanding how to retrieve and process user input is essential. Express.js provides several ways to access user data, each designed for different scenarios.
Think of these different input methods as various ways people might communicate with you:
- URL Parameters: Like someone addressing you by name - direct and built into the conversation structure
- Query Parameters: Similar to asking follow-up questions - additional information that refines the main request
- Request Body: Like handing over a detailed document - contains substantial information too complex for simple parameters
- Headers: Akin to the context of a conversation - metadata about how to interpret the main message
- Cookies: Similar to remembering past conversations - maintaining state between interactions
URL Parameters vs. Query Parameters
Before diving into query parameters, let's clarify the difference between URL (route) parameters and query parameters.
URL Components
https://example.com/users/123?sort=name&order=asc
Blue: URL parameter (part of route path)
Green: Query parameters (key-value pairs after ?)
| Feature | URL Parameters | Query Parameters |
|---|---|---|
| Syntax | /users/:id | /users?id=123 |
| Typically used for | Resource identification | Filtering, sorting, pagination |
| Required vs Optional | Usually required | Usually optional |
| Express access | req.params.id | req.query.id |
| Route definition | Explicitly defined | Implicit (no special setup) |
Working with Query Parameters
Query parameters are the key-value pairs that appear after the question mark (?) in a URL. They're useful for filtering, sorting, searching, and pagination.
Accessing Query Parameters
Express provides query parameters through the req.query object - no additional middleware required!
Basic Query Parameter Usage
// URL: /api/products?category=electronics&sort=price&minPrice=100
app.get('/api/products', (req, res) => {
console.log(req.query);
// Output: { category: 'electronics', sort: 'price', minPrice: '100' }
const { category, sort, minPrice } = req.query;
console.log(category); // 'electronics'
console.log(sort); // 'price'
console.log(minPrice); // '100' (string, not number!)
// Use the parameters to filter and sort products...
res.json({ message: 'Products filtered successfully', params: req.query });
});
Important Query Parameter Characteristics
- All query parameter values are strings by default
- You need to manually convert to numbers, booleans, etc.
- Missing parameters are
undefined, not null - Parameter names are case-sensitive
- Array parameters can be passed using repeated keys or special syntax
Advanced Query Parameter Handling
Type Conversion
Always convert query parameters to appropriate types:
Type Conversion for Query Parameters
app.get('/api/products', (req, res) => {
// Extract query parameters
const {
category,
minPrice,
maxPrice,
inStock,
page,
limit
} = req.query;
// Type conversion
const filters = {
category: category, // Keep as string
minPrice: minPrice ? parseFloat(minPrice) : undefined,
maxPrice: maxPrice ? parseFloat(maxPrice) : undefined,
inStock: inStock === 'true', // Convert to boolean
page: page ? parseInt(page, 10) : 1,
limit: limit ? parseInt(limit, 10) : 20
};
// Validation
if (filters.minPrice && isNaN(filters.minPrice)) {
return res.status(400).json({ error: 'minPrice must be a number' });
}
if (filters.page < 1 || isNaN(filters.page)) {
return res.status(400).json({ error: 'page must be a positive number' });
}
// Use converted & validated parameters
console.log(filters);
// { category: 'electronics', minPrice: 100, inStock: true, page: 1, limit: 20 }
// ... fetch and filter products
res.json({ message: 'Success', filters });
});
Handling Array Parameters
Query parameters can also represent arrays using different notation styles:
Array Query Parameters
// URL: /api/products?colors=red&colors=blue&colors=green
// OR: /api/products?colors[]=red&colors[]=blue&colors[]=green
// OR: /api/products?colors=red,blue,green (custom handling)
app.get('/api/products', (req, res) => {
const { colors } = req.query;
// If multiple values with same key are provided
console.log(colors);
// It could be array: ['red', 'blue', 'green']
// Or single value: 'red,blue,green' or just 'red'
// Ensure consistent array handling
let colorsArray;
if (Array.isArray(colors)) {
// Already an array
colorsArray = colors;
} else if (typeof colors === 'string' && colors.includes(',')) {
// Comma-separated string
colorsArray = colors.split(',');
} else if (colors) {
// Single value
colorsArray = [colors];
} else {
// No value provided
colorsArray = [];
}
console.log(colorsArray); // ['red', 'blue', 'green']
// Use the array for filtering...
res.json({ colorsSelected: colorsArray });
});
Real-World Query Parameter Patterns
Pagination
Pagination is one of the most common uses for query parameters:
Implementing Pagination
app.get('/api/articles', (req, res) => {
// Get pagination parameters
const page = parseInt(req.query.page || '1', 10);
const limit = parseInt(req.query.limit || '10', 10);
// Validate pagination parameters
if (page < 1) {
return res.status(400).json({ error: 'Page must be at least 1' });
}
if (limit < 1 || limit > 100) {
return res.status(400).json({ error: 'Limit must be between 1 and 100' });
}
// Calculate offsets
const startIndex = (page - 1) * limit;
const endIndex = page * limit;
// Mock database query
const allArticles = Array.from({ length: 100 }, (_, i) => ({
id: i + 1,
title: `Article ${i + 1}`,
createdAt: new Date()
}));
// Get current page of data
const paginatedArticles = allArticles.slice(startIndex, endIndex);
// Create pagination metadata
const totalArticles = allArticles.length;
const totalPages = Math.ceil(totalArticles / limit);
// Construct response with pagination links
const results = {
data: paginatedArticles,
pagination: {
total: totalArticles,
page,
limit,
pages: totalPages,
hasMore: endIndex < totalArticles
},
links: {
self: `/api/articles?page=${page}&limit=${limit}`,
first: `/api/articles?page=1&limit=${limit}`,
last: `/api/articles?page=${totalPages}&limit=${limit}`,
prev: page > 1 ? `/api/articles?page=${page - 1}&limit=${limit}` : null,
next: page < totalPages ? `/api/articles?page=${page + 1}&limit=${limit}` : null
}
};
res.json(results);
});
Filtering and Searching
Query parameters are ideal for implementing search and filter functionality:
Implementing Search and Filters
app.get('/api/users', (req, res) => {
// Extract search and filter parameters
const {
search,
role,
status,
startDate,
endDate,
sortBy = 'createdAt',
sortOrder = 'desc'
} = req.query;
// Start with full dataset (mock database)
let users = [
{ id: 1, name: 'Alice Johnson', email: 'alice@example.com', role: 'admin', status: 'active', createdAt: '2023-01-15' },
{ id: 2, name: 'Bob Smith', email: 'bob@example.com', role: 'user', status: 'inactive', createdAt: '2023-02-20' },
{ id: 3, name: 'Carol Williams', email: 'carol@example.com', role: 'editor', status: 'active', createdAt: '2023-03-10' }
// ... more users
];
// Apply text search (case-insensitive)
if (search) {
const searchLower = search.toLowerCase();
users = users.filter(user =>
user.name.toLowerCase().includes(searchLower) ||
user.email.toLowerCase().includes(searchLower)
);
}
// Apply role filter
if (role) {
users = users.filter(user => user.role === role);
}
// Apply status filter
if (status) {
users = users.filter(user => user.status === status);
}
// Apply date range filter
if (startDate) {
users = users.filter(user => new Date(user.createdAt) >= new Date(startDate));
}
if (endDate) {
users = users.filter(user => new Date(user.createdAt) <= new Date(endDate));
}
// Apply sorting
users.sort((a, b) => {
const valueA = a[sortBy];
const valueB = b[sortBy];
if (sortOrder.toLowerCase() === 'asc') {
return valueA > valueB ? 1 : -1;
} else {
return valueA < valueB ? 1 : -1;
}
});
// Return filtered and sorted results
res.json({
count: users.length,
data: users,
filters: { search, role, status, startDate, endDate },
sorting: { sortBy, sortOrder }
});
});
Request Body Parsing
While query parameters work well for simple data, request bodies are better for complex or larger data submissions. Express provides middleware to parse different types of request bodies.
What is Body Parsing?
Body parsing is the process of extracting data from the request body and converting it into a JavaScript object that your application can work with. Unlike query parameters, Express doesn't parse request bodies automatically - you need middleware.
Built-in Express Body Parsers
Express includes built-in middleware for parsing JSON and URL-encoded bodies:
Setting Up Body Parsing Middleware
const express = require('express');
const app = express();
// Parse JSON bodies (Content-Type: application/json)
app.use(express.json());
// Parse URL-encoded bodies (Content-Type: application/x-www-form-urlencoded)
app.use(express.urlencoded({ extended: true }));
// Now req.body will be populated in route handlers
app.post('/api/users', (req, res) => {
console.log(req.body); // Access parsed request body
// Process the data...
const { username, email, age } = req.body;
// Validation example
if (!username || !email) {
return res.status(400).json({ error: 'Username and email are required' });
}
// Create new user...
res.status(201).json({
message: 'User created successfully',
user: { username, email, age }
});
});
Body Parser Options
The built-in body parsers accept various options:
// JSON parser options
app.use(express.json({
limit: '1mb', // Maximum request body size (default: '100kb')
strict: true, // Only parse arrays and objects (default: true)
reviver: null, // Function to transform parsed values
type: 'application/json' // Content-Type to parse
}));
// URL-encoded parser options
app.use(express.urlencoded({
extended: true, // Use qs library for parsing (allows nested objects)
limit: '1mb', // Maximum request body size
parameterLimit: 1000, // Max number of parameters
type: 'application/x-www-form-urlencoded' // Content-Type to parse
}));
JSON Body Parsing in Detail
JSON is the most common format for API request bodies. Let's look at how to work with JSON bodies effectively.
Complete JSON Body Handling Example
const express = require('express');
const app = express();
// Configure JSON parser with error handling
app.use(express.json({
limit: '500kb', // Increase size limit
strict: true,
type: ['application/json', '+json'] // Accept more Content-Types
}));
// Error handler for JSON parsing errors
app.use((err, req, res, next) => {
if (err instanceof SyntaxError && err.status === 400 && 'body' in err) {
// Handle JSON parsing error
return res.status(400).json({
error: 'Invalid JSON in request body',
details: err.message
});
}
next(err); // Pass other errors to next handler
});
// POST endpoint that accepts JSON data
app.post('/api/products', (req, res) => {
try {
// Destructure and validate required fields
const { name, price, category, description } = req.body;
if (!name || name.trim() === '') {
return res.status(400).json({ error: 'Product name is required' });
}
if (typeof price !== 'number' || price <= 0) {
return res.status(400).json({ error: 'Price must be a positive number' });
}
if (!category) {
return res.status(400).json({ error: 'Category is required' });
}
// Handle nested data
const { inventory, dimensions, manufacturer } = req.body;
// Defaults for optional fields
const stockLevel = inventory?.quantity || 0;
const isInStock = stockLevel > 0;
// Create database representation of product
const newProduct = {
id: Date.now().toString(),
name,
price,
category,
description: description || '',
inventory: {
quantity: stockLevel,
isInStock,
reorderLevel: inventory?.reorderLevel || 5
},
dimensions: dimensions || { width: 0, height: 0, depth: 0 },
manufacturer: manufacturer || 'Unknown',
createdAt: new Date().toISOString()
};
// Save to database (mock implementation)
// db.products.insert(newProduct);
// Return success response with created product
res.status(201).json({
message: 'Product created successfully',
product: newProduct
});
} catch (error) {
console.error('Error processing product:', error);
res.status(500).json({ error: 'Internal server error' });
}
});
URL-encoded Form Data
URL-encoded form data is common when working with traditional HTML forms. Express can parse this with the urlencoded middleware.
Working with URL-encoded Forms
const express = require('express');
const app = express();
// Configure URL-encoded parser
app.use(express.urlencoded({ extended: true }));
// Form submission endpoint
app.post('/login', (req, res) => {
const { username, password, rememberMe } = req.body;
console.log(username); // e.g., 'john_doe'
console.log(password); // e.g., 'secret123'
console.log(rememberMe); // 'on' if checkbox checked, undefined if not
// Type conversion for checkbox
const shouldRemember = rememberMe === 'on';
// Validate credentials (mock implementation)
if (username === 'john_doe' && password === 'secret123') {
// Success - redirect to dashboard
return res.redirect('/dashboard');
} else {
// Failed login - redirect back to login with error
return res.redirect('/login?error=invalid_credentials');
}
});
Extended Option Explained
The extended: true option allows for rich objects and arrays to be encoded into the URL-encoded format, using the qs library.
Extended URL-encoding Example
<!-- HTML Form with nested data -->
<form action="/api/profile" method="POST">
<input name="user[name]" value="John Doe">
<input name="user[email]" value="john@example.com">
<input name="address[street]" value="123 Main St">
<input name="address[city]" value="New York">
<input name="hobbies[]" value="reading">
<input name="hobbies[]" value="hiking">
<button type="submit">Submit</button>
</form>
// Server-side handling
app.post('/api/profile', (req, res) => {
console.log(req.body);
/* With extended: true, outputs:
{
user: {
name: 'John Doe',
email: 'john@example.com'
},
address: {
street: '123 Main St',
city: 'New York'
},
hobbies: ['reading', 'hiking']
}
*/
/* With extended: false, would output:
{
'user[name]': 'John Doe',
'user[email]': 'john@example.com',
'address[street]': '123 Main St',
'address[city]': 'New York',
'hobbies[]': ['reading', 'hiking']
}
*/
res.send('Profile updated');
});
Best Practices for Request Data Handling
Validation
Always validate and sanitize user input from query parameters and request bodies:
Using express-validator for Validation
const express = require('express');
const { body, query, validationResult } = require('express-validator');
const app = express();
app.use(express.json());
// Validation chain for user creation
app.post('/api/users',
// Validate and sanitize request body
[
body('username')
.trim()
.isLength({ min: 3, max: 20 })
.withMessage('Username must be between 3-20 characters')
.isAlphanumeric()
.withMessage('Username must contain only letters and numbers'),
body('email')
.isEmail()
.withMessage('Must provide a valid email')
.normalizeEmail(),
body('password')
.isLength({ min: 6 })
.withMessage('Password must be at least 6 characters long')
.matches(/\d/)
.withMessage('Password must contain at least one number'),
body('age')
.optional()
.isInt({ min: 18, max: 120 })
.withMessage('Age must be between 18-120')
],
(req, res) => {
// Check for validation errors
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
}
// Process validated data
const { username, email, password, age } = req.body;
// Create user logic...
res.status(201).json({
message: 'User created successfully',
user: { username, email, age }
});
}
);
Consistent Error Handling
Create consistent error responses for both query parameter and body validation:
Consistent Error Handler
// Centralized error handling middleware
function handleValidationErrors(req, res, next) {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({
success: false,
errors: errors.array().map(err => ({
parameter: err.param,
location: err.location, // 'body', 'query', 'params', etc.
message: err.msg,
value: err.value
}))
});
}
next();
}
// Using centralized handler with validation chains
app.get('/api/products',
[
query('minPrice').optional().isFloat({ min: 0 }).withMessage('Min price must be a positive number'),
query('maxPrice').optional().isFloat({ min: 0 }).withMessage('Max price must be a positive number'),
query('page').optional().isInt({ min: 1 }).withMessage('Page must be a positive integer')
],
handleValidationErrors,
(req, res) => {
// Process valid request
// ...
}
);
Security Considerations
Protect your API from common security issues:
- Size Limits: Set appropriate size limits to prevent DoS attacks
- Content Type Verification: Validate the Content-Type header
- Sanitization: Sanitize input to prevent XSS and injection attacks
- Rate Limiting: Implement rate limiting to prevent abuse
Combined Example: E-commerce Search API
Let's look at a comprehensive example that combines query parameters and request body handling for an e-commerce search API:
Advanced Product Search API
const express = require('express');
const { body, query, validationResult } = require('express-validator');
const router = express.Router();
// Middleware
router.use(express.json());
// Validation middleware
const validate = (req, res, next) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({
success: false,
errors: errors.array()
});
}
next();
};
// Advanced search endpoint with both query params and request body
router.post('/products/search',
// Query parameter validation
[
query('page').optional().isInt({ min: 1 }).toInt(),
query('limit').optional().isInt({ min: 1, max: 100 }).toInt(),
query('sort').optional().isIn(['price', 'name', 'rating', 'newest']),
query('order').optional().isIn(['asc', 'desc'])
],
// Request body validation
[
body('keywords').optional().isString(),
body('priceRange.min').optional().isFloat({ min: 0 }).toFloat(),
body('priceRange.max').optional().isFloat({ min: 0 }).toFloat(),
body('categories').optional().isArray(),
body('categories.*').optional().isString(),
body('brands').optional().isArray(),
body('brands.*').optional().isString(),
body('ratings').optional().isArray(),
body('ratings.*').optional().isInt({ min: 1, max: 5 }).toInt(),
body('features').optional().isArray(),
body('features.*').optional().isString()
],
validate,
(req, res) => {
// Extract and process query parameters
const page = req.query.page || 1;
const limit = req.query.limit || 20;
const sort = req.query.sort || 'newest';
const order = req.query.order || 'desc';
// Extract and process request body
const {
keywords,
priceRange = {},
categories = [],
brands = [],
ratings = [],
features = []
} = req.body;
// Build search criteria
const searchCriteria = {
// Text search
keywords,
// Price range
minPrice: priceRange.min,
maxPrice: priceRange.max,
// Categories, brands, ratings, features
categories,
brands,
ratings,
features,
// Pagination and sorting
sort,
order,
page,
limit
};
console.log('Search criteria:', searchCriteria);
// Mock database query
// const results = await productService.search(searchCriteria);
// Return paginated results
res.json({
success: true,
pagination: {
page,
limit,
// total: results.total,
// pages: Math.ceil(results.total / limit)
},
criteria: searchCriteria,
// data: results.items
data: [] // Placeholder for actual results
});
}
);
module.exports = router;
Practical Exercises
Exercise 1: Build a Blog API with Filtering
Create an Express API endpoint for retrieving blog posts with these query parameters:
author- Filter by author namecategory- Filter by categorytags- Filter by tags (array)search- Search in title and contentstartDateandendDate- Filter by publication date rangepageandlimit- For paginationsortBy- Sort by field (title, date, views)order- Sort order (asc, desc)
Implement validation for all parameters and provide proper error responses.
Exercise 2: Create a User Profile Update API
Create a PUT endpoint for updating user profiles that accepts a JSON body with these fields:
displayName- Optional, string, 3-50 charsemail- Optional, valid email formatpassword- Optional, min 8 chars, mixed case, numbersbio- Optional, string, max 500 charspreferences- Optional, object with theme (light/dark) and notifications (boolean)socialLinks- Optional, array of objects with platform and url
Implement validation for all fields and return appropriate error messages.
Additional Resources
Documentation
- Express.js - req.query documentation
- Express.js - req.body documentation
- Body Parser middleware documentation