What is Mongoose and Why Use It?
Mongoose is an Object Data Modeling (ODM) library for MongoDB and Node.js. It provides a schema-based solution to model your application data and includes built-in type casting, validation, query building, business logic hooks and more.
Think of Mongoose as a bridge between your Node.js application's object-oriented code and MongoDB's document-oriented data. It offers a higher level of abstraction that makes working with MongoDB more structured and convenient for developers.
Why Use Mongoose Instead of Native MongoDB Driver?
Schema Definition
While MongoDB is schema-less by design, many applications benefit from having a consistent data structure. Mongoose lets you define schemas to ensure your documents follow a specific structure.
Data Validation
Mongoose provides built-in and custom validators to ensure data integrity. This saves you from writing extensive validation code in your application logic.
Middleware (Hooks)
With pre and post hooks, you can define functions to execute at specific stages of a document's lifecycle (like before saving or after finding).
Type Casting
Mongoose handles the conversion between JavaScript types and MongoDB storage formats, reducing errors from type mismatches.
Query Building
Mongoose provides a powerful, chainable query API that makes building complex queries more intuitive and readable than using raw MongoDB queries.
Business Logic Integration
You can attach methods to models and documents, allowing you to encapsulate business logic directly in your data models.
MongoDB Native Driver vs. Mongoose
Native MongoDB Driver
// Connection
const { MongoClient } = require('mongodb');
const client = new MongoClient(uri);
await client.connect();
const db = client.db('test');
// Creating a document
await db.collection('users').insertOne({
name: 'John Doe',
email: 'john@example.com',
age: 30
});
// Finding documents
const users = await db.collection('users')
.find({ age: { $gt: 25 } })
.toArray();
Mongoose
// Connection
const mongoose = require('mongoose');
await mongoose.connect(uri);
// Schema definition
const userSchema = new mongoose.Schema({
name: { type: String, required: true },
email: { type: String, required: true, unique: true },
age: { type: Number, min: 18 }
});
// Model creation
const User = mongoose.model('User', userSchema);
// Creating a document
await User.create({
name: 'John Doe',
email: 'john@example.com',
age: 30
});
// Finding documents
const users = await User.find({ age: { $gt: 25 } });
Installation and Connection Setup
Let's start by installing Mongoose and setting up a connection to MongoDB.
Installing Mongoose
npm install mongoose
Connecting to MongoDB
// db.js - Database connection module
const mongoose = require('mongoose');
require('dotenv').config();
// Connection URI (use environment variables for security)
const uri = process.env.MONGODB_URI;
// Connect with options
mongoose.connect(uri, {
useNewUrlParser: true,
useUnifiedTopology: true,
// useCreateIndex: true, // Note: No longer needed in Mongoose 6+
// useFindAndModify: false // Note: No longer needed in Mongoose 6+
})
.then(() => console.log('MongoDB connected successfully'))
.catch(err => {
console.error('MongoDB connection error:', err);
process.exit(1);
});
// Optional: Log when Mongoose reconnects after losing connection
mongoose.connection.on('reconnected', () => {
console.log('MongoDB reconnected!');
});
// Optional: Log when Mongoose encounters an error
mongoose.connection.on('error', err => {
console.error('MongoDB connection error:', err);
});
// Optional: Log when Mongoose disconnects
mongoose.connection.on('disconnected', () => {
console.log('MongoDB disconnected!');
});
// Handle application termination
process.on('SIGINT', async () => {
await mongoose.connection.close();
console.log('MongoDB connection closed due to app termination');
process.exit(0);
});
module.exports = mongoose;
Connection Options Explained
| Option | Description | Default |
|---|---|---|
useNewUrlParser |
Use the new URL parser | true (in Mongoose 6+) |
useUnifiedTopology |
Use the new Server Discovery and Monitoring engine | true (in Mongoose 6+) |
connectTimeoutMS |
Timeout for initial connection in milliseconds | 30000 (30 seconds) |
socketTimeoutMS |
Timeout for operations in milliseconds | 30000 (30 seconds) |
maxPoolSize |
Maximum number of connections in the connection pool | 100 |
serverSelectionTimeoutMS |
Timeout for server selection in milliseconds | 30000 (30 seconds) |
Note: In Mongoose 6.0+, several options like useNewUrlParser, useUnifiedTopology, useFindAndModify, and useCreateIndex were made true by default or deprecated.
Schemas and Models
Schemas and models are the foundation of Mongoose. A schema defines the structure of documents within a collection, while a model is a compiled version of the schema that provides an interface for CRUD operations.
Defining a Schema
// user.schema.js
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
// Create a schema
const userSchema = new Schema({
// Basic types
firstName: {
type: String,
required: true,
trim: true
},
lastName: {
type: String,
required: true,
trim: true
},
email: {
type: String,
required: true,
unique: true,
lowercase: true,
trim: true,
match: [/^\w+([.-]?\w+)*@\w+([.-]?\w+)*(\.\w{2,3})+$/, 'Please enter a valid email']
},
password: {
type: String,
required: true,
minlength: 8
},
// Numbers
age: {
type: Number,
min: 18,
max: 120
},
// Dates
birthDate: Date,
// Booleans
isActive: {
type: Boolean,
default: true
},
// Nested object
address: {
street: String,
city: String,
state: String,
zipCode: String,
country: {
type: String,
default: 'USA'
}
},
// Arrays
hobbies: [String],
// Array of objects
education: [{
institution: String,
degree: String,
major: String,
startYear: Number,
endYear: Number
}],
// Timestamps
createdAt: {
type: Date,
default: Date.now,
immutable: true
},
updatedAt: {
type: Date,
default: Date.now
}
}, {
// Schema options
timestamps: true, // Adds createdAt and updatedAt automatically
collection: 'users' // Explicitly set collection name (otherwise, Mongoose pluralizes the model name)
});
module.exports = userSchema;
Schema Types
Mongoose provides various built-in SchemaTypes that map to MongoDB data types:
| SchemaType | Description | JavaScript Type |
|---|---|---|
String |
String values | String |
Number |
Integer or float values | Number |
Date |
Date and time values | Date |
Boolean |
True or false values | Boolean |
Buffer |
Binary data | Buffer |
ObjectId |
MongoDB's unique identifiers | mongoose.Schema.Types.ObjectId |
Array |
Arrays of other types | Array |
Map |
Key-value pairs | Map |
Decimal128 |
Precise decimal values | mongoose.Schema.Types.Decimal128 |
Mixed |
Any type of data (flexible) | mongoose.Schema.Types.Mixed |
Schema Options
The second parameter to the Schema constructor is an options object that customizes the behavior of the schema:
const userSchema = new Schema({
// schema definition
}, {
timestamps: true, // Adds createdAt and updatedAt
collection: 'users', // Customizes collection name
strict: true, // Prevents fields not in schema from being saved
versionKey: '__v', // Name of version key
id: true, // Enables a virtual 'id' getter for _id
toJSON: { virtuals: true }, // Include virtuals when document is converted to JSON
toObject: { virtuals: true } // Include virtuals when document is converted to object
});
Creating a Model
Once you've defined a schema, you can create a model from it:
// user.model.js
const mongoose = require('mongoose');
const userSchema = require('./user.schema');
// Create model from schema
const User = mongoose.model('User', userSchema);
module.exports = User;
In your application code, you can then import and use the model:
// app.js or a route file
const User = require('./models/user.model');
// Create a new user
const createUser = async (userData) => {
try {
const newUser = new User(userData);
await newUser.save();
return newUser;
} catch (error) {
console.error('Error creating user:', error);
throw error;
}
};
// Alternative creation method
const createUserAlternative = async (userData) => {
try {
return await User.create(userData);
} catch (error) {
console.error('Error creating user:', error);
throw error;
}
};
Schema Validation
One of Mongoose's most valuable features is built-in validation. You can define validation rules in your schema to ensure that only valid data gets saved to the database.
Built-in Validators
const productSchema = new mongoose.Schema({
// Required field
name: {
type: String,
required: true
},
// String with min and max length
description: {
type: String,
minlength: 10,
maxlength: 1000
},
// Number with min and max values
price: {
type: Number,
required: true,
min: [0, 'Price cannot be negative'],
max: [100000, 'Price exceeds maximum allowed']
},
// String with regex pattern
sku: {
type: String,
match: [/^[A-Z]{2}-\d{4}$/, 'SKU must be in format XX-0000']
},
// String with enum values
category: {
type: String,
enum: {
values: ['Electronics', 'Clothing', 'Home', 'Books', 'Toys'],
message: '{VALUE} is not a supported category'
}
},
// Custom validator for URL
imageUrl: {
type: String,
validate: {
validator: function(v) {
return /^(http|https):\/\/[^ "]+$/.test(v);
},
message: props => `${props.value} is not a valid URL`
}
}
});
Custom Validators
You can create more complex validation logic with custom validator functions:
const orderSchema = new mongoose.Schema({
items: [{
productId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Product',
required: true
},
quantity: {
type: Number,
required: true,
min: 1
},
price: {
type: Number,
required: true,
min: 0
}
}],
totalAmount: {
type: Number,
required: true,
min: 0,
// Custom validator to check if totalAmount matches sum of item prices
validate: {
validator: function(v) {
if (this.items && this.items.length > 0) {
const calculatedTotal = this.items.reduce(
(sum, item) => sum + (item.price * item.quantity), 0
);
// Allow for small floating point differences
return Math.abs(v - calculatedTotal) < 0.01;
}
return true; // Skip validation if no items
},
message: 'Total amount does not match sum of item prices'
}
},
shippingAddress: {
street: {
type: String,
required: true
},
city: {
type: String,
required: true
},
state: {
type: String,
required: true
},
zipCode: {
type: String,
required: true
}
},
paymentMethod: {
type: String,
required: true,
enum: ['Credit Card', 'PayPal', 'Bank Transfer']
},
status: {
type: String,
required: true,
enum: ['Pending', 'Processing', 'Shipped', 'Delivered', 'Cancelled'],
default: 'Pending'
}
});
// Custom validation for status transitions
orderSchema.path('status').validate(function(value) {
// If this is a new order, allow any valid status
if (this.isNew) return true;
// Get the previous document to check the current status
return Order.findById(this._id)
.then(order => {
if (!order) return true; // Document not found, let it pass
const currentStatus = order.status;
// Define valid status transitions
const validTransitions = {
'Pending': ['Processing', 'Cancelled'],
'Processing': ['Shipped', 'Cancelled'],
'Shipped': ['Delivered', 'Cancelled'],
'Delivered': [], // No further transitions allowed
'Cancelled': [] // No further transitions allowed
};
// Check if the new status is a valid transition from the current status
return validTransitions[currentStatus].includes(value);
})
.catch(err => {
console.error('Validation error:', err);
return false;
});
}, 'Invalid status transition');
Handling Validation Errors
When validation fails, Mongoose returns a ValidationError that you can handle in your application:
// Route handler example
app.post('/api/users', async (req, res) => {
try {
const newUser = await User.create(req.body);
res.status(201).json(newUser);
} catch (error) {
// Check if it's a validation error
if (error.name === 'ValidationError') {
// Extract validation error messages
const errors = {};
for (const field in error.errors) {
errors[field] = error.errors[field].message;
}
return res.status(400).json({
message: 'Validation failed',
errors
});
}
// Handle other types of errors
console.error('Error creating user:', error);
res.status(500).json({ message: 'Internal server error' });
}
});
Validation Best Practices
- Be Explicit: Define validation rules for all critical fields
- Provide Error Messages: Custom error messages make debugging easier
- Server and Client Validation: Validate on both ends for better user experience and security
- Balance Validation: Too strict validation can frustrate users, too loose can compromise data integrity
- Test Edge Cases: Ensure your validation handles empty strings, zero values, etc.
Middleware (Hooks)
Mongoose middleware (also called hooks) are functions that run during the execution of asynchronous operations like document save or query execution. They allow you to inject logic before or after these operations.
Types of Middleware
- Document Middleware: Acts on document instances (save, validate, remove)
- Query Middleware: Acts on query operations (find, findOne, etc.)
- Aggregate Middleware: Acts on aggregation operations
- Model Middleware: Acts on model operations (insertMany)
Pre and Post Hooks
You can define middleware to run before (pre) or after (post) specific operations:
const userSchema = new mongoose.Schema({
username: {
type: String,
required: true,
unique: true
},
email: {
type: String,
required: true,
unique: true
},
password: {
type: String,
required: true
},
passwordChangedAt: Date,
role: {
type: String,
enum: ['user', 'admin', 'editor'],
default: 'user'
},
active: {
type: Boolean,
default: true,
select: false // Won't be returned in queries by default
}
});
// PRE-SAVE MIDDLEWARE
// Hash password before saving
userSchema.pre('save', async function(next) {
// 'this' refers to the document being saved
const user = this;
// Skip if password hasn't been modified
if (!user.isModified('password')) return next();
try {
// Example using bcrypt (you would need to install bcrypt)
// const salt = await bcrypt.genSalt(10);
// user.password = await bcrypt.hash(user.password, salt);
// For demonstration, we'll just add a prefix
user.password = 'hashed_' + user.password;
next();
} catch (error) {
next(error);
}
});
// Update passwordChangedAt when password changes
userSchema.pre('save', function(next) {
if (!this.isModified('password') || this.isNew) return next();
// Set passwordChangedAt to current time (with a small delay to ensure
// the timestamp is after the JWT token was issued)
this.passwordChangedAt = new Date(Date.now() - 1000);
next();
});
// PRE-FIND MIDDLEWARE
// Only find active users unless specifically requested
userSchema.pre(/^find/, function(next) {
// 'this' refers to the query, not the document
this.find({ active: { $ne: false } });
next();
});
// POST-SAVE MIDDLEWARE
userSchema.post('save', function(doc, next) {
console.log('New user saved:', doc.username);
// You could send a welcome email, log activity, etc.
next();
});
// PRE-REMOVE MIDDLEWARE
userSchema.pre('remove', async function(next) {
// Clean up related data before removing user
// For example, delete all posts by this user
// await Post.deleteMany({ author: this._id });
console.log(`Preparing to delete user ${this._id}`);
next();
});
Error Handling in Middleware
userSchema.pre('save', function(next) {
// Check if email is from a blocked domain
const blockedDomains = ['spam.com', 'temporary.com'];
const emailDomain = this.email.split('@')[1];
if (blockedDomains.includes(emailDomain)) {
// Pass an error to next() to trigger error handling
return next(new Error('Email domain not allowed'));
}
next();
});
// You can also use async/await with error handling
userSchema.pre('save', async function(next) {
try {
// Perform some async operation
await someAsyncOperation();
next();
} catch (error) {
next(error); // Pass the error to next()
}
});
Real-World Example: Audit Logging Middleware
// Define an audit schema
const auditSchema = new mongoose.Schema({
action: {
type: String,
enum: ['create', 'update', 'delete', 'read'],
required: true
},
collection: {
type: String,
required: true
},
documentId: {
type: mongoose.Schema.Types.ObjectId,
required: true
},
changes: mongoose.Schema.Types.Mixed,
performedBy: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
performedAt: {
type: Date,
default: Date.now
},
ipAddress: String
});
const Audit = mongoose.model('Audit', auditSchema);
// Apply audit logging middleware to a schema
function addAuditMiddleware(schema, collectionName) {
// Log document creation
schema.post('save', async function(doc) {
try {
// Get current user from request context (simplistic example)
const userId = global.currentUserId; // Set by your auth middleware
const ipAddress = global.currentIpAddress;
await Audit.create({
action: 'create',
collection: collectionName,
documentId: doc._id,
changes: doc.toObject(),
performedBy: userId,
ipAddress
});
} catch (error) {
console.error('Audit logging error:', error);
// Don't stop the flow for logging errors
}
});
// Log document updates
schema.pre('findOneAndUpdate', function() {
// Store the query to use in post middleware
this._update = this.getUpdate();
});
schema.post('findOneAndUpdate', async function(doc) {
if (!doc) return; // No document was updated
try {
const userId = global.currentUserId;
const ipAddress = global.currentIpAddress;
await Audit.create({
action: 'update',
collection: collectionName,
documentId: doc._id,
changes: this._update,
performedBy: userId,
ipAddress
});
} catch (error) {
console.error('Audit logging error:', error);
}
});
// Similar handlers for delete operations...
}
// Apply to user schema
addAuditMiddleware(userSchema, 'users');
Queries, Statics, and Instance Methods
Mongoose schemas can be extended with custom methods at three levels: instance methods, static methods, and query helpers. These allow you to encapsulate business logic in your data models.
Instance Methods
Instance methods are callable on individual document instances:
// User schema with instance methods
const userSchema = new mongoose.Schema({
firstName: String,
lastName: String,
email: String,
password: String,
role: {
type: String,
enum: ['user', 'admin'],
default: 'user'
}
});
// Add an instance method
userSchema.methods.getFullName = function() {
return `${this.firstName} ${this.lastName}`;
};
// Method to check password
userSchema.methods.comparePassword = async function(candidatePassword) {
// In a real application, you would use bcrypt here
// return await bcrypt.compare(candidatePassword, this.password);
// Simplified example
return candidatePassword === this.password;
};
// Method to check if user has admin access
userSchema.methods.isAdmin = function() {
return this.role === 'admin';
};
// Usage example
const User = mongoose.model('User', userSchema);
async function loginUser(email, password) {
// Find the user
const user = await User.findOne({ email });
if (!user) return null;
// Use instance method to verify password
const isMatch = await user.comparePassword(password);
if (!isMatch) return null;
// Get full name using instance method
console.log(`${user.getFullName()} logged in`);
return user;
}
Static Methods
Static methods are callable on the model itself, not on instances:
// Add static methods to the schema
userSchema.statics.findByEmail = function(email) {
return this.findOne({ email });
};
userSchema.statics.findAdmins = function() {
return this.find({ role: 'admin' });
};
// Method to find users created in last N days
userSchema.statics.findRecentUsers = function(days = 7) {
const date = new Date();
date.setDate(date.getDate() - days);
return this.find({
createdAt: { $gte: date }
}).sort({ createdAt: -1 });
};
// Usage example
async function getAdminEmails() {
const admins = await User.findAdmins();
return admins.map(admin => admin.email);
}
async function getUserByEmail(email) {
return await User.findByEmail(email);
}
Query Helpers
Query helpers add custom methods to the Mongoose query object, allowing for chainable syntax:
// Add query helpers
userSchema.query.byRole = function(role) {
return this.where({ role });
};
userSchema.query.byName = function(name) {
const regex = new RegExp(name, 'i');
return this.where({
$or: [
{ firstName: regex },
{ lastName: regex }
]
});
};
userSchema.query.active = function() {
return this.where({ active: true });
};
// Usage example
async function findUsers() {
// Chain query helpers
const activeAdmins = await User.find()
.byRole('admin')
.active()
.select('firstName lastName email')
.sort('lastName');
// Search by name with query helper
const usersNamedJohn = await User.find()
.byName('John')
.active();
return {
activeAdmins,
usersNamedJohn
};
}
Virtual Properties
Virtual properties don't get persisted to MongoDB but can be used like regular properties:
// Add a virtual property for full name
userSchema.virtual('fullName')
.get(function() {
return `${this.firstName} ${this.lastName}`;
})
.set(function(value) {
const parts = value.split(' ');
this.firstName = parts[0];
this.lastName = parts.slice(1).join(' ');
});
// Virtual property that calculates age from birthDate
userSchema.virtual('age').get(function() {
if (!this.birthDate) return null;
const today = new Date();
const birthDate = new Date(this.birthDate);
let age = today.getFullYear() - birthDate.getFullYear();
const monthDiff = today.getMonth() - birthDate.getMonth();
if (monthDiff < 0 || (monthDiff === 0 && today.getDate() < birthDate.getDate())) {
age--;
}
return age;
});
// Make sure virtuals are included when converting to JSON
userSchema.set('toJSON', { virtuals: true });
userSchema.set('toObject', { virtuals: true });
// Usage example
const john = await User.findOne({ firstName: 'John' });
console.log(john.fullName); // "John Doe"
console.log(john.age); // 35 (calculated from birthDate)
// Set the full name
john.fullName = 'John Robert Doe';
console.log(john.firstName); // "John"
console.log(john.lastName); // "Robert Doe"
Modeling Relationships in Mongoose
MongoDB is a document-oriented database, but it still allows you to model relationships between documents. Mongoose offers several ways to implement these relationships.
Referencing (Normalization)
Referencing means storing the ID of one document in another document. This approach is similar to foreign keys in relational databases.
// User model with referenced posts
const userSchema = new mongoose.Schema({
username: { type: String, required: true, unique: true },
email: { type: String, required: true, unique: true },
password: { type: String, required: true },
// Here we don't store the actual posts, just their IDs
});
const User = mongoose.model('User', userSchema);
// Post model with reference to user
const postSchema = new mongoose.Schema({
title: { type: String, required: true },
content: { type: String, required: true },
// Reference to User model
author: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
required: true
},
// Reference to Category model
category: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Category'
},
createdAt: { type: Date, default: Date.now }
});
const Post = mongoose.model('Post', postSchema);
// Comment model with references to both post and user
const commentSchema = new mongoose.Schema({
content: { type: String, required: true },
// Reference to Post model
post: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Post',
required: true
},
// Reference to User model
author: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
required: true
},
createdAt: { type: Date, default: Date.now }
});
const Comment = mongoose.model('Comment', commentSchema);
Retrieving related data using populate():
// Find a post and populate the author information
const post = await Post.findById(postId)
.populate('author', 'username email') // Only include these fields
.populate('category');
console.log(post.title);
console.log(post.author.username); // Access populated user data
// Find a post with its comments and their authors
const postWithComments = await Post.findById(postId).lean();
// Get comments for the post
const comments = await Comment.find({ post: postId })
.populate('author', 'username')
.sort({ createdAt: -1 });
// Add comments to the post object
postWithComments.comments = comments;
// Nested population
const posts = await Post.find()
.populate({
path: 'author',
select: 'username',
// You can even populate fields in the author
populate: {
path: 'profile',
select: 'avatar'
}
})
.limit(10);
Embedding (Denormalization)
Embedding means storing the entire related document directly within the parent document. This is useful for one-to-few relationships where the child data is always loaded with the parent.
// Product schema with embedded reviews
const productSchema = new mongoose.Schema({
name: { type: String, required: true },
price: { type: Number, required: true, min: 0 },
description: String,
// Embedded reviews - the complete review data is stored in the product document
reviews: [{
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
rating: {
type: Number,
min: 1,
max: 5,
required: true
},
comment: String,
createdAt: {
type: Date,
default: Date.now
}
}]
});
const Product = mongoose.model('Product', productSchema);
// Usage example - adding a review to a product
async function addReview(productId, userId, rating, comment) {
const product = await Product.findById(productId);
if (!product) {
throw new Error('Product not found');
}
// Add review to embedded array
product.reviews.push({
user: userId,
rating,
comment
});
// Calculate average rating
const totalRating = product.reviews.reduce((sum, review) => sum + review.rating, 0);
product.averageRating = totalRating / product.reviews.length;
// Save the product with the new review
await product.save();
return product;
}
Hybrid Approach
In some cases, a hybrid approach combining embedding and referencing works best:
// User schema with embedded address but referenced orders
const userSchema = new mongoose.Schema({
username: { type: String, required: true },
email: { type: String, required: true },
// Embedded address (one-to-one relationship)
address: {
street: String,
city: String,
state: String,
zipCode: String,
country: String
},
// Just the count of orders (for performance)
orderCount: {
type: Number,
default: 0
},
// Recent orders summary (embed a few, reference the rest)
recentOrders: [{
orderId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Order'
},
total: Number,
date: Date,
status: String
}]
});
// Order schema references user but embeds items
const orderSchema = new mongoose.Schema({
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
required: true
},
// Embedded line items
items: [{
product: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Product'
},
name: String, // Denormalized for performance
price: Number, // Denormalized for performance
quantity: Number
}],
totalAmount: Number,
status: String,
paymentMethod: String,
// Denormalized shipping address (copied from user at order time)
shippingAddress: {
street: String,
city: String,
state: String,
zipCode: String,
country: String
}
});
// Update user when an order is placed
orderSchema.post('save', async function(order) {
try {
// Update user's order count
await User.findByIdAndUpdate(order.user, {
$inc: { orderCount: 1 },
$push: {
recentOrders: {
$each: [{
orderId: order._id,
total: order.totalAmount,
date: order.createdAt,
status: order.status
}],
$sort: { date: -1 },
$slice: 5 // Keep only the 5 most recent orders
}
}
});
} catch (error) {
console.error('Error updating user after order save:', error);
}
});
Choosing a Relationship Pattern
| Relationship Type | Recommended Pattern | Example |
|---|---|---|
| One-to-One | Embedding or referencing | User and Profile |
| One-to-Few | Embedding | Product and Reviews (if limited) |
| One-to-Many | Referencing | User and Orders |
| Many-to-Many | Referencing with an array or a separate model | Students and Courses |
Considerations When Modeling Relationships
- Data Access Patterns: How frequently the data is read vs. written
- Data Size: Large arrays can impact document size and performance
- Data Consistency: Embedded data can become outdated if not carefully managed
- Query Performance: Consider indexing for fields used in lookups and joins
- Document Size Limit: MongoDB documents have a 16MB size limit
Practical Activities
Activity 1: Blog API with Mongoose
Create a RESTful API for a blog application using Express and Mongoose with the following features:
- Define Mongoose schemas and models for:
- Users (with validation for email, password)
- Posts (with references to authors)
- Comments (with references to posts and users)
- Categories (with references to posts)
- Implement middleware for password hashing and timestamp updates
- Create custom methods for retrieving related data
- Build Express routes for CRUD operations on all resources
- Implement proper error handling and validation
Activity 2: E-commerce Data Modeling
Design and implement Mongoose models for an e-commerce application:
- Design schemas for:
- Products (with categories, variants, inventory tracking)
- Users (with address, payment information)
- Orders (with line items, payment status, shipping details)
- Reviews (with ratings, comments)
- Decide on appropriate relationship patterns (embedding vs. referencing)
- Implement virtual properties for calculated fields (e.g., order total)
- Create static methods for common queries (e.g., finding top-rated products)
- Add middleware for inventory management (e.g., reducing stock when orders are placed)
Activity 3: Advanced Mongoose Features
Extend your blog or e-commerce application with advanced Mongoose features:
- Implement optimistic concurrency control using versioning
- Create a plugin for adding audit trails to model operations
- Use discriminators for handling different user types with shared base schema
- Implement text search using MongoDB's text indexes
- Add geospatial querying for finding nearby resources (e.g., stores)
- Create complex aggregation pipelines for reporting (e.g., sales by category)
Advanced Mongoose Techniques
Here are some advanced techniques that can enhance your Mongoose applications:
Plugins
Plugins are reusable packages of schema functionality:
// Define a plugin for adding timestamps
function timestampPlugin(schema, options) {
// Add createdAt and updatedAt fields
schema.add({
createdAt: {
type: Date,
default: Date.now,
immutable: true
},
updatedAt: {
type: Date,
default: Date.now
}
});
// Add pre-save middleware to update the updatedAt field
schema.pre('save', function(next) {
if (this.isModified() && !this.isNew) {
this.updatedAt = new Date();
}
next();
});
// Add pre-update middleware
schema.pre(['updateOne', 'findOneAndUpdate'], function(next) {
this.set({ updatedAt: new Date() });
next();
});
}
// Use the plugin in multiple schemas
userSchema.plugin(timestampPlugin);
postSchema.plugin(timestampPlugin);
commentSchema.plugin(timestampPlugin);
Discriminators (Inheritance)
Discriminators provide a way to implement inheritance-like functionality in Mongoose:
// Base schema for all types of users
const userSchema = new mongoose.Schema({
email: { type: String, required: true, unique: true },
password: { type: String, required: true },
name: { type: String, required: true },
createdAt: { type: Date, default: Date.now },
role: { type: String, required: true },
}, { discriminatorKey: 'role' });
const User = mongoose.model('User', userSchema);
// Customer model (inherits from User)
const Customer = User.discriminator('customer', new mongoose.Schema({
shippingAddresses: [{
street: String,
city: String,
state: String,
zipCode: String,
isPrimary: Boolean
}],
orders: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'Order'
}]
}));
// Admin model (inherits from User)
const Admin = User.discriminator('admin', new mongoose.Schema({
permissions: [String],
department: String,
adminLevel: {
type: Number,
min: 1,
max: 5,
default: 1
}
}));
// Usage
async function createUsers() {
// Create a customer
const customer = await Customer.create({
email: 'customer@example.com',
password: 'hashed_password',
name: 'John Customer',
shippingAddresses: [{
street: '123 Main St',
city: 'Anytown',
state: 'CA',
zipCode: '12345',
isPrimary: true
}]
});
// Create an admin
const admin = await Admin.create({
email: 'admin@example.com',
password: 'hashed_password',
name: 'Jane Admin',
permissions: ['users.view', 'users.edit', 'products.manage'],
department: 'IT',
adminLevel: 3
});
// Find all users regardless of role
const allUsers = await User.find();
// This will include both customers and admins
console.log(`Total users: ${allUsers.length}`);
}
Aggregation Framework
Mongoose provides a wrapper around MongoDB's powerful aggregation framework:
// Get sales statistics by product category
async function getProductCategoryStats() {
return await Order.aggregate([
// Unwind the items array to work with individual items
{ $unwind: '$items' },
// Look up the product for each item
{
$lookup: {
from: 'products',
localField: 'items.product',
foreignField: '_id',
as: 'productData'
}
},
// Unwind the productData array (which has one element)
{ $unwind: '$productData' },
// Group by category and calculate statistics
{
$group: {
_id: '$productData.category',
totalSales: { $sum: { $multiply: ['$items.price', '$items.quantity'] } },
count: { $sum: '$items.quantity' },
averagePrice: { $avg: '$items.price' }
}
},
// Sort by total sales descending
{ $sort: { totalSales: -1 } },
// Add a category name field from the _id
{
$project: {
category: '$_id',
totalSales: 1,
count: 1,
averagePrice: 1,
_id: 0
}
}
]);
}
// Get customer purchase statistics
async function getCustomerPurchaseStats() {
return await Order.aggregate([
// Match only completed orders
{ $match: { status: 'completed' } },
// Group by customer
{
$group: {
_id: '$user',
totalSpent: { $sum: '$totalAmount' },
orderCount: { $sum: 1 },
averageOrderValue: { $avg: '$totalAmount' },
firstOrder: { $min: '$createdAt' },
lastOrder: { $max: '$createdAt' }
}
},
// Look up customer details
{
$lookup: {
from: 'users',
localField: '_id',
foreignField: '_id',
as: 'customerData'
}
},
// Unwind the customerData array
{ $unwind: '$customerData' },
// Project only the fields we need
{
$project: {
_id: 1,
customer: {
name: '$customerData.name',
email: '$customerData.email'
},
totalSpent: 1,
orderCount: 1,
averageOrderValue: 1,
daysSinceFirstOrder: {
$divide: [
{ $subtract: [new Date(), '$firstOrder'] },
1000 * 60 * 60 * 24 // Convert milliseconds to days
]
},
daysSinceLastOrder: {
$divide: [
{ $subtract: [new Date(), '$lastOrder'] },
1000 * 60 * 60 * 24
]
}
}
},
// Sort by total spent
{ $sort: { totalSpent: -1 } },
// Limit to top 100 customers
{ $limit: 100 }
]);
}
Transactions
For operations that need to be atomic across multiple documents or collections, use transactions:
// Transfer funds between accounts
async function transferFunds(fromAccountId, toAccountId, amount) {
// Start a session
const session = await mongoose.startSession();
session.startTransaction();
try {
// Find the source account and update its balance
const fromAccount = await Account.findOneAndUpdate(
{ _id: fromAccountId, balance: { $gte: amount } },
{ $inc: { balance: -amount } },
{ new: true, session }
);
if (!fromAccount) {
throw new Error('Insufficient funds or account not found');
}
// Find the destination account and update its balance
const toAccount = await Account.findOneAndUpdate(
{ _id: toAccountId },
{ $inc: { balance: amount } },
{ new: true, session }
);
if (!toAccount) {
throw new Error('Destination account not found');
}
// Create a transaction record
await Transaction.create([{
fromAccount: fromAccountId,
toAccount: toAccountId,
amount,
type: 'transfer',
date: new Date()
}], { session });
// Commit the transaction
await session.commitTransaction();
session.endSession();
return { fromAccount, toAccount };
} catch (error) {
// Abort the transaction on error
await session.abortTransaction();
session.endSession();
throw error;
}
}
Text Search
MongoDB's text search capabilities can be accessed through Mongoose:
// Create a text index on the product schema
productSchema.index({
name: 'text',
description: 'text',
tags: 'text'
}, {
weights: {
name: 10, // Name is most important
tags: 5, // Tags are next
description: 1 // Description is least important
},
name: 'product_text_index'
});
// Search for products
async function searchProducts(query, options = {}) {
const { limit = 20, page = 1, sortBy = 'relevance' } = options;
const skip = (page - 1) * limit;
let sort = {};
if (sortBy === 'relevance') {
sort = { score: { $meta: 'textScore' } };
} else if (sortBy === 'price_asc') {
sort = { price: 1 };
} else if (sortBy === 'price_desc') {
sort = { price: -1 };
} else if (sortBy === 'newest') {
sort = { createdAt: -1 };
}
// Perform the text search
const products = await Product.find(
{ $text: { $search: query } },
{ score: { $meta: 'textScore' } }
)
.sort(sort)
.skip(skip)
.limit(limit);
// Get total count for pagination
const totalProducts = await Product.countDocuments({
$text: { $search: query }
});
return {
products,
pagination: {
totalProducts,
totalPages: Math.ceil(totalProducts / limit),
currentPage: page,
limit
}
};
}
Geospatial Queries
MongoDB has robust support for geospatial queries, which Mongoose can leverage:
// Store location with geospatial index
const storeSchema = new mongoose.Schema({
name: { type: String, required: true },
address: {
street: String,
city: String,
state: String,
zipCode: String
},
location: {
type: {
type: String,
enum: ['Point'],
required: true
},
coordinates: {
type: [Number], // [longitude, latitude]
required: true
}
},
storeHours: {
monday: String,
tuesday: String,
wednesday: String,
thursday: String,
friday: String,
saturday: String,
sunday: String
}
});
// Create a 2dsphere index for geospatial queries
storeSchema.index({ location: '2dsphere' });
const Store = mongoose.model('Store', storeSchema);
// Find stores near a location
async function findNearbyStores(longitude, latitude, maxDistance = 10000) {
return await Store.find({
location: {
$near: {
$geometry: {
type: 'Point',
coordinates: [longitude, latitude]
},
$maxDistance: maxDistance // in meters
}
}
})
.select('name address location storeHours')
.limit(10);
}
// Find stores within a polygon (e.g., delivery area)
async function findStoresInArea(polygonCoordinates) {
return await Store.find({
location: {
$geoWithin: {
$geometry: {
type: 'Polygon',
coordinates: [polygonCoordinates] // Array of [lng, lat] points
}
}
}
});
}
Performance and Best Practices
As your application grows, optimizing Mongoose performance becomes increasingly important. Here are some key best practices:
Use Lean Queries
When you only need the data and don't need to modify the document, use .lean() to get plain JavaScript objects instead of full Mongoose documents. This can significantly improve performance for read-heavy operations.
// Regular query returns Mongoose documents
const normalUsers = await User.find({ active: true });
// Lean query returns plain JavaScript objects (much faster)
const leanUsers = await User.find({ active: true }).lean();
Proper Indexing
Indexes dramatically improve query performance but come with a write penalty. Index fields that are frequently used in queries, sorting, or as foreign keys.
// Single field index
userSchema.index({ email: 1 });
// Compound index
userSchema.index({ lastName: 1, firstName: 1 });
// Text index
productSchema.index({ name: 'text', description: 'text' });
// Geospatial index
storeSchema.index({ location: '2dsphere' });
// Unique index
userSchema.index({ username: 1 }, { unique: true });
Select Only Needed Fields
Limit the fields you retrieve to only what you need, especially when working with large documents.
// Only select fields you need
const users = await User.find({ active: true })
.select('firstName lastName email')
.lean();
// Exclude specific fields
const products = await Product.find()
.select('-description -longText')
.lean();
Limit and Paginate Results
Always limit the number of documents returned, especially for endpoints accessed by end users.
// Pagination function
async function getPaginatedResults(model, query = {}, options = {}) {
const {
page = 1,
limit = 10,
sort = { createdAt: -1 },
select = '',
populate = null
} = options;
const skip = (page - 1) * limit;
// Build the query
let queryBuilder = model.find(query)
.skip(skip)
.limit(limit)
.sort(sort);
// Add field selection if specified
if (select) {
queryBuilder = queryBuilder.select(select);
}
// Add population if specified
if (populate) {
queryBuilder = queryBuilder.populate(populate);
}
// Execute the query
const results = await queryBuilder.lean();
// Get total count for pagination info
const totalResults = await model.countDocuments(query);
return {
results,
pagination: {
total: totalResults,
totalPages: Math.ceil(totalResults / limit),
currentPage: page,
limit
}
};
}
// Usage example
const productsPage = await getPaginatedResults(Product,
{ category: 'Electronics' },
{
page: 2,
limit: 20,
sort: { price: -1 },
select: 'name price rating',
populate: { path: 'reviews', select: 'rating comment' }
}
);
Be Careful with Populate
The populate() method is convenient but can lead to performance issues if not used carefully. Limit the fields you populate and the depth of population.
// BAD: Deep population with no field selection
const users = await User.find()
.populate('posts')
.populate({
path: 'comments',
populate: {
path: 'author',
populate: {
path: 'profile'
}
}
});
// GOOD: Selective population with field selection
const users = await User.find()
.select('name email')
.populate({
path: 'posts',
select: 'title createdAt',
options: { limit: 5, sort: { createdAt: -1 } }
})
.lean();
Use Bulk Operations
For operations that affect multiple documents, use Mongoose's bulk operations for better performance.
// Less efficient: Update documents one by one
async function deactivateOldUsers(date) {
const users = await User.find({ lastActive: { $lt: date } });
for (const user of users) {
user.active = false;
await user.save();
}
}
// More efficient: Use updateMany
async function deactivateOldUsers(date) {
const result = await User.updateMany(
{ lastActive: { $lt: date } },
{ active: false }
);
return result.nModified;
}
// Using bulkWrite for different operations
async function processUserBatch(operations) {
return await User.bulkWrite([
{
insertOne: {
document: { name: 'New User', email: 'new@example.com' }
}
},
{
updateOne: {
filter: { email: 'existing@example.com' },
update: { $set: { name: 'Updated User' } }
}
},
{
deleteOne: {
filter: { email: 'remove@example.com' }
}
}
]);
}
Consider Schema Design Carefully
Schema design significantly impacts performance. Be thoughtful about when to embed vs. reference, and about which fields to index.
- Embed: Data that's always accessed together and doesn't grow unbounded
- Reference: Large subdocuments or collections that grow over time
- Denormalize: Frequently accessed data to reduce joins, but be careful about keeping it in sync
Use Appropriate Connection Options
Configure your Mongoose connection for optimal performance:
mongoose.connect(uri, {
// Connection pool size
maxPoolSize: 100,
// How long to maintain idle connections
socketTimeoutMS: 45000,
// Enable/disable SCRAM authentication
authMechanism: 'SCRAM-SHA-1',
// Whether to retry failed operations
retryWrites: true,
// Replicaset options if using MongoDB replication
replicaSet: 'rs0',
// For read preference if using replication
readPreference: 'primaryPreferred'
});
Benchmarking Tips
When optimizing, measure performance to ensure your changes are actually helpful:
// Simple timing function
async function timeExecution(name, fn) {
console.time(name);
const result = await fn();
console.timeEnd(name);
return result;
}
// Compare different query approaches
async function compareQueryPerformance() {
// Regular query with document conversion
await timeExecution('Regular Query', () =>
User.find({ active: true }).exec()
);
// Lean query
await timeExecution('Lean Query', () =>
User.find({ active: true }).lean().exec()
);
// Query with field selection
await timeExecution('Selected Fields', () =>
User.find({ active: true }).select('name email').lean().exec()
);
// Query with indexable condition
await timeExecution('Indexed Query', () =>
User.find({ email: 'test@example.com' }).lean().exec()
);
}
Robust Error Handling
Proper error handling is crucial for building reliable applications. Mongoose operations can throw various types of errors that should be handled appropriately.
Common Mongoose Error Types
| Error Type | Description | Common Causes |
|---|---|---|
| ValidationError | Schema validation failed | Invalid data provided to model |
| CastError | Failed to cast value to required type | Invalid ObjectId, wrong data type |
| MongoError | Error from MongoDB driver | Duplicate key, connection issues |
| DocumentNotFoundError | Document not found for update operation | Using findOneAndUpdate with runValidators when no document matches |
| MissingSchemaError | Schema not defined for model | Trying to access a model that doesn't exist |
Handling Validation Errors
// Express route handler with validation error handling
app.post('/api/users', async (req, res) => {
try {
const user = new User(req.body);
await user.save();
res.status(201).json(user);
} catch (error) {
if (error.name === 'ValidationError') {
// Extract validation error messages
const errors = {};
for (const field in error.errors) {
errors[field] = error.errors[field].message;
}
return res.status(400).json({
message: 'Validation failed',
errors
});
}
// Handle other types of errors
console.error('Error creating user:', error);
res.status(500).json({ message: 'Server error' });
}
});
Handling MongoDB Errors
// Handle MongoDB-specific errors
app.post('/api/products', async (req, res) => {
try {
const product = await Product.create(req.body);
res.status(201).json(product);
} catch (error) {
if (error.name === 'MongoError' || error.name === 'MongoServerError') {
// Duplicate key error
if (error.code === 11000) {
const field = Object.keys(error.keyValue)[0];
const value = error.keyValue[field];
return res.status(409).json({
message: `${field} already exists with value: ${value}`
});
}
// Connection timeout
if (error.code === 50) {
return res.status(503).json({
message: 'Database connection timeout, please try again'
});
}
}
// Handle validation errors
if (error.name === 'ValidationError') {
// ... validation error handling ...
}
// Handle cast errors
if (error.name === 'CastError') {
return res.status(400).json({
message: `Invalid ${error.path}: ${error.value}`
});
}
// Log unknown errors
console.error('Database error:', error);
res.status(500).json({ message: 'Server error' });
}
});
Creating Custom Error Handlers
// Define custom error classes
class AppError extends Error {
constructor(message, statusCode) {
super(message);
this.statusCode = statusCode;
this.status = `${statusCode}`.startsWith('4') ? 'fail' : 'error';
this.isOperational = true;
Error.captureStackTrace(this, this.constructor);
}
}
class ValidationError extends AppError {
constructor(errors) {
super('Validation failed', 400);
this.errors = errors;
this.name = 'ValidationError';
}
}
class NotFoundError extends AppError {
constructor(entity, id) {
super(`${entity} with ID ${id} not found`, 404);
this.name = 'NotFoundError';
}
}
// Helper function to handle Mongoose errors
function handleMongooseError(error) {
if (error.name === 'ValidationError') {
const errors = {};
for (const field in error.errors) {
errors[field] = error.errors[field].message;
}
return new ValidationError(errors);
}
if (error.name === 'CastError') {
return new AppError(`Invalid ${error.path}: ${error.value}`, 400);
}
if (error.name === 'MongoError' && error.code === 11000) {
const field = Object.keys(error.keyValue)[0];
const value = error.keyValue[field];
return new AppError(`Duplicate field value: ${field}=${value}`, 409);
}
return error;
}
// Express middleware for global error handling
function globalErrorHandler(err, req, res, next) {
// Handle Mongoose errors
const error = handleMongooseError(err);
// Set default values
const statusCode = error.statusCode || 500;
const status = error.status || 'error';
// Response format
const response = {
status,
message: error.message
};
// Add validation errors if present
if (error.errors) {
response.errors = error.errors;
}
// Add stack trace in development
if (process.env.NODE_ENV === 'development') {
response.stack = error.stack;
}
// Send the response
res.status(statusCode).json(response);
}
// Apply global error handler
app.use(globalErrorHandler);
// Usage in route handlers
app.get('/api/users/:id', async (req, res, next) => {
try {
const user = await User.findById(req.params.id);
if (!user) {
throw new NotFoundError('User', req.params.id);
}
res.json(user);
} catch (error) {
next(error); // Pass to global error handler
}
});
Error Handling Best Practices
- Centralize Error Handling: Use a consistent approach across your application
- Be Specific: Return specific error types and messages
- Log Properly: Log detailed error information for debugging while keeping user responses simple
- Validate Early: Validate input data before database operations
- Use Transactions: For multi-document operations that need to be atomic
- Safe Operations: Use methods like
findOneAndUpdateover find-then-update patterns
Summary and Key Takeaways
Mongoose Fundamentals
- Mongoose is an ODM (Object Data Modeling) library for MongoDB and Node.js
- It provides a schema-based solution to model application data
- Schemas define the structure of your documents and add validation
- Models are compiled from schemas and provide an interface to the database
Data Modeling
- Consider carefully when to embed vs. reference related data
- Use embedding for one-to-few relationships where data is accessed together
- Use referencing for one-to-many or many-to-many relationships
- Consider a hybrid approach for complex scenarios
Schema Features
- Validation ensures data integrity before saving to the database
- Middleware (hooks) allows code execution at specific points in the document lifecycle
- Methods and statics add business logic to your models
- Virtuals provide computed properties that aren't stored in MongoDB
Performance Optimization
- Use
lean()for read-only operations - Create indexes for frequently queried fields
- Select only needed fields to reduce document size
- Use pagination to limit results
- Use bulk operations for multiple document operations
Error Handling
- Implement consistent error handling across your application
- Handle different types of Mongoose errors appropriately
- Use custom error classes for better error management
- Implement global error handling in Express
Further Reading and Resources
- Mongoose Documentation
- MongoDB Data Modeling Introduction
- Mongoose GitHub Repository
- Getting Started with MongoDB and Mongoose
- "Web Development with Node and Express" by Ethan Brown
- "MongoDB: The Definitive Guide" by Shannon Bradshaw, Eoin Brazil, and Kristina Chodorow
- "Node.js Design Patterns" by Mario Casciaro and Luciano Mammino