Mongoose Validation and Middleware

Advanced Techniques for Data Integrity and Business Logic

Building Robust MongoDB Applications

In our previous lecture, we covered the fundamentals of Mongoose schemas, models, and relationships. Today, we'll dive deeper into two crucial aspects of Mongoose that enable you to build robust, maintainable MongoDB applications: validation and middleware.

These features allow you to enforce data integrity at the schema level and implement complex business logic at specific points in the document lifecycle. Mastering these concepts will help you create applications that are not only functional but also reliable and scalable.

flowchart LR A[Application] --> B[Validation Layer] B --> C[Middleware Layer] C --> D[(MongoDB)] E[User Input] --> A subgraph Validation F[Schema Rules] G[Custom Validators] H[Update Validators] end subgraph Middleware I[Pre Hooks] J[Post Hooks] K[Query Middleware] L[Document Middleware] end

In this lecture, we'll explore:

Advanced Validation Techniques

Mongoose provides a powerful validation framework that goes far beyond simple type checking. Let's explore advanced validation techniques to ensure your data meets complex business requirements.

Recap: Basic Schema Validation


const mongoose = require('mongoose');
const { Schema } = mongoose;

const productSchema = new Schema({
  name: {
    type: String,
    required: true,
    trim: true,
    minlength: 2,
    maxlength: 100
  },
  price: {
    type: Number,
    required: true,
    min: [0, 'Price cannot be negative'],
    max: 1000000
  },
  description: {
    type: String,
    trim: true,
    maxlength: 2000
  },
  category: {
    type: String,
    required: true,
    enum: {
      values: ['Electronics', 'Clothing', 'Books', 'Home', 'Sports'],
      message: '{VALUE} is not a supported category'
    }
  },
  inStock: {
    type: Boolean,
    default: true
  },
  createdAt: {
    type: Date,
    default: Date.now,
    immutable: true // Cannot be changed once set
  }
});
            

Custom Validators

Custom validators allow you to implement complex validation logic that can't be expressed with built-in validators.


const userSchema = new Schema({
  username: {
    type: String,
    required: true,
    validate: {
      validator: function(v) {
        // Username must be alphanumeric with optional underscores and hyphens
        return /^[a-zA-Z0-9_-]+$/.test(v);
      },
      message: props => `${props.value} is not a valid username. Use only letters, numbers, underscores, and hyphens.`
    }
  },
  email: {
    type: String,
    required: true,
    validate: {
      validator: function(v) {
        // Complex email validation regex
        return /^([\w-\.]+@([\w-]+\.)+[\w-]{2,4})?$/.test(v);
      },
      message: props => `${props.value} is not a valid email address.`
    }
  },
  password: {
    type: String,
    required: true,
    validate: {
      validator: function(v) {
        // Password must have at least 8 characters, one uppercase, one lowercase, and one number
        const passwordRegex = /^(?=.*[a-z])(?=.*[A-Z])(?=.*\d).{8,}$/;
        return passwordRegex.test(v);
      },
      message: 'Password must be at least 8 characters long and contain at least one uppercase letter, one lowercase letter, and one number.'
    }
  },
  age: {
    type: Number,
    min: 18,
    max: 120,
    validate: {
      validator: function(v) {
        // Additional custom logic: age must be a whole number
        return Number.isInteger(v);
      },
      message: 'Age must be a whole number.'
    }
  }
});
            

Asynchronous Validators

For validation that requires database lookups or API calls, Mongoose supports asynchronous validators.


const userSchema = new Schema({
  email: {
    type: String,
    required: true,
    validate: {
      validator: async function(email) {
        // Check if the email is already in use
        const existingUser = await mongoose.models.User.findOne({ email });
        return !existingUser; // Return false if user with this email exists
      },
      message: 'Email is already in use.'
    }
  },
  username: {
    type: String,
    required: true,
    validate: {
      validator: async function(username) {
        // Check if username is already taken
        const existingUser = await mongoose.models.User.findOne({ username });
        
        // Also check against a list of reserved usernames
        const reservedUsernames = ['admin', 'root', 'system', 'moderator'];
        
        return !existingUser && !reservedUsernames.includes(username.toLowerCase());
      },
      message: 'Username is already taken or reserved.'
    }
  },
  website: {
    type: String,
    validate: {
      validator: async function(url) {
        if (!url) return true; // Optional field
        
        try {
          // Check if the website is reachable
          // In a real application, you might use a proper HTTP client
          // This is a simplified example
          const response = await fetch(url, { method: 'HEAD' });
          return response.ok;
        } catch (error) {
          return false;
        }
      },
      message: 'Website is not reachable.'
    }
  }
});
            
Important Note on Asynchronous Validators

Asynchronous validators must return a Promise. The validation passes if the Promise resolves to true and fails if it resolves to false or rejects with an error.

Be careful with asynchronous validators that query the same collection. For new documents, they work fine, but for updates, they might incorrectly detect the document being updated as a duplicate.

Conditional Validation

In many real-world scenarios, validation of one field depends on the value of another field. Mongoose allows you to implement this conditional logic.


const orderSchema = new Schema({
  orderType: {
    type: String,
    enum: ['standard', 'express', 'international'],
    required: true
  },
  
  // Shipping address (required for all orders except digital)
  shippingAddress: {
    street: String,
    city: String,
    state: String,
    zipCode: String,
    country: {
      type: String,
      default: 'USA'
    }
  },
  
  // International shipping details (required only for international orders)
  internationalDetails: {
    customsDeclaration: String,
    taxId: String,
    importLicense: String
  },
  
  // Digital delivery details (for digital products)
  digitalDelivery: {
    email: String,
    downloadLink: String
  },
  
  isDigital: {
    type: Boolean,
    default: false
  }
});

// Conditional validation: Shipping address is required unless it's a digital-only order
orderSchema.path('shippingAddress.street').required(function() {
  return !this.isDigital;
}, 'Shipping address is required for physical orders');

orderSchema.path('shippingAddress.city').required(function() {
  return !this.isDigital;
}, 'Shipping city is required for physical orders');

// Conditional validation: International details required only for international orders
const internationalDetailsValidator = function() {
  return this.orderType === 'international';
};

orderSchema.path('internationalDetails.customsDeclaration').required(
  internationalDetailsValidator,
  'Customs declaration is required for international orders'
);

orderSchema.path('internationalDetails.taxId').required(
  internationalDetailsValidator,
  'Tax ID is required for international orders'
);

// Conditional validation: Digital delivery details required only for digital orders
orderSchema.path('digitalDelivery.email').required(function() {
  return this.isDigital;
}, 'Email is required for digital delivery');
            

Cross-Field Validation

Sometimes validation needs to compare multiple fields. You can implement cross-field validation using custom validators.


const accountSchema = new Schema({
  username: {
    type: String,
    required: true
  },
  password: {
    type: String,
    required: true
  },
  passwordConfirmation: {
    type: String,
    required: true,
    validate: {
      validator: function(value) {
        // Check if password and passwordConfirmation match
        return value === this.password;
      },
      message: 'Passwords do not match.'
    }
  },
  dateOfBirth: {
    type: Date,
    required: true
  },
  retirementDate: {
    type: Date,
    validate: {
      validator: function(value) {
        if (!value) return true; // Optional field
        
        // Retirement date must be after date of birth
        return value > this.dateOfBirth;
      },
      message: 'Retirement date must be after date of birth.'
    }
  },
  
  // Price range validation
  priceMin: {
    type: Number,
    min: 0
  },
  priceMax: {
    type: Number,
    min: 0,
    validate: {
      validator: function(value) {
        if (!value || !this.priceMin) return true; // Both fields are optional
        
        // priceMax must be greater than or equal to priceMin
        return value >= this.priceMin;
      },
      message: 'Maximum price must be greater than or equal to minimum price.'
    }
  }
});
            

Array Validation

MongoDB's flexible schema allows for arrays of various types. Mongoose provides ways to validate both the entire array and individual elements.


const courseSchema = new Schema({
  title: {
    type: String,
    required: true
  },
  
  // Array of simple values with validation
  tags: {
    type: [String],
    validate: {
      validator: function(array) {
        // Must have at least one tag
        return array.length > 0;
      },
      message: 'Course must have at least one tag.'
    }
  },
  
  // Array of objects with nested validation
  lessons: {
    type: [{
      title: {
        type: String,
        required: true
      },
      duration: {
        type: Number,
        required: true,
        min: 1,
        max: 240 // Maximum duration in minutes
      },
      content: {
        type: String,
        required: true,
        minlength: 10
      }
    }],
    validate: {
      validator: function(array) {
        // Must have at least one lesson
        return array.length > 0;
      },
      message: 'Course must have at least one lesson.'
    }
  },
  
  // Custom validation for each element in an array
  prerequisites: {
    type: [String],
    validate: [
      {
        validator: function(array) {
          // Check if all prerequisites are unique
          return new Set(array).size === array.length;
        },
        message: 'Prerequisites must be unique.'
      },
      {
        validator: function(array) {
          // Check if all prerequisites are in proper format
          return array.every(item => /^[A-Z]{3}[0-9]{3}$/.test(item));
        },
        message: 'Prerequisites must be in the format XXX000.'
      }
    ]
  }
});
            

Update Validators

By default, MongoDB's update operations bypass Mongoose's validation. You can enable validation for updates by setting the runValidators option.


// Update a user with validation
const updateUser = async (userId, updates) => {
  try {
    const user = await User.findByIdAndUpdate(
      userId,
      updates,
      {
        new: true, // Return the updated document
        runValidators: true, // Enable update validators
        context: 'query' // Make `this` in validators refer to the query
      }
    );
    
    return user;
  } catch (error) {
    // Handle validation errors
    if (error.name === 'ValidationError') {
      // Process validation errors
      console.error('Validation error:', error.message);
    }
    
    throw error;
  }
};
            
⚠️ Update Validator Limitations
  • Update validators run for each field being updated
  • The document being updated may not be loaded into memory, so some validators (esp. those using this) may not work as expected
  • Setting context: 'query' makes this refer to the query object
  • Custom validators that compare fields won't work unless you load the document first
  • For complex validation during updates, consider using the find-and-update pattern

Schema-wide Validation

Some validation needs to examine the entire document. You can use the validate hook for this purpose.


const projectSchema = new Schema({
  title: {
    type: String,
    required: true
  },
  startDate: {
    type: Date,
    required: true
  },
  endDate: {
    type: Date
  },
  budget: {
    type: Number,
    min: 0
  },
  expenses: {
    type: [{
      description: String,
      amount: Number,
      date: Date
    }]
  },
  status: {
    type: String,
    enum: ['planning', 'ongoing', 'completed', 'cancelled'],
    default: 'planning'
  }
});

// Schema-level validation
projectSchema.pre('validate', function(next) {
  const project = this;
  
  // Ensure end date is after start date
  if (project.endDate && project.startDate && project.endDate < project.startDate) {
    project.invalidate('endDate', 'End date must be after start date.');
  }
  
  // Validate that expenses don't exceed budget
  if (project.budget && project.expenses && project.expenses.length > 0) {
    const totalExpenses = project.expenses.reduce((sum, expense) => sum + expense.amount, 0);
    
    if (totalExpenses > project.budget) {
      project.invalidate('expenses', `Total expenses (${totalExpenses}) exceed budget (${project.budget}).`);
    }
  }
  
  // Validate based on status
  if (project.status === 'completed' && !project.endDate) {
    project.invalidate('endDate', 'End date is required for completed projects.');
  }
  
  next();
});
            

Custom Error Messages

Clear, contextual error messages improve the developer and user experience. Mongoose allows for dynamic error messages that can include field values and other context.


const productSchema = new Schema({
  name: {
    type: String,
    required: [true, 'Product name is required']
  },
  price: {
    type: Number,
    required: [true, 'Product price is required'],
    min: [0, 'Price cannot be negative'],
    validate: {
      validator: function(value) {
        // Price must be a multiple of 0.01 (no more than 2 decimal places)
        return (value * 100) % 1 === 0;
      },
      message: props => `${props.value} is not a valid price. Price must have at most 2 decimal places.`
    }
  },
  sku: {
    type: String,
    required: [true, 'SKU is required'],
    validate: {
      validator: function(value) {
        return /^[A-Z]{2}-\d{4}$/.test(value);
      },
      message: props => `${props.value} is not a valid SKU. Format must be XX-0000.`
    }
  },
  category: {
    type: String,
    required: [true, 'Category is required'],
    enum: {
      values: ['Electronics', 'Clothing', 'Books', 'Home', 'Sports'],
      message: props => `${props.value} is not a supported category. Supported categories: Electronics, Clothing, Books, Home, Sports.`
    }
  }
});
            

Custom Validation Error Handling

When validation fails, Mongoose returns a ValidationError containing detailed information about each invalid field. You can extract and format these errors for user-friendly responses.


// Express route handler with validation error handling
app.post('/api/products', async (req, res) => {
  try {
    const product = new Product(req.body);
    await product.save();
    res.status(201).json(product);
  } catch (error) {
    if (error.name === 'ValidationError') {
      // Format validation errors
      const formattedErrors = formatValidationErrors(error);
      return res.status(400).json({
        message: 'Validation error',
        errors: formattedErrors
      });
    }
    
    // Handle other types of errors
    console.error('Error creating product:', error);
    res.status(500).json({ message: 'Server error' });
  }
});

// Helper function to format validation errors
function formatValidationErrors(error) {
  const formattedErrors = {};
  
  // Loop through all validation errors
  for (const field in error.errors) {
    const fieldError = error.errors[field];
    
    // Format based on error type
    if (fieldError.kind === 'required') {
      formattedErrors[field] = `${fieldError.path} is required`;
    } else if (fieldError.kind === 'enum') {
      formattedErrors[field] = `${fieldError.value} is not a valid option for ${fieldError.path}`;
    } else if (fieldError.kind === 'min') {
      formattedErrors[field] = `${fieldError.path} must be at least ${fieldError.properties.min}`;
    } else if (fieldError.kind === 'max') {
      formattedErrors[field] = `${fieldError.path} must be at most ${fieldError.properties.max}`;
    } else {
      // Use the error message directly
      formattedErrors[field] = fieldError.message;
    }
  }
  
  return formattedErrors;
}
            

Middleware (Hooks) in Depth

Mongoose middleware (hooks) allows you to execute code at specific points in the lifecycle of documents and queries. They are incredibly powerful for implementing business logic, data transformations, and maintaining data consistency.

flowchart LR A[Document Creation] --> B{Pre-save Middleware} B -->|Pass| C[Save to Database] C --> D{Post-save Middleware} D --> E[Document Saved] B -->|Fail/Error| F[Save Cancelled]

Types of Middleware

Mongoose supports four types of middleware:

Document Middleware

Document middleware runs for operations on a specific document. In document middleware, this refers to the document being processed.


const userSchema = new Schema({
  firstName: String,
  lastName: String,
  email: {
    type: String,
    required: true,
    lowercase: true,
    trim: true
  },
  password: {
    type: String,
    required: true
  },
  fullName: String,
  passwordChangedAt: Date,
  lastLogin: Date,
  active: {
    type: Boolean,
    default: true
  }
});

// Pre-save middleware: Set fullName before saving
userSchema.pre('save', function(next) {
  // 'this' refers to the document being saved
  if (this.firstName && this.lastName) {
    this.fullName = `${this.firstName} ${this.lastName}`;
  }
  
  next();
});

// Pre-save middleware: Hash password before saving
userSchema.pre('save', async function(next) {
  // Skip if password hasn't been modified
  if (!this.isModified('password')) return next();
  
  try {
    // Simulated password hashing (in real app, use bcrypt)
    this.password = 'hashed_' + this.password;
    
    // Update passwordChangedAt timestamp if password changed
    if (!this.isNew) {
      this.passwordChangedAt = new Date();
    }
    
    next();
  } catch (error) {
    next(error);
  }
});

// Post-save middleware: Log after document is saved
userSchema.post('save', function(doc, next) {
  console.log(`User saved: ${doc.email}`);
  
  // You could send a welcome email, log to audit trail, etc.
  
  next();
});

// Pre-remove middleware: Clean up related data
userSchema.pre('remove', async function(next) {
  try {
    // Delete related data (e.g., user posts)
    // await Post.deleteMany({ user: this._id });
    
    console.log(`Preparing to delete user: ${this._id}`);
    next();
  } catch (error) {
    next(error);
  }
});

// Post-remove middleware: Cleanup after user is removed
userSchema.post('remove', function(doc, next) {
  console.log(`User removed: ${doc.email}`);
  next();
});
            

Query Middleware

Query middleware runs for query operations. In query middleware, this refers to the query object, not the document.


// Pre-find middleware: Automatically exclude inactive users
userSchema.pre(/^find/, function(next) {
  // 'this' refers to the query, not the document
  this.find({ active: { $ne: false } });
  
  // Record query start time for performance monitoring
  this.startTime = Date.now();
  
  next();
});

// Post-find middleware: Log query performance
userSchema.post(/^find/, function(docs, next) {
  // Log query execution time
  console.log(`Query took ${Date.now() - this.startTime}ms`);
  
  // Log result count
  console.log(`Found ${docs.length} documents`);
  
  next();
});

// Pre-findOne middleware: Add specific conditions
userSchema.pre('findOne', function(next) {
  // Add additional conditions to the query
  this.populate('profile'); // Automatically populate related data
  
  next();
});

// Pre-update middleware: Prevent updating certain fields
userSchema.pre(['updateOne', 'findOneAndUpdate', 'findByIdAndUpdate'], function(next) {
  // Get the update operations
  const update = this.getUpdate();
  
  // Prevent updating certain fields
  if (update.$set && update.$set.role === 'admin') {
    return next(new Error('Cannot update role to admin through this endpoint'));
  }
  
  // Add updated timestamp
  if (!update.$set) update.$set = {};
  update.$set.updatedAt = new Date();
  
  next();
});
            

Aggregate Middleware

Aggregate middleware runs for aggregation operations. In aggregate middleware, this refers to the aggregation object.


// Pre-aggregate middleware: Modify pipeline or add stages
userSchema.pre('aggregate', function(next) {
  // 'this' refers to the aggregation object
  
  // Get the aggregation pipeline
  const pipeline = this.pipeline();
  
  // Add a match stage at the beginning to filter out inactive users
  pipeline.unshift({ $match: { active: { $ne: false } } });
  
  console.log('Aggregate pipeline:', JSON.stringify(pipeline));
  
  next();
});

// Post-aggregate middleware
userSchema.post('aggregate', function(result, next) {
  console.log(`Aggregation returned ${result.length} results`);
  
  next();
});
            

Model Middleware

Model middleware runs for operations at the model level. In model middleware, this refers to the model.


// Pre-insertMany middleware
userSchema.pre('insertMany', function(next, docs) {
  console.log(`Inserting ${docs.length} documents`);
  
  // Modify documents before insertion
  docs.forEach(doc => {
    if (doc.firstName && doc.lastName) {
      doc.fullName = `${doc.firstName} ${doc.lastName}`;
    }
    
    // Set creation timestamp
    doc.createdAt = new Date();
  });
  
  next();
});

// Post-insertMany middleware
userSchema.post('insertMany', function(docs, next) {
  console.log(`Inserted ${docs.length} documents successfully`);
  next();
});
            

Error Handling in Middleware

Middleware can be used to catch and handle errors, or to trigger errors based on business rules.


// Catching errors in middleware
userSchema.pre('save', function(next) {
  try {
    // Potentially error-throwing operation
    if (this.email && this.email.endsWith('forbidden-domain.com')) {
      throw new Error('Email domain not allowed');
    }
    
    next();
  } catch (error) {
    // Pass error to next to trigger error handling
    next(error);
  }
});

// Async error handling with try/catch
userSchema.pre('save', async function(next) {
  try {
    // Async operation that might throw
    await someAsyncOperation();
    next();
  } catch (error) {
    next(error);
  }
});

// Using next(error) to abort the operation with a specific error
userSchema.pre('save', function(next) {
  if (this.role === 'admin' && !this.adminVerified) {
    return next(new Error('Admin accounts require verification'));
  }
  
  next();
});

// Post-error middleware
userSchema.post('save', function(error, doc, next) {
  // Convert certain errors to more user-friendly messages
  if (error.name === 'MongoError' && error.code === 11000) {
    // Duplicate key error
    next(new Error('An account with that email already exists'));
  } else {
    // Pass other errors through
    next(error);
  }
});
            

Middleware Execution Order

Understanding the execution order of middleware is crucial. Middleware runs in series based on the order in which they're defined.


// Middleware execution order demonstration
userSchema.pre('save', function(next) {
  console.log('First pre-save middleware');
  next();
});

userSchema.pre('save', function(next) {
  console.log('Second pre-save middleware');
  next();
});

userSchema.post('save', function(doc, next) {
  console.log('First post-save middleware');
  next();
});

userSchema.post('save', function(doc, next) {
  console.log('Second post-save middleware');
  next();
});

// When saving a user, the output will be:
// 1. "First pre-save middleware"
// 2. "Second pre-save middleware"
// 3. [Document is saved to the database]
// 4. "First post-save middleware"
// 5. "Second post-save middleware"
            
Middleware Execution Chain

If any pre middleware calls next() with an error, Mongoose skips all remaining pre middleware and post middleware and returns the error to the callback.

Post middleware is only executed if the operation completes successfully (unless it's a post-error middleware).

Parallel Middleware Execution

By default, middleware executes serially, but you can run independent middleware in parallel for better performance.


// Parallel middleware execution
const schema = new Schema({ name: String });

// Parallel middleware
schema.pre('save', true, function(next, done) {
  // 'next' is called right away to let Mongoose know that
  // this middleware is asynchronous
  next();
  
  // Perform async work
  setTimeout(() => {
    console.log('First parallel middleware completed');
    done(); // Call done when the middleware is complete
  }, 100);
});

schema.pre('save', true, function(next, done) {
  next();
  
  setTimeout(() => {
    console.log('Second parallel middleware completed');
    done();
  }, 50);
});

// This post middleware will only run after all the parallel
// middleware has completed
schema.post('save', function(doc) {
  console.log('Post middleware executed');
});
            

Real-World Validation and Middleware Patterns

Let's explore some common real-world patterns that combine validation and middleware to solve practical problems.

Pattern 1: Automatic Slug Generation

Generate and validate URL-friendly slugs for blog posts, products, etc.


const slugify = require('slugify');

const postSchema = new Schema({
  title: {
    type: String,
    required: [true, 'Post title is required'],
    trim: true,
    minlength: [3, 'Title must be at least 3 characters long'],
    maxlength: [100, 'Title cannot exceed 100 characters']
  },
  slug: {
    type: String,
    unique: true,
    lowercase: true,
    index: true
  },
  content: {
    type: String,
    required: [true, 'Post content is required']
  },
  author: {
    type: Schema.Types.ObjectId,
    ref: 'User',
    required: true
  },
  published: {
    type: Boolean,
    default: false
  },
  publishedAt: Date
});

// Generate slug from title before validation
postSchema.pre('validate', async function(next) {
  if (!this.isModified('title')) return next();
  
  // Generate slug from title
  let slug = slugify(this.title, {
    lower: true,
    strict: true // Remove special characters
  });
  
  // Ensure slug uniqueness
  const existingPost = await this.constructor.findOne({ slug });
  
  if (existingPost && !existingPost._id.equals(this._id)) {
    // Append a random string to make the slug unique
    const randomStr = Math.random().toString(36).substring(2, 7);
    slug = `${slug}-${randomStr}`;
  }
  
  this.slug = slug;
  next();
});

// Update publishedAt timestamp when post is published
postSchema.pre('save', function(next) {
  if (this.isModified('published') && this.published && !this.publishedAt) {
    this.publishedAt = new Date();
  }
  
  next();
});
            

Pattern 2: Geocoding Addresses

Automatically convert addresses to geographic coordinates.


// In a real app, you would use a proper geocoding service like Google Maps Geocoding API
// This is a simplified example
const geocodeAddress = async (address) => {
  // Simulated geocoding response
  return {
    lat: 37.7749,
    lng: -122.4194
  };
};

const locationSchema = new Schema({
  address: {
    street: {
      type: String,
      required: [true, 'Street address is required']
    },
    city: {
      type: String,
      required: [true, 'City is required']
    },
    state: {
      type: String,
      required: [true, 'State is required']
    },
    zipCode: {
      type: String,
      required: [true, 'Zip code is required'],
      validate: {
        validator: function(v) {
          return /^\d{5}(-\d{4})?$/.test(v);
        },
        message: props => `${props.value} is not a valid zip code`
      }
    },
    country: {
      type: String,
      default: 'USA'
    }
  },
  location: {
    type: {
      type: String,
      enum: ['Point'],
      default: 'Point'
    },
    coordinates: {
      type: [Number], // [longitude, latitude]
      index: '2dsphere' // Create geospatial index
    }
  },
  formattedAddress: String,
  addressUpdatedAt: Date
});

// Geocode address before saving
locationSchema.pre('save', async function(next) {
  // Check if address has been modified
  const addressFields = ['address.street', 'address.city', 'address.state', 'address.zipCode', 'address.country'];
  const isAddressModified = addressFields.some(field => this.isModified(field));
  
  if (!isAddressModified) return next();
  
  try {
    // Format full address
    const { street, city, state, zipCode, country } = this.address;
    this.formattedAddress = `${street}, ${city}, ${state} ${zipCode}, ${country}`;
    
    // Geocode the address
    const { lat, lng } = await geocodeAddress(this.formattedAddress);
    
    // Update coordinates [longitude, latitude]
    this.location.coordinates = [lng, lat];
    
    // Update timestamp
    this.addressUpdatedAt = new Date();
    
    next();
  } catch (error) {
    next(error);
  }
});
            

Pattern 3: Document Version Control

Implement optimistic concurrency control with versioning.


const documentSchema = new Schema({
  title: {
    type: String,
    required: true
  },
  content: {
    type: String,
    required: true
  },
  author: {
    type: Schema.Types.ObjectId,
    ref: 'User',
    required: true
  },
  version: {
    type: Number,
    default: 1
  },
  // Store previous versions
  history: [{
    title: String,
    content: String,
    version: Number,
    modifiedAt: Date,
    modifiedBy: {
      type: Schema.Types.ObjectId,
      ref: 'User'
    }
  }],
  lastModifiedBy: {
    type: Schema.Types.ObjectId,
    ref: 'User'
  },
  lastModifiedAt: {
    type: Date,
    default: Date.now
  }
});

// Pre-save middleware for version control
documentSchema.pre('save', async function(next) {
  if (!this.isModified('title') && !this.isModified('content')) {
    return next();
  }
  
  try {
    // Check if it's not a new document
    if (!this.isNew) {
      // Add current state to history before updating
      this.history.push({
        title: this.title,
        content: this.content,
        version: this.version,
        modifiedAt: this.lastModifiedAt,
        modifiedBy: this.lastModifiedBy
      });
      
      // Increment version
      this.version += 1;
    }
    
    // Update modification timestamp
    this.lastModifiedAt = new Date();
    
    next();
  } catch (error) {
    next(error);
  }
});

// Method to revert to a specific version
documentSchema.methods.revertToVersion = async function(versionNumber, userId) {
  if (versionNumber >= this.version || versionNumber < 1) {
    throw new Error(`Invalid version number: ${versionNumber}`);
  }
  
  // Find the version in history
  const versionToRevert = this.history.find(v => v.version === versionNumber);
  
  if (!versionToRevert) {
    throw new Error(`Version ${versionNumber} not found in history`);
  }
  
  // Add current state to history
  this.history.push({
    title: this.title,
    content: this.content,
    version: this.version,
    modifiedAt: this.lastModifiedAt,
    modifiedBy: this.lastModifiedBy
  });
  
  // Restore fields from the historical version
  this.title = versionToRevert.title;
  this.content = versionToRevert.content;
  
  // Update metadata
  this.version += 1;
  this.lastModifiedAt = new Date();
  this.lastModifiedBy = userId;
  
  // Save the document
  return await this.save();
};
            

Pattern 4: Password Reset and Expiry

Implement secure password reset tokens with automatic expiration.


const crypto = require('crypto');

const userSchema = new Schema({
  email: {
    type: String,
    required: true,
    unique: true,
    lowercase: true
  },
  password: {
    type: String,
    required: true,
    minlength: 8,
    select: false // Don't include password in query results by default
  },
  resetPasswordToken: String,
  resetPasswordExpires: Date,
  passwordChangedAt: Date
});

// Method to generate password reset token
userSchema.methods.createPasswordResetToken = function() {
  // Generate a random token
  const resetToken = crypto.randomBytes(32).toString('hex');
  
  // Hash the token and store it in the database
  this.resetPasswordToken = crypto
    .createHash('sha256')
    .update(resetToken)
    .digest('hex');
  
  // Set expiration (10 minutes)
  this.resetPasswordExpires = Date.now() + 10 * 60 * 1000;
  
  // Save the changes
  return resetToken;
};

// Pre-save middleware to hash password
userSchema.pre('save', async function(next) {
  // Skip if password hasn't been modified
  if (!this.isModified('password')) return next();
  
  try {
    // In a real app, you would use bcrypt or Argon2
    this.password = 'hashed_' + this.password;
    
    // Update passwordChangedAt timestamp
    this.passwordChangedAt = Date.now() - 1000; // Subtract 1 second for safety
    
    next();
  } catch (error) {
    next(error);
  }
});

// Method to verify if token has expired
userSchema.methods.isResetTokenValid = function(token) {
  // Hash the provided token
  const hashedToken = crypto
    .createHash('sha256')
    .update(token)
    .digest('hex');
  
  // Check if the token matches and hasn't expired
  return (
    this.resetPasswordToken === hashedToken &&
    this.resetPasswordExpires > Date.now()
  );
};

// Query middleware to check password reset token expiry
userSchema.pre('findOne', function(next) {
  // If querying by reset token, also check if it's still valid
  if (this._conditions.resetPasswordToken) {
    this.where('resetPasswordExpires').gt(Date.now());
  }
  
  next();
});
            

Pattern 5: Automatic Data Syncing

Keep denormalized data in sync across collections.


// User schema with a subset of denormalized data
const userSchema = new Schema({
  name: {
    type: String,
    required: true
  },
  email: {
    type: String,
    required: true,
    unique: true
  },
  role: {
    type: String,
    enum: ['user', 'admin'],
    default: 'user'
  },
  avatar: String
});

// Post schema with denormalized author information
const postSchema = new Schema({
  title: {
    type: String,
    required: true
  },
  content: {
    type: String,
    required: true
  },
  // Reference to author
  author: {
    type: Schema.Types.ObjectId,
    ref: 'User',
    required: true
  },
  // Denormalized author information
  authorInfo: {
    name: String,
    email: String,
    avatar: String
  },
  createdAt: {
    type: Date,
    default: Date.now
  }
});

// Pre-save middleware to populate denormalized author info
postSchema.pre('save', async function(next) {
  if (this.isNew || this.isModified('author')) {
    try {
      // Find the author
      const author = await mongoose.model('User').findById(this.author);
      
      if (!author) {
        return next(new Error('Author not found'));
      }
      
      // Update denormalized fields
      this.authorInfo = {
        name: author.name,
        email: author.email,
        avatar: author.avatar
      };
      
      next();
    } catch (error) {
      next(error);
    }
  } else {
    next();
  }
});

// Post-save middleware on User model to update denormalized data
userSchema.post('save', async function(doc) {
  // Skip if not modifying fields that are denormalized
  if (!doc.isModified('name') && !doc.isModified('email') && !doc.isModified('avatar')) {
    return;
  }
  
  try {
    // Update all posts with this author
    await mongoose.model('Post').updateMany(
      { author: doc._id },
      {
        $set: {
          'authorInfo.name': doc.name,
          'authorInfo.email': doc.email,
          'authorInfo.avatar': doc.avatar
        }
      }
    );
  } catch (error) {
    console.error('Error updating denormalized data:', error);
  }
});
            

Best Practices for Validation and Middleware

Validation Best Practices

  • Validate Early: Catch errors before they reach the database
  • Layer Your Validation: Use Mongoose validation, custom validators, and application-level validation
  • Be Specific with Error Messages: Clear error messages help developers and users understand what went wrong
  • Test Edge Cases: Validate for null, undefined, empty strings, etc.
  • Use Async Validation Carefully: Be aware of performance implications
  • Consider Client-Side Validation: Provide immediate feedback to users while still validating on the server

Middleware Best Practices

  • Keep Middleware Focused: Each middleware should do one thing well
  • Handle Errors Properly: Use try/catch and next(error) patterns
  • Be Mindful of Performance: Avoid expensive operations in frequently used middleware
  • Use Middleware for Cross-Cutting Concerns: Auditing, logging, timestamps, etc.
  • Understand Context: Know when this refers to a document, query, or model
  • Don't Abuse Middleware: Some logic belongs in controllers or services, not middleware

General Best Practices

  • Security First: Use middleware for sensitive operations like password hashing
  • Consistency is Key: Use similar patterns across your application
  • Documentation: Comment middleware to explain what it does and why
  • Testing: Write unit tests for custom validators and middleware
  • Keep Models Focused: Split complex models into smaller, more manageable parts
  • Use Plugins: For common patterns, consider creating reusable plugins

Common Anti-Patterns to Avoid

  • Excessive Middleware: Too many middleware hooks can make code hard to follow
  • Not Calling next(): Forgetting to call next() will hang your application
  • Heavy Processing in Middleware: Long-running operations can block other operations
  • Circular Dependencies: Avoid middleware that can trigger infinite loops
  • Overly Complex Validation: If validation is too complex, consider moving logic to a service layer
  • Inconsistent Error Handling: Establish consistent patterns for handling validation errors

Practical Activities

Activity 1: User Registration System

Implement a complete user registration system with validation and middleware:

  1. Create a User schema with appropriate validation for:
    • Username (alphanumeric, unique)
    • Email (valid format, unique)
    • Password (minimum length, complexity requirements)
    • Profile information (name, bio, etc.)
  2. Implement middleware for:
    • Password hashing
    • Default avatar generation
    • Welcome email sending (simulated)
    • Audit logging
  3. Create Express routes for:
    • User registration
    • Profile editing
    • Password changing
  4. Implement proper error handling for validation errors

Activity 2: Blog Post System with Comments

Create a blog post system with validation and middleware:

  1. Create schemas for:
    • Posts (title, content, author, tags, slug)
    • Comments (content, author, post reference)
  2. Implement validation for:
    • Required fields
    • Content length
    • Valid references
  3. Add middleware for:
    • Automatic slug generation
    • Comment count updating
    • Markdown-to-HTML conversion (simulated)
    • Notification generation when commented on
  4. Implement API endpoints for CRUD operations

Activity 3: E-commerce Product Inventory

Develop an e-commerce product inventory system:

  1. Create schemas for:
    • Products (name, description, price, stock)
    • Categories (name, description)
    • Inventory Transactions (product, quantity, type)
  2. Implement validation for:
    • Price (positive, proper format)
    • Stock levels (non-negative)
    • Required fields and relationships
  3. Add middleware for:
    • Stock level updates when transactions occur
    • Low stock alerts
    • Price history tracking
    • Automatic SKU generation
  4. Create API endpoints for product management and inventory operations

Further Reading and Resources

Coming Up: Population and Relationships

In our next session, we'll dive deeper into MongoDB relationships and Mongoose's population feature. We'll explore advanced techniques for working with related data, handling nested documents, and optimizing queries involving multiple collections.