Node.js Core Modules

Introduction to Core Modules

Node.js ships with a set of core modules that provide essential functionality for building server-side applications. These modules are immediately available for use without any additional installation and form the backbone of the Node.js ecosystem.

What are Core Modules?

Core modules are built-in JavaScript libraries included in the Node.js runtime. They provide low-level functionality like file system operations, networking, path manipulation, and more. They're carefully optimized for performance and designed with Node.js's event-driven architecture in mind.

flowchart TD A[Node.js Application] --> B[Core Modules] A --> C[Third-Party Modules] A --> D[Local Modules] B --> E[fs: File System] B --> F[http: HTTP/HTTPS] B --> G[path: Path Utilities] B --> H[os: Operating System] B --> I[util: Utilities] B --> J[events: Event Emitter] B --> K[stream: Streaming] B --> L[..."and many more"]

Using Core Modules

Core modules are imported using the require() function. Since they're built into Node.js, you only need to specify the module name:

// Importing core modules
const fs = require('fs');
const http = require('http');
const path = require('path');

// Using multiple core modules together
http.createServer((req, res) => {
  const filePath = path.join(__dirname, 'public', 'index.html');
  
  fs.readFile(filePath, (err, content) => {
    if (err) {
      res.writeHead(500);
      res.end('Server Error');
      return;
    }
    
    res.writeHead(200, { 'Content-Type': 'text/html' });
    res.end(content);
  });
}).listen(3000, () => {
  console.log('Server running on port 3000');
});

The Toolbox Analogy

Think of Node.js core modules as a professional toolbox that comes standard with every purchase:

You don't have to go to the hardware store (npm) to get these essential tools
Each tool (module) has a specific purpose and is designed for particular tasks
Some tools (like fs, http) you'll use daily, while others (like zlib, dgram) are for specialized tasks
You can always add more specialized tools (third-party modules) to your collection, but the standard toolbox provides everything needed for most common tasks
Like a good toolbox, core modules are organized logically so you can find what you need quickly

File System (fs) Module

The file system module provides an API for interacting with the file system in a way that closely models the standard POSIX functions. It's one of the most frequently used core modules in Node.js applications.

flowchart LR A[fs Module] --> B[File Reading] A --> C[File Writing] A --> D[Directory Operations] A --> E[File Information] A --> F[File Watching] B --> B1[readFile] B --> B2[readFileSync] B --> B3[createReadStream] C --> C1[writeFile] C --> C2[writeFileSync] C --> C3[createWriteStream] D --> D1[mkdir] D --> D2[readdir] D --> D3[rmdir] E --> E1[stat] E --> E2[access] E --> E3[chmod] F --> F1[watch] F --> F2[watchFile]

Key fs Module Operations

Reading Files

Node.js provides multiple ways to read files, each with different characteristics:

// Asynchronous file reading with callback
fs.readFile('example.txt', 'utf8', (err, data) => {
  if (err) {
    console.error('Error reading file:', err);
    return;
  }
  console.log(data);
});

// Synchronous file reading (blocks the event loop)
try {
  const data = fs.readFileSync('example.txt', 'utf8');
  console.log(data);
} catch (err) {
  console.error('Error reading file:', err);
}

// Using Promises API (modern approach)
fs.promises.readFile('example.txt', 'utf8')
  .then(data => console.log(data))
  .catch(err => console.error('Error reading file:', err));

// Async/await with Promises API
async function readFileContent() {
  try {
    const data = await fs.promises.readFile('example.txt', 'utf8');
    console.log(data);
  } catch (err) {
    console.error('Error reading file:', err);
  }
}
readFileContent();

// Stream-based reading (for large files)
const readStream = fs.createReadStream('largefile.txt', { encoding: 'utf8' });
readStream.on('data', (chunk) => {
  console.log(`Received ${chunk.length} bytes of data`);
});
readStream.on('end', () => {
  console.log('Finished reading file');
});
readStream.on('error', (err) => {
  console.error('Error reading file:', err);
});

Writing Files

Similarly, Node.js offers multiple approaches to write files:

// Asynchronous file writing with callback
fs.writeFile('output.txt', 'Hello, World!', (err) => {
  if (err) {
    console.error('Error writing file:', err);
    return;
  }
  console.log('File written successfully');
});

// Synchronous file writing
try {
  fs.writeFileSync('output.txt', 'Hello, World!');
  console.log('File written successfully');
} catch (err) {
  console.error('Error writing file:', err);
}

// Appending to files
fs.appendFile('log.txt', 'New log entry\n', (err) => {
  if (err) {
    console.error('Error appending to file:', err);
    return;
  }
  console.log('Data appended to file');
});

// Stream-based writing (for large data)
const writeStream = fs.createWriteStream('output-stream.txt');
writeStream.write('First chunk of data\n');
writeStream.write('Second chunk of data\n');
writeStream.end('Final chunk of data');
writeStream.on('finish', () => {
  console.log('All data has been written');
});
writeStream.on('error', (err) => {
  console.error('Error writing to file:', err);
});

Directory Operations

The fs module also provides functions for working with directories:

// Creating directories
fs.mkdir('new-directory', (err) => {
  if (err) {
    console.error('Error creating directory:', err);
    return;
  }
  console.log('Directory created');
});

// Creating nested directories
fs.mkdir('parent/child/grandchild', { recursive: true }, (err) => {
  if (err) {
    console.error('Error creating nested directories:', err);
    return;
  }
  console.log('Nested directories created');
});

// Reading directory contents
fs.readdir('some-directory', (err, files) => {
  if (err) {
    console.error('Error reading directory:', err);
    return;
  }
  console.log('Directory contents:', files);
});

// Removing directories
fs.rmdir('empty-directory', (err) => {
  if (err) {
    console.error('Error removing directory:', err);
    return;
  }
  console.log('Directory removed');
});

Real-World Example: Log Rotation

Here's a practical example of using the fs module to implement a simple log rotation system:

const fs = require('fs');
const path = require('path');

class SimpleLogger {
  constructor(options = {}) {
    this.logDir = options.logDir || 'logs';
    this.logFile = options.logFile || 'app.log';
    this.maxSize = options.maxSize || 1024 * 1024; // 1 MB default
    this.maxFiles = options.maxFiles || 5;
    
    // Create log directory if it doesn't exist
    if (!fs.existsSync(this.logDir)) {
      fs.mkdirSync(this.logDir, { recursive: true });
    }
    
    this._initLogStream();
  }
  
  _initLogStream() {
    this.logPath = path.join(this.logDir, this.logFile);
    this.stream = fs.createWriteStream(this.logPath, { flags: 'a' });
    
    // Check current file size
    if (fs.existsSync(this.logPath)) {
      const stats = fs.statSync(this.logPath);
      if (stats.size > this.maxSize) {
        this._rotateLog();
      }
    }
  }
  
  _rotateLog() {
    // Close current stream
    this.stream.end();
    
    // Rotate existing log files
    for (let i = this.maxFiles - 1; i > 0; i--) {
      const oldFile = path.join(this.logDir, `${this.logFile}.${i}`);
      const newFile = path.join(this.logDir, `${this.logFile}.${i + 1}`);
      
      if (fs.existsSync(oldFile)) {
        if (i === this.maxFiles - 1) {
          // Remove oldest log file if we've reached max
          fs.unlinkSync(oldFile);
        } else {
          // Rename file to next index
          fs.renameSync(oldFile, newFile);
        }
      }
    }
    
    // Rename current log file
    fs.renameSync(
      this.logPath,
      path.join(this.logDir, `${this.logFile}.1`)
    );
    
    // Create a new log stream
    this._initLogStream();
  }
  
  log(message) {
    const timestamp = new Date().toISOString();
    const logEntry = `${timestamp} - ${message}\n`;
    
    this.stream.write(logEntry);
    
    // Check if we need to rotate after this write
    const stats = fs.statSync(this.logPath);
    if (stats.size > this.maxSize) {
      this._rotateLog();
    }
  }
  
  close() {
    if (this.stream) {
      this.stream.end();
    }
  }
}

// Usage
const logger = new SimpleLogger({ logDir: 'app-logs' });
logger.log('Application started');
// More log entries...
// logger.close(); // When shutting down

This example demonstrates several fs module functions working together to implement a real application feature. In production, you might use a specialized logging library like winston or pino, but understanding how to work with the fs module allows you to build custom solutions when needed.

HTTP Module

The HTTP module is central to Node.js's identity as a platform for building web servers and services. It provides the foundation for creating HTTP servers and making HTTP client requests.

flowchart TD A[HTTP Module] --> B[Server] A --> C[Client] B --> B1[http.createServer()] B --> B2[server.listen()] B --> B3[Request Object] B --> B4[Response Object] C --> C1[http.get()] C --> C2[http.request()]

Creating an HTTP Server

At its core, an HTTP server in Node.js listens for requests and sends back responses:

const http = require('http');

// Create an HTTP server
const server = http.createServer((req, res) => {
  // Request information
  console.log(`Request Method: ${req.method}`);
  console.log(`URL: ${req.url}`);
  console.log(`Headers:`, req.headers);
  
  // Set response headers
  res.setHeader('Content-Type', 'text/html');
  res.setHeader('X-Powered-By', 'Node.js');
  
  // Set status code
  res.statusCode = 200;
  
  // Send response body
  res.end(`
    <!DOCTYPE html>
    <html>
      <head>
        <title>Node.js HTTP Server</title>
      </head>
      <body>
        <h1>Hello from Node.js!</h1>
        <p>You requested: ${req.url}</p>
      </body>
    </html>
  `);
});

// Start the server on port 3000
server.listen(3000, () => {
  console.log('Server running at http://localhost:3000/');
});

Handling Different Routes

A basic routing system can be implemented by checking the request URL:

const http = require('http');
const fs = require('fs');
const path = require('path');

const server = http.createServer((req, res) => {
  // Parse the URL path
  const url = req.url;
  
  // Handle different routes
  if (url === '/' || url === '/home') {
    res.writeHead(200, { 'Content-Type': 'text/html' });
    res.end('<h1>Home Page</h1>');
  } 
  else if (url === '/about') {
    res.writeHead(200, { 'Content-Type': 'text/html' });
    res.end('<h1>About Page</h1><p>This is a simple Node.js server.</p>');
  } 
  else if (url === '/api/users') {
    // Return JSON data
    const users = [
      { id: 1, name: 'Alice Smith' },
      { id: 2, name: 'Bob Johnson' }
    ];
    
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify(users));
  } 
  else {
    // 404 Not Found
    res.writeHead(404, { 'Content-Type': 'text/html' });
    res.end('<h1>404 - Page Not Found</h1>');
  }
});

server.listen(3000, () => {
  console.log('Server running at http://localhost:3000/');
});

Serving Static Files

A common task is serving static files like HTML, CSS, JavaScript, and images:

const http = require('http');
const fs = require('fs');
const path = require('path');

const server = http.createServer((req, res) => {
  // Protect against directory traversal attacks
  const sanitizedUrl = path.normalize(req.url).replace(/^(\.\.[\/\\])+/, '');
  let filePath = path.join(__dirname, 'public', sanitizedUrl === '/' ? 'index.html' : sanitizedUrl);
  
  // Get the file extension
  const extname = path.extname(filePath);
  
  // Initial content type
  let contentType = 'text/html';
  
  // Check ext and set content type
  switch (extname) {
    case '.js':
      contentType = 'text/javascript';
      break;
    case '.css':
      contentType = 'text/css';
      break;
    case '.json':
      contentType = 'application/json';
      break;
    case '.png':
      contentType = 'image/png';
      break;
    case '.jpg':
    case '.jpeg':
      contentType = 'image/jpeg';
      break;
  }
  
  // Read file
  fs.readFile(filePath, (err, content) => {
    if (err) {
      if (err.code === 'ENOENT') {
        // Page not found
        fs.readFile(path.join(__dirname, 'public', '404.html'), (err, content) => {
          res.writeHead(404, { 'Content-Type': 'text/html' });
          res.end(content, 'utf8');
        });
      } else {
        // Some server error
        res.writeHead(500);
        res.end(`Server Error: ${err.code}`);
      }
    } else {
      // Success
      res.writeHead(200, { 'Content-Type': contentType });
      res.end(content, 'utf8');
    }
  });
});

server.listen(3000, () => {
  console.log('Server running at http://localhost:3000/');
});

Making HTTP Requests

The HTTP module also allows Node.js to act as a client, making requests to other servers:

const http = require('http');

// Simple GET request
http.get('http://api.example.com/data', (res) => {
  let data = '';
  
  // A chunk of data has been received
  res.on('data', (chunk) => {
    data += chunk;
  });
  
  // The whole response has been received
  res.on('end', () => {
    console.log(JSON.parse(data));
  });
}).on('error', (err) => {
  console.error(`Error: ${err.message}`);
});

// More complex POST request
const postData = JSON.stringify({
  name: 'John Doe',
  email: 'john@example.com'
});

const options = {
  hostname: 'api.example.com',
  port: 80,
  path: '/users',
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': Buffer.byteLength(postData)
  }
};

const req = http.request(options, (res) => {
  let data = '';
  
  res.on('data', (chunk) => {
    data += chunk;
  });
  
  res.on('end', () => {
    console.log(`Response status: ${res.statusCode}`);
    console.log(JSON.parse(data));
  });
});

req.on('error', (err) => {
  console.error(`Problem with request: ${err.message}`);
});

// Write data to request body
req.write(postData);
req.end();

HTTP vs HTTPS

Node.js also provides an https module that works almost identically to the http module but uses TLS/SSL encryption. For production applications, you should typically use HTTPS to secure your connections.

const https = require('https');
const fs = require('fs');

const options = {
  key: fs.readFileSync('private-key.pem'),
  cert: fs.readFileSync('certificate.pem')
};

https.createServer(options, (req, res) => {
  res.writeHead(200);
  res.end('Hello, secure world!');
}).listen(443);

Real-World Example: Simple API Proxy

Here's a practical example of using the HTTP module to create an API proxy that forwards requests to another server and caches the results:

const http = require('http');
const url = require('url');

// Simple in-memory cache
const cache = new Map();
const CACHE_DURATION = 60 * 1000; // 1 minute in milliseconds

const server = http.createServer((req, res) => {
  // Parse the requested URL
  const parsedUrl = url.parse(req.url, true);
  const path = parsedUrl.pathname;
  
  // Only handle GET requests to /api/
  if (req.method === 'GET' && path.startsWith('/api/')) {
    // Create a unique cache key based on full URL
    const cacheKey = req.url;
    
    // Check if we have a cached response
    if (cache.has(cacheKey)) {
      const cachedResponse = cache.get(cacheKey);
      
      // Check if cache is still valid
      if (Date.now() - cachedResponse.timestamp < CACHE_DURATION) {
        console.log(`Cache hit for ${cacheKey}`);
        
        // Set headers from cached response
        Object.entries(cachedResponse.headers).forEach(([key, value]) => {
          res.setHeader(key, value);
        });
        
        // Send cached response body
        res.writeHead(200);
        res.end(cachedResponse.data);
        return;
      } else {
        // Cache expired
        cache.delete(cacheKey);
      }
    }
    
    // Remove /api from path to get the actual API endpoint
    const apiPath = path.slice(4);
    
    // Target API server details
    const apiOptions = {
      hostname: 'api.example.com',
      port: 80,
      path: apiPath + (parsedUrl.search || ''),
      method: 'GET',
      headers: {
        'User-Agent': 'Node.js API Proxy'
      }
    };
    
    // Forward the request to the target API
    const apiReq = http.request(apiOptions, (apiRes) => {
      let responseData = '';
      
      // Collect response data
      apiRes.on('data', (chunk) => {
        responseData += chunk;
      });
      
      apiRes.on('end', () => {
        // Store response in cache
        cache.set(cacheKey, {
          data: responseData,
          headers: apiRes.headers,
          timestamp: Date.now()
        });
        
        console.log(`Cached response for ${cacheKey}`);
        
        // Forward headers from API response
        Object.entries(apiRes.headers).forEach(([key, value]) => {
          res.setHeader(key, value);
        });
        
        // Send response to client
        res.writeHead(apiRes.statusCode);
        res.end(responseData);
      });
    });
    
    apiReq.on('error', (err) => {
      console.error(`API request error: ${err.message}`);
      res.writeHead(500);
      res.end('Internal Server Error');
    });
    
    apiReq.end();
  } else {
    // Handle unsupported routes/methods
    res.writeHead(404);
    res.end('Not Found');
  }
});

server.listen(3000, () => {
  console.log('Proxy server running at http://localhost:3000/');
  
  // Periodically clean up expired cache entries
  setInterval(() => {
    const now = Date.now();
    let expiredCount = 0;
    
    for (const [key, value] of cache.entries()) {
      if (now - value.timestamp > CACHE_DURATION) {
        cache.delete(key);
        expiredCount++;
      }
    }
    
    if (expiredCount > 0) {
      console.log(`Cleaned up ${expiredCount} expired cache entries`);
    }
  }, CACHE_DURATION);
});

This example demonstrates a practical use case that combines the HTTP module with other core modules to create a useful microservice. Such a proxy can help reduce load on external APIs, improve response times for clients, and add a layer of abstraction between your application and third-party services.

Path Module

The path module provides utilities for working with file and directory paths. It helps solve cross-platform path issues and provides many helpful path manipulation functions.

flowchart LR A[Path Module] --> B[path.join()] A --> C[path.resolve()] A --> D[path.basename()] A --> E[path.dirname()] A --> F[path.extname()] A --> G[path.parse()] A --> H[path.normalize()] A --> I[path.relative()]

Key Path Operations

const path = require('path');

// Join path segments (handles path separators for current OS)
const fullPath = path.join(__dirname, 'public', 'css', 'styles.css');
console.log(fullPath); // e.g., /home/user/project/public/css/styles.css

// Resolve to an absolute path (like navigation in a file system)
const absolutePath = path.resolve('docs', 'example.md');
console.log(absolutePath); // e.g., /home/user/current-dir/docs/example.md

// Get the base filename
const fileName = path.basename('/users/docs/report.pdf');
console.log(fileName); // report.pdf

// Get the base filename without extension
const fileNameWithoutExt = path.basename('/users/docs/report.pdf', '.pdf');
console.log(fileNameWithoutExt); // report

// Get directory name
const dirName = path.dirname('/users/docs/report.pdf');
console.log(dirName); // /users/docs

// Get file extension
const extension = path.extname('script.js');
console.log(extension); // .js

// Parse a path into its components
const parsedPath = path.parse('/home/user/docs/report.pdf');
console.log(parsedPath);
// {
//   root: '/',
//   dir: '/home/user/docs',
//   base: 'report.pdf',
//   ext: '.pdf',
//   name: 'report'
// }

// Normalize a path (resolves '..' and '.')
const normalizedPath = path.normalize('/users/./docs/../docs/report.pdf');
console.log(normalizedPath); // /users/docs/report.pdf

// Get relative path from one path to another
const relativePath = path.relative('/home/user/docs', '/home/user/photos');
console.log(relativePath); // ../photos

Why Path Module Matters

The path module solves several important problems:

Cross-platform compatibility: Windows uses backslashes (\) for paths while Unix-like systems use forward slashes (/)
Directory traversal security: Properly handling paths helps prevent directory traversal attacks
Relative path resolution: Converting relative paths to absolute paths reliably
Path normalization: Handling redundant separators, dots, and other path anomalies

Here's a practical example that creates a file tree crawler using the path module:

const fs = require('fs');
const path = require('path');

function crawlDirectory(directory, callback) {
  // Read directory contents
  fs.readdir(directory, { withFileTypes: true }, (err, entries) => {
    if (err) {
      return callback(err);
    }
    
    // Process each file/directory
    let remaining = entries.length;
    
    // Handle empty directories
    if (remaining === 0) {
      return callback(null, []);
    }
    
    const results = [];
    
    entries.forEach(entry => {
      // Create full path to this file/directory
      const fullPath = path.join(directory, entry.name);
      
      if (entry.isDirectory()) {
        // Recursively crawl subdirectories
        crawlDirectory(fullPath, (err, subEntries) => {
          if (err) {
            return callback(err);
          }
          
          // Add directory and its entries to results
          results.push({
            type: 'directory',
            name: entry.name,
            path: fullPath,
            children: subEntries
          });
          
          // Check if we've processed all entries
          if (--remaining === 0) {
            callback(null, results);
          }
        });
      } else {
        // Add file info to results
        results.push({
          type: 'file',
          name: entry.name,
          path: fullPath,
          extension: path.extname(entry.name)
        });
        
        // Check if we've processed all entries
        if (--remaining === 0) {
          callback(null, results);
        }
      }
    });
  });
}

// Example usage
crawlDirectory('./project', (err, fileTree) => {
  if (err) {
    console.error('Error crawling directory:', err);
    return;
  }
  
  console.log(JSON.stringify(fileTree, null, 2));
});

Other Essential Core Modules

Events Module

Provides the EventEmitter class, which is fundamental to Node.js's event-driven architecture.

const EventEmitter = require('events');

class Logger extends EventEmitter {
  log(message) {
    // Record log message
    console.log(`Logged: ${message}`);
    
    // Emit an event
    this.emit('logged', { message });
  }
}

const logger = new Logger();

// Register event listener
logger.on('logged', (data) => {
  console.log(`Event emitted with: ${data.message}`);
});

logger.log('Hello, world!');

URL Module

Provides utilities for URL parsing and formatting.

const url = require('url');

// Parse a URL string
const parsedUrl = new URL('https://example.com/path?query=123#section');

console.log(parsedUrl.hostname); // example.com
console.log(parsedUrl.pathname); // /path
console.log(parsedUrl.searchParams.get('query')); // 123
console.log(parsedUrl.hash); // #section

// Create a URL
const newUrl = new URL('path', 'https://base.com');
console.log(newUrl.href); // https://base.com/path

OS Module

Provides operating system-related utility methods.

const os = require('os');

// System information
console.log(`Hostname: ${os.hostname()}`);
console.log(`Platform: ${os.platform()}`);
console.log(`Architecture: ${os.arch()}`);
console.log(`CPU cores: ${os.cpus().length}`);
console.log(`Total memory: ${os.totalmem() / 1024 / 1024 / 1024} GB`);
console.log(`Free memory: ${os.freemem() / 1024 / 1024 / 1024} GB`);
console.log(`Uptime: ${os.uptime() / 60 / 60} hours`);

// Network interfaces
console.log('Network interfaces:');
console.log(os.networkInterfaces());

Stream Module

Provides an abstract interface for working with streaming data.

const fs = require('fs');
const zlib = require('zlib');

// Pipe example: compress a file
fs.createReadStream('input.txt')
  .pipe(zlib.createGzip())
  .pipe(fs.createWriteStream('input.txt.gz'))
  .on('finish', () => {
    console.log('File compressed');
  });

Util Module

Provides utility functions useful for developers.

const util = require('util');
const fs = require('fs');

// Convert callback-based function to Promise-based
const readFilePromise = util.promisify(fs.readFile);

// Now we can use promises/async-await
async function readConfig() {
  try {
    const data = await readFilePromise('config.json', 'utf8');
    return JSON.parse(data);
  } catch (err) {
    console.error('Error reading config:', err);
    return {};
  }
}

readConfig().then(config => {
  console.log('Config loaded:', config);
});

Crypto Module

Provides cryptographic functionality.

const crypto = require('crypto');

// Hash a string
function hashPassword(password) {
  const hash = crypto.createHash('sha256');
  hash.update(password);
  return hash.digest('hex');
}

console.log(hashPassword('secret123'));

// Encrypt and decrypt data
function encrypt(text, key) {
  const algorithm = 'aes-256-cbc';
  const iv = crypto.randomBytes(16);
  const cipher = crypto.createCipheriv(algorithm, Buffer.from(key), iv);
  
  let encrypted = cipher.update(text, 'utf8', 'hex');
  encrypted += cipher.final('hex');
  
  return {
    iv: iv.toString('hex'),
    encryptedData: encrypted
  };
}

function decrypt(encrypted, key) {
  const algorithm = 'aes-256-cbc';
  const iv = Buffer.from(encrypted.iv, 'hex');
  const decipher = crypto.createDecipheriv(algorithm, Buffer.from(key), iv);
  
  let decrypted = decipher.update(encrypted.encryptedData, 'hex', 'utf8');
  decrypted += decipher.final('utf8');
  
  return decrypted;
}

// Generate a 32-byte key (256 bits)
const key = crypto.randomBytes(32).toString('hex');
const text = 'Sensitive information';

const encryptedData = encrypt(text, key);
console.log('Encrypted:', encryptedData);

const decryptedText = decrypt(encryptedData, key);
console.log('Decrypted:', decryptedText);

Practical Exercise

Exercise 1: Build a Simple File Explorer

Create a web-based file explorer using Node.js core modules that allows users to navigate directories and view file contents.

// file_explorer.js
const http = require('http');
const fs = require('fs');
const path = require('path');
const url = require('url');

// Base directory to explore (adjust as needed)
const baseDir = process.cwd();

const server = http.createServer((req, res) => {
  // Parse the URL
  const parsedUrl = url.parse(req.url, true);
  const pathname = decodeURIComponent(parsedUrl.pathname);
  
  // Handle directory listing
  if (pathname === '/' || pathname.endsWith('/')) {
    const dirPath = path.join(baseDir, pathname === '/' ? '' : pathname);
    
    // Protect against directory traversal
    if (!dirPath.startsWith(baseDir)) {
      res.writeHead(403, { 'Content-Type': 'text/html' });
      res.end('<h1>Access Denied</h1>');
      return;
    }
    
    try {
      const files = fs.readdirSync(dirPath);
      
      // Get parent directory path
      const parentDir = path.relative(baseDir, dirPath).split(path.sep).slice(0, -1).join('/');
      const parentLink = parentDir ? `/${parentDir}/` : '/';
      
      // Prepare HTML content
      let html = `
        <!DOCTYPE html>
        <html>
        <head>
          <title>File Explorer: ${pathname}</title>
          <style>
            body { font-family: Arial, sans-serif; margin: 20px; }
            h1 { color: #333; }
            .breadcrumb { margin-bottom: 20px; background: #f5f5f5; padding: 10px; }
            .directory { font-weight: bold; color: #0066cc; }
            table { width: 100%; border-collapse: collapse; }
            th, td { text-align: left; padding: 8px; border-bottom: 1px solid #ddd; }
            th { background-color: #f2f2f2; }
            tr:hover { background-color: #f5f5f5; }
          </style>
        </head>
        <body>
          <h1>File Explorer</h1>
          
          <div class="breadcrumb">
            Location: <a href="/">Root</a>
      `;
      
      // Add breadcrumb navigation
      if (pathname !== '/') {
        const parts = pathname.split('/').filter(Boolean);
        let currentPath = '';
        
        for (const part of parts) {
          currentPath += `/${part}`;
          html += ` / <a href="${currentPath}">${part}</a>`;
        }
      }
      
      html += `
          </div>
          
          <table>
            <thead>
              <tr>
                <th>Name</th>
                <th>Type</th>
                <th>Size</th>
                <th>Modified</th>
              </tr>
            </thead>
            <tbody>
      `;
      
      // Add parent directory link if not at root
      if (pathname !== '/') {
        html += `
          <tr>
            <td><a href="${parentLink}">..</a></td>
            <td>Directory</td>
            <td>-</td>
            <td>-</td>
          </tr>
        `;
      }
      
      // Process each file/directory
      for (const file of files) {
        const filePath = path.join(dirPath, file);
        const stats = fs.statSync(filePath);
        const isDirectory = stats.isDirectory();
        const relativePath = path.join(pathname, file);
        
        // Skip hidden files (starting with .)
        if (file.startsWith('.')) continue;
        
        const fileSize = isDirectory ? '-' : formatFileSize(stats.size);
        const modified = stats.mtime.toLocaleString();
        const fileType = isDirectory ? 'Directory' : getFileType(file);
        
        html += `
          <tr>
            <td>
              <a href="${relativePath}${isDirectory ? '/' : ''}" class="${isDirectory ? 'directory' : ''}">
                ${file}
              </a>
            </td>
            <td>${fileType}</td>
            <td>${fileSize}</td>
            <td>${modified}</td>
          </tr>
        `;
      }
      
      html += `
            </tbody>
          </table>
        </body>
        </html>
      `;
      
      res.writeHead(200, { 'Content-Type': 'text/html' });
      res.end(html);
    } catch (err) {
      res.writeHead(500, { 'Content-Type': 'text/html' });
      res.end(`<h1>Error</h1><p>${err.message}</p>`);
    }
  } 
  // Handle file viewing
  else {
    const filePath = path.join(baseDir, pathname);
    
    // Protect against directory traversal
    if (!filePath.startsWith(baseDir)) {
      res.writeHead(403, { 'Content-Type': 'text/html' });
      res.end('<h1>Access Denied</h1>');
      return;
    }
    
    try {
      const stats = fs.statSync(filePath);
      
      if (stats.isDirectory()) {
        // Redirect to directory with trailing slash
        res.writeHead(302, { 'Location': `${pathname}/` });
        res.end();
        return;
      }
      
      // Get file extension and determine content type
      const ext = path.extname(filePath).toLowerCase();
      const contentType = getContentType(ext);
      
      // Check if file should be displayed in browser
      if (isBinaryFile(ext)) {
        // Serve file for download
        res.writeHead(200, {
          'Content-Type': contentType,
          'Content-Disposition': `attachment; filename="${path.basename(filePath)}"`
        });
        fs.createReadStream(filePath).pipe(res);
      } else {
        // Read file and display its contents
        const content = fs.readFileSync(filePath, 'utf8');
        const parent = path.dirname(pathname);
        const parentLink = parent === '/' ? '/' : `${parent}/`;
        
        // Generate HTML for text files
        const html = `
          <!DOCTYPE html>
          <html>
          <head>
            <title>File: ${path.basename(filePath)}</title>
            <style>
              body { font-family: Arial, sans-serif; margin: 20px; }
              h1 { color: #333; }
              .breadcrumb { margin-bottom: 20px; background: #f5f5f5; padding: 10px; }
              .file-info { margin-bottom: 20px; }
              .content { white-space: pre-wrap; border: 1px solid #ddd; padding: 15px; }
              .content.code { font-family: monospace; background-color: #f8f8f8; }
            </style>
          </head>
          <body>
            <h1>File: ${path.basename(filePath)}</h1>
            
            <div class="breadcrumb">
              <a href="${parentLink}">Back to directory</a>
            </div>
            
            <div class="file-info">
              <strong>Size:</strong> ${formatFileSize(stats.size)}<br>
              <strong>Modified:</strong> ${stats.mtime.toLocaleString()}<br>
              <strong>Type:</strong> ${getFileType(filePath)}
            </div>
            
            <h2>Content:</h2>
            <div class="content ${isCodeFile(ext) ? 'code' : ''}">${escapeHtml(content)}</div>
          </body>
          </html>
        `;
        
        res.writeHead(200, { 'Content-Type': 'text/html' });
        res.end(html);
      }
    } catch (err) {
      res.writeHead(404, { 'Content-Type': 'text/html' });
      res.end(`<h1>404 - File Not Found</h1><p>${err.message}</p>`);
    }
  }
});

// Helper functions
function formatFileSize(bytes) {
  if (bytes === 0) return '0 Bytes';
  
  const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];
  const i = Math.floor(Math.log(bytes) / Math.log(1024));
  
  return parseFloat((bytes / Math.pow(1024, i)).toFixed(2)) + ' ' + sizes[i];
}

function getFileType(filename) {
  const ext = path.extname(filename).toLowerCase();
  
  const types = {
    '.html': 'HTML Document',
    '.css': 'CSS Stylesheet',
    '.js': 'JavaScript File',
    '.json': 'JSON File',
    '.txt': 'Text File',
    '.md': 'Markdown File',
    '.jpg': 'JPEG Image',
    '.jpeg': 'JPEG Image',
    '.png': 'PNG Image',
    '.gif': 'GIF Image',
    '.pdf': 'PDF Document',
    '.zip': 'ZIP Archive',
    '.mp3': 'MP3 Audio',
    '.mp4': 'MP4 Video'
  };
  
  return types[ext] || 'File';
}

function getContentType(ext) {
  const types = {
    '.html': 'text/html',
    '.css': 'text/css',
    '.js': 'text/javascript',
    '.json': 'application/json',
    '.txt': 'text/plain',
    '.md': 'text/plain',
    '.jpg': 'image/jpeg',
    '.jpeg': 'image/jpeg',
    '.png': 'image/png',
    '.gif': 'image/gif',
    '.pdf': 'application/pdf',
    '.zip': 'application/zip',
    '.mp3': 'audio/mpeg',
    '.mp4': 'video/mp4'
  };
  
  return types[ext] || 'application/octet-stream';
}

function isBinaryFile(ext) {
  const binaryExts = ['.jpg', '.jpeg', '.png', '.gif', '.pdf', '.zip', '.mp3', '.mp4'];
  return binaryExts.includes(ext);
}

function isCodeFile(ext) {
  const codeExts = ['.js', '.html', '.css', '.json', '.md'];
  return codeExts.includes(ext);
}

function escapeHtml(text) {
  return text
    .replace(/&/g, '&')
    .replace(//g, '>')
    .replace(/"/g, '"')
    .replace(/'/g, ''');
}

const PORT = 3000;
server.listen(PORT, () => {
  console.log(`File Explorer running at http://localhost:${PORT}/`);
});

Instructions:

Create a file named file_explorer.js with the code above
Run the server with node: node file_explorer.js
Open your browser and navigate to http://localhost:3000/
You should see a file explorer interface showing the contents of the current directory
Click on directories to navigate through the file system
Click on text files to view their content in the browser

Challenge: Enhance the file explorer with the following features:

Add file upload functionality
Add the ability to create new directories
Add basic text editing capabilities for text files
Implement file deletion functionality
Add search functionality to find files or directories

Exercise 2: Build a Weather CLI Application

Create a command-line interface (CLI) application that fetches weather data from an API and displays it in the terminal.

// weather_cli.js
const https = require('https');
const readline = require('readline');

// Create readline interface for user input
const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout
});

// ANSI color codes for terminal output
const colors = {
  reset: '\x1b[0m',
  bright: '\x1b[1m',
  red: '\x1b[31m',
  green: '\x1b[32m',
  yellow: '\x1b[33m',
  blue: '\x1b[34m',
  cyan: '\x1b[36m'
};

// Function to make API request
function getWeather(location) {
  return new Promise((resolve, reject) => {
    // Note: You would need to sign up for a free API key from a weather service
    // like OpenWeatherMap or WeatherAPI to use in a real application
    // This is a mock API URL - replace with a real one
    const apiUrl = `https://api.example.com/weather?location=${encodeURIComponent(location)}&units=metric`;
    
    https.get(apiUrl, (res) => {
      let data = '';
      
      // A chunk of data has been received
      res.on('data', (chunk) => {
        data += chunk;
      });
      
      // The whole response has been received
      res.on('end', () => {
        if (res.statusCode !== 200) {
          reject(new Error(`API Error: ${res.statusCode} ${data}`));
          return;
        }
        
        try {
          const weatherData = JSON.parse(data);
          resolve(weatherData);
        } catch (error) {
          reject(new Error('Error parsing API response'));
        }
      });
    }).on('error', (err) => {
      reject(err);
    });
  });
}

// Function to display weather data nicely
function displayWeather(weatherData) {
  // For the example, we're using mock data structure
  // In a real app, adjust this to match the structure of your API response
  
  console.log(`\n${colors.bright}${colors.yellow}Weather for ${weatherData.location.name}, ${weatherData.location.country}${colors.reset}`);
  console.log(`${colors.cyan}${'-'.repeat(50)}${colors.reset}`);
  
  console.log(`${colors.bright}Current Conditions:${colors.reset} ${weatherData.current.condition}`);
  console.log(`${colors.bright}Temperature:${colors.reset} ${weatherData.current.temp_c}°C / ${weatherData.current.temp_f}°F`);
  console.log(`${colors.bright}Feels Like:${colors.reset} ${weatherData.current.feelslike_c}°C / ${weatherData.current.feelslike_f}°F`);
  console.log(`${colors.bright}Humidity:${colors.reset} ${weatherData.current.humidity}%`);
  console.log(`${colors.bright}Wind:${colors.reset} ${weatherData.current.wind_kph} km/h from ${weatherData.current.wind_dir}`);
  console.log(`${colors.bright}Pressure:${colors.reset} ${weatherData.current.pressure_mb} mb`);
  console.log(`${colors.bright}Visibility:${colors.reset} ${weatherData.current.vis_km} km`);
  console.log(`${colors.bright}Last Updated:${colors.reset} ${weatherData.current.last_updated}`);
  
  if (weatherData.forecast && weatherData.forecast.length > 0) {
    console.log(`\n${colors.bright}${colors.green}3-Day Forecast:${colors.reset}`);
    console.log(`${colors.cyan}${'-'.repeat(50)}${colors.reset}`);
    
    weatherData.forecast.forEach(day => {
      console.log(`${colors.bright}${day.date}:${colors.reset} ${day.condition}`);
      console.log(`  High: ${day.maxtemp_c}°C / ${day.maxtemp_f}°F`);
      console.log(`  Low: ${day.mintemp_c}°C / ${day.mintemp_f}°F`);
      console.log(`  Chance of Rain: ${day.daily_chance_of_rain}%`);
      console.log();
    });
  }
}

// Mock weather data for testing when API is not available
function getMockWeatherData(location) {
  return {
    location: {
      name: location,
      country: 'Mock Country',
      lat: 40.7128,
      lon: -74.0060
    },
    current: {
      temp_c: 22.5,
      temp_f: 72.5,
      feelslike_c: 23.1,
      feelslike_f: 73.6,
      condition: 'Partly cloudy',
      humidity: 65,
      wind_kph: 15,
      wind_dir: 'NW',
      pressure_mb: 1015,
      vis_km: 10,
      last_updated: '2025-05-04 15:30'
    },
    forecast: [
      {
        date: '2025-05-05',
        maxtemp_c: 24,
        maxtemp_f: 75.2,
        mintemp_c: 18,
        mintemp_f: 64.4,
        condition: 'Sunny',
        daily_chance_of_rain: 10
      },
      {
        date: '2025-05-06',
        maxtemp_c: 26,
        maxtemp_f: 78.8,
        mintemp_c: 19,
        mintemp_f: 66.2,
        condition: 'Cloudy with sunny periods',
        daily_chance_of_rain: 20
      },
      {
        date: '2025-05-07',
        maxtemp_c: 23,
        maxtemp_f: 73.4,
        mintemp_c: 17,
        mintemp_f: 62.6,
        condition: 'Light rain',
        daily_chance_of_rain: 80
      }
    ]
  };
}

// Main application function
async function runWeatherApp() {
  console.log(`${colors.bright}${colors.blue}===== Weather CLI App =====${colors.reset}`);
  console.log('Get current weather and forecast information for any location.');
  
  let running = true;
  
  while (running) {
    const location = await new Promise(resolve => {
      rl.question(`\n${colors.green}Enter a location (or type 'exit' to quit): ${colors.reset}`, answer => {
        resolve(answer.trim());
      });
    });
    
    if (location.toLowerCase() === 'exit') {
      running = false;
      continue;
    }
    
    if (!location) {
      console.log(`${colors.red}Please enter a valid location.${colors.reset}`);
      continue;
    }
    
    console.log(`\nFetching weather data for ${location}...`);
    
    try {
      // In a real application, you would use getWeather(location)
      // For this example, we're using mock data
      const weatherData = getMockWeatherData(location);
      displayWeather(weatherData);
    } catch (error) {
      console.error(`${colors.red}Error: ${error.message}${colors.reset}`);
    }
  }
  
  console.log(`\n${colors.bright}Thank you for using the Weather CLI App!${colors.reset}`);
  rl.close();
}

// Start the application
runWeatherApp();

Instructions:

Create a file named weather_cli.js with the code above
Run the application with node: node weather_cli.js
Enter a location when prompted to see the weather forecast
Type 'exit' to quit the application

Note: This example uses mock data. To make it work with real weather data, you would need to:

Sign up for a free API key from a weather service provider (e.g., OpenWeatherMap, WeatherAPI)
Replace the mock API URL with the real API endpoint
Update the displayWeather function to match the structure of the actual API response

Challenge: Enhance the weather CLI app with the following features:

Save favorite locations that can be quickly accessed
Add support for different units (metric/imperial)
Add more detailed forecast information
Create ASCII art representations of weather conditions
Add command-line arguments using the process.argv array to allow specifying a location directly (e.g., node weather_cli.js "New York")

Summary

Core Modules are built-in JavaScript libraries included in Node.js that provide essential functionality
The fs module provides file system operations like reading/writing files and working with directories
The http module allows you to create web servers and make HTTP requests
The path module provides utilities for working with file and directory paths cross-platform
Other important modules include:
- events: The foundation of Node.js's event-driven architecture
- url: Tools for parsing and formatting URLs
- os: Access to operating system information
- stream: Interface for working with streaming data
- util: Utility functions for developers
- crypto: Cryptographic functionality
Core modules are accessed using the require() function with just the module name
Most core modules provide both synchronous and asynchronous variants of their methods
Core modules can be combined to create powerful server-side applications

Next Lesson Preview

In our next session, we'll dive into Express.js, a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications. You'll learn how to create routes, middleware, and handle HTTP requests with this powerful framework that has become the de facto standard for building web applications with Node.js.

Introduction to Core Modules

What are Core Modules?

Using Core Modules

The Toolbox Analogy

File System (fs) Module

Key fs Module Operations

Reading Files

Writing Files

Directory Operations

Real-World Example: Log Rotation

HTTP Module

Creating an HTTP Server

Handling Different Routes

Serving Static Files

Making HTTP Requests

HTTP vs HTTPS

Real-World Example: Simple API Proxy

Path Module

Key Path Operations

Why Path Module Matters

Other Essential Core Modules

Events Module

URL Module

OS Module

Stream Module

Util Module

Crypto Module

Practical Exercise

Exercise 1: Build a Simple File Explorer

Exercise 2: Build a Weather CLI Application

Summary

Further Reading

Next Lesson Preview