Introduction to Offline-First Development
In our previous lectures, we covered Progressive Web App principles and Service Worker implementation. Today, we'll explore how to build truly offline-first applications — web apps that treat offline functionality as a core feature rather than an edge case.
Offline-first development inverts the traditional approach to web applications. Instead of assuming constant connectivity and treating offline as an error state, offline-first applications assume intermittent connectivity and design around it. This paradigm shift creates more resilient applications that work reliably regardless of network conditions.
Analogy: Planning for Rain
Traditional web development is like planning an outdoor event assuming perfect weather. Offline-first development is like planning that same event with contingencies for rain — you hope for sunshine, but you're prepared for anything. You might bring tents, have an indoor backup location, and provide umbrellas.
Similarly, offline-first applications are designed to work in ideal conditions (good connectivity) but have well-thought-out strategies for handling poor network conditions or complete disconnection. The result is an experience that remains valuable to users regardless of their connectivity status.
graph LR
A[Traditional Approach] -->|"Start with"| B[Online Experience]
B -->|"Add as fallback"| C[Offline Error Handling]
D[Offline-First Approach] -->|"Start with"| E[Offline Experience]
E -->|"Enhance with"| F[Online Capabilities]
Designing for Offline User Experience
Key Principles
- Transparency: Clearly communicate network status to users
- Reliability: Ensure core functionality works offline
- Predictability: Make offline behavior consistent and understandable
- Optimism: Assume actions will eventually succeed when back online
- Progressive Enhancement: Add features when connectivity improves
Common UX Patterns
Network Status Indicator
// HTML Structure
<div class="network-status">
<span class="status-indicator"></span>
<span class="status-text">Online</span>
</div>
// CSS Styling
.network-status {
display: flex;
align-items: center;
padding: 5px 10px;
border-radius: 20px;
font-size: 14px;
transition: background-color 0.3s;
}
.status-indicator {
width: 10px;
height: 10px;
border-radius: 50%;
margin-right: 6px;
transition: background-color 0.3s;
}
.online .status-indicator {
background-color: #4CAF50;
}
.offline .status-indicator {
background-color: #F44336;
}
// JavaScript for updating status
function updateNetworkStatus() {
const statusElement = document.querySelector('.network-status');
const statusText = document.querySelector('.status-text');
if (navigator.onLine) {
statusElement.classList.remove('offline');
statusElement.classList.add('online');
statusText.textContent = 'Online';
} else {
statusElement.classList.remove('online');
statusElement.classList.add('offline');
statusText.textContent = 'Offline';
}
}
// Listen for online/offline events
window.addEventListener('online', updateNetworkStatus);
window.addEventListener('offline', updateNetworkStatus);
// Initial status check
document.addEventListener('DOMContentLoaded', updateNetworkStatus);
Optimistic UI
Optimistic UI updates the interface immediately to reflect the expected outcome of an action, rather than waiting for server confirmation. This creates a responsive experience even when offline.
Optimistic UI Example
// Function to handle a "like" action optimistically
function handleLikeButtonClick(postId) {
const likeButton = document.querySelector(`#post-${postId} .like-button`);
const likeCount = document.querySelector(`#post-${postId} .like-count`);
// Get current state
const isLiked = likeButton.classList.contains('liked');
const currentCount = parseInt(likeCount.textContent, 10);
// Optimistically update UI
if (isLiked) {
likeButton.classList.remove('liked');
likeCount.textContent = currentCount - 1;
} else {
likeButton.classList.add('liked');
likeCount.textContent = currentCount + 1;
}
// Create a record of this action for later sync
const actionData = {
type: 'like',
postId: postId,
action: isLiked ? 'unlike' : 'like',
timestamp: new Date().toISOString()
};
// Store the action for later synchronization
saveActionForSync(actionData)
.then(() => {
// Try to send immediately if online
if (navigator.onLine) {
return syncAction(actionData);
}
})
.catch(error => {
console.error('Error handling like action:', error);
// Revert UI change if there was an error saving the action
if (isLiked) {
likeButton.classList.add('liked');
likeCount.textContent = currentCount;
} else {
likeButton.classList.remove('liked');
likeCount.textContent = currentCount;
}
// Notify user
showToast('Failed to save your action. Please try again.');
});
}
// Function to save action for later sync
async function saveActionForSync(action) {
// Open IndexedDB database
const db = await openDatabase();
// Add to pending actions store
return db.add('pendingActions', action);
}
State Management and Conflict Resolution
Offline-first applications need clear strategies for managing state and resolving conflicts that arise from offline operations:
graph TD
A[Conflict Detection] --> B{Resolution Strategy?}
B -->|Last Write Wins| C[Use most recent timestamp]
B -->|Client Priority| D[Client changes override server]
B -->|Server Priority| E[Server state overrides client]
B -->|Merge| F[Combine changes if possible]
B -->|Manual Resolution| G[Ask user to resolve conflict]
Conflict Resolution Example
// Function to resolve conflicts between local and server data
function resolveConflict(localItem, serverItem) {
// Strategy 1: Last write wins (based on timestamp)
const localUpdatedAt = new Date(localItem.updatedAt);
const serverUpdatedAt = new Date(serverItem.updatedAt);
if (localUpdatedAt > serverUpdatedAt) {
return localItem; // Local changes are newer
} else {
return serverItem; // Server changes are newer
}
// Strategy 2: Field-level merging (more complex)
/*
const merged = { ...serverItem };
// Identify fields changed in local version
for (const key in localItem) {
if (localItem[key] !== serverItem[key]) {
// Check if field has been modified since last sync
if (localItem[`${key}_changedAt`] > serverItem.updatedAt) {
merged[key] = localItem[key];
}
}
}
return merged;
*/
// Strategy 3: Ask user to resolve conflict
/*
return {
type: 'conflict',
localVersion: localItem,
serverVersion: serverItem,
needsResolution: true
};
*/
}
// Function to handle sync of data with conflicts
async function syncItemWithConflictResolution(localItem) {
try {
// Fetch latest version from server
const response = await fetch(`/api/items/${localItem.id}`);
if (!response.ok) {
throw new Error(`Failed to fetch server version: ${response.status}`);
}
const serverItem = await response.json();
// Check if server version has changed since last sync
if (serverItem.version !== localItem.lastSyncedVersion) {
// We have a conflict - both local and server versions changed
console.log('Conflict detected, resolving...');
// Apply conflict resolution strategy
const resolvedItem = resolveConflict(localItem, serverItem);
if (resolvedItem.needsResolution) {
// Store conflict for user resolution
await storeConflict(resolvedItem);
return { status: 'conflict-stored', item: resolvedItem };
}
// Save resolved version to server
await saveToServer(resolvedItem);
// Update local version with resolved version
await updateLocalItem(resolvedItem);
return { status: 'conflict-resolved', item: resolvedItem };
} else {
// No conflict, just update server with our changes
await saveToServer(localItem);
return { status: 'updated', item: localItem };
}
} catch (error) {
console.error('Error syncing item:', error);
return { status: 'error', error };
}
}
Client-Side Data Storage
Offline-first applications require robust client-side storage mechanisms. Let's explore the primary options available in modern browsers.
Storage Options Comparison
| Storage Type | Capacity | Persistence | API Type | Best For |
|---|---|---|---|---|
| IndexedDB | ~50-80% of disk space | Until deleted | Async/Promise-based | Structured data, large datasets, complex queries |
| Cache API | ~50-80% of disk space | Until deleted | Async/Promise-based | HTTP responses, resource caching |
| localStorage | ~5-10MB | Until deleted | Synchronous | Simple key-value pairs, settings |
| sessionStorage | ~5-10MB | Browser session | Synchronous | Temporary session data |
| Cookies | ~4KB | Configurable | Synchronous | Authentication, cross-request data |
IndexedDB
IndexedDB is the most powerful client-side storage option, providing a transactional database system in the browser. It's well-suited for offline-first applications due to its:
- Large storage capacity
- Structured data storage
- Support for indexes
- Transactional operations
- Asynchronous API (non-blocking)
Basic IndexedDB Setup
// Helper function to open database connection
function openDatabase() {
return new Promise((resolve, reject) => {
const request = indexedDB.open('MyOfflineApp', 1);
// Handle database upgrade (runs if database doesn't exist or version changes)
request.onupgradeneeded = (event) => {
const db = event.target.result;
// Create object stores (similar to tables in SQL)
if (!db.objectStoreNames.contains('articles')) {
const articlesStore = db.createObjectStore('articles', { keyPath: 'id' });
// Create indexes for faster querying
articlesStore.createIndex('by_category', 'category', { unique: false });
articlesStore.createIndex('by_date', 'publishedAt', { unique: false });
}
if (!db.objectStoreNames.contains('pendingActions')) {
const actionsStore = db.createObjectStore('pendingActions', {
keyPath: 'id',
autoIncrement: true
});
actionsStore.createIndex('by_type', 'type', { unique: false });
}
console.log('Database setup complete');
};
request.onsuccess = (event) => {
const db = event.target.result;
resolve(db);
};
request.onerror = (event) => {
console.error('IndexedDB error:', event.target.error);
reject('Error opening database');
};
});
}
// Wrap IndexedDB operations in a more usable API
class IndexedDBStorage {
constructor(dbName, version) {
this.dbName = dbName;
this.version = version;
this.db = null;
}
// Open the database connection
async open() {
if (this.db) return this.db;
this.db = await new Promise((resolve, reject) => {
const request = indexedDB.open(this.dbName, this.version);
request.onupgradeneeded = (event) => {
this._setupDatabase(event.target.result);
};
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
return this.db;
}
// Set up database schema (override in subclasses)
_setupDatabase(db) {
// Implementation in subclasses
}
// Add an item to an object store
async add(storeName, item) {
const db = await this.open();
return new Promise((resolve, reject) => {
const transaction = db.transaction(storeName, 'readwrite');
const store = transaction.objectStore(storeName);
const request = store.add(item);
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
// Get an item by key
async get(storeName, key) {
const db = await this.open();
return new Promise((resolve, reject) => {
const transaction = db.transaction(storeName, 'readonly');
const store = transaction.objectStore(storeName);
const request = store.get(key);
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
// Put (add or update) an item
async put(storeName, item) {
const db = await this.open();
return new Promise((resolve, reject) => {
const transaction = db.transaction(storeName, 'readwrite');
const store = transaction.objectStore(storeName);
const request = store.put(item);
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
// Delete an item
async delete(storeName, key) {
const db = await this.open();
return new Promise((resolve, reject) => {
const transaction = db.transaction(storeName, 'readwrite');
const store = transaction.objectStore(storeName);
const request = store.delete(key);
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
// Get all items from a store
async getAll(storeName) {
const db = await this.open();
return new Promise((resolve, reject) => {
const transaction = db.transaction(storeName, 'readonly');
const store = transaction.objectStore(storeName);
const request = store.getAll();
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
// Query using an index
async getAllByIndex(storeName, indexName, value) {
const db = await this.open();
return new Promise((resolve, reject) => {
const transaction = db.transaction(storeName, 'readonly');
const store = transaction.objectStore(storeName);
const index = store.index(indexName);
const request = index.getAll(value);
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
// Close the database connection
close() {
if (this.db) {
this.db.close();
this.db = null;
}
}
}
// Example usage
const db = new IndexedDBStorage('MyOfflineApp', 1);
// Save an article
async function saveArticle(article) {
try {
// Add timestamp for sync management
article.updatedAt = new Date().toISOString();
await db.put('articles', article);
console.log('Article saved:', article.id);
} catch (error) {
console.error('Error saving article:', error);
}
}
// Retrieve articles by category
async function getArticlesByCategory(category) {
try {
return await db.getAllByIndex('articles', 'by_category', category);
} catch (error) {
console.error('Error fetching articles:', error);
return [];
}
}
Using the Cache API
The Cache API is designed specifically for storing HTTP responses and is a perfect companion to Service Workers for offline resource management.
Cache API Example
// Utility functions for the Cache API
class CacheManager {
constructor(cacheName) {
this.cacheName = cacheName;
}
// Add a response to the cache
async cacheResponse(request, response) {
const cache = await caches.open(this.cacheName);
return cache.put(request, response);
}
// Get a response from the cache
async getResponse(request) {
const cache = await caches.open(this.cacheName);
return cache.match(request);
}
// Delete a response from the cache
async deleteResponse(request) {
const cache = await caches.open(this.cacheName);
return cache.delete(request);
}
// Add a complete resource (URL + response) to the cache
async cacheResource(url, options = {}) {
try {
const request = new Request(url, options);
const response = await fetch(request);
if (!response.ok) {
throw new Error(`Failed to fetch ${url}: ${response.status}`);
}
// Clone the response before caching it
const responseToCache = response.clone();
await this.cacheResponse(request, responseToCache);
return response;
} catch (error) {
console.error('Failed to cache resource:', error);
throw error;
}
}
// Clear all cached responses
async clearCache() {
return caches.delete(this.cacheName);
}
// Get all cached URLs
async getCachedUrls() {
const cache = await caches.open(this.cacheName);
const requests = await cache.keys();
return requests.map(request => request.url);
}
}
// Example usage
const contentCache = new CacheManager('content-v1');
// Pre-cache key resources
async function preCacheContent() {
try {
await contentCache.cacheResource('/api/articles/featured');
await contentCache.cacheResource('/api/categories');
console.log('Content pre-cached successfully');
} catch (error) {
console.error('Error pre-caching content:', error);
}
}
// Fetch from cache with network fallback
async function getArticleSafely(id) {
const articleUrl = `/api/articles/${id}`;
try {
// Try from cache first
const cachedResponse = await contentCache.getResponse(articleUrl);
if (cachedResponse) {
console.log('Article retrieved from cache');
return cachedResponse.json();
}
// If not in cache, fetch from network and cache
console.log('Article not in cache, fetching from network');
const response = await contentCache.cacheResource(articleUrl);
return response.json();
} catch (error) {
console.error('Failed to get article:', error);
// Handle error gracefully, maybe return a default article
return {
id,
title: 'Article Unavailable',
content: 'This article could not be loaded. Please try again when you\'re online.'
};
}
}
LocalStorage for Simple Data
LocalStorage provides a simple key-value storage mechanism that's useful for small amounts of data and application settings.
LocalStorage Wrapper
// Simple localStorage wrapper with JSON handling
class LocalStore {
constructor(prefix = '') {
this.prefix = prefix ? `${prefix}_` : '';
}
// Generate prefixed key
_key(key) {
return `${this.prefix}${key}`;
}
// Set a value (with automatic JSON conversion)
set(key, value) {
try {
const serialized = JSON.stringify(value);
localStorage.setItem(this._key(key), serialized);
return true;
} catch (error) {
console.error('Error saving to localStorage:', error);
return false;
}
}
// Get a value (with automatic JSON parsing)
get(key, defaultValue = null) {
try {
const serialized = localStorage.getItem(this._key(key));
if (serialized === null) return defaultValue;
return JSON.parse(serialized);
} catch (error) {
console.error('Error reading from localStorage:', error);
return defaultValue;
}
}
// Remove a value
remove(key) {
localStorage.removeItem(this._key(key));
}
// Check if a key exists
has(key) {
return localStorage.getItem(this._key(key)) !== null;
}
// Clear all values with this prefix
clear() {
if (!this.prefix) {
localStorage.clear();
return;
}
// Only clear keys with our prefix
const keys = Object.keys(localStorage);
for (const key of keys) {
if (key.startsWith(this.prefix)) {
localStorage.removeItem(key);
}
}
}
// Get all key-value pairs with this prefix
getAll() {
const result = {};
const keys = Object.keys(localStorage);
for (const key of keys) {
// Only include keys with our prefix
if (key.startsWith(this.prefix)) {
const shortKey = key.slice(this.prefix.length);
result[shortKey] = this.get(shortKey);
}
}
return result;
}
}
// Example usage
const settings = new LocalStore('app_settings');
const userPrefs = new LocalStore('user_prefs');
// Save user theme preference
function saveThemePreference(theme) {
userPrefs.set('theme', theme);
}
// Load user settings
function loadUserSettings() {
return {
theme: userPrefs.get('theme', 'light'),
fontSize: userPrefs.get('fontSize', 'medium'),
notifications: userPrefs.get('notifications', true)
};
}
// Track offline changes
function markPageAsVisitedOffline(pageId) {
const offlineHistory = new LocalStore('offline_history');
const history = offlineHistory.get('pages', []);
history.push({
pageId,
timestamp: Date.now()
});
offlineHistory.set('pages', history);
}
Data Synchronization Strategies
Offline-first applications need to synchronize local data with the server when connectivity is restored. Let's explore different synchronization strategies.
Background Sync API
The Background Sync API allows Service Workers to defer actions until the user has stable connectivity, even if the user has closed the tab or browser.
Background Sync Implementation
// In your application JavaScript
async function saveComment(postId, text) {
const comment = {
postId,
text,
createdAt: new Date().toISOString(),
status: 'pending' // Starts as pending
};
// Save comment to local database
try {
const db = await openDatabase();
const commentId = await db.add('comments', comment);
// Register for background sync
if ('serviceWorker' in navigator && 'SyncManager' in window) {
const registration = await navigator.serviceWorker.ready;
await registration.sync.register('sync-comments');
console.log('Background sync registered for comments');
} else {
// No background sync support, try immediate sync
await synchronizeComments();
}
return commentId;
} catch (error) {
console.error('Error saving comment:', error);
throw error;
}
}
// In your service worker
self.addEventListener('sync', event => {
if (event.tag === 'sync-comments') {
event.waitUntil(synchronizeComments());
}
});
// Shared synchronization function (can be called from either place)
async function synchronizeComments() {
const db = await openDatabase();
// Get all pending comments
const pendingComments = await db.getAllByIndex('comments', 'by_status', 'pending');
if (pendingComments.length === 0) {
console.log('No pending comments to synchronize');
return;
}
console.log(`Syncing ${pendingComments.length} comments`);
// Process each comment
const syncResults = await Promise.allSettled(
pendingComments.map(async (comment) => {
try {
// Send to server
const response = await fetch('/api/comments', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(comment)
});
if (!response.ok) {
throw new Error(`Server responded with ${response.status}`);
}
const serverComment = await response.json();
// Update local comment with server data
comment.status = 'synced';
comment.id = serverComment.id; // Use server-generated ID
await db.put('comments', comment);
return { success: true, comment };
} catch (error) {
console.error(`Failed to sync comment ${comment.id}:`, error);
return { success: false, comment, error };
}
})
);
// Count results
const successful = syncResults.filter(result => result.status === 'fulfilled' && result.value.success).length;
const failed = syncResults.length - successful;
console.log(`Comment sync complete: ${successful} succeeded, ${failed} failed`);
// If any failed, we can notify the user or retry later
if (failed > 0) {
// Maybe schedule another sync attempt
// Or notify the user that some changes couldn't be saved
}
return { successful, failed };
}
Periodic Sync (Advanced)
Periodic Background Sync allows your app to synchronize data regularly in the background, even when the user isn't actively using your application.
Periodic Sync Example
// In your application JavaScript
async function registerPeriodicNewsSync() {
// Check for support
if ('serviceWorker' in navigator && 'periodicSync' in ServiceWorkerRegistration.prototype) {
try {
// Get permission (this will prompt the user)
const status = await navigator.permissions.query({
name: 'periodic-background-sync'
});
if (status.state === 'granted') {
// Register for periodic sync
const registration = await navigator.serviceWorker.ready;
await registration.periodicSync.register('sync-news', {
minInterval: 24 * 60 * 60 * 1000 // Once per day (in ms)
});
console.log('Periodic sync registered for news updates');
return true;
}
console.log('Periodic sync permission not granted');
return false;
} catch (error) {
console.error('Error registering periodic sync:', error);
return false;
}
} else {
console.log('Periodic background sync not supported');
return false;
}
}
// In your service worker
self.addEventListener('periodicsync', event => {
if (event.tag === 'sync-news') {
event.waitUntil(downloadLatestNews());
}
});
// Function to download and cache latest news
async function downloadLatestNews() {
try {
console.log('Downloading latest news articles');
// Fetch the latest articles
const response = await fetch('/api/articles/latest');
if (!response.ok) {
throw new Error(`Failed to fetch latest news: ${response.status}`);
}
const articles = await response.json();
// Store in IndexedDB
const db = await openDatabase();
// Start a transaction for all operations
const tx = db.transaction('articles', 'readwrite');
const store = tx.objectStore('articles');
// Process each article
for (const article of articles) {
// Add a timestamp
article.downloadedAt = new Date().toISOString();
// Store in IndexedDB
await store.put(article);
// Also cache any images
if (article.imageUrl) {
const imageCache = await caches.open('news-images');
await imageCache.add(article.imageUrl);
}
}
// Complete the transaction
await tx.complete;
// Update metadata about the sync
const meta = new LocalStore('app_meta');
meta.set('lastNewsSync', {
timestamp: Date.now(),
articleCount: articles.length
});
console.log(`Downloaded ${articles.length} news articles`);
// Optionally show a notification to the user
if (articles.length > 0 && 'Notification' in window) {
const permission = await Notification.requestPermission();
if (permission === 'granted') {
self.registration.showNotification('News Update', {
body: `${articles.length} new articles available for offline reading`,
icon: '/icons/news-icon.png'
});
}
}
return true;
} catch (error) {
console.error('Error downloading news:', error);
return false;
}
}
Custom Sync Queue
When Background Sync API isn't available or you need more control, you can implement a custom sync queue:
Custom Sync Queue Implementation
// SyncQueue class for managing pending operations
class SyncQueue {
constructor(dbName = 'SyncQueueDB', storeName = 'pendingOperations') {
this.dbName = dbName;
this.storeName = storeName;
this.db = null;
}
// Initialize the database
async init() {
if (this.db) return this.db;
this.db = await new Promise((resolve, reject) => {
const request = indexedDB.open(this.dbName, 1);
request.onupgradeneeded = (event) => {
const db = event.target.result;
// Create the pending operations store
if (!db.objectStoreNames.contains(this.storeName)) {
const store = db.createObjectStore(this.storeName, {
keyPath: 'id',
autoIncrement: true
});
// Add indexes for better querying
store.createIndex('by_url', 'url', { unique: false });
store.createIndex('by_method', 'method', { unique: false });
store.createIndex('by_status', 'status', { unique: false });
store.createIndex('by_timestamp', 'timestamp', { unique: false });
}
};
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
return this.db;
}
// Add an operation to the queue
async enqueue(operation) {
await this.init();
// Ensure required fields
if (!operation.url) throw new Error('Operation must have a URL');
if (!operation.method) throw new Error('Operation must have a method');
// Set defaults
const timestamp = Date.now();
const fullOperation = {
...operation,
status: 'pending',
timestamp,
retryCount: 0
};
// Store in IndexedDB
return new Promise((resolve, reject) => {
const tx = this.db.transaction(this.storeName, 'readwrite');
const store = tx.objectStore(this.storeName);
const request = store.add(fullOperation);
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
// Get all pending operations
async getPendingOperations() {
await this.init();
return new Promise((resolve, reject) => {
const tx = this.db.transaction(this.storeName, 'readonly');
const store = tx.objectStore(this.storeName);
const index = store.index('by_status');
const request = index.getAll('pending');
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
// Update operation status
async updateOperationStatus(id, status, result = null) {
await this.init();
return new Promise((resolve, reject) => {
const tx = this.db.transaction(this.storeName, 'readwrite');
const store = tx.objectStore(this.storeName);
// First get the current operation
const getRequest = store.get(id);
getRequest.onsuccess = (event) => {
const operation = event.target.result;
if (!operation) {
reject(new Error(`Operation ${id} not found`));
return;
}
// Update the operation
operation.status = status;
if (result) operation.result = result;
if (status === 'failed') operation.retryCount += 1;
operation.lastUpdated = Date.now();
// Save the updated operation
const updateRequest = store.put(operation);
updateRequest.onsuccess = () => resolve(operation);
updateRequest.onerror = (event) => reject(event.target.error);
};
getRequest.onerror = (event) => reject(event.target.error);
});
}
// Process the queue
async processQueue() {
// Get all pending operations
const pendingOps = await this.getPendingOperations();
if (pendingOps.length === 0) {
console.log('No pending operations to process');
return { processed: 0, succeeded: 0, failed: 0 };
}
console.log(`Processing ${pendingOps.length} pending operations`);
let succeeded = 0;
let failed = 0;
// Process each operation
await Promise.all(pendingOps.map(async (operation) => {
try {
// Prepare the fetch options
const options = {
method: operation.method,
headers: operation.headers || {},
mode: operation.mode || 'cors',
credentials: operation.credentials || 'same-origin'
};
// Add body for non-GET requests
if (operation.method !== 'GET' && operation.body) {
options.body = operation.body;
}
// Execute the fetch
const response = await fetch(operation.url, options);
// Check if successful
if (!response.ok) {
throw new Error(`Server responded with ${response.status}`);
}
// Parse the response
const result = await response.json();
// Mark as successful
await this.updateOperationStatus(operation.id, 'completed', result);
succeeded++;
} catch (error) {
console.error(`Failed to process operation ${operation.id}:`, error);
// Determine if we should retry
const shouldRetry = operation.retryCount < 3; // Max 3 retries
// Mark as failed or permanent failure
const newStatus = shouldRetry ? 'failed' : 'permanent_failure';
await this.updateOperationStatus(operation.id, newStatus, {
error: error.message
});
failed++;
}
}));
console.log(`Queue processing complete: ${succeeded} succeeded, ${failed} failed`);
return {
processed: pendingOps.length,
succeeded,
failed
};
}
// Helper for automatic periodic processing
startAutoProcessing(interval = 60000) {
// Check for connections periodically
this.processingInterval = setInterval(() => {
if (navigator.onLine) {
this.processQueue()
.then(result => {
if (result.processed > 0) {
console.log(`Auto-processed ${result.processed} operations`);
}
})
.catch(error => {
console.error('Error in auto-processing:', error);
});
}
}, interval);
// Also process when coming online
window.addEventListener('online', () => {
console.log('Device is online, processing sync queue');
this.processQueue().catch(console.error);
});
console.log(`Auto-processing started with ${interval}ms interval`);
return this;
}
// Stop automatic processing
stopAutoProcessing() {
if (this.processingInterval) {
clearInterval(this.processingInterval);
this.processingInterval = null;
console.log('Auto-processing stopped');
}
return this;
}
}
// Example usage
const syncQueue = new SyncQueue();
// Initialize and start auto-processing
syncQueue.init()
.then(() => syncQueue.startAutoProcessing())
.catch(console.error);
// Wrap fetch operations to handle offline scenarios
async function fetchWithOfflineSupport(url, options = {}) {
try {
// Try the normal fetch first
const response = await fetch(url, options);
return response;
} catch (error) {
// If fetch fails (likely offline), enqueue for later
if (!navigator.onLine) {
console.log(`Device is offline, enqueueing request to ${url}`);
// Prepare body for storage
let body = options.body;
// If body is FormData, convert to object
if (body instanceof FormData) {
const formObject = {};
for (const [key, value] of body.entries()) {
formObject[key] = value;
}
body = JSON.stringify(formObject);
} else if (typeof body === 'object') {
body = JSON.stringify(body);
}
// Enqueue the operation
await syncQueue.enqueue({
url,
method: options.method || 'GET',
headers: options.headers,
body,
mode: options.mode,
credentials: options.credentials
});
// Return a fake response
return new Response(JSON.stringify({
success: false,
offline: true,
message: 'Your request has been saved and will be processed when you\'re back online.'
}), {
status: 200,
headers: { 'Content-Type': 'application/json' }
});
}
// If not an offline issue, rethrow the error
throw error;
}
}
Bi-directional Sync with Version Vectors
For more complex applications where data can be modified both online and offline, you may need sophisticated conflict resolution:
Version Vector Synchronization
// Simplified version vector implementation for conflict detection
class VersionVector {
constructor(nodes = {}) {
this.nodes = { ...nodes };
}
// Increment the counter for a node
increment(nodeId) {
this.nodes[nodeId] = (this.nodes[nodeId] || 0) + 1;
return this;
}
// Compare with another version vector
compare(other) {
if (!(other instanceof VersionVector)) {
throw new Error('Can only compare with another VersionVector');
}
let aGreater = false;
let bGreater = false;
// Check all keys in this vector
for (const nodeId in this.nodes) {
const aValue = this.nodes[nodeId] || 0;
const bValue = other.nodes[nodeId] || 0;
if (aValue > bValue) aGreater = true;
if (aValue < bValue) bGreater = true;
}
// Check keys that are in other but not in this
for (const nodeId in other.nodes) {
if (this.nodes[nodeId] === undefined && other.nodes[nodeId] > 0) {
bGreater = true;
}
}
// Determine relationship
if (aGreater && bGreater) return 'conflict';
if (aGreater) return 'after';
if (bGreater) return 'before';
return 'equal';
}
// Merge with another version vector
merge(other) {
if (!(other instanceof VersionVector)) {
throw new Error('Can only merge with another VersionVector');
}
const result = new VersionVector(this.nodes);
for (const nodeId in other.nodes) {
result.nodes[nodeId] = Math.max(
result.nodes[nodeId] || 0,
other.nodes[nodeId]
);
}
return result;
}
// Serialize to JSON
toJSON() {
return this.nodes;
}
// Create from JSON
static fromJSON(json) {
return new VersionVector(json);
}
}
// Document class with version tracking
class SyncedDocument {
constructor(id, data = {}, vector = new VersionVector()) {
this.id = id;
this.data = { ...data };
this.vector = vector;
}
// Update document data
update(nodeId, changes) {
// Apply changes
this.data = { ...this.data, ...changes };
// Update version vector
this.vector.increment(nodeId);
// Return updated document
return this;
}
// Handle sync with remote version
sync(remoteDoc) {
// Compare version vectors
const comparison = this.vector.compare(remoteDoc.vector);
if (comparison === 'equal') {
// Documents are identical, nothing to do
return { action: 'none', document: this };
}
if (comparison === 'before') {
// Remote version is newer, take remote version
return { action: 'update_local', document: remoteDoc };
}
if (comparison === 'after') {
// Local version is newer, update remote
return { action: 'update_remote', document: this };
}
// We have a conflict, need to merge
// For this example, we'll use a simple last-write-wins strategy for each field
// In practice, you might use a more sophisticated merge strategy
const mergedData = { ...this.data };
for (const key in remoteDoc.data) {
// Simple strategy: If remote field is newer, use it
if (remoteDoc.data[key]._lastModified > (this.data[key]?._lastModified || 0)) {
mergedData[key] = remoteDoc.data[key];
}
}
// Create merged document with merged version vector
const mergedDoc = new SyncedDocument(
this.id,
mergedData,
this.vector.merge(remoteDoc.vector)
);
return { action: 'merge', document: mergedDoc };
}
// Serialize to JSON
toJSON() {
return {
id: this.id,
data: this.data,
vector: this.vector.toJSON()
};
}
// Create from JSON
static fromJSON(json) {
return new SyncedDocument(
json.id,
json.data,
VersionVector.fromJSON(json.vector)
);
}
}
// Example usage
async function syncDocument(docId, nodeId) {
try {
// Get local version
const db = await openDatabase();
const localDocData = await db.get('documents', docId);
if (!localDocData) {
throw new Error(`Document ${docId} not found locally`);
}
const localDoc = SyncedDocument.fromJSON(localDocData);
// Get remote version
const response = await fetch(`/api/documents/${docId}`);
if (!response.ok) {
throw new Error(`Failed to fetch remote document: ${response.status}`);
}
const remoteDocData = await response.json();
const remoteDoc = SyncedDocument.fromJSON(remoteDocData);
// Perform sync
const result = localDoc.sync(remoteDoc);
// Handle the result
switch (result.action) {
case 'none':
console.log('Documents already in sync');
break;
case 'update_local':
console.log('Updating local document with remote changes');
await db.put('documents', result.document.toJSON());
break;
case 'update_remote':
console.log('Updating remote document with local changes');
await fetch(`/api/documents/${docId}`, {
method: 'PUT',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(result.document.toJSON())
});
break;
case 'merge':
console.log('Merging conflicting changes');
// Update both local and remote
await db.put('documents', result.document.toJSON());
await fetch(`/api/documents/${docId}`, {
method: 'PUT',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(result.document.toJSON())
});
break;
}
return result.document;
} catch (error) {
console.error(`Error syncing document ${docId}:`, error);
throw error;
}
}
flowchart TB
A[Local Document] --> C{Version Comparison}
B[Remote Document] --> C
C -->|equal| D[No Action]
C -->|local before remote| E[Update Local]
C -->|local after remote| F[Update Remote]
C -->|conflict| G[Merge Changes]
E --> H[Update Local Storage]
F --> I[Send Update to Server]
G --> H
G --> I
Building an Offline-First Architecture
Let's explore how to architect an offline-first application that combines all the concepts we've covered.
Layered Data Access Pattern
A key pattern for offline-first applications is to use a layered data access approach that abstracts where data comes from:
graph TD
A[UI Components] --> B[Data Access Layer]
B --> C[Cache Strategy]
C --> D[Local Storage]
C --> E[Network API]
E -.->|If Online| F[Server]
D -->|Cached Data| C
Layered Data Access Implementation
// Data access layer that abstracts data sources
class DataService {
constructor() {
// Initialize storage
this.db = null;
this.initDatabase();
// Track online status
this.isOnline = navigator.onLine;
window.addEventListener('online', () => {
this.isOnline = true;
this.syncData();
});
window.addEventListener('offline', () => {
this.isOnline = false;
});
}
// Initialize the database
async initDatabase() {
try {
this.db = await new Promise((resolve, reject) => {
const request = indexedDB.open('OfflineFirstDB', 1);
request.onupgradeneeded = (event) => {
const db = event.target.result;
// Create object stores for different data types
if (!db.objectStoreNames.contains('articles')) {
const store = db.createObjectStore('articles', { keyPath: 'id' });
store.createIndex('by_category', 'category', { unique: false });
store.createIndex('by_date', 'publishedAt', { unique: false });
store.createIndex('by_status', 'syncStatus', { unique: false });
}
if (!db.objectStoreNames.contains('comments')) {
const store = db.createObjectStore('comments', {
keyPath: 'id',
autoIncrement: true
});
store.createIndex('by_article', 'articleId', { unique: false });
store.createIndex('by_status', 'syncStatus', { unique: false });
}
if (!db.objectStoreNames.contains('userActions')) {
const store = db.createObjectStore('userActions', {
keyPath: 'id',
autoIncrement: true
});
store.createIndex('by_type', 'type', { unique: false });
store.createIndex('by_status', 'syncStatus', { unique: false });
}
};
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
console.log('Database initialized');
// Initial sync
if (this.isOnline) {
this.syncData();
}
} catch (error) {
console.error('Failed to initialize database:', error);
}
}
// Generic database operations
async add(storeName, item) {
await this.waitForDb();
return new Promise((resolve, reject) => {
const tx = this.db.transaction(storeName, 'readwrite');
const store = tx.objectStore(storeName);
// Add sync status for tracking
item.syncStatus = 'pending';
item.updatedAt = new Date().toISOString();
const request = store.add(item);
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
async get(storeName, id) {
await this.waitForDb();
return new Promise((resolve, reject) => {
const tx = this.db.transaction(storeName, 'readonly');
const store = tx.objectStore(storeName);
const request = store.get(id);
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
async put(storeName, item) {
await this.waitForDb();
return new Promise((resolve, reject) => {
const tx = this.db.transaction(storeName, 'readwrite');
const store = tx.objectStore(storeName);
// Update sync status and timestamp
item.syncStatus = 'pending';
item.updatedAt = new Date().toISOString();
const request = store.put(item);
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
async getAll(storeName, indexName = null, indexValue = null) {
await this.waitForDb();
return new Promise((resolve, reject) => {
const tx = this.db.transaction(storeName, 'readonly');
const store = tx.objectStore(storeName);
let request;
if (indexName && indexValue !== null) {
const index = store.index(indexName);
request = index.getAll(indexValue);
} else {
request = store.getAll();
}
request.onsuccess = (event) => resolve(event.target.result);
request.onerror = (event) => reject(event.target.error);
});
}
// Wait for database to be initialized
async waitForDb() {
if (this.db) return;
let attempts = 0;
while (!this.db && attempts < 50) {
await new Promise(resolve => setTimeout(resolve, 100));
attempts++;
}
if (!this.db) {
throw new Error('Database initialization timed out');
}
}
// Sync data with the server
async syncData() {
if (!this.isOnline) return false;
try {
console.log('Starting data synchronization');
// Sync pending articles
await this.syncPendingItems('articles', '/api/articles');
// Sync pending comments
await this.syncPendingItems('comments', '/api/comments');
// Sync user actions
await this.syncUserActions();
// Fetch new data from server
await this.fetchLatestData();
console.log('Data synchronization completed');
return true;
} catch (error) {
console.error('Data synchronization failed:', error);
return false;
}
}
// Sync pending items to the server
async syncPendingItems(storeName, apiEndpoint) {
try {
// Get all pending items
const pendingItems = await this.getAll(storeName, 'by_status', 'pending');
if (pendingItems.length === 0) {
console.log(`No pending ${storeName} to sync`);
return { success: true, count: 0 };
}
console.log(`Syncing ${pendingItems.length} ${storeName}`);
// Process each item
const results = await Promise.allSettled(
pendingItems.map(async (item) => {
try {
// Determine if it's a new item or an update
const isNew = !item.serverId;
const method = isNew ? 'POST' : 'PUT';
const url = isNew ? apiEndpoint : `${apiEndpoint}/${item.serverId}`;
// Send to server
const response = await fetch(url, {
method,
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(item)
});
if (!response.ok) {
throw new Error(`Server responded with ${response.status}`);
}
const serverItem = await response.json();
// Update local item with server data
item.syncStatus = 'synced';
// If it's a new item, store the server ID
if (isNew && serverItem.id) {
item.serverId = serverItem.id;
}
// Update other fields from server
item.lastSynced = new Date().toISOString();
// Save updated item
await this.put(storeName, item);
return { success: true, item };
} catch (error) {
console.error(`Failed to sync ${storeName} item:`, error);
return { success: false, item, error };
}
})
);
// Count results
const succeeded = results.filter(result =>
result.status === 'fulfilled' && result.value.success
).length;
console.log(`${storeName} sync: ${succeeded}/${pendingItems.length} succeeded`);
return { success: true, count: succeeded };
} catch (error) {
console.error(`Error syncing ${storeName}:`, error);
return { success: false, error };
}
}
// Sync user actions (likes, follows, etc.)
async syncUserActions() {
try {
// Get all pending actions
const pendingActions = await this.getAll('userActions', 'by_status', 'pending');
if (pendingActions.length === 0) {
console.log('No pending user actions to sync');
return { success: true, count: 0 };
}
console.log(`Syncing ${pendingActions.length} user actions`);
// Process each action
const results = await Promise.allSettled(
pendingActions.map(async (action) => {
try {
// Send to appropriate endpoint based on action type
let endpoint;
switch (action.type) {
case 'like':
endpoint = `/api/likes/${action.isAdd ? 'add' : 'remove'}`;
break;
case 'bookmark':
endpoint = `/api/bookmarks/${action.isAdd ? 'add' : 'remove'}`;
break;
case 'follow':
endpoint = `/api/follows/${action.isAdd ? 'add' : 'remove'}`;
break;
default:
endpoint = '/api/user-actions';
}
// Send to server
const response = await fetch(endpoint, {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(action.data)
});