Node.js
Node.js is a JavaScript runtime built on V8 and libuv. It runs a single-threaded event loop that handles I/O without blocking β instead of waiting for a database query or file read to finish, it registers a callback and moves on. When the operation completes, the callback fires.
This model means one Node process can handle thousands of concurrent connections that traditional thread-per-request servers would need thousands of OS threads for. The flip side: that single thread also runs your business logic. A CPU-bound operation (image processing, heavy computation, ML inference) blocks the entire event loop and stalls every other request. For CPU-heavy work, use Worker Threads or offload to a separate service written in something else.
π’ Junior
npm and package.json
npm init -y # create package.json with defaults
npm install express # install a dependency
npm install -D jest # install a dev dependency (not bundled in prod)
npm install --save-exact lodash # pin the exact version (no ^ semver range)
npm run dev # run the "dev" script
npm start # run the "start" script (shorthand)
npm test # run the "test" script (shorthand)
package.json is the project manifest:
{
"name": "my-api",
"version": "1.0.0",
"main": "src/index.js",
"scripts": {
"start": "node src/index.js",
"dev": "nodemon src/index.js",
"test": "jest --coverage",
"lint": "eslint src/"
},
"dependencies": {
"express": "^4.18.2"
},
"devDependencies": {
"jest": "^29.0.0",
"nodemon": "^3.0.0"
},
"engines": {
"node": ">=20.0.0" // document the required Node version
}
}
package-lock.json β exact dependency tree, committed to version control. Ensures everyone installs the identical versions.
Async Patterns: Callbacks β Promises β async/await
Node APIs evolved through three styles. Youβll encounter all three in the wild.
Callbacks (original Node style):
const fs = require('fs');
// Error-first callback: first arg is always error (or null)
fs.readFile('data.txt', 'utf8', (err, data) => {
if (err) {
console.error('Failed to read:', err.message);
return;
}
console.log(data);
});
// Code continues executing immediately while the file is being read
console.log('This prints before the file content');
Promises (Node 8+):
const fs = require('fs').promises;
fs.readFile('data.txt', 'utf8')
.then(data => console.log(data))
.catch(err => console.error(err.message));
// Chaining
fetchUser(userId)
.then(user => fetchOrders(user.id))
.then(orders => res.json(orders))
.catch(next);
async/await (recommended β Node 10+):
async function loadConfig() {
try {
const raw = await fs.readFile('config.json', 'utf8'); // pause here until done
const data = JSON.parse(raw);
return data;
} catch (err) {
console.error('Config error:', err.message);
throw err; // re-throw so the caller knows
}
}
// Parallel execution β don't await sequentially if tasks are independent
async function loadAll() {
// BAD β sequential, takes 2x as long
const users = await fetchUsers();
const orders = await fetchOrders();
// GOOD β parallel, takes max(fetchUsers, fetchOrders)
const [users, orders] = await Promise.all([fetchUsers(), fetchOrders()]);
}
Non-Blocking I/O
// β BLOCKING β readFileSync freezes the entire process until complete
const data = fs.readFileSync('huge-file.csv');
// While waiting here, ALL other incoming requests are blocked
// β
NON-BLOCKING β process continues; callback fires when ready
fs.readFile('huge-file.csv', 'utf8', (err, data) => {
// Handle data here
});
// Other requests can be served while the file is reading
// β
MODERN β async/await, still non-blocking
const data = await fs.promises.readFile('huge-file.csv', 'utf8');
Rule: Never use any *Sync method inside a request handler or any hot code path.
Essential Built-in Modules
const fs = require('fs/promises'); // file system (async API)
const path = require('path'); // cross-platform path manipulation
const os = require('os'); // system info
const crypto = require('crypto'); // hashing, encryption
const http = require('http'); // raw HTTP server
const events = require('events'); // EventEmitter
const stream = require('stream'); // stream base classes
// Path β always use path.join instead of string concatenation
path.join(__dirname, 'public', 'index.html'); // platform-safe
path.resolve('./relative'); // absolute path
path.extname('file.ts'); // '.ts'
path.basename('/foo/bar/file.ts'); // 'file.ts'
path.dirname('/foo/bar/file.ts'); // '/foo/bar'
// OS
os.cpus().length; // number of CPU cores
os.totalmem(); // total RAM in bytes
os.freemem(); // free RAM in bytes
os.platform(); // 'linux', 'darwin', 'win32'
os.tmpdir(); // system temp directory
// Crypto
const hash = crypto.createHash('sha256').update('password').digest('hex');
const uuid = crypto.randomUUID(); // built-in UUID generation (Node 15.6+)
const rand = crypto.randomBytes(32).toString('hex'); // secure random token
Environment Variables
Never hardcode secrets or environment-specific values.
# .env file (never commit this)
DATABASE_URL=postgres://user:pass@localhost:5432/mydb
JWT_SECRET=super-secret-key-at-least-32-chars
PORT=3000
NODE_ENV=development
npm install dotenv
require('dotenv').config(); // load .env into process.env β call at startup
const port = parseInt(process.env.PORT || '3000', 10);
const dbUrl = process.env.DATABASE_URL;
const jwtSecret = process.env.JWT_SECRET;
if (!jwtSecret) {
throw new Error('JWT_SECRET environment variable is required');
}
π‘ Medior
CommonJS vs ES Modules
| Β | CommonJS (CJS) | ES Modules (ESM) |
|---|---|---|
| Syntax | require() / module.exports |
import / export |
| Loading | Synchronous (blocking) | Asynchronous |
| Tree shaking | No | Yes (bundlers can eliminate unused exports) |
__dirname |
Built-in | Not available β must reconstruct |
| Top-level await | No | Yes |
| Default in Node | Yes | Opt-in via .mjs or "type":"module" |
// ESM β reconstruct __dirname (not available natively)
import { fileURLToPath } from 'url';
import { dirname } from 'path';
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
// Mixing: CJS can't import ESM with require(). ESM can import CJS with import.
// If a dependency is ESM-only, you must use ESM yourself or dynamic import():
const chalk = await import('chalk'); // dynamic import works in CJS too
Event Loop β The Core
Nodeβs event loop is what makes non-blocking I/O possible. It cycles through phases:
βββββββββββββββββββββββββββ
ββ>β timers β setTimeout / setInterval callbacks
β ββββββββββββββ¬βββββββββββββ
β ββββββββββββββ΄βββββββββββββ
β β pending callbacks β I/O errors from previous tick
β ββββββββββββββ¬βββββββββββββ
β ββββββββββββββ΄βββββββββββββ
β β idle, prepare β Node internal use only
β ββββββββββββββ¬βββββββββββββ
β ββββββββββββββ΄βββββββββββββ
β β poll β β Node waits here for I/O events
β ββββββββββββββ¬βββββββββββββ (blocks until callback or timer deadline)
β ββββββββββββββ΄βββββββββββββ
β β check β setImmediate callbacks
β ββββββββββββββ¬βββββββββββββ
β ββββββββββββββ΄βββββββββββββ
ββββ close callbacks β socket.on('close', ...) etc.
βββββββββββββββββββββββββββ
Between every phase: process.nextTick() and Promise microtasks run (always before the next phase).
setTimeout(() => console.log('timeout'), 0);
setImmediate(() => console.log('immediate'));
process.nextTick(() => console.log('nextTick'));
Promise.resolve().then(() => console.log('promise'));
// Output (inside an I/O callback): nextTick β promise β immediate β timeout
// Note: timeout vs immediate order is NON-DETERMINISTIC outside an I/O callback
Practical rule: Donβt overthink the event loop β just remember:
process.nextTickfires before anything else in the current iteration- Promises fire before timers
setImmediatefires beforesetTimeoutwhen both are in an I/O callback
Streams
Streams process data piece-by-piece (chunks) rather than loading everything into memory. Essential for large files, HTTP bodies, and database result sets.
Four types: Readable, Writable, Duplex, Transform
const { pipeline } = require('stream/promises'); // promisified pipeline (Node 15+)
// Piping streams β the most common pattern
// pipeline handles backpressure and error propagation automatically
await pipeline(
fs.createReadStream('large-file.csv'), // Readable
csvParser(), // Transform (parse CSV rows)
new WritableStream({ objectMode: true, // Writable
write(row, enc, cb) {
db.insert(row).then(() => cb()).catch(cb);
}
})
);
// Consuming a readable with async iteration (Node 12+)
async function processLines(filePath) {
const rl = readline.createInterface({ input: fs.createReadStream(filePath) });
for await (const line of rl) {
await processLine(line); // one line at a time β constant memory usage
}
}
// Piping HTTP response directly to a file (zero in-memory buffering)
app.get('/download/:file', async (req, res) => {
const stream = fs.createReadStream(`./files/${req.params.file}`);
stream.pipe(res); // stream file directly to HTTP response
});
// Creating a Transform stream
const { Transform } = require('stream');
const toUpperCase = new Transform({
transform(chunk, encoding, callback) {
callback(null, chunk.toString().toUpperCase()); // push transformed data
}
});
fs.createReadStream('input.txt')
.pipe(toUpperCase)
.pipe(fs.createWriteStream('output.txt'));
Backpressure β the mechanism that prevents a fast readable from overwhelming a slow writable. pipe() handles it automatically. When writing manually:
function writeWithBackpressure(readable, writable) {
readable.on('data', (chunk) => {
const canContinue = writable.write(chunk);
if (!canContinue) {
readable.pause(); // pause the readable
writable.once('drain', () => readable.resume()); // resume when writable is ready
}
});
}
Worker Threads
The event loop is single-threaded β one heavy computation freezes all requests. Worker Threads provide real OS threads with their own V8 heap.
// main.js
const { Worker, isMainThread, workerData, parentPort } = require('worker_threads');
if (isMainThread) {
// Main thread: spawn a worker
function runWorker(data) {
return new Promise((resolve, reject) => {
const worker = new Worker(__filename, { workerData: data });
worker.on('message', resolve);
worker.on('error', reject);
worker.on('exit', code => {
if (code !== 0) reject(new Error(`Worker stopped with exit code ${code}`));
});
});
}
app.get('/compute', async (req, res, next) => {
try {
const result = await runWorker({ input: req.query.n });
res.json({ result });
} catch (err) {
next(err);
}
});
} else {
// Worker thread: perform CPU work
const result = heavyComputation(workerData.input);
parentPort.postMessage(result);
}
SharedArrayBuffer β zero-copy shared memory:
// Share a buffer between main and worker without copying
const sharedBuffer = new SharedArrayBuffer(4);
const arr = new Int32Array(sharedBuffer);
const worker = new Worker('./worker.js', { workerData: { sharedBuffer } });
// Both threads read/write arr[0] directly β use Atomics for safe access
Atomics.add(arr, 0, 1); // atomic increment
Atomics.wait(arr, 0, 0); // wait until arr[0] changes from 0
Atomics.notify(arr, 0, 1); // wake one waiting thread
EventEmitter
The core pub/sub mechanism in Node. Most built-in objects (streams, HTTP server, sockets) extend EventEmitter.
const EventEmitter = require('events');
class OrderService extends EventEmitter {
async create(orderData) {
const order = await db.createOrder(orderData);
this.emit('order:created', order); // fire event
return order;
}
async ship(orderId) {
const order = await db.shipOrder(orderId);
this.emit('order:shipped', order);
return order;
}
}
const orderService = new OrderService();
// Subscribe to events β decoupled side effects
orderService.on('order:created', (order) => emailService.sendConfirmation(order));
orderService.on('order:created', (order) => inventoryService.reserve(order));
orderService.on('order:shipped', (order) => notificationService.push(order.userId, 'Shipped!'));
// once() β one-time listener
orderService.once('order:created', () => console.log('First order ever!'));
Always add an
errorlistener β unhandled'error'events crash the process:emitter.on('error', (err) => logger.error(err));
Error Handling & Graceful Shutdown
// Catch unhandled errors at the process level β last safety net
process.on('uncaughtException', (err) => {
logger.error('Uncaught exception', err);
process.exit(1); // state is now undefined β must exit
});
process.on('unhandledRejection', (reason, promise) => {
logger.error('Unhandled rejection', { reason, promise });
process.exit(1);
});
// Graceful shutdown β finish in-flight requests before dying
const server = app.listen(3000);
async function gracefulShutdown(signal) {
logger.info(`${signal} received, starting graceful shutdown`);
server.close(async () => { // stop accepting new TCP connections
try {
await db.pool.end(); // let ongoing DB queries finish, then close
await redisClient.quit();
logger.info('Graceful shutdown complete');
process.exit(0);
} catch (err) {
logger.error('Error during shutdown', err);
process.exit(1);
}
});
// Force kill if it takes too long (e.g., stuck queries)
setTimeout(() => {
logger.warn('Forcing exit after timeout');
process.exit(1);
}, 30_000).unref(); // .unref() so this timer doesn't keep the process alive by itself
}
process.on('SIGTERM', () => gracefulShutdown('SIGTERM')); // Docker/K8s stop
process.on('SIGINT', () => gracefulShutdown('SIGINT')); // Ctrl+C
π΄ Senior
Cluster Module
Node is single-threaded β one process uses one CPU core. The Cluster module forks the process, one per core. The OS load-balances incoming connections across workers.
import cluster from 'cluster';
import { cpus } from 'os';
import { createServer } from './app.js';
if (cluster.isPrimary) {
const numWorkers = cpus().length;
console.log(`Primary ${process.pid} starting ${numWorkers} workers`);
for (let i = 0; i < numWorkers; i++) {
cluster.fork();
}
// Auto-restart dead workers
cluster.on('exit', (worker, code, signal) => {
console.warn(`Worker ${worker.process.pid} died (${signal || code}), restarting`);
cluster.fork();
});
// IPC between primary and workers
cluster.on('message', (worker, msg) => {
if (msg.type === 'cache:invalidate') {
for (const id in cluster.workers) {
cluster.workers[id].send(msg);
}
}
});
} else {
createServer().listen(3000);
console.log(`Worker ${process.pid} started`);
}
In practice, use PM2 (
pm2 start app.js -i max) β cluster mode with process monitoring, zero-downtime reloads, and log management without any cluster code.
Memory Management
V8 uses generational garbage collection:
ββββββββββββββββββββββββββββββββββββββββ
β Young Generation β
β ββ Nursery (new allocations) β Minor GC: fast, frequent
β ββ Intermediate (survived 1 GC) β
ββββββββββββββββββββββββββββββββββββββββ€
β Old Generation β Major GC: slow, infrequent
β (survived 2+ minor GCs) β
ββββββββββββββββββββββββββββββββββββββββ
Metaspace (off-heap)
ββ Compiled JS code, V8 metadata
Common memory leak sources:
// 1. Global cache that grows without bound
const cache = {};
app.get('/item/:id', (req, res) => {
cache[req.params.id] = bigObject; // never evicted β LEAK
});
// Fix: use a proper LRU cache
const LRU = require('lru-cache');
const cache = new LRU({ max: 1000, ttl: 1000 * 60 * 5 }); // 1000 items, 5min TTL
// 2. Event listeners never removed
class SomeService {
init() {
this.handler = (data) => this.process(data);
globalEventBus.on('event', this.handler); // holds reference to 'this'
}
destroy() {
globalEventBus.off('event', this.handler); // MUST remove
}
}
// 3. Closures capturing large objects
function createHandler(largeBuffer) {
return (req, res) => {
// largeBuffer stays in memory as long as this function exists
res.send(largeBuffer.slice(0, 10));
};
}
// 4. Streams not consumed β Node buffers them
const readable = getReadableStream();
// If you never pipe or consume, data events buffer up β OOM
readable.resume(); // drain without processing if you don't need the data
Diagnosing memory leaks:
# Start with heap profiling
node --inspect app.js
# β Open Chrome DevTools β Memory β Heap Snapshot
# β Take snapshot, do work, take another, compare
# Memory usage stats
const { heapUsed, heapTotal, rss, external } = process.memoryUsage();
# heapUsed: JS objects on the heap
# heapTotal: V8's allocated heap (grows, rarely shrinks)
# rss: Resident Set Size β total process memory including C++ addons
# external: Memory used by C++ objects bound to JS (Buffers)
# Set heap limit explicitly (default ~1.5GB, may be too low for large workloads)
node --max-old-space-size=4096 app.js # 4GB limit
Performance Profiling
# CPU profiling β generates a v8.log
node --prof app.js
# Run load test, then:
node --prof-process isolate-*.log > profile.txt
# Built-in performance hooks
const { performance, PerformanceObserver } = require('perf_hooks');
const obs = new PerformanceObserver((list) => {
list.getEntries().forEach(entry => {
console.log(`${entry.name}: ${entry.duration.toFixed(2)}ms`);
});
});
obs.observe({ type: 'measure' });
performance.mark('db-start');
const results = await db.query('SELECT ...');
performance.mark('db-end');
performance.measure('database-query', 'db-start', 'db-end');
libuv Thread Pool
Nodeβs event loop is single-threaded, but some operations run in libuvβs thread pool:
fs.*(most file operations)dns.lookup()crypto.pbkdf2,crypto.scrypt,crypto.randomBytes- Native addons that use the thread pool
Default pool size: 4 threads. Under high load, pool tasks queue up.
# Increase thread pool size (before Node starts)
UV_THREADPOOL_SIZE=16 node app.js
# dns.resolve (async DNS) vs dns.lookup (uses thread pool)
# In high-traffic APIs, use dns.resolve to avoid thread pool contention
import dns from 'dns/promises';
await dns.resolve4('api.example.com'); // doesn't use thread pool
Senior Gotchas
process.nextTick()starvation β recursively schedulingnextTickfrom within anextTickcallback starves the event loop. UsesetImmediateinstead when you need to defer without hogging the loop.setTimeout(fn, 0)is not instant β minimum delay is ~1ms. For βas soon as possible after I/O,β usesetImmediate.require()is synchronous and cached β everyrequire()call after the first returns the exact same exported object (singleton per process). This means module-level state is shared across all callers.clustervsworker_threadsβcluster= multiple processes with isolated memory (for scaling HTTP throughput);worker_threads= multiple threads with shared memory (for CPU work). They solve different problems.- Memory is NOT automatically freed β even if you delete references, the GC decides when to collect. Use
heapUsedand heap snapshots to verify leaks are fixed. - Async stack traces are truncated β
Error.captureStackTraceand--async-contextcan help, but stack traces acrossawaitboundaries are lossy. Use structured logging with correlation IDs instead. - Donβt create a new DB connection per request β always use a connection pool (
pg.Pool, Mongooseβs built-in pool). Connections are expensive to open. BuffervsstringvsArrayBufferβBufferis Nodeβs efficient binary container (subclass ofUint8Array). Converting between strings and buffers has CPU and memory cost β stay in binary if youβre transforming binary data.