Streams and I/O operations

Node.jsJunior~2 min read

What a stream is in Node.js

A stream is a sequential source or destination for data, letting you process data in chunks, without waiting for the entire content to load.

In other words:

Streams are a way to read or write data gradually, rather than all at once.

Examples of streams in Node.js:

Stream type	Example use	Module
Readable	Reading a file, receiving data from an HTTP request	`fs.createReadStream()`, `http.IncomingMessage`
Writable	Writing to a file, sending an HTTP response	`fs.createWriteStream()`, `http.ServerResponse`
Duplex	A TCP socket (can be both read and written)	`net.Socket`
Transform	Compression, encryption, processing data on the fly	`zlib.createGzip()`, `crypto.createCipher()`

Example: reading a file with a stream

javascript

const fs = require('fs');

const readStream = fs.createReadStream('bigfile.txt', 'utf8');

readStream.on('data', chunk => {
  console.log('Got a chunk of data:', chunk.length);
});

readStream.on('end', () => {
  console.log('The file was read in full.');
});

Here, Node.js doesn't load the entire file into memory, it reads it in chunks and processes them as they arrive.

Why streams are the primary way to handle I/O

If you read a 10 GB file via fs.readFile(), Node.js will try to load the whole file into RAM, and crash with an OOM (out of memory) error. Streams instead read piece by piece, keeping memory use constant.

2. High performance

Data can be processed as soon as it arrives, without waiting for the entire file, request, or response to be ready.

For example:

a video feed from a camera can be transcoded on the fly;
an HTTP request can start being served to the client right away, with no buffering of the entire response.

3. Asynchrony and backpressure

Streams are built on the EventEmitter architecture, run asynchronously, and support backpressure, a mechanism that stops a receiver from being flooded with data faster than it can process it.

For example:

javascript

readable.pipe(writable);

If the writable side temporarily can't keep up, Node.js automatically pauses reading until the writable side frees up.

4. Composability

Streams can be chained (piped) together like a pipeline:

javascript

const fs = require('fs');
const zlib = require('zlib');

fs.createReadStream('input.txt')
  .pipe(zlib.createGzip())
  .pipe(fs.createWriteStream('input.txt.gz'));

Here, the data:

is read from the file,
gets compressed,
and is written back out, all with no intermediate temp files and no loading into memory.

Inside Node.js

Streams are built on the stream module and underpin most of the built-in APIs:

http (requests and responses),
fs (file operations),
net (TCP),
zlib, crypto (compression, encryption),
process.stdin, process.stdout.

Summary

Streams in Node.js are an interface for working with data in pieces, which:

lets you read, write, and process data gradually;

saves memory and speeds up I/O;

supports asynchronous work and backpressure;

is a universal mechanism for all I/O in Node.js (files, network, HTTP, processes, encryption, and more).

Short Answer

Interview ready

Premium

A concise answer to help you respond confidently on this topic during an interview.