Suggest an edit

Improve this article

Refine the answer for “Why is chunked I/O more efficient?”. Your changes go to moderation before they’re published.

Approval required

Content

What you’re changing

Title (EN)

Short answer (EN)

Shown above the full answer for quick recall.

Answer (EN)

## What chunked I/O is

**Chunked I/O** (or **streaming / block-wise I/O**) is a way to **process data in pieces (chunks)**,
instead of loading a whole file or response into memory at once.

Example:
Say we have a **2 GB** file, a regular read (`fs.readFile`) would try to:

- load all 2 GB into memory at once,
- then process the result.

**Chunked I/O** (via `fs.createReadStream`), instead, reads the file, say, **64 KB** at a time and immediately "pumps" those pieces through the handler.

## Why this is more efficient

### 1. Memory savings

With regular I/O:

- data loads **entirely** into RAM;
- for large files, this can cause an **out-of-memory** crash.

With chunked I/O:

- only a **small piece of data** sits in memory at any moment;
- memory use stays **constant and stable**.

| Method | Memory for a 2 GB file |
|---|---|
| `fs.readFile()` | ~2 GB |
| `fs.createReadStream()` | ~64 KB |

### 2. Earlier processing

Chunked I/O lets you **start processing without waiting for the read to finish**.

Example:
If you're reading a large video file and streaming it to a client over HTTP,
with streams, the client **starts receiving video right away**, rather than after the whole file loads.

This cuts **latency** and speeds up delivery.

### 3. Asynchrony and backpressure

Stream APIs in Node.js run **asynchronously** and support **backpressure**,
if the receiver can't keep up with processing, the source **pauses** reading so it doesn't flood memory.

```javascript
readable.pipe(writable); // automatically regulates transfer speed
```

This keeps the **data flow stable** and prevents memory or queue overflow.

### 4. Composability (pipelines)

Chunked I/O combines easily,
data can flow **sequentially through a chain of handlers** (a pipe):

```javascript
fs.createReadStream('video.mp4')
  .pipe(zlib.createGzip())      // compression
  .pipe(fs.createWriteStream('video.mp4.gz'));
```

It all happens **in a stream**, with no intermediate files or temp buffers.

### 5. Fewer system-level blocks

Large I/O operations often cause blocking at the OS or disk level.
Chunked I/O splits them into **small system calls**,
improving parallelism and reducing load on the disk and the OS scheduler.

### 6. HTTP/1.1 and "Transfer-Encoding: chunked"

In the HTTP protocol, chunked transfer lets a server **send a response as it's generated**,
even without knowing the data's full size ahead of time.

That gives you:

- an instant start to the client's download;
- the ability to stream content on the fly (video, JSON responses, SSE, and so on);
- time and memory savings on the server side.

## Comparing regular and chunked I/O

| Criterion | Regular I/O | Chunked I/O |
|---|---|---|
| Data processing | After a full load | As it arrives |
| Memory use | High | Minimal |
| Response time | Long (wait for the whole file) | Instant (streaming) |
| Scalability | Poor (each request eats memory) | Excellent (thousands of connections) |
| Backpressure support | No | Yes |
| Good for | Small files | Large files, network streams |

## Summary

> **Chunked I/O** is more efficient because it:
>
> - works **in pieces**, not all at once;
> - **saves memory**;
> - allows **processing data on the fly**;
> - **speeds up responses**;
> - supports **flow control (backpressure)**;
> - is a great fit for **streaming, HTTP servers, and file/network work**.

Markdown · drag & drop images · ⌘B / ⌘I shortcuts550 words

For the reviewer

Note to the moderator (optional)

Visible only to the moderator. Helps review go faster.