Backpressure in Node.js

Backpressure is a flow-control mechanism that prevents a fast data producer from overwhelming a slower data consumer.

Learn software engineering

It is especially important in Node.js because applications frequently process data using streams—files, HTTP requests, database exports, compression pipelines, video streams, and network connections.

The Basic Problem

Imagine a Node.js application reading a large file and sending it over a network connection:

Large File
    ↓
Readable Stream
    ↓
Node.js Application
    ↓
Writable Stream
    ↓
Network

Suppose the file can be read at:

100 MB/s

but the network can only send:

10 MB/s

Without backpressure, the producer keeps generating data faster than the consumer can process it.

The extra data has to wait somewhere—usually memory.

Producer
100 MB/s
   ↓
████████████████████
        BUFFER
████████████████████
   ↓
Consumer
10 MB/s

Over time, memory consumption can grow dramatically.

Backpressure tells the producer:

Slow down. The consumer hasn’t finished processing the previous data yet.

Backpressure in Node.js Streams

Consider:

const fs = require("fs");

const readable = fs.createReadStream("large-file.mp4");
const writable = fs.createWriteStream("copy.mp4");

readable.on("data", (chunk) => {
  writable.write(chunk);
});

At first glance, this looks fine.

The problem is that writable.write() doesn’t always mean:

"The data has already been written."

Instead, Node.js may place that data into an internal buffer.

The important part is the return value of write().

const canContinue = writable.write(chunk);

It returns either:

true

or:

false

true means the internal buffer still has room.

false means the buffer has reached its configured threshold and the producer should temporarily stop sending data.

Handling Backpressure Manually

A safer implementation is:

const fs = require("fs");

const readable = fs.createReadStream("large-file.mp4");
const writable = fs.createWriteStream("copy.mp4");

readable.on("data", (chunk) => {
  const canContinue = writable.write(chunk);

  if (!canContinue) {
    readable.pause();
  }
});

writable.on("drain", () => {
  readable.resume();
});

readable.on("end", () => {
  writable.end();
});

The important sequence is:

read()
  ↓
write(chunk)
  ↓
write() returns false
  ↓
pause()
  ↓
consumer processes buffered data
  ↓
"drain" event
  ↓
resume()

The "drain" event tells you that the writable stream’s buffer has cleared enough for writing to continue.

highWaterMark

Node.js streams have a buffering threshold controlled by:

highWaterMark

Example:

const fs = require("fs");

const stream = fs.createWriteStream("output.txt", {
  highWaterMark: 64 * 1024
});

Here the threshold is:

64 × 1024
= 65,536 bytes
≈ 64 KB

Conceptually:

Producer
   ↓
┌──────────────────┐
│ Internal Buffer  │
│                  │
│ highWaterMark    │
└──────────────────┘
   ↓
Consumer

When the buffer reaches the threshold, write() begins returning false.

An important detail: highWaterMark is a threshold, not necessarily a hard memory limit. Calling write() repeatedly after it returns false can continue adding buffered data.

The Easier Solution: pipe()

Usually you shouldn’t implement this flow manually.

Node.js streams provide:

readable.pipe(writable);

For example:

const fs = require("fs");

const readable = fs.createReadStream("large-file.mp4");
const writable = fs.createWriteStream("copy.mp4");

readable.pipe(writable);

pipe() automatically coordinates the readable and writable streams, including backpressure.

Conceptually:

Readable
   │
   ▼
┌──────────┐
│  Buffer  │
└──────────┘
   │
   ▼
Writable

Consumer slow?
     ↓
Pause producer
     ↓
Buffer drains
     ↓
Resume producer

Even Better: pipeline()

For production code, pipeline() is often preferable because it also handles stream errors and cleanup more safely.

const fs = require("fs");
const { pipeline } = require("stream");

pipeline(
  fs.createReadStream("input.mp4"),
  fs.createWriteStream("output.mp4"),
  (error) => {
    if (error) {
      console.error("Pipeline failed:", error);
      return;
    }

    console.log("Pipeline completed.");
  }
);

There is also a Promise-based version:

const fs = require("fs");
const { pipeline } = require("stream/promises");

async function copyFile() {
  await pipeline(
    fs.createReadStream("input.mp4"),
    fs.createWriteStream("output.mp4")
  );
}

copyFile().catch(console.error);

This becomes particularly useful when several transformations are involved:

await pipeline(
  source,
  transformA,
  transformB,
  destination
);

Backpressure propagates through the pipeline.

Backpressure with Async Iteration

Readable streams also support async iteration:

for await (const chunk of readable) {
  // process chunk
}

If you’re manually writing those chunks to another stream, you still need to respect the writable side:

const { once } = require("events");

for await (const chunk of readable) {
  if (!writable.write(chunk)) {
    await once(writable, "drain");
  }
}

writable.end();

This gives a useful mental model:

Produce
   ↓
Can consumer accept more?
   ↓
 YES ──────→ Continue
   ↓ NO
   ↓
 Wait
   ↓
Consumer catches up
   ↓
Continue

Backpressure Is Bigger Than Streams

The same concept appears throughout backend systems.

Imagine your API receives:

10,000 requests/sec

but a downstream database can safely process only:

2,000 operations/sec

Something needs to regulate the flow.

Systems commonly use mechanisms such as:

Queues
Concurrency limits
Rate limiting
Batching
Buffers
Worker pools
Load shedding

All address variations of the same fundamental problem:

What happens when work arrives faster than it can be processed?

Why Backpressure Matters

Without effective flow control, a Node.js application can experience:

Growing memory usage
        ↓
More garbage collection
        ↓
Longer latency
        ↓
Event-loop pressure
        ↓
Poor throughput
        ↓
Possible out-of-memory crash

Backpressure changes the strategy from:

"Process everything as quickly as it arrives."

to:

"Accept work at a rate the downstream system can sustain."

Key Takeaway

Backpressure in Node.js is the mechanism that coordinates fast producers with slower consumers.

For writable streams, remember this pattern:

if (!writable.write(chunk)) {
  // stop producing
}

then wait for:

writable.on("drain", () => {
  // continue producing
});

For most stream pipelines, prefer:

pipeline(source, transform, destination);

because Node.js handles the backpressure propagation for you.

Producer → Buffer → Consumer is the core model. When the consumer slows down, a well-designed Node.js system makes the producer slow down too.

About the Author

codeflare

Administrator

Latest tech news and coding tips.

Visit Website View All Posts