Streams in Node.js

Data Pipelining: The Power of Streams

Streams represent Node.js's most robust tool for managing data. Without streams, parsing a 2GB file requires allocating 2GB of V8 heap memory—which promptly crashes the application.

Memory Efficiency via Chunks

Streams fracture massive data sets into manageable, sequential buffers. As a chunk is read, it is processed, sent downstream, and subjected to garbage collection long before the file completes reading.

The Four Stream Typologies

• Readable: Abstractions acting as a source (e.g., fs.createReadStream, HTTP IncomingRequest).
• Writable: Abstractions acting as a destination (e.g., fs.createWriteStream).
• Duplex: Both Readable and Writable (e.g., raw TCP Sockets).
• Transform: Duplex streams that mutate data mid-flight (e.g., zlib compression, cryptographic ciphers).

Backpressure

If a Readable stream pumps data faster than a Writable stream can consume it over a slow network connection, the internal buffer overflows. The .pipe() method automatically manages "Backpressure," instructing the Readable stream to pause fetching until the Writable stream drains.