intermediate 17 min read

Streams & Buffers

Efficient data handling with streams and buffers for large datasets.

Streams

Streams are objects that let you read data from a source or write data to a destination in continuous fashion.

const fs = require('fs');\nconst zlib = require('zlib');\n\n// Readable stream\nconst readStream = fs.createReadStream('./large-file.txt', 'utf8');\n\n// Writable stream\nconst writeStream = fs.createWriteStream('./output.txt');\n\n// Pipe: read -> write\nreadStream.pipe(writeStream);\n\n// Transform stream (compress)\nconst gzip = zlib.createGzip();\nconst source = fs.createReadStream('./data.txt');\nconst dest = fs.createWriteStream('./data.txt.gz');\nsource.pipe(gzip).pipe(dest);

Stream Events

readStream.on('data', (chunk) => {\n  console.log(`Received ${chunk.length} bytes`);\n});\n\nreadStream.on('end', () => {\n  console.log('Finished reading');\n});\n\nreadStream.on('error', (err) => {\n  console.error('Stream error:', err);\n});

Buffers

const buf = Buffer.alloc(10);         // Create a 10-byte buffer\nconst buf2 = Buffer.from('Hello');   // Buffer from string\nconst buf3 = Buffer.concat([buf, buf2]); // Concatenate\n\nconsole.log(buf2.toString());        // 'Hello'\nconsole.log(buf2.length);            // 5\nconsole.log(buf2[0]);                // 72 (ASCII 'H')

Examples

const { Readable, Transform, Writable } = require('stream');

// Custom readable stream
const source = new Readable({
  read() {
    this.push('Line 1\n');
    this.push('Line 2\n');
    this.push(null); // Signal end
  }
});

// Transform stream (uppercase)
const upper = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

source.pipe(upper).pipe(process.stdout);

Table of Contents

Related Lessons

Streams & Buffers

Streams

Stream Events

Buffers

Examples

Your Notes

Discussion

Flashcards