From 39fa9fa85c6f240ec982dd2972b5ad8aa20359af Mon Sep 17 00:00:00 2001 From: Tristian Flanagan Date: Thu, 5 Nov 2015 14:54:10 -0500 Subject: [PATCH] doc: sort stream alphabetically Reorders, with no contextual changes, the stream documentation alphabetically. PR-URL: https://github.com/nodejs/node/pull/3662 Reviewed-By: Evan Lucas Reviewed-By: James M Snell Reviewed-By: Jeremiah Senkpiel --- doc/api/stream.markdown | 1116 +++++++++++++++++++-------------------- 1 file changed, 554 insertions(+), 562 deletions(-) diff --git a/doc/api/stream.markdown b/doc/api/stream.markdown index 2e9ea45151aecd..90289b7cf7400c 100644 --- a/doc/api/stream.markdown +++ b/doc/api/stream.markdown @@ -93,6 +93,17 @@ server.listen(1337); // error: Unexpected token o ``` +### Class: stream.Duplex + +Duplex streams are streams that implement both the [Readable][] and +[Writable][] interfaces. See above for usage. + +Examples of Duplex streams include: + +* [tcp sockets][] +* [zlib streams][] +* [crypto streams][] + ### Class: stream.Readable @@ -145,52 +156,13 @@ Examples of readable streams include: * [child process stdout and stderr][] * [process.stdin][] -#### Event: 'readable' - -When a chunk of data can be read from the stream, it will emit a -`'readable'` event. - -In some cases, listening for a `'readable'` event will cause some data -to be read into the internal buffer from the underlying system, if it -hadn't already. - -```javascript -var readable = getReadableStreamSomehow(); -readable.on('readable', function() { - // there is some data to read now -}); -``` - -Once the internal buffer is drained, a `readable` event will fire -again when more data is available. - -The `readable` event is not emitted in the "flowing" mode with the -sole exception of the last one, on end-of-stream. - -The 'readable' event indicates that the stream has new information: -either new data is available or the end of the stream has been reached. -In the former case, `.read()` will return that data. In the latter case, -`.read()` will return null. For instance, in the following example, `foo.txt` -is an empty file: - -```javascript -var fs = require('fs'); -var rr = fs.createReadStream('foo.txt'); -rr.on('readable', function() { - console.log('readable:', rr.read()); -}); -rr.on('end', function() { - console.log('end'); -}); -``` +#### Event: 'close' -The output of running this script is: +Emitted when the stream and any of its underlying resources (a file +descriptor, for example) have been closed. The event indicates that +no more events will be emitted, and no further computation will occur. -``` -bash-3.2$ node test.js -readable: null -end -``` +Not all streams will emit the 'close' event. #### Event: 'data' @@ -228,101 +200,75 @@ readable.on('end', function() { }); ``` -#### Event: 'close' - -Emitted when the stream and any of its underlying resources (a file -descriptor, for example) have been closed. The event indicates that -no more events will be emitted, and no further computation will occur. - -Not all streams will emit the 'close' event. - #### Event: 'error' * {Error Object} Emitted if there was an error receiving data. -#### readable.read([size]) - -* `size` {Number} Optional argument to specify how much data to read. -* Return {String | Buffer | null} - -The `read()` method pulls some data out of the internal buffer and -returns it. If there is no data available, then it will return -`null`. - -If you pass in a `size` argument, then it will return that many -bytes. If `size` bytes are not available, then it will return `null`, -unless we've ended, in which case it will return the data remaining -in the buffer. +#### Event: 'readable' -If you do not specify a `size` argument, then it will return all the -data in the internal buffer. +When a chunk of data can be read from the stream, it will emit a +`'readable'` event. -This method should only be called in paused mode. In flowing mode, -this method is called automatically until the internal buffer is -drained. +In some cases, listening for a `'readable'` event will cause some data +to be read into the internal buffer from the underlying system, if it +hadn't already. ```javascript var readable = getReadableStreamSomehow(); readable.on('readable', function() { - var chunk; - while (null !== (chunk = readable.read())) { - console.log('got %d bytes of data', chunk.length); - } + // there is some data to read now }); ``` -If this method returns a data chunk, then it will also trigger the -emission of a [`'data'` event][]. - -Note that calling `readable.read([size])` after the `end` event has been -triggered will return `null`. No runtime error will be raised. - -#### readable.setEncoding(encoding) - -* `encoding` {String} The encoding to use. -* Return: `this` +Once the internal buffer is drained, a `readable` event will fire +again when more data is available. -Call this function to cause the stream to return strings of the -specified encoding instead of Buffer objects. For example, if you do -`readable.setEncoding('utf8')`, then the output data will be -interpreted as UTF-8 data, and returned as strings. If you do -`readable.setEncoding('hex')`, then the data will be encoded in -hexadecimal string format. +The `readable` event is not emitted in the "flowing" mode with the +sole exception of the last one, on end-of-stream. -This properly handles multi-byte characters that would otherwise be -potentially mangled if you simply pulled the Buffers directly and -called `buf.toString(encoding)` on them. If you want to read the data -as strings, always use this method. +The 'readable' event indicates that the stream has new information: +either new data is available or the end of the stream has been reached. +In the former case, `.read()` will return that data. In the latter case, +`.read()` will return null. For instance, in the following example, `foo.txt` +is an empty file: ```javascript -var readable = getReadableStreamSomehow(); -readable.setEncoding('utf8'); -readable.on('data', function(chunk) { - assert.equal(typeof chunk, 'string'); - console.log('got %d characters of string data', chunk.length); +var fs = require('fs'); +var rr = fs.createReadStream('foo.txt'); +rr.on('readable', function() { + console.log('readable:', rr.read()); +}); +rr.on('end', function() { + console.log('end'); }); ``` -#### readable.resume() +The output of running this script is: -* Return: `this` +``` +bash-3.2$ node test.js +readable: null +end +``` -This method will cause the readable stream to resume emitting `data` -events. +#### readable.isPaused() -This method will switch the stream into flowing mode. If you do *not* -want to consume the data from a stream, but you *do* want to get to -its `end` event, you can call [`readable.resume()`][] to open the flow of -data. +* Return: `Boolean` + +This method returns whether or not the `readable` has been **explicitly** +paused by client code (using `readable.pause()` without a corresponding +`readable.resume()`). ```javascript -var readable = getReadableStreamSomehow(); -readable.resume(); -readable.on('end', function() { - console.log('got to the end, but did not read anything'); -}); +var readable = new stream.Readable + +readable.isPaused() // === false +readable.pause() +readable.isPaused() // === true +readable.resume() +readable.isPaused() // === false ``` #### readable.pause() @@ -346,24 +292,6 @@ readable.on('data', function(chunk) { }); ``` -#### readable.isPaused() - -* Return: `Boolean` - -This method returns whether or not the `readable` has been **explicitly** -paused by client code (using `readable.pause()` without a corresponding -`readable.resume()`). - -```javascript -var readable = new stream.Readable - -readable.isPaused() // === false -readable.pause() -readable.isPaused() // === true -readable.resume() -readable.isPaused() // === false -``` - #### readable.pipe(destination[, options]) * `destination` {[Writable][] Stream} The destination for writing data @@ -416,54 +344,137 @@ reader.on('end', function() { Note that `process.stderr` and `process.stdout` are never closed until the process exits, regardless of the specified options. -#### readable.unpipe([destination]) +#### readable.read([size]) -* `destination` {[Writable][] Stream} Optional specific stream to unpipe +* `size` {Number} Optional argument to specify how much data to read. +* Return {String | Buffer | null} -This method will remove the hooks set up for a previous `pipe()` call. +The `read()` method pulls some data out of the internal buffer and +returns it. If there is no data available, then it will return +`null`. -If the destination is not specified, then all pipes are removed. +If you pass in a `size` argument, then it will return that many +bytes. If `size` bytes are not available, then it will return `null`, +unless we've ended, in which case it will return the data remaining +in the buffer. -If the destination is specified, but no pipe is set up for it, then -this is a no-op. +If you do not specify a `size` argument, then it will return all the +data in the internal buffer. + +This method should only be called in paused mode. In flowing mode, +this method is called automatically until the internal buffer is +drained. ```javascript var readable = getReadableStreamSomehow(); -var writable = fs.createWriteStream('file.txt'); -// All the data from readable goes into 'file.txt', -// but only for the first second -readable.pipe(writable); -setTimeout(function() { - console.log('stop writing to file.txt'); - readable.unpipe(writable); - console.log('manually close the file stream'); - writable.end(); -}, 1000); +readable.on('readable', function() { + var chunk; + while (null !== (chunk = readable.read())) { + console.log('got %d bytes of data', chunk.length); + } +}); ``` -#### readable.unshift(chunk) +If this method returns a data chunk, then it will also trigger the +emission of a [`'data'` event][]. -* `chunk` {Buffer | String} Chunk of data to unshift onto the read queue +Note that calling `readable.read([size])` after the `end` event has been +triggered will return `null`. No runtime error will be raised. -This is useful in certain cases where a stream is being consumed by a -parser, which needs to "un-consume" some data that it has -optimistically pulled out of the source, so that the stream can be -passed on to some other party. +#### readable.resume() -Note that `stream.unshift(chunk)` cannot be called after the `end` event -has been triggered; a runtime error will be raised. +* Return: `this` -If you find that you must often call `stream.unshift(chunk)` in your -programs, consider implementing a [Transform][] stream instead. (See API -for Stream Implementors, below.) +This method will cause the readable stream to resume emitting `data` +events. + +This method will switch the stream into flowing mode. If you do *not* +want to consume the data from a stream, but you *do* want to get to +its `end` event, you can call [`readable.resume()`][] to open the flow of +data. ```javascript -// Pull off a header delimited by \n\n -// use unshift() if we get too much -// Call the callback with (error, header, stream) -var StringDecoder = require('string_decoder').StringDecoder; -function parseHeader(stream, callback) { - stream.on('error', callback); +var readable = getReadableStreamSomehow(); +readable.resume(); +readable.on('end', function() { + console.log('got to the end, but did not read anything'); +}); +``` + +#### readable.setEncoding(encoding) + +* `encoding` {String} The encoding to use. +* Return: `this` + +Call this function to cause the stream to return strings of the +specified encoding instead of Buffer objects. For example, if you do +`readable.setEncoding('utf8')`, then the output data will be +interpreted as UTF-8 data, and returned as strings. If you do +`readable.setEncoding('hex')`, then the data will be encoded in +hexadecimal string format. + +This properly handles multi-byte characters that would otherwise be +potentially mangled if you simply pulled the Buffers directly and +called `buf.toString(encoding)` on them. If you want to read the data +as strings, always use this method. + +```javascript +var readable = getReadableStreamSomehow(); +readable.setEncoding('utf8'); +readable.on('data', function(chunk) { + assert.equal(typeof chunk, 'string'); + console.log('got %d characters of string data', chunk.length); +}); +``` + +#### readable.unpipe([destination]) + +* `destination` {[Writable][] Stream} Optional specific stream to unpipe + +This method will remove the hooks set up for a previous `pipe()` call. + +If the destination is not specified, then all pipes are removed. + +If the destination is specified, but no pipe is set up for it, then +this is a no-op. + +```javascript +var readable = getReadableStreamSomehow(); +var writable = fs.createWriteStream('file.txt'); +// All the data from readable goes into 'file.txt', +// but only for the first second +readable.pipe(writable); +setTimeout(function() { + console.log('stop writing to file.txt'); + readable.unpipe(writable); + console.log('manually close the file stream'); + writable.end(); +}, 1000); +``` + +#### readable.unshift(chunk) + +* `chunk` {Buffer | String} Chunk of data to unshift onto the read queue + +This is useful in certain cases where a stream is being consumed by a +parser, which needs to "un-consume" some data that it has +optimistically pulled out of the source, so that the stream can be +passed on to some other party. + +Note that `stream.unshift(chunk)` cannot be called after the `end` event +has been triggered; a runtime error will be raised. + +If you find that you must often call `stream.unshift(chunk)` in your +programs, consider implementing a [Transform][] stream instead. (See API +for Stream Implementors, below.) + +```javascript +// Pull off a header delimited by \n\n +// use unshift() if we get too much +// Call the callback with (error, header, stream) +var StringDecoder = require('string_decoder').StringDecoder; +function parseHeader(stream, callback) { + stream.on('error', callback); stream.on('readable', onReadable); var decoder = new StringDecoder('utf8'); var header = ''; @@ -528,6 +539,16 @@ myReader.on('readable', function() { }); ``` +### Class: stream.Transform + +Transform streams are [Duplex][] streams where the output is in some way +computed from the input. They implement both the [Readable][] and +[Writable][] interfaces. See above for usage. + +Examples of Transform streams include: + +* [zlib streams][] +* [crypto streams][] ### Class: stream.Writable @@ -547,25 +568,6 @@ Examples of writable streams include: * [child process stdin](child_process.html#child_process_child_stdin) * [process.stdout][], [process.stderr][] -#### writable.write(chunk[, encoding][, callback]) - -* `chunk` {String | Buffer} The data to write -* `encoding` {String} The encoding, if `chunk` is a String -* `callback` {Function} Callback for when this chunk of data is flushed -* Returns: {Boolean} True if the data was handled completely. - -This method writes some data to the underlying system, and calls the -supplied callback once the data has been fully handled. - -The return value indicates if you should continue writing right now. -If the data had to be buffered internally, then it will return -`false`. Otherwise, it will return `true`. - -This return value is strictly advisory. You MAY continue to write, -even if it returns `false`. However, writes will be buffered in -memory, so it is best not to do this excessively. Instead, wait for -the `drain` event before writing more data. - #### Event: 'drain' If a [`writable.write(chunk)`][] call returns false, then the `drain` @@ -600,40 +602,11 @@ function writeOneMillionTimes(writer, data, encoding, callback) { } ``` -#### writable.cork() - -Forces buffering of all writes. - -Buffered data will be flushed either at `.uncork()` or at `.end()` call. - -#### writable.uncork() - -Flush all data, buffered since `.cork()` call. - -#### writable.setDefaultEncoding(encoding) - -* `encoding` {String} The new default encoding - -Sets the default encoding for a writable stream. - -#### writable.end([chunk][, encoding][, callback]) - -* `chunk` {String | Buffer} Optional data to write -* `encoding` {String} The encoding, if `chunk` is a String -* `callback` {Function} Optional callback for when the stream is finished - -Call this method when no more data will be written to the stream. If -supplied, the callback is attached as a listener on the `finish` event. +#### Event: 'error' -Calling [`write()`][] after calling [`end()`][] will raise an error. +* {Error object} -```javascript -// write 'hello, ' and then end with 'world!' -var file = fs.createWriteStream('example.txt'); -file.write('hello, '); -file.end('world!'); -// writing more now is not allowed! -``` +Emitted if there was an error when writing or piping data. #### Event: 'finish' @@ -686,34 +659,59 @@ reader.pipe(writer); reader.unpipe(writer); ``` -#### Event: 'error' +#### writable.cork() -* {Error object} +Forces buffering of all writes. -Emitted if there was an error when writing or piping data. +Buffered data will be flushed either at `.uncork()` or at `.end()` call. -### Class: stream.Duplex +#### writable.end([chunk][, encoding][, callback]) -Duplex streams are streams that implement both the [Readable][] and -[Writable][] interfaces. See above for usage. +* `chunk` {String | Buffer} Optional data to write +* `encoding` {String} The encoding, if `chunk` is a String +* `callback` {Function} Optional callback for when the stream is finished -Examples of Duplex streams include: +Call this method when no more data will be written to the stream. If +supplied, the callback is attached as a listener on the `finish` event. -* [tcp sockets][] -* [zlib streams][] -* [crypto streams][] +Calling [`write()`][] after calling [`end()`][] will raise an error. +```javascript +// write 'hello, ' and then end with 'world!' +var file = fs.createWriteStream('example.txt'); +file.write('hello, '); +file.end('world!'); +// writing more now is not allowed! +``` -### Class: stream.Transform +#### writable.setDefaultEncoding(encoding) -Transform streams are [Duplex][] streams where the output is in some way -computed from the input. They implement both the [Readable][] and -[Writable][] interfaces. See above for usage. +* `encoding` {String} The new default encoding -Examples of Transform streams include: +Sets the default encoding for a writable stream. -* [zlib streams][] -* [crypto streams][] +#### writable.uncork() + +Flush all data, buffered since `.cork()` call. + +#### writable.write(chunk[, encoding][, callback]) + +* `chunk` {String | Buffer} The data to write +* `encoding` {String} The encoding, if `chunk` is a String +* `callback` {Function} Callback for when this chunk of data is flushed +* Returns: {Boolean} True if the data was handled completely. + +This method writes some data to the underlying system, and calls the +supplied callback once the data has been fully handled. + +The return value indicates if you should continue writing right now. +If the data had to be buffered internally, then it will return +`false`. Otherwise, it will return `true`. + +This return value is strictly advisory. You MAY continue to write, +even if it returns `false`. However, writes will be buffered in +memory, so it is best not to do this excessively. Instead, wait for +the `drain` event before writing more data. ## API for Stream Implementors @@ -796,6 +794,49 @@ methods described in [API for Stream Consumers][] above. Otherwise, you can potentially cause adverse side effects in programs that consume your streaming interfaces. +### Class: stream.Duplex + + + +A "duplex" stream is one that is both Readable and Writable, such as a +TCP socket connection. + +Note that `stream.Duplex` is an abstract class designed to be extended +with an underlying implementation of the `_read(size)` and +[`_write(chunk, encoding, callback)`][] methods as you would with a +Readable or Writable stream class. + +Since JavaScript doesn't have multiple prototypal inheritance, this +class prototypally inherits from Readable, and then parasitically from +Writable. It is thus up to the user to implement both the lowlevel +`_read(n)` method as well as the lowlevel +[`_write(chunk, encoding, callback)`][] method on extension duplex classes. + +#### new stream.Duplex(options) + +* `options` {Object} Passed to both Writable and Readable + constructors. Also has the following fields: + * `allowHalfOpen` {Boolean} Default=true. If set to `false`, then + the stream will automatically end the readable side when the + writable side ends and vice versa. + * `readableObjectMode` {Boolean} Default=false. Sets `objectMode` + for readable side of the stream. Has no effect if `objectMode` + is `true`. + * `writableObjectMode` {Boolean} Default=false. Sets `objectMode` + for writable side of the stream. Has no effect if `objectMode` + is `true`. + +In classes that extend the Duplex class, make sure to call the +constructor so that the buffering settings can be properly +initialized. + +### Class: stream.PassThrough + +This is a trivial implementation of a [Transform][] stream that simply +passes the input bytes across to the output. Its purpose is mainly +for examples and testing, but there are occasionally use cases where +it can come in handy as a building block for novel sorts of streams. + ### Class: stream.Readable @@ -807,15 +848,115 @@ Please see above under [API for Stream Consumers][] for how to consume streams in your programs. What follows is an explanation of how to implement Readable streams in your programs. -#### Example: A Counting Stream - - - -This is a basic example of a Readable stream. It emits the numerals -from 1 to 1,000,000 in ascending order, and then ends. +#### new stream.Readable([options]) -```javascript -var Readable = require('stream').Readable; +* `options` {Object} + * `highWaterMark` {Number} The maximum number of bytes to store in + the internal buffer before ceasing to read from the underlying + resource. Default=16kb, or 16 for `objectMode` streams + * `encoding` {String} If specified, then buffers will be decoded to + strings using the specified encoding. Default=null + * `objectMode` {Boolean} Whether this stream should behave + as a stream of objects. Meaning that stream.read(n) returns + a single value instead of a Buffer of size n. Default=false + +In classes that extend the Readable class, make sure to call the +Readable constructor so that the buffering settings can be properly +initialized. + +#### readable.\_read(size) + +* `size` {Number} Number of bytes to read asynchronously + +Note: **Implement this method, but do NOT call it directly.** + +This method is prefixed with an underscore because it is internal to the +class that defines it and should only be called by the internal Readable +class methods. All Readable stream implementations must provide a _read +method to fetch data from the underlying resource. + +When _read is called, if data is available from the resource, `_read` should +start pushing that data into the read queue by calling `this.push(dataChunk)`. +`_read` should continue reading from the resource and pushing data until push +returns false, at which point it should stop reading from the resource. Only +when _read is called again after it has stopped should it start reading +more data from the resource and pushing that data onto the queue. + +Note: once the `_read()` method is called, it will not be called again until +the `push` method is called. + +The `size` argument is advisory. Implementations where a "read" is a +single call that returns data can use this to know how much data to +fetch. Implementations where that is not relevant, such as TCP or +TLS, may ignore this argument, and simply provide data whenever it +becomes available. There is no need, for example to "wait" until +`size` bytes are available before calling [`stream.push(chunk)`][]. + +#### readable.push(chunk[, encoding]) + +* `chunk` {Buffer | null | String} Chunk of data to push into the read queue +* `encoding` {String} Encoding of String chunks. Must be a valid + Buffer encoding, such as `'utf8'` or `'ascii'` +* return {Boolean} Whether or not more pushes should be performed + +Note: **This method should be called by Readable implementors, NOT +by consumers of Readable streams.** + +If a value other than null is passed, The `push()` method adds a chunk of data +into the queue for subsequent stream processors to consume. If `null` is +passed, it signals the end of the stream (EOF), after which no more data +can be written. + +The data added with `push` can be pulled out by calling the `read()` method +when the `'readable'`event fires. + +This API is designed to be as flexible as possible. For example, +you may be wrapping a lower-level source which has some sort of +pause/resume mechanism, and a data callback. In those cases, you +could wrap the low-level source object by doing something like this: + +```javascript +// source is an object with readStop() and readStart() methods, +// and an `ondata` member that gets called when it has data, and +// an `onend` member that gets called when the data is over. + +util.inherits(SourceWrapper, Readable); + +function SourceWrapper(options) { + Readable.call(this, options); + + this._source = getLowlevelSourceObject(); + var self = this; + + // Every time there's data, we push it into the internal buffer. + this._source.ondata = function(chunk) { + // if push() returns false, then we need to stop reading from source + if (!self.push(chunk)) + self._source.readStop(); + }; + + // When the source ends, we push the EOF-signaling `null` chunk + this._source.onend = function() { + self.push(null); + }; +} + +// _read will be called when the stream wants to pull more data in +// the advisory size argument is ignored in this case. +SourceWrapper.prototype._read = function(size) { + this._source.readStart(); +}; +``` + +#### Example: A Counting Stream + + + +This is a basic example of a Readable stream. It emits the numerals +from 1 to 1,000,000 in ascending order, and then ends. + +```javascript +var Readable = require('stream').Readable; var util = require('util'); util.inherits(Counter, Readable); @@ -951,220 +1092,6 @@ SimpleProtocol.prototype._read = function(n) { // with the parsed header data. ``` - -#### new stream.Readable([options]) - -* `options` {Object} - * `highWaterMark` {Number} The maximum number of bytes to store in - the internal buffer before ceasing to read from the underlying - resource. Default=16kb, or 16 for `objectMode` streams - * `encoding` {String} If specified, then buffers will be decoded to - strings using the specified encoding. Default=null - * `objectMode` {Boolean} Whether this stream should behave - as a stream of objects. Meaning that stream.read(n) returns - a single value instead of a Buffer of size n. Default=false - -In classes that extend the Readable class, make sure to call the -Readable constructor so that the buffering settings can be properly -initialized. - -#### readable.\_read(size) - -* `size` {Number} Number of bytes to read asynchronously - -Note: **Implement this method, but do NOT call it directly.** - -This method is prefixed with an underscore because it is internal to the -class that defines it and should only be called by the internal Readable -class methods. All Readable stream implementations must provide a _read -method to fetch data from the underlying resource. - -When _read is called, if data is available from the resource, `_read` should -start pushing that data into the read queue by calling `this.push(dataChunk)`. -`_read` should continue reading from the resource and pushing data until push -returns false, at which point it should stop reading from the resource. Only -when _read is called again after it has stopped should it start reading -more data from the resource and pushing that data onto the queue. - -Note: once the `_read()` method is called, it will not be called again until -the `push` method is called. - -The `size` argument is advisory. Implementations where a "read" is a -single call that returns data can use this to know how much data to -fetch. Implementations where that is not relevant, such as TCP or -TLS, may ignore this argument, and simply provide data whenever it -becomes available. There is no need, for example to "wait" until -`size` bytes are available before calling [`stream.push(chunk)`][]. - -#### readable.push(chunk[, encoding]) - -* `chunk` {Buffer | null | String} Chunk of data to push into the read queue -* `encoding` {String} Encoding of String chunks. Must be a valid - Buffer encoding, such as `'utf8'` or `'ascii'` -* return {Boolean} Whether or not more pushes should be performed - -Note: **This method should be called by Readable implementors, NOT -by consumers of Readable streams.** - -If a value other than null is passed, The `push()` method adds a chunk of data -into the queue for subsequent stream processors to consume. If `null` is -passed, it signals the end of the stream (EOF), after which no more data -can be written. - -The data added with `push` can be pulled out by calling the `read()` method -when the `'readable'`event fires. - -This API is designed to be as flexible as possible. For example, -you may be wrapping a lower-level source which has some sort of -pause/resume mechanism, and a data callback. In those cases, you -could wrap the low-level source object by doing something like this: - -```javascript -// source is an object with readStop() and readStart() methods, -// and an `ondata` member that gets called when it has data, and -// an `onend` member that gets called when the data is over. - -util.inherits(SourceWrapper, Readable); - -function SourceWrapper(options) { - Readable.call(this, options); - - this._source = getLowlevelSourceObject(); - var self = this; - - // Every time there's data, we push it into the internal buffer. - this._source.ondata = function(chunk) { - // if push() returns false, then we need to stop reading from source - if (!self.push(chunk)) - self._source.readStop(); - }; - - // When the source ends, we push the EOF-signaling `null` chunk - this._source.onend = function() { - self.push(null); - }; -} - -// _read will be called when the stream wants to pull more data in -// the advisory size argument is ignored in this case. -SourceWrapper.prototype._read = function(size) { - this._source.readStart(); -}; -``` - - -### Class: stream.Writable - - - -`stream.Writable` is an abstract class designed to be extended with an -underlying implementation of the [`_write(chunk, encoding, callback)`][] method. - -Please see above under [API for Stream Consumers][] for how to consume -writable streams in your programs. What follows is an explanation of -how to implement Writable streams in your programs. - -#### new stream.Writable([options]) - -* `options` {Object} - * `highWaterMark` {Number} Buffer level when [`write()`][] starts - returning false. Default=16kb, or 16 for `objectMode` streams - * `decodeStrings` {Boolean} Whether or not to decode strings into - Buffers before passing them to [`_write()`][]. Default=true - * `objectMode` {Boolean} Whether or not the `write(anyObj)` is - a valid operation. If set you can write arbitrary data instead - of only `Buffer` / `String` data. Default=false - -In classes that extend the Writable class, make sure to call the -constructor so that the buffering settings can be properly -initialized. - -#### writable.\_write(chunk, encoding, callback) - -* `chunk` {Buffer | String} The chunk to be written. Will **always** - be a buffer unless the `decodeStrings` option was set to `false`. -* `encoding` {String} If the chunk is a string, then this is the - encoding type. If chunk is a buffer, then this is the special - value - 'buffer', ignore it in this case. -* `callback` {Function} Call this function (optionally with an error - argument) when you are done processing the supplied chunk. - -All Writable stream implementations must provide a [`_write()`][] -method to send data to the underlying resource. - -Note: **This function MUST NOT be called directly.** It should be -implemented by child classes, and called by the internal Writable -class methods only. - -Call the callback using the standard `callback(error)` pattern to -signal that the write completed successfully or with an error. - -If the `decodeStrings` flag is set in the constructor options, then -`chunk` may be a string rather than a Buffer, and `encoding` will -indicate the sort of string that it is. This is to support -implementations that have an optimized handling for certain string -data encodings. If you do not explicitly set the `decodeStrings` -option to `false`, then you can safely ignore the `encoding` argument, -and assume that `chunk` will always be a Buffer. - -This method is prefixed with an underscore because it is internal to -the class that defines it, and should not be called directly by user -programs. However, you **are** expected to override this method in -your own extension classes. - -#### writable.\_writev(chunks, callback) - -* `chunks` {Array} The chunks to be written. Each chunk has following - format: `{ chunk: ..., encoding: ... }`. -* `callback` {Function} Call this function (optionally with an error - argument) when you are done processing the supplied chunks. - -Note: **This function MUST NOT be called directly.** It may be -implemented by child classes, and called by the internal Writable -class methods only. - -This function is completely optional to implement. In most cases it is -unnecessary. If implemented, it will be called with all the chunks -that are buffered in the write queue. - - -### Class: stream.Duplex - - - -A "duplex" stream is one that is both Readable and Writable, such as a -TCP socket connection. - -Note that `stream.Duplex` is an abstract class designed to be extended -with an underlying implementation of the `_read(size)` and -[`_write(chunk, encoding, callback)`][] methods as you would with a -Readable or Writable stream class. - -Since JavaScript doesn't have multiple prototypal inheritance, this -class prototypally inherits from Readable, and then parasitically from -Writable. It is thus up to the user to implement both the lowlevel -`_read(n)` method as well as the lowlevel -[`_write(chunk, encoding, callback)`][] method on extension duplex classes. - -#### new stream.Duplex(options) - -* `options` {Object} Passed to both Writable and Readable - constructors. Also has the following fields: - * `allowHalfOpen` {Boolean} Default=true. If set to `false`, then - the stream will automatically end the readable side when the - writable side ends and vice versa. - * `readableObjectMode` {Boolean} Default=false. Sets `objectMode` - for readable side of the stream. Has no effect if `objectMode` - is `true`. - * `writableObjectMode` {Boolean} Default=false. Sets `objectMode` - for writable side of the stream. Has no effect if `objectMode` - is `true`. - -In classes that extend the Duplex class, make sure to call the -constructor so that the buffering settings can be properly -initialized. - - ### Class: stream.Transform A "transform" stream is a duplex stream where the output is causally @@ -1186,9 +1113,44 @@ also implement the `_flush()` method. (See below.) * `options` {Object} Passed to both Writable and Readable constructors. -In classes that extend the Transform class, make sure to call the -constructor so that the buffering settings can be properly -initialized. +In classes that extend the Transform class, make sure to call the +constructor so that the buffering settings can be properly +initialized. + +#### Events: 'finish' and 'end' + +The [`finish`][] and [`end`][] events are from the parent Writable +and Readable classes respectively. The `finish` event is fired after +`.end()` is called and all chunks have been processed by `_transform`, +`end` is fired after all data has been output which is after the callback +in `_flush` has been called. + +#### transform.\_flush(callback) + +* `callback` {Function} Call this function (optionally with an error + argument) when you are done flushing any remaining data. + +Note: **This function MUST NOT be called directly.** It MAY be implemented +by child classes, and if so, will be called by the internal Transform +class methods only. + +In some cases, your transform operation may need to emit a bit more +data at the end of the stream. For example, a `Zlib` compression +stream will store up some internal state so that it can optimally +compress the output. At the end, however, it needs to do the best it +can with what is left, so that the data will be complete. + +In those cases, you can implement a `_flush` method, which will be +called at the very end, after all the written data is consumed, but +before emitting `end` to signal the end of the readable side. Just +like with `_transform`, call `transform.push(chunk)` zero or more +times, as appropriate, and call `callback` when the flush operation is +complete. + +This method is prefixed with an underscore because it is internal to +the class that defines it, and should not be called directly by user +programs. However, you **are** expected to override this method in +your own extension classes. #### transform.\_transform(chunk, encoding, callback) @@ -1238,41 +1200,6 @@ the class that defines it, and should not be called directly by user programs. However, you **are** expected to override this method in your own extension classes. -#### transform.\_flush(callback) - -* `callback` {Function} Call this function (optionally with an error - argument) when you are done flushing any remaining data. - -Note: **This function MUST NOT be called directly.** It MAY be implemented -by child classes, and if so, will be called by the internal Transform -class methods only. - -In some cases, your transform operation may need to emit a bit more -data at the end of the stream. For example, a `Zlib` compression -stream will store up some internal state so that it can optimally -compress the output. At the end, however, it needs to do the best it -can with what is left, so that the data will be complete. - -In those cases, you can implement a `_flush` method, which will be -called at the very end, after all the written data is consumed, but -before emitting `end` to signal the end of the readable side. Just -like with `_transform`, call `transform.push(chunk)` zero or more -times, as appropriate, and call `callback` when the flush operation is -complete. - -This method is prefixed with an underscore because it is internal to -the class that defines it, and should not be called directly by user -programs. However, you **are** expected to override this method in -your own extension classes. - -#### Events: 'finish' and 'end' - -The [`finish`][] and [`end`][] events are from the parent Writable -and Readable classes respectively. The `finish` event is fired after -`.end()` is called and all chunks have been processed by `_transform`, -`end` is fired after all data has been output which is after the callback -in `_flush` has been called. - #### Example: `SimpleProtocol` parser v2 The example above of a simple protocol parser can be implemented @@ -1351,13 +1278,79 @@ SimpleProtocol.prototype._transform = function(chunk, encoding, done) { // with the parsed header data. ``` +### Class: stream.Writable -### Class: stream.PassThrough + -This is a trivial implementation of a [Transform][] stream that simply -passes the input bytes across to the output. Its purpose is mainly -for examples and testing, but there are occasionally use cases where -it can come in handy as a building block for novel sorts of streams. +`stream.Writable` is an abstract class designed to be extended with an +underlying implementation of the [`_write(chunk, encoding, callback)`][] method. + +Please see above under [API for Stream Consumers][] for how to consume +writable streams in your programs. What follows is an explanation of +how to implement Writable streams in your programs. + +#### new stream.Writable([options]) + +* `options` {Object} + * `highWaterMark` {Number} Buffer level when [`write()`][] starts + returning false. Default=16kb, or 16 for `objectMode` streams + * `decodeStrings` {Boolean} Whether or not to decode strings into + Buffers before passing them to [`_write()`][]. Default=true + * `objectMode` {Boolean} Whether or not the `write(anyObj)` is + a valid operation. If set you can write arbitrary data instead + of only `Buffer` / `String` data. Default=false + +In classes that extend the Writable class, make sure to call the +constructor so that the buffering settings can be properly +initialized. + +#### writable.\_write(chunk, encoding, callback) + +* `chunk` {Buffer | String} The chunk to be written. Will **always** + be a buffer unless the `decodeStrings` option was set to `false`. +* `encoding` {String} If the chunk is a string, then this is the + encoding type. If chunk is a buffer, then this is the special + value - 'buffer', ignore it in this case. +* `callback` {Function} Call this function (optionally with an error + argument) when you are done processing the supplied chunk. + +All Writable stream implementations must provide a [`_write()`][] +method to send data to the underlying resource. + +Note: **This function MUST NOT be called directly.** It should be +implemented by child classes, and called by the internal Writable +class methods only. + +Call the callback using the standard `callback(error)` pattern to +signal that the write completed successfully or with an error. + +If the `decodeStrings` flag is set in the constructor options, then +`chunk` may be a string rather than a Buffer, and `encoding` will +indicate the sort of string that it is. This is to support +implementations that have an optimized handling for certain string +data encodings. If you do not explicitly set the `decodeStrings` +option to `false`, then you can safely ignore the `encoding` argument, +and assume that `chunk` will always be a Buffer. + +This method is prefixed with an underscore because it is internal to +the class that defines it, and should not be called directly by user +programs. However, you **are** expected to override this method in +your own extension classes. + +#### writable.\_writev(chunks, callback) + +* `chunks` {Array} The chunks to be written. Each chunk has following + format: `{ chunk: ..., encoding: ... }`. +* `callback` {Function} Call this function (optionally with an error + argument) when you are done processing the supplied chunks. + +Note: **This function MUST NOT be called directly.** It may be +implemented by child classes, and called by the internal Writable +class methods only. + +This function is completely optional to implement. In most cases it is +unnecessary. If implemented, it will be called with all the chunks +that are buffered in the write queue. ## Simplified Constructor API @@ -1370,22 +1363,16 @@ This can be done by passing the appropriate methods as constructor options: Examples: -### Readable +### Duplex ```javascript -var readable = new stream.Readable({ +var duplex = new stream.Duplex({ read: function(n) { // sets this._read under the hood // push data onto the read queue, passing null // will signal the end of the stream (EOF) this.push(chunk); - } -}); -``` - -### Writable -```javascript -var writable = new stream.Writable({ + }, write: function(chunk, encoding, next) { // sets this._write under the hood @@ -1396,18 +1383,6 @@ var writable = new stream.Writable({ // or -var writable = new stream.Writable({ - writev: function(chunks, next) { - // sets this._writev under the hood - - // An optional error can be passed as the first argument - next() - } -}); -``` - -### Duplex -```javascript var duplex = new stream.Duplex({ read: function(n) { // sets this._read under the hood @@ -1416,29 +1391,24 @@ var duplex = new stream.Duplex({ // will signal the end of the stream (EOF) this.push(chunk); }, - write: function(chunk, encoding, next) { - // sets this._write under the hood + writev: function(chunks, next) { + // sets this._writev under the hood // An optional error can be passed as the first argument next() } }); +``` -// or - -var duplex = new stream.Duplex({ +### Readable +```javascript +var readable = new stream.Readable({ read: function(n) { // sets this._read under the hood // push data onto the read queue, passing null // will signal the end of the stream (EOF) this.push(chunk); - }, - writev: function(chunks, next) { - // sets this._writev under the hood - - // An optional error can be passed as the first argument - next() } }); ``` @@ -1466,6 +1436,29 @@ var transform = new stream.Transform({ }); ``` +### Writable +```javascript +var writable = new stream.Writable({ + write: function(chunk, encoding, next) { + // sets this._write under the hood + + // An optional error can be passed as the first argument + next() + } +}); + +// or + +var writable = new stream.Writable({ + writev: function(chunks, next) { + // sets this._writev under the hood + + // An optional error can be passed as the first argument + next() + } +}); +``` + ## Streams: Under the Hood @@ -1493,40 +1486,6 @@ The purpose of streams, especially with the `pipe()` method, is to limit the buffering of data to acceptable levels, so that sources and destinations of varying speed will not overwhelm the available memory. -### `stream.read(0)` - -There are some cases where you want to trigger a refresh of the -underlying readable stream mechanisms, without actually consuming any -data. In that case, you can call `stream.read(0)`, which will always -return null. - -If the internal read buffer is below the `highWaterMark`, and the -stream is not currently reading, then calling `read(0)` will trigger -a low-level `_read` call. - -There is almost never a need to do this. However, you will see some -cases in Node.js's internals where this is done, particularly in the -Readable stream class internals. - -### `stream.push('')` - -Pushing a zero-byte string or Buffer (when not in [Object mode][]) has an -interesting side effect. Because it *is* a call to -[`stream.push()`][], it will end the `reading` process. However, it -does *not* add any data to the readable buffer, so there's nothing for -a user to consume. - -Very rarely, there are cases where you have no data to provide now, -but the consumer of your stream (or, perhaps, another bit of your own -code) will know when to check again, by calling `stream.read(0)`. In -those cases, you *may* call `stream.push('')`. - -So far, the only use case for this functionality is in the -[tls.CryptoStream][] class, which is deprecated in Node.js/io.js v1.0. If you -find that you have to use `stream.push('')`, please consider another -approach, because it almost certainly indicates that something is -horribly wrong. - ### Compatibility with Older Node.js Versions @@ -1684,6 +1643,39 @@ JSONParseStream.prototype._flush = function(cb) { }; ``` +### `stream.read(0)` + +There are some cases where you want to trigger a refresh of the +underlying readable stream mechanisms, without actually consuming any +data. In that case, you can call `stream.read(0)`, which will always +return null. + +If the internal read buffer is below the `highWaterMark`, and the +stream is not currently reading, then calling `read(0)` will trigger +a low-level `_read` call. + +There is almost never a need to do this. However, you will see some +cases in Node.js's internals where this is done, particularly in the +Readable stream class internals. + +### `stream.push('')` + +Pushing a zero-byte string or Buffer (when not in [Object mode][]) has an +interesting side effect. Because it *is* a call to +[`stream.push()`][], it will end the `reading` process. However, it +does *not* add any data to the readable buffer, so there's nothing for +a user to consume. + +Very rarely, there are cases where you have no data to provide now, +but the consumer of your stream (or, perhaps, another bit of your own +code) will know when to check again, by calling `stream.read(0)`. In +those cases, you *may* call `stream.push('')`. + +So far, the only use case for this functionality is in the +[tls.CryptoStream][] class, which is deprecated in Node.js/io.js v1.0. If you +find that you have to use `stream.push('')`, please consider another +approach, because it almost certainly indicates that something is +horribly wrong. [EventEmitter]: events.html#events_class_events_eventemitter [Object mode]: #stream_object_mode