From b5c2933c9b14c969e93c9714279f29f977aa1164 Mon Sep 17 00:00:00 2001 From: Benjamin Steephenson Date: Tue, 3 Mar 2015 09:59:07 -0500 Subject: [PATCH 1/7] docs:events clarify emitter.listener() behavior Clarifies that emitter.listener() returns a copy, not a reference Resolves issue #9022 --- doc/api/events.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/api/events.markdown b/doc/api/events.markdown index a98eb5ed3fd5..857d5658bbd5 100644 --- a/doc/api/events.markdown +++ b/doc/api/events.markdown @@ -104,7 +104,7 @@ Note that `emitter.setMaxListeners(n)` still has precedence over ### emitter.listeners(event) -Returns an array of listeners for the specified event. +Returns a copy of the array of listeners for the specified event. server.on('connection', function (stream) { console.log('someone connected!'); From ec67d78df8f87b9c1b62f9f4750c7144a569b93d Mon Sep 17 00:00:00 2001 From: James M Snell Date: Wed, 1 Jul 2015 12:52:47 -0700 Subject: [PATCH 2/7] doc: two minor stream doc improvements per: https://github.com/joyent/node/issues/14596 1. document that a runtime error will occur if you attempt to unshift after the end event 2. document that calling read after the end event will return null and will not trigger a runtime error --- doc/api/stream.markdown | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/doc/api/stream.markdown b/doc/api/stream.markdown index 8434a4333c20..dfc9a6d32d4f 100644 --- a/doc/api/stream.markdown +++ b/doc/api/stream.markdown @@ -228,7 +228,7 @@ returns it. If there is no data available, then it will return If you pass in a `size` argument, then it will return that many bytes. If `size` bytes are not available, then it will return `null`, -unless we've ended, in which case it will return the data remaining +unless we've ended, in which case it will return the data remaining in the buffer. If you do not specify a `size` argument, then it will return all the @@ -251,6 +251,9 @@ readable.on('readable', function() { If this method returns a data chunk, then it will also trigger the emission of a [`'data'` event][]. +Note that calling `readable.read([size])` after the `end` event has been +triggered will return `null`. No runtime error will be raised. + #### readable.setEncoding(encoding) * `encoding` {String} The encoding to use. @@ -422,6 +425,9 @@ parser, which needs to "un-consume" some data that it has optimistically pulled out of the source, so that the stream can be passed on to some other party. +Note that `stream.unshift(chunk)` cannot be called after the `end` event +has been triggered; a runtime error will be raised. + If you find that you must often call `stream.unshift(chunk)` in your programs, consider implementing a [Transform][] stream instead. (See API for Stream Implementors, below.) From 003c1b23df4b08df3f3e5a036eb24ed13121eb06 Mon Sep 17 00:00:00 2001 From: fresheneesz Date: Thu, 25 Jun 2015 16:05:40 -0700 Subject: [PATCH 3/7] doc: clarify Readable._read and Readable.push Minor clarifications around Readable._read and Readable.push to make their implementation/usage easier to understand. https://github.com/joyent/node/issues/14124#issuecomment-115392674 --- doc/api/stream.markdown | 45 ++++++++++++++++++----------------------- 1 file changed, 20 insertions(+), 25 deletions(-) diff --git a/doc/api/stream.markdown b/doc/api/stream.markdown index dfc9a6d32d4f..71a8e4ebb98b 100644 --- a/doc/api/stream.markdown +++ b/doc/api/stream.markdown @@ -944,24 +944,22 @@ initialized. * `size` {Number} Number of bytes to read asynchronously -Note: **Implement this function, but do NOT call it directly.** +Note: **Implement this method, but do NOT call it directly.** -This function should NOT be called directly. It should be implemented -by child classes, and only called by the internal Readable class -methods. +This method is prefixed with an underscore because it is internal to the +class that defines it and should only be called by the internal Readable +class methods. All Readable stream implementations must provide a _read +method to fetch data from the underlying resource. -All Readable stream implementations must provide a `_read` method to -fetch data from the underlying resource. +When _read is called, if data is available from the resource, `_read` should +start pushing that data into the read queue by calling `this.push(dataChunk)`. +`_read` should continue reading from the resource and pushing data until push +returns false, at which point it should stop reading from the resource. Only +when _read is called again after it has stopped should it start reading +more data from the resource and pushing that data onto the queue. -This method is prefixed with an underscore because it is internal to -the class that defines it, and should not be called directly by user -programs. However, you **are** expected to override this method in -your own extension classes. - -When data is available, put it into the read queue by calling -`readable.push(chunk)`. If `push` returns false, then you should stop -reading. When `_read` is called again, you should start pushing more -data. +Note: once the `_read()` method is called, it will not be called again until +the `push` method is called. The `size` argument is advisory. Implementations where a "read" is a single call that returns data can use this to know how much data to @@ -977,19 +975,16 @@ becomes available. There is no need, for example to "wait" until Buffer encoding, such as `'utf8'` or `'ascii'` * return {Boolean} Whether or not more pushes should be performed -Note: **This function should be called by Readable implementors, NOT +Note: **This method should be called by Readable implementors, NOT by consumers of Readable streams.** -The `_read()` function will not be called again until at least one -`push(chunk)` call is made. - -The `Readable` class works by putting data into a read queue to be -pulled out later by calling the `read()` method when the `'readable'` -event fires. +If a value other than null is passed, The `push()` method adds a chunk of data +into the queue for subsequent stream processors to consume. If `null` is +passed, it signals the end of the stream (EOF), after which no more data +can be written. -The `push()` method will explicitly insert some data into the read -queue. If it is called with `null` then it will signal the end of the -data (EOF). +The data added with `push` can be pulled out by calling the `read()` method +when the `'readable'`event fires. This API is designed to be as flexible as possible. For example, you may be wrapping a lower-level source which has some sort of From 6528aed1b929c03c05c59ebed664786c8a7c1ac5 Mon Sep 17 00:00:00 2001 From: Daniel Rentz Date: Thu, 25 Jun 2015 21:43:56 +0200 Subject: [PATCH 4/7] Update util.markdown Fix for https://github.com/joyent/node/issues/25559 (Typo in example of util.deprecate() documentation) --- doc/api/util.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/api/util.markdown b/doc/api/util.markdown index cc639ddb4943..3765963406c2 100644 --- a/doc/api/util.markdown +++ b/doc/api/util.markdown @@ -261,7 +261,7 @@ through the `constructor.super_` property. Marks that a method should not be used any more. - exports.puts = exports.deprecate(function() { + exports.puts = util.deprecate(function() { for (var i = 0, len = arguments.length; i < len; ++i) { process.stdout.write(arguments[i] + '\n'); } From b3a8fec6fd2dd149e77e4e8b84c34ddb096a54f9 Mon Sep 17 00:00:00 2001 From: James M Snell Date: Mon, 6 Jul 2015 11:18:06 -0700 Subject: [PATCH 5/7] doc: stream.unshift does not reset reading state Per https://github.com/joyent/node/issues/14604, Document that performing an `unshift` during a read can have unexpected results. Following the `unshift` with a `push('')` resets the reading state appropriately. Also indicate that doing an `unshift` during a read is not optimal and should be avoided. --- doc/api/stream.markdown | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/doc/api/stream.markdown b/doc/api/stream.markdown index 71a8e4ebb98b..9686786c49c9 100644 --- a/doc/api/stream.markdown +++ b/doc/api/stream.markdown @@ -466,6 +466,13 @@ function parseHeader(stream, callback) { } } ``` +Note that, unlike `stream.push(chunk)`, `stream.unshift(chunk)` will not +end the reading process by resetting the internal reading state of the +stream. This can cause unexpected results if `unshift` is called during a +read (i.e. from within a `_read` implementation on a custom stream). Following +the call to `unshift` with an immediate `stream.push('')` will reset the +reading state appropriately, however it is best to simply avoid calling +`unshift` while in the process of performing a read. #### readable.wrap(stream) @@ -905,6 +912,10 @@ SimpleProtocol.prototype._read = function(n) { // back into the read queue so that our consumer will see it. var b = chunk.slice(split); this.unshift(b); + // calling unshift by itself does not reset the reading state + // of the stream; since we're inside _read, doing an additional + // push('') will reset the state appropriately. + this.push(''); // and let them know that we are done parsing the header. this.emit('header', this.header); @@ -946,19 +957,19 @@ initialized. Note: **Implement this method, but do NOT call it directly.** -This method is prefixed with an underscore because it is internal to the -class that defines it and should only be called by the internal Readable -class methods. All Readable stream implementations must provide a _read +This method is prefixed with an underscore because it is internal to the +class that defines it and should only be called by the internal Readable +class methods. All Readable stream implementations must provide a _read method to fetch data from the underlying resource. -When _read is called, if data is available from the resource, `_read` should -start pushing that data into the read queue by calling `this.push(dataChunk)`. -`_read` should continue reading from the resource and pushing data until push -returns false, at which point it should stop reading from the resource. Only -when _read is called again after it has stopped should it start reading +When _read is called, if data is available from the resource, `_read` should +start pushing that data into the read queue by calling `this.push(dataChunk)`. +`_read` should continue reading from the resource and pushing data until push +returns false, at which point it should stop reading from the resource. Only +when _read is called again after it has stopped should it start reading more data from the resource and pushing that data onto the queue. -Note: once the `_read()` method is called, it will not be called again until +Note: once the `_read()` method is called, it will not be called again until the `push` method is called. The `size` argument is advisory. Implementations where a "read" is a @@ -978,12 +989,12 @@ becomes available. There is no need, for example to "wait" until Note: **This method should be called by Readable implementors, NOT by consumers of Readable streams.** -If a value other than null is passed, The `push()` method adds a chunk of data -into the queue for subsequent stream processors to consume. If `null` is -passed, it signals the end of the stream (EOF), after which no more data +If a value other than null is passed, The `push()` method adds a chunk of data +into the queue for subsequent stream processors to consume. If `null` is +passed, it signals the end of the stream (EOF), after which no more data can be written. -The data added with `push` can be pulled out by calling the `read()` method +The data added with `push` can be pulled out by calling the `read()` method when the `'readable'`event fires. This API is designed to be as flexible as possible. For example, From 970e2f1c4a9cf1f3ab0f4191a5ea2bf1cab4e002 Mon Sep 17 00:00:00 2001 From: James M Snell Date: Mon, 6 Jul 2015 11:50:17 -0700 Subject: [PATCH 6/7] doc: readable event clarification per https://github.com/joyent/node/issues/14597 Indicate that `'readable'` indicates only that data can be read from the stream, not that there is actually data to be consumed. `readable.read([size])` can still return null. Includes an example that illustrates the point. --- doc/api/stream.markdown | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/doc/api/stream.markdown b/doc/api/stream.markdown index 9686786c49c9..872898f29745 100644 --- a/doc/api/stream.markdown +++ b/doc/api/stream.markdown @@ -167,6 +167,32 @@ again when more data is available. The `readable` event is not emitted in the "flowing" mode with the sole exception of the last one, on end-of-stream. +Note that the `'readable'` event indicates only that data *can* be +read from the stream. It does not indicate whether there is actual +data to be consumed. The callback passed to handle the `'readable'` +event must be prepared to handle a `null` response from +`readable.read([size])`. For instance, in the following example, `foo.txt` +is an empty file: + +```javascript +var fs = require('fs'); +var rr = fs.createReadStream('foo.txt'); +rr.on('readable', function() { + console.log('readable:', rr.read()); +}); +rr.on('end', function() { + console.log('end'); +}); +``` + +The output of running this script is: + +``` +bash-3.2$ node test.js +readable: null +end +``` + #### Event: 'data' * `chunk` {Buffer | String} The chunk of data. From 4ed55ffa6000d5b06a7c70eaab1442058a35944c Mon Sep 17 00:00:00 2001 From: James M Snell Date: Thu, 9 Jul 2015 16:35:32 -0700 Subject: [PATCH 7/7] doc: additional refinement to readable event Per https://github.com/joyent/node/pull/25635#discussion_r33973696 Additional refinement to the clarification on the `readable` event --- doc/api/stream.markdown | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/doc/api/stream.markdown b/doc/api/stream.markdown index 872898f29745..b327480fe536 100644 --- a/doc/api/stream.markdown +++ b/doc/api/stream.markdown @@ -167,11 +167,10 @@ again when more data is available. The `readable` event is not emitted in the "flowing" mode with the sole exception of the last one, on end-of-stream. -Note that the `'readable'` event indicates only that data *can* be -read from the stream. It does not indicate whether there is actual -data to be consumed. The callback passed to handle the `'readable'` -event must be prepared to handle a `null` response from -`readable.read([size])`. For instance, in the following example, `foo.txt` +The 'readable' event indicates that the stream has new information: +either new data is available or the end of the stream has been reached. +In the former case, `.read()` will return that data. In the latter case, +`.read()` will return null. For instance, in the following example, `foo.txt` is an empty file: ```javascript