[BUG] Callback called multiple times

[olegario96]

Describe the bug
I'm currently using your lib to parsing a stream received from a S3 bucket. But after some time I've noticed the following error:

{
    "errorType": "Error",
    "errorMessage": "Callback called multiple times",
    "code": "ERR_MULTIPLE_CALLBACK",
    "stack": ["Error [ERR_MULTIPLE_CALLBACK]: Callback called multiple times", "    at CsvParserStream.afterTransform (_stream_transform.js:89:31)", "    at callNext (/var/task/node_modules/@fast-csv/parse/build/src/CsvParserStream.js:93:28)", "    at /var/task/node_modules/@fast-csv/parse/build/src/CsvParserStream.js:119:28", "    at CsvParserStream.transformRow (/var/task/node_modules/@fast-csv/parse/build/src/CsvParserStream.js:165:13)", "    at iterate (/var/task/node_modules/@fast-csv/parse/build/src/CsvParserStream.js:116:25)", "    at Immediate._onImmediate (/var/task/node_modules/@fast-csv/parse/build/src/CsvParserStream.js:97:40)", "    at processImmediate (internal/timers.js:456:21)", "    at process.topLevelDomainCallback (domain.js:137:15)"]
}

Parsing or Formatting?

• Formatting
• Parsing

To Reproduce
Here's the current coding that's triggering this error:

return new Promise((resolve): void => {
  stream
    .pipe(parseCsv())
    .on('data', async (row: string[]) => {
      buffer.push(row);
      if (buffer.length === this.bufferLimit) {
        const bufferToBeConsumed = buffer;
        buffer = [];
        await this.upsertCsvRowsByExportType(bufferToBeConsumed, filename);
      }
    })
    .once('end', async () => {
      if (buffer.length) {
        const bufferToBeConsumed = buffer;
        buffer = [];
        await this.upsertCsvRowsByExportType(bufferToBeConsumed, filename);
        resolve();
      }
    });
});

The lib isn't used in any other place.

Expected behavior
No error should occur during parsing process.

Screenshots

Desktop (please complete the following information):

• Lambda function
• Node Version 12.13
• Lib Version 4.3.1

Additional context
Add any other context about the problem here.

[olegario96]

Fixed after remove this lib from my dependencies and use csv-parse :)

[rhanka]

I have the same "Callback called multiple times" each time I try to have the fast-csv parse stream pipe into another one... is this a fast-csv (new?) bug ?

[doug-martin]

@rhanka would you mind testing out v4.3.0 to see if the bug exists in that version? I'm wondering if its related to this commit 84ecdf6

[rhanka]

The same occurs with v4.3.1, v4.3.0, v4.2.0 and v4.1.6

[doug-martin]

@rhanka thank you for checking. These types of issues are sometimes hard to reproduce, I'll investigate shortly.

[seeroush]

I've identified the problem on this line:

fast-csv/packages/parse/src/CsvParserStream.ts

Lines 79 to 86 in 23bf809

	try {
	const { lines } = this;
	const newLine = lines + this.decoder.write(data);
	const rows = this.parse(newLine, true);
	return this.processRows(rows, done);
	} catch (e) {
	return done(e);
	}

It occurs in both _flush() and _transform(). The processRows() -> callNext() function calls the done() callback, and then is caught by _flush() and _transform(), where the done() callback is invoked again. The error was reproduced by parsing a CSV file containing duplicate headers.

[seeroush]

It looks like this may only occur when running from certain versions of Node. The following block will result in the reported error in Node v8.15.0. It does not cause a problem when running under the currently configured v14.8.0.

const CSV_CONTENT = ['a,b,b,c', '1,2,2,3'].join(EOL,);
csv.parseString(CSV_CONTENT, {headers: true})

[chrisnorwood]

I've been receiving this same error while trying to parse a streamed csv file.

The basic code is just:

const someStreamFromS3 = ... ;
const myRecords = [];
csv.parseStream(someStreamFromS3).on("data", data => myRecords.push(data));

The error occurs regardless of whether I include an on("error", ...) callback or not.

Node Version: 10.18.1
@fast-csv/parse: 4.3.1

[doug-martin]

Just issue #464 to try to address this, instead of calling done again in the catch block it will now re throw the error.

I'll update once I publish

[doug-martin]

Just published v4.3.2, I'll leave this issue open for a couple of days in case it does not completely address it.

Thank you for the bug report!