Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance improvements. #378

Merged
merged 1 commit into from
Apr 3, 2019
Merged

Performance improvements. #378

merged 1 commit into from
Apr 3, 2019

Conversation

juanjoDiaz
Copy link
Collaborator

Some serious performance improvements.

Running Benchmak.js

  • on a Macbook Pro 2015 2,2 GHz and 15 GB
  • on Node.js 11.9.0
  • on a simple repetitive dataset (const sampleData = Array(100000).fill({ carModel: 'BMW', price: 15000, color: 'red', transmission: 'manual' });)
  • with all default options

I get this:

 > new json2csv.parse x 8.25 ops/sec ±4.51% (25 runs sampled)
 > new json2csv.parseAsync x 6.30 ops/sec ±3.41% (34 runs sampled)
 > local json2csv.Transform x 8.26 ops/sec ±2.86% (43 runs sampled)
 > current json2csv.parse x 2.09 ops/sec ±2.61% (10 runs sampled)
 > current json2csv.parseAsync x 2.05 ops/sec ±3.77% (15 runs sampled)
 > json2csv.Transform x 2.09 ops/sec ±2.17% (15 runs sampled)
 > json-2-csv x 3.41 ops/sec ±1.20% (21 runs sampled)
 > papaparse x 5.68 ops/sec ±6.21% (18 runs sampled)

Fastest is new json2csv.Transform

So json2csv gets 4 times faster and goes from the slowest to the fastest library around the block.
The results are similar passing the fields options (all of the above just get equally faster), and in safari and edge. However, in chrome json2csv is still not doing too well I need to investigate why.

I'm sure that there is more room for improvement and I still need to test with more advanced options. But this is a good first step.

As side effects:

  • We properly preserve unicode new line and tab (before they where converted to \n and \t)
  • Empty lines logic is correct (before it wouldn't work if someone used the fields options and has an object with fields but none of which match the given options).

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.2%) to 99.756% when pulling 260fe53 on juanjoDiaz:performance_improvements into d56c752 on zemirco:master.

@coveralls
Copy link

coveralls commented Apr 1, 2019

Coverage Status

Coverage remained the same at 100.0% when pulling 10cf3cb on juanjoDiaz:performance_improvements into d56c752 on zemirco:master.

@knownasilya knownasilya merged commit 1f91351 into zemirco:master Apr 3, 2019
@juanjoDiaz juanjoDiaz deleted the performance_improvements branch April 29, 2019 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants