Automatically parse DB bigints as JavaScript Ints #83
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://trello.com/c/RIATlpAS In the [charging-module-api](https://github.com/defra/charging-module-api) we hit an issue caused by integer overflow. We hold all our monetary values as pennies rather than pounds and pence. This is common practise in financial systems as it avoids some issues with rounding in equations. The issue was all our monetary fields were set as `integer` but we found examples of bill runs with values larger than 2147483647 (the max an integer can hold). To solve the issue we needed to set the DB fields to `bigint`. [Bigint's](https://www.postgresql.org/docs/10/datatype-numeric.html#DATATYPE-INT) can hold values in the range of -9223372036854775808 to +9223372036854775807 in PostgreSQL. JavaScript on the otherhand uses [Number](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number) to manage integers. An integer in JavaScript has a range of -9007199254740991 to 9007199254740991. If you need to go as large as PostgreSQL you need to use a [JavaScript BigInt](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt). Because of this discrepency the PostgreSQL driver for Node by default returns all BigInt values as strings. This is a [design decision by PostgreSQL](brianc/node-postgres#353) to avoid any loss of precision. However, if you are confident you won't be effected by this precision loss (which we are) you can tell the PostgreSQL driver to return an integer instead. So as we do not expect to generate bill runs with values greater than 9.0071993e+13 (I don't even know how to say what that number is!) this change adds the config needed so we can forget about BigInt's in the rest of our code.
StuAA78
approved these changes
Dec 10, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://trello.com/c/RIATlpAS
In the charging-module-api we hit an issue caused by integer overflow. We hold all our monetary values as pennies rather than pounds and pence. This is common practise in financial systems as it avoids some issues with rounding in equations.
The issue was all our monetary fields were set as
integerbut we found examples of bill runs with values larger than 2147483647 (the max an integer can hold). To solve the issue we needed to set the DB fields tobigint. Bigint's can hold values in the range of -9223372036854775808 to +9223372036854775807 in PostgreSQL.JavaScript on the other hand uses Number to manage integers. An integer in JavaScript has a range of -9007199254740991 to 9007199254740991. If you need to go as large as PostgreSQL you need to use a JavaScript BigInt.
Because of this discrepancy, the PostgreSQL driver for Node by default returns all BigInt values as strings. This is a design decision by PostgreSQL to avoid any loss of precision.
However, if you are confident you won't be affected by this precision loss (which we are) you can tell the PostgreSQL driver to return an integer instead.
So as we do not expect to generate bill runs with values greater than 9.0071993e+13 (I don't even know how to say what that number is!) this change adds the config needed so we can forget about BigInt's in the rest of our code.