fix(relationJoins): decimal values should not suffer precision loss #4771

Weakky · 2024-03-14T11:36:06Z

Overview

BigDecimal::from_f64 introduces precision loss even on the latest version.

Example:

let a = BigDecimal::from_f64(95993.57) // == BigDecimal("95993.57000000001");
let b = BigDecimal::from_str(&95993.57.to_string()) // == BigDecimal("95993.57");

dbg!(&a.as_bigint_and_exponent()) // (9599357000000001, 11)
dbg!(&b.as_bigint_and_exponent()) // (9599357, 2)

github-actions · 2024-03-14T11:40:16Z

WASM Size

Engine	This PR	Base branch	Diff
Postgres	2.068MiB	2.068MiB	-137.000B
Postgres (gzip)	816.101KiB	816.227KiB	-129.000B
Mysql	2.038MiB	2.038MiB	-137.000B
Mysql (gzip)	802.532KiB	802.605KiB	-75.000B
Sqlite	1.928MiB	1.929MiB	-260.000B
Sqlite (gzip)	762.057KiB	762.290KiB	-238.000B

codspeed-hq · 2024-03-14T11:44:56Z

CodSpeed Performance Report

Merging #4771 will not alter performance

_{Comparing fix/relation-join-decimal (5948261) with main (93f79ec)}

Summary

✅ 11 untouched benchmarks

github-actions · 2024-03-14T11:51:05Z

✅ WASM query-engine performance won't change substantially (1.000x)

Full benchmark report

DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.19.1 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p999
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - ~50K)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     289 ms/iter       (287 ms … 293 ms)    291 ms    293 ms    293 ms
Web Assembly: Latest       375 ms/iter       (374 ms … 376 ms)    375 ms    376 ms    376 ms
Web Assembly: Current      374 ms/iter       (372 ms … 375 ms)    375 ms    375 ms    375 ms
Node API: Current          200 ms/iter       (194 ms … 205 ms)    205 ms    205 ms    205 ms

summary for movies.findMany() (all - ~50K)
  Web Assembly: Current
   1.87x slower than Node API: Current
   1.29x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  12'118 µs/iter (11'411 µs … 15'730 µs) 11'969 µs 15'730 µs 15'730 µs
Web Assembly: Latest    15'121 µs/iter (14'746 µs … 17'365 µs) 15'104 µs 17'365 µs 17'365 µs
Web Assembly: Current   15'360 µs/iter (14'984 µs … 17'091 µs) 15'326 µs 17'091 µs 17'091 µs
Node API: Current        7'938 µs/iter   (7'760 µs … 8'141 µs)  8'008 µs  8'141 µs  8'141 µs

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   1.94x slower than Node API: Current
   1.27x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'879 µs/iter   (1'763 µs … 3'023 µs)  1'861 µs  2'820 µs  3'023 µs
Web Assembly: Latest     2'416 µs/iter   (2'316 µs … 4'095 µs)  2'397 µs  3'354 µs  4'095 µs
Web Assembly: Current    2'420 µs/iter   (2'324 µs … 3'205 µs)  2'412 µs  2'968 µs  3'205 µs
Node API: Current        1'424 µs/iter   (1'317 µs … 2'189 µs)  1'417 µs  1'950 µs  2'189 µs

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   1.7x slower than Node API: Current
   1.29x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     554 ms/iter       (547 ms … 575 ms)    556 ms    575 ms    575 ms
Web Assembly: Latest       736 ms/iter       (729 ms … 756 ms)    744 ms    756 ms    756 ms
Web Assembly: Current      736 ms/iter       (732 ms … 747 ms)    737 ms    747 ms    747 ms
Node API: Current          466 ms/iter       (455 ms … 475 ms)    475 ms    475 ms    475 ms

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.58x slower than Node API: Current
   1.33x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  76'129 µs/iter (75'885 µs … 76'596 µs) 76'500 µs 76'596 µs 76'596 µs
Web Assembly: Latest       103 ms/iter       (103 ms … 103 ms)    103 ms    103 ms    103 ms
Web Assembly: Current      103 ms/iter       (103 ms … 104 ms)    103 ms    104 ms    104 ms
Node API: Current       60'410 µs/iter (59'746 µs … 61'086 µs) 60'781 µs 61'086 µs 61'086 µs

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.71x slower than Node API: Current
   1.35x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     981 ms/iter     (973 ms … 1'003 ms)    981 ms  1'003 ms  1'003 ms
Web Assembly: Latest     1'244 ms/iter   (1'230 ms … 1'262 ms)  1'256 ms  1'262 ms  1'262 ms
Web Assembly: Current    1'222 ms/iter   (1'215 ms … 1'239 ms)  1'226 ms  1'239 ms  1'239 ms
Node API: Current          880 ms/iter       (860 ms … 903 ms)    890 ms    903 ms    903 ms

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.39x slower than Node API: Current
   1.25x slower than Web Assembly: Baseline
   1.02x faster than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     137 ms/iter       (136 ms … 138 ms)    137 ms    138 ms    138 ms
Web Assembly: Latest       171 ms/iter       (170 ms … 172 ms)    172 ms    172 ms    172 ms
Web Assembly: Current      170 ms/iter       (168 ms … 173 ms)    172 ms    173 ms    173 ms
Node API: Current          104 ms/iter       (102 ms … 106 ms)    104 ms    106 ms    106 ms

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.64x slower than Node API: Current
   1.24x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     871 µs/iter     (820 µs … 1'387 µs)    873 µs  1'256 µs  1'387 µs
Web Assembly: Latest     1'194 µs/iter   (1'132 µs … 1'856 µs)  1'197 µs  1'614 µs  1'856 µs
Web Assembly: Current    1'192 µs/iter   (1'133 µs … 1'755 µs)  1'197 µs  1'565 µs  1'755 µs
Node API: Current          786 µs/iter     (726 µs … 1'110 µs)    798 µs    929 µs  1'110 µs

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.52x slower than Node API: Current
   1.37x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     863 µs/iter     (819 µs … 1'457 µs)    867 µs  1'165 µs  1'457 µs
Web Assembly: Latest     1'193 µs/iter   (1'138 µs … 1'703 µs)  1'196 µs  1'550 µs  1'703 µs
Web Assembly: Current    1'211 µs/iter   (1'141 µs … 2'009 µs)  1'206 µs  1'756 µs  2'009 µs
Node API: Current          760 µs/iter     (706 µs … 1'065 µs)    781 µs    931 µs  1'065 µs

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.59x slower than Node API: Current
   1.4x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

After changes in 5948261

fix(relationJoins): decimal values should be normalized

3430fac

Weakky requested a review from a team as a code owner March 14, 2024 11:36

Weakky requested review from Druue and removed request for a team March 14, 2024 11:36

Weakky added 3 commits March 14, 2024 12:51

reuse utils

0203e33

fix sqlite conversion as well

ed34b52

fix driver-adadapters conversion too

5948261

Weakky changed the title ~~fix(relationJoins): decimal values should be normalized~~ fix(relationJoins): decimal values should not suffer precision loss Mar 14, 2024

Jolg42 requested review from SevInf and removed request for Druue March 14, 2024 13:04

Weakky added this to the 5.12.0 milestone Mar 14, 2024

SevInf approved these changes Mar 20, 2024

View reviewed changes

Weakky merged commit ff01c8c into main Mar 22, 2024
204 checks passed

Weakky deleted the fix/relation-join-decimal branch March 22, 2024 11:06

Weakky mentioned this pull request Jul 22, 2024

fix(qe): json and top-level f32 coercion should be in sync #4961

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(relationJoins): decimal values should not suffer precision loss #4771

fix(relationJoins): decimal values should not suffer precision loss #4771

Weakky commented Mar 14, 2024 •

edited

Loading

github-actions bot commented Mar 14, 2024 •

edited

Loading

codspeed-hq bot commented Mar 14, 2024 •

edited

Loading

github-actions bot commented Mar 14, 2024 •

edited

Loading

fix(relationJoins): decimal values should not suffer precision loss #4771

fix(relationJoins): decimal values should not suffer precision loss #4771

Conversation

Weakky commented Mar 14, 2024 • edited Loading

Overview

github-actions bot commented Mar 14, 2024 • edited Loading

WASM Size

codspeed-hq bot commented Mar 14, 2024 • edited Loading

CodSpeed Performance Report

Merging #4771 will not alter performance

Summary

github-actions bot commented Mar 14, 2024 • edited Loading

✅ WASM query-engine performance won't change substantially (1.000x)

Weakky commented Mar 14, 2024 •

edited

Loading

github-actions bot commented Mar 14, 2024 •

edited

Loading

codspeed-hq bot commented Mar 14, 2024 •

edited

Loading

github-actions bot commented Mar 14, 2024 •

edited

Loading