Skip to content

Commit

Permalink
decimal: eliminate heap allocations in big.Int arithmetic
Browse files Browse the repository at this point in the history
  • Loading branch information
@eapenkin
eapenkin authored Jan 1, 2024
2 parents 38b0a97 + 35c5ef2 commit 975ce54
Show file tree
Hide file tree
Showing 7 changed files with 264 additions and 131 deletions.
7 changes: 7 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Changelog

## [0.1.20] - 2024-01-01

### Changed

- Eliminated heap allocations in big.Int arithmetic.
- Improved documentation.

## [0.1.19] - 2023-12-18

### Changed
Expand Down
67 changes: 33 additions & 34 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,7 @@ This package is designed specifically for use in transactional financial systems
- **Simple String Representation** - Decimals are represented without the complexities
of scientific or engineering notation.
- **Correctness** - Fuzz testing is used to [cross-validate] arithmetic operations
against the [cockroachdb] and [shopspring] decimal packages.

against the [cockroachdb/apd] and [shopspring/decimal] packages.

## Getting Started

Expand Down Expand Up @@ -66,9 +65,9 @@ func main() {
fmt.Println(d.FMA(e, f)) // 8 * 12.5 + 2.567
fmt.Println(d.Pow(2)) // 8 ^ 2

fmt.Println(d.Quo(e)) // 8 / 12.5
fmt.Println(d.Quo(e)) // 8 ÷ 12.5
fmt.Println(d.QuoRem(e)) // 8 div 12.5, 8 mod 12.5
fmt.Println(d.Inv()) // 1 / 8
fmt.Println(d.Inv()) // 1 ÷ 8

// Rounding to 2 decimal places
fmt.Println(g.Round(2)) // 7.90
Expand Down Expand Up @@ -98,20 +97,20 @@ For examples related to financial calculations, see the `money` package

Comparison with other popular packages:

| Feature | govalues | [cockroachdb] v3.2.1 | [shopspring] v1.3.1 |
| ---------------- | ------------ | -------------------- | ------------------- |
| Speed | High | Medium | Low[^reason] |
| Mutability | Immutable | Mutable[^reason] | Immutable |
| Memory Footprint | Low | Medium | High |
| Panic Free | Yes | Yes | No[^divzero] |
| Precision | 19 digits | Arbitrary | Arbitrary |
| Default Rounding | Half to even | Half up | Half away from 0 |
| Context | Implicit | Explicit | Implicit |
| Feature | govalues | [cockroachdb/apd] v3.2.1 | [shopspring/decimal] v1.3.1 |
| ---------------- | ------------ | ------------------------ | --------------------------- |
| Speed | High | Medium | Low[^reason] |
| Mutability | Immutable | Mutable[^reason] | Immutable |
| Memory Footprint | Low | Medium | High |
| Panic Free | Yes | Yes | No[^divzero] |
| Precision | 19 digits | Arbitrary | Arbitrary |
| Default Rounding | Half to even | Half up | Half away from 0 |
| Context | Implicit | Explicit | Implicit |

[^reason]: decimal package was created simply because shopspring's decimal was
too slow and cockroachdb's decimal was mutable.
[^reason]: decimal package was created simply because [shopspring/decimal] was
too slow and [cockroachdb/apd] was mutable.

[^divzero]: [shopspring]'s decimal panics on division by zero.
[^divzero]: [shopspring/decimal] panics on division by zero.

### Benchmarks

Expand All @@ -122,22 +121,22 @@ pkg: github.com/govalues/decimal-tests
cpu: AMD Ryzen 7 3700C with Radeon Vega Mobile Gfx
```

| Test Case | Expression | govalues | [cockroachdb] v3.2.1 | [shopspring] v1.3.1 | govalues vs cockroachdb | govalues vs shopspring |
| ----------- | -------------------- | -------: | -------------------: | ------------------: | ----------------------: | ---------------------: |
| Add | 2 + 3 | 15.53n | 46.68n | 142.30n | +200.45% | +816.00% |
| Mul | 2 * 3 | 15.64n | 52.83n | 137.35n | +237.76% | +778.20% |
| QuoFinite | 2 / 4 | 51.65n | 179.60n | 619.40n | +247.76% | +1099.34% |
| QuoInfinite | 2 / 3 | 568.80n | 935.20n | 2749.00n | +64.43% | +383.30% |
| Pow | 1.1^60 | 1.28µ | 3.28µ | 16.03µ | +156.99% | +1156.09% |
| Pow | 1.01^600 | 4.31µ | 10.43µ | 37.00µ | +142.15% | +758.69% |
| Pow | 1.001^6000 | 7.54µ | 20.39µ | 651.51µ | +170.58% | +8544.78% |
| Parse | 1 | 17.14n | 77.64n | 129.15n | +353.00% | +653.50% |
| Parse | 123.456 | 36.15n | 201.85n | 235.25n | +458.37% | +550.76% |
| Parse | 123456789.1234567890 | 98.90n | 210.95n | 475.05n | +113.30% | +380.33% |
| String | 1 | 5.18n | 21.43n | 208.00n | +313.99% | +3918.16% |
| String | 123.456 | 42.31n | 67.55n | 226.55n | +59.66% | +435.52% |
| String | 123456789.1234567890 | 76.04n | 209.50n | 329.95n | +175.49% | +333.89% |
| Telco | see [specification] | 134.00n | 947.60n | 3945.50n | +607.13% | +2844.40% |
| Test Case | Expression | govalues | [cockroachdb/apd] v3.2.1 | [shopspring/decimal] v1.3.1 | govalues vs cockroachdb | govalues vs shopspring |
| ----------- | -------------------- | -------: | -----------------------: | --------------------------: | ----------------------: | ---------------------: |
| Add | 5 + 6 | 16.89n | 80.96n | 140.50n | +379.48% | +732.10% |
| Mul | 2 * 3 | 16.85n | 58.14n | 145.30n | +245.15% | +762.57% |
| QuoExact | 2 ÷ 4 | 66.00n | 193.25n | 619.15n | +192.78% | +838.03% |
| QuoInfinite | 2 ÷ 3 | 453.30n | 961.00n | 2767.00n | +112.01% | +510.41% |
| Pow | 1.1^60 | 1.04µ | 3.42µ | 15.76µ | +227.72% | +1408.43% |
| Pow | 1.01^600 | 3.57µ | 10.70µ | 35.70µ | +200.11% | +901.23% |
| Pow | 1.001^6000 | 6.19µ | 20.72µ | 634.41µ | +234.65% | +10148.95% |
| Parse | 1 | 18.10n | 85.66n | 136.75n | +373.23% | +655.52% |
| Parse | 123.456 | 54.16n | 197.25n | 238.45n | +264.20% | +340.27% |
| Parse | 123456789.1234567890 | 111.00n | 238.20n | 498.00n | +114.59% | +348.65% |
| String | 1 | 5.70n | 20.89n | 203.25n | +266.24% | +3464.23% |
| String | 123.456 | 42.74n | 75.71n | 235.65n | +77.14% | +451.36% |
| String | 123456789.1234567890 | 72.34n | 215.90n | 331.20n | +198.47% | +357.87% |
| Telco | see [specification] | 148.00n | 1075.00n | 4010.50n | +626.35% | +2609.80% |

The benchmark results shown in the table are provided for informational purposes only and may vary depending on your specific use case.

Expand Down Expand Up @@ -170,7 +169,7 @@ This ensures alignment with the project's objectives and roadmap.
[licenseb]: https://img.shields.io/github/license/govalues/decimal?color=blue
[awesome]: https://github.com/avelino/awesome-go#financial
[awesomeb]: https://awesome.re/mentioned-badge.svg
[cockroachdb]: https://pkg.go.dev/github.com/cockroachdb/apd
[shopspring]: https://pkg.go.dev/github.com/shopspring/decimal
[cockroachdb/apd]: https://pkg.go.dev/github.com/cockroachdb/apd
[shopspring/decimal]: https://pkg.go.dev/github.com/shopspring/decimal
[specification]: https://speleotrove.com/decimal/telcoSpec.html
[cross-validate]: https://github.com/govalues/decimal-tests/blob/main/decimal_fuzz_test.go
118 changes: 84 additions & 34 deletions coefficient.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package decimal

import (
"math/big"
"sync"
)

// fint (Fast INTeger) is a wrapper around uint64.
Expand Down Expand Up @@ -174,7 +175,7 @@ func (x fint) rshDown(shift int) fint {
}

// prec returns length of x in decimal digits.
// prec assumes that 0 has zero digits.
// prec assumes that 0 has no digits.
func (x fint) prec() int {
left, right := 0, len(pow10)
for left < right {
Expand All @@ -188,8 +189,9 @@ func (x fint) prec() int {
return left
}

// tzeroes returns number of trailing zeros in x.
func (x fint) tzeros() int {
// ntz returns number of trailing zeros in x.
// ntz assumes that 0 has no trailing zeros.
func (x fint) ntz() int {
left, right := 1, x.prec()
for left < right {
mid := (left + right) / 2
Expand All @@ -215,12 +217,6 @@ func (x fint) hasPrec(prec int) bool {
return x >= pow10[prec-1]
}

// bint converts uint64 to *big.Int.
func (x fint) bint() *bint {
z := new(big.Int).SetUint64(uint64(x))
return (*bint)(z)
}

// bint (Big INTeger) is a wrapper around big.Int.
type bint big.Int

Expand Down Expand Up @@ -328,10 +324,11 @@ var bpow10 = [...]*bint{
newBintFromPow10(99),
}

// newBintFromPow10 returns 10^exp as *big.Int.
func newBintFromPow10(exp int) *bint {
z := new(big.Int).Exp(big.NewInt(10), big.NewInt(int64(exp)), nil)
return (*bint)(z)
// newBintFromPow10 creates a *big.Int equal to 10^power.
func newBintFromPow10(power int) *bint {
z := (*bint)(new(big.Int))
z.pow10(power)
return z
}

func (z *bint) sign() int {
Expand All @@ -350,15 +347,19 @@ func (z *bint) setBint(x *bint) {
(*big.Int)(z).Set((*big.Int)(x))
}

func (z *bint) setInt64(x int64) {
(*big.Int)(z).SetInt64(x)
}

func (z *bint) setFint(x fint) {
(*big.Int)(z).SetUint64(uint64(x))
}

// fint converts *big.Int to uint64.
// If z cannot be represented as uint64, the result is undefined.
func (z *bint) fint() fint {
i := (*big.Int)(z).Uint64()
return fint(i)
f := (*big.Int)(z).Uint64()
return fint(f)
}

// add calculates z = x + y.
Expand Down Expand Up @@ -397,50 +398,78 @@ func (z *bint) mul(x, y *bint) {
(*big.Int)(z).Mul((*big.Int)(x), (*big.Int)(y))
}

// exp calculates z = x^y.
// If y is negative, the result is unpredictable.
func (z *bint) exp(x, y *bint) {
(*big.Int)(z).Exp((*big.Int)(x), (*big.Int)(y), nil)
}

// pow10 calculates z = 10^power.
// If power is negative, the result is unpredictable.
func (z *bint) pow10(power int) {
x := getBint()
defer putBint(x)
x.setInt64(10)
y := getBint()
defer putBint(y)
y.setInt64(int64(power))
z.exp(x, y)
}

// quo calculates z = x / y.
func (z *bint) quo(x, y *bint) {
(*big.Int)(z).Quo((*big.Int)(x), (*big.Int)(y))
r := getBint()
defer putBint(r)
// Passing r to prevent heap allocations.
z.quoRem(x, y, r)
}

// quoRem calculates z and r such that x = z * y + r.
func (z *bint) quoRem(x, y *bint) *bint {
_, r := (*big.Int)(z).QuoRem((*big.Int)(x), (*big.Int)(y), new(big.Int))
return (*bint)(r)
func (z *bint) quoRem(x, y, r *bint) {
(*big.Int)(z).QuoRem((*big.Int)(x), (*big.Int)(y), (*big.Int)(r))
}

func (z *bint) isOdd() bool {
return (*big.Int)(z).Bit(0) != 0
}

// lsh (Left Shift) calculates x * 10^shift.
// lsh (Left Shift) calculates z = x * 10^shift.
func (z *bint) lsh(x *bint, shift int) {
var y *bint
if shift < len(bpow10) {
y = bpow10[shift]
} else {
y = newBintFromPow10(shift)
y = getBint()
defer putBint(y)
y.pow10(shift)
}
z.mul(x, y)
}

// fsa (Fused Shift and Addition) calculates x * 10^shift + y.
func (z *bint) fsa(shift int, y byte) {
z.lsh(z, shift)
z.add(z, fint(y).bint())
// fsa (Fused Shift and Addition) calculates z = x * 10^shift + f.
func (z *bint) fsa(x *bint, shift int, f fint) {
y := getBint()
defer putBint(y)
y.setFint(f)
z.lsh(x, shift)
z.add(z, y)
}

// rshDown (Right Shift) calculates x / 10^shift and rounds result towards zero.
// rshDown (Right Shift) calculates z = x / 10^shift and rounds
// result towards zero.
func (z *bint) rshDown(x *bint, shift int) {
var y *bint
if shift < len(bpow10) {
y = bpow10[shift]
} else {
y = newBintFromPow10(shift)
y = getBint()
defer putBint(y)
y.pow10(shift)
}
z.quo(x, y)
}

// rshHalfEven (Right Shift) calculates x / 10^shift and
// rshHalfEven (Right Shift) calculates z = x / 10^shift and
// rounds result using "half to even" rule.
func (z *bint) rshHalfEven(x *bint, shift int) {
// Special cases
Expand All @@ -453,13 +482,17 @@ func (z *bint) rshHalfEven(x *bint, shift int) {
return
}
// General case
var y *bint
var y, r *bint
if shift < len(bpow10) {
y = bpow10[shift]
} else {
y = newBintFromPow10(shift)
y = getBint()
defer putBint(y)
y.pow10(shift)
}
r := z.quoRem(x, y)
r = getBint()
defer putBint(r)
z.quoRem(x, y, r)
r.dbl(r) // r = r * 2
switch y.cmp(r) {
case -1:
Expand All @@ -473,8 +506,8 @@ func (z *bint) rshHalfEven(x *bint, shift int) {
}

// prec returns length of *big.Int in decimal digits.
// It considers 0 to have zero digits.
// If *big.Int is negative, the result is unpredictable.
// prec assumes that 0 has no digits.
// If z is negative, the result is unpredictable.
//
// z.prec() provides a more efficient approach than len(z.string())
// when dealing with decimals having less than len(bpow10) digits.
Expand All @@ -497,7 +530,7 @@ func (z *bint) prec() int {
}

// hasPrec checks if *big.Int has a given number of digits or more.
// It considers 0 to have zero digits.
// hasPrec assumes that 0 has no digits.
// If *big.Int is negative, the result is unpredictable.
//
// z.hasPrec() provides a more efficient approach than (z.prec() >= prec)
Expand All @@ -513,3 +546,20 @@ func (z *bint) hasPrec(prec int) bool {
// General case
return z.cmp(bpow10[prec-1]) >= 0
}

// pool is a cache of reusable *big.Int instances.
var pool = sync.Pool{
New: func() any {
return (*bint)(new(big.Int))
},
}

// getBint obtains a *big.Int from the pool.
func getBint() *bint {
return pool.Get().(*bint)
}

// putBint returns the *big.Int into the pool.
func putBint(b *bint) {
pool.Put(b)
}
6 changes: 3 additions & 3 deletions coefficient_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -366,7 +366,7 @@ func TestFint_prec(t *testing.T) {
}
}

func TestFint_tzeros(t *testing.T) {
func TestFint_ntz(t *testing.T) {
cases := []struct {
x fint
want int
Expand Down Expand Up @@ -415,9 +415,9 @@ func TestFint_tzeros(t *testing.T) {
{math.MaxUint64, 0},
}
for _, tt := range cases {
got := tt.x.tzeros()
got := tt.x.ntz()
if got != tt.want {
t.Errorf("%v.tzeros() = %v, want %v", tt.x, got, tt.want)
t.Errorf("%v.ntz() = %v, want %v", tt.x, got, tt.want)
}
}
}
Expand Down
Loading

0 comments on commit 975ce54

Please sign in to comment.