lotus v1.9.0

[jennijuju]

Lotus v1.9.0 Release

🗺 What's left for the release

Note: This section is still WIP and should be finalized on Apr 9th.

• Integrate proof v6.1.0 #5816
• Integrate proof v7.0.0 and new ffi #6129
• Update to go-fil-markets-v1.2.5 #5941
• Binaries for RCs (see [Feature Request] Offer pre-built binaries for Lotus #6030)
• Lotus Lite: Check backing node version on startup Lotus Lite: Check backing node version on startup #6146
• api doc

🚢 Estimated shipping date

• 2021-04-13 v1.7.0-rc1 1.9.0-rc1
• 2021-05-03 1.9.0-rc2
• 2021-05-06 1.9.0-rc3
• 2021-05-14 1.9.0-rc5
• 2021-04-27 2021-05-10 2021-05-17 v1.9.0

🔦 Highlights

Changelog

This is an optional Lotus release that introduces various improvements to the sealing, mining, and deal-making processes.

Highlights

• OpenRPC Support (OpenRPC Support #5843)
• Take latency into account when making interactive deals (Take latency into account when making interactive deals #5876)
• Update go-commp-utils for >10x faster client commp calculation (Update go-commp-utils for >10x faster client commp calculation #5892)
• add lotus client cancel-retrieval cmd to lotus CLI (add lotus client cancel-retrieval cmd to lotus CLI #5871)
• add inspect-deal command to lotus client (add inspect-deal command to lotus client #5833)
• Local retrieval support (Local retrieval support #5917)
• go-fil-markets v1.1.9 -> v1.2.5
  • For a detailed changelog see https://github.com/filecoin-project/go-fil-markets/blob/master/CHANGELOG.md
• rust-fil-proofs v5.4.1 -> v6.1.0
  • For a detailed changelog see https://github.com/filecoin-project/rust-fil-proofs/blob/master/CHANGELOG.md

Changes

• storagefsm: Apply global events even in broken states (storagefsm: Apply global events even in broken states #5962)
• Default the AlwaysKeepUnsealedCopy flag to true (Default the AlwaysKeepUnsealedCopy flag to true #5743)
• splitstore: compact hotstore prior to garbage collection (splitstore: compact hotstore prior to garbage collection #5778)
• ipfs-force bootstrapper update (ipfs-force bootstrapper update #5799)
• better logging when unsealing fails (better logging when unsealing fails #5851)
• perf: add cache for gas permium estimation (perf: add cache for gas permium estimation #5709)
• backupds: Compact log on restart (backupds: Compact log on restart #5875)
• backupds: Improve truncated log handling (backupds: Improve truncated log handling #5891)
• State CLI improvements (State CLI improvements)
• API proxy struct codegen (API proxy struct codegen #5854)
• move DI stuff for paychmgr into modules (move DI stuff for paychmgr into modules #5791)
• Implement Event observer and Settings for 3rd party dep injection (Implement Event observer and Settings for 3rd party dep injection #5693)
• Export developer and network commands for consumption by derivatives of Lotus (Export developer and network commands for consumption by derivatives of Lotus #5864)
• mock sealer: Simulate randomness sideeffects (mock sealer: Simulate randomness sideeffects #5805)
• localstorage: Demote reservation stat error to debug (localstorage: Demote reservation stat error to debug #5976)
• shed command to unpack miner info dumps (shed command to unpack miner info dumps #5800)
• Add two utils to Lotus-shed (Add two utils to Lotus-shed #5867)
• add shed election estimate command (add shed election estimate command  #5092)
• Add --actor flag in lotus-shed sectors terminate (Add --actor flag in lotus-shed sectors terminate #5819)
• Move lotus mpool clear to lotus-shed (chore: Move lotus mpool clear to lotus-shed #5900)
• Centralize everything on ipfs/go-log/v2 (Centralize everything on ipfs/go-log/v2 #5974)
• expose NextID from nice market actor interface (expose NextID from nice market actor interface #5850)
• add available options for perm on error (chore: add available options for perm on error #5814)
• API docs clarification: Document StateSearchMsg replaced message behavior (API docs clarification: Document StateSearchMsg replaced message behavior #5838)
• api: Document StateReplay replaced message behavior (api: Document StateReplay replaced message behavior #5840)
• add godocs to miner objects (add godocs to miner objects. #2184)
• Add description to the client deal CLI command (Add description to the client deal CLI command #5999)
• lint: don't skip builtin (lint: don't skip builtin #5881)
• use deal duration from actors (use deal duration from actors #5270)
• remote calc winningpost proof (remote calc winningpost proof #5884)
• packer: other network images (other network images #5930)
• Convert the chainstore lock to RW (Convert the chainstore lock to RW #5971)
• Remove CachedBlockstore (Remove CachedBlockstore #5972)
• remove messagepool CapGasFee duplicate code (remove messagepool CapGasFee duplicate code #5992)

Fixes

• return buffers after canceling badger operation (fix: return buffers after canceling badger operation #5796)
• avoid holding a lock while calling the View callback (fix: avoid holding a lock while calling the View callback #5792)
• storagefsm: Trigger input processing when below limits (storagefsm: Trigger input processing when below limits #5801)
• After importing a previously deleted key, be able to delete it again (After importing a previously deleted key, be able to delete it again. #4653)
• fix StateManager.Replay on reward actor (fix: StateManager.Replay on reward actor #5804)
• make sure atomic 64bit fields are 64bit aligned (fix: make sure atomic 64bit fields are 64bit aligned #5794)
• Import secp sigs in paych tests (Import secp sigs in paych tests #5879)
• fix ci build-macos (fix ci build-macos #5934)
• Fix creation of remainder account when it's not a multisig (Fix creation of remainder account when it's not a multisig #5807)
• Fix fallback chainstore (Fix fallback chainstore #6003)
• fix 4857: show help for set-addrs (fix 4857: show help for set-addrs #5943)
• fix health report (fix health report #6011)

✅ Release Checklist

First steps:

• Fork a new branch (release/vX.Y.Z) from master and make any further release related changes to this branch. If any "non-trivial" changes get added to the release, uncheck all the checkboxes and return to this stage.
• Prep the changelog using bin/mkreleaselog, and add it to CHANGELOG.md
• Bump the version in version.go in the master branch to vX.(Y+1).0-dev - Bump master version to 1.7.1-dev #6028

Prepping an RC:

• version string in build/version.go has been updated (in the release/vX.Y.Z branch).
• tag commit with vX.Y.Z-rcN
• cut a pre-release here

Testing an RC:

• Stage 0 - Automated Testing

  • Automated Testing
    • CI: Ensure that all tests are passing. * - there is one test that is flaky)
    • Testground tests

• Stage 1 - Internal Testing

  • Upgrade our testnet infra
    • 1 bootstrap node
    • 1 miner
    • Scratch nodes
    • Wait 24 hours, confirm nodes stay in sync
    • Remaining testnet infra
  • Upgrade our mainnet infra
  • Subset of development full archival nodes
  • Subset of bootstrappers (1 per region)
  • Confirm nodes stay in sync
  • Metrics report
    • Block validation time
    • Memory / CPU usage
    • Number of goroutines
    • IPLD block read latency
    • Bandwidth usage
  • If anything has worsened significantly, investigate + fix
  • Confirm the following work (some combination of Testground / Calibnet / Mainnet / beta users)
    • Seal a sector
    • make a deal
    • Submit a PoSt
    • (optional) let a sector go faulty, and see it be recovered

• Stage 2 - Community Dev Testing

  • Inform beta miners (@lotus-early-testers-miner in Filecoin Slack #fil-lotus)
  • Ask close ecosystem partners to test their projects (@lotus-early-testers-eco-dev in Filecoin slack #fil-lotus)
    • Powergate
    • Glif
    • Zondax
    • Stats dashboard
    • Community dashboards
    • Infura
    • Sentinel
    • Protofire
    • Fleek

• Stage 3 - Community Prod Testing

  • Documentation
    • Ensure that CHANGELOG.md is up to date
    • Check if any config updates are needed
  • Invite the wider community through (link to the release issue):
    • Create a lotus disucssion, example here
    • Link the disucssion in #fil-lotus on Filecoin slack

• Stage 4 - Release

  • Final preparation
    • Verify that version string in version.go has been updated.
    • Ensure that CHANGELOG.md is up to date
    • Ensure that README.md is up to date
    • Merge release-vX.Y.Z into the releases branch.
    • Tag this merge commit (on the releases branch) with vX.Y.Z.
    • Cut the release here.
    • Final announcements
      • Update network.filecoin.io
      • Add a comment when the final release is tagged, example here
      • repost in #fil-lotus in filecoin slack

• Post-Release

  • Merge the releases branch back into master, ignoring the changes to version.go (keep the -dev version from master).
  • Update RELEASE_ISSUE_TEMPLATE.md with any improvements determined from this latest release iteration.
  • Create an issue using RELEASE_ISSUE_TEMPLATE.md for the next release.

❤️ Contributors

< list generated by bin/mkreleaselog >

Would you like to contribute to Lotus and don't know how? Well, there are a few places you can get started:

• TODO

⁉️ Do you have questions?

Leave a comment here

[jennijuju]

@dirkmc IIUC, the new market release is not 100% backward compatible and we may want to avoid that if possible. Do we have plans to fix that? If so, can it be done by next Monday?

[dirkmc]

I will do a test as part of #5941 to check if the release is backwards compatible between 1.6.0 and 1.7.0

[f8-ptrk]

4 - ensure the docs are in sync

[vanvank]

can we upgrade using the latest master which is 1.7.0 for April 12th

[ProgramAPE]

The version of rust-fil-proofs will be changed to 1.6.0? Why not the 6.1.0+ series?

[BigLep]

Note that while this was previously listed as Lotus v1.7.0, we have redacted this to v$next until #5948 get addressed, which covers Lotus version naming and how it relates to Network versions.

[jennijuju]

modularized lotus-miner wouldnt get in this release

[jennijuju]

The team has decided on the version convention philosophy and will update in the discussion and have it in lotus readme soon. but as the result of that conversation, this release will be a feature release that has version v1.7.0

[jennijuju]

proof v7.0.0 appears to have issue with gpu as well and this is currently blocked by it

[jennijuju]

proof v7.0.0 appears to have issue with gpu as well and this is currently blocked by it

this is done now

[BigLep]

@jennijuju : FYI I updated the dates in "Estimated shipping date". Feel free to adjust further.

[BigLep]

@jennijuju : FYI I added checklist steps for going back and updating the release template based on lessons learned. This is in the template as well in #6236