Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

State diff results in parent actor not found #273

Closed
iand opened this issue Nov 27, 2020 · 3 comments · Fixed by #449
Closed

State diff results in parent actor not found #273

iand opened this issue Nov 27, 2020 · 3 comments · Fixed by #449
Assignees
Labels
kind/bug Kind: Bug P1 P1: Must be resolved

Comments

@iand
Copy link
Contributor

iand commented Nov 27, 2020

Describe the bug:

When processing actor states we get a list of actor state changes using state.Diff(parentStateTree, stateTree). We iterate the list of changes and process each one to extract and persist the state. The following actor consistently fails:

Height: 200021 
State root: bafy2bzaced6y7fdn4sdsrv6tsn3klwg33jybrihgvw7g26iwwp5bnedmj46dy
"Code": "bafkqaetgnfwc6mrpon2g64tbm5sw22lomvza", 
"Head": "bafy2bzacecahsf5ews4k4rjebw5inskbzjcvjw4v4pe2yozskok5jpvfgo6uq", 
"Name": "fil/2/storageminer", 
"Error": "failed to extract parsed actor state: creating miner state extraction context: loading previous miner f062897 at tipset {bafy2bzaceaqqqoxx34olk7tuyev6jz32m2eozjed2lk7brajgxknmppcsbmvq} epoch 200021: actor not found", 
"Address": "f062897"

Steps to Reproduce:

Visor Version

Found using this work in progress commit 95d218b

@iand iand added the kind/bug Kind: Bug label Nov 27, 2020
@iand
Copy link
Contributor Author

iand commented Dec 2, 2020

This is possibly a freshly created miner that doesn't have a previous state.

Another instance

{"height": 287133, "actor": "fil/2/storageminer", "address": "f094124", "error": "failed to extract parsed actor state: creating miner state extraction context: loading previous miner f094124 at tipset {bafy2bzacedodnmolpmswax5ooig4mbtv3sipgyeuferowf6gu2m2na5ceeosm,bafy2bzacedj5zswymdjwvnfc2an7pfaf5gim733krr57zevdwy5omkhvrox4o,bafy2bzacedpddii5bvf3d62omgpoeyxjyicz5zaspywi5d5m2zhhcvbnxnrss,bafy2bzacebq4zuxxkwsuf4ew43feulhidivxiqx2wkvx7eonfh2feepog5osq,bafy2bzaceadufcsoolc2ibqdrqy66c4yrw3t2x2zcyx5fybvevjvegacezbq4,bafy2bzacean2xj5nhtrt5cljqqwwpc5oxkmujixklwluxwpzrhw5hz7stghpg,bafy2bzacedcs3odk5bu4tjr5nwi3ns6y2sbjzws6w7s3nqyxyqovyua4hydom} epoch 287133: actor not found"}

@placer14 placer14 added the P1 P1: Must be resolved label Dec 4, 2020
@frrist frrist self-assigned this Jan 7, 2021
@frrist
Copy link
Member

frrist commented Mar 30, 2021

To debug I picked account actor f0466359 from the visor_processing_report table as it has a similar error:

[{"Code": "bafkqadlgnfwc6mzpmfrwg33vnz2a", "Head": "bafy2bzacebgdwerb5ucln5nep7iwvbhlyy5fpxrzlq3pqurkhrmhl7qns3fci", "Name": "fil/3/account", "Error": "failed to extract parsed actor state: getting actor: actor not found", "Address": "f0466359"}]

(the command ./visor --lens="lotusrepo" --repo="~/.lotus" walk --from=624269 --to=624269 --tasks=actorstatesraw can be used to reproduce this)

Based on debugging so far, these errors are being returned from here:
https://github.com/filecoin-project/lotus/blob/1f62c07789535980e848fd805c8165dafd04a5c7/chain/state/statetree.go#L337

Will continue to update as I debug.

@frrist
Copy link
Member

frrist commented Mar 31, 2021

frrist added a commit that referenced this issue Apr 14, 2021
frrist added a commit that referenced this issue Apr 16, 2021
* fix(tasks): use corret actor info for actor state extraction
- fixes #273

* fix(lens): remove OptimizedStateGetActorWithFallback
- fixes #405

* fix(market-task): fix market actor task and gen vector
- fixes #465

* polish(vector): support msapproval model validation

* test(vector): generate new test vectors

* ci: increase instance size due to test vectors
iand added a commit that referenced this issue Apr 20, 2021
* fix: ignore false positive gosec failure in wait package (#421)

* fix: embed genesis block in executable (#420)

* feat: add benchmarking of vectors (#423)

* feat: add benchmarking of vectors

* feat: add benchmarking of vectors

* Use nicer test and benchmark names

* Chore/sqlotus dag prefetch option (#430)

* Disable preloader by default, toggle with LOTUS_CHAINSTORE_PRELOAD_RECENTS

* Switch to a cli string as suggested by @iand

* Remove one more inefficiency for the R/O lens

filecoin-project/go-bs-postgres-chainnotated@b8b5d500e368d2ed

* Forgotten option/helptext spec

* Add support for static Get() prefetch

* close TipSetObs in walker and watcher

* fix(indexer): only close TIpSetIndexer in walk and watch

- subsequent calls to close will hang forever

* chore: increase linter timout in ci

* chore: update for Lotus 1.5.3 (#439)

* chore: update for Lotus 1.5.3

* fixup! chore: update for Lotus 1.5.3

* Update statediff version

Co-authored-by: frrist <[email protected]>

* Release notes for v0.5.5 (#440)

* chore: update lotus to 1.6.0 (#443)

* chore: upgrade lotus dependency to 1.6.0-rc1 (#447)

* chore: update lotus to 1.6.0 (#455)

* Release notes for v0.5.6 (#456)

* fix: ensure persistence semaphore channel is drained on close (#457)

* fix: ensure persistence semaphore channel is drained on close

* Add hidden window flag for testing

* Reduce actor happy path logging

* Make sure actor errors reported are appended to correct list

* Release notes for v0.5.7 (#458)

* fix(tasks): use correct ActorInfo for actor state extraction (#449)

* fix(tasks): use corret actor info for actor state extraction
- fixes #273

* fix(lens): remove OptimizedStateGetActorWithFallback
- fixes #405

* fix(market-task): fix market actor task and gen vector
- fixes #465

* polish(vector): support msapproval model validation

* test(vector): generate new test vectors

* ci: increase instance size due to test vectors

* Add MinerSectorInfo for extended sectors (#448)

Fixes #441. Extended sectors have new deadlines but this is not recorded anywhere.

* Fix makefile conflicts

Co-authored-by: Peter Rabbitson <[email protected]>
Co-authored-by: frrist <[email protected]>
Co-authored-by: Hector Sanjuan <[email protected]>
iand pushed a commit that referenced this issue Apr 20, 2021
…ply and revert events (#408)

chore: reuse watcher in lily watch command (#422)

Merge master into lily (#425)

* fix: ignore false positive gosec failure in wait package (#421)

* fix: embed genesis block in executable (#420)

* feat: add benchmarking of vectors (#423)

* feat: add benchmarking of vectors

* feat: add benchmarking of vectors

* Use nicer test and benchmark names

fix(linter): make deps before linting in CI (#437)

merge master into lily (#432)

* fix(indexer): only close TIpSetIndexer in walk and watch
* chore: increase linter timeout in ci

scheduler job control + watch, walk, and job cli (#414)

merge master into lily (#442)

* fix: ignore false positive gosec failure in wait package (#421)

* fix: embed genesis block in executable (#420)

* feat: add benchmarking of vectors (#423)

* feat: add benchmarking of vectors

* feat: add benchmarking of vectors

* Use nicer test and benchmark names

* Chore/sqlotus dag prefetch option (#430)

* Disable preloader by default, toggle with LOTUS_CHAINSTORE_PRELOAD_RECENTS

* Switch to a cli string as suggested by @iand

* Remove one more inefficiency for the R/O lens

filecoin-project/go-bs-postgres-chainnotated@b8b5d500e368d2ed

* Forgotten option/helptext spec

* Add support for static Get() prefetch

* close TipSetObs in walker and watcher

* fix(indexer): only close TIpSetIndexer in walk and watch

- subsequent calls to close will hang forever

* chore: increase linter timout in ci

* chore: update for Lotus 1.5.3 (#439)

* chore: update for Lotus 1.5.3

* fixup! chore: update for Lotus 1.5.3

* Update statediff version

Co-authored-by: frrist <[email protected]>

Co-authored-by: Ian Davis <[email protected]>
Co-authored-by: Peter Rabbitson <[email protected]>

chore: remove confidence argument from lily walk command (#444)

chore: reorganise visor commands (#445)

feat: add storage configuration to daemon (#446)

* feat: add storage configuration to daemon

* Start wiring up storage to config

* Finish wiring up storage config

* check error return when ensuring config

* Close config file on error

feat: Add --api parameter to daemon client commands (#450)

* feat: Add --api parameter to daemon client commands

* Make api default value a valid multiaddr

* Add api token parameter

Merge master into lily (#463)

* fix: ignore false positive gosec failure in wait package (#421)

* fix: embed genesis block in executable (#420)

* feat: add benchmarking of vectors (#423)

* feat: add benchmarking of vectors

* feat: add benchmarking of vectors

* Use nicer test and benchmark names

* Chore/sqlotus dag prefetch option (#430)

* Disable preloader by default, toggle with LOTUS_CHAINSTORE_PRELOAD_RECENTS

* Switch to a cli string as suggested by @iand

* Remove one more inefficiency for the R/O lens

filecoin-project/go-bs-postgres-chainnotated@b8b5d500e368d2ed

* Forgotten option/helptext spec

* Add support for static Get() prefetch

* close TipSetObs in walker and watcher

* fix(indexer): only close TIpSetIndexer in walk and watch

- subsequent calls to close will hang forever

* chore: increase linter timout in ci

* chore: update for Lotus 1.5.3 (#439)

* chore: update for Lotus 1.5.3

* fixup! chore: update for Lotus 1.5.3

* Update statediff version

Co-authored-by: frrist <[email protected]>

* Release notes for v0.5.5 (#440)

* chore: update lotus to 1.6.0 (#443)

* chore: upgrade lotus dependency to 1.6.0-rc1 (#447)

* chore: update lotus to 1.6.0 (#455)

* Release notes for v0.5.6 (#456)

* fix: ensure persistence semaphore channel is drained on close (#457)

* fix: ensure persistence semaphore channel is drained on close

* Add hidden window flag for testing

* Reduce actor happy path logging

* Make sure actor errors reported are appended to correct list

* Release notes for v0.5.7 (#458)

* Update to latest lotus commit

* Add missing support/tools/go.sum

* Fix merge bugs in makefile

* Fix merge bugs in makefile

Co-authored-by: Peter Rabbitson <[email protected]>
Co-authored-by: frrist <[email protected]>

feat: clean up command line flags for single-run jobs (#464)

* feat: clean up command line flags for single-run jobs
* Fix dockerfile

Co-authored-by: frrist <[email protected]>

polish(lily): add job name flag (#467)

- closes #452

polish(job): print job type, params, and errors in list (#468)

- closes #451

fix: ensure event channel is initialised before watch starts (#471)

chore: merge master into lily branch (#472)

* fix: ignore false positive gosec failure in wait package (#421)

* fix: embed genesis block in executable (#420)

* feat: add benchmarking of vectors (#423)

* feat: add benchmarking of vectors

* feat: add benchmarking of vectors

* Use nicer test and benchmark names

* Chore/sqlotus dag prefetch option (#430)

* Disable preloader by default, toggle with LOTUS_CHAINSTORE_PRELOAD_RECENTS

* Switch to a cli string as suggested by @iand

* Remove one more inefficiency for the R/O lens

filecoin-project/go-bs-postgres-chainnotated@b8b5d500e368d2ed

* Forgotten option/helptext spec

* Add support for static Get() prefetch

* close TipSetObs in walker and watcher

* fix(indexer): only close TIpSetIndexer in walk and watch

- subsequent calls to close will hang forever

* chore: increase linter timout in ci

* chore: update for Lotus 1.5.3 (#439)

* chore: update for Lotus 1.5.3

* fixup! chore: update for Lotus 1.5.3

* Update statediff version

Co-authored-by: frrist <[email protected]>

* Release notes for v0.5.5 (#440)

* chore: update lotus to 1.6.0 (#443)

* chore: upgrade lotus dependency to 1.6.0-rc1 (#447)

* chore: update lotus to 1.6.0 (#455)

* Release notes for v0.5.6 (#456)

* fix: ensure persistence semaphore channel is drained on close (#457)

* fix: ensure persistence semaphore channel is drained on close

* Add hidden window flag for testing

* Reduce actor happy path logging

* Make sure actor errors reported are appended to correct list

* Release notes for v0.5.7 (#458)

* fix(tasks): use correct ActorInfo for actor state extraction (#449)

* fix(tasks): use corret actor info for actor state extraction
- fixes #273

* fix(lens): remove OptimizedStateGetActorWithFallback
- fixes #405

* fix(market-task): fix market actor task and gen vector
- fixes #465

* polish(vector): support msapproval model validation

* test(vector): generate new test vectors

* ci: increase instance size due to test vectors

* Add MinerSectorInfo for extended sectors (#448)

Fixes #441. Extended sectors have new deadlines but this is not recorded anywhere.

* Fix makefile conflicts

Co-authored-by: Peter Rabbitson <[email protected]>
Co-authored-by: frrist <[email protected]>
Co-authored-by: Hector Sanjuan <[email protected]>
iand added a commit that referenced this issue Apr 20, 2021
* Create a Lily node by embedding lotus into visor to observe tipset apply and revert events (#408)

* chore: reuse watcher in lily watch command (#422)

* Merge master into lily (#425)

* fix: ignore false positive gosec failure in wait package (#421)

* fix: embed genesis block in executable (#420)

* feat: add benchmarking of vectors (#423)

* feat: add benchmarking of vectors

* feat: add benchmarking of vectors

* Use nicer test and benchmark names

* fix(linter): make deps before linting in CI (#437)

* merge master into lily (#432)

* fix(indexer): only close TIpSetIndexer in walk and watch
* chore: increase linter timeout in ci

* scheduler job control + watch, walk, and job cli (#414)

* merge master into lily (#442)

* fix: ignore false positive gosec failure in wait package (#421)

* fix: embed genesis block in executable (#420)

* feat: add benchmarking of vectors (#423)

* feat: add benchmarking of vectors

* feat: add benchmarking of vectors

* Use nicer test and benchmark names

* Chore/sqlotus dag prefetch option (#430)

* Disable preloader by default, toggle with LOTUS_CHAINSTORE_PRELOAD_RECENTS

* Switch to a cli string as suggested by @iand

* Remove one more inefficiency for the R/O lens

filecoin-project/go-bs-postgres-chainnotated@b8b5d500e368d2ed

* Forgotten option/helptext spec

* Add support for static Get() prefetch

* close TipSetObs in walker and watcher

* fix(indexer): only close TIpSetIndexer in walk and watch

- subsequent calls to close will hang forever

* chore: increase linter timout in ci

* chore: update for Lotus 1.5.3 (#439)

* chore: update for Lotus 1.5.3

* fixup! chore: update for Lotus 1.5.3

* Update statediff version

Co-authored-by: frrist <[email protected]>

Co-authored-by: Ian Davis <[email protected]>
Co-authored-by: Peter Rabbitson <[email protected]>

* chore: remove confidence argument from lily walk command (#444)

* chore: reorganise visor commands (#445)

* feat: add storage configuration to daemon (#446)

* feat: add storage configuration to daemon

* Start wiring up storage to config

* Finish wiring up storage config

* check error return when ensuring config

* Close config file on error

* feat: Add --api parameter to daemon client commands (#450)

* feat: Add --api parameter to daemon client commands

* Make api default value a valid multiaddr

* Add api token parameter

* Merge master into lily (#463)

* fix: ignore false positive gosec failure in wait package (#421)

* fix: embed genesis block in executable (#420)

* feat: add benchmarking of vectors (#423)

* feat: add benchmarking of vectors

* feat: add benchmarking of vectors

* Use nicer test and benchmark names

* Chore/sqlotus dag prefetch option (#430)

* Disable preloader by default, toggle with LOTUS_CHAINSTORE_PRELOAD_RECENTS

* Switch to a cli string as suggested by @iand

* Remove one more inefficiency for the R/O lens

filecoin-project/go-bs-postgres-chainnotated@b8b5d500e368d2ed

* Forgotten option/helptext spec

* Add support for static Get() prefetch

* close TipSetObs in walker and watcher

* fix(indexer): only close TIpSetIndexer in walk and watch

- subsequent calls to close will hang forever

* chore: increase linter timout in ci

* chore: update for Lotus 1.5.3 (#439)

* chore: update for Lotus 1.5.3

* fixup! chore: update for Lotus 1.5.3

* Update statediff version

Co-authored-by: frrist <[email protected]>

* Release notes for v0.5.5 (#440)

* chore: update lotus to 1.6.0 (#443)

* chore: upgrade lotus dependency to 1.6.0-rc1 (#447)

* chore: update lotus to 1.6.0 (#455)

* Release notes for v0.5.6 (#456)

* fix: ensure persistence semaphore channel is drained on close (#457)

* fix: ensure persistence semaphore channel is drained on close

* Add hidden window flag for testing

* Reduce actor happy path logging

* Make sure actor errors reported are appended to correct list

* Release notes for v0.5.7 (#458)

* Update to latest lotus commit

* Add missing support/tools/go.sum

* Fix merge bugs in makefile

* Fix merge bugs in makefile

Co-authored-by: Peter Rabbitson <[email protected]>
Co-authored-by: frrist <[email protected]>

* feat: clean up command line flags for single-run jobs (#464)

* feat: clean up command line flags for single-run jobs
* Fix dockerfile

Co-authored-by: frrist <[email protected]>

* polish(lily): add job name flag (#467)

- closes #452

* polish(job): print job type, params, and errors in list (#468)

- closes #451

* fix: ensure event channel is initialised before watch starts (#471)

* chore: merge master into lily branch (#472)

* fix: ignore false positive gosec failure in wait package (#421)

* fix: embed genesis block in executable (#420)

* feat: add benchmarking of vectors (#423)

* feat: add benchmarking of vectors

* feat: add benchmarking of vectors

* Use nicer test and benchmark names

* Chore/sqlotus dag prefetch option (#430)

* Disable preloader by default, toggle with LOTUS_CHAINSTORE_PRELOAD_RECENTS

* Switch to a cli string as suggested by @iand

* Remove one more inefficiency for the R/O lens

filecoin-project/go-bs-postgres-chainnotated@b8b5d500e368d2ed

* Forgotten option/helptext spec

* Add support for static Get() prefetch

* close TipSetObs in walker and watcher

* fix(indexer): only close TIpSetIndexer in walk and watch

- subsequent calls to close will hang forever

* chore: increase linter timout in ci

* chore: update for Lotus 1.5.3 (#439)

* chore: update for Lotus 1.5.3

* fixup! chore: update for Lotus 1.5.3

* Update statediff version

Co-authored-by: frrist <[email protected]>

* Release notes for v0.5.5 (#440)

* chore: update lotus to 1.6.0 (#443)

* chore: upgrade lotus dependency to 1.6.0-rc1 (#447)

* chore: update lotus to 1.6.0 (#455)

* Release notes for v0.5.6 (#456)

* fix: ensure persistence semaphore channel is drained on close (#457)

* fix: ensure persistence semaphore channel is drained on close

* Add hidden window flag for testing

* Reduce actor happy path logging

* Make sure actor errors reported are appended to correct list

* Release notes for v0.5.7 (#458)

* fix(tasks): use correct ActorInfo for actor state extraction (#449)

* fix(tasks): use corret actor info for actor state extraction
- fixes #273

* fix(lens): remove OptimizedStateGetActorWithFallback
- fixes #405

* fix(market-task): fix market actor task and gen vector
- fixes #465

* polish(vector): support msapproval model validation

* test(vector): generate new test vectors

* ci: increase instance size due to test vectors

* Add MinerSectorInfo for extended sectors (#448)

Fixes #441. Extended sectors have new deadlines but this is not recorded anywhere.

* Fix makefile conflicts

Co-authored-by: Peter Rabbitson <[email protected]>
Co-authored-by: frrist <[email protected]>
Co-authored-by: Hector Sanjuan <[email protected]>

Co-authored-by: Ian Davis <[email protected]>
Co-authored-by: Peter Rabbitson <[email protected]>
Co-authored-by: Hector Sanjuan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Kind: Bug P1 P1: Must be resolved
Projects
None yet
3 participants