Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: guard concurrent accesses to node api #412

Merged
merged 1 commit into from
Mar 9, 2021

Conversation

iand
Copy link
Contributor

@iand iand commented Mar 9, 2021

Following investigation by @placer14 we suspect the panic in #411 is due to concurrent modifications of the node field in the actor state task. When originally written this code was sequential but recent changes have introduced concurrent accesses to the field, namely when the Close method is called. I speculate this can happen when another task fails while an actor state task is still spawning goroutines. The task failure is detected by the indexer which closes all lenses used by the other running tasks, setting the node field to nil which is then passed to a new goroutine by the still-running task.

This change guards accesses to the node field with mutexes in each of the tasks. The actor state task checks whether the api lens is available before processing and if it isn't returns a non-fatal error for each actor which will be logged in visor_processing_reports for that tipset+task+actor combination.

Also removed the node field from the messages task since it is not used.

Fixes #411

@iand iand requested a review from placer14 March 9, 2021 12:21
@placer14
Copy link
Contributor

placer14 commented Mar 9, 2021

Thanks for putting this together, Ian.

@placer14 placer14 merged commit de2eda0 into master Mar 9, 2021
@placer14 placer14 deleted the fix/avoid-panic-closed-lens branch March 9, 2021 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Panic in actorstate.NewPowerStateExtractionContext
3 participants