Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(task): Create chainvis views and refresher #77

Merged
merged 2 commits into from
Oct 9, 2020

Conversation

placer14
Copy link
Contributor

@placer14 placer14 commented Oct 8, 2020

This PR serves two purposes:

  1. It is a proposal for supporting optional views within Visor. The idea here is that certain views are not always worth maintaining during normal processing. Within, we define a pattern for a materialized view which can be instantiated without data and optionally refreshed by passing in --chainvis-refresh-rate=1m with some value greater than 0 (which disables the refresh loop entirely).
  2. It is a proposal to support chain visualization views for queries associated with https://github.com/DigitalMOB2/filecoin-lotus-explorer-playground/. This view is a contract for data we promise to support for this external dependency. This will be a UI dependency which sits alongside Grafana as one of out first custom visualizations on top of Sentinel.

Overall, this pattern intends to add zero operational overhead and minimal developer overhead (specifically in maintaining the underlying tables supporting these views). Refresh is completely optional. Creation of materialized views without data initialized are virtually free to maintain in schema. Cost of refreshing views is managed by configurable refresh rate.

Potential areas of improvement:

  • capture metrics for view refresh duration
  • CI process to test migrations

@placer14 placer14 requested review from iand and frrist October 8, 2020 06:37
Copy link
Contributor

@iand iand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a minor nit

Nice use of the task interface.

return wait.RepeatUntil(ctx, r.refreshRate, r.refreshView)
}

func (r *ChainVisRefresher) refreshView(ctx context.Context) (bool, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit: would be cleaner to have these views in a slice that is looped over

Copy link
Member

@frrist frrist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@placer14 placer14 merged commit 0d67282 into master Oct 9, 2020
@placer14 placer14 deleted the feat/chainvis-view branch October 9, 2020 02:27
placer14 added a commit that referenced this pull request Oct 14, 2020
* origin/master:
  fix: remove errgrp from UnindexedBlockData persist
  fix: verify there are no missing migrations before migrating (#89)
  chore: add tests for reward and power actor state extracters (#83)
  feat: support v2 actor codes (#84)
  feat(debug): Process actor by head without persistance (#86)
  fix: use debugf logging method in message processor (#82)
  feat(task): Create chainvis views and refresher (#77)
  perf: minor optimization of market actor diffing (#78)
  chore: fail database tests if VISOR_TEST_DB not set (#79)
  test(storage): add test to check for duplicate schema migrations (#80)
  in-process lotus "repo" lens (#75)

 Conflicts:
        tasks/actorstate/miner.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants