Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Lake] Create pdr_contracts table, and cleanup <asset, timeframe, source> columns from pdr_tables. Use <contract_id> to join. #616

Closed
7 tasks
idiom-bytes opened this issue Feb 14, 2024 · 0 comments
Labels

Comments

@idiom-bytes
Copy link
Member

Motivation

There is residual duplication of data as a result of initial subgraph->lake integration, and reducing the scope of e2e work (by working with fewer tables).

Columns like: <asset, timeframe, source> are duplicated across various tables as a result.

This metadata comes from the contract, and exists in subgraph-predictContracts

DoD

  • Fetch contract metadata from subgraph predictContracts
  • Store contract metadata in lake (this is small data ~40 contracts ATM)
  • Create contract_utils, that that make it easy to get the contract data from the lake, into memory.
  • Reconcile contract_utils with remainder of predictoor/backend/ getting contract metadata by fetching from the subgraph.
  • All tables should drop <asset, timeframe, source> and start using <contract_id> to join/lookup/filter based on this data.
  • All tables/queries, have been updated to use <contract_id> instead.
  • Dashboards & remaining code, may choose to use contract_utils to get this info from lake, rather than querying subgraph. Similarly to OHLCVDataFactory, we may want want to stop querying the subgraph directly, fill the lake, and get the answer from there.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant