Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ingest/pipeline: update LedgerBackends to be InboundAdapters #5401

Closed
sreuland opened this issue Jul 26, 2024 · 1 comment
Closed

ingest/pipeline: update LedgerBackends to be InboundAdapters #5401

sreuland opened this issue Jul 26, 2024 · 1 comment

Comments

@sreuland
Copy link
Contributor

sreuland commented Jul 26, 2024

What problem does your feature solve

no clear path on how to use network tx meta as the source of origin in a data transformation pipeline using ingest/pipeline sdk per #5400

What would you like to see?

  • update ledgerbackend.LedgerBackend to become an InboundAdapter from the new ingest/pipeline sdk:
    *. LedgerBackend.Publish(ctx context.Context, outboundChannel Channel)
    • update the LedgerBackend implementations to implement Publish method, Allows the backend to be run as an async worker routine, until its ledger range is satisfied or forever if unbounded range. It will run a loop to obtain each LedgerCloseMeta internally via GetLedger(), and serialize and send it as Message to pipeline on outboundChannel, thus, provides the origin Event for application pipelines
      • ledgerbackend.BufferedStorageBackend
      • ledgerbackend.CaptiveStellarCore

Depends on #5400

What alternatives are there?

@sreuland
Copy link
Contributor Author

sreuland commented Aug 6, 2024

after processors lib design review, we decided to go with a different approach of pipelines created from observable streams using functional operators, this infra not needed and obsoleted by #5412.

@sreuland sreuland closed this as completed Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

1 participant