Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Components should acknowledge that they have received the data #11673

Open
michalpristas opened this issue Aug 15, 2024 · 3 comments
Open

Components should acknowledge that they have received the data #11673

michalpristas opened this issue Aug 15, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@michalpristas
Copy link

Component(s)

No response

Is your feature request related to a problem? Please describe.

Components should acknowledge that they have received the data from the upstream and take over the responsibility to move them forward without blocking the upstream

Describe the solution you'd like

The benefit of blocking is that in many cases an error can be propagated back upstream which can be reacted to. For example in a simple otlp->otlp forwarding pipeline, the result can ultimately be returned from the otlp receiver to the client trying to send the data. Then the client has the ability to react as they see appropriate. However, this only works for synchronous components, and to my knowledge we could do a better job declaring components as sync vs async (again maybe in consumer.Capabilities or metadata.yaml). This would make it easier to validate expectations.
@djaglowski

Describe alternatives you've considered

No response

Additional context

No response

@djaglowski
Copy link
Member

I think synchronous components already do this.

My proposal is basically that we should add a sync vs async flag to consumer.Capabilities so that we can reason about and ultimately make observable the behavior of the downstream components.

Examples:

  • Ensure a collector is running synchronously end-to-end in order to guarantee that errors are back propagated from exporters to receivers.
  • Easily identify components which may be (1) added substantial latency to the data or (2) burying errors.

@mx-psi
Copy link
Member

mx-psi commented Nov 12, 2024

Should this be transferred to opentelemetry-collector?

@djaglowski
Copy link
Member

@mx-psi I agree it should be moved

@mx-psi mx-psi transferred this issue from open-telemetry/opentelemetry-collector-contrib Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants