Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept transactions containing unfinished spans #1244

Closed
Tracked by #1690
bruno-garcia opened this issue Apr 27, 2022 · 2 comments
Closed
Tracked by #1690

Accept transactions containing unfinished spans #1244

bruno-garcia opened this issue Apr 27, 2022 · 2 comments
Assignees
Milestone

Comments

@bruno-garcia
Copy link
Member

https://getsentry.atlassian.net/browse/INGEST-1109

Currently, Relay will drop transactions that contain Spans that don't have an end timestamp.

The code in question:

match (span.start_timestamp.value(), span.timestamp.value()) {
(Some(start), Some(end)) => {
if end < start {
return Err(ProcessingAction::InvalidTransaction(
"end timestamp in span is smaller than start timestamp",
));
}
}
(_, None) => {
// XXX: Maybe do the same as event.timestamp?
return Err(ProcessingAction::InvalidTransaction(
"span is missing timestamp",
));
}
(None, _) => {
// XXX: Maybe copy timestamp over?
return Err(ProcessingAction::InvalidTransaction(
"span is missing start_timestamp",
));
}
}

match (span.start_timestamp.value(), span.timestamp.value()) {
(Some(start), Some(end)) => {
if end < start {
return Err(ProcessingAction::InvalidTransaction(
"end timestamp in span is smaller than start timestamp",
));
}
}
(_, None) => {
// XXX: Maybe do the same as event.timestamp?
return Err(ProcessingAction::InvalidTransaction(
"span is missing timestamp",
));
}
(None, _) => {
// XXX: Maybe copy timestamp over?
return Err(ProcessingAction::InvalidTransaction(
"span is missing start_timestamp",
));
}
}

In practice, it's common in SDKs that a transaction is finished while an asynchronous task didn't complete yet. This in turn results in confusion, particularly when SDKs are being developed as the engineer doesn't see any transactions showing up.

Different SDKs chose different approaches on how to handle it. But it's mainly:

We've identified the different implementations in the SDKs and are now leaning towards the latter with the reasoning that it's better to see what was going on during the transaction (could affect performance) even though it didn't complete. And the status code will help clearly indicate that this span didn't complete with the transaction.

Either way, handling this in the SDK adds to the complexity that now must iterate over all spans before capturing transactions to handle this.

With that, we agreed in a DACI that Relay should be changed to deal with transactions that contain unfinished spans:

Relay will no longer drop transactions that contain Spans without end timestamp

In this approach, the SDKs can remove the logic of handling unfinished spans. Relay now will be free to take the best approach to the problem:

Relay will add the transaction’s timestamp to the Span and the status deadline_exceeded, to indicate this span didn't complete during the transaction.

@jan-auer
Copy link
Member

Thanks, added to backlog and will implement this. In the future, please feel free to directly submit a PR :)

@jjbayer
Copy link
Member

jjbayer commented Dec 16, 2022

@bruno-garcia this has now been implemented and deployed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants