[Error Count Discrepancy] Address discrepancy between error count and number of errors observed in Replay Details

[jas-kas]

Problem:

Today, there are many instances where the error count seen on the Replay Index as well as in the counter on the Replay Details (top-right bubble) is not reflective of the number of errors seen in the actual Replay and reflected in our breadcrumbs/console/issues list. This has been reported by our alpha partners.

This can result in a user seeing a Replay that has 2 errors on the Index page but once they click in, the Replay actually has no errors.

We know this can happen as a result of... Server-side inbound filters.

This may also be related to... Filtering error events on the browser SDK - docs - It is still under investigation if this is the case with the SDK team.

We need a solution to address this, as today it leads to customers not trusting our data and confusion.

Solution:

TBD.

cc @billyvg @ryan953

[billyvg]

Re: the filtering on browser SDK....

In the replays SDK we keep an aggregate count of errors by sending a list of error ids to the backend. We use addGlobalEventProcessor to listen for error events and 1) tag the replayId to the event and 2) grab the error id to later send to replays backend. It's possible that some of these errors could get filtered by the browser SDK (e.g. beforeSend? or a default integration?) and would make our counts inaccurate.

[jas-kas]

Example:

• https://sentry.io/organizations/sentry/replays/javascript:81dd27f10bc342d09a2443b52105508f/?f_c_logLevel=issue&f_c_logLevel=warning&referrer=%2Forganizations%2F%3AorgId%2Freplays%2F (10 in error count vs. 6 in replay)

[mydea]

See: getsentry/sentry-javascript#6285

[jas-kas]

[Messaging around this to the user explaining that there may be a discrepancy]

[mydea]

Closing this in favor of getsentry/sentry-javascript#6285! I'll add a note to the readme as well.