You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As a product manager and user of the OKR Metrics Dashboard, I want the dashboard to convey the most accurate information so that I can be confident in what is being conveyed and make decisions off of it.
Description/Use Case
Currently the OKR Metrics Dashboard conveys the volume of results designated to be routed (based on the qty of test performed instances) to receivers of ReportStream. The Azure app event that contains this information is ITEM_ROUTED; however, this does not take into account actions to occur after this event is logged (e.g., filtering out conditions, filtering out pos/ negs, resends, items that get stuck in poison queue or last mile) – which ultimately leads to an inaccurate representation of the quantity of results that get sent to the receiver.
The request is to create or modify an event to contain the metadata necessary for the OKR Metrics Dashboard, but can more accurately capture the delivered/ sent status of the result to the receiver.
Additional context of the issue:
On Aug 16, 2024, the OKR Metrics Dashboard displayed that 18272 test performed observations were routed to IL PHD. Although this is accurate in terms of how many messages were passed through the "route" step, this is not closely accurate in conveying what was received by the receiver. What happened was that on "8/12/2024, it was 108 messages sent to IL without PID-3-1. [And engagement engineer] requested Simple Report to ask [them] to resubmit messages that missing PID-3-1. At 8/17/2023, it was 100 messages sent to IL without PID-3-1. I corrected those messages and resubmitted to their production folder and verified that they all there... [in addition, there were] Negative results. IL only wants Positive results to route to them. Therefore, those messages have been filtered out by conditionFilter." This means that using the ITEM_ROUTED is insufficient for capturing the volume of results received by the STLT, as it does not take into about receiver's filter settings and any resends.
Risks/Impacts/Considerations
Evaluate how this may impact current and future observability/ monitoring for the engagement team/ RS admins.
Ensure that the following metadata continues to be captured:
child reportID and/ or tracking ID
observation containing the tests performed (code system, code [typically LOINC], display name)
info that enables identification and filtering of AOEs (typically the LOINC display name (mentioned above) works
date/time
sender
receiver
topic
It would be nice to have all the metadata that is currently provided within the ITEM_ROUTED event
Dev Notes
Consider leveraging the last logged event in the pipeline (send)
Because this impacts how the OKR Metrics Dashboard will import data, please denote the exact time that changes in prod will go into effect. This will help with switching the data import settings.
Reports in send step may contain thousands of items so a single event wont be able to hold all the condition data. One option is to create a "ItemSentEvent" for each ITEM in the report.
Another option is the ItemSentEvent can contain the trackingId and SubmittedReportId which Jessica S can use to link to an event that contains the condition info
Acceptance Criteria
Create/ modify azure event, that captures the sent/ delivery state of the report/ result, to contain the necessary info for the OKR Metrics Dashboard
Notify @jsutantio exact time when changes go into effect in prod
The text was updated successfully, but these errors were encountered:
As a product manager and user of the OKR Metrics Dashboard, I want the dashboard to convey the most accurate information so that I can be confident in what is being conveyed and make decisions off of it.
Description/Use Case
Currently the OKR Metrics Dashboard conveys the volume of results designated to be routed (based on the qty of test performed instances) to receivers of ReportStream. The Azure app event that contains this information is ITEM_ROUTED; however, this does not take into account actions to occur after this event is logged (e.g., filtering out conditions, filtering out pos/ negs, resends, items that get stuck in poison queue or last mile) – which ultimately leads to an inaccurate representation of the quantity of results that get sent to the receiver.
The request is to create or modify an event to contain the metadata necessary for the OKR Metrics Dashboard, but can more accurately capture the delivered/ sent status of the result to the receiver.
Additional context of the issue:
On Aug 16, 2024, the OKR Metrics Dashboard displayed that 18272 test performed observations were routed to IL PHD. Although this is accurate in terms of how many messages were passed through the "route" step, this is not closely accurate in conveying what was received by the receiver. What happened was that on "8/12/2024, it was 108 messages sent to IL without PID-3-1. [And engagement engineer] requested Simple Report to ask [them] to resubmit messages that missing PID-3-1. At 8/17/2023, it was 100 messages sent to IL without PID-3-1. I corrected those messages and resubmitted to their production folder and verified that they all there... [in addition, there were] Negative results. IL only wants Positive results to route to them. Therefore, those messages have been filtered out by conditionFilter."
This means that using the ITEM_ROUTED is insufficient for capturing the volume of results received by the STLT, as it does not take into about receiver's filter settings and any resends.
Risks/Impacts/Considerations
Evaluate how this may impact current and future observability/ monitoring for the engagement team/ RS admins.
Ensure that the following metadata continues to be captured:
Dev Notes
Acceptance Criteria
The text was updated successfully, but these errors were encountered: