Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

15766: item sent event #15920

Merged
merged 5 commits into from
Sep 25, 2024
Merged

Conversation

mkalish
Copy link
Collaborator

@mkalish mkalish commented Sep 18, 2024

This PR adds a new ITEM_SENT azure event

Test Steps:

  1. Send a report with more than one item through the pipeline, the following example should work with the default orgs
curl --location 'http://localhost:7071/api/reports' \
--header 'client: ignore.ignore-full-elr ' \
--header 'Content-Type: application/hl7-v2' \
--header 'x-functions-key: 09xifjf....444s' \
--header 'Authorization: Bearer dummy' \
--data-raw 'MSH|^~\&|CDC PRIME - Atlanta, Georgia (Dekalb)^2.16.840.1.114222.4.1.237821^ISO|Avante at Ormond Beach^10D0876999^CLIA|PRIME_DOH|Prime ReportStream|20210210170737||ORU^R01^ORU_R01|371784|P|2.5.1|||NE|NE|USA||||PHLabReportNoAck^ELR_Receiver^2.16.840.1.113883.9.99^ISO
SFT|Centers for Disease Control and Prevention|0.1-SNAPSHOT|PRIME ReportStream|0.1-SNAPSHOT||20210210
PID|1||2a14112c-ece1-4f82-915c-7b3a8d152eda^^^Avante at Ormond Beach^PI||Buckridge^Kareem^Millie^^^^L||19580810|F||2106-3^White^HL70005^^^^2.5.1|688 Leighann Inlet^^South Rodneychester^IG^67071^^^^48077||7275555555:1:^PRN^^[email protected]^1^211^2240784|||||||||U^Unknown^HL70189||||||||N
ORC|RE|73a6e9bd-aaec-418e-813a-0ad33366ca85|73a6e9bd-aaec-418e-813a-0ad33366ca85|||||||||1629082607^Eddin^Husam^^^^^^CMS&2.16.840.1.113883.3.249&ISO^^^^NPI||^WPN^^^1^386^6825220|20210209||||||Avante at Ormond Beach|170 North King Road^^Ormond Beach^FL^32174^^^^12127|^WPN^^[email protected]^1^407^7397506|^^^^32174
OBR|1|73a6e9bd-aaec-418e-813a-0ad33366ca85|0cba76f5-35e0-4a28-803a-2f31308aae9b|94558-4^SARS-CoV-2 (COVID-19) Ag [Presence] in Respiratory specimen by Rapid immunoassay^LN|||202102090000-0600|202102090000-0600||||||||1629082607^Eddin^Husam^^^^^^CMS&2.16.840.1.113883.3.249&ISO^^^^NPI|^WPN^^^1^386^6825220|||||202102090000-0600|||F
OBX|1|CWE|94558-4^SARS-CoV-2 (COVID-19) Ag [Presence] in Respiratory specimen by Rapid immunoassay^LN||260415000^Not detected^SCT|||N^Normal (applies to non-numeric results)^HL70078|||F|||202102090000-0600|||CareStart COVID-19 Antigen test_Access Bio, Inc._EUA^^99ELR||202102090000-0600||||Avante at Ormond Beach^^^^^CLIA&2.16.840.1.113883.4.7&ISO^^^^10D0876999^CLIA|170 North King Road^^Ormond Beach^FL^32174^^^^12127
OBX|2|CWE|95418-0^Whether patient is employed in a healthcare setting^LN||Y^Yes^HL70136||||||F|||202102090000-0600|||||||||||||||QST
OBX|3|CWE|95417-2^First test for condition of interest^LN||Y^Yes^HL70136||||||F|||202102090000-0600|||||||||||||||QST
OBX|4|CWE|95421-4^Resides in a congregate care setting^LN||N^No^HL70136||||||F|||202102090000-0600|||||||||||||||QST
OBX|5|CWE|95419-8^Has symptoms related to condition of interest^LN||N^No^HL70136||||||F|||202102090000-0600|||||||||||||||QST
SPM|1|0cba76f5-35e0-4a28-803a-2f31308aae9b||258500001^Nasopharyngeal swab^SCT||||71836000^Nasopharyngeal structure (body structure)^SCT^^^^2020-09-01|||||||||202102090000-0600|202102090000-0600
MSH|^~\&|CDC PRIME - Atlanta, Georgia (Dekalb)^2.16.840.1.114222.4.1.237821^ISO|Avante at Ormond Beach^10D0876999^CLIA|PRIME_DOH|Prime ReportStream|20210210170737||ORU^R01^ORU_R01|371784|P|2.5.1|||NE|NE|USA||||PHLabReportNoAck^ELR_Receiver^2.16.840.1.113883.9.99^ISO
SFT|Centers for Disease Control and Prevention|0.1-SNAPSHOT|PRIME ReportStream|0.1-SNAPSHOT||20210210
PID|1||2a14112c-ece1-4f82-915c-7b3a8d152eda^^^Avante at Ormond Beach^PI||Buckridge^Kareem^Millie^^^^L||19580810|F||2106-3^White^HL70005^^^^2.5.1|688 Leighann Inlet^^South Rodneychester^IG^67071^^^^48077||7275555555:1:^PRN^^[email protected]^1^211^2240784|||||||||U^Unknown^HL70189||||||||N
ORC|RE|73a6e9bd-aaec-418e-813a-0ad33366ca85|73a6e9bd-aaec-418e-813a-0ad33366ca85|||||||||1629082607^Eddin^Husam^^^^^^CMS&2.16.840.1.113883.3.249&ISO^^^^NPI||^WPN^^^1^386^6825220|20210209||||||Avante at Ormond Beach|170 North King Road^^Ormond Beach^FL^32174^^^^12127|^WPN^^[email protected]^1^407^7397506|^^^^32174
OBR|1|73a6e9bd-aaec-418e-813a-0ad33366ca85|0cba76f5-35e0-4a28-803a-2f31308aae9b|94558-4^SARS-CoV-2 (COVID-19) Ag [Presence] in Respiratory specimen by Rapid immunoassay^LN|||202102090000-0600|202102090000-0600||||||||1629082607^Eddin^Husam^^^^^^CMS&2.16.840.1.113883.3.249&ISO^^^^NPI|^WPN^^^1^386^6825220|||||202102090000-0600|||F
OBX|1|CWE|94558-4^SARS-CoV-2 (COVID-19) Ag [Presence] in Respiratory specimen by Rapid immunoassay^LN||260415000^Not detected^SCT|||N^Normal (applies to non-numeric results)^HL70078|||F|||202102090000-0600|||CareStart COVID-19 Antigen test_Access Bio, Inc._EUA^^99ELR||202102090000-0600||||Avante at Ormond Beach^^^^^CLIA&2.16.840.1.113883.4.7&ISO^^^^10D0876999^CLIA|170 North King Road^^Ormond Beach^FL^32174^^^^12127
SPM|1|0cba76f5-35e0-4a28-803a-2f31308aae9b||258500001^Nasopharyngeal swab^SCT||||71836000^Nasopharyngeal structure (body structure)^SCT^^^^2020-09-01|||||||||202102090000-0600|202102090000-0600'
  1. Wait for it to make it through the send step
  2. Verify that an ITEM_SENT event is generated for each item in the delivered report and that the data is correct, specifically looking at the childItemIndex and submittedItemIndex

Changes

  • Adds a new function for getting the report at a particular step from a child id/index
  • Updates the code to send an ITEM_SENT event

Checklist

Testing

  • Tested locally?
  • Ran ./prime test or ./gradlew testSmoke against local Docker ReportStream container?
  • (For Changes to /frontend-react/...) Ran npm run lint:write?
  • Added tests?

Process

  • Are there licensing issues with any new dependencies introduced?
  • Includes a summary of what a code reviewer should test/verify?
  • Updated the release notes?
  • Database changes are submitted as a separate PR?
  • DevOps team has been notified if PR requires ops support?

Linked Issues

To Be Done

Specific Security-related subjects a reviewer should pay specific attention to

Copy link

github-actions bot commented Sep 18, 2024

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails

Scanned Manifest Files

Copy link

github-actions bot commented Sep 18, 2024

Test Results

1 232 tests  ±0   1 228 ✅ ±0   7m 38s ⏱️ -6s
  162 suites ±0       4 💤 ±0 
  162 files   ±0       0 ❌ ±0 

Results for commit e6909dc. ± Comparison against base commit db9921c.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Sep 18, 2024

Integration Test Results

 53 files  ±0   53 suites  ±0   27m 29s ⏱️ +6s
410 tests ±0  401 ✅ ±0  9 💤 ±0  0 ❌ ±0 
413 runs  ±0  404 ✅ ±0  9 💤 ±0  0 ❌ ±0 

Results for commit e6909dc. ± Comparison against base commit db9921c.

♻️ This comment has been updated with latest results.

@mkalish mkalish force-pushed the platform/kalish/15766-azure-event-item-sent branch from 689cb04 to 90ada17 Compare September 19, 2024 21:00
@mkalish mkalish marked this pull request as ready for review September 19, 2024 21:07
@mkalish mkalish requested a review from a team as a code owner September 19, 2024 21:07
@mkalish mkalish added the platform Platform Team label Sep 20, 2024
@@ -616,6 +624,45 @@ class ActionHistory(
)
}

val lineages = Report.createItemLineagesFromDb(header, sentReportId)
lineages?.forEach { itemLineage ->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This algorithm is okay for now, but just thinking out loud, downloading blobs and parsing them just to get Item information EVERY TIME we need it in the various steps seems inefficient. If we were storing "Item" in the database, the bundle digest is something that would fit nicely there and then we could just get it from the DB (given it does not contain PII). Alternatively, we could also use the covid pipelines metadata table strategy probably.

I don't think any of this is worth considering until we gather the external message monitoring requirements.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. My ultimate preference would to have a clearer delineation of the uses cases since this doesn't even feel like monitoring and is more an analytics/business reporting question

@@ -520,4 +526,33 @@ class ReportGraph(

return select
}

/**
* Fetches all descendant report records in a recursive manner.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"ancestor" instead of "descendant"?

@@ -565,6 +572,7 @@ class ActionHistory(
result: String,
header: WorkflowEngine.Header,
reportEventService: IReportStreamEventService,
reportService: ReportService,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this be better as a class member?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was really messy with how the action history gets instantiated to do it that way (also why I'm passed in the reportEventService that way originally). I was bucketing this is in things to clean up if we refactor how some this works

return reportGraph.getRootReports(childReportId).distinctBy { it.reportId }
}

fun getReportsForStep(childReportId: ReportId, childIndex: Int, task: TaskAction): ReportFile? {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kdoc? Also confusing because query only returns one but method name is getReports?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, little copy pasta; this actually fetches a specific report for an item and action. Will update the name

@@ -213,6 +212,17 @@ class ReportGraph(
return descendantReportRecords(txn, cte, searchedForTaskActions).fetchInto(ReportFile::class.java)
}

fun getAncestorReports(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kdoc?

@mkalish
Copy link
Collaborator Author

mkalish commented Sep 24, 2024

Back to you @arnejduranovic

Copy link

sonarcloud bot commented Sep 25, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
45.3% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud

@mkalish mkalish merged commit 51a8a01 into master Sep 25, 2024
21 of 22 checks passed
@mkalish mkalish deleted the platform/kalish/15766-azure-event-item-sent branch September 25, 2024 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform Platform Team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants