Add opt-out import and cclf-import workflows #957
Conversation
Updates opt-out import workflows to correctly point to the right directories.
Adds initial workflow scaffolding to add cclf import lambdas. Actual deploy scripts will need to be implemented once the lambdas are rolled.
Using PLT-366 workflows, mirror them for cclf-import workflows.
austincanada
requested review from
gsf,
kyeah,
alex-dzeda and
laurenkrugen-navapbc
|
@austincanada With this PR up, can I close #950? |
@gsf |
| aws s3 cp --no-progress ../shared_files/synthetic1800MedicareFiles/test/T\#EFT.ON.ACO.NGD1800.DPRF.D181120.T1000009 \ | ||
| s3://bfd-test-eft/bfdeft01/bcda/in/T.NGD.DPC.RSP.D$(date +'%y%m%d').T$(date +'%H%M%S')1.IN | ||
|
|
||
| # TODO Check bucket for response file |
There was a problem hiding this comment.
See where I've figured out one way to check the response file: https://github.com/CMSgov/ab2d-lambdas/blob/ac828968f2f17c4440b0724bf196e8939b39c935/.github/workflows/opt-out-import-test-integration.yml#L49-L53
There was a problem hiding this comment.
We don't have response files for BCDA, so we'll just want to do the database check below if possible (or maybe just verify that the file was deleted from S3 after being ingested by the app code.)
Will need some arbitrary waiting period / loop or something to allow the lambda to run.
There was a problem hiding this comment.
Do we have any examples of us verifying the file was deleted or checking the database? Looking through ab2d and dpc there's no implementation that I can find to mirror them.
There was a problem hiding this comment.
No response files? That does change things. Hmm agreed then that the best check would be a job in the workflow to run a self-hosted runner and check the database. Good future work to schedule
🎫 Ticket
https://jira.cms.gov/browse/BCDA-7899
🛠 Changes
Added cclf-import workflows.
Added Sean's opt-out-import workflows.
Added unit testing workflow for cclf and opt-out workflow.
ℹ️ Context for reviewers
Creating a merge PR in order to test run the workflows. Some things will probably have to change within the workflows themselves.
✅ Acceptance Validation
Workflows completed:
Lambdas updated:
🔒 Security Implications
If any security implications apply, add Jason Ashbaugh (GitHub username: StewGoin) as a reviewer and do not merge this PR without his approval.