-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add undeliveredUploads feature to UploadStats. #197
Conversation
Update the GetUploadStats query to search for and provide the response for "undelivered uploads" and the associated counts as part of the query response.
@@ -72,6 +71,18 @@ class UploadStatsLoader: CosmosLoader() { | |||
+ ") r where r.totalCount > 1" | |||
) | |||
|
|||
val unDeliveredUploadsQuery = ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This query is sufficient for detecting explicitly undelivered uploads with reports that contain a failure status. However, we need a second query to look for uploads where this report is missing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the code to fetch the uploads where the report is missing for the stage, "blob-file-copy". For a given dataStreamId, dataStreamRoute and the start and the end dates, we check for any difference in the total uploads with stage-action = "metadata-verify" and stage-action ="blob-file-copy", get the respective undelivered upload ids as well.
Update the GetUploadStats query to search for and provide the response for "undelivered uploads" and the associated counts as part of the query response (Send the counts, filename and uploadId).
…es for blob-file-copy in teh reports container
Get the latest from develop.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There looks to be some opportunities to improve the efficiency by reducing the number of queries run. Also, there may be some holes in the logic we should revisit. Approving so we can get this deployed, knowing we may be taking on some tech debt.
Quality Gate passedIssues Measures |
Note: Need to work on query optimization and other improvements |
Update the GetUploadStats query to search for and provide the response for "undelivered uploads" and the associated counts as part of the query response.