Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix db error reporting #93

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

peterwilton-ttam
Copy link

This is an apparent fix of the bug mentioned here. This bug obfuscates any error response stored by self._async_table.get_artifact_in_task(...) in DBResponse.body.

Basically, when there is a generic DB error, self._async_table.get_artifact_in_task(...) returns a DBResponse with .body attribute of type string (containing the error message) and a .response_code of 500, indicating an error. The string-type body is set in db_utils.aiopg_exception_handling.

ArtifactsApi._filter_artifacts_by_attempt_id(...) expects artifacts.body to be an iterable, which is what get_artifact_in_task returns when no error has occurred (and fetch_single==False, which is the case in get_artifact_in_task). This fix avoids the call to ArtifactsApi._filter_artifacts_by_attempt_id(...) when DBResponse.response_code is not 200.

@peterwilton-ttam
Copy link
Author

Here is the full comment on gitter for thoroughness:


I'm trying to take a look at the artifacts associated with one of our SFN executions but when trying to call:

Step(...).task.data

We run into the following error:

ServiceException: Metadata request (/flows/{Flow}/runs/{Run}/steps/start/tasks/{Task}/artifacts) failed (code 500): 500 Internal Server Error

Looking at the logs from our metadata service, I see the following error:

Traceback (most recent call last):
    File "/opt/latest/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 418, in start
        resp = await task
    File "/opt/latest/lib/python3.7/site-packages/aiohttp/web_app.py", line 458, in _handle
        resp = await handler(request)
    File "/opt/latest/lib/python3.7/site-packages/services/metadata_service/api/artifact.py", line 140, in get_artifacts_by_task
        artifacts.body)
    File "/opt/latest/lib/python3.7/site-packages/services/metadata_service/api/artifact.py", line 355, in _filter_artifacts_by_attempt_id
        attempt_id = ArtificatsApi._get_latest_attempt_id(artifacts)
    File "/opt/latest/lib/python3.7/site-packages/services/metadata_service/api/artifact.py", line 349, in _get_latest_attempt_id
        if artifact['attempt_id'] > attempt_id:
TypeError: string indices must be integers

From what I can tell, this can only occur when the result of this call returns a DBResponse object with a body of type string. And it looks like this would only happen in the case of error handling. Unfortunately it looks like the underlying error message is obfuscated by this error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant