Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pk5/fix integration tests for release #195

Merged
merged 3 commits into from
May 16, 2024

Conversation

patrykkulik-microsoft
Copy link
Collaborator

  • We were hitting an error in the integration tests because when running list_credential method we do not retry on failure. When running the CLI normally (outside of the testing framework), the retries happen automatically which means that the command would succeed when running outside of the testing framework. This seems like an external issue but can be fixed by adding a short while loop to retry the list_credential method
  • I have also changed the publisher name in our tests because the US has some publishers laying around with the same name, which meant we could not actually run the tests with the old publisher name

@jordlay
Copy link
Collaborator

jordlay commented May 15, 2024

Logic looks good to me. As discussed, check with Dave as to whether we should merge this now or not

artifact_manifest_name=config.acrManifestName,
).as_dict()
break
except ServiceResponseError as error:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I think if you hit an exception that isn't a ServiceResponseError the code will just exit. Is that right? If not, there's an infinite loop.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is just going to fail with another exception but we are not going to catch it

# This retry logic is to handle the ServiceResponseError that is hit in the integration tests.
# This error is not hit when running the cli normally because the CLI framework automatically retries,
# the testing framework does not support automatic retries.
while retries < 2:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of questions:

  • Are automatic retries going to race/conflict with these ones? I'm not sure what would happen if they did - maybe it's fine.
  • Why aren't these retries in the test code if it's a limitation in the test framework?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The automatic retires should not race with this one because the ServiceResponseError is only raised when running in a test.
  • I have discussed with the rest of the team about fixing in the testing code and we decided that it would be complicated to figure out (if even possible) so we should just fix it in the main code and chase the CLI team about this problem

@patrykkulik-microsoft patrykkulik-microsoft merged commit f5a787a into main May 16, 2024
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants