Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial version of regression test framework #1115

Draft
wants to merge 1 commit into
base: develop-v1.18.0
Choose a base branch
from

Conversation

bradh
Copy link
Contributor

@bradh bradh commented Jan 18, 2024

This is a concept draft for regression testing. It was inspired by the iden regression going unnoticed.

We do have unit testing, but it doesn't really exercise a lot of the functionality.

So this makes use of pytest (which is assumed installed - its a package on ubuntu) to do black-box testing. The main check at this stage is that heif-info produces the right result. There are a bunch of other things that could be tested.

Key decisions are:

  1. Is this worth the effort to maintain?
  2. Are the tests reliable? (I'm not sure what I have is reliable).
  3. Is there an additional / better source of test data?
  4. Do we want to do image comparisons against known-good snapshots (e.g. PNG files)

A couple of other things I noticed:

  • The output of heif-info --dump-boxes puts the MIME type, main brand and compatible brand at the start when you are using it interactively, and at the end when redirected to a file (e.g. see the last three lines in any of the .dump files).
  • The output of --help goes to stderr. That is probably OK for a mis-use, but might not be appropriate when requested, since its not really an error at this point.

Thoughts / opinions welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant