Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test coverage for new code #244

Open
dzeber opened this issue Feb 22, 2021 · 2 comments
Open

Add test coverage for new code #244

dzeber opened this issue Feb 22, 2021 · 2 comments
Assignees

Comments

@dzeber
Copy link
Contributor

dzeber commented Feb 22, 2021

The refactored evaluations and report interface are still missing test coverage. Some may be able to be refactored out of the old versions.

@dzeber dzeber added this to the Initial beta release milestone Feb 22, 2021
@dzeber
Copy link
Contributor Author

dzeber commented Jun 12, 2021

I'm still working on sorting through the old code and tests and looking at what can be ported over. I will try to get this done on Monday. I've started making some updates on this branch.

@dzeber
Copy link
Contributor Author

dzeber commented Jun 15, 2021

@gleonard-m I've been going through the deprecated tests and code and adding comments on this branch. I've been making the following changes:

  • Removed code from presc.deprecated that is no longer needed or already ported to presc.evaluations.
    • either removing the entire module or parts
  • In some cases I've moved the deprecated functions to the test module itself and backported to the current implementation to preserve the test logic
    • hopefully this makes it clear how the test logic applies to the refactored evaluations
  • Some code that we may want to use in the future can be left in presc.deprecated for now
  • Removed tests from tests.deprecated for removed code.
  • Noted tests for which the logic can be ported to the new presc.evaluations code.
  • Maked some minor updates to the example notebooks based on the deprecated code so that they still run. In the future we can update them to use the refactored code.

All of these updates are commented in the code itself, under presc/deprecated and tests/deprecated.

So far there are test modules for conditional_metric and conditional_distribution, but the tests mainly check that the configuration options filter through.

  • We should add test coverage for each of the modules under presc.evaluations (except _base_evaluation which is not currently used) - I expect the pattern will be similar for each.
  • The main thing is that each one needs coverage for the compute_<...>() function, which actually runs the computation, to check that it gives the right outputs on a simple problem and possibly some edge cases.
  • We should also have tests that run the display_result() function which produces the graph, but at this point I think we only need to check that it runs without choking, not actually validate the outputs (especially since we'll be refactoring the graphing code).

The deprecated test code has some test cases and sample datasets that may be useful, but I'm not sure how much of it we want to port over. We also have a small dataset available through the fixtures in conftest. It is stored at tests/fixtures/dataset.pkl and was generated using tests/generate_data.py. The main goal at the moment is to make sure all the computations get run in a test to catch bugs, etc. We can discuss later if we want to run them through a wider variety of datasets/cases.

I will finish up tomorrow by updating the spatial distributions code and file a PR for that branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants