Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate tests to use dvc-tracked baseline images #1131

Closed
28 tasks done
seisman opened this issue Mar 26, 2021 · 0 comments · Fixed by #1154
Closed
28 tasks done

Migrate tests to use dvc-tracked baseline images #1131

seisman opened this issue Mar 26, 2021 · 0 comments · Fixed by #1154
Labels
help wanted Helping hands are appreciated maintenance Boring but important stuff for the core devs
Milestone

Comments

@seisman
Copy link
Member

seisman commented Mar 26, 2021

After #1036, we start to use the new dvc based workflow for baseline images. We need to migrate existing baseline images to dvc, and also rewrite some tests. Please read the Testing Plots section in the contributing guides for the details of the new dvc based workflow. (Related to #963)

This issue tracks the progress of the migration.

Main Tasks

  1. Migrate existing baseline images to dvc
  2. Migrate @check_figures_equal tests to @pytest.mark.mpl_image_compare

Migrate existing baseline images to dvc

For each item, you should see the test file pygmt/tests/test_xxx.py and baseline images pygmt/tests/baseline/test_xxx_*.png.

Migrate @check_figures_equal tests to @pytest.mark.mpl_image_compare

Some tests in the following files use @check_figures_equal, and should be migrated to @pytest.mark.mpl_image_compare.


Here are some notes for the migration based on my own experience. These are all open for comments and suggestions:

  1. Tests should focus on testing PyGMT-specific codes. Some tests that test wrapper aliases can be removed.
  2. Use SI units and long-form parameters, unless there is a special reason
  3. Each test should focus on testing one feature/bug.
  4. Although we no longer store baseline images in the git repository, please try to generate smaller images if possible (e.g., set the figure width to 15c, not 30c, use a simpler dataset, use low-resolution coastline or earth relief data).
  5. Think about how to generated baseline images that are easy to verify their correctness. For example, when plotting a data point on a figure, it's easier to verify that the data is plotted at the correct position when frame and gridlines are also shown.
  6. You don't have to migrate all tests in one test file in one PR. Adding/updating too many baseline images also makes the PR difficult to review. Try to change no more than 5 tests in one PR.
@seisman seisman added maintenance Boring but important stuff for the core devs help wanted Helping hands are appreciated labels Mar 26, 2021
@seisman seisman pinned this issue Mar 26, 2021
michaelgrund added a commit that referenced this issue Mar 29, 2021
As discussed in #1131, this PR moves the reference figures in fig.contour to use dvc.
@seisman seisman added this to the 0.4.0 milestone Mar 30, 2021
weiji14 added a commit that referenced this issue Apr 1, 2021
Related to #1131, and is almost like reverting #589.
weiji14 added a commit that referenced this issue Apr 1, 2021
Related to #1131, and is almost like reverting #589.
michaelgrund added a commit that referenced this issue Apr 1, 2021
michaelgrund added a commit that referenced this issue Apr 2, 2021
michaelgrund added a commit that referenced this issue Apr 3, 2021
michaelgrund added a commit that referenced this issue Apr 3, 2021
@seisman seisman unpinned this issue Apr 6, 2021
@weiji14 weiji14 changed the title Migrate tests to used dvc-tracked baseline images Migrate tests to use dvc-tracked baseline images Apr 28, 2021
weiji14 added a commit that referenced this issue May 26, 2021
Final round of tests to migrate to DVC for the
`grdview` plotting module. This is related to #1131,
and is almost like reverting #589 actually.

* Migrate test_grdview baseline images to dvc
* Update fig.grdview baseline images for GMT 6.2.0rc1
* Plot fancy frame for test_grdview_with_perspective
* Update fig.grdview baseline images for GMT 6.2.0rc2
* Use projection="Q15c+" for test_grdview_with_perspective
sixy6e pushed a commit to sixy6e/pygmt that referenced this issue Dec 21, 2022
Final round of tests to migrate to DVC for the
`grdview` plotting module. This is related to GenericMappingTools#1131,
and is almost like reverting GenericMappingTools#589 actually.

* Migrate test_grdview baseline images to dvc
* Update fig.grdview baseline images for GMT 6.2.0rc1
* Plot fancy frame for test_grdview_with_perspective
* Update fig.grdview baseline images for GMT 6.2.0rc2
* Use projection="Q15c+" for test_grdview_with_perspective
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Helping hands are appreciated maintenance Boring but important stuff for the core devs
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant