Create a wheel/sdist with the baseline images

[SidharthBansal]

PR Summary

• Create a wheel/sdist with just the baseline images; upload it to testpypi (so that one can do pip install mpl-baseline-images); don’t bother removing baseline images from the main repo yet.
• Changes in the MANIFEST for the matplotlib package is required for not including the matplotlib_baseline_images package.
  Uploading on test.pypi
• Created tests/test_data. Tests/test_data contains the images used in the tests and we don't want need in the baseline_images package.
• Symlink baseline images out.
• Changes in the travis, appvoyer and azure pipeline to install matplotlib_baseline_images package after matplotlib package is installed
• Changes in the .flake8 to ignore the import errors in the lib/matplotlib/tests/__init__.py and 'lib/mpl_toolkits/tests/_init.py`

In case of running the tests/modifying the tests containing the baseline-images, the developer needs to install the matplotlib_baseline_images package.

PR Checklist

• Has Pytest style unit tests
• Code is Flake 8 compliant
• New features are documented, with examples if plot related
• Documentation is sphinx and numpydoc compliant
• Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
• Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

[SidharthBansal]

Description about the commits:
Moved Baseline images to a sub-wheel/matplotlib-baseline-images commit moves the baseline images out of the lib folder to the newly created package directory matplotlib-baseline images

License added to mpl-baseline-only-package Created license for the mpl-baseline images. It is same License as /LICENSE/LICENSE

sub-wheels/matplotlib-baseline-images/MANIFEST.in added Added Manifest to include the baseline images folder, setup, readme, license, and setup.cfg.template for the package

Modified setup.cfg.template i deleted the instructions for disabling the baseline images to be installed implicitly as it is a separate package after merging this pr.

Moved mpl_baseline_images and mpl_toolkit_baseline_images to baseline_images folder Also included mpl-toolkit_baseline_images in the package

Modified links for the changed location of baseline images Changed links in the decorators for the image comparisons and in the tools/triage_tests.py

Added Setup.py changes for the baseline images package Fetched the version and then strip the version as it contains 'version_number+pep_encoding'. So, used [0] of the list after split('+').
Then used the setup to initialise the values.

Added Readme for the baseline images package Finally added the basic documentation in Readme

mpl-baseline-images package uploaded to test pypi -> https://test.pypi.org/project/matplotlib.baseline-images/3.2.1/

Thanks for the help. Kindly review @anntzer and suggest the changes.

[anntzer]

Sorry for the delay in reviewing.

Unfortunately, neither the wheel nor the sdist seem correct. You can check this by unpacking the whl and the tar.gz that you built and uploaded to testpypi: the whl contains only metadata and no baseline images and the tar.gz contains everything, not just the baseline images. Do you understand the problem?

An additional point is that decorators.py should not look up the baseline images relative to itself (because you can imagine that this won't work if e.g. matplotlib is directly installed and the baseline images are editably installed, or vice-versa). Although your modification could be kept as a fallback (in case matplotlib is editably installed and the baseline images not installed at all, but happen to be findable next to the rest of the library because that's how the git repository is laid out), the correct fix should be to instead e.g. import matplotlib_baseline_images and look up the images relative to matplotlib_baseline_images.__file__ (assuming you decide to create an empty matplotlib_baseline_images/__init__.py after all just for the purposes of locating things relative to it), or to use something like importlib.resources.path (https://docs.python.org/3/library/importlib.html#importlib.resources.path) or importlib.metadata.files (https://docs.python.org/3/library/importlib.metadata.html#files) to locate the files of that package.

Finally, changes to setup.cfg should be accompanied with the corresponding changes in the toplevel setupext.py.

[SidharthBansal]

changes to setup.cfg should be accompanied with the corresponding changes in the toplevel setupext.py.

I realized that sample data doesn't need any changes. Only test data/baseline images need changes. So, pushed commit for deletion of related feature from setupext.py

[anntzer]

Agreed that sample_data should be left in-place.

[SidharthBansal]

Changed path in decorators in this commit

[SidharthBansal]

A developer can check the sdist and wheel generated for the mpl package by running python3 setup.py sdist bdist_wheel from the terminal from the dist folder. It will not have sub-wheel directory

For the baseline-images package, kindly check by cd sub-wheels/matplotlib-baseline-images and then python3 setup.py sdist bdist_wheel from the sib-wheels/matplotlib-baseline-images/dist folder. It will only contain baseline images directory related files and folders.

[SidharthBansal]

Rebased

[tacaswell]

You can put local paths into a requirements.txt file like so: https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format

✔ 22:49:56 $ git diff 
diff --git a/requirements/testing/travis_all.txt b/requirements/testing/travis_all.txt
index 3f42a603f6..040389e207 100644
--- a/requirements/testing/travis_all.txt
+++ b/requirements/testing/travis_all.txt
@@ -8,3 +8,4 @@ pytest-timeout
 pytest-xdist
 python-dateutil
 tornado
+-e sub-wheels/matplotlib-baseline-images
diff --git a/sub-wheels/matplotlib-baseline-images/setup.py b/sub-wheels/matplotlib-baseline-images/setup.py
index ff4e1e4f95..67a3607d29 100644
--- a/sub-wheels/matplotlib-baseline-images/setup.py
+++ b/sub-wheels/matplotlib-baseline-images/setup.py
@@ -17,5 +17,5 @@ setup(
     package_dir={"": "lib"},
     packages=find_packages("lib"),
     include_package_data=True,
-    install_requires=["matplotlib=={}".format(__version__)],
+    # install_requires=["matplotlib=={}".format(__version__)],
 )

which means instead of having to document "please go into the sub-package and install it" we can have people do pip install -r requirements/testing/travis_all.txt (maybe renaming that file building on #15617 ?) so that to @jklymak 's point we have a very easy installation instruction installs the test images + pytest and friends.

I took out the version checking in the inner setup.py because it was failing for me due to a dirty git tree, I suspect a bit more thought needs to go into how to make that the right level of strictness.

[tacaswell]

changing the requirement file would also mean the changes to all of the ci configs could be reverted (as it would be done in a central place).

[tacaswell]

Maybe not on this pass, but we should think about how packages can what package carries their test images.

Maybe entrypoints?

[tacaswell]

My heavy-handed approach in my suggestion is probably too much, is this something that will be handled in a follow on PR?

[tacaswell]

This seems to all be technically correct I have a couple of questions:

For this PR:

• has there been thought about how to make the test-package-discovery (maybe https://github.com/takluyver/entrypoints)
• what is the plan for version checking / aligning?

I don't want to hold the PR for final answers, but just to make sure we have not painted ourselves in to a corner.

I assume the questions of the human developer and CI workflows are being addressed in #17793 ?

[SidharthBansal]

We haven't thought about the entrypoints yet. The version of mpl and mpl_baseline_images package will be handled in the followup PRs I guess.

[anntzer]

re: entrypoints: I think a much simpler solution would be to make the location of the baseline images configurable, e.g. def image_comparison(..., *, baseline_images_package=...) and then e.g. astropy would just do image_comparison = functool.partial(image_comparison, baseline_images_package=...) (or have global state settable via set_baseline_images_package(...), or possibly make the configurable item a mapping of package names to baseline package names (so that different packages don't step on another). In any case I'd rather not involve more packaging tooling.

re: versioning: I think the right way is to just declare baselines as immutable (if you want to change an image just rename it, e.g. from "foo.png" to "foo.1.png" to "foo.2.png" etc.) This doesn't really help re: FreeType version, but I guess that is a separate question? Right now (especially for the baseline wheel/sdist) we still have a canonical FT version, I guess later we can e.g. add a separate subdirectory for alternative FT versions?

PS: looks like upgrading setuptools fixed the azure build?

[SidharthBansal]

Yes

[SidharthBansal]

Moved to #17793