Packaging for 3.14.0 doesnt include sub-packages

[jayvdb]

The PyPI packages only include package 'health_check', excluding sub-packages `health_check.contrib', etc

[jayvdb]

Also it looks like the template isnt being installed.

[skarzi]

I have updated django_health_check to 3.14.0 in several projects and it works.

How do you importing contrib and other sub-packages?
And what do you mean by template isn't being installed?

[jayvdb]

The contents of the 3.14.0 wheel looks ok.

Installing from the sdist using setup.py does not work correctly using latest setuptools.

c.f. https://build.opensuse.org/request/show/835160 openSUSE workaround to get it working on rolling release which uses latest versions of all build related packages. I haven't investigated if it works on some older versions of setuptools - there is a narrow window of versions that support setup.cfg fully (the .cfg syntax seems to be evolving even now), so it is possible that some versions work, and others dont.

For the record, the build env this occurred on was

[   19s] [125/191] cumulate python38-base-3.8.5-2.1
[   19s] [126/191] cumulate python3-appdirs-1.4.4-2.1
[   19s] [127/191] cumulate python3-attrs-19.3.0-2.2
[   19s] [128/191] cumulate python3-billiard-3.6.3.0-41.5
[   19s] [129/191] cumulate python3-iniconfig-1.0.1-1.1
[   19s] [130/191] cumulate python3-mock-4.0.2-1.2
[   19s] [131/191] cumulate python3-more-itertools-8.4.0-1.2
[   19s] [132/191] cumulate python3-olefile-0.46-2.4
[   19s] [133/191] cumulate python3-ordered-set-3.1.1-4.2
[   19s] [134/191] cumulate python3-pluggy-0.13.1-1.4
[   19s] [135/191] cumulate python3-py-1.9.0-1.3
[   19s] [136/191] cumulate python3-pycparser-2.20-1.5
[   19s] [137/191] cumulate python3-pyparsing-2.4.7-2.2
[   19s] [138/191] cumulate python3-toml-0.10.1-2.1
[   19s] [139/191] cumulate python3-vine-1.3.0-1.7
[   19s] [140/191] cumulate python3-wcwidth-0.2.5-1.2
[   19s] [141/191] cumulate rpm-build-python-4.15.1-7.2
[   19s] [142/191] cumulate systemd-rpm-macros-4-3.1
[   19s] [143/191] cumulate libpython3_8-1_0-3.8.5-2.1
[   19s] [144/191] cumulate libxcrypt-devel-4.4.17-1.1
[   19s] [145/191] cumulate linux-glibc-devel-5.7-1.3
[   19s] [146/191] cumulate python3-pytz-2020.1-1.2
[   19s] [147/191] cumulate python3-six-1.15.0-1.2
[   19s] [148/191] cumulate glibc-locale-base-2.31-6.3
[   19s] [149/191] cumulate libctf0-2.34-1.13
[   19s] [150/191] cumulate python3-future-0.18.2-1.6
[   19s] [151/191] cumulate python3-sqlparse-0.3.1-2.5
[   19s] [152/191] cumulate permissions-config-1550_20200909-23.1
[   19s] [153/191] cumulate python38-3.8.5-2.1
[   19s] [154/191] cumulate gettext-tools-0.20.2-2.3
[   19s] [155/191] cumulate aaa_base-84.87+git20200909.ee4a72c-1.1
[   19s] [156/191] cumulate rpm-4.15.1-7.2
[   19s] [157/191] cumulate aaa_base-malloccheck-84.87+git20200909.ee4a72c-1.1
[   19s] [158/191] cumulate glibc-locale-2.31-6.3
[   19s] [159/191] cumulate permissions-20200909.1550-23.1
[   19s] [160/191] cumulate python3-amqp-2.6.1-1.2
[   19s] [161/191] cumulate python3-asgiref-3.2.10-9.2
[   19s] [162/191] cumulate rpm-config-SUSE-0.g56-1.2
[   19s] [163/191] cumulate python3-packaging-20.4-1.2
[   19s] [164/191] cumulate glibc-devel-2.31-6.3
[   19s] [165/191] cumulate python3-cffi-1.14.2-1.1
[   19s] [166/191] cumulate librpmbuild9-4.15.1-7.2
[   19s] [167/191] cumulate build-compare-20200727T175347.d95eb35-1.2
[   19s] [168/191] cumulate python3-Pillow-7.2.0-3.1
[   19s] [169/191] cumulate python3-argon2-cffi-20.1.0-1.2
[   19s] [170/191] cumulate post-build-checks-84.87+git20200819.5847424-1.1
[   19s] [171/191] cumulate python3-setuptools-44.1.1-1.1
[   19s] [172/191] cumulate rpmlint-mini-1.10-21.4
[   19s] [173/191] cumulate pam-1.4.0-1.3
[   19s] [174/191] cumulate gcc10-10.2.1+git583-1.2
[   19s] [175/191] cumulate rpmlint-Factory-1.0-98.2
[   19s] [176/191] cumulate gcc-10-2.1
[   19s] [177/191] cumulate python3-setuptools_scm-4.1.2-1.2
[   19s] [178/191] cumulate python3-kombu-4.6.11-1.2
[   19s] [179/191] cumulate python3-Django-3.1.1-1.1
[   19s] [180/191] cumulate python3-pytest-6.0.1-1.1
[   19s] [181/191] cumulate shadow-4.8.1-2.3
[   19s] [182/191] cumulate gcc-PIE-10-2.1
[   19s] [183/191] cumulate sysuser-shadow-3.0-4.1
[   19s] [184/191] cumulate python3-pytest-django-3.9.0-2.1
[   19s] [185/191] cumulate python3-celery-4.4.6-1.3
[   19s] [186/191] cumulate system-group-hardware-20170617-12.1
[   19s] [187/191] cumulate redis-6.0.8-1.1
[   19s] [188/191] cumulate python3-redis-3.5.3-1.2
[   19s] [189/191] cumulate libutempter0-1.2.0-2.1
[   19s] [190/191] cumulate util-linux-2.35.1-2.3
[   19s] [191/191] cumulate rpm-build-4.15.1-7.2

more complete logs can be found at https://build.opensuse.org/package/show/devel:languages:python:django/python-django-health-check and logging in, after which you can "branch" (fork) that package to edit the .spec file to remove the hacks which will let you see it fail. But I was able to reproduce the problem locally, so that shouldn't be necessary.

[codingjoe]

Hi @jayvdb,

I checked, both the source and binary release include all packages and templates.
Maybe you could elaborate a bit more on your setup and I could try to help you find a solution. However, it seems your problem is highly specific to your setup and not to the package itself.

Best,
Joe

[codingjoe]

Please forgive that I removed your comment form the other issue. It was unrelated and could be misleading to others.

[jayvdb]

It has nothing to do with me. This is a openSUSE distribution package, and the problem exists for any other distro and any other user of the sdist. The problem is not that the files are not in the sdist. They are in there. The problem is the setup.py doesnt install them. Anyway, as you are deleting feedback, and we have a workaround, I will disengage. Hopefully someone else can explain it better.

[codingjoe]

Hi @jayvdb,

To be honest, I was not aware there was an OpenSUSE distro of this package. In any case, after reviewing the spec-file, it seems the issue that the spec is using the wrong source. It uses the GitHub tar-balls which are not official releases.

You could use the official PyPi sdist builds, those will work, see also:

python3 -m pip install --force-reinstall --no-binary :all: django-health-check==3.14.0

A little person side note:
I am truly sorry, if the removal of your comment offended you. I notified about the removal to avoid any hard feelings while making a humble effort to maintain this repo and its discussions. Please let know if I can be of any help, to simplify the OpenSUSE release.

Best,
Joe

[jayvdb]

@codingjoe , distro packages build from source and run tests in an offline build environment to avoid any chance of pollutants. This package no longer provides tests in the PyPI sdist, so we must fetch the tarball from GitHub as a fallback to get a properly versioned testsuite. I mentioned this problem in my (now deleted) comment on the other issue.

In addition, pip has a dotted history with regards to https://reproducible-builds.org/ which is needed by distros, frequently not working, but occasionally it worked in some cases. pypa/pip#7808 suggests it might be working now in some cases, but without adequate tests it would be foolish for distros to switch to using pip only to find it breaks again and they need to revert all of their spec files to the old syntax.
pypa/pip#6505 and others indicate that it isnt working for at least one large class of cases.

[codingjoe]

I see. So, the other issue is about releases missing on PyPi, not files missing in the dists. That being said, we could easily add the tests and docs back into the sdist. Its two lines really, that currently explicitly remove the folders. We could remove those lines, what do you think?
https://github.com/KristianOellegaard/django-health-check/blob/36e37a5409ef7fce9286f5fa9c24a185592df59a/MANIFEST.in#L2-L3

[jayvdb]

Re-adding docs and tests to the sdist would certainly help.

The suggested change above seems dubious, but I havent checked it carefully. That typically causes a package called tests to be installed by setup.py, but it depends on several other variables in setup.cfg. The only way to know for certain is to build sdists and install them, and rinse and repeat until the packaging does the right thing, and no more. check-manifest helps ensure the sdist is comprehensive, but check-manifest doesnt prevent setuptools from installing tests or other sibling directories that are included in the sdist, and check-manifest needs an exclude list configuration if you want to not have everything in the sdist- often maintainers feel that some files dont belong in the sdist, and I usually exclude a lot of the dot files from the sdists of the packages I maintain for similar reasons.

[codingjoe]

Hi @jayvdb,

Thanks for the hint about check-manifest. I didn't know this one.
I did a thorough look at the Manifest.in file and explicitly added everything that should be in the sdist build and removed all GitHub/Fussyfox related files. I opened a PR #271 and would love to get your feedback.
Oh, and the setup.cfg file defines the package explicitly, so tests isn't being installed. I double-checked.

Best,
Joe

[jayvdb]

Re-checked with v3.16.5, and I am using the GitHub tarball.

The sub-packages and index.html are still not installing.
Quite odd.

I have a little hack in my .spec file which "fixes" this

sed -i 's/packages = health_check/packages = find:/' setup.cfg

That fixes the installation of sub-packages, but I still need to manually install the index.html

And the tests being installed is very likely a byproduct of my hack above.