tests/*.zim files not included in release tarball

[legoktm]

If you download https://files.pythonhosted.org/packages/26/8e/201f1ed560f83f5fd4a87fe3ec52960d078f1a31c8f1670ac16cb92fe3a3/libzim-0.0.3.post0.tar.gz it's missing the two zim files in tests/ so the tests fail. I believe we need a MANIFEST.in file for this.

Also, from a licensing standpoint, it would be nice if we could use zim files with fully public domain content so then I don't have to keep track of the specific copyright status for Debian.

[legoktm]

Also if we used different zim files we could make them much smaller...they're 33M right now.

[kelson42]

@rgaudin @legoktm Maybe we should publish the tarbal on download.openzim.org/release ?

[rgaudin]

I also think we should not include 33M of ZIM files for tests.

• wikipedia_es_chemistry_mini.zim doesn't seem to be used (14M)
• Do we need large ZIM files (wikipedia_es_physics_mini.zim 20M)?
• Should we need it, should we download it it tests?

I think it's important we test with ZIM files not created with pylibzim but there are many small ones. We could have an longer test that downloads a larger ZIM file also if we think it's relevant. I think there were issues with large number of articles in the very early days of this.

[kelson42]

I'm in favour of downloading ZIM files if they are more than 100KB.

[legoktm]

With my Debian hat on:

• I would like all the tests in the tarball to pass without needing anything extra (tests could check if the extra file exists and skip if not).
• If something does want to download stuff, it needs to be possible to disable that functionality (e.g. one other Python library I maintain will skip network tests if NO_NETWORK environment variable is set).

Avoiding large ZIM files in the tarball would be nice too, but the other two are more important.

[legoktm]

Might also be worth looking at what ZIMs kiwix-lib is using: https://github.com/kiwix/kiwix-lib/tree/master/test/data. wikipedia_en_ray_charles_mini_2020-03.zim is ~550k which seems much more reasonable.

[legoktm]

I created #76 as an alternative solution, it just skips those tests if the file is missing.