polytopes_db_4d converted to new style package

[embray]

This spkg is mentioned in #14467 and was apparently "huge" (and hence optional). But it is no longer listed under build/pkgs nor does it appear to be available on the download mirrors.

---

Here is a branch to convert it to a new style experimental (but, maybe optional?) package.

the tarball is the old *.spkg file, but as it's a tar file, it works

tarball url (see in checksum.ini), but it's the same as one gets
via ./sage -p polytopes_db_4d (which still works with old style pkfs)

A few options:

1. If someone has a copy of this package we can convert it to a new-style SPKG (we will probably need to upload the tarball somewhere with enough space to host it).

Done.

CC: @mkoeppe @vbraun

Component: packages: huge

Author: Dima Pasechnik

Branch/Commit: b49f89a

Reviewer: Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/26029

[slel]

comment:1

Related discussions:

• Ask Sage question 43179: Problem installing package (2018-07)
• sage-devel discussion (2018-08)

[slel]

comment:2

The package is listed at:

• http://files.sagemath.org/spkg/huge/

[slel]

comment:3

And it's available from each of the SageMath download mirrors. E.g.

• LIP6 Paris: http://www-ftp.lip6.fr/pub/math/sagemath/spkg/huge/
• Riken Japan: ftp://ftp.riken.jp/sagemath/spkg/huge/index.html

etc.

The problems encountered with this huge package might instead have to do with

• Sage Trac ticket 26011: Copying package files is really slow

[embray]

comment:4

As I wrote on sage-devel, we could keep supporting this package if it were repackaged as a .tar.<whatever compression method works best>, and we added an appropriate build/pkgs/polytopes_db_4d for it. We would then want to move it to a standard location on the mirrors, not under "spkg/huge". That, or leave it under "spkg/huge" (if there is some advantage to that--can mirrors opt not to carry "huge" packages?) but modify sage-download-file to be able to find tarballs under that path on the mirrors.

[slel]

comment:6

The following shows sage can find packages in "huge":

$ sage --info polytopes_db_4d
Attempting to get on-line info for package polytopes_db_4d
>>> Checking online list of optional packages.
>>> Checking online list of experimental packages.
>>> Checking online list of huge packages.
>>> Found polytopes_db_4d-1.0
...

The package cannot be installed with sage -i because it is an "old-style SPKG"

$ sage -i polytopes_db_4d
...
Error: package 'polytopes_db_4d' not found
Note: if it is an old-style package, use -p instead of -i to install it

but sage -p can install "old-style spkg" packages that still work, including this one.

$ sage -p polytopes_db_4d
Attempting to download package polytopes_db_4d
>>> Checking online list of optional packages.
>>> Checking online list of experimental packages.
>>> Checking online list of huge packages.
>>> Found polytopes_db_4d-1.0
================================== NOTE ==================================
You are about to download and install an old-style package. While this
might still work fine, old-style packages are unmaintained and deprecated.

This package will be removed in future versions of SageMath. If you care
about this package, you should make a proper new-style package instead.
For more information about making Sage packages, see
http://doc.sagemath.org/html/en/developer/packaging.html
==========================================================================
Are you sure (automatically continuing in 30 seconds) [Y/n]?

Once this is done and after restarting Sage, one can follow the
SageMath documentation on using the palp_database of reflexive Hodge 4-polytopes.
I tried it in Sage 8.4 as installed on the "k8s" CoCalc server for development,
which currently runs Ubuntu 17.04.

sage: version()
'SageMath version 8.4, Release Date: 2018-10-17'

sage: from sage.geometry.polyhedron.palp_database import Reflexive4dHodge
sage: ref = Reflexive4dHodge(1, 101)
sage: next(iter(ref)).Vrepresentation()
(A vertex at (-1, -1, -1, -1),
 A vertex at (0, 0, 0, 1),
 A vertex at (0, 0, 1, 0),
 A vertex at (0, 1, 0, 0),
 A vertex at (1, 0, 0, 0))

[embray]

comment:7

Retargeting some of my tickets (somewhat optimistically for now).

[embray]

comment:8

Removing most of the rest of my open tickets out of the 8.7 milestone, which should be closed.

[embray]

comment:9

According to Andrey Novoseltev:

Data originates from http://hep.itp.tuwien.ac.at/~kreuzer/CY/

The whole list can be generated using PALP (a standard package in Sage), but when they have done it first, it took about a year on multiple computers, so there is sense in just downloading these polytopes. Sage package contains precomputed data for Hodge numbers as well, allowing quick search for particular cases. Volker knows more about exact details, I was just happily using the result.

While we are at it: it would be awesome if installation of this package after upgrade was faster. "Installation" just means unpacking files and copying them somewhere, but it takes something like 45 minutes. Some time ago I asked to use parallel decompressor if it is available in the system, which helped a lot, but then it stopped again.

As I wrote on sage-devel, perhaps it would make sense to host this data (as well as the 3D polytopes; why not?) on an online database somewhere that can be easily queried (including by the code in Sage which uses this data)? This way it is possible to use as-needed without downloading the entire, very large database (though perhaps with the possibility of pre-downloading/caching all or parts of it as needed).

It would be very easy to write a simple web service, and perhaps the data and service could be hosted on sagemath.org. This might also be a good opportunity for some MitM synergy. There is a workshop later this year on mathematical data, and frameworks and services for accessing them, so that might be a good opportunity to work on such a project (I would be happy to help, especially if someone can help me with the math end of things as I have no idea what a reflexive Hodge polytope is much less how it would be represented in a MitM framework): https://opendreamkit.org/2019/08/17/WorkshopOnDataInMathematics/

[dimpase]

comment:10

A reflexive polytope is a full-dimensional lattice polytope (i.e. the vertices have integer coordinates), containing the origin in the interior, so that its dual
is also a lattice polytope. All this is readily available in Sage.

[novoselt]

comment:11

3D polytopes take so little space, that there is no sense at all removing them from Sage in their current form. As for 4D, there are probably very few people who are using them and so there is a lack of effort to do something about them. On the other hand, since there are papers referring to the use of this package in Sage, it would be nice to support repeating those computations in some way, even if installation is "non-standard".

[dimpase]

Author: Dima Pasechnik

[dimpase]

Commit: 2ed5a53

[dimpase]

New commits:

2ed5a53

converting polytopes_db_4d into a new style package

[dimpase]

Branch: u/dimpase/packages/refpoly4d

[tscrim]

comment:13

Do we want this to be experimental or optional? I feel like optional would be the better option for this.

[mkoeppe]

comment:14

I also think it should be "optional"

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

b49f89a

make it optional

[sagetrac-git]

Changed commit from 2ed5a53 to b49f89a

[dimpase]

comment:16

OK, done

[tscrim]

comment:17

Thanks. I am ready to set a positive review. Any objections?

[tscrim]

Reviewer: Travis Scrimshaw

[mkoeppe]

comment:18

No objection; I haven't tried it though because downloading was taking too long.

[tscrim]

comment:19

It built for me without any problems, so I am setting it to positive.

[vbraun]

Changed branch from u/dimpase/packages/refpoly4d to b49f89a