Undo the breaking of old-style packages

[jdemeyer]

Ticket #19004 has the very user-unfriendly consequence that all old-style packages are effectively gone:

$ ./sage -i chomp
Attempting to download package chomp
Error: could not find a package matching chomp
       Try 'sage --package list' to see the available packages
       Did you mean: csdp, gmp, cbc, cmake, cvxopt?

Instead of removing support for old-style packages, we should deprecate them.

Depends on #19101

CC: @nathanncohen @vbraun @jhpalmieri

Component: scripts

Author: Jeroen Demeyer

Branch/Commit: 3dda704

Reviewer: Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/19158

[jdemeyer]

Dependencies: #19101

[vbraun]

comment:2

IMHO the only thing that needs to be done is make chomp a new-style package.

The mixture of old and new packages has always been broken. Let it die. We can support old packages as legacy but not mash it up with new packages in an utterly confusing manner to users.

[jdemeyer]

comment:3

1. There has been no deprecation at all for this, which really goes against Sage's 1-year deprecation policy. You cannot just break user's work flows like this.

2. Old-style packages still appear in various places in the documentation:

• Various places document sage -i chomp (or other old-style packages)
• sage --optional shows both new-style and old-style packages
• We even have a page in the developer documentation about creating old-style packages.

[vbraun]

comment:4

1. There is no deprecation period for fixing bugs.
2. The documentation should be fixed (Often we should just point people towards pip; Suggesting some ancient unmaintained spkg is yet another bug in itself)

[jdemeyer]

comment:5

Replying to @vbraun:

1. There is no deprecation period for fixing bugs.

The fact that sage -i chomp used to install chomp is a bug??

[vbraun]

comment:6

That sage -i python used to install an ancient Python is a bug. That the undesirable behavior of silently installing ancient packages happened to be somewhat(1) desirable for chomp doesn't make it less of a bug.

(1) The chomp spkg actually doesn't compile with modern gcc due to tighter C++ checking

../include/capd/matrixAlgorithms/floatMatrixAlgorithms.hpp:245:159: error: redeclaration of 'template<class MatrixType> int chomp::matrixAlgorithms::symMatrixDiagonalize(const MatrixType&, MatrixType&, typename MatrixType::ScalarType)' may not have default arguments [-fpermissive]
 int symMatrixDiagonalize(const MatrixType& A, MatrixType& D, typename MatrixType::ScalarType diagonalizingRelTolerance = typename MatrixType::ScalarType(1e-5))
                                                                                                                                                               ^
../include/capd/matrixAlgorithms/floatMatrixAlgorithms.hpp:326:177: error: redeclaration of 'template<class MatrixType> typename MatrixType::ScalarType chomp::matrixAlgorithms::spectralRadiusOfSymMatrix(const MatrixType&, typename MatrixType::ScalarType)' may not have default arguments [-fpermissive]

[jdemeyer]

comment:7

Replying to @vbraun:

That sage -i python used to install an ancient Python is a bug.

Agreed. But #19004 wasn't the right fix.

That the undesirable behavior of silently installing ancient packages happened to be somewhat(1) desirable for chomp doesn't make it less of a bug.

This ticket isn't about chomp at all, I'm just using that package as one example of an old-style package.

[vbraun]

comment:8

IMHO we should have removed old spkgs a long time ago. It was ok during the transition period but has been causing us constant pain since then. We are much better of with #19004 than without. For every old-style package that you might actually want to install I can name you ten that are going to get in the way if you accidentally install them.

[jdemeyer]

comment:9

Replying to @vbraun:

IMHO we should have removed old spkgs a long time ago.

But we didn't, so we are currently stuck with them.

It was ok during the transition period but has been causing us constant pain since then.

Maybe, but like I said: #19004 isn't the solution. The solution is warning users that they are installing old-style packages with a proper deprecation warning.

[vbraun]

comment:10

Replying to @jdemeyer:

Maybe, but like I said: #19004 isn't the solution. The solution is warning users that they are installing old-style packages with a proper deprecation warning.

No, the solution is to not install broken/deprecated packages. Or, rather, make it difficult enough so that nobody does it accidentally.

[jdemeyer]

Branch: u/jdemeyer/ticket/19158

[jdemeyer]

Commit: 39f1e4d

[jdemeyer]

New commits:

31ff6e7	Parse sage-spkg options in sage-spkg
cb05746	Add sage --location option to run sage-location
e856a8e	Change sage -i to install with dependencies
e105028	Revert sage-spkg changes from Trac #19004
39f1e4d	Deprecate old-style packages

[jdemeyer]

Author: Jeroen Demeyer

[vbraun]

comment:13

I'm against

[jdemeyer]

comment:14

Replying to @vbraun:

I'm against

Well, I am against #19004.

Is there any compromise possible? A solution which still allows users to install old-style packages?

[vbraun]

comment:15

You can install old-style packages, they just aren't searched for. If you want to try to compile chomp just run e.g.

sage -f http://files.sagemath.org/spkg/optional/chomp-20130518.p1.spkg

[jdemeyer]

comment:16

Replying to @vbraun:

You can install old-style packages, they just aren't searched for.

That's "you can" for a very dubious value of "can". To users, it really looks like the package is completely gone.

[vbraun]

comment:18

And thats good because they are either huge nuisances (like "python") or just broken.

[jdemeyer]

comment:19

We obviously cannot resolve this here, so I'm writing to sage-devel.

[jhpalmieri]

comment:20

As I said on #19105, you shouldn't search "standard" packages on-line.

[jdemeyer]

comment:21

Replying to @jhpalmieri:

As I said on #19105, you shouldn't search "standard" packages on-line.

Sure, that's an easy change (although not very relevant if we merge #19105).

But I'm confused by your comment. Does it imply you're in favour of this ticket now?

[jhpalmieri]

comment:22

I don't think I ever said I wasn't in favor of it. My last post on the sage-devel thread said, "I could also be easily persuaded to (B) if we don't allow "sage -i python" to install the old python-2.5 package."

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

efd2725

Do not consider "standard" and "archive" packages

[sagetrac-git]

Changed commit from 39f1e4d to efd2725

[jdemeyer]

comment:24

Replying to @jhpalmieri:

My last post on the sage-devel thread said, "I could also be easily persuaded to (B) if we don't allow "sage -i python" to install the old python-2.5 package."

Great! My last commit does that, since we no longer look for standard packages:

Attempting to download package python
>>> Checking online list of optional packages.
>>> Checking online list of experimental packages.
>>> Checking online list of huge packages.
Error: could not find a package matching python
       Try 'sage --package list' to see the available packages
       Did you mean: cython, ipython, python2, python3, patch?

[jdemeyer]

comment:25

Given William Stein's support, I am still very convinced that we should move forward with this ticket.

[tscrim]

comment:26

I was able to install (old-style) chomp without any problems with this ticket. Moreover it doesn't let me install python. Overall I'm ready to set this to positive review except for an interface question.

Could we add an option to the warning that if the user presses Y or enter (after perhaps a mandatory 5 second reading period) that the old-style spkg installs? If this is too difficult, then could we drop the sleep to 15 seconds or so (it's not that long of a message)?

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

3dda704

Improve prompt

[sagetrac-git]

Changed commit from efd2725 to 3dda704

[tscrim]

comment:28

LGTM. Thanks!

[tscrim]

Reviewer: Travis Scrimshaw

[vbraun]

comment:29

Urgh thats so ugly I just threw up in my mouth a little. Ever wondered why none of the gnu coreutils does the ask + timeout?

I expect you to fix the topcom/TOPCOM issue and do something about the totally broken packages.

[jhpalmieri]

comment:30

Replying to @vbraun:

I expect you to fix the topcom/TOPCOM issue and do something about the totally broken packages.

See #19220 for TOPCOM. That and python are the two serious problems with old-style packages, as far as I understand. Other old-style packages may be broken, but that's not new. I don't know of any other packages which will break the Sage installation if you try to install them.

[tscrim]

comment:31

Python is handled on #19105.

[vbraun]

comment:32

Well somebody has to go through the list. Fun things:

• kash3 is binary-only. Somehow that wasn't OK for gap3 but you just re-enabled another optional binary-only package.
• Anything that is pip installable is wayyy outdated, terrible UX to just use ancient cruft even if it works. Nobody even made a list of which packages fall in that category
• The undocumented: soya_cvs-2006.05.09.spkg has no SPKG.txt, wtf is it and why is it a 9-year old cvs snapshot?

And by "somebody", I mean somebody who was is favor of this ticket.

[vbraun]

Changed branch from u/jdemeyer/ticket/19158 to 3dda704