src/bin/sage: Delegate handling of options of sage-the-distribution, specifying a subset that is supported by sagelib proper #29111

mkoeppe · 2020-01-30T16:12:13Z

The sage script has a number of options that only make sense for sage-the-distribution, but do not make sense in downstream packaging of sage. Consequently, downstream packagers may install simple custom versions of the script that only provide a subset of the options.

Examples:

sage -sqlite -- as discussed in spkg-configure.m4 for sqlite #29002/sqlite3 CLI executable should be checked for at configure time #29092 -- because from a downstream packaging point of view, sage is not really responsible for providing sqlite, it's provided by justs another system package
sage -upgrade -- only makes sense with sage-the-distribution built from source

We should specify the subset of sage command line options that must be supported in any deployment/packaging of sage. This is so that user packages can work reliably.

Examples:

sage -c SAGECOMMAND -- obviously
sage -python -c COMMAND
sage -t -- because it is used by user packages for testing sage sources
sage -sh ..... ?

In this ticket, we actually split src/bin/sage into sagelib functionality and sage-the-distribution functionality. src/bin/sage delegates unknown options to build/bin/sage-site.

Ideally, distributions should be able to use an unmodified src/bin/sage.

Context:

https://groups.google.com/d/msg/sage-packaging/BmkxIBdwbvE/fRMl2sjdBQAJ
Meta-ticket: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages #29060: Meta-ticket: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages

See also:

src/doc/bootstrap: Generate src/doc/en/reference/repl/options.txt #29884: src/doc/bootstrap: Generate src/doc/en/reference/repl/options.rst

Depends on #29878
Depends on #29289
Depends on #29884

CC: @kiwifb @isuruf @dimpase @embray @saraedum @antonio-rojas @slel @EmmanuelCharpentier @orlitzky @kcrisman

Component: scripts

Author: Matthias Koeppe, John Palmieri

Branch: 3953671

Reviewer: Matthias Koeppe, John Palmieri, François Bissey

Issue created by migration from https://trac.sagemath.org/ticket/29111

The text was updated successfully, but these errors were encountered:

mkoeppe · 2020-01-30T16:21:00Z

comment:1

As I said on #29092, we should also move corresponding sage-the-distribution tests (such as those for sage -sqlite) mentioned in #29002) from src (which is for sagelib) to somewhere in build (which is for sage-the-distribution). Downstream packagers would ignore the sage-the-distribution tests.

mkoeppe · 2020-04-14T06:33:01Z

comment:3

pushing these forward to 9.2

jhpalmieri · 2020-06-16T20:15:08Z

comment:6

I've asked this question before: if Sage is built with the system's Python, should we still keep sage --python ...? What is the difference between sage --python and sage --sqlite3 (and what about sage --gap, sage --singular, etc.)?

jhpalmieri · 2020-06-16T20:24:43Z

comment:7

Here are some arguments that should always be allowed:

sage -t
sage -c
sage -h
sage -n, sage --notebook
sage --nodotsage
sage -v, sage --version

"Advanced" arguments:

sage --preparse
sage --min
sage -q
sage --dumpversion
sage --startuptime
sage --fixdoctests
sage --coverage
sage --coverageall
sage --grep, sage --search_src, sage --grepdoc, sage --search_doc
sage --rst2ipynb, sage --ipynb2rst

Maybe also sage --sh.

(I'm not saying is a complete list, but I would suggest these as a starting point.)

jhpalmieri · 2020-06-16T20:42:45Z

comment:8

By the way, sage -twistd doesn't work: there is no twistd executable since Sage 9.1. Plus we are removing the twisted package in #29754. See #29878.

kiwifb · 2020-06-16T21:03:19Z

comment:9

Hum, this ticket reminds me that I need to clean the version in sage-on-gentoo

fbissey@moonloop ~ $ sage -advanced
SageMath version 9.2.beta1, Release Date: 2020-06-13

Running Sage:
  file.[sage|py|spyx] -- run given .sage, .py or .spyx file
  -advanced           -- print this advanced help message
  -c <cmd>            -- Evaluates cmd as sage code
  -h, -?              -- print short help message
  -min [...]          -- do not populate global namespace (must be first
                         option)
  -preparse <file.sage> -- preparse file.sage and produce corresponding file.sage.py
  -q                  -- quiet; start with no banner
  -gthread, -qthread, -q4thread, -wthread, -pylab
                      -- pass the option through to ipython
  -v, -version        -- display Sage version information
  -dumpversion        -- print Sage version

Running the notebook:
  --notebook=[...]    -- start the Sage notebook (valid options are
                         'default', 'sagenb', 'jupyter' and 'export').
                         Current default is 'export' sagenb to jupyter.
                         See the output of sage --notebook --help
                         for more details and examples of how to pass
                         optional arguments
  -inotebook [...]    -- start the *insecure* Sage notebook (deprecated)
  -n, -notebook [...] -- start the default Sage notebook (options are the
                         same as for the notebook command in Sage).  See the
                         output of sage -n -h for more details

Running external programs:
  -cleaner            -- run the Sage cleaner
  -cython [...]       -- run Cython with given arguments
  -ecl [...]          -- run Common Lisp
  -gap [...]          -- run Sage's Gap with given arguments
  -gdb                -- run Sage under the control of gdb
  -gp [...]           -- run Sage's PARI/GP calculator with given arguments
  -ipython [...]      -- run Sage's IPython using the default environment (not
                         Sage), passing additional options to IPython
  -ipython3 [...]     -- same as above, but using Python 3
  -jupyter [...]      -- run Sage's Jupyter with given arguments
  -kash [...]         -- run Sage's Kash with given arguments
                         (not installed currently, run sage -i kash)
  -lisp [...]         -- run Lisp interpreter included with Sage
  -M2 [...]           -- run Sage's Macaulay2 with given arguments
                         (not installed currently, run sage -i macaulay2)
  -maxima [...]       -- run Sage's Maxima with given arguments
  -mwrank [...]       -- run Sage's mwrank with given arguments
  -polymake [...]     -- run Sage's Polymake with given arguments
                         (not installed currently, run sage -i polymake)
  -python [...]       -- run the Python interpreter
  -python2 [...]      -- run the Python 2 interpreter
  -python3 [...]      -- run the Python 3 interpreter
  -R [...]            -- run Sage's R with given arguments
  -sh [...]           -- run $SHELL (/bin/bash) with Sage environment variables
  -singular [...]     -- run Sage's singular with given arguments
  -sqlite3 [...]      -- run Sage's sqlite3 with given arguments
  -twistd [...]       -- run Twisted server

Testing the Sage library:
  -startuptime [module] -- display how long each component of Sage takes to
                         start up; optionally specify a module to get more
                         details about that particular module
  -t [options] <--all|files|dir>
                      -- test examples in .py, .pyx, .sage, .tex or .rst files
                         selected options:
                           --long - include lines with the phrase 'long time'
                           --verbose - print debugging output during the test
                           --all - test all files
                           --sagenb - test all sagenb files
                           --optional - controls which optional tests are run
                           --initial - only show the first failure per block
                           --debug - drop into PDB after an unexpected error
                           --failed - only test files that failed last test
                           --warn-long [timeout] - warning if doctest is slow
                           --only-errors - only output failures, not successes
                           --gc=GC - control garbarge collection (ALWAYS:
                                     collect garbage before every test; NEVER:
                                     disable gc; DEFAULT: Python default)
                           --help - show all testing options
  -tp <N> [...]       -- like -t above, but tests in parallel using N threads
                         with 0 interpreted as a sensible default
  -testall [options]  -- test all source files, docs, and examples.  options
                         like -t

File conversion:
  -rst2ipynb [...]    -- Generates Jupyter notebook (.ipynb) from standalone
                         reStructuredText source.
                         (not installed currently, run sage -i rst2ipynb)
  -ipynb2rst [...]    -- Generates a reStructuredText source file from
                         a Jupyter notebook (.ipynb).

Valgrind memory debugging:
  -cachegrind         -- run Sage using Valgrind's cachegrind tool.  The log
                         files are named sage-cachegrind.PID can be found in
                         
  -callgrind          -- run Sage using Valgrind's callgrind tool.  The log
                         files are named sage-callgrind.PID can be found in
                         
  -massif             -- run Sage using Valgrind's massif tool.  The log
                         files are named sage-massif.PID can be found in
                         
  -memcheck           -- run Sage using Valgrind's memcheck tool.  The log
                         files are named sage-memcheck.PID can be found in
                         
  -omega              -- run Sage using Valgrind's omega tool.  The log
                         files are named sage-omega.PID can be found in
                         
  -valgrind           -- this is an alias for -memcheck

You can also use -- before a long option, e.g., 'sage --optional'.

I have to drop the py2/py3 stuff for just defaut python. I dropped -sh the day I dropped sage-env altogether, the main point of keeping sage -sh around was to check or use the environment set by sage-env.

mkoeppe · 2020-06-16T22:17:23Z

comment:10

Dropping sage -sh is a bad idea. User packages need this

mkoeppe · 2020-06-16T22:19:01Z

comment:11

Replying to @jhpalmieri:

I've asked this question before: if Sage is built with the system's Python, should we still keep sage --python ...? What is the difference between sage --python and sage --sqlite3 (and what about sage --gap, sage --singular, etc.)?

sage -python means: run the python that sage runs in.
In distributions, this could just be plain Python.

Install scripts of user packages need this.

mkoeppe · 2020-06-16T22:26:31Z

Dependencies: #29878

kiwifb · 2020-06-16T22:29:34Z

comment:13

Replying to @mkoeppe:

Dropping sage -sh is a bad idea. User packages need this

Certainly, vanilla sage does need it. In sage-on-gentoo at best you would have a normal shell since I dropped sage-env altogether.

mkoeppe · 2020-06-16T22:32:13Z

comment:14

Replying to @kiwifb:

Replying to @mkoeppe:

Dropping sage -sh is a bad idea. User packages need this

Certainly, vanilla sage does need it. In sage-on-gentoo at best you would have a normal shell since I dropped sage-env altogether.

Yes, vanilla shell is fine. The point is that user packages need the interface.

mkoeppe · 2020-06-16T22:33:29Z

Changed dependencies from #29878 to #29878, #29289

kiwifb · 2020-06-16T22:36:26Z

comment:16

Now, I understand what you mean. User scripts may start with sage -sh and what not. I completely forgot about that when I removed it - even though that was one of the reason I was keeping it around. I will re-instate it as a vanilla shell or no-op for compatibility.

mkoeppe · 2020-06-16T22:39:02Z

comment:17

Great!

mkoeppe · 2020-06-16T23:01:16Z

Branch: u/mkoeppe/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution

mkoeppe · 2020-06-16T23:02:49Z

comment:19

Here's a beginning of what I have in mind.

New commits:

`b200d70`	`trac 29878: remove "sage --twistd" command-line argument`
`e0b6112`	`build/bin/sage-spkg, src/bin/sage: Remove support for installing old-style SPKGs`
`20b2a93`	`Merge branch 't/29289/remove_support_for_installing_old_style_spkgs__deprecated_in_sage_6_9' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution`
`152e581`	`src/bin/sage: Delegate options that pertain to SAGE_ROOT or sage-the-distribution to the new script build/bin/sage-site`

mkoeppe · 2020-06-16T23:02:49Z

Commit: 152e581

jhpalmieri · 2020-06-16T23:12:47Z

comment:20

Okay, I'm fine with keeping sage --python. What about sage --singular et al.? Should src/bin/sage be constructed by ./configure, adding arguments when the corresponding packages are going to be installed by Sage? Or is that too complicated?

mkoeppe · 2020-06-16T23:14:17Z

comment:22

Replying to @jhpalmieri:

Okay, I'm fine with keeping sage --python. What about sage --singular et al.?

I think those should be moved to sage-site as well. (Also the printing of these in the help text...)

Should src/bin/sage be constructed by ./configure, adding arguments when the corresponding packages are going to be installed by Sage? Or is that too complicated?

I think that's too complicated

orlitzky · 2020-06-16T23:40:03Z

comment:23

sage --grep, sage --search_src, sage --grepdoc, sage --search_doc

At the risk of starting another holy war, we shouldn't expect these to work on a distribution because we shouldn't expect the source files to be available after installation. My current sage.git checkout contains 71MiB of *.py files that are entirely redundant after the optimized .opt-2.pyc files are generated. Someone is eventually going to realize that they shouldn't be installed after this gets more popular.

mkoeppe · 2020-06-16T23:44:32Z

comment:24

We could just conditionalize these ones based on whether SAGE_SRC is set or not by sage-env...

jhpalmieri · 2020-06-22T05:35:33Z

comment:109

Good idea. From http://fricas.sourceforge.net/history.html, it looks like FriCAS is a fork of Axiom, but I just installed it, and it doesn't install axiom (at least not in SAGE_ROOT/local/bin, nor is axiom listed in its package manifest). It installs fricas and efricas, so we could eventually add those, but it's not like anyone's been clamoring for them.

mkoeppe · 2020-06-22T05:46:51Z

comment:110

Then let's just drop sage --axiom. Good catch!

dimpase · 2020-06-22T06:03:31Z

comment:111

is axiom package an old-style one?

It's probably very outdated. also, my impression is that fricas fork is the most active of all the forks of the original axiom forks.
The others are http://www.axiom-developer.org/ (axiom) and http://www.open-axiom.org/links.html (OpenAxiom)

mkoeppe · 2020-06-22T17:42:46Z

comment:112

That's right, there is no new-style axiom package.

sagetrac-git · 2020-06-22T17:44:41Z

Changed commit from 4a3d36e to 3a0193c

sagetrac-git · 2020-06-22T17:44:41Z

Branch pushed to git repo; I updated commit sha1. New commits:

`3a0193c`	`src/bin/sage: Remove handling of 'sage -axiom'`

mkoeppe · 2020-06-22T17:45:10Z

comment:114

Done. Needs review

jhpalmieri · 2020-06-22T18:52:31Z

comment:115

I'm happy with everything. Who else would like to weigh in?

mkoeppe · 2020-06-23T20:04:22Z

Changed reviewer from Matthias Koeppe, John Palmieri, ... to Matthias Koeppe, John Palmieri, François Bissey

kiwifb · 2020-06-23T20:46:53Z

comment:117

Sorry, I was busy and forgot about it. We have reached an agreement with what goes in and what's next. I think we have bashed it enough and it is ready to go.

mkoeppe · 2020-06-23T21:11:38Z

comment:118

Thanks!

jhpalmieri · 2020-06-23T21:44:36Z

comment:119

mkoeppe · 2020-06-23T21:50:03Z

comment:120

Follow-ups:

Install src/bin scripts by sagelib's setup.py, not make #21559: Install src/bin scripts by sagelib's setup.py, not make
Change handling of 'sage --APP' #29930: Change handling of sage --APP

vbraun · 2020-07-08T22:50:50Z

comment:121

Merge confilct

sagetrac-git · 2020-07-08T22:59:01Z

Branch pushed to git repo; I updated commit sha1. New commits:

`c8bab39`	`Merge tag '9.2.beta4' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution`

sagetrac-git · 2020-07-08T22:59:01Z

Changed commit from 3a0193c to c8bab39

sagetrac-git · 2020-07-08T22:59:44Z

Changed commit from c8bab39 to 3953671

sagetrac-git · 2020-07-08T22:59:44Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`3953671`	`Merge tag '9.2.beta4' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution`

vbraun · 2020-07-10T19:34:16Z

Changed branch from u/mkoeppe/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution to 3953671

mkoeppe · 2022-04-25T02:25:50Z

Changed commit from 3953671 to none

mkoeppe · 2022-04-25T02:25:50Z

comment:126

Follow-up in #33753

mkoeppe added this to the sage-9.1 milestone Jan 30, 2020

mkoeppe added p: major / 3 labels Jan 30, 2020

mkoeppe added the c: scripts label Jan 31, 2020

mkoeppe modified the milestones: sage-9.1, sage-9.2 Apr 14, 2020

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~Specify a subset of sage command line options that are supported by sagelib - rather than sage-the-distribution~~ src/bin/sage: Delegate handling of options of sage-the-distribution, specifying a subset that is supported by sagelib proper Jun 16, 2020

kiwifb added s: positive review and removed s: needs review labels Jun 23, 2020

vbraun added s: needs work and removed s: positive review labels Jul 8, 2020

mkoeppe added s: positive review and removed s: needs work labels Jul 8, 2020

vbraun removed the s: positive review label Jul 10, 2020

vbraun closed this as completed in cc19847 Jul 10, 2020

jhpalmieri mentioned this issue Jul 10, 2020

Deprecate deprecated sage-cython script harder, remove use of it in the main sage script #29923

Closed

src/bin/sage: Delegate handling of options of sage-the-distribution, specifying a subset that is supported by sagelib proper #29111

src/bin/sage: Delegate handling of options of sage-the-distribution, specifying a subset that is supported by sagelib proper #29111

Comments

mkoeppe commented Jan 30, 2020

mkoeppe commented Jan 30, 2020

mkoeppe commented Apr 14, 2020

This comment has been minimized.

jhpalmieri commented Jun 16, 2020

jhpalmieri commented Jun 16, 2020

jhpalmieri commented Jun 16, 2020

kiwifb commented Jun 16, 2020

mkoeppe commented Jun 16, 2020

mkoeppe commented Jun 16, 2020

mkoeppe commented Jun 16, 2020

kiwifb commented Jun 16, 2020

mkoeppe commented Jun 16, 2020

mkoeppe commented Jun 16, 2020

kiwifb commented Jun 16, 2020

mkoeppe commented Jun 16, 2020

mkoeppe commented Jun 16, 2020

mkoeppe commented Jun 16, 2020

mkoeppe commented Jun 16, 2020

jhpalmieri commented Jun 16, 2020

This comment has been minimized.

mkoeppe commented Jun 16, 2020

orlitzky commented Jun 16, 2020

mkoeppe commented Jun 16, 2020

jhpalmieri commented Jun 22, 2020

mkoeppe commented Jun 22, 2020

dimpase commented Jun 22, 2020

mkoeppe commented Jun 22, 2020

sagetrac-git mannequin commented Jun 22, 2020

sagetrac-git mannequin commented Jun 22, 2020

mkoeppe commented Jun 22, 2020

jhpalmieri commented Jun 22, 2020

mkoeppe commented Jun 23, 2020

kiwifb commented Jun 23, 2020

mkoeppe commented Jun 23, 2020

jhpalmieri commented Jun 23, 2020

mkoeppe commented Jun 23, 2020

vbraun commented Jul 8, 2020

sagetrac-git mannequin commented Jul 8, 2020

sagetrac-git mannequin commented Jul 8, 2020

sagetrac-git mannequin commented Jul 8, 2020

sagetrac-git mannequin commented Jul 8, 2020

vbraun commented Jul 10, 2020

mkoeppe commented Apr 25, 2022

mkoeppe commented Apr 25, 2022