Add script packages build/pkgs/_prereq, build/pkgs/_bootstrap #29124

mkoeppe · 2020-01-31T18:39:33Z

As a follow-up to #29053, we move the build/pkgs/*.txt files to a more systematic location. This will simplify some scripts.

build/pkgs/_prereq/distros/debian.txt etc. - records the minimal requirements (system packages) for installing from source. (we move the current build/pkgs/debian.txt etc. there)
build/pkgs/_bootstrap/distros/debian.txt etc. - would record the additional requirements for running ./bootstrap (we move the current build/pkgs/debian-bootstrap.txt there)

Add script-type _spkg_check package that installs tox, mock, pytest etc. #28998: Add script-type _spkg_check package that installs tox, mock, pytest etc.
Add script package _recommended and generate "recommended system packages" info #29557: build/pkgs/_recommended - records additional recommended packages such as texlive-xetex (see https://wiki.sagemath.org/prerequisites/Ubuntu)

Possible follow-up:

build/pkgs/_toolchain/dependencies - to replace some special Makefile targets currently in build/make/Makefile.in.

Depends on #30940
Depends on #29655

CC: @dimpase @egourgoulhon @jhpalmieri @kiwifb @tobiasdiez @seblabbe @slel @kliem

Component: build

Keywords: sd111

Author: Matthias Koeppe

Branch/Commit: e8da106

Reviewer: Sébastien Labbé, Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29124

The text was updated successfully, but these errors were encountered:

mkoeppe · 2020-04-15T19:26:31Z

comment:4

for example on fedora-26-standard sagetex testsuite fails:

! LaTeX Error: File `makecmds.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: 
! Emergency stop.
<read *> 
         
l.78 \RequirePackage
                    {ifpdf}^^M

mkoeppe · 2020-04-16T15:41:29Z

Branch: u/mkoeppe/add_script_packages_build_pkgs__prereq__build_pkgs__toolchain__build_pkgs__bootstrap___recommended

mkoeppe · 2020-04-16T15:43:22Z

Commit: bf2dfdf

mkoeppe · 2020-04-16T15:43:22Z

comment:6

Installation manual also recommends Tcl/Tk - what are they used for in Sage?
http://doc.sagemath.org/html/en/installation/source.html#tcl-tk

New commits:

`bf2dfdf`	`build/pkgs/_recommended: New`

mkoeppe · 2020-04-16T15:43:43Z

Author: Matthias Koeppe

mkoeppe · 2020-04-23T18:15:01Z

Dependencies: #29557

mkoeppe · 2020-04-23T18:15:01Z

Changed branch from u/mkoeppe/add_script_packages_build_pkgs__prereq__build_pkgs__toolchain__build_pkgs__bootstrap___recommended to none

mkoeppe · 2020-04-23T18:15:01Z

Changed commit from bf2dfdf to none

mkoeppe · 2020-11-23T07:55:17Z

Changed dependencies from #29557 to #30951

mkoeppe · 2020-11-23T07:55:17Z

comment:14

Best done on top of #30951

mkoeppe · 2020-11-24T01:31:59Z

Changed dependencies from #30951 to #30951, #30947

mkoeppe · 2021-01-02T20:31:07Z

Changed dependencies from #30940 to #30940, #29655

sagetrac-git · 2021-01-02T20:46:05Z

Changed commit from 9988c5f to 6ff6545

sagetrac-git · 2021-01-02T20:46:05Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`6f3b864`	`build/pkgs/[s-t]*/SPKG.rst: Reformat title in format 'spkg_name: Short description'`
`23558f8`	`build/pkgs/[t-z]*/SPKG.rst: Reformat title in format 'spkg_name: Short description' -- completes coverage of all standard packages`
`17789df`	`build/pkgs/[z-z]*/SPKG.rst: Reformat title in format 'spkg_name: Short description' -- completes coverage of all optional packages`
`944408d`	`build/pkgs/[a-z]*/SPKG.rst: Reformat title in format 'spkg_name: Short description' -- completes coverage of all experimental packages`
`feafa60`	`src/doc/en/installation/standard_packages.rst: Remove`
`58c63e1`	`src/doc/bootstrap: Add brief explanation of the standard/optional/experimental package types`
`54d4fb1`	`extra line after version`
`17f2f24`	`Merge branch 't/29655/improve_build_pkgs___spkg_rst' into t/29124/script-packages-prereq-toolchain-bootstrap`
`e7a768b`	`build/pkgs/{_bootstrap,_prereq}/SPKG.rst: Reformat title`
`6ff6545`	`Move build/pkgs/opensuse*.txt to build/pkgs/{_bootstrap,_prereq}/distros`

mkoeppe · 2021-01-02T20:48:36Z

comment:64

Still needs review

dimpase · 2021-01-03T09:52:14Z

comment:65

./bootstrap is getting very slow - 3 times slower than ./configure, over 60 sec on a relatively fast machine!

dimpase · 2021-01-03T15:23:34Z

comment:66

On this branch I'm getting

sage -t --warn-long 53.2 --random-seed=0 src/sage/tests/cmdline.py
**********************************************************************
File "src/sage/tests/cmdline.py", line 219, in sage.tests.cmdline.test_executable
Failed example:
    print(out)  # optional - build
Expected:
    Found local metadata for sqlite-...
    SQLite
    ======
    ...
    SQLite is a software library that implements a self-contained,
    serverless, zero-configuration, transactional SQL database engine.
    ...
Got:
    sqlite: An SQL database engine
    ==============================
    <BLANKLINE>
    Description
    -----------
    <BLANKLINE>
    SQLite is a software library that implements a self-contained,
    serverless, zero-configur
...

and one more similar failure in this file.
Not sure whether it's coming from this ticket, or one of dependencies.

dimpase · 2021-01-03T15:23:34Z

Changed reviewer from Sébastien Labbé to Sébastien Labbé, Dima Pasechnik

mkoeppe · 2021-01-03T19:03:40Z

comment:67

Thanks for catching this. It's coming from #29655

sagetrac-git · 2021-01-03T19:18:01Z

Changed commit from 6ff6545 to b57fecc

sagetrac-git · 2021-01-03T19:18:01Z

Branch pushed to git repo; I updated commit sha1. New commits:

`2eac7a1`	`src/doc/bootstrap: Use sage-spkg-info directly instead of 'sage --info', much faster now`
`3fe8d4c`	`src/sage/tests/cmdline.py: Adjust doctests`
`b57fecc`	`Merge branch 't/29655/improve_build_pkgs___spkg_rst' into t/29124/script-packages-prereq-toolchain-bootstrap`

dimpase · 2021-01-04T10:57:11Z

comment:70

hmm, now I see

File "src/sage/tests/cmdline.py", line 219, in sage.tests.cmdline.test_executable
Failed example:
    print(out)  # optional - build
Expected:
    SQLite
    ======
    ...
    SQLite is a software library that implements a self-contained,
    serverless, zero-configuration, transactional SQL database engine.
    ...
Got:
    sqlite: An SQL database engine
    ==============================
    <BLANKLINE>
    Description
    -----------
    <BLANKLINE>
    SQLite is a software library that implements a self-contained,
    serverless, zero-configuration, transactional SQL database engine.
    <BLANKLINE>
...

(and the same in line 232)

So more changes of these test outputs are needed.

The different heading (sqlite: An SQL database engine) comes from src/doc/en/reference/spkg/sqlite.rst, as far as me and grep can tell.

sagetrac-git · 2021-01-04T17:20:36Z

Branch pushed to git repo; I updated commit sha1. New commits:

`49ad8ae`	`src/sage/tests/cmdline.py: Adjust doctest properly`
`e8da106`	`Merge branch 't/29655/improve_build_pkgs___spkg_rst' into t/29124/script-packages-prereq-toolchain-bootstrap`

sagetrac-git · 2021-01-04T17:20:36Z

Changed commit from b57fecc to e8da106

dimpase · 2021-01-04T18:43:09Z

comment:73

OK, good.

mkoeppe · 2021-01-05T06:35:50Z

comment:74

Thanks!

vbraun · 2021-01-24T10:38:15Z

Changed branch from u/mkoeppe/script-packages-prereq-toolchain-bootstrap to e8da106

mkoeppe added this to the sage-9.1 milestone Jan 31, 2020

mkoeppe added c: build labels Jan 31, 2020