Skip to content
This repository has been archived by the owner on Jan 30, 2023. It is now read-only.

Commit

Permalink
Merge #31386
Browse files Browse the repository at this point in the history
  • Loading branch information
Matthias Koeppe committed Dec 6, 2021
2 parents cc60cfe + 81e9c9a commit af47b5f
Show file tree
Hide file tree
Showing 9 changed files with 728 additions and 229 deletions.
46 changes: 32 additions & 14 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -349,54 +349,72 @@ Simplified directory layout (only essential files/directories):
SAGE_ROOT Root directory (sage-x.y.z in Sage tarball)
├── build
│ └── pkgs Every package is a subdirectory here
│ ├── 4ti2
│ ├── 4ti2/
│ …
│ └── zn_poly
│ └── zn_poly/
├── configure Top-level configure script
├── COPYING.txt Copyright information
├── pkgs Source trees of Python distribution packages
│ ├── sage-conf
│ │ ├── sage_conf.py
│ │ └── setup.py
│ ├── sage-docbuild
│ │ ├── sage_docbuild
│ │ ├── sage_docbuild/
│ │ └── setup.py
│ ├── sage-setup
│ │ ├── sage_setup/
│ │ └── setup.py
│ ├── sage-sws2rst
│ │ ├── sage_sws2rst
│ │ ├── sage_sws2rst/
│ │ └── setup.py
│ └── sagemath-standard
│ ├── bin
│ ├── sage
│ ├── bin/
│ ├── sage -> ../../src/sage
│ └── setup.py
├── local (SAGE_LOCAL) Compiled packages are installed here
├── local (SAGE_LOCAL) Installation hierarchy for non-Python packages
│ ├── bin Executables
│ ├── include C/C++ headers
│ ├── lib Shared libraries
│ ├── lib Shared libraries, architecture-dependent data
│ ├── share Databases, architecture-independent data, docs
│ │ └── doc Viewable docs of Sage and of some components
│ └── var
│ ├── lib/sage List of installed packages
│ └── tmp/sage Temporary files when building Sage
│ ├── lib/sage
│ │ ├── installed/
│ │ │ Records of installed non-Python packages
│ │ ├── scripts/ Scripts for uninstalling installed packages
│ │ └── venv-python3.9 (SAGE_VENV)
│ │ │ Installation hierarchy (virtual environment)
│ │ │ for Python packages
│ │ ├── bin/ Executables and installed scripts
│ │ ├── lib/python3.9/site-packages/
│ │ │ Python modules/packages are installed here
│ │ └── var/lib/sage/
│ │ └── wheels/
│ │ Python wheels for all installed Python packages
│ │
│ └── tmp/sage/ Temporary files when building Sage
├── logs
│ ├── dochtml.log Log of the documentation build
│ ├── install.log Full install log
│ └── pkgs Build logs of individual packages
│ ├── alabaster-0.7.12.log
│ …
│ └── zn_poly-0.9.2.log
├── m4 M4 macros for configure
├── m4 M4 macros for generating the configure script
│ └── *.m4
├── Makefile Running "make" uses this file
├── prefix -> SAGE_LOCAL Convenience symlink to the installation tree
├── README.md This file
├── sage Script to start Sage
├── src Monolithic Sage library source tree
│ ├── bin Scripts that Sage uses internally
│ ├── doc Sage documentation sources
│ └── sage The Sage library source code
│ ├── bin/ Scripts that Sage uses internally
│ ├── doc/ Sage documentation sources
│ └── sage/ The Sage library source code
├── upstream Source tarballs of packages
│ ├── Babel-2.9.1.tar.gz
│ …
│ └── zn_poly-0.9.2.tar.gz
├── venv -> SAGE_VENV Convenience symlink to the virtual environment
└── VERSION.txt
```
For more details see [our Developer's Guide](https://doc.sagemath.org/html/en/developer/coding_basics.html#files-and-directory-structure).
Expand Down
144 changes: 33 additions & 111 deletions pkgs/sagemath-standard/setup.py
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
#!/usr/bin/env python

from __future__ import print_function

import os
import sys
import time
Expand All @@ -28,21 +26,34 @@
sage.env.SAGE_SRC = os.getcwd()
from sage.env import *

from sage_setup.excepthook import excepthook
sys.excepthook = excepthook

from sage_setup.setenv import setenv
setenv()

#########################################################
### Configuration
#########################################################

if len(sys.argv) > 1 and sys.argv[1] == "sdist":
if len(sys.argv) > 1 and (sys.argv[1] == "sdist" or sys.argv[1] == "egg_info"):
sdist = True
else:
sdist = False

if sdist:
cmdclass = {}
else:
from sage_setup.excepthook import excepthook
sys.excepthook = excepthook

from sage_setup.setenv import setenv
setenv()

from sage_setup.command.sage_build import sage_build
from sage_setup.command.sage_build_cython import sage_build_cython
from sage_setup.command.sage_build_ext import sage_build_ext
from sage_setup.command.sage_install import sage_install_and_clean

cmdclass = dict(build=sage_build,
build_cython=sage_build_cython,
build_ext=sage_build_ext,
install=sage_install_and_clean)

#########################################################
### Testing related stuff
#########################################################
Expand All @@ -52,131 +63,42 @@
if os.path.exists(sage.misc.lazy_import_cache.get_cache_file()):
os.unlink(sage.misc.lazy_import_cache.get_cache_file())


from sage_setup.command.sage_build import sage_build
from sage_setup.command.sage_build_cython import sage_build_cython
from sage_setup.command.sage_build_ext import sage_build_ext


#########################################################
### Discovering Sources
#########################################################

# TODO: This should be quiet by default
print("Discovering Python/Cython source code....")
t = time.time()

from sage_setup.optional_extension import is_package_installed_and_updated

if sdist:
# No need to compute distributions. This avoids a dependency on Cython
# just to make an sdist.
distributions = None
python_packages = []
python_modules = []
cython_modules = []
else:
# TODO: This should be quiet by default
print("Discovering Python/Cython source code....")
t = time.time()
from sage_setup.optional_extension import is_package_installed_and_updated
distributions = ['']
optional_packages_with_extensions = ['mcqd', 'bliss', 'tdlib', 'primecount',
'coxeter3', 'fes', 'sirocco', 'meataxe']
distributions += ['sagemath-{}'.format(pkg)
for pkg in optional_packages_with_extensions
if is_package_installed_and_updated(pkg)]
log.warn('distributions = {0}'.format(distributions))
from sage_setup.find import find_python_sources
python_packages, python_modules, cython_modules = find_python_sources(
SAGE_SRC, ['sage'], distributions=distributions)

from sage_setup.find import find_python_sources
python_packages, python_modules, cython_modules = find_python_sources(
SAGE_SRC, ['sage'], distributions=distributions)

log.debug('python_packages = {0}'.format(python_packages))

print("Discovered Python/Cython sources, time: %.2f seconds." % (time.time() - t))

log.debug('python_packages = {0}'.format(python_packages))
print("Discovered Python/Cython sources, time: %.2f seconds." % (time.time() - t))

from sage_setup.command.sage_install import sage_install_and_clean

#########################################################
### Distutils
#########################################################

code = setup(
packages = python_packages,
package_data = {
'sage.libs.gap': ['sage.gaprc'],
'sage.interfaces': ['sage-maxima.lisp'],
'sage.doctest': ['tests/*'],
'sage': ['ext_data/*',
'ext_data/kenzo/*',
'ext_data/singular/*',
'ext_data/singular/function_field/*',
'ext_data/images/*',
'ext_data/doctest/*',
'ext_data/doctest/invalid/*',
'ext_data/doctest/rich_output/*',
'ext_data/doctest/rich_output/example_wavefront/*',
'ext_data/gap/*',
'ext_data/gap/joyner/*',
'ext_data/mwrank/*',
'ext_data/notebook-ipython/*',
'ext_data/nbconvert/*',
'ext_data/graphs/*',
'ext_data/pari/*',
'ext_data/pari/dokchitser/*',
'ext_data/pari/buzzard/*',
'ext_data/pari/simon/*',
'ext_data/magma/*',
'ext_data/magma/latex/*',
'ext_data/magma/sage/*',
'ext_data/valgrind/*',
'ext_data/threejs/*']
},
scripts = [## The sage script
'bin/sage',
## Other scripts that should be in the path also for OS packaging of sage:
'bin/sage-eval',
'bin/sage-runtests', # because it is useful for doctesting user scripts too
'bin/sage-fixdoctests', # likewise
'bin/sage-coverage', # because it is useful for coverage-testing user scripts too
'bin/sage-cython', # deprecated, might be used in user package install scripts
## Helper scripts invoked by sage script
## (they would actually belong to something like libexec)
'bin/sage-cachegrind',
'bin/sage-callgrind',
'bin/sage-massif',
'bin/sage-omega',
'bin/sage-valgrind',
'bin/sage-venv-config',
'bin/sage-version.sh',
'bin/sage-cleaner',
## Only makes sense in sage-the-distribution. TODO: Move to another installation script.
'bin/sage-list-packages',
'bin/sage-location',
## Uncategorized scripts in alphabetical order
'bin/math-readline',
'bin/sage-env',
# sage-env-config -- installed by sage_conf
# sage-env-config.in -- not to be installed',
'bin/sage-gdb-commands',
'bin/sage-grep',
'bin/sage-grepdoc',
'bin/sage-inline-fortran',
'bin/sage-ipynb2rst',
'bin/sage-ipython',
'bin/sage-native-execute',
'bin/sage-notebook',
'bin/sage-num-threads.py',
'bin/sage-open',
'bin/sage-preparse',
'bin/sage-python',
'bin/sage-rebase.bat',
'bin/sage-rebase.sh',
'bin/sage-rebaseall.bat',
'bin/sage-rebaseall.sh',
'bin/sage-run',
'bin/sage-run-cython',
'bin/sage-startuptime.py',
'bin/sage-update-src',
'bin/sage-update-version',
],
cmdclass = dict(build=sage_build,
build_cython=sage_build_cython,
build_ext=sage_build_ext,
install=sage_install_and_clean),
cmdclass = cmdclass,
ext_modules = cython_modules)
12 changes: 6 additions & 6 deletions pkgs/sagemath-standard/tox.ini
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,15 @@ envlist =
# Build dependencies according to requirements.txt (all versions fixed).
# Use ONLY the wheels built and stored by the Sage distribution (no PyPI):
#
# ./sage -sh -c '(cd pkgs/sagelib && tox -v -v -v -e python-sagewheels-nopypi)'
# ./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e python-sagewheels-nopypi)'
#
python-sagewheels-nopypi,
#
# Build and test without using the concrete dependencies specified by requirements.txt,
# using the dependencies declared in pyproject.toml and setup.cfg (install-requires) only:
# Still use ONLY the wheels built and stored by the Sage distribution (no PyPI).
#
# ./sage -sh -c '(cd pkgs/sagelib && tox -v -v -v -e python-sagewheels-nopypi-norequirements)'
# ./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e python-sagewheels-nopypi-norequirements)'
#
python-sagewheels-nopypi-norequirements,
#
Expand All @@ -44,7 +44,7 @@ envlist =
# and additionally allow packages from PyPI.
# Because all versions are fixed, we "should" end up using the prebuilt wheels.
#
# ./sage -sh -c '(cd pkgs/sagelib && tox -v -v -v -e python-sagewheels)'
# ./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e python-sagewheels)'
#
python-sagewheels,
#
Expand Down Expand Up @@ -81,8 +81,8 @@ passenv =
PKG_CONFIG_PATH
# Parallel build
SAGE_NUM_THREADS
# SAGE_LOCAL only for finding the wheels
sagewheels: SAGE_LOCAL
# SAGE_VENV only for finding the wheels
sagewheels: SAGE_VENV
# Location of the wheels (needs to include a PEP 503 compliant
# Simple Repository index, i.e., a subdirectory "simple")
sagewheels: SAGE_SPKG_WHEELS
Expand All @@ -92,7 +92,7 @@ setenv =
HOME={envdir}
# We supply pip options by environment variables so that they
# apply both to the installation of the dependencies and of the package
sagewheels: PIP_FIND_LINKS=file://{env:SAGE_SPKG_WHEELS:{env:SAGE_LOCAL:{toxinidir}/../../../../local}/var/lib/sage/wheels}
sagewheels: PIP_FIND_LINKS=file://{env:SAGE_SPKG_WHEELS:{env:SAGE_VENV:{toxinidir}/../../../../venv}/var/lib/sage/wheels}
nopypi: PIP_NO_INDEX=true
# No build isolation for PEP 517 packages - use what is already in the environment
# Note that this pip env "NO" variable uses inverted logic:
Expand Down
28 changes: 21 additions & 7 deletions src/doc/en/developer/coding_basics.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -152,17 +152,31 @@ Then in the file ``SAGE_ROOT/src/sage/all.py``, add a line ::

from sage.measure_theory.all import *

Non-Python Sage source code and supporting files should be placed in
appropriate subdirectories of ``SAGE_ROOT/src/sage/ext_data/``. They will then be
automatically copied to the corresponding subdirectories of
``SAGE_ROOT/local/share/sage/ext/`` during the build process and can be
accessed at runtime using ``SAGE_EXTCODE``. For example, if ``file`` is placed
in ``SAGE_ROOT/src/sage/ext_data/directory/`` it can be accessed with ::
Non-Python Sage source code and supporting files can be included in one
of the following places:

- In the directory of the Python code that uses that file. When the
Sage library is installed, the file will be installed in the same
location as the Python code. For example,
``SAGE_ROOT/src/sage/interfaces/maxima.py`` needs to use the file
``SAGE_ROOT/src/sage/interfaces/maxima.lisp`` at runtime, so it refers
to it as ::

os.path.join(os.path.dirname(__file__), 'sage-maxima.lisp')

- In an appropriate subdirectory of ``SAGE_ROOT/src/sage/ext_data/``.
(At runtime, it is then available in the directory indicated by
``SAGE_EXTCODE``). For example, if ``file`` is placed in
``SAGE_ROOT/src/sage/ext_data/directory/`` it can be accessed with ::

from sage.env import SAGE_EXTCODE
file = os.path.join(SAGE_EXTCODE, 'directory', 'file')

``SAGE_EXTCODE`` is used because not all distributions have ``SAGE_ROOT``.
In both cases, the files must be listed (explicitly or via wildcards) in
the section ``options.package_data`` of the file
``SAGE_ROOT/pkgs/sagemath-standard/setup.cfg.m4`` (or the corresponding
file of another distribution).



Learn by copy/paste
Expand Down
8 changes: 8 additions & 0 deletions src/doc/en/developer/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,14 @@ Sage Coding Details
coding_in_cython
coding_in_other

Packaging the Sage Library
--------------------------

.. toctree::
:maxdepth: 3

packaging_sage_library

Packaging Third-Party Code
--------------------------

Expand Down
Loading

0 comments on commit af47b5f

Please sign in to comment.