Meta-ticket: Clear out __init__.py files in preparation for namespace packages

[mkoeppe]

(Part of the modularization effort, #29705)

Whenever there are two distribution packages that provide modules with a common prefix of Python packages, that prefix needs to be a native namespace package, i.e., there cannot be an __init__.py file.

For example,

• sagemath-tdlib will provide sage.graphs.graph_decompositions.tdlib,
• sagemath-rw will provide sage.graphs.graph_decompositions.rankwidth,
• sagemath-graphs will provide all of the rest of sage.graphs.graph_decompositions (and most of sage.graphs).
  Then, none of
• sage,
• sage.graphs,
• sage.graphs.graph_decomposition
  can be an ordinary Python package (with __init__.py file), but rather each of them has to be a native namespace package (no __init__.py file).

In this meta-ticket, we make sure that all such __init__.py files are empty. (The actual removal of empty __init__.py files will be done in #28925.)

Not all __init__.py files need to be empty. For example,

• all of sage.cpython and sage.structure (after Move sage.structure.formal_sum to sage.modules #29870) is intended to be in one distribution package, sagemath-objects. Hence sage.cpython and sage.structure keep their nonempty __init__.py files, and we may even add a comment that documents that they are intended to stay ordinary packages. (Remove some monkey patching in src/sage/__init__.py #31420 clears sage.__init__ and adds such a comment to sage.cpython.__init__)
• sage.libs.giac is intended to be in one distribution package, sagemath-giac. Hence sage.libs.giac will keep its nonempty __init__.py file (but sage.libs will not).

With positively-reviewed tickets #31420/#32479/#32506/#32507

$ find src/sage -name '__init__.py' | xargs wc -l | grep -v '^ *0'
       2 src/sage/crypto/__init__.py
       3 src/sage/crypto/mq/__init__.py
       3 src/sage/dynamics/__init__.py
       3 src/sage/dynamics/cellular_automata/__init__.py
      42 src/sage/combinat/words/__init__.py
      38 src/sage/combinat/designs/__init__.py            #32508
      13 src/sage/combinat/chas/__init__.py               #32508
      11 src/sage/combinat/path_tableaux/__init__.py      #32508
      49 src/sage/combinat/__init__.py                    #32508
      10 src/sage/combinat/cluster_algebra_quiver/__init__.py  #32508
      51 src/sage/combinat/species/__init__.py            #32508
      31 src/sage/combinat/sf/__init__.py                 #32508
      29 src/sage/combinat/crystals/__init__.py           #32508
      10 src/sage/combinat/ncsym/__init__.py              #32508
      35 src/sage/combinat/posets/__init__.py             #32508
      11 src/sage/combinat/ncsf_qsym/__init__.py          #32508
       7 src/sage/combinat/integer_lists/__init__.py            # OK, 1 distro
      36 src/sage/combinat/rigged_configurations/__init__.py  #32508
     127 src/sage/combinat/root_system/__init__.py        #32508, #32840
       9 src/sage/combinat/matrices/__init__.py
       3 src/sage/doctest/__init__.py                           # OK, 1 distro sagemath-repl
     826 src/sage/features/__init__.py                          # OK, 1 distro
       4 src/sage/repl/rich_output/__init__.py                  # OK, 1 distro sagemath-repl
      39 src/sage/__init__.py                             #32489
     359 src/sage/libs/giac/__init__.py                         # OK, 1 distro sagemath-giac
       2 src/sage/libs/ntl/__init__.py                          # OK, 1 distro
       1 src/sage/libs/gap/__init__.py                          # OK, 1 distro
     219 src/sage/libs/pari/__init__.py                         # OK, 1 distro
      55 src/sage/cpython/__init__.py                           # OK, 1 distro sagemath-objects
       2 src/sage/sat/converters/__init__.py
       4 src/sage/sat/solvers/__init__.py
       2 src/sage/matroids/__init__.py
     114 src/sage/rings/polynomial/pbori/__init__.py            # OK, 1 distro sagemath-brial
       2 src/sage/rings/polynomial/__init__.py            #32509
      10 src/sage/modules/with_basis/__init__.py                # OK, 1 distro sagemath-categories
       2 src/sage/structure/__init__.py                         # OK, 1 distro sagemath-objects
      29 src/sage/categories/__init__.py                  #32508
       2 src/sage/media/__init__.py                             # deprecated

Tickets:

• Remove some monkey patching in src/sage/__init__.py #31420/Remove monkey patching of inspect.isfunction in sage.__init__ #32479/Move load_ipython_extension from sage.__init__ to sage.repl.__init__ #32489 sage
• Remove useless 'file is not empty' comments from __init__.py files #32506 Remove useless 'file is not empty' comments from __init__.py files
• Remove useless 'from . import all' from some __init__.py files #32507 Remove useless 'from . import all' and similar from some __init__.py files
• Eliminate use of __init__.py files for supplying package docstrings #32508 Eliminate use of __init__.py files for supplying package docstrings
• Clear out __init__.py in sage.rings.polynomial #32509 Clear out __init__.py in sage.rings.polynomial
• Fix unpickle_global when register_unpickle_override appears in a module not loaded at startup #32840 Fix unpickle_global when register_unpickle_override appears in a module not loaded at startup

Related:

• Distributions sagemath-{flint-arb,homfly,giac,gap}, add "sage_setup: distribution" to Cython modules #30666 Add "sage_setup: distribution" directives to all Cython modules needing external libraries -- related because the dependencies on C/C++ libraries guide the layout of the namespace packages.

CC: @tscrim @pjbruin @kiwifb @kliem @dcoudert

Component: refactoring

Reviewer: Michael Orlitzky

Issue created by migration from https://trac.sagemath.org/ticket/32501

[mkoeppe]

comment:6

A serious case:

$ cat src/sage/rings/polynomial/__init__.py
# Resolve a cyclic import
import sage.rings.convert.mpfi

We need to get rid of this one -- sage.rings.polynomial must become a namespace package

[mkoeppe]

comment:9

Replying to @mkoeppe:

A serious case:

$ cat src/sage/rings/polynomial/__init__.py
# Resolve a cyclic import
import sage.rings.convert.mpfi

We need to get rid of this one -- sage.rings.polynomial must become a namespace package

This one was added it #25848 ("Some preparations for binding=True") for a stalled effort in #22747/#26254

[mkoeppe]

comment:11

We can close this meta-ticket because it only tracks one open ticket.

[tscrim]

comment:12

I think it would be good to keep it open until the final ticket is done.

[orlitzky]

Author: Michael Orlitzky

[orlitzky]

comment:18

Everything in the "Tickets:" list has been closed: fixed.

[orlitzky]

Reviewer: Michael Orlitzky

[orlitzky]

Changed author from Michael Orlitzky to none