Add cmr (Combinatorial Matrix Recognition library) and Cython interface

[mkoeppe]

📚 Description

• Adding an SPKG for the Combinatorial Matrix Recognition Library (successor of TUtest) by Matthias Walter
  https://discopt.github.io/cmr/, and dependencies
• Cython interface in sage.libs.cmr
• New matrix backend class Matrix_cmr_chr_sparse
• Classes for decomposition nodes (CMR_DEC_...)
• New methods for Matrix_integer_dense along the lines of the existing methods smith_form etc.; delegate to Matrix_cmr_chr_sparse methods
• Definitions of all terms
• Cite literature
• Examples and tests for all methods/classes
• Link to CMR documentation - https://discopt.github.io/cmr/ (maybe use Package breathe - Sphinx plugin for project documentation using Doxygen #37577)
• Methods for matroids
• Subclasses of Matroid corresponding node types of the Seymour decomposition
• Connect to modular decomposition of graphs
• Connect to sage.graphs.connectivity.spqr_tree?
• Matrix class that is backed by a DecompositionNode
• Graph backend class
• Wrap remaining functions of CMR
• pip-installable package sagemath-cmr (depending on sagemath-modules, Modularization of sagelib: Distributions sagemath-{brial,combinat,eclib,flint,gap,giac,glpk,graphs,groups,homfly,lcalc,libbraiding,libecm,linbox,modules,mpmath,ntl,pari,plot,polyhedra,schemes,singular,standard-no-symbolics,symbolics} #35095)

Follow-ups:

• sage.tensor.modules: Add methods is_unimodular etc. #38139

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

• Upstream: cmr/dec.h API cannot distinguish CMR_DEC_IRREGULAR, CMR_DEC_UNKNOWN, CMR_DEC_SERIES_PARALLEL discopt/cmr#32
• Depends on FiniteRankFreeModule.isomorphism_with_fixed_basis: Make it an actual isomorphism #37512
• Depends on MatrixSpace: Support constructing Hom(CombinatorialFreeModule) #37514
• Depends on Make homsets of FiniteDimensionalModulesWithBasis finite-dimensional #37516
• Depends on Border matrix #37606
• Depends on matrix, Graph.incidence_matrix, LinearMatroid.representation: Support constructing Hom(CombinatorialFreeModule) elements #37692
• Depends on Update conda-lock files, rename macOS conda-lock files to match CI Conda, remove unexplained duplicate lock files #37998 (merged here)

@xuluze @jsantillan3 @discopt

[mkoeppe]

Does not build yet; discopt/cmr#29

[mkoeppe]

In -minimal configurations, our boost is too old or too cropped

  [cmr-git]   /sage/local/var/tmp/sage/build/cmr-git/src/src/cmr/matroid.hpp:7:10: fatal error: boost/numeric/ublas/matrix.hpp: No such file or directory
  [cmr-git]    #include <boost/numeric/ublas/matrix.hpp>
  [cmr-git]             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  [cmr-git]   compilation terminated.

[github-actions]

Documentation preview for this PR (built with commit 8341677; changes) is ready! 🎉

[mkoeppe]

Rebased away from the boost upgrade, updated CMR to latest

[mkoeppe]

I've adapted the interface to the new CMR version.

There are a few doctest failures that I don't understand

sage -t --random-seed=129552849186103186121087193282398427711 src/sage/matrix/matrix_cmr_sparse.pyx
**********************************************************************
File "src/sage/matrix/matrix_cmr_sparse.pyx", line 734, in sage.matrix.matrix_cmr_sparse.Matrix_cmr_chr_sparse.modulus
Failed example:
    M.modulus()
Expected:
    1
Got:
    <BLANKLINE>
**********************************************************************
File "src/sage/matrix/matrix_cmr_sparse.pyx", line 789, in sage.matrix.matrix_cmr_sparse.Matrix_cmr_chr_sparse.strong_modulus
Failed example:
    M.strong_modulus()
Expected:
    1
Got:
    <BLANKLINE>
**********************************************************************
File "src/sage/matrix/matrix_cmr_sparse.pyx", line 837, in sage.matrix.matrix_cmr_sparse.Matrix_cmr_chr_sparse.is_k_modular
Failed example:
    M.is_k_modular(1)
Expected:
    True
Got:
    False
**********************************************************************
File "src/sage/matrix/matrix_cmr_sparse.pyx", line 876, in sage.matrix.matrix_cmr_sparse.Matrix_cmr_chr_sparse.is_strongly_k_modular
Failed example:
    M.is_strongly_k_modular(1)
Expected:
    True
Got:
    False
**********************************************************************

[mkoeppe]

Also I haven't done anything yet on the Python side about the new terminology "equimodular" that CMR now uses.

[xuluze]

I have encountered the following underflow issue in the parent_rows_and_columns function for a children node.

sage: from sage.matrix.matrix_cmr_sparse import Matrix_cmr_chr_sparse
sage: R12_large = Matrix_cmr_chr_sparse(MatrixSpace(ZZ, 9, 12, sparse=True), [[1, -1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1], 
....: [0, 0, 0, 1, -1, 0, 0, 0, 1 , 1, 1, 1], 
....: [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1], 
....: [ 1,  0,  1,  0,  0,  0,  0,  0,  1,  1,  0,  0],
....: [ 0,  1,  1,  0,  0,  0,  0,  0,  0,  0, -1, -1],
....: [ 0,  0,  0,  1,  0,  1,  0,  0,  1,  1,  0,  0],
....: [ 0,  0,  0,  0,  1,  1,  0,  0,  0,  0, -1, -1],
....: [ 0,  0,  0,  0,  0,  0,  1,  0,  1,  0,  1,  0],
....: [ 0,  0,  0,  0,  0,  0,  0,  1,  0,  1,  0,  1]])
sage: result, certificate = R12_large.is_totally_unimodular(certificate=True)
sage: certificate._children()[0].parent_rows_and_columns()
((2, 7, 8, 0, 3), (6, 7, 8, 9, 10, 11, 18446744073709551615))

Any ideas for how to fix it? Thank you!

[mkoeppe]

I see the same on my machine

[discopt]

I'll can have a look. At least I cannot reproduce it standalone by reading from a file.

[mkoeppe]

this number is uint64_t(-1)

[discopt]

I have encountered the following underflow issue in the parent_rows_and_columns function for a children node.

sage: from sage.matrix.matrix_cmr_sparse import Matrix_cmr_chr_sparse
sage: R12_large = Matrix_cmr_chr_sparse(MatrixSpace(ZZ, 9, 12, sparse=True), [[1, -1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1], 
....: [0, 0, 0, 1, -1, 0, 0, 0, 1 , 1, 1, 1], 
....: [0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1], 
....: [ 1,  0,  1,  0,  0,  0,  0,  0,  1,  1,  0,  0],
....: [ 0,  1,  1,  0,  0,  0,  0,  0,  0,  0, -1, -1],
....: [ 0,  0,  0,  1,  0,  1,  0,  0,  1,  1,  0,  0],
....: [ 0,  0,  0,  0,  1,  1,  0,  0,  0,  0, -1, -1],
....: [ 0,  0,  0,  0,  0,  0,  1,  0,  1,  0,  1,  0],
....: [ 0,  0,  0,  0,  0,  0,  0,  1,  0,  1,  0,  1]])
sage: result, certificate = R12_large.is_totally_unimodular(certificate=True)
sage: certificate._children()[0].parent_rows_and_columns()
((2, 7, 8, 0, 3), (6, 7, 8, 9, 10, 11, 18446744073709551615))

This is exactly the issue of the 3-sum: the submatrix indexed by these rows and columns is (excluding the last one):

 1  1  1  1  1  1
 1  0  1  0  1  0
 0  1  0  1  0  1
 0  0  1  1  1  1
 0  0  1  1  0  0

The SIZE_MAX entry for the last column indicates the artificial column with two 1s in the last rows and 0s otherwise (over F_2):

 1  1  1  1  1  1 0
 1  0  1  0  1  0 0
 0  1  0  1  0  1 0
 0  0  1  1  1  1 1
 0  0  1  1  0  0 1

The second matrix has a SIZE_MAX row as the first row, which means that the first two are 1s and 0s everywhere else (and that this row is not part of the input matrix).

1  1  0  0  0  0  0  0 
1  1  1 -1  0  0  0  0 
1  1  0  0  0  1 -1  0 
1  0  1  0  1  0  0  0 
0 -1  0  1  1  0  0  0 
1  0  0  0  0  1  0  1 
0 -1  0  0  0  0  1  1 

I'm very open to changing this behavior, so we can have a discussion offline.