Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong result with tdlib for treewidth #38159

Closed
2 tasks done
dcoudert opened this issue Jun 6, 2024 · 19 comments · Fixed by #38214
Closed
2 tasks done

Wrong result with tdlib for treewidth #38159

dcoudert opened this issue Jun 6, 2024 · 19 comments · Fixed by #38214

Comments

@dcoudert
Copy link
Contributor

dcoudert commented Jun 6, 2024

Steps To Reproduce

sage: from sage.graphs.graph_decompositions.tree_decomposition import width_of_tree_decomposition
sage: G = Graph('q~~|~q{mLhUo}?v?EO`{?h`?wo@w?fo?As_?lG?Cp_?Uc??{?_?|???Y_G?@gH??{?_?A{???Dp???Dk???Ax????l????Dh????Qo???GkG???_lG????Q_???I@p?G???uO????As????@@gH????FG??A??AW_??A??d????CG@w?C????PE?@????D@G?C???t?O??????')
sage: G.treewidth(algorithm='sage')
5
sage: G.treewidth(algorithm='tdlib')
6
sage: T = G.treewidth(algorithm='tdlib', certificate=True)
sage: width_of_tree_decomposition(G, T)
6

Expected Behavior

sage: G.treewidth(algorithm='tdlib')
5

Actual Behavior

sage: G.treewidth(algorithm='tdlib')
6

Additional Information

No response

Environment

- **OS**: macOS 12.7.5 and fedora 39
- **Sage Version**: 10.4.beta8

Checklist

  • I have searched the existing issues for a bug report that matches the one I want to file, without success.
  • I have read the documentation and troubleshoot guide
@dcoudert
Copy link
Contributor Author

dcoudert commented Jun 6, 2024

Another example with only 10 nodes

sage: G = Graph('I~~}vPlr_')
sage: G.treewidth(algorithm='sage')
5
sage: G.treewidth(algorithm='tdlib')
6

@dimpase
Copy link
Member

dimpase commented Jun 6, 2024

@felix-salfelder

@felix-salfelder
Copy link

I'm not familiar with the compressed string representation. Please remind me how to convert 'I~~}vPlr_' into a graph.

Which version of tdlib (aka treedec) is this, and which algorithm does it use?

Thanks

@mkoeppe
Copy link
Contributor

mkoeppe commented Jun 6, 2024

See also:

@felix-salfelder
Copy link

I figured that 'I~~}vPlr_' has 10 vertices and 32 edges.

It looks like this in gr format

p tw 10 32
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 10
2 3
2 4
2 5
2 6
2 7
2 8
2 9
2 10
3 4
3 5
3 6
3 8
3 9
4 5
4 6
4 7
5 6
5 7
5 8
5 9
5 10
6 9
6 10
7 10

With this input, tdecomp --ex17 (develop branch) gives me

s td 9 6 10
b 1 1 5
b 2 1 2 5
b 3 1 2 5 7
b 4 1 2 4 5 7
b 5 1 2 4 5 7 10
b 6 1 2 4 5 6 10
b 7 1 2 3 4 5 6
b 8 1 2 3 5 8
b 9 2 3 5 6 9
1 2
2 3
3 4
4 5
5 6
6 7
7 8
7 9

Similar decompositions also from fi, ppfi and bmd (all with tw 5, if tw is the size of a biggest bag minus 1).

I can't say much about the cython sage integration, and obviously the results may depend on vertex order and such things. Do you actually get a suboptimal decomposition? Which one?

@dcoudert
Copy link
Contributor Author

dcoudert commented Jun 7, 2024

The graph has 10 vertices and 33 edges

p tw 10 33
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 10
2 3
2 4
2 5
2 6
2 7
2 8
2 9
2 10
3 4
3 5
3 6
3 8
3 9
4 5
4 6
4 7
5 6
5 7
5 8
5 9
5 10
6 9
6 10
7 10
8 9

and we get the decomposition:

s td 4 7 10
b 1 1 2 5 6 7 10
b 2 1 2 3 4 5 6 7
b 3 1 2 3 5 6 8
b 4 2 3 5 6 8 9
1 2
2 3
3 4

We clearly have to use a newer version of tdlib. We currently call a former function sage_exact_decomposition from version 0.3.1.p0. We will have to figure out how to use the current interfaces.

@felix-salfelder
Copy link

felix-salfelder commented Jun 7, 2024 via email

@dimpase
Copy link
Member

dimpase commented Jun 7, 2024

On Thu, Jun 06, 2024 at 11:33:16PM -0700, David Coudert wrote:

The graph has 10 vertices and 33 edges

Thanks, interesting. I used a tool "showg" to decode the string. Then
hand edited the output... It had to go wrong.

The current algorithms also work on the 33 edge graph, but I havent
tried the dated exact decomposition (cutset I suppose), as it's not
enabled in tdecomp.

We clearly have to use a newer version of tdlib. We currently call a
former function sage_exact_decomposition from version 0.3.1.p0. We
will have to figure out how to use the current interfaces.

The new python bindings need work. Perhaps it should be split out, and
become more "pythonic", with a dependency on libtreedec provided by the
main package. Time will tell, suggestions welcome.

The best would be to have a Python package on PyPI, dependent on libtreedec. Then it would become the only point of entry to tdlib in Sage, simplifying things.

@felix-salfelder
Copy link

felix-salfelder commented Jun 7, 2024 via email

@mkoeppe
Copy link
Contributor

mkoeppe commented Jun 7, 2024

We clearly have to use a newer version of tdlib.

vbraun pushed a commit to vbraun/sage that referenced this issue Jun 10, 2024
    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

- FIxes sagemath#30813
- Fixes sagemath#38159

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->

- Depends on sagemath#38144 (merged here for testing)
    
URL: sagemath#38163
Reported by: Matthias Köppe
Reviewer(s): David Coudert
@dimpase
Copy link
Member

dimpase commented Jun 10, 2024

On Fri, Jun 07, 2024 at 05:09:40AM -0700, Dima Pasechnik wrote: The best would be to have a Python package on PyPI, dependent on libtreedec. Then it would become the only point of entry to tdlib in Sage, simplifying things.
If I understand correctly, we should eventually build a shared library.
[...]

The question is whether to use Cython, or instead use Boost/Python or pybind11. The latter is a Sage package, and is used in scipy, contourpy, etc. It's said to be lighter and easier that Boost/Python.

@dimpase
Copy link
Member

dimpase commented Jun 10, 2024

@dcoudert ##38163 should have a test showing that the error here is tested. Can you please communicate this,

@dimpase
Copy link
Member

dimpase commented Jun 10, 2024

The question is whether to use Cython, or instead use Boost/Python or pybind11.

one can also can take Cython/C++ code in Sage and modify it appropriately.

@dimpase
Copy link
Member

dimpase commented Jun 10, 2024

@dcoudert ##38163 should have a test showing that the error here is tested. Can you please communicate this,

it's actually does not fix the issue here, according to my testing.

@dcoudert
Copy link
Contributor Author

@dcoudert ##38163 should have a test showing that the error here is tested. Can you please communicate this,

it's actually does not fix the issue here, according to my testing.

Have you tried running make sagemath_tdlib ? it's working for me on macOS and fedora 39...

In fact, I'm missing a proper documentation on how to install each of our optional packages. I have the filling that we have many ways and that we should sometimes favor one of them, but not always the sage depending on the package...

@dimpase
Copy link
Member

dimpase commented Jun 10, 2024

make sagemath_tdlib is not working for me. I get a rather obscure error:

...
[tdlib-0.9.3.p0] Finished installing tdlib-0.9.3.p0
make --no-print-directory sagemath_environment-SAGE_VENV-no-deps
[sagemath_environment-10.4.beta8] Setting up build directory /mnt/opt/Sage/sage-clang/local/var/lib/sage/venv-python3.11/var/tmp/sage/build/sagemath_environment-10.4.beta8
[sagemath_environment-10.4.beta8] Host system: Linux hilbert 6.0.8-gentoo-x86_64 #2 SMP PREEMPT_DYNAMIC Sun Dec 18 22:31:12 GMT 2022 x86_64 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz GenuineIntel GNU/Linux
[sagemath_environment-10.4.beta8] C compiler: gcc, Using built-in specs., COLLECT_GCC=gcc, COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/13/lto-wrapper, Target: x86_64-pc-linux-gnu, Configured with: /var/tmp/portage/sys-devel/gcc-13.2.1_p20240210/work/gcc-13-20240210/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/13 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/13/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/13 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/13/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/13/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13 --disable-silent-rules --disable-dependency-tracking --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/13/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --disable-libunwind-exceptions --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 13.2.1_p20240210 p14' --with-gcc-major-version-only --enable-libstdcxx-time --enable-lto --disable-libstdcxx-pch --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --disable-multilib --with-multilib-list=m64 --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libssp --disable-libada --disable-cet --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --without-zstd --without-isl --enable-default-pie --enable-default-ssp --disable-fixincludes --with-build-config=bootstrap-O3, Thread model: posix, Supported LTO compression algorithms: zlib, gcc version 13.2.1 20240210 (Gentoo 13.2.1_p20240210 p14)
[sagemath_environment-10.4.beta8] No stamp file for package 'sagemath_environment' in /mnt/opt/Sage/sage-clang/local/var/lib/sage/venv-python3.11/var/lib/sage/installed
[sagemath_environment-10.4.beta8] No spkg-legacy-uninstall script; nothing to do
[sagemath_environment-10.4.beta8] [spkg-install] * Creating isolated environment: venv+pip...
[sagemath_environment-10.4.beta8] [spkg-install] * Installing packages in isolated environment:
[sagemath_environment-10.4.beta8] [spkg-install]   - setuptools >= 68.1.1
[sagemath_environment-10.4.beta8] [spkg-install]   - wheel >=0.36.2
[sagemath_environment-10.4.beta8] [spkg-install] > /mnt/opt/Sage/sage-clang/local/var/lib/sage/venv-python3.11/bin/python3 -m
[sagemath_environment-10.4.beta8] [spkg-install]   pip --python /tmp/build-env-_ris87b1/bin/python install --use-pep517 --no-
[sagemath_environment-10.4.beta8] [spkg-install]   warn-script-location --no-compile -r /tmp/build-reqs-_yylu9p_.txt
[sagemath_environment-10.4.beta8] [spkg-install] < Looking in links: file:///mnt/opt/Sage/sage-clang/local/var/lib/sage/venv-
[sagemath_environment-10.4.beta8] [spkg-install]   python3.11/var/lib/sage/wheels
[sagemath_environment-10.4.beta8] [spkg-install] < ERROR: Could not find a version that satisfies the requirement wheel>=0.36.2
[sagemath_environment-10.4.beta8] [spkg-install]   (from versions: none)
[sagemath_environment-10.4.beta8] [spkg-install] < ERROR: No matching distribution found for wheel>=0.36.2
[sagemath_environment-10.4.beta8] [spkg-install] 
[sagemath_environment-10.4.beta8] [spkg-install] Traceback (most recent call last):
[sagemath_environment-10.4.beta8] [spkg-install]   File "/usr/lib/python3.11/site-packages/build/__main__.py", line 178, in _handle_build_error
[sagemath_environment-10.4.beta8] [spkg-install]     yield
[sagemath_environment-10.4.beta8] [spkg-install]   File "/usr/lib/python3.11/site-packages/build/__main__.py", line 429, in main
[sagemath_environment-10.4.beta8] [spkg-install]     built = build_call(
[sagemath_environment-10.4.beta8] [spkg-install]             ^^^^^^^^^^^
[sagemath_environment-10.4.beta8] [spkg-install]   File "/usr/lib/python3.11/site-packages/build/__main__.py", line 268, in build_package_via_sdist
[sagemath_environment-10.4.beta8] [spkg-install]     sdist = _build(isolation, srcdir, outdir, 'sdist', config_settings, skip_dependency_check, installer)
[sagemath_environment-10.4.beta8] [spkg-install]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[sagemath_environment-10.4.beta8] [spkg-install]   File "/usr/lib/python3.11/site-packages/build/__main__.py", line 170, in _build
[sagemath_environment-10.4.beta8] [spkg-install]     return _build_in_isolated_env(srcdir, outdir, distribution, config_settings, installer)
[sagemath_environment-10.4.beta8] [spkg-install]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[sagemath_environment-10.4.beta8] [spkg-install]   File "/usr/lib/python3.11/site-packages/build/__main__.py", line 135, in _build_in_isolated_env
[sagemath_environment-10.4.beta8] [spkg-install]     env.install(builder.build_system_requires)
[sagemath_environment-10.4.beta8] [spkg-install]   File "/usr/lib/python3.11/site-packages/build/env.py", line 136, in install
[sagemath_environment-10.4.beta8] [spkg-install]     self._env_backend.install_requirements(requirements)
[sagemath_environment-10.4.beta8] [spkg-install]   File "/usr/lib/python3.11/site-packages/build/env.py", line 265, in install_requirements
[sagemath_environment-10.4.beta8] [spkg-install]     run_subprocess(cmd)
[sagemath_environment-10.4.beta8] [spkg-install]   File "/usr/lib/python3.11/site-packages/build/_ctx.py", line 71, in run_subprocess
[sagemath_environment-10.4.beta8] [spkg-install]     subprocess.run(cmd, capture_output=True, check=True, env=env)
[sagemath_environment-10.4.beta8] [spkg-install]   File "/usr/lib/python3.11/subprocess.py", line 571, in run
[sagemath_environment-10.4.beta8] [spkg-install]     raise CalledProcessError(retcode, process.args,
[sagemath_environment-10.4.beta8] [spkg-install] subprocess.CalledProcessError: Command '['/mnt/opt/Sage/sage-clang/local/var/lib/sage/venv-python3.11/bin/python3', '-m', 'pip', '--python', '/tmp/build-env-_ris87b1/bin/python', 'install', '--use-pep517', '--no-warn-script-location', '--no-compile', '-r', '/tmp/build-reqs-_yylu9p_.txt']' returned non-zero exit status 1.
[sagemath_environment-10.4.beta8] [spkg-install] 
[sagemath_environment-10.4.beta8] [spkg-install] ERROR Command '['/mnt/opt/Sage/sage-clang/local/var/lib/sage/venv-python3.11/bin/python3', '-m', 'pip', '--python', '/tmp/build-env-_ris87b1/bin/python', 'install', '--use-pep517', '--no-warn-script-location', '--no-compile', '-r', '/tmp/build-reqs-_yylu9p_.txt']' returned non-zero exit status 1.
[sagemath_environment-10.4.beta8] [spkg-install] ***************************************************************************************************************************************************************************************************
[sagemath_environment-10.4.beta8] [spkg-install] Failure building sdist and wheel
[sagemath_environment-10.4.beta8] [spkg-install] ***************************************************************************************************************************************************************************************************
[sagemath_environment-10.4.beta8] ************************************************************************
[sagemath_environment-10.4.beta8] Error installing package sagemath_environment-10.4.beta8
[sagemath_environment-10.4.beta8] ************************************************************************
[sagemath_environment-10.4.beta8] Please email sage-devel (http://groups.google.com/group/sage-devel)
[sagemath_environment-10.4.beta8] explaining the problem and including the log files
[sagemath_environment-10.4.beta8]   /mnt/opt/Sage/sage-clang/logs/pkgs/sagemath_environment-10.4.beta8.log
[sagemath_environment-10.4.beta8] and
[sagemath_environment-10.4.beta8]   /mnt/opt/Sage/sage-clang/config.log
[sagemath_environment-10.4.beta8] Describe your computer, operating system, etc.
[sagemath_environment-10.4.beta8] If you want to try to fix the problem yourself, *don't* just cd to
[sagemath_environment-10.4.beta8] /mnt/opt/Sage/sage-clang/local/var/lib/sage/venv-python3.11/var/tmp/sage/build/sagemath_environment-10.4.beta8 and type 'make' or whatever is appropriate.
[sagemath_environment-10.4.beta8] Instead, the following commands setup all environment variables
[sagemath_environment-10.4.beta8] correctly and load a subshell for you to debug the error:
[sagemath_environment-10.4.beta8]   (cd '/mnt/opt/Sage/sage-clang/local/var/lib/sage/venv-python3.11/var/tmp/sage/build/sagemath_environment-10.4.beta8' && '/mnt/opt/Sage/sage-clang/sage' --buildsh)
[sagemath_environment-10.4.beta8] When you are done debugging, you can type "exit" to leave the subshell.
[sagemath_environment-10.4.beta8] ************************************************************************
make[4]: *** [Makefile:3611: sagemath_environment-SAGE_VENV-no-deps] Error 1
make[3]: *** [Makefile:3611: /mnt/opt/Sage/sage-clang/local/var/lib/sage/venv-python3.11/var/lib/sage/installed/sagemath_environment-10.4.beta8] Error 2
make[2]: *** [Makefile:3067: all-build] Error 2
make[2]: Leaving directory '/mnt/opt/Sage/sage-clang/build/make'
***************************************************************
Error building Sage.

Also, a clean install

./configure --enable-system-site-packages --enable-sagemath_tdlib && make build

errors out with the same error.
This probably means that sagemath_environment has broken --enable-system-site-packages.
@orlitzky

PS. the error is actually not so obscure - there is a lookup happening for wheels for setuptools and wheel - something that is almost sure to fail if the latter come from the system.

@dimpase
Copy link
Member

dimpase commented Jun 10, 2024

One way or the other, #38163 should add the test from the example here.

@dcoudert
Copy link
Contributor Author

I agree (#38163 (comment))

@mkoeppe
Copy link
Contributor

mkoeppe commented Jun 10, 2024

Just open a PR for that please.

dimpase added a commit to dimpase/sage that referenced this issue Jun 13, 2024
vbraun pushed a commit to vbraun/sage that referenced this issue Jun 16, 2024
    
This will fix sagemath#38159

Add a missing test

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ x] The title is concise and informative.
- [ x] The description explains in detail what this PR is about.
- [ x] I have linked a relevant issue or discussion.
- [ x] I have created tests covering the changes.
- [  ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
sagemath#38163
    
URL: sagemath#38214
Reported by: Dima Pasechnik
Reviewer(s): David Coudert
vbraun pushed a commit to vbraun/sage that referenced this issue Jun 16, 2024
    
This will fix sagemath#38159

Add a missing test

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ x] The title is concise and informative.
- [ x] The description explains in detail what this PR is about.
- [ x] I have linked a relevant issue or discussion.
- [ x] I have created tests covering the changes.
- [  ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
sagemath#38163
    
URL: sagemath#38214
Reported by: Dima Pasechnik
Reviewer(s): David Coudert
vbraun pushed a commit to vbraun/sage that referenced this issue Jun 17, 2024
    
This will fix sagemath#38159

Add a missing test

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ x] The title is concise and informative.
- [ x] The description explains in detail what this PR is about.
- [ x] I have linked a relevant issue or discussion.
- [ x] I have created tests covering the changes.
- [  ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
sagemath#38163
    
URL: sagemath#38214
Reported by: Dima Pasechnik
Reviewer(s): David Coudert
@mkoeppe mkoeppe added this to the sage-10.4 milestone Jul 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants