Basic setup for Python interface using SWIG #216

WagnerMarcos · 2021-07-27T19:12:51Z

Signed-off-by: Marcos Wagner [email protected]

🎉 New feature

Summary

Basic setup for a Python interface using SWIG to address #210 . (Related to #101) This creates a python module with the cpp classes with an associated .i file. To achieve dual interface output with Python and Ruby, the ruby.i and the python.i used at compilation had to be moved into separate subdirectories, to avoid collisions between the auxiliary files created by SWIG for Python and Ruby.

Test it

To test this interface

Build ign-math with colcon
Run export PYTHONPATH=/ws/install/lib/python
Open a Python interactive session running python3.
Inside of the interactive session run import ignition.math.
Library inside python ready to use.

Checklist

Signed all commits for DCO
Added tests
Added example and/or tutorial
Updated documentation (as needed)
Updated migration guide (as needed)
codecheck passed (See contributing)
All tests passed (See test coverage)
While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge

Signed-off-by: Marcos Wagner <[email protected]>

Signed-off-by: Franco Cipollone <[email protected]>

francocipollone · 2021-07-28T04:14:33Z

Some of the changes I've pushed:

Install path was modified to install math.py in install folder which simplifies PYTHONPATH setting up. 3e31a3a
Module was renamed from pymath to math. 3e31a3a
Examples were added for Angle, Vector2, and Vector3 py classes. f0ad4df
Fixes ruby_TEST after moving ruby.i to a subfolder fab308e
The configuration using Swig 3(bionic) and Swig 4(focal) is slightly different. Added support for both, however if we want to target focal it would be a bit simpler. 5fa9264

codecov · 2021-07-28T04:28:21Z

Codecov Report

Merging #216 (e6373fb) into ign-math6 (9519ce7) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           ign-math6     #216   +/-   ##
==========================================
  Coverage      99.21%   99.21%           
==========================================
  Files             65       65           
  Lines           6089     6089           
==========================================
  Hits            6041     6041           
  Misses            48       48

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9519ce7...e6373fb. Read the comment docs.

Signed-off-by: Marcos Wagner <[email protected]>

…on test files to some of the classes that had an associated .i file. Signed-off-by: LolaSegura <[email protected]>

Signed-off-by: Marcos Wagner <[email protected]>

WagnerMarcos · 2021-07-30T18:34:49Z

The interface has been implemented for Python using SWIG for the files that already had Ruby implementation. At the moment we have pushed tests for these files, to match the Ruby implementation.

Something to note is that when running add_test() for Python in the CMakeList.txt the --gtest_output=xml argument was not added to match the Ruby add_test(), because it was not being able to run it. Ruby is using the test-result folder inside the build directory as output for the xml produced, but no files are created when the colcon test cmd is run. Even after these two cases, running colcon test-result outputs the correct amount of passed, errors, failures and skipped tests (including the Python and Ruby ones)

francocipollone

Looks good, I added some comments.

examples/vector2_example.py

src/CMakeLists.txt

src/GaussMarkovProcess_TEST.py

src/Rand_TEST.py

src/Vector2_TEST.py

src/Vector3_TEST.py

src/Vector2_TEST.py

src/python_TEST.py

Signed-off-by: LolaSegura <[email protected]>

francocipollone

LGTM, I left some minor comments.

src/Vector4_TEST.py

src/Vector2_TEST.py

src/CMakeLists.txt

francocipollone · 2021-08-02T15:49:45Z

I've just left some minor comments however It is in a good shape to be reviewed @chapulina.
One comment, there is no a defined style for python in the ignition contribution, so let us know if we must align to a different style.

Signed-off-by: LolaSegura <[email protected]>

scpeters · 2021-08-13T07:40:45Z

I think the following will fix the ABI checker: gazebo-tooling/release-tools#497

src/CMakeLists.txt

francocipollone · 2021-08-16T17:07:48Z

src/CMakeLists.txt

+
+  set(CMAKE_SWIG_OUTDIR "${CMAKE_BINARY_DIR}/lib/python")
+  if(CMAKE_VERSION VERSION_GREATER 3.8.0)
+    SWIG_ADD_LIBRARY(${SWIG_PY_LIB} LANGUAGE python SOURCES python/python.i ${sources})


We are recompiling sources instead of just linking, for both python and ruby bindings.

Solved at 0ce31c6

It fails now, it's like now it can't find libignition-math6.so but I couldn't replicate this locally, locally it works. Will check

I can reproduce the problem on my machine. I'm looking into it

scpeters

the test failure in the GitHub actions occur because make test is run without make install. We created the FAKE_INSTALL target in the test folder to support testing without make install. I've updated the python tests to use FAKE_INSTALL_PREFIX in 3e9109a. It required moving the definition of the FAKE_INSTALL_PREFIX variable to the root CMakeLists.txt in order to be accessed by both the src and test folders, which seems acceptable to me

scpeters · 2021-08-16T21:51:24Z

src/python_TEST.py

+import ignition.math
+
+
+class TestVector3(unittest.TestCase):


nit: rename TestVector3 since this is a more general test

Done. e6373fb

scpeters · 2021-08-16T23:04:39Z

looks like the only remaining CI issue is that 9e2dafc needs to be signed @francocipollone

Co-authored-by: Louise Poubel <[email protected]> Signed-off-by: Franco Cipollone <[email protected]>

Signed-off-by: Michael Carroll <[email protected]>

Signed-off-by: Franco Cipollone <[email protected]>

Move the variable definition to the root CMakeLists.txt Signed-off-by: Steve Peters <[email protected]>

Point to fake install lib folder. Signed-off-by: Steve Peters <[email protected]>

Signed-off-by: Franco Cipollone <[email protected]>

francocipollone · 2021-08-17T16:23:47Z

the test failure in the GitHub actions occur because make test is run without make install. We created the FAKE_INSTALL target in the test folder to support testing without make install. I've updated the python tests to use FAKE_INSTALL_PREFIX in 3e9109a. It required moving the definition of the FAKE_INSTALL_PREFIX variable to the root CMakeLists.txt in order to be accessed by both the src and test folders, which seems acceptable to me

I see, thanks @scpeters for both the patch and the explanation.

looks like the only remaining CI issue is that 9e2dafc needs to be signed @francocipollone

Fixed!

scpeters · 2021-08-17T18:20:12Z

examples/angle_example.py

+a1 = ignition.math.Angle(1.5707)
+a2 = ignition.math.Angle(0.7854)
+print("a1 = {} radians, {} degrees\n".format(a1.Radian(), a1.Degree()))
+print("a2 = {} radians, {} degrees\n".format(a2.Radian(), a2.Degree()))


nit for the future (not this pull request): I find f"" strings are more readable than using .format()

for example:

print(f"a2 = {a2.Radian()} radians, {a2.Degree(} degrees\n")

chapulina · 2021-08-17T19:49:49Z

I just pulled the latest ign-math6 branch and got into a weird compile error:

set_target_properties Can not find target to add properties to: _pymath

Full log

[0.020s] Invoking command in '/home/chapulina/dev_bionic/ws_fortress/build/ignition-math6': /usr/bin/cmake /home/chapulina/dev_bionic/ws_fortress/src/ign-math -DBUILD_TESTING=true -DCMAKE_INSTALL_PREFIX=/home/chapulina/dev_bionic/ws_fortress/install
[0.062s] -- The C compiler identification is GNU 8.4.0
[0.102s] -- The CXX compiler identification is GNU 8.4.0
[0.111s] -- Detecting C compiler ABI info
[0.150s] -- Detecting C compiler ABI info - done
[0.159s] -- Check for working C compiler: /usr/bin/cc - skipped
[0.159s] -- Detecting C compile features
[0.159s] -- Detecting C compile features - done
[0.161s] -- Detecting CXX compiler ABI info
[0.200s] -- Detecting CXX compiler ABI info - done
[0.209s] -- Check for working CXX compiler: /usr/bin/c++ - skipped
[0.209s] -- Detecting CXX compile features
[0.210s] -- Detecting CXX compile features - done
[0.215s] -- ignition-math6 version 6.8.0
[0.215s] -- Operating system is Linux
[0.218s] -- Found CPack generators: DEB
[0.223s] -- Looking for eigen3 - found
[0.223s] 
[0.232s] -- Searching for swig - found.
[0.696s] -- Searching for Ruby - found.
[0.704s] -- Searching for Python - found.
[1.374s] -- 
[1.375s] -- Searching for host SSE information
[1.413s] -- SSE2 found
[1.413s] -- SSE3 found
[1.413s] -- SSE4.1 found
[1.413s] -- SSE4.2 found
[1.422s] -- Configuring library: ignition-math6
[1.423s] -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
[1.472s] -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
[1.472s] -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
[1.520s] -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
[1.520s] -- Performing Test COMPILER_HAS_DEPRECATED_ATTR
[1.570s] -- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
[1.572s] -- Adding 43 UNIT tests
[1.593s] -- Adding 6 UNIT tests
[1.614s] �[31mCMake Error at src/CMakeLists.txt:120 (set_target_properties):
[1.614s]   set_target_properties Can not find target to add properties to: _pymath
[1.614s] 
[1.614s] �[0m
[1.614s] �[31mCMake Error at src/CMakeLists.txt:126 (target_compile_options):
[1.614s]   Cannot specify compile options for target "_pymath" which is not built by
[1.614s]   this project.
[1.615s] 
[1.615s] �[0m
[1.615s] �[31mCMake Error at src/CMakeLists.txt:131 (install):
[1.615s]   install TARGETS given target "_pymath" which does not exist.
[1.615s] 
[1.615s] �[0m
[1.632s] -- Adding 1 INTEGRATION tests
[1.648s] -- No tests have been specified for PERFORMANCE
[1.663s] -- No tests have been specified for REGRESSION
[1.681s] -- Configuring library: ignition-math6-eigen3
[1.682s] -- Adding 1 UNIT tests
[1.720s] -- Adding codecheck target
[1.720s] -- Build configuration successful
[1.720s] -- Build type: RelWithDebInfo
[1.721s] -- Install prefix: /home/chapulina/dev_bionic/ws_fortress/install
[1.722s] -- Looking for ronn to generate manpages - found
[1.759s] -- Found Doxygen: /usr/bin/doxygen (found version "1.8.13") found components: doxygen missing components: dot
[1.801s] -- Configuring incomplete, errors occurred!
[1.801s] See also "/home/chapulina/dev_bionic/ws_fortress/build/ignition-math6/CMakeFiles/CMakeOutput.log".
[1.802s] See also "/home/chapulina/dev_bionic/ws_fortress/build/ignition-math6/CMakeFiles/CMakeError.log".
[1.807s] Invoked command in '/home/chapulina/dev_bionic/ws_fortress/build/ignition-math6' returned '1': /usr/bin/cmake /home/chapulina/dev_bionic/ws_fortress/src/ign-math -DBUILD_TESTING=true -DCMAKE_INSTALL_PREFIX=/home/chapulina/dev_bionic/ws_fortress/install

I'm not sure yet how to fix my workspace. Looking into it

Update

Solved for now with sudo apt purge .*swig.*

scpeters · 2021-08-17T20:53:10Z

I just pulled the latest ign-math6 branch and got into a weird compile error:

set_target_properties Can not find target to add properties to: _pymath

Full log
I'm not sure yet how to fix my workspace. Looking into it

Update

Solved for now with sudo apt purge .*swig.*

what OS and swig version do you have?

we should consider reverting this if others can reproduce the problem

chapulina · 2021-08-17T22:15:34Z

what OS and swig version do you have?

This happened on Bionic, swig/bionic 3.0.12-1 amd64. I see this is the same version installed in the successful CI build. I can try to reproduce in a clean container later.

francocipollone · 2021-08-18T13:10:06Z

I couldn't reproduce the error in bionic with the same swig version. I'm running the container with ign-rocker.

Solved for now with sudo apt purge .swig.

So removing swig and installing it again did the trick? That's interesting.

chapulina · 2021-08-18T19:34:45Z

So removing swig and installing it again did the trick?

Not exactly 😅 Removing SWIG just allowed me to skip that entire block of code, so I didn't run into the problem and could compile the core library. Reinstalling SWIG brings back the issue to me.

My problem is the CMake version, trying a few things here: #223

Basic setup for Python interface using SWIG

02c1b08

Signed-off-by: Marcos Wagner <[email protected]>

github-actions bot added 🏢 edifice Ignition Edifice 🏯 fortress Ignition Fortress 🏰 citadel Ignition Citadel 🔮 dome Ignition Dome labels Jul 27, 2021

WagnerMarcos requested a review from francocipollone July 27, 2021 19:13

WagnerMarcos assigned WagnerMarcos and LolaSegura Jul 27, 2021

francocipollone added 4 commits July 27, 2021 23:43

Fixes install path and name math instead of pymath.

3e31a3a

Signed-off-by: Franco Cipollone <[email protected]>

Adds py examples

f0ad4df

Signed-off-by: Franco Cipollone <[email protected]>

Fixes ruby_TEST set up.

fab308e

Signed-off-by: Franco Cipollone <[email protected]>

Support both Swig3 and Swig4 versions.

5fa9264

Signed-off-by: Franco Cipollone <[email protected]>

Changes in cmake to run python tests with colcon

4d09fe6

Signed-off-by: Marcos Wagner <[email protected]>

WagnerMarcos force-pushed the wagnermarcos/python_interface branch from bca2353 to 4d09fe6 Compare July 29, 2021 14:55

LolaSegura and others added 2 commits July 29, 2021 16:56

Adds line in the src/CMakeLists.txt file to solve CI error. Adds pyth…

5f3d676

…on test files to some of the classes that had an associated .i file. Signed-off-by: LolaSegura <[email protected]>

Adding Vector3 and Vector4 python interface tests

6522274

Signed-off-by: Marcos Wagner <[email protected]>

francocipollone suggested changes Jul 30, 2021

View reviewed changes

francocipollone marked this pull request as ready for review July 30, 2021 20:59

francocipollone requested a review from scpeters as a code owner July 30, 2021 20:59

francocipollone mentioned this pull request Jul 30, 2021

Documentation, Style and scripting interfaces update #101

Closed

45 tasks

Addressed comments.

9e443c9

Signed-off-by: LolaSegura <[email protected]>

francocipollone suggested changes Aug 2, 2021

View reviewed changes

src/Vector4_TEST.py Outdated Show resolved Hide resolved

src/Vector4_TEST.py Outdated Show resolved Hide resolved

src/Vector2_TEST.py Outdated Show resolved Hide resolved

src/CMakeLists.txt Outdated Show resolved Hide resolved

src/CMakeLists.txt Outdated Show resolved Hide resolved

Address comments.

f3c457f

Signed-off-by: LolaSegura <[email protected]>

WagnerMarcos mentioned this pull request Aug 12, 2021

Adds Line2, Line3, SignalStats, Temperature python interface #220

Merged

8 tasks

scpeters mentioned this pull request Aug 13, 2021

abi script: clean build folder between builds gazebo-tooling/release-tools#497

Merged

scpeters reviewed Aug 13, 2021

View reviewed changes

src/CMakeLists.txt Outdated Show resolved Hide resolved

francocipollone reviewed Aug 16, 2021

View reviewed changes

scpeters reviewed Aug 16, 2021

View reviewed changes

francocipollone and others added 6 commits August 17, 2021 13:19

Update CMakeLists.txt

440a6d1

Co-authored-by: Louise Poubel <[email protected]> Signed-off-by: Franco Cipollone <[email protected]>

Remove Marya from CODEOWNERS (#217)

be29035

Signed-off-by: Michael Carroll <[email protected]>

Fixes ABI break.

4a1b193

Signed-off-by: Franco Cipollone <[email protected]>

Avoid recompiling sources for the bindings.

4b02a9d

Signed-off-by: Franco Cipollone <[email protected]>

Use FAKE_INSTALL_PREFIX for PYTHONPATH in tests

02e065a

Move the variable definition to the root CMakeLists.txt Signed-off-by: Steve Peters <[email protected]>

Set LD_LIBRARY_PATH for python tests

3fca80f

Point to fake install lib folder. Signed-off-by: Steve Peters <[email protected]>

francocipollone force-pushed the wagnermarcos/python_interface branch from 2192b29 to 3fca80f Compare August 17, 2021 16:21

Addresses nit

e6373fb

Signed-off-by: Franco Cipollone <[email protected]>

scpeters approved these changes Aug 17, 2021

View reviewed changes

francocipollone approved these changes Aug 17, 2021

View reviewed changes

scpeters merged commit 2330061 into ign-math6 Aug 17, 2021

scpeters deleted the wagnermarcos/python_interface branch August 17, 2021 19:07

LolaSegura mentioned this pull request Aug 17, 2021

Adds python interface to Quaternion, Pose3, Matrix3 and Matrix4 #221

Merged

8 tasks

chapulina mentioned this pull request Aug 18, 2021

SWIG Python compilation with CMake >= 3.13 #223

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Basic setup for Python interface using SWIG #216

Basic setup for Python interface using SWIG #216

WagnerMarcos commented Jul 27, 2021 •

edited by francocipollone

Loading

francocipollone commented Jul 28, 2021

codecov bot commented Jul 28, 2021 •

edited

Loading

WagnerMarcos commented Jul 30, 2021 •

edited

Loading

francocipollone left a comment

francocipollone left a comment

francocipollone commented Aug 2, 2021

scpeters commented Aug 13, 2021

francocipollone Aug 16, 2021

francocipollone Aug 16, 2021

francocipollone Aug 16, 2021

scpeters Aug 16, 2021

scpeters left a comment

scpeters Aug 16, 2021

francocipollone Aug 17, 2021

scpeters commented Aug 16, 2021

francocipollone commented Aug 17, 2021 •

edited

Loading

scpeters Aug 17, 2021

chapulina commented Aug 17, 2021 •

edited

Loading

scpeters commented Aug 17, 2021 •

edited

Loading

chapulina commented Aug 17, 2021

francocipollone commented Aug 18, 2021

chapulina commented Aug 18, 2021

		import ignition.math


		class TestVector3(unittest.TestCase):

Basic setup for Python interface using SWIG #216

Basic setup for Python interface using SWIG #216

Conversation

WagnerMarcos commented Jul 27, 2021 • edited by francocipollone Loading

🎉 New feature

Summary

Test it

Checklist

francocipollone commented Jul 28, 2021

codecov bot commented Jul 28, 2021 • edited Loading

Codecov Report

WagnerMarcos commented Jul 30, 2021 • edited Loading

francocipollone left a comment

Choose a reason for hiding this comment

francocipollone left a comment

Choose a reason for hiding this comment

francocipollone commented Aug 2, 2021

scpeters commented Aug 13, 2021

francocipollone Aug 16, 2021

Choose a reason for hiding this comment

francocipollone Aug 16, 2021

Choose a reason for hiding this comment

francocipollone Aug 16, 2021

Choose a reason for hiding this comment

scpeters Aug 16, 2021

Choose a reason for hiding this comment

scpeters left a comment

Choose a reason for hiding this comment

scpeters Aug 16, 2021

Choose a reason for hiding this comment

francocipollone Aug 17, 2021

Choose a reason for hiding this comment

scpeters commented Aug 16, 2021

francocipollone commented Aug 17, 2021 • edited Loading

scpeters Aug 17, 2021

Choose a reason for hiding this comment

chapulina commented Aug 17, 2021 • edited Loading

scpeters commented Aug 17, 2021 • edited Loading

chapulina commented Aug 17, 2021

francocipollone commented Aug 18, 2021

chapulina commented Aug 18, 2021

WagnerMarcos commented Jul 27, 2021 •

edited by francocipollone

Loading

codecov bot commented Jul 28, 2021 •

edited

Loading

WagnerMarcos commented Jul 30, 2021 •

edited

Loading

francocipollone commented Aug 17, 2021 •

edited

Loading

chapulina commented Aug 17, 2021 •

edited

Loading

scpeters commented Aug 17, 2021 •

edited

Loading