Sphinx Python Documentation #342

jstone-lucasfilm · 2020-01-15T21:00:09Z

Add support for the generation of Sphinx documentation for MaterialX Python, using the approach described here:

https://pybind11.readthedocs.io/en/stable/advanced/misc.html#generating-documentation-using-sphinx

…AcademySoftwareFoundation#342) * Build fixes - Make Viewer conditionally require render builds - Remove duplicate installs of resources and libraries by fixing up search path logic to allow for current path as well as parent path. Current install puts the resources/ and libraries/ under the same directory as bin/. - Add in pop() method on FilePath to allow to get parent. * Create Util file and pull out some Material class logic into it. * Movement of UI / introspection utilities the render utils. * - Fix up extension storage and usage for image file dialog so it's not fixed. - Fix so that we use v-flip for textures so texture are not flipped. * Review fixes. Add in VariantBlock search function based on generic predicate function. * Review fixes.

StefanHabel · 2023-10-10T01:19:16Z

I'd like to take a stab at this for the ASWF Dev Days 2023.

If I’m understanding this right, the work here roughly consists of two parts:

Integrating a Sphinx build of the MaterialX Python API documentation into the MaterialX CMake build
Adding/extending docstrings for MaterialX Python bindings using pybind11

While it may not be possible/feasible to add/curate docstrings for all MaterialX Python bindings during Dev Days, it should be possible to lay the foundation for building API documentation using Sphinx on the day.

Proposed approach/outcome:

An optional extra build target similar to the existing cmake --build . --target MaterialXDocs (which uses doxygen to generate C++ API documentation), possibly named MaterialXDocsPython.
Documentation in HTML format that is generated in a sibling folder next to the existing C++ API documentation (as opposed to living in the same html folder).

Proposed artifacts:

A new CMake module at cmake/modules/FindSphinx.cmake to be able to find the sphinx-build executable (Windows: sphinx-build.exe) from documents/CMakeLists.txt using find_package(Sphinx).
- cf. https://eb2.co/blog/2012/03/sphinx-and-cmake-beautiful-documentation-for-c-projects/
- cf. https://www.vortech.nl/en/integrating-sphinx-in-cmake/
Modifications to documents/CMakeLists.txt to create a new build target named MaterialXDocsPython, which would run sphinx-build to generate Python API developer documentation, in a directory that is separate to the existing doxygen-generated html documentation, e.g. named html-sphinx.
- https://www.sphinx-doc.org/en/master/tutorial/getting-started.html
A Sphinx configuration template file (e.g. documents/sphinx-conf.py.in) that would be filled with values of configuration variables like MATERIALX_LIBRARY_VERSION as part of the build, in order to generate the conf.py file that sphinx-build uses.
- This would be similar to the existing developer/Doxyfile.in from which documents/CMakeLists.txt generates Doxyfile for configuring the doxygen run.
- https://www.sphinx-doc.org/en/master/usage/configuration.html
Integration of existing developer guide contents in Markdown format from documents/DeveloperGuide/ into the Sphinx-generated Python API documentation, e.g. using the myst_parser extension:
A new paragraph in the Building API Documentation section in documents/DeveloperGuide/MainPage.md to describe how to build the Python API documentation using Sphinx.
Integration of docstrings from MaterialX Python bindings that are defined in source/PyMaterialX/ from compiled Python C extensions, e.g. from lib/PyMaterialXCore.cpython-312-darwin.so etc.

Work in progress. This PR adds a new build target named `MaterialXDocsPython`, which generates Python API documentation using Sphinx. The existing developer guide contents are incorporated into the new HTML documentation, which lives side-by-side to the existing Doxygen-generated C++ API documentation. The docstrings of the Python modules were tweaked to describe what the individual modules are responsible for. Signed-off-by: Stefan Habel <[email protected]>

…Documentation Signed-off-by: Jonathan Stone <[email protected]>

…fixes

StefanHabel · 2024-09-20T23:41:39Z

I'd like to continue our work on this from last year’s ASWF Dev Days next week:
#1567

Changes: - Replaced relative image sources with absolute image URLs so that images load when viewed at https://pypi.org/project/MaterialX/ - Streamlined headline levels: page title (if any) uses `#`, first level uses `##`, second level uses `###`, etc. - Used markdown tables with table headers for formatting image galleries - Streamlined figure captions to appear _below_ figures - Used HTML comment syntax for comments in XML blocks - Other minor formatting changes, e.g. formatting the `MaterialXView` executable in a code font Split from AcademySoftwareFoundation#1567. Update AcademySoftwareFoundation#342. Signed-off-by: Stefan Habel <[email protected]>

This PR adds support for generating Python API documentation in HTML format using Sphinx from the MaterialX Python that are built in the `lib/` directory. A new CMake build option named `MATERIALX_BUILD_PYTHON_DOCS` allows developers to turn generating Python API documentation on. When `MATERIALX_BUILD_PYTHON_DOCS` is set to `ON`, `MATERIALX_BUILD_PYTHON` is set to `ON` as well, ensuring we have Python modules for which to build the Python API docs. The core functionality of generating Python API documentation lives in a new directory named `documents/PythonAPI/`. It is controlled with a new `CMakeLists.txt` file in that directory, which defines a new target named `MaterialXDocsPython`, similar to the existing target `MaterialXDocs` that generates API documentation for the MaterialX C++ API. To facilitate the curation and addition of docstrings in the implementation files within `source/PyMaterialX/`, this PR adds a new helper macro named `PYMATERIALX_DOCSTRING` that allows developers of Python modules to define docstrings using the following pattern: ```cpp PYMATERIALX_DOCSTRING(R"docstring( ...markdown text here... )docstring"); ``` Revised docstrings for modules and classes are to be added in subsequent PRs separately. Documentation in markdown format from the existing `DeveloperGuide` is integrated into the new Python API documentation by way of symlinking the four main `.md` files into the `documents/PythonAPI/sources/` directory from which Sphinx generates the resulting HTML documentation. Warnings that are issued when generating the documentation via Sphinx are to be addressed in a separate PR for the markdown files: AcademySoftwareFoundation#2037 To build the docs from scratch on macOS, I've used the following build script, naming it `build.sh` in the `MaterialX` checkout directory: ```bash #!/usr/bin/env tcsh echo build.sh: Updating Git submodules... git submodule update --init --recursive # Create and activate a virtual environment for installing Python dependencies python3 -m venv /tmp/venv source /tmp/venv/bin/activate.csh echo build.sh: Installing dependencies... python3 -m pip install myst_parser # https://pypi.org/project/myst-parser/ echo build.sh: Making build directory and changing into it... mkdir build cd build echo build.sh: Configuring... cmake .. \ --fresh \ -DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX15.0.sdk \ -DMATERIALX_BUILD_PYTHON=ON \ -DMATERIALX_BUILD_VIEWER=ON \ -DMATERIALX_BUILD_GRAPH_EDITOR=ON \ -DMATERIALX_BUILD_DOCS=ON \ -DMATERIALX_BUILD_PYTHON_DOCS=ON \ -DMATERIALX_BUILD_TESTS=ON \ && \ echo build.sh: Building... \ && \ cmake --build . -j 8 \ && \ echo build.sh: Building target MaterialXDocs... \ && \ cmake --build . --target MaterialXDocs \ && \ echo build.sh: Building target MaterialXDocsPython... \ && \ cmake --build . --target MaterialXDocsPython \ && \ afplay /System/Library/Sounds/Blow.aiff # Deactivate the virtual environment deactivate ``` The build output currently ends with the following messages: ```python The parsed MaterialX Python API consists of: * 11 modules * 48 functions * 139 classes * 1175 methods * 6 exception types WARNING: 48 functions look like they do not have docstrings yet. WARNING: 1019 methods look like they do not have docstrings yet. WARNING: 32 functions look like their parameters have not all been named using `py::arg()`. WARNING: 499 methods look like their parameters have not all been named using `py::arg()`. build succeeded, 168 warnings. The HTML pages are in .. [100%] Built target MaterialXDocsPython ``` Split from AcademySoftwareFoundation#1567. Update AcademySoftwareFoundation#342. Signed-off-by: Stefan Habel <[email protected]>

This PR adds a docstring in markdown format to the `PyMaterialXCore` module. The docstring is defined in a macro named `PyMaterialXCore_DOCSTRING` that uses the new `PYMATERIALX_DOCSTRING` macro to allow us to place the first and last lines in lines by themselves, rather than starting the docstring contents immediately after the `R"docstring(` marker. The `PyMaterialXCore_DOCSTRING` macro is placed in a separate header file named `__doc__.md.h` which lives side-by-side with the `PyModule.cpp` file in which it is included. Split from AcademySoftwareFoundation#1567. Depends on AcademySoftwareFoundation#2038. Update AcademySoftwareFoundation#342. Signed-off-by: Stefan Habel <[email protected]>

This PR adds a docstring in markdown format to the `PyMaterialXFormat` module. The docstring is defined in a macro named `PyMaterialXFormat_DOCSTRING` that uses the new `PYMATERIALX_DOCSTRING` macro that is introduced in AcademySoftwareFoundation#2038 to allow us to place the first and last lines in lines by themselves, rather than starting the docstring contents immediately after the `R"docstring(` marker. The `PyMaterialXFormat_DOCSTRING` macro is placed in a separate header file named `__doc__.md.h` which lives side-by-side with the `PyModule.cpp` file in which it is included. Note that the docstrings for individual classes, methods, and functions are to be added in separate PRs. Note that this PR also renames `readFromXmlFileBase()` to `readFromXmlFile()` allowing us to remove the alias from `python/MaterialX/main.py`. Split from AcademySoftwareFoundation#1567. Depends on AcademySoftwareFoundation#2038. Update AcademySoftwareFoundation#342. Signed-off-by: Stefan Habel <[email protected]>

This PR adds docstrings in markdown format to the following modules: - `PyMaterialXGenGlsl` - `PyMaterialXGenMdl` - `PyMaterialXGenMsl` - `PyMaterialXGenOsl` - `PyMaterialXGenShader` The docstrings are defined in macros named `PyMaterialXGen*_DOCSTRING` that use the new `PYMATERIALX_DOCSTRING` macro that is introduced in AcademySoftwareFoundation#2038 to allow us to place the first and last lines in lines by themselves, rather than starting the docstring contents immediately after the `R"docstring(` marker. The `PyMaterialXGen*_DOCSTRING` macros are placed in separate header files named `__doc__.md.h` which live side-by-side with their corresponding `PyModule.cpp` file in which it is included. Note that the docstrings for individual classes, methods, and functions are to be added in separate PRs. Split from AcademySoftwareFoundation#1567. Depends on AcademySoftwareFoundation#2038. Update AcademySoftwareFoundation#342. Signed-off-by: Stefan Habel <[email protected]>

This PR adds docstrings in markdown format to the following modules: - `PyMaterialXRender` - `PyMaterialXRenderGlsl` - `PyMaterialXRenderMsl` - `PyMaterialXRenderOsl` The docstrings are defined in macros named `PyMaterialXRender*_DOCSTRING` that use the new `PYMATERIALX_DOCSTRING` macro that is introduced in AcademySoftwareFoundation#2038 to allow us to place the first and last lines in lines by themselves, rather than starting the docstring contents immediately after the `R"docstring(` marker. The `PyMaterialXRender*_DOCSTRING` macros are placed in separate header files named `__doc__.md.h` which live side-by-side with their corresponding `PyModule.cpp` files in which they are included. Note that the docstrings for individual classes, methods, and functions are to be added in separate PRs. Split from AcademySoftwareFoundation#1567. Depends on AcademySoftwareFoundation#2038. Update AcademySoftwareFoundation#342. Signed-off-by: Stefan Habel <[email protected]>

This PR adds support for generating Python API documentation in HTML format using Sphinx from the MaterialX Python that are built in the `lib/` directory. A new CMake build option named `MATERIALX_BUILD_PYTHON_DOCS` allows developers to turn generating Python API documentation on. When `MATERIALX_BUILD_PYTHON_DOCS` is set to `ON`, `MATERIALX_BUILD_PYTHON` is set to `ON` as well, ensuring we have Python modules for which to build the Python API docs. The core functionality of generating Python API documentation lives in a new directory named `documents/PythonAPI/`. It is controlled with a new `CMakeLists.txt` file in that directory, which defines a new target named `MaterialXDocsPython`, similar to the existing target `MaterialXDocs` that generates API documentation for the MaterialX C++ API. To facilitate the curation and addition of docstrings in the implementation files within `source/PyMaterialX/`, this PR adds a new helper macro named `PYMATERIALX_DOCSTRING` that allows developers of Python modules to define docstrings using the following pattern: ```cpp PYMATERIALX_DOCSTRING(R"docstring( ...markdown text here... )docstring"); ``` Revised docstrings for modules and classes are to be added in subsequent PRs separately. Documentation in markdown format from the existing `DeveloperGuide` is integrated into the new Python API documentation by way of symlinking the four main `.md` files into the `documents/PythonAPI/sources/` directory from which Sphinx generates the resulting HTML documentation. Warnings that are issued when generating the documentation via Sphinx are to be addressed in a separate PR for the markdown files: AcademySoftwareFoundation#2037 To build the docs from scratch on macOS, I've used the following build script, naming it `build.sh` in the `MaterialX` checkout directory: ```bash echo build.sh: Updating Git submodules... git submodule update --init --recursive python3 -m venv /tmp/venv source /tmp/venv/bin/activate.csh echo build.sh: Installing dependencies... python3 -m pip install myst_parser # https://pypi.org/project/myst-parser/ echo build.sh: Making build directory and changing into it... mkdir build cd build echo build.sh: Configuring... cmake .. \ --fresh \ -DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX15.0.sdk \ -DMATERIALX_BUILD_PYTHON=ON \ -DMATERIALX_BUILD_VIEWER=ON \ -DMATERIALX_BUILD_GRAPH_EDITOR=ON \ -DMATERIALX_BUILD_DOCS=ON \ -DMATERIALX_BUILD_PYTHON_DOCS=ON \ -DMATERIALX_BUILD_TESTS=ON \ && \ echo build.sh: Building... \ && \ cmake --build . -j 8 \ && \ echo build.sh: Building target MaterialXDocs... \ && \ cmake --build . --target MaterialXDocs \ && \ echo build.sh: Building target MaterialXDocsPython... \ && \ cmake --build . --target MaterialXDocsPython \ && \ afplay /System/Library/Sounds/Blow.aiff deactivate ``` The build output currently ends with the following messages: ```python The parsed MaterialX Python API consists of: * 11 modules * 48 functions * 139 classes * 1175 methods * 6 exception types WARNING: 48 functions look like they do not have docstrings yet. WARNING: 1019 methods look like they do not have docstrings yet. WARNING: 32 functions look like their parameters have not all been named using `py::arg()`. WARNING: 499 methods look like their parameters have not all been named using `py::arg()`. build succeeded, 168 warnings. The HTML pages are in .. [100%] Built target MaterialXDocsPython ``` Split from AcademySoftwareFoundation#1567. Update AcademySoftwareFoundation#342. Signed-off-by: Stefan Habel <[email protected]>

This PR adds a docstring in markdown format to the `PyMaterialXCore` module. The docstring is defined in a macro named `PyMaterialXCore_DOCSTRING` that uses the new `PYMATERIALX_DOCSTRING` macro to allow us to place the first and last lines in lines by themselves, rather than starting the docstring contents immediately after the `R"docstring(` marker. The `PyMaterialXCore_DOCSTRING` macro is placed in a separate header file named `__doc__.md.h` which lives side-by-side with the `PyModule.cpp` file in which it is included. Split from AcademySoftwareFoundation#1567. Depends on AcademySoftwareFoundation#2038. Update AcademySoftwareFoundation#342. Signed-off-by: Stefan Habel <[email protected]>

This PR adds support for generating Python API documentation in HTML format using Sphinx from the MaterialX Python that are built in the `lib/` directory. A new CMake build option named `MATERIALX_BUILD_PYTHON_DOCS` allows developers to turn generating Python API documentation on. When `MATERIALX_BUILD_PYTHON_DOCS` is set to `ON`, `MATERIALX_BUILD_PYTHON` is set to `ON` as well, ensuring we have Python modules for which to build the Python API docs. The core functionality of generating Python API documentation lives in a new directory named `documents/PythonAPI/`. It is controlled with a new `CMakeLists.txt` file in that directory, which defines a new target named `MaterialXDocsPython`, similar to the existing target `MaterialXDocs` that generates API documentation for the MaterialX C++ API. To facilitate the curation and addition of docstrings in the implementation files within `source/PyMaterialX/`, this PR adds a new helper macro named `PYMATERIALX_DOCSTRING` that allows developers of Python modules to define docstrings using the following pattern: ```cpp PYMATERIALX_DOCSTRING(R"docstring( ...markdown text here... )docstring"); ``` Revised docstrings for modules and classes are to be added in subsequent PRs separately. Documentation in markdown format from the existing `DeveloperGuide` is integrated into the new Python API documentation by way of symlinking the four main `.md` files into the `documents/PythonAPI/sources/` directory from which Sphinx generates the resulting HTML documentation. Warnings that are issued when generating the documentation via Sphinx are to be addressed in a separate PR for the markdown files: AcademySoftwareFoundation#2037 To build the docs from scratch on macOS, I've used the following build script, naming it `build.sh` in the `MaterialX` checkout directory: ```bash echo build.sh: Updating Git submodules... git submodule update --init --recursive python3 -m venv /tmp/venv source /tmp/venv/bin/activate.csh echo build.sh: Installing dependencies... python3 -m pip install myst_parser # https://pypi.org/project/myst-parser/ echo build.sh: Making build directory and changing into it... mkdir build cd build echo build.sh: Configuring... cmake .. \ --fresh \ -DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX15.0.sdk \ -DMATERIALX_BUILD_PYTHON=ON \ -DMATERIALX_BUILD_VIEWER=ON \ -DMATERIALX_BUILD_GRAPH_EDITOR=ON \ -DMATERIALX_BUILD_DOCS=ON \ -DMATERIALX_BUILD_PYTHON_DOCS=ON \ -DMATERIALX_BUILD_TESTS=ON \ && \ echo build.sh: Building... \ && \ cmake --build . -j 8 \ && \ echo build.sh: Building target MaterialXDocs... \ && \ cmake --build . --target MaterialXDocs \ && \ echo build.sh: Building target MaterialXDocsPython... \ && \ cmake --build . --target MaterialXDocsPython \ && \ afplay /System/Library/Sounds/Blow.aiff deactivate ``` The build output currently ends with the following messages: ```python The parsed MaterialX Python API consists of: * 11 modules * 48 functions * 139 classes * 1175 methods * 6 exception types WARNING: 48 functions look like they do not have docstrings yet. WARNING: 1019 methods look like they do not have docstrings yet. WARNING: 32 functions look like their parameters have not all been named using `py::arg()`. WARNING: 499 methods look like their parameters have not all been named using `py::arg()`. build succeeded, 168 warnings. The HTML pages are in .. [100%] Built target MaterialXDocsPython ``` Split from AcademySoftwareFoundation#1567. Update AcademySoftwareFoundation#342. Signed-off-by: Stefan Habel <[email protected]>

Mostly copied from corresponding C++ header files, but using backticks to automatically link to relevant sections in the Python API documentation, e.g. `Document` to link to `PyMaterialXCore.Document.html`. Using `mod.attr("<name>").doc() = ...` syntax so that this patch only adds new lines, rather than modifying existing lines. Using `:see:` fields to link to the corresponding C++ API docs page. Demo of generated HTML page available here: https://stefanhabel.github.io/generated/PyMaterialXCore.html Split from AcademySoftwareFoundation#1567. Update AcademySoftwareFoundation#342. Signed-off-by: Stefan Habel <[email protected]>

jstone-lucasfilm added enhancement help wanted and removed enhancement labels Jan 15, 2020

jstone-lucasfilm changed the title ~~[Lucasfilm] Sphinx Python Documentation~~ Sphinx Python Documentation Jul 20, 2022

jstone-lucasfilm mentioned this issue Oct 13, 2023

Sphinx Python Documentation #1567

Closed

StefanHabel mentioned this issue Oct 22, 2023

Crash in MaterialXCore::Element::setChildIndex() due to off-by-one error in index check #1581

Closed

jstone-lucasfilm added a commit to StefanHabel/MaterialX that referenced this issue Oct 25, 2023

Merge branch 'main' into AcademySoftwareFoundation#342-Sphinx-Python-…

37f1ef5

…Documentation Signed-off-by: Jonathan Stone <[email protected]>

StefanHabel mentioned this issue Nov 6, 2023

Resolved Python module import dependencies #1595

Merged

jstone-lucasfilm added a commit to StefanHabel/MaterialX that referenced this issue Nov 27, 2023

Merge branch 'main' into AcademySoftwareFoundation#342-Python-script-…

5207806

…fixes

jstone-lucasfilm removed the help wanted label Jan 9, 2024

jstone-lucasfilm assigned StefanHabel Sep 23, 2024

jstone-lucasfilm added the help wanted label Sep 23, 2024

StefanHabel mentioned this issue Sep 29, 2024

Updated markdown documentation files #2037

Merged

StefanHabel mentioned this issue Sep 30, 2024

Generating Python API docs #2038

Open

StefanHabel mentioned this issue Sep 30, 2024

Improvements to PyMaterialX docstrings #2039

Merged

StefanHabel mentioned this issue Sep 30, 2024

Adding a docstring for PyMaterialXFormat. #2040

Closed

StefanHabel mentioned this issue Sep 30, 2024

Adding docstrings for PyMaterialXGen*. #2041

Closed

StefanHabel mentioned this issue Sep 30, 2024

Adding docstrings for PyMaterialXRender*. #2042

Closed

jstone-lucasfilm added a commit to StefanHabel/MaterialX that referenced this issue Oct 1, 2024

Merge branch 'main' into AcademySoftwareFoundation#342-markdown-changes

f9de866

jstone-lucasfilm added a commit to StefanHabel/MaterialX that referenced this issue Oct 3, 2024

Merge branch 'main' into AcademySoftwareFoundation#342-PythonAPI-docs

ae0f0c0

StefanHabel added a commit to StefanHabel/MaterialX that referenced this issue Oct 6, 2024

Merge branch 'main' into AcademySoftwareFoundation#342-PythonAPI-docs

02b8bf9

StefanHabel mentioned this issue Oct 7, 2024

Docstrings for PyMaterialXCore classes. #2051

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sphinx Python Documentation #342

Sphinx Python Documentation #342

jstone-lucasfilm commented Jan 15, 2020

StefanHabel commented Oct 10, 2023

StefanHabel commented Sep 20, 2024

Sphinx Python Documentation #342

Sphinx Python Documentation #342

Comments

jstone-lucasfilm commented Jan 15, 2020

StefanHabel commented Oct 10, 2023

StefanHabel commented Sep 20, 2024