Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid error in multipledispatch #12

Closed
wants to merge 3 commits into from
Closed

Conversation

flying-sheep
Copy link
Collaborator

Helps avoid a problem in scverse/scanpy#915

@ivirshup
Copy link
Member

ivirshup commented Dec 5, 2019

It's not throwing an error, but it does throw a number of warnings. This happens a lot with the doc build process, so I'm not sure how important this is.

Definitely important, there's a few formatting issues, here's a screenshot of the build docs:

image

image

Most obviously:

  • Multiple params sections being joined, with strange intermediates
  • Weird indentation level
  • Math not formatting
  • Spurious "gearys_c" at the top of the file

Here's the docstrings that generated this:

Docstrings
    """\
    Params
    ------
    g
        Connectivity graph as a scipy sparse matrix. Should have shape:
        `(n_obs, n_obs)`.
    vals
        Values to calculate Geary's C for. If one dimensional, should have
        shape `(n_obs,)`. If two dimensional (i.e calculating Geary's C for
        multiple variables) should have shape `(n_vars, n_obs)`.
    """
    """\
    Calculate `Geary's C`_ , as used by
    `VISION <https://doi.org/10.1038/s41467-019-12235-0>`_.

    Geary's C is a measure of autocorrelation for some measure on a graph. This
    can be to whether measures are correlated between neighboring cells. Lower
    values indicate greater correlation.

    ..math

        C =
        \frac{
            (N - 1)\sum_{i,j} w_{i,j} (x_i - x_j)^2
        }{
            2W \sum_i (x_i - \bar{x})^2
        }

    Params
    ------
    adata
    vals
        Values to calculate Geary's C for. If this is two dimensional, should
        be of shape `(n_features, n_cells)`. Otherwise should be of shape
        `(n_cells,)`. This matrix can be selected from elements of the anndata
        object by using key word arguments: `layer`, `obsm`, `obsp`, or
        `use_raw`.
    use_graph
        Key to use for graph in anndata object. If not provided, default
        neighbors connectivities will be used instead.
    layer
        Key for `adata.layers` to choose `vals`.
    obsm
        Key for `adata.obsm` to choose `vals`.
    obsp
        Key for `adata.obsp` to choose `vals`.
    use_raw
        Whether to use `adata.raw.X` for `vals`.

    Returns
    -------
    If vals is two dimensional, returns a 1 dimensional ndarray array. Returns
    a scalar if `vals` is 1d.

    .. _Geary's C: https://en.wikipedia.org/wiki/Geary's_C
    """

Here's the output from building the docs where only api/scanpy.metrics.gearys_c.rst should need to be generated and built:

Build output
Running Sphinx v2.2.1
loading pickled environment... done
[autosummary] generating autosummary for: api/index.rst, api/plotting.rst, api/scanpy.Neighbors.compute_eigen.rst, api/scanpy.Neighbors.compute_neighbors.rst, api/scanpy.Neighbors.compute_transitions.rst, api/scanpy.Neighbors.connectivities.rst, api/scanpy.Neighbors.distances.rst, api/scanpy.Neighbors.distances_dpt.rst, api/scanpy.Neighbors.eigen_basis.rst, api/scanpy.Neighbors.eigen_values.rst, ..., external/scanpy.external.tl.cyclone.rst, external/scanpy.external.tl.palantir.rst, external/scanpy.external.tl.phate.rst, external/scanpy.external.tl.phenograph.rst, external/scanpy.external.tl.sandbag.rst, index.rst, installation.rst, references.rst, release_notes.rst, tutorials.rst
WARNING: [autosummary] failed to import 'scanpy._settings.ScanpyConfig.getdoc': no module named scanpy._settings.ScanpyConfig.getdoc
[autosummary] generating autosummary for: /Users/isaac/github/scanpy/docs/api/scanpy.metrics.gearys_c.rst
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: 0 added, 3 changed, 0 removed
reading sources... [100%] api/scanpy.metrics.gearys_c                                                
WARNING: autodoc: failed to import attribute 'ScanpyConfig.getdoc' from module 'scanpy._settings'; the following exception was raised:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sphinx/util/inspect.py", line 230, in safe_getattr
    return getattr(obj, name, *defargs)
AttributeError: type object 'ScanpyConfig' has no attribute 'getdoc'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sphinx/ext/autodoc/importer.py", line 71, in import_object
    obj = attrgetter(obj, attrname)
  File "/usr/local/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py", line 226, in get_attr
    return autodoc_attrgetter(self.env.app, obj, name, *defargs)
  File "/usr/local/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py", line 1501, in autodoc_attrgetter
    return safe_getattr(obj, name, *defargs)
  File "/usr/local/lib/python3.7/site-packages/sphinx/util/inspect.py", line 246, in safe_getattr
    raise AttributeError(name)
AttributeError: getdoc

WARNING: autodoc: failed to import function 'pbmc3k_processed' from module 'scanpy.datasets'; the following exception was raised:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sphinx/util/inspect.py", line 230, in safe_getattr
    return getattr(obj, name, *defargs)
AttributeError: module 'scanpy.datasets' has no attribute 'pbmc3k_processed'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sphinx/ext/autodoc/importer.py", line 71, in import_object
    obj = attrgetter(obj, attrname)
  File "/usr/local/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py", line 226, in get_attr
    return autodoc_attrgetter(self.env.app, obj, name, *defargs)
  File "/usr/local/lib/python3.7/site-packages/sphinx/ext/autodoc/__init__.py", line 1501, in autodoc_attrgetter
    return safe_getattr(obj, name, *defargs)
  File "/usr/local/lib/python3.7/site-packages/sphinx/util/inspect.py", line 246, in safe_getattr
    raise AttributeError(name)
AttributeError: pbmc3k_processed

/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:4: WARNING: Unexpected section title.

Inputs: <spmatrix, ndarray>
----------------------------
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:6: WARNING: Unexpected section title or transition.

------
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:16: WARNING: Unexpected section title.

Inputs: <AnnData>
------------------
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:28: WARNING: Unexpected indentation.
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:29: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:31: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:34: WARNING: Unexpected section title.

Params
------
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:37: WARNING: Unexpected indentation.
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:42: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:55: WARNING: Unexpected section title.

Returns
-------
looking for now-outdated files... none found
pickling environment... done
checking consistency... /Users/isaac/github/scanpy/docs/api/scanpy.api.Neighbors.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.datasets.blobs.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.datasets.krumsiek11.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.datasets.moignard15.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.datasets.paul15.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.datasets.pbmc3k.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.datasets.pbmc68k_reduced.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.datasets.toggleswitch.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.export_to.spring_project.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.logging.print_versions.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.bbknn.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.calculate_qc_metrics.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.dca.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.downsample_counts.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.filter_cells.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.filter_genes.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.filter_genes_dispersion.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.highly_variable_genes.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.log1p.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.magic.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.mnn_correct.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.neighbors.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.normalize_per_cell.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.pca.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.recipe_seurat.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.recipe_weinreb17.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.recipe_zheng17.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.regress_out.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.scale.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.pp.subsample.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.queries.mitochondrial_genes.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read_10x_h5.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read_10x_mtx.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read_csv.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read_excel.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read_h5ad.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read_hdf.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read_loom.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read_mtx.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read_text.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.read_umi_tools.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.set_figure_params.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.cyclone.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.diffmap.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.dpt.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.draw_graph.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.leiden.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.louvain.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.paga.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.pca.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.phate.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.rank_genes_groups.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.sandbag.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.score_genes.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.score_genes_cell_cycle.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.sim.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.tsne.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.api.tl.umap.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.datasets.pbmc3k_processed.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.exporting.cellbrowser.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.exporting.spring_project.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.pl.phate.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.pp.bbknn.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.pp.dca.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.pp.magic.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.pp.mnn_correct.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.tl.cyclone.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.tl.palantir.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.tl.phate.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.tl.phenograph.rst: WARNING: document isn't included in any toctree
/Users/isaac/github/scanpy/docs/api/scanpy.external.tl.sandbag.rst: WARNING: document isn't included in any toctree
done
preparing documents... done
writing output... [100%] index                                                                       
/Users/isaac/github/scanpy/docs/api/scanpy._settings.ScanpyConfig.rst:39:<autosummary>:1: WARNING: py:obj reference target not found: scanpy._settings.ScanpyConfig.getdoc
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:67: WARNING: py:class reference target not found: Multiply dispatched method
generating indices...  genindex py-modindexdone
writing additional pages...  search/usr/local/lib/python3.7/site-packages/sphinx_rtd_theme/search.html:20: RemovedInSphinx30Warning: To modify script_files in the theme is deprecated. Please insert a <script> tag directly in your theme instead.
  {{ super() }}
done
copying static files... ... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 87 warnings.

The HTML pages are in _build/html.

I think the relevant lines are:


/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:4: WARNING: Unexpected section title.

Inputs: <spmatrix, ndarray>
----------------------------
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:6: WARNING: Unexpected section title or transition.

------
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:16: WARNING: Unexpected section title.

Inputs: <AnnData>
------------------
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:28: WARNING: Unexpected indentation.
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:29: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:31: WARNING: Definition list ends without a blank line; unexpected unindent.
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:34: WARNING: Unexpected section title.

Params
------
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:37: WARNING: Unexpected indentation.
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:42: WARNING: Block quote ends without a blank line; unexpected unindent.
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:55: WARNING: Unexpected section title.

Returns
-------
/Users/isaac/github/scanpy/scanpy/metrics/__init__.py:docstring of scanpy.metrics.gearys_c:67: WARNING: py:class reference target not found: Multiply dispatched method

@flying-sheep
Copy link
Collaborator Author

flying-sheep commented Dec 6, 2019

Ugh, that getdoc error … no idea why that surfaced.

That docstring is buggy:

  • It’s not a raw string and the \f of \frac and \s of \sum are invalid escape sequences.
  • It’s .. math, not ..math.

@ivirshup
Copy link
Member

ivirshup commented Dec 7, 2019

With correct formatting for the math equation, it just doesn't show up:

image

?sc.metrics.gearys_c shows:

Multiply dispatched method: gearys_c

Inputs: <spmatrix, ndarray>
----------------------------
Params
    ------
    g
        Connectivity graph as a scipy sparse matrix. Should have shape:
        `(n_obs, n_obs)`.
    vals
        Values to calculate Geary's C for. If one dimensional, should have
        shape `(n_obs,)`. If two dimensional (i.e calculating Geary's C for
        multiple variables) should have shape `(n_vars, n_obs)`.

Inputs: <AnnData>
------------------
Calculate `Geary's C`_ , as used by
`VISION <https://doi.org/10.1038/s41467-019-12235-0>`_.

Geary's C is a measure of autocorrelation for some measure on a graph. This
can be to whether measures are correlated between neighboring cells. Lower
values indicate greater correlation.


.. math

        C =
        \frac{
            (N - 1)\sum_{i,j} w_{i,j} (x_i - x_j)^2
        }{
            2W \sum_i (x_i - \bar{x})^2
        }


Params
------
adata
vals
    Values to calculate Geary's C for. If this is two dimensional, should
    be of shape `(n_features, n_cells)`. Otherwise should be of shape
    `(n_cells,)`. This matrix can be selected from elements of the anndata
    object by using key word arguments: `layer`, `obsm`, `obsp`, or
    `use_raw`.
use_graph
    Key to use for graph in anndata object. If not provided, default
    neighbors connectivities will be used instead.
layer
    Key for `adata.layers` to choose `vals`.
obsm
    Key for `adata.obsm` to choose `vals`.
obsp
    Key for `adata.obsp` to choose `vals`.
use_raw
    Whether to use `adata.raw.X` for `vals`.

Returns
-------
If vals is two dimensional, returns a 1 dimensional ndarray array. Returns
a scalar if `vals` is 1d.

.. _Geary's C: https://en.wikipedia.org/wiki/Geary's_C

Other signatures:
    spmatrix, spmatrix
    csr_matrix, csr_matrix
    spmatrix, DataFrame
    spmatrix, Series

@flying-sheep
Copy link
Collaborator Author

Still not correct, it’s .. math::. Sorry!

But yes, the way they paste together the docstrings seems buggy. The first section including the underline is indented too much. Just one of the “Params” sections seems to be converted to a “Parameters” section…

I guess without proper support through a sphinx extension, we won’t be happy with this. So what you could do is

  1. Simply manually add a docstring to the multiply-dispatched function instead of specifying them inline. I think it either has a doc property it uses or simply __doc__, please check their implementation for what to override.
  2. File a bug there about the wrongly pasted together docstrings
  3. File a feature request there so they supply a sphinx extension that handles their stuff properly.

@flying-sheep flying-sheep deleted the multiple-dispatch branch October 31, 2023 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants