Errors with pandas 1.4 #811

karlann · 2022-01-24T11:50:22Z

Hey!

I updated my pandas version to the latest 1.4 and started getting errors with scvelo plotting and velocity functions. The same plotting functions work fine in scanpy and rolling back to pandas 1.3.5 also solves the issues.

import scanpy as sc
import scvelo as scv

adata = scv.datasets.pancreas()
scv.pp.filter_and_normalize(adata, min_shared_counts=20, n_top_genes=2000)
scv.pp.moments(adata, n_pcs=30, n_neighbors=30)

sc.pl.umap(adata, color = 'clusters') #Works as it should
scv.pl.umap(adata, color = 'clusters') #Results in AttributeError: 'Series' object has no attribute 'categories'

scv.tl.velocity(adata) #Results in AttributeError: 'Series' object has no attribute 'categories'

Error output

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [7], in <module>
----> 1 scv.tl.velocity(adata)

File ~/anaconda3/envs/scvelo2/lib/python3.8/site-packages/scvelo/tools/velocity.py:314, in velocity(data, vkey, mode, fit_offset, fit_offset2, filter_genes, groups, groupby, groups_for_fit, constrain_ratio, use_raw, use_latent_time, perc, min_r2, min_likelihood, r2_adjusted, use_highly_variable, diff_kinetics, copy, **kwargs)
    310     moments(adata)
    312 logg.info("computing velocities", r=True)
--> 314 strings_to_categoricals(adata)
    316 if mode is None or (mode == "dynamical" and "fit_alpha" not in adata.var.keys()):
    317     mode = "stochastic"

File ~/anaconda3/envs/scvelo2/lib/python3.8/site-packages/scvelo/tools/utils.py:301, in strings_to_categoricals(adata)
    296     return (
    297         is_string_dtype(values) or is_integer_dtype(values) or is_bool_dtype(values)
    298     )
    300 df = adata.obs
--> 301 df_keys = [key for key in df.columns if is_valid_dtype(df[key])]
    302 for key in df_keys:
    303     c = df[key]

File ~/anaconda3/envs/scvelo2/lib/python3.8/site-packages/scvelo/tools/utils.py:301, in <listcomp>(.0)
    296     return (
    297         is_string_dtype(values) or is_integer_dtype(values) or is_bool_dtype(values)
    298     )
    300 df = adata.obs
--> 301 df_keys = [key for key in df.columns if is_valid_dtype(df[key])]
    302 for key in df_keys:
    303     c = df[key]

File ~/anaconda3/envs/scvelo2/lib/python3.8/site-packages/scvelo/tools/utils.py:297, in strings_to_categoricals.<locals>.is_valid_dtype(values)
    295 def is_valid_dtype(values):
    296     return (
--> 297         is_string_dtype(values) or is_integer_dtype(values) or is_bool_dtype(values)
    298     )

File ~/anaconda3/envs/scvelo2/lib/python3.8/site-packages/pandas/core/dtypes/common.py:1322, in is_bool_dtype(arr_or_dtype)
   1319     return False
   1321 if isinstance(dtype, CategoricalDtype):
-> 1322     arr_or_dtype = arr_or_dtype.categories
   1323     # now we use the special definition for Index
   1325 if isinstance(arr_or_dtype, ABCIndex):
   1326     # Allow Index[object] that is all-bools or Index["boolean"]

File ~/anaconda3/envs/scvelo2/lib/python3.8/site-packages/pandas/core/generic.py:5583, in NDFrame.__getattr__(self, name)
   5576 if (
   5577     name not in self._internal_names_set
   5578     and name not in self._metadata
   5579     and name not in self._accessors
   5580     and self._info_axis._can_hold_identifiers_and_holds_name(name)
   5581 ):
   5582     return self[name]
-> 5583 return object.__getattribute__(self, name)

AttributeError: 'Series' object has no attribute 'categories'

Versions

scvelo==0.2.4  scanpy==1.8.2  anndata==0.7.8  loompy==3.0.6  numpy==1.21.5  scipy==1.7.3  matplotlib==3.5.1  sklearn==1.0.2  pandas==1.4.0

ivirshup · 2022-01-31T19:35:56Z

This looks related to a bug in pandas 1.4.0, which I think will be fixed in the next release: pandas-dev/pandas#45615

stela2502 · 2022-02-01T07:41:02Z

I have the same problem - is there a fix or workaround available?

WeilerP · 2022-02-01T07:45:33Z

@stela2502, @karlann for the time being, you can downgrade to pandas==1.3.5.

akhst7 · 2022-02-12T02:31:07Z

I have pandas 1.4.0 but the problem still persists. Scv.pl can't create any plots while scanpy can create the plot from the same adata.

Any fixes for this ?

Thanks.

scv.pl.velocity_embedding_stream(adata, basis='umap', color='SCT_snn_res.0.8')

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [20], in <module>
----> 1 scv.pl.velocity_embedding_stream(adata, basis='umap', color='SCT_snn_res.0.8')

File /opt/homebrew/Caskroom/miniforge/base/envs/velocity/lib/python3.10/site-packages/scvelo/plotting/velocity_embedding_stream.py:252, in velocity_embedding_stream(adata, basis, vkey, density, smooth, min_mass, cutoff_perc, arrow_color, arrow_size, arrow_style, max_length, integration_direction, linewidth, n_neighbors, recompute, color, use_raw, layer, color_map, colorbar, palette, size, alpha, perc, X, V, X_grid, V_grid, sort_order, groups, components, legend_loc, legend_fontsize, legend_fontweight, xlabel, ylabel, title, fontsize, figsize, dpi, frameon, show, save, ax, ncols, **kwargs)
    249 ax.streamplot(X_grid[0], X_grid[1], V_grid[0], V_grid[1], **stream_kwargs)
    251 size = 8 * default_size(adata) if size is None else size
--> 252 ax = scatter(
    253     adata,
    254     layer=layer,
    255     color=color,
    256     size=size,
    257     title=title,
    258     ax=ax,
    259     zorder=0,
    260     **scatter_kwargs,
    261 )
    263 savefig_or_show(dpi=dpi, save=save, show=show)
    264 if show is False:

File /opt/homebrew/Caskroom/miniforge/base/envs/velocity/lib/python3.10/site-packages/scvelo/plotting/scatter.py:308, in scatter(adata, basis, x, y, vkey, color, use_raw, layer, color_map, colorbar, palette, size, alpha, linewidth, linecolor, perc, groups, sort_order, components, projection, legend_loc, legend_loc_lines, legend_fontsize, legend_fontweight, legend_fontoutline, legend_align_text, xlabel, ylabel, title, fontsize, figsize, xlim, ylim, add_density, add_assignments, add_linfit, add_polyfit, add_rug, add_text, add_text_pos, add_margin, add_outline, outline_width, outline_color, n_convolve, smooth, normalize_data, rescale_color, color_gradients, dpi, frameon, zorder, ncols, nrows, wspace, hspace, show, save, ax, **kwargs)
    305     color = default_color(adata, add_outline)
    306 if "cmap" not in kwargs:
    307     kwargs["cmap"] = (
--> 308         default_color_map(adata, color) if color_map is None else color_map
    309     )
    310 if "s" not in kwargs:
    311     kwargs["s"] = default_size(adata) if size is None else size

File /opt/homebrew/Caskroom/miniforge/base/envs/velocity/lib/python3.10/site-packages/scvelo/plotting/utils.py:345, in default_color_map(adata, c)
    343 def default_color_map(adata, c):
    344     cmap = None
--> 345     if isinstance(c, str) and c in adata.obs.keys() and not is_categorical(adata, c):
    346         c = adata.obs[c]
    347     elif isinstance(c, int):

File /opt/homebrew/Caskroom/miniforge/base/envs/velocity/lib/python3.10/site-packages/scvelo/plotting/utils.py:55, in is_categorical(data, c)
     53     return cat(data)  # if data is categorical/array
     54 if not is_view(data):  # if data is anndata view
---> 55     strings_to_categoricals(data)
     56 return isinstance(c, str) and c in data.obs.keys() and cat(data.obs[c])

File /opt/homebrew/Caskroom/miniforge/base/envs/velocity/lib/python3.10/site-packages/scvelo/tools/utils.py:301, in strings_to_categoricals(adata)
    296     return (
    297         is_string_dtype(values) or is_integer_dtype(values) or is_bool_dtype(values)
    298     )
    300 df = adata.obs
--> 301 df_keys = [key for key in df.columns if is_valid_dtype(df[key])]
    302 for key in df_keys:
    303     c = df[key]

File /opt/homebrew/Caskroom/miniforge/base/envs/velocity/lib/python3.10/site-packages/scvelo/tools/utils.py:301, in <listcomp>(.0)
    296     return (
    297         is_string_dtype(values) or is_integer_dtype(values) or is_bool_dtype(values)
    298     )
    300 df = adata.obs
--> 301 df_keys = [key for key in df.columns if is_valid_dtype(df[key])]
    302 for key in df_keys:
    303     c = df[key]

File /opt/homebrew/Caskroom/miniforge/base/envs/velocity/lib/python3.10/site-packages/scvelo/tools/utils.py:297, in strings_to_categoricals.<locals>.is_valid_dtype(values)
    295 def is_valid_dtype(values):
    296     return (
--> 297         is_string_dtype(values) or is_integer_dtype(values) or is_bool_dtype(values)
    298     )

File /opt/homebrew/Caskroom/miniforge/base/envs/velocity/lib/python3.10/site-packages/pandas/core/dtypes/common.py:1322, in is_bool_dtype(arr_or_dtype)
   1319     return False
   1321 if isinstance(dtype, CategoricalDtype):
-> 1322     arr_or_dtype = arr_or_dtype.categories
   1323     # now we use the special definition for Index
   1325 if isinstance(arr_or_dtype, ABCIndex):
   1326     # Allow Index[object] that is all-bools or Index["boolean"]

File /opt/homebrew/Caskroom/miniforge/base/envs/velocity/lib/python3.10/site-packages/pandas/core/generic.py:5583, in NDFrame.__getattr__(self, name)
   5576 if (
   5577     name not in self._internal_names_set
   5578     and name not in self._metadata
   5579     and name not in self._accessors
   5580     and self._info_axis._can_hold_identifiers_and_holds_name(name)
   5581 ):
   5582     return self[name]
-> 5583 return object.__getattribute__(self, name)

AttributeError: 'Series' object has no attribute 'categories'

AnnData object with n_obs × n_vars = 14662 × 3000
    obs: 'n_genes_by_counts', 'log1p_n_genes_by_counts', 'total_counts', 'log1p_total_counts', 'pct_counts_in_top_50_genes', 'pct_counts_in_top_100_genes', 'pct_counts_in_top_200_genes', 'pct_counts_in_top_500_genes', 'total_counts_mt', 'pct_counts_mt', 'initial_size_spliced', 'initial_size_unspliced', 'initial_size', 'n_counts', 'velocity_self_transition', 'leiden', 'SCT_snn_res.0.8', 'SCT_snn_res.0.5', 'SCT_snn_res.0.6'
    var: 'gene_name', 'gene_id', 'mt', 'n_cells_by_counts', 'mean_counts', 'log1p_mean_counts', 'pct_dropout_by_counts', 'total_counts', 'log1p_total_counts', 'gene_count_corr', 'means', 'dispersions', 'dispersions_norm', 'highly_variable', 'velocity_gamma', 'velocity_qreg_ratio', 'velocity_r2', 'velocity_genes'
    uns: 'leiden', 'leiden_colors', 'neighbors', 'pca', 'umap', 'velocity_graph', 'velocity_graph_neg', 'velocity_params'
    obsm: 'X_pca', 'X_tsne', 'X_umap', 'velocity_pca', 'velocity_umap'
    varm: 'PCs'
    layers: 'Ms', 'Mu', 'spliced', 'unspliced', 'variance_velocity', 'velocity'
    obsp: 'connectivities', 'distances'

sc.pl.umap(adata, color='SCT_snn_res.0.8')

WeilerP · 2022-02-12T06:10:05Z

@akhst7, as the issue already says, the problem is pandas==1.4.0. Please see my previous reply and downgrade to pandas==1.3.5.

akhst7 · 2022-02-12T15:10:25Z

@WeilerP, Ok. I misread your post. Thanks.

akhst7 · 2022-02-14T00:43:25Z

Just updated pandas to 1.4.1 and it works.

Excludes `pandas==1.4.0` to prevent failure as reported in theislab#811.

Excludes `pandas==1.4.0` to prevent failure as reported in #811.

WeilerP · 2022-03-14T07:25:01Z

Closed via #838.

* Add .editorconfig (#529) * Add TestCleanObsNames (#530) Add test class for `scvelo/core/_anndata.py::clean_obs_names`. * Refactor `clean_obs_names` (#532) * Rename input arguments from `data` to `adata`, `copy` to `inplace`, `ID_length` to `id_length`, `base` to `alphabet`. * Rename variables to something more meaningful and use snake as well as lower case, consistently. * Make use of Pandas functionality. * Use regex expression. * Move unit tests to `tests/` (#535) * Move `scvelo/core/tests` to `tests/core` Moves unit tests `scvelo/core/tests` to `tests/core`. This will, for example, prevent installation files becoming large when including data or ground truth figures. * Add `tests/__init__.py` Makes `tests/` a package. This is needed to e.g. import from `tests/core` later on. * Update pytest.ini Remove `scvelo` from testpaths. * Fix and refactor `filter_genes` (#537) * Add `multiply` Adds function to calculate the element-wise product of two arrays or an array and a sparse matrix. * Fix and refactor `filter_genes` The original implementation fails when `adata.layers['unspliced']` is sparse but `adata.layers['spliced']` dense. The element-wise multiplication is now performend using `multiply`. * Add column generation for adata.obs/.var (#544) * Fix and update docstrings Update docstrings to follow codebase style. * Add option to add columns to adata.obs * Adds `obs_col_names`, `min_obs_cols`, `max_obs_cols` to composite strategy `get_adata`. Using `obs_col_names`, the column names can be set manually, the other arguments allow setting the minimum and maximum number of columns generated. * Updates unit tests to encounter for and test changes. * Add option to add columns to adata.var * Adds `var_col_names`, `min_var_cols`, `max_var_cols` to composite strategy `get_adata`. Using `var_col_names`, the column names can be set manually, the other arguments allow setting the minimum and maximum number of columns generated. * Updates unit tests to encounter for and test changes. * Add unit tests for `cleanup` (#548) * Add method `_subset_columns` to `TestBase` Adds method to sample from column names in `adata.obs` and `adata.var`. * Add `TestCleanup` Adds test class to unit test `scvelo/core/_anndata.py::cleanup`. * Fix TestCleanup::test_cleanup_some (#550) * Refactor `cleanup` (#552) * Rename argument `data` to `adata` * Rename argument `copy` to `inplace` * Update type hint in `cleanup` * Rename variables Rename variables to use more informative names. * Add `TestGetInitialSize` (#554) * Add `TestGetSize` (#556) * Refactor module import (#560) * Delete `pl.py`, `pp.py` and `tl.py`. * Update `__init__.py` to no longer rely on `pl.py`, `pp.py` and `tl.py`. * Update `get_modality` (#567) * Update `get_modality` Support passing `None` for argument `modality`. In this case, `adata.X` is returned. * Add `TestGetModality::test_modality_equals_none` Add unit test to check that correct modality is extracted if `None` is passed for `modality` in `get_modality`. * Refactor and generalize `get_size` (#569) * Rename `layer` to `modality` Rename argument `layer` to `modality` in `get_size`. This unifies naming convention for the argument across the code base and prepares generalizing `get_size` to arbitrary modalities. * Generalize `get_size` to arbitrary modalities This allows calculating the size per observation for any modality, from `adata.X`, `adata.layers` or `adata.obsm`. * Update docstrings * Add TestObsDf (#579) Unit tests `scvelo.core._anndata.py::obs_df`. * Add TestVarDf (#583) Unit tests `scvelo.core._anndata.py::var_df`. * Update TestObsDf (#585) The addition assert statement checks that the index of the returned data frame is correct. * Fix typo in default layer names (#587) * Add `TestShowProportions` (#589) Unit tests `scvelo.core._anndata.py`::show_proportions`. * Add `TestSetInitialSize` (#591) * Reduce maximum number of obs/vars (#593) Reduce maximum size of generated AnnData objects in some tests. * Update ci.yml (#546) * Run tests on ubuntu-latest using both Python 3.7 and 3.8. * Increase verbosity of pytest. * Add timeout to tests. * Update `.gitignore` [ci skip] (#595) Add files and directories created by `pytest-cov`. * Update `_anndata.py` (#596) Change order of default layers to match temporal ordering in biological process. * Fix TestShowProportions (#598) In TestShowProportions::test_layers_not_specified, layers should not be specified but `layers=None` passed instead. * Update `TestCleanObsNames` (#600) Check that `inplace` argument is working correctly. * Test modality passed as string (#602) Add unit tests to `TestMakeDense` and `TestMakeSparse` when `modality` is passed as a string. * Fix `test_not_existing_modality` (#605) Generate layer names with at least two characters to exclude the case `layer='X'`. * Refactor unit test for `l2_norm` (#607) Converts unit test to a test class. 2D arrays and sparse matrices are now generated as well to increase coverage to 100%. * Add `varm` to AnnData strategy (#609) * Add `varm` to AnnData strategy Strategy can now also add entries to `adata.varm`. This is tested by extending the existing unit tests. * Set `max_obs` and `max_vars` to `5` This is an attempt to make the unit tests not in the CI on GitHub due to a time out error by Hypothesis. * Add `TestMerge` (#611) Add test class to unit test `scvelo/core/_anndata.py::merge`. * Update `TestVarDf` (#615) Reduce maximum size of generated AnnData. * Catch error in steady state calculation (#614) * Fix steady state calculation Raise ValueError when `beta` and `gamma` are not positive. * Update `TestSplicingDynamics` Add unit test to verify ValueError is raised when calculating steady states with inadmissible parameter values. * Fix AnnData strategy (#616) Reset `min_varm` and `max_varm` if `varm_keys` is specified. * Update `TestSplicingDynamics` (#618) Add unit tests to * test 2d time points, * verify correct initial states are set/returned, * verify correct steady states are returned. * Fix `TestCleanup` (#620) * Update `DynamicsBase` (#622) Remove redundant return statements to increase test coverage. The abstract methods cannot be tested. * Add `TestGetDf` (#624) Unit tests for `scvelo/core/_anndata.py::get_df`. * Add module `datasets` (#636) * Add module `datasets` The file `datasets.py` is moved to the module and renamed to `_datasets.py`. * Deprecate `pancreatic_endocrinogenesis` * Add `datasets/_simulate.py` Adds file `datasets/_simulate.py` and moves `simulation`, `unspliced`, `spliced` and `vectorize` from `datasets/_datasets.py` to the new file. * Cleanup `datasets/_datasets.py` (#638) * Sort functions alphabetically * Add TODO * Fix `DeprecationWarning` [ci skip] (#640) * Add argument `file_path` (#642) Allow saving/reading datasets to/from custom paths. * Fix gillespie simulation (#644) * Add unit tests for `scvelo.datasets` (#646) * Add `test_datasets.py` Add unit tests for datasets accessible through scvelo. * Add `test_simulate.py` Add unit tests for `scvelo.datasets.simulation`. * Add new datasets to `datasets/__init__.py` (#677) * Fix `csr_vcorrcoef` (#679) * Fix `csr_vcorrcoef` Fixes calculation of Pearson's correlation in `csr_vcorrcoef`. * Add `tests/preprocessing` * Add `TestCsrVcorrcoef` Test class for unit testing `csr_vcorrcoef`. * Update Python versions in CI (#680) * Run linting check in Python 3.8 * Run unit tests in Python 3.9 * Fix and unit test `get_mean_var` (#698) * Fix passing `perc` not affecting result Up to now, passing `perc` does not affect the output as only `data` but not `X` is manipulated. * Fix different size parameters Mean and variance calculation use different size parameters instead of one and the same. * Fix sparse input with `ignore_zeros` or nan values Mean and variance cannot be calculated due to a shape mismatch. * Add `TestGetMeanVar` Unit test `test_get_mean_var`. * Add unit tests for `preprocessing/utils.py` (#700) * Add `TestCheckIfValidDtype` Unit tests `check_if_valid_dtype`. * Add `TestFilter` Unit tests `_filter`. * Add `TestLog1p` Unit tests `log1p`. * Add `TestMaterializeAsNdarray` Unit tests `materialize_as_ndarray`. * Add `TestNotYetNormalized` Unit tests `not_yet_normalized`. * Add `TestFilterGenes` Unit tests `filter_genes`. * Add `TestCountsPerCellQuantile` * Add `TestNormalizePerCell` * Add `tests/_data/` Add subsetted datasets to use in unit tests. * Add `confest.py` Add file to declare fixture for entire testing directory. * Add `TestFilterGenesDispersion` Unit test `filter_genes_dispersion`. * Add `TestFilterAndNormalize` Unit test `filter_and_normalize`. * Add `TestRecipeVelocity` Unit test `recipe_velocity`. * Update AnnData generation for testing (#785) * Update `test_base.py::get_adata` Exclude `"X"` as key of `.layers`, `.obsm` and `.varm`. * Update `test_base.py::TestAdataGeneration` Check that `"X"` is not a key in `.layers`, `.obsm`, `.varm`. * Update `test_anndata.py` Remove redundant statements in `TestCleanup::test_cleanup_some` and `TestSetInitialSize::test_added_columns` according to changes in AnnData generation for tests. * Add `pull_request_template.md` [ci skip] (#787) Add template for pull requests. * Speed up CI (#789) * Update `ci.yml` Remove testing of dataset download from job `test`. Instead, a new job `test-dataset-downloads` is added. * Update `ci.yml` Print execution time of 25 slowest tests. * Update `test_datasets.py` Skip `test_datasets.py::TestDataSets` if Python version is not 3.8 and OS is not Linux. * Refactor `neighbors` (#795) * Split body of `neighbors` over several functions Split body of `neighbors` over several function for easier unit testing. * Add type hints to arguments of `neighbors` * Fix typo in docstrings * Fix `_get_rep` (#797) The variable `rep` is referenced before assignment if the if-else case is not entered. This is fixed by adding a final `else` case. In the following, only the variable `rep` is used. * Refactor testing in CI (#801) * Update `requirements-dev.txt` Add `pytest-cov` to the installed libraries. * Update `ci.yml` Update how dependencies are installed in the `test` job. * Increase Hypothesis deadline in CI (#804) To prevent test failure due to exceeding deadlines, the default deadline of Hypothesis is increased to 500 milliseconds when running the tests in the CI. * Unit test `neighbors.py` (#799) * Add `test_neighbors.py` Add file to host unit tests for `preprocessing/neighbors.py`. * Add `TestGetNeighs` * Add `TestGetNNeighs` * Add `TestGetDuplicateCells` * Add `TestRemoveDuplicateCells` * Add preprocessed adata for testing Add preprocessed versions of `dentategyrus_50obs.h5ad`, `dentategyurs_100obs.h5ad`, `pancreas_50obs.h5ad`, and `pancreas_100obs.h5ad`. * Add fixtures for preprocessed datasets Add fixtures to use preprocessed datasets in unit tests. * Add fixture `adata` The fixture `adata` is used to easily load raw or preprocessed datasets in unit tests. This allows running one and the same unit test on all available datasets. * Add `TestSelectConnectivities` Add unit tests for `select_connectivities`. * Add `TestSelectDistances` Add unit tests for `select_distances`. * Add `TestSetDiagonal` Add unit tests for `set_diagonal`. * Add `TestNeighborsToBeRecomputed` Add unit tests for `neighbors_to_be_recomputed`. * Add `TestVerifyNeighbors` Add unit tests for `verify_neighbors`. * Add `TestGetConnectivities` and accompanying files Add unit tests for `get_connectivities` and saved CSR matrices to verify results. * Add `TestGetCsrFromIndices` Add unit tests for `get_csr_from_indices`. * Add `TestComputeConnectivitiesUmap` Add unit tests for `compute_connectivities_umap` * Add `TestGetRep` Add unit tests for `_get_rep`. * Add `TestSetPCA` Add unit tests for `_set_pca`. * Add `TestGetHnswNeighbors` Add unit tests for `_get_hnsw_neighbors` and corresponding CSR matrices saved using `.npz` format. * Add `TestGetScanpyNeighbors` Add unit tests for `_get_scanpy_neighbors` and corresponding CSR matrices saved using `.npz` format. * Add `TestGetSklearnNeighbors` Add unit tests for `_get_sklearn_neighbors` and corresponding CSR matrices saved using `.npz` format. * Add `TestNeighbors` Adds unit tests for the function `neighbors`. * Unit test `moments.py` (#806) * Add `test_moments.py` Add file to host unit tests of functions in `moments.py`. * Add `TestGetMoments` Add unit tests for `get_moments` and corresponding ground truth moments saved using `.npy` format. * Add `TestSecondOrderMomentsU` Add unit tests for `second_order_moments_u`. * Add `TestSecondOrderMoments` Add unit tests for `second_order_moments` and corresponding moments saved using `.npy` format. * Add `TestMagicImpute` Add unit tests for `magic_imput` and corresponding imputed data saved using `.npy` format. * Add `TestMoments` Add unit tests for `moments` and corresponding first order moments saved using `.npy` format. * Update `requirements-dev.txt` Add `magic-impute` to run unit tests on CI. * Update `docs/` [ci skip] (#810) Recommend `kb count` instead of `loompy fromfq`. * Update `requirements.txt` (#838) Excludes `pandas==1.4.0` to prevent failure as reported in #811. * Fix `optimization.py::get_weight` (#839) Fix `scvelo.tools.optimization.py::get_weight` to work correctly when `perc` is numeric. * Fix inference with `fit_scaling=False` (#848) * Update `DynamicsRecovery` Fix parameter inference to not fit scaling when `fit_scaling=False`. * Update `pre-commit` setup (#855) * Update `requirements-dev.txt` Update versions of `black` and `isort`. * Update `.pre-commit-config.yaml` Update repo paths and versions. * Run `black` Run code formatting with latest version of `black`. * Add argument `file_path` (#853) Adds argument `file_path` to datasets added in `scvelo=0.2.4.`. * Update `CONTRIBUTING.rst` [ci skip] (#911) Details workaround to issue where `pip install -e '.[dev]'` leads to errors on Windows. * Fix saving stream embedding figure (#900) * Fix missing write_key for embedding stream plot. * Set save instead of wire_key when saving embedding stream plot. * Fix `clean_obs_names` (#921) * Fix bug with clean_obs_names Switched implementation to use the previously ignored id_length parameter. New approach finds a substring of length id_length where each of the letters are in the alphabet * Extend tests to cover more cases for clean_obs_names * Fix Pandas display precision (#907) * Update `core/_anndata.py` Update how Pandas displayed precision is set. * Update `core/_anndata.py` Add TODO comment to add a unit test to test the `precision` argument. * Fix documentation issues [ci skip] (#929) * Update `CONTRIBUTING.rst` [ci skip] Update branch name convention. * Update `docs/requirements.txt` [ci skip] Pin `Jinja` version. * Update `docs/source/conf.py` [ci skip] Update `add_stylesheet` with `add_css_file`. The former is deprecated. * Fix typo in `index.rst` [ci skip] * Fix link to bone marrow dataset [ci skip] * Fix `velocity_embedding` docs [ci skip] * Add key contributors section [ci skip] (#930) * Add `_key_contributors.rst` [ci skip] Add file with key contributors and their roles. * Show key contributors on index page [ci skip] * Fix deprecation of keyword argument `copy` (#932) * Add `core/_utils.py` * Add `core/_utils.py::deprecated_arg_names` Modify wrapper from Scanpy to replace old with new keyword argument. The wrapper is modified s.t. `copy=True` corresponds to `inplace=False`, and vice versa. * Make `deprecated_arg_names` importable from `core` * Update import of `deprecated_arg_names` Import local version instead of the Scanpy version s.t. the keyword argument `copy` is handled correctly. * Update `CONTRIBUTING.rst` [ci skip] (#938) Remove reference to `develop` branch as it will be removed in the next release. * Fix linear regression unit tests (#940) * Fix `TestLinearRegression::test_perfect_fit` Replace `array` strategy to generate `x` with `sampled_from`. * Fix `TestLinearRegression::test_perfect_fit_2d` Replace `array` strategy to generate `x` with `sampled_from`. * Update `test_neighbors.py` (#941) Remove failing and non-essential unit tests. See #922. * Update release notes [ci skip] (#945) * Remove trailing white spaces * Add notes for `0.2.5` release Co-authored-by: Isaac Virshup <[email protected]> Co-authored-by: Oisin-M <[email protected]> Co-authored-by: Dries Schaumont <[email protected]>

streelmanlab · 2023-06-08T19:18:02Z

Hi I had a similar error. When trying to use velocity_embedding_stream, I got the following error:

Traceback (most recent call last):
File "", line 1, in
File "/storage/home/hcoda1/6/ggruenhagen3/p-js585-0/George/rich_project_pb1/conda_envs/velo3/lib/python3.11/site-packages/scvelo/plotting/velocity_embedding_stream.py", line 252, in velocity_embedding_stream
ax = scatter(
^^^^^^^^
File "/storage/home/hcoda1/6/ggruenhagen3/p-js585-0/George/rich_project_pb1/conda_envs/velo3/lib/python3.11/site-packages/scvelo/plotting/scatter.py", line 668, in scatter
set_legend(
File "/storage/home/hcoda1/6/ggruenhagen3/p-js585-0/George/rich_project_pb1/conda_envs/velo3/lib/python3.11/site-packages/scvelo/plotting/utils.py", line 555, in set_legend
obs_vals.cat.categories = obs_vals.cat.categories.astype(str)
^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/home/hcoda1/6/ggruenhagen3/p-js585-0/George/rich_project_pb1/conda_envs/velo3/lib/python3.11/site-packages/pandas/core/base.py", line 178, in setattr
object.setattr(self, key, value)
File "/storage/home/hcoda1/6/ggruenhagen3/p-js585-0/George/rich_project_pb1/conda_envs/velo3/lib/python3.11/site-packages/pandas/core/accessor.py", line 99, in _setter
return self._delegate_property_set(name, new_values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/home/hcoda1/6/ggruenhagen3/p-js585-0/George/rich_project_pb1/conda_envs/velo3/lib/python3.11/site-packages/pandas/core/arrays/categorical.py", line 2460, in _delegate_property_set
return setattr(self._parent, name, new_values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: property 'categories' of 'Categorical' object has no setter

I found the source code for the current verison of scvelo and changed the offending line of the version I had.
From: obs_vals.cat.categories = obs_vals.cat.categories.astype(str)
To:
str_cats = obs_vals.cat.categories.astype(str)
obs_vals = obs_vals.cat.set_categories(str_cats, rename=True)

I don't know why I had this error in the first place since I'm using a verison of scvelo where this bug was fixed.

Versions: python==3.11.3, scvelo==0.2.5, scanpy==1.9.3, pandas=2.0.2, numpy==1.23.5 (I had to downgrade my numpy version, maybe that has something to do with it #1077)

WeilerP · 2023-06-08T19:43:22Z

@streelmanlab, this has been fixed in #1065.

JFerrer-B · 2023-09-26T16:54:59Z

Hi,

I deleted this line from the utils.py file corresponding to the functino get_legend() and now it works.
"obs_vals.cat.categories = obs_vals.cat.categories.astype(str)"

Not need to downgrade pandas.

vasikara17 · 2023-10-12T08:52:17Z

Hello,
unfortunately I still have the same problem with the following versions
pd.version
'2.1.1'
scv.version
'0.2.5'

Is there anything we can do apart from changing the pandas version because I do not have the 1.4.0

ryf0810 · 2023-10-13T20:24:32Z

Hi,

I deleted this line from the utils.py file corresponding to the functino get_legend() and now it works. "obs_vals.cat.categories = obs_vals.cat.categories.astype(str)"

Not need to downgrade pandas.

This works for me, thank you so much!

vasikara17 · 2023-10-15T20:50:55Z

Thank you very much for your reply! I am sorry to ask something so simple but I am not very familiar with this bugs in python. Where do I to change this line of code exactly? With utils.py you mean the library? Gesendet von Outlook für Android<https://aka.ms/AAb9ysg>

…

________________________________ From: ryf0810 ***@***.***> Sent: Friday, October 13, 2023 10:24:45 PM To: theislab/scvelo ***@***.***> Cc: Vasiliki Karagiannakou ***@***.***>; Comment ***@***.***> Subject: Re: [theislab/scvelo] Errors with pandas 1.4 (Issue #811) Hi, I deleted this line from the utils.py file corresponding to the functino get_legend() and now it works. "obs_vals.cat.categories = obs_vals.cat.categories.astype(str)" Not need to downgrade pandas. This works for me, thank you so much! — Reply to this email directly, view it on GitHub<#811 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BCWWOME3YH46SIJUBSNH5BLX7GPQ3ANCNFSM5MVE23BQ>. You are receiving this because you commented.Message ID: ***@***.***> Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH) Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de<https://www.helmholtz-munich.de/> Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

ryf0810 · 2023-10-17T14:13:32Z

Thank you very much for your reply! I am sorry to ask something so simple but I am not very familiar with this bugs in python. Where do I to change this line of code exactly? With utils.py you mean the library? Gesendet von Outlook für Androidhttps://aka.ms/AAb9ysg
…
________________________________ From: ryf0810 @.> Sent: Friday, October 13, 2023 10:24:45 PM To: theislab/scvelo @.> Cc: Vasiliki Karagiannakou @.>; Comment @.> Subject: Re: [theislab/scvelo] Errors with pandas 1.4 (Issue #811) Hi, I deleted this line from the utils.py file corresponding to the functino get_legend() and now it works. "obs_vals.cat.categories = obs_vals.cat.categories.astype(str)" Not need to downgrade pandas. This works for me, thank you so much! — Reply to this email directly, view it on GitHub<#811 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCWWOME3YH46SIJUBSNH5BLX7GPQ3ANCNFSM5MVE23BQ. You are receiving this because you commented.Message ID: @.***> Helmholtz Zentrum München – Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH) Ingolstädter Landstraße 1, D-85764 Neuherberg, https://www.helmholtz-munich.de https://www.helmholtz-munich.de/ Geschäftsführung: Prof. Dr. med. Dr. h.c. Matthias Tschöp | Aufsichtsratsvorsitzende: MinDir’in Prof. Dr. Veronika von Messling Registergericht: Amtsgericht München HRB 6466 | USt-IdNr. DE 129521671

Hi Vasikara,

Comment this line of code 'str_cats = obs_vals.cat.categories.astype(str)' in utils.py under plotting folder, the function is called set_legend in latest version.

Hope this helps!

* update environment for windows using micromamba * try Windows environment resolved using micromamba * add pip dependencies * downgrade matplotlib to 3.7.3 (https://stackoverflow.com/questions/77128061/ydata-profiling-profilereport-attributeerror-module-matplotlib-cbook-has-no) * downgrade pandas (theislab/scvelo#811 (comment)) * remove pip section * #80 (comment) * woops updated env in R code too * downgrade python (theislab/scvelo#1124 (comment)) * matplotlib 3.6.3 * pandas 1.5.2 * numpy 1.21.1 * fix version bump * remove useless pip --------- Co-authored-by: Kevin Rue <[email protected]>

karlann added the bug Something isn't working label Jan 24, 2022

WeilerP added a commit to WeilerP/scvelo that referenced this issue Mar 14, 2022

Update requirements.txt

3188e1d

Excludes `pandas==1.4.0` to prevent failure as reported in theislab#811.

WeilerP mentioned this issue Mar 14, 2022

Exclude pandas==1.4 #838

Merged

WeilerP added a commit that referenced this issue Mar 14, 2022

Update requirements.txt (#838)

b9893ed

Excludes `pandas==1.4.0` to prevent failure as reported in #811.

WeilerP closed this as completed Mar 14, 2022

WeilerP mentioned this issue Apr 14, 2022

Error plotting velocity graph #863

Closed

goepp mentioned this issue Nov 9, 2023

AttributeError: can't set attribute 'categories' eleozzr/desc#58

Closed

kevinrue mentioned this issue Aug 30, 2024

update scvelo for Windows kevinrue/velociraptor#80

Closed

kevinrue added a commit to kevinrue/velociraptor that referenced this issue Aug 30, 2024

downgrade pandas (theislab/scvelo#811 (comment))

3c3338d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Errors with pandas 1.4 #811

Errors with pandas 1.4 #811

karlann commented Jan 24, 2022

ivirshup commented Jan 31, 2022

stela2502 commented Feb 1, 2022

WeilerP commented Feb 1, 2022

akhst7 commented Feb 12, 2022 •

edited

Loading

WeilerP commented Feb 12, 2022

akhst7 commented Feb 12, 2022

akhst7 commented Feb 14, 2022

WeilerP commented Mar 14, 2022

streelmanlab commented Jun 8, 2023

WeilerP commented Jun 8, 2023

JFerrer-B commented Sep 26, 2023

vasikara17 commented Oct 12, 2023

ryf0810 commented Oct 13, 2023

vasikara17 commented Oct 15, 2023 via email

ryf0810 commented Oct 17, 2023

Errors with pandas 1.4 #811

Errors with pandas 1.4 #811

Comments

karlann commented Jan 24, 2022

ivirshup commented Jan 31, 2022

stela2502 commented Feb 1, 2022

WeilerP commented Feb 1, 2022

akhst7 commented Feb 12, 2022 • edited Loading

WeilerP commented Feb 12, 2022

akhst7 commented Feb 12, 2022

akhst7 commented Feb 14, 2022

WeilerP commented Mar 14, 2022

streelmanlab commented Jun 8, 2023

WeilerP commented Jun 8, 2023

JFerrer-B commented Sep 26, 2023

vasikara17 commented Oct 12, 2023

ryf0810 commented Oct 13, 2023

vasikara17 commented Oct 15, 2023 via email

ryf0810 commented Oct 17, 2023

akhst7 commented Feb 12, 2022 •

edited

Loading