Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error in tl.velocity: LinAlgError: SVD did not converge #856

Closed
SimoniMD opened this issue Apr 9, 2022 · 3 comments
Closed

Error in tl.velocity: LinAlgError: SVD did not converge #856

SimoniMD opened this issue Apr 9, 2022 · 3 comments
Labels
bug Something isn't working

Comments

@SimoniMD
Copy link

SimoniMD commented Apr 9, 2022

Hi all,

Getting a bug so close to the end! Hope someone has any ideas...

Here is my sample

> sample500
View of AnnData object with n_obs × n_vars = 500 × 58367
    obs: 'barcode', 'batch'
    var: 'gene_id'
    obsm: 'X_umap'
    layers: 'matrix', 'spliced', 'unspliced'

scv.pp.filter_and_normalize(sample500)
scv.pp.moments(sample500)
scv.tl.velocity(sample500, mode = "stochastic")
scv.tl.velocity_graph(sample500)
scv.pl.velocity_embedding(sample500, basis = 'umap')

I know it's happening on the tl.velocity line because i can split up the code and it stops there.

computing velocities
---------------------------------------------------------------------------
LinAlgError                               Traceback (most recent call last)
[<ipython-input-25-646b6f385bd1>](https://localhost:8080/#) in <module>()
----> 1 scv.tl.velocity(sample500, mode = "stochastic")
      2 scv.tl.velocity_graph(sample500)
      3 scv.pl.velocity_embedding(sample500, basis = 'umap')

5 frames
<__array_function__ internals> in pinv(*args, **kwargs)

<__array_function__ internals> in svd(*args, **kwargs)

[/usr/local/lib/python3.7/dist-packages/numpy/linalg/linalg.py](https://localhost:8080/#) in _raise_linalgerror_svd_nonconvergence(err, flag)
     95 
     96 def _raise_linalgerror_svd_nonconvergence(err, flag):
---> 97     raise LinAlgError("SVD did not converge")
     98 
     99 def _raise_linalgerror_lstsq(err, flag):

LinAlgError: SVD did not converge
Versions 
scvelo==0.2.4  scanpy==1.9.1  anndata==0.8.0  loompy==3.0.7  numpy==1.21.5  scipy==1.4.1  matplotlib==3.5.1  sklearn==1.0.2  pandas==1.3.5
@SimoniMD SimoniMD added the bug Something isn't working label Apr 9, 2022
@WeilerP
Copy link
Member

WeilerP commented Apr 12, 2022 •
edited
Loading

@SimoniMD, seems like a numerical issue, i.e., not sure what to do about it in scvelo. Is this the entire stack trace? If not, could you pleas provide all of it that I can see at which step exactly it fails? TBH, ATM, I don't even know where a SVD would be calculated/needed. Does the error also occur if you use mode='determinstic'? You could also check if the error persists when installing the develop branch on GitHub.

@WeilerP
Copy link
Member

WeilerP commented May 4, 2022

Closing this for now. Possibly related to #874.

@WeilerP WeilerP closed this as completed May 4, 2022
@WeilerP WeilerP mentioned this issue May 4, 2022
Open
@BenjaminDEMAILLE
Copy link

Same problem

---------------------------------------------------------------------------
LinAlgError                               Traceback (most recent call last)
Cell In[21], [line 2](vscode-notebook-cell:?execution_count=21&line=2)
      [1](vscode-notebook-cell:?execution_count=21&line=1) scv.pp.moments(adata)
----> [2](vscode-notebook-cell:?execution_count=21&line=2) scv.tl.velocity(adata, mode = "stochastic")
      [3](vscode-notebook-cell:?execution_count=21&line=3) scv.tl.velocity_graph(adata)
      [4](vscode-notebook-cell:?execution_count=21&line=4) scv.pl.velocity_embedding(adata, basis = 'umap')

File [/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:428](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:428), in velocity(data, vkey, mode, fit_offset, fit_offset2, filter_genes, groups, groupby, groups_for_fit, constrain_ratio, use_raw, use_latent_time, perc, min_r2, min_likelihood, r2_adjusted, use_highly_variable, diff_kinetics, copy, **kwargs)
    [419](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:419)         _adata = adata if groups is None else adata[cell_subset]
    [420](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:420)         velo = Velocity(
    [421](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:421)             _adata,
    [422](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:422)             residual=residual,
   (...)
    [426](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:426)             use_highly_variable=use_highly_variable,
    [427](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:427)         )
--> [428](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:428)     velo.compute_stochastic(fit_offset, fit_offset2, mode, perc=perc)
    [430](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:430) write_residuals(adata, vkey, velo._residual, cell_subset)
    [431](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:431) write_residuals(adata, f"variance_{vkey}", velo._residual2, cell_subset)

File [/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:147](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:147), in Velocity.compute_stochastic(self, fit_offset, fit_offset2, mode, perc)
    [141](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:141) res2_std = (cov_us - _gamma2 * var_ss - _offset2).std(0)
    [143](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:143) # solve multiple regression
    [144](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:144) self._offset[idx], self._offset2[idx], self._gamma[idx] = (
    [145](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:145)     maximum_likelihood(_Ms, _Mu, _Mus, _Mss, fit_offset, fit_offset2)
    [146](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:146)     if mode == "bayes"
--> [147](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:147)     else leastsq_generalized(
    [148](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:148)         _Ms,
    [149](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:149)         _Mu,
    [150](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:150)         var_ss,
    [151](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:151)         cov_us,
    [152](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:152)         res_std,
    [153](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:153)         res2_std,
    [154](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:154)         fit_offset,
    [155](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:155)         fit_offset2,
    [156](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:156)         perc,
    [157](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:157)     )
    [158](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:158) )
    [160](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:160) self._residual = self._Mu - self._gamma * self._Ms
    [161](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/velocity.py:161) if fit_offset:

File [/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/optimization.py:173](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/optimization.py:173), in leastsq_generalized(x, y, x2, y2, res_std, res2_std, fit_offset, fit_offset2, perc)
    [171](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/optimization.py:171)     for i in range(n_var):
    [172](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/optimization.py:172)         A = np.c_[x[:, i]]
--> [173](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/optimization.py:173)         gamma[i] = np.linalg.pinv(A.T.dot(A)).dot(A.T.dot(y[:, i]))
    [175](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/optimization.py:175) offset[np.isnan(offset)] = 0
    [176](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/scvelo/tools/optimization.py:176) offset_ss[np.isnan(offset_ss)] = 0

File <__array_function__ internals>:200, in pinv(*args, **kwargs)

File [/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1983](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1983), in pinv(a, rcond, hermitian)
   [1981](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1981)     return wrap(res)
   [1982](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1982) a = a.conjugate()
-> [1983](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1983) u, s, vt = svd(a, full_matrices=False, hermitian=hermitian)
   [1985](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1985) # discard small singular values
   [1986](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1986) cutoff = rcond[..., newaxis] * amax(s, axis=-1, keepdims=True)

File <__array_function__ internals>:200, in svd(*args, **kwargs)

File [/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1642](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1642), in svd(a, full_matrices, compute_uv, hermitian)
   [1639](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1639)         gufunc = _umath_linalg.svd_n_s
   [1641](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1641) signature = 'D->DdD' if isComplexType(t) else 'd->ddd'
-> [1642](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1642) u, s, vh = gufunc(a, signature=signature, extobj=extobj)
   [1643](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1643) u = u.astype(result_t, copy=False)
   [1644](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:1644) s = s.astype(_realType(result_t), copy=False)

File [/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:98](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:98), in _raise_linalgerror_svd_nonconvergence(err, flag)
     [97](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:97) def _raise_linalgerror_svd_nonconvergence(err, flag):
---> [98](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/numpy/linalg/linalg.py:98)     raise LinAlgError("SVD did not converge")

LinAlgError: SVD did not converge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@WeilerP @BenjaminDEMAILLE @SimoniMD