fix sc.tl.umap for umap-0.5

[ivirshup]

@Koncopd, this seems to work as a bare minimum for making sc.tl.umap work for umap-0.5. ingest still does not work, and that definitely looks complicated to fix.

What would you say to adding a line like

if umap.__version__ >= 0.5:
    raise ImportError("Ingest currently require umap-learn < 0.5")

We could then remove the pin, and free up user's environments.

[Koncopd]

I am investigating what could be done now to fix ingest. I really don't like requiring a different and older version of umap for ingest.

[ivirshup]

Ideally yes, but it would be nice to minimize the loss of functionality in one environment.

I think a short term, sorta hacky, solution could be mimicking NNDescent.__getstate__/ NNDescent.__setstate__. Longer term I think we could try to get something in pynndescent like initialization from a prebuilt graph.

[Koncopd]

I'll try to fix this over the weekend.

[Koncopd]

@ivirshup i see you opened the issue about graph initialization, though i haven't checked yet the things that were added. Do we wait for a new release of pynndescent and use the new features or we try to fix ingest with the current version of pynndescent (i have some code already that partially works)?

upd: ah, i see init_graph should work without hacks now.

[ivirshup]

Since we don't know when a release of pynndescent will go out, I think it's fine to keep this a little hacky for now. I think it can be less hacky than now doing something like this:

from_init = pynndescent.NNDescent(train, n_neighbors=15, init_graph=indices)
from_init._rp_forest = rp_forest
query_indices_init, query_distances_init = from_scratch.query(test)

Once a release of pynndescent comes out we can support doing it the proper way.

I'd say it's up to you whether you want to have the kinda hacky solution or not. I definitely don't want UMAP to be pinned to below 0.5 when we release 1.7 proper, and it would be good for ingest to work with UMAP 0.5.

The only downside I see to the kinda hacky solution as an intermediate is that you're fixing it twice. I don't think it'll be hard to go from this to the clean version however.

---

I haven't looked into what needs to happen for the UMAP embedding transfer stuff to work. Is that pretty straight forward?

[ivirshup]

Are you happy with me to merge this as is, then you can open a PR for ingest support?

[Koncopd]

Yes, i am fine with it.

[ivirshup]

@meeseeksdev backport 1.7.x