How do I use the API to created scaled signature?

[phiweger]

Like, from within Python, how can I do the equivalent of

sourmash compute ... --scaled 100 ...

import sourmash_lib as sm
sig = sm.MinHash(ksize=16, n=1000)
# now scale

Thanks a lot!

[phiweger]

Went into the code base hand found some questions :)

I assume I can initialize a scaled signature like so:

import sourmash_lib as sm
scaled=10000
sig = sm.MinHash(ksize=16, n=1000, max_hash=sm.MAX_HASH/scaled)

Recarding what "scaled" does conceptually. It seems to me that it places an upper bound on the hash (space). When I then add_sequence, what happens to kmers that hash to above that upper bound? Are they discarded? I.e., having initialized the signature as scaled, can I treat it from then on (programmatically) as I would an unscaled signature, trusting in that it takes care of the scaling thing?

You could tell me to look at

mh = new KmerMinHash(n, ksize, is_protein, seed, max_hash)

but I am not very proficient in C++ :(

[ctb]

On Fri, Jun 30, 2017 at 09:14:51AM +0000, Adrian Viehweger wrote: Went into the code base hand found some questions :) I assume I can initialize a scaled signature like so: ``` import sourmash_lib as sm scaled=10000 sig = sm.MinHash(ksize=16, n=1000, max_hash=sm.MAX_HASH/scaled) ```

yes!

Recarding what "scaled" does conceptually. It seems to me that it places an upper bound on the hash (space). When I then `add_sequence`, what happens to kmers that hash to above that upper bound? Are they discarded? I.e., having initialized the signature as scaled, can I treat it from then on (programmatically) as I would an unscaled signature, trusting in that it takes care of the scaling thing?

yes, it places an upper bound on hash space, and they are discarded! you can treat signatures programmatically the same way, yes. but you cannot compare scaled to non-scaled without doing some extra work, and if you compute signatures with two different scaled values you must downsample them to the common scaled value. We have functions to do to that, will go find them and get back to you :)

[phiweger]

thank you, that would be great.

I'm glad I finally got my head around minhash scaling.

btw: are there any references to this scaling technique?

[ctb]

no, not yet. it's mildly novel and still under exploration.

[luizirber]

As a side note, I have a [WIP PR][0] to try to make CLI commands available as Python functions. [0]: #245

[phiweger]

that would be convenient 👍 although I would miss that "in the code trenches" feeling ;)

[phiweger]

@ctb

We have functions to do to that, will go find them and get back to you :)

Did you have a chance to look?

[ctb]

Sorry for the long delay...

checkout the MinHash.downsample_scaled function. Usage:

            max_scaled = max(node.data.minhash.scaled, sig.minhash.scaled)
            mh1 = node.data.minhash.downsample_scaled(max_scaled)
            mh2 = sig.minhash.downsample_scaled(max_scaled)

[ctb]

Please re-open if you have any questions!

[ctb]

#436 adds an example into the docs!