-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ONT flag default settings #134
Comments
Providing some concrete data to support this change. Here are some figures and associated PDF for the relevant part of my thesis. This is also relevant for #133 In the end, the parameters I used to run mykrobe for Nanore data are
|
To discuss. Essentially we drop our FNs for isoniazid, ethambutol and rifampicin significantly. There is a price, which is elevated FPs for isoniazid. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm raising this so it is on our radar and we don't forget, but the default behaviour when the
--ont
flag is given seems to need more investigation.Currently, when this flag is set, the following parameters are set/overridden
mykrobe/src/mykrobe/cmds/amr.py
Lines 344 to 361 in 6a7e7f6
And this also runs the simulations.
As part of mbhall88/head_to_head_pipeline#75 and mbhall88/head_to_head_pipeline#76 I have been trying some different error rates. In order to stop mykrobe from overriding the error rate I was giving though I couldn't provide the
--ont
flag. So I provided-e 0.08 --ploidy haploid
but didn't realise there were the other settings that this flag also changed. The results from the-e 0.08 --ploidy haploid
settings were considerably better than using the default--ont
settings. But what I had accidentally discovered was that if I also provided the other default parameters such as--model kmer_count --conf_percent_cutoff 90
, which are normally set when--ont
is passed, I get much worse results than just setting the error rate and ploidy. By "worse" I mean I get a lot more FNs. It does indeed improve the FPs (actually make FPR better than Illumina), but at a big cost to the FNR.So it seems like either the kmer count model, or the simulations of confidence thresholds (or both) need further work.
The text was updated successfully, but these errors were encountered: