-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[web] ~100 seconds to load model/InferenceSession #11217
Comments
When I try to load this model in with the C# wrapper on Win10 x64, I get this: |
@jdluzen Hmm that's weird - from what I can see it doesn't appear to be an invalid file. It loads correctly and works fine with the web runtime and on https://netron.app Searching for Note that it also takes a very long time to render in Netron due to the large node count. Netron says that there are about 28k nodes (I'll update the first post to mention this). |
There are 41,085 OPs.
|
Thanks @PINTO0309! I've just done a few tests of load/init times for different runtimes:
I wonder if this is a useful test case for finding bottlenecks in the init? Given that Pytorch handles the init much quicker, and given that SwinIR is a fairly popular model, it seems like it's worth looking into? Edit: But I guess it could also be a "problem" with the conversion - ops getting inefficiently converted into many ops, or something? Not sure who to ping here, or if it's appropriate to ping, so apologies if not! @snnn @edgchen1 |
The input size is not fixed and therefore not fully optimized. It is also true that even with optimization, loading protocolbuffers into onnxruntime is several times slower than in Pytorch. If it were me, I would use Torchscript. |
Describe the bug
I'm using this ONNX file for this browser-based SwinIR super-resolution demo, and it works great. The only problem is that it takes about 100 seconds to load the ONNX model. I think this is because the model file has a very large number of nodes (about 28k according to Netron).
I converted this .pth model using the following code (as documented here):
Urgency
None
System information
To Reproduce
Run the following code:
Or you can just open this page, and the model will start loading (see console): https://josephrocca.github.io/super-resolution-js/
Expected behavior
I'm not sure what to expect here, because it may be that a model with so many nodes just inherently takes this long to load, but 100s does seem excessive given how fast the
.pth
loads in python (a few seconds at the most). I guess the root of the issue could also be to do with the behavior oftorch.onnx.export
.The text was updated successfully, but these errors were encountered: