Understanding Model Process from Tutorials

[ZoltanV-V]

Hello,

I'm still a beginner to neuromorphic computing and I've so far only explored the various lava-dl tutorials so I'm sorry if this might be a really novice question to ask or if I might have just misunderstood something. I was reading through the end-to-end MNIST classifier tutorial for lava and I had a question about creating a process for the classifier model. The ImageClassifier Process - does it essentially work the same as loading the hdf5 with NetX?

I understood from lava-dl tutorials that Lava processes currently cannot be trained directly so training would have to be done with either Lava-dl's Slayer or Bootstrap. After training, you can save the model in hdf5 format and NetX can later convert it into a Lava Process. What would be the advantage of defining a Process from scratch (like ImageClassifier Process) vs just using NetX?

A follow-up question also: the .npy weights file, is from a model pre-trained using lava-dl?

[bamsumit]

Hi @ZoltanV-V, the ImageClassifier process describes the MNIST network. It is in a way similar to NetX. The main difference is that NetX automatically loads the trained parameters and connects the relevant neurons and synapses and creates the lava process for you whereas the example show how you can do it all yourself. It is definitely useful to understand how you can describe your process manually as in the NMNIST example. However, as the networks grow complex, NetX automation provides one call creation of lava process for your trained network and, as a result, minimizes any errors that may happen during implementation.

A follow-up question also: the .npy weights file, is from a model pre-trained using lava-dl?

No. That network was trained using ANN-SNN conversion.

[ZoltanV-V]

Thank you so much, that cleared up everything!

No. That network was trained using ANN-SNN conversion.

does this mean if we have an ANN model converted to SNN (say using snntoolbox) we can also load it as a lava process this way?

[bamsumit]

Yes. You can. It should also be quite easy to generate a hdf5 NetX specification for third party frameworks and use lava-dl NetX. You can look at the description of hdf5 standard here.