Synaptic delays in Dense Process

[a-t-0]

User story

As a user, I want to be able to specify synaptic delays in the Dense Process to enable algorithmic work.

Note: @a-t-0 suggests dense = Dense(.., delay=5) as an interface to generate a synaptic connection that induces a delay of 5 timesteps before submitting a spike to the output neuron.

Note: A PR has already been filed by another external community member.

Conditions of satisfaction

• User is able to specify a single synaptic delay for all synapses at the Process level.
• User is able to specify different synaptic delays for all synapses at the Process level.
• Synaptic delays are implemented in ProcessModels (float and bit accurate model) for Python CPU compute resource.

Acceptance tests

• Specifying a single synaptic delay for all synapses for float/bit accurate
• Specifying a delay matrix for the synapses for float/bit accurate
• Input validation tests (empty matrix/list, negative values, etc.)

Tasks

• Review PR Implementing Synaptic Delays with Delay Process #267 and figure out what is working and what is not working (PP, 2 hours)
• Figure out user interface (Dense or DelayDense process?) (PP, 2 hours)
• Clean up Implementing Synaptic Delays with Delay Process #267 and add missing parts (PP, 2 days)
• Write unit tests (PP, 1 day)
• Review and merge (1 day)
• Create new user stories for additional features and Loihi enablement (1 hour)

Excerpt from original issue by @a-t-0

• If there is a work-around to simulate a synaptic delay among LIF neurons, please let me know, it would be valuable until the delay is implemented. I thought perhaps including an array of neurons and synapses between the input and output neuron of the Dense() object, might be able to simulate the delay. Alternatively, perhaps there exist other objects that do support the delay between LIF neurons.
• If you have suggestions on where to start implementing a delay for the Dense() object, and or relevant dependencies, please do share.

Thank you everyone for your work on this wonderful platform!

[kds300]

Hello,

I have been working on implementing synaptic delays for lava, and have a working version of it. Instead of directly modifying the Dense() process, I copied the code for that and made a modified I called Delay(). I use a 2D matrix of delays similar to how the weights are input into the Dense() process. I'd be more than happy to share the code that I have so far, and submit a pull request for implementing it once it is finished (it still needs to be tested and optimized).

[awintel]

This sounds great. We haven't been able to prioritize support for delays ourselves so far. Thus we are glad that community members are adding new such features.
Submitting a PR is the way to move this forward. Once submitted, members of our team will take a look and give feedback on the PR. It might make sense to either factor our shared code with DENSE or simply merge it with DENSE.

[frothga]

Here are some things I'd like to add to the user story:

• Support for sparse connectivity, not only dense.
• The minimum delay is 1 cycle (rather than a larger number like 2), both when running as software simulation and while running on neuromorphic hardware.
• Depth of delay is arbitrary. The compiler finds the most efficient way to implement long delays and handles mixed delays off a single neuron.

[PhilippPlank]

@frothga Thanks for the suggestions. These features will not be part of this story, but we will add them to existing user stories or create new ones.
We are currently implementing support for delays in software simulation for Dense as a start.