Braille letter reading: A benchmark for spatio-temporal pattern recognition on neuromorphic hardware
You can find the Frontiers publication "Braille letter reading: A benchmark for spatio-temporal pattern recognition on neuromorphic hardware" here.
Braille is a tactile writing system used by people who are visually impaired. These characters have rectangular blocks called cells that have tiny bumps called raised dots. The number and arrangement of these dots distinguish one character from another. For more details and background information see here.
The dataset is composed of different levels of complexity from single letters to words. The 27 letters (Space + A - Z) have been recorded using the iCub fingertip sliding over 3d printed stimuli. For that, the fingertip was mounted on a 3-axis robot (omega.3, forcedimensions) and moved over single braille letters 50 times each with similar velocity (0.01 m/s) at a sampling frequency of 40Hz. The data is converted into spike trains afterward. Delta coding is used for the conversion. No additional noise is added because the analog recordings already contain sensor noise. Binary events ('ON'/'OFF') are created when a predefined threshold is reached followed by a refractory period. At the end of the refractory period, change is accumulated again, until the threshold is reached and a new event is elicit. Thresholds and refractory period are (0.5 for ON and OFF) and (0.0025 sec) respectively. The recordings of the single letters spike trains are combined to compose words.
| Experimental Setup | Encoding Scheme |
|---|---|
 |
 |
| Scanning | Sample-based | Event-based |
|---|---|---|
 |
 |
 |
- Install Python, PyTorch, NumPy, scikit-learn, pandas and matplotlib for plotting
- Download the dataset from Zenodo
- Extract the files and add them in the main folder of this repository
- Run the jupiter notebook for the feedforawrd SNN and/or the recurrent SNN
- If you want to use encoding thresholds not already contained run this file with your personal parameters again and change the data loading in the notebooks accordingly