Local Stochastic Gates Model Example (Supervised)

• Single-file python script that defines and trains Local STG NNet with Linear classifier
• Here we use MNIST subset (first 10K samples for train set and first 1K samples for test)
• The hyperparameters were not tuned to the best, you can try to modify LocalSTG hidden dimension (the dafault is 128), regularization loss term multiplier (the dafault is 0.5) or Trainer arguments.

Samples (Test ACC 86.5% on 1K subset):

Original Image	Sparse Image

Citations:

• Feature Selection using Stochastic Gates
• Locally Sparse Neural Networks for Tabular Biomedical Data