Rational Supervised Learning

Rational Networks in Supervized Learning such as MNIST, CIFAR and Imagenet Classification Tasks.

Rational functions outperformes every non-learnable one (cf. Padé Activation Units: ...).

Rational are also here used for lesioning. They replace Residual Blocks in a ResNet101:

Eval	Lesion	L2.B3	L3.B13	L3.B19	L4.B2
training	Standard	100.9	120.2	90.5	58.9
	Rational	101.1	120.3	104.0	91.1
testing	Standard	93.1	102.0	97.1	81.7
	Rational	90.5	102.6	97.6	85.3
% dropped params		0.63	2.51	2.51	10.0

Dependencies

• Python 3.6+
• PyTorch
• rational-activations

Using Rational Neural Networks

If you want to use (Recurrent) Rational Networks, an additional README is provided in each folder to explain how to train the networks, make score tables, plots, ...etc

Related Repo

Rational RL - for (Recurrent) Rational Networks on Reinforcement Learning