pytorch-slimming

This is a PyTorch re-implementation of algorithm presented in "Learning Efficient Convolutional Networks Through Network Slimming (ICCV2017)." . The official source code is based on Torch. For more info, visit the author's webpage!.

CIFAR10-VGG16BN	Baseline	Trained with Sparsity (1e-4)	Pruned (0.7 Pruned)	Fine-tuned (40epochs)
Top1 Accuracy (%)	93.62	93.77	10.00	93.56
Parameters	20.04M	20.04M	2.42M	2.42M

Pruned Ratio	0	0.1	0.2	0.3	0.4	0.5	0.6	0.7
Top1 Accuracy (%) without Fine-tuned	93.77	93.72	93.76	93.75	93.75	93.40	37.83	10.00
Parameters(M) / macc(M)	20.04/ 398.44	15.9/ 349.22	12.28/ 307.78	9.12/ 272.94	6.74/ 247.86	4.62/ 231.86	3.14/ 222.17	2.42/ 210.84

Pruned Ratio	architecture
0	[64, 64, 'M', 128, 128, 'M', 256, 256, 256, 256, 'M', 512, 512, 512, 512, 'M', 512, 512, 512, 512]
0.1	[60, 64, 'M', 128, 128, 'M', 256, 255, 253, 245, 'M', 436, 417, 425, 462, 'M', 463, 465, 472, 424]
0.2	[58, 64, 'M', 128, 128, 'M', 256, 255, 250, 233, 'M', 360, 336, 329, 398, 'M', 420, 412, 435, 341]
0.3	[56, 64, 'M', 128, 128, 'M', 256, 254, 249, 227, 'M', 284, 239, 244, 351, 'M', 369, 364, 384, 255]
0.4	[52, 64, 'M', 128, 128, 'M', 256, 254, 247, 218, 'M', 218, 162, 166, 294, 'M', 317, 315, 318, 165]
0.5	[52, 64, 'M', 128, 128, 'M', 256, 254, 245, 214, 'M', 179, 117, 116, 229, 'M', 228, 220, 210, 111]
0.6	[51, 64, 'M', 128, 128, 'M', 256, 254, 245, 213, 'M', 165, 85, 92, 153, 'M', 83, 86, 87, 111]
0.7	[49, 64, 'M', 128, 128, 'M', 256, 254, 234, 198, 'M', 114, 41, 24, 11, 'M', 14, 13, 19, 104]

Baseline

python main.py

Trained with Sparsity

python main.py -sr --s 0.0001

Pruned

python prune.py --model model_best.pth.tar --save pruned.pth.tar --percent 0.7

Fine-tuned

python main.py -refine pruned.pth.tar --epochs 40

Reference

@InProceedings{Liu_2017_ICCV,
    author = {Liu, Zhuang and Li, Jianguo and Shen, Zhiqiang and Huang, Gao and Yan, Shoumeng and Zhang, Changshui},
    title = {Learning Efficient Convolutional Networks Through Network Slimming},
    booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
    month = {Oct},
    year = {2017}
}