README.md

ANN

Asymmetric Non-local Neural Networks for Semantic Segmentation

Introduction

Official Repo

Code Snippet

Abstract

The non-local module works as a particularly useful technique for semantic segmentation while criticized for its prohibitive computation and GPU memory occupation. In this paper, we present Asymmetric Non-local Neural Network to semantic segmentation, which has two prominent components: Asymmetric Pyramid Non-local Block (APNB) and Asymmetric Fusion Non-local Block (AFNB). APNB leverages a pyramid sampling module into the non-local block to largely reduce the computation and memory consumption without sacrificing the performance. AFNB is adapted from APNB to fuse the features of different levels under a sufficient consideration of long range dependencies and thus considerably improves the performance. Extensive experiments on semantic segmentation benchmarks demonstrate the effectiveness and efficiency of our work. In particular, we report the state-of-the-art performance of 81.3 mIoU on the Cityscapes test set. For a 256x128 input, APNB is around 6 times faster than a non-local block on GPU while 28 times smaller in GPU running memory occupation. Code is available at: this https URL.

Results and models

Cityscapes

Method	Backbone	Crop Size	Lr schd	Mem (GB)	Inf time (fps)	Device	mIoU	mIoU(ms+flip)	config	download
ANN	R-50-D8	512x1024	40000	6	3.71	V100	77.40	78.57	config	model | log
ANN	R-101-D8	512x1024	40000	9.5	2.55	V100	76.55	78.85	config	model | log
ANN	R-50-D8	769x769	40000	6.8	1.70	V100	78.89	80.46	config	model | log
ANN	R-101-D8	769x769	40000	10.7	1.15	V100	79.32	80.94	config	model | log
ANN	R-50-D8	512x1024	80000	-	-	V100	77.34	78.65	config	model | log
ANN	R-101-D8	512x1024	80000	-	-	V100	77.14	78.81	config	model | log
ANN	R-50-D8	769x769	80000	-	-	V100	78.88	80.57	config	model | log
ANN	R-101-D8	769x769	80000	-	-	V100	78.80	80.34	config	model | log

ADE20K

Method	Backbone	Crop Size	Lr schd	Mem (GB)	Inf time (fps)	Device	mIoU	mIoU(ms+flip)	config	download
ANN	R-50-D8	512x512	80000	9.1	21.01	V100	41.01	42.30	config	model | log
ANN	R-101-D8	512x512	80000	12.5	14.12	V100	42.94	44.18	config	model | log
ANN	R-50-D8	512x512	160000	-	-	V100	41.74	42.62	config	model | log
ANN	R-101-D8	512x512	160000	-	-	V100	42.94	44.06	config	model | log

Pascal VOC 2012 + Aug

Method	Backbone	Crop Size	Lr schd	Mem (GB)	Inf time (fps)	Device	mIoU	mIoU(ms+flip)	config	download
ANN	R-50-D8	512x512	20000	6	20.92	V100	74.86	76.13	config	model | log
ANN	R-101-D8	512x512	20000	9.5	13.94	V100	77.47	78.70	config	model | log
ANN	R-50-D8	512x512	40000	-	-	V100	76.56	77.51	config	model | log
ANN	R-101-D8	512x512	40000	-	-	V100	76.70	78.06	config	model | log

Citation

@inproceedings{zhu2019asymmetric,
  title={Asymmetric non-local neural networks for semantic segmentation},
  author={Zhu, Zhen and Xu, Mengde and Bai, Song and Huang, Tengteng and Bai, Xiang},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={593--602},
  year={2019}
}