Hunting Sparsity: Density-Guided Contrastive Learning for Semi-Supervised Semantic Segmentation

Hunting Sparsity: Density-Guided Contrastive Learning for Semi-Supervised Semantic Segmentation
Xiaoyang Wang, Bingfeng Zhang, Limin Yu, and Jimin Xiao.
In CVPR 2023.

Abstract: Inspired by density-based unsupervised clustering, we propose to leverage feature density to locate sparse regions within feature clusters defined by label and pseudo labels. The hypothesis is that lower-density features tend to be under-trained compared with those densely gathered. Therefore, we propose to apply regularization on the structure of the cluster by tackling the sparsity to increase intra-class compactness in feature space. With this goal, we present a Density-Guided Contrastive Learning (DGCL) strategy to push anchor features in sparse regions toward cluster centers approximated by high-density positive keys. The heart of our method is to estimate feature density which is defined as neighbor compactness. We design a multi-scale density estimation module to obtain the density from multiple nearest-neighbor graphs for robust density modeling. Moreover, a unified training framework is proposed to combine label-guided self-training and densityguided geometry regularization to form complementary supervision on unlabeled data.

Getting Started

Installation

cd DGCL
conda create -n dgcl python=3.10
conda activate dgcl
pip install -r requirements.txt

Pretrained Weights

Download pretrained wegiths ResNet-101

├── DGCL/
    └── resnet101.pth

Data Preparation

├── Path_to_Pascal
    ├── JPEGImages
    └── SegmentationClassAug
    
├── Path_to_Cityscapes
    ├── leftImg8bit
    └── gtFine

Training

Navigate into experiments/pascal/732 and modify config.yaml and train.sh.

sh train.sh <num_gpu> <port>

Citation

@inproceedings{wang2023dgcl,
  title= {Hunting Sparsity: Density-Guided Contrastive Learning for Semi-Supervised Semantic Segmentation},
  author={Wang, Xiaoyang and Zhang, Bingfeng and Yu, Limin and Xiao, Jimin},
  booktitle={CVPR},
  year={2023},
}

Acknowledgement

This project borrows codes from U2PL and ReCo. Thanks for their great work!

Contact

For questions, please contact: [email protected]