Panoptic Feature Pyramid Networks

This is an unofficial implementation of Panoptic-FPN in a gluon-cv style, we implemented this framework in a fully Gluon API, please stay tuned!

Main Results

Cityscapes

• panoptic_fpn_resnet50_v1b_citys

-	PQ	SQ	RQ	N
All	55.4	77.9	69.3	19
Things	52.4	78.1	66.6	8
Stuff	57.6	77.7	71.2	11

Installation

1. Install cuda 10.0 and mxnet 1.4.0.

sudo pip3 install mxnet-cu100==1.4.0.post0

2. Clone the code, and install gluoncv with setup.py.

cd panoptic-fpn-gluon
sudo python3 setup.py build
sudo python3 setup.py install

Preparation

Cityscapes

1. Download Cityscapes datasets follow the official tutorials and create a soft link.

ln -s $DOWNLOAD_PATH ~/.mxnet/datasets/citys

You can also download from Cityscapes and execute the command above.

2. Create Panoptic images for training and Inference, the code can be found here. Then execute the command below:

python3 createPanopticImgs.py --dataset-folder ~/.mxnet/datasets/citys/gtFine/ --output-folder ~/.mxnet/datasets/citys/gtFine/

Note that the correct data structure is shown below:

$ ls ~/.mxnet/datasets/citys
├── gtFine
│   ├── train/
│   ├── val/
│   ├── test/
│   ├── cityscapes_panoptic_train/
│   ├── cityscapes_panoptic_val/
│   ├── cityscapes_panoptic_test/
│   ├── cityscapes_panoptic_train.json
│   └── cityscapes_panoptic_val.json
├── leftImg8bit
│   ├── train/
│   ├── val/
│   └── test/

3. More preparations can also refer to GluonCV.

4. All experiments are performed on 8 * 2080ti GPU with Python3.5, cuda10.0 and cudnn7.5.0.

COCO

• TODO

Structure

* Model : $ROOT/gluoncv/model_zoo/panoptic/
* Train & valid scripts : $ROOT/scripts/panoptic/
* Metric : $ROOT/gluoncv/utils/metric/

Training & Inference

Cityscapes

1. Clone the training scripts here, then train panoptic_fpn_resnet50_v1b_citys with:

python3 train_panoptic_fpn.py --network resnet50_v1b --use-fpn --gpus 0,1,2,3,4,5,6,7 --num-workers 32 --log-interval 10 --save-interval 20 --val-interval 10 --epochs 700 --lr-decay-epoch 430,590 --lr-warmup 1600

Note that we follow the training settings described in original paper.

2. Clone the validation scripts here, then validate panoptic_fpn_resnet50_v1b_citys with:

python3 eval_panoptic_fpn.py --network resnet50_v1b --gpus 0,1,2,3,4,5,6,7 --pretrained ./XXX.params

COCO

• TODO

Reference

• Panoptic FPN: Alexander Kirillov, Ross Girshick, Kaiming He, Piotr Dollár.
  "Panoptic Feature Pyramid Networks." CVPR (2019 oral). [paper]