Code for On Point Affiliation in Feature Upsampling, which is an extension for the NeurIPS 2022 paper SAPA: Similarity-Aware Point Affiliation for Feature Upsampling.
The code is tested on Python 3.8.8 and PyTorch 1.9.0.
For the base module (SAPA-B),
cd sapa
python setup.py develop
And for SAPA with dynamic selection (SAPA-D),
cd deformsapa
python setup.py develop
Segformer-B1 | FLOPs | Params | mIoU | bIoU |
---|---|---|---|---|
Bilinear | 15.9 | 13.7 | 42.11 | 28.16 |
Deconv | +34.4 | +3.5 | 40.71 | 25.94 |
PixelShuffle | +34.4 | +3.5 | 41.50 | 26.58 |
CARAFE | +1.5 | +0.4 | 42.82 | 29.84 |
IndexNet | +30.7 | +12.6 | 41.50 | 28.27 |
A2U | +0.4 | +0.1 | 41.45 | 27.31 |
FADE | +2.7 | +0.3 | 43.06 | 31.68 |
SAPA-I | +0.8 | +0 | 43.05 | 30.25 |
SAPA-B | +1.0 | +0.1 | 43.20 | 30.96 |
SAPA-D | +5.3 | +0.8 | 44.68 | 31.74 |
UPerNet | Backbone | Params | mIoU | bIoU |
---|---|---|---|---|
Bilinear | R50 | 66.5 | 41.09 | 28.04 |
Deconv | R50 | +7.1 | 41.43 | 27.72 |
PixelShuffle | R50 | +28.3 | 41.35 | 27.49 |
CARAFE | R50 | +0.3 | 41.49 | 28.29 |
IndexNet | R50 | +25.2 | 41.42 | 27.88 |
A2U | R50 | +0.1 | 41.37 | 27.71 |
FADE | R50 | +0.1 | 41.83 | 27.92 |
SAPA-I | R50 | +0 | 41.51 | 28.60 |
SAPA-B | R50 | +0.1 | 41.47 | 28.27 |
SAPA-D | R50 | +0.1 | 42.60 | 28.97 |
Bilinear | R101 | 85.5 | 43.33 | 30.21 |
SAPA-D | R101 | +0.1 | 44.31 | 31.47 |
Faster R-CNN | Backbone | Params | ||||||
---|---|---|---|---|---|---|---|---|
Nearest | R50 | 46.8 | 37.4 | 58.1 | 40.4 | 21.2 | 41.0 | 48.1 |
Deconv | R50 | +2.4 | 37.3 | 57.8 | 40.3 | 21.3 | 41.1 | 48.0 |
PixelShuffle | R50 | +9.4 | 37.5 | 58.5 | 40.4 | 21.5 | 41.5 | 48.3 |
CARAFE | R50 | +0.3 | 38.6 | 59.9 | 42.2 | 23.3 | 42.2 | 49.7 |
IndexNet | R50 | +8.4 | 37.6 | 58.4 | 40.9 | 21.5 | 41.3 | 49.2 |
A2U | R50 | +0.1 | 37.3 | 58.7 | 40.0 | 21.7 | 41.1 | 48.5 |
FADE | R50 | +0.2 | 38.5 | 59.6 | 41.8 | 23.1 | 42.2 | 49.3 |
SAPA-I | R50 | +0 | 37.7 | 59.2 | 40.6 | 22.2 | 41.2 | 48.4 |
SAPA-B | R50 | +0.1 | 37.8 | 59.2 | 40.6 | 22.4 | 41.4 | 49.1 |
SAPA-D | R50 | +0.6 | 39.2 | 60.8 | 42.7 | 23.5 | 42.9 | 50.3 |
Nearest | R101 | 65.8 | 39.4 | 60.1 | 43.1 | 22.4 | 43.7 | 51.1 |
SAPA-D | R101 | +0.6 | 40.6 | 61.8 | 44.1 | 24.3 | 45.0 | 52.8 |