Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update l1 config #405

Merged
merged 2 commits into from
Jan 3, 2023
Merged

update l1 config #405

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b84316d
add l1 config
Dec 22, 2022
f82abe5
update l1 config
Dec 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion configs/pruning/mmcls/l1-norm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,13 @@

L1-norm pruning is a classical filter pruning algorithm. It prunes filers(channels) according to the l1-norm of the weight of a conv layer.

We use ItePruneAlgorithm and L1MutableChannelUnit to implement l1-norm pruning. Please refer to xxxx for more configuration detail.
We use ItePruneAlgorithm and L1MutableChannelUnit to implement l1-norm pruning. Please refer to [Pruning User Guide](../../../../docs/en/user_guides/pruning_user_guide.md) for more configuration detail.

| Model | Top-1 | Gap | Flop(G) | Pruned | Parameters | Pruned | Config | Download |
| ----------------- | ----- | ----- | ------- | ------ | ---------- | ------ | ------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ResNet34 | 73.62 | - | 3.68 | - | 2.18 | - | [mmcls](https://github.com/open-mmlab/mmclassification/blob/1.x/configs/resnet/resnet34_8xb32_in1k.py) | [model](https://download.openmmlab.com/mmclassification/v0/resnet/resnet34_8xb32_in1k_20210831-f257d4e6.pth) \| [log](https://download.openmmlab.com/mmclassification/v0/resnet/resnet34_8xb32_in1k_20210831-f257d4e6.log.json) |
| ResNet34_Pruned_A | 73.61 | -0.01 | 3.10 | 15.8% | 2.01 | 7.8% | [config](./l1-norm_resnet34_8xb32_in1k_a.py) | [model](https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmrazor/v1/pruning/l1-norm/l1-norm_resnet34_8xb32_in1k_a.pth) \| [log](https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmrazor/v1/pruning/l1-norm/l1-norm_resnet34_8xb32_in1k_a.json) |
| ResNet34_Pruned_B | 73.20 | -0.42 | 2.79 | 24.2% | 1.95 | 10.6% | [config](./l1-norm_resnet34_8xb32_in1k_a.py) | [model](https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmrazor/v1/pruning/l1-norm/l1-norm_resnet34_8xb32_in1k_b.pth) \| [log](https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmrazor/v1/pruning/l1-norm/l1-norm_resnet34_8xb32_in1k_b.json) |
| ResNet34_Pruned_C | 73.89 | +0.27 | 3.40 | 7.6% | 2.02 | 7.3% | [config](./l1-norm_resnet34_8xb32_in1k_a.py) | [model](https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmrazor/v1/pruning/l1-norm/l1-norm_resnet34_8xb32_in1k_c.pth) \| [log](https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmrazor/v1/pruning/l1-norm/l1-norm_resnet34_8xb32_in1k_c.json) |

**Note:** There is a different implementation from the original paper. We pruned the layers related to the shortcut with a shared pruning decision, while the original paper pruned them separately in *Pruned C*. This may be why our *Pruned C* outperforms *Prune A* and *Prune B*, while *Pruned C* is worst in the original paper.
27 changes: 16 additions & 11 deletions ...ls/l1-norm/l1-norm_resnet34_8xb32_in1k.py → .../l1-norm/l1-norm_resnet34_8xb32_in1k_a.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,31 +1,36 @@
_base_ = ['mmcls::resnet/resnet34_8xb32_in1k.py']

un_prune = 1.0
stage_ratio_1 = 0.7
stage_ratio_2 = 0.7
stage_ratio_3 = 0.7
stage_ratio_4 = 1.0
stage_ratio_4 = un_prune

# the config template of target_pruning_ratio can be got by
# python ./tools/pruning/get_channel_units.py {config_file} --choice
# python ./tools/get_channel_units.py {config_file} --choice
target_pruning_ratio = {
'backbone.conv1_(0, 64)_64': stage_ratio_1,
# stage 1
'backbone.conv1_(0, 64)_64': un_prune, # short cut layers
'backbone.layer1.0.conv1_(0, 64)_64': stage_ratio_1,
'backbone.layer1.1.conv1_(0, 64)_64': stage_ratio_1,
'backbone.layer1.2.conv1_(0, 64)_64': stage_ratio_1,
'backbone.layer2.0.conv1_(0, 128)_128': stage_ratio_2,
'backbone.layer2.0.conv2_(0, 128)_128': stage_ratio_2,
'backbone.layer1.2.conv1_(0, 64)_64': un_prune,
# stage 2
'backbone.layer2.0.conv1_(0, 128)_128': un_prune,
'backbone.layer2.0.conv2_(0, 128)_128': un_prune, # short cut layers
'backbone.layer2.1.conv1_(0, 128)_128': stage_ratio_2,
'backbone.layer2.2.conv1_(0, 128)_128': stage_ratio_2,
'backbone.layer2.3.conv1_(0, 128)_128': stage_ratio_2,
'backbone.layer3.0.conv1_(0, 256)_256': stage_ratio_3,
'backbone.layer3.0.conv2_(0, 256)_256': stage_ratio_3,
'backbone.layer2.3.conv1_(0, 128)_128': un_prune,
# stage 3
'backbone.layer3.0.conv1_(0, 256)_256': un_prune,
'backbone.layer3.0.conv2_(0, 256)_256': un_prune, # short cut layers
'backbone.layer3.1.conv1_(0, 256)_256': stage_ratio_3,
'backbone.layer3.2.conv1_(0, 256)_256': stage_ratio_3,
'backbone.layer3.3.conv1_(0, 256)_256': stage_ratio_3,
'backbone.layer3.4.conv1_(0, 256)_256': stage_ratio_3,
'backbone.layer3.5.conv1_(0, 256)_256': stage_ratio_3,
'backbone.layer3.5.conv1_(0, 256)_256': un_prune,
# stage 4
'backbone.layer4.0.conv1_(0, 512)_512': stage_ratio_4,
'backbone.layer4.0.conv2_(0, 512)_512': stage_ratio_4,
'backbone.layer4.0.conv2_(0, 512)_512': un_prune, # short cut layers
'backbone.layer4.1.conv1_(0, 512)_512': stage_ratio_4,
'backbone.layer4.2.conv1_(0, 512)_512': stage_ratio_4
}
Expand Down
38 changes: 38 additions & 0 deletions configs/pruning/mmcls/l1-norm/l1-norm_resnet34_8xb32_in1k_b.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
_base_ = ['./l1-norm_resnet34_8xb32_in1k_a.py']

un_prune = 1.0
stage_ratio_1 = 0.5
stage_ratio_2 = 0.4
stage_ratio_3 = 0.6
stage_ratio_4 = un_prune

# the config template of target_pruning_ratio can be got by
# python ./tools/get_channel_units.py {config_file} --choice
target_pruning_ratio = {
# stage 1
'backbone.conv1_(0, 64)_64': un_prune, # short cut layers
'backbone.layer1.0.conv1_(0, 64)_64': stage_ratio_1,
'backbone.layer1.1.conv1_(0, 64)_64': stage_ratio_1,
'backbone.layer1.2.conv1_(0, 64)_64': un_prune,
# stage 2
'backbone.layer2.0.conv1_(0, 128)_128': un_prune,
'backbone.layer2.0.conv2_(0, 128)_128': un_prune, # short cut layers
'backbone.layer2.1.conv1_(0, 128)_128': stage_ratio_2,
'backbone.layer2.2.conv1_(0, 128)_128': stage_ratio_2,
'backbone.layer2.3.conv1_(0, 128)_128': un_prune,
# stage 3
'backbone.layer3.0.conv1_(0, 256)_256': un_prune,
'backbone.layer3.0.conv2_(0, 256)_256': un_prune, # short cut layers
'backbone.layer3.1.conv1_(0, 256)_256': stage_ratio_3,
'backbone.layer3.2.conv1_(0, 256)_256': stage_ratio_3,
'backbone.layer3.3.conv1_(0, 256)_256': stage_ratio_3,
'backbone.layer3.4.conv1_(0, 256)_256': stage_ratio_3,
'backbone.layer3.5.conv1_(0, 256)_256': un_prune,
# stage 4
'backbone.layer4.0.conv1_(0, 512)_512': stage_ratio_4,
'backbone.layer4.0.conv2_(0, 512)_512': un_prune, # short cut layers
'backbone.layer4.1.conv1_(0, 512)_512': stage_ratio_4,
'backbone.layer4.2.conv1_(0, 512)_512': stage_ratio_4
}

model = dict(target_pruning_ratio=target_pruning_ratio, )
34 changes: 34 additions & 0 deletions configs/pruning/mmcls/l1-norm/l1-norm_resnet34_8xb32_in1k_c.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
_base_ = ['./l1-norm_resnet34_8xb32_in1k_a.py']

un_prune = 1.0

# the config template of target_pruning_ratio can be got by
# python ./tools/get_channel_units.py {config_file} --choice
target_pruning_ratio = {
# stage 1
'backbone.conv1_(0, 64)_64': un_prune, # short cut layers
'backbone.layer1.0.conv1_(0, 64)_64': un_prune,
'backbone.layer1.1.conv1_(0, 64)_64': un_prune,
'backbone.layer1.2.conv1_(0, 64)_64': un_prune,
# stage 2
'backbone.layer2.0.conv1_(0, 128)_128': un_prune,
'backbone.layer2.0.conv2_(0, 128)_128': un_prune, # short cut layers
'backbone.layer2.1.conv1_(0, 128)_128': un_prune,
'backbone.layer2.2.conv1_(0, 128)_128': un_prune,
'backbone.layer2.3.conv1_(0, 128)_128': un_prune,
# stage 3
'backbone.layer3.0.conv1_(0, 256)_256': un_prune,
'backbone.layer3.0.conv2_(0, 256)_256': 0.8, # short cut layers
'backbone.layer3.1.conv1_(0, 256)_256': un_prune,
'backbone.layer3.2.conv1_(0, 256)_256': un_prune,
'backbone.layer3.3.conv1_(0, 256)_256': un_prune,
'backbone.layer3.4.conv1_(0, 256)_256': un_prune,
'backbone.layer3.5.conv1_(0, 256)_256': un_prune,
# stage 4
'backbone.layer4.0.conv1_(0, 512)_512': un_prune,
'backbone.layer4.0.conv2_(0, 512)_512': un_prune, # short cut layers
'backbone.layer4.1.conv1_(0, 512)_512': un_prune,
'backbone.layer4.2.conv1_(0, 512)_512': un_prune
}

model = dict(target_pruning_ratio=target_pruning_ratio, )