Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in distill, how print teacher model accuracy of data #360

Closed
Manningchan opened this issue Nov 24, 2022 · 6 comments
Closed

in distill, how print teacher model accuracy of data #360

Manningchan opened this issue Nov 24, 2022 · 6 comments
Assignees
Labels

Comments

@Manningchan
Copy link

No description provided.

@wilxy wilxy self-assigned this Nov 25, 2022
@wilxy wilxy added the usage label Nov 25, 2022
@wilxy
Copy link
Contributor

wilxy commented Nov 25, 2022

Thanks for your issue.
You can set the type of val_cfg to mmrazor.SingleTeacherDistillValLoop like this config.
Then you can get the accuracy of the teacher model during training like this log:

{"student.accuracy/top1": 29.884000778198242, "student.accuracy/top5": 55.50400161743164, "step": 1}
{"teacher.accuracy/top1": 73.62600708007812, "teacher.accuracy/top5": 91.59000396728516, "step": 1}

@Manningchan
Copy link
Author

Thanks for your issue. You can set the type of val_cfg to mmrazor.SingleTeacherDistillValLoop like this config. Then you can get the accuracy of the teacher model during training like this log:

{"student.accuracy/top1": 29.884000778198242, "student.accuracy/top5": 55.50400161743164, "step": 1}
{"teacher.accuracy/top1": 73.62600708007812, "teacher.accuracy/top5": 91.59000396728516, "step": 1}

thanks for your reply, but i used mmseg,if i add val_cfg,i still can not get the teacher accuracy,and if i add student_recorders and teacher_recorders, it will occurs TypeError: GeneralDistill: SingleTeacherDistiller: init() got an unexpected keyword argument 'student_recorders'. i can not find ‘ConfigurableDistiller‘,

@wilxy
Copy link
Contributor

wilxy commented Nov 25, 2022

Please provide your training config and the output of pip list | grep "mmcv\|mmrazor\|^torch".

@Manningchan
Copy link
Author

grep "mmcv|mmrazor|^torch
my download mmrazor is master and i see your provide is dev-1.x, can i modify some codes to print the teacher model accuracy in master branch? the config and output of mmcv,mmrazor,torch is below, the model is a mask semantic segmentation model defined by myself

config:
algorithm = dict(
type='GeneralDistill',
architecture=dict(
type='MMSegArchitecture',
model=student,
),
distiller=dict(
type='SingleTeacherDistiller',
teacher=teacher,
teacher_trainable=False,
#student_recorders=dict(
# conv_seg=dict(type='',source='decode_head.conv_seg')),
#teacher_recorders=dict(
# conv_seg=dict(type='',source='decode_head.conv_seg')),
components=[
dict(
student_module='decode_head.conv_seg1',
teacher_module='decode_head.conv_seg1',
losses=[
dict(
type='ChannelWiseDivergence_cls',
name='loss_cwd_logits_cls',
tau=1,
loss_weight=5,
)
]),
dict(
student_module='decode_head.conv_seg2',
teacher_module='decode_head.conv_seg2',
losses=[
dict(
type='ChannelWiseDivergence_mask',
name='loss_cwd_logits_mask',
tau=1,
loss_weight=5,
)
])
]),
)

environment

mmcv-full 1.6.0
mmrazor 0.3.1
torch 1.12.1+cu113
torchaudio 0.12.1+cu113
torchvision 0.13.1+cu113

@wilxy
Copy link
Contributor

wilxy commented Nov 25, 2022

Your MMRazor version is 0.3.1, which does not support the recorder mechanism and SingleTeacherDistillValLoop.
Please update the version to 1.0.0rc1 by using the dev-1.x branch.

@LKJacky LKJacky closed this as completed Dec 9, 2022
@wilxy
Copy link
Contributor

wilxy commented Dec 9, 2022

Please reopen it if there are any problems in the future.

humu789 pushed a commit to humu789/mmrazor that referenced this issue Feb 13, 2023
humu789 pushed a commit to humu789/mmrazor that referenced this issue Feb 13, 2023
* fix pose demo and windows build (open-mmlab#307)

* init

* Update nms_rotated.cpp

* add postprocessing_masks gpu version (open-mmlab#276)

* add postprocessing_masks gpu version

* default device cpu

* pre-commit fix

Co-authored-by: hadoop-basecv <[email protected]>

* fixed a bug causes text-recognizer to fail when (non-NULL) empty bboxes list is passed (open-mmlab#310)

* [Fix] include missing <type_traits> for formatter.h (open-mmlab#313)

* fix formatter

* relax GCC version requirement

* fix

* fix lint

* fix lint

* [Fix] MMEditing cannot save results when testing (open-mmlab#336)

* fix show

* lint

* remove redundant codes

* resolve comment

* type hint

* docs(build): fix typo (open-mmlab#352)

* docs(build): add missing build option

* docs(build): add onnx install

* style(doc): trim whitespace

* docs(build): revert install onnx

* docs(build): add ncnn LD_LIBRARY_PATH

* docs(build): fix path error

* fix openvino export tmp model, add binary flag (open-mmlab#353)

* init circleci (open-mmlab#348)

* fix wrong input mat type (open-mmlab#362)

* fix wrong input mat type

* fix lint

* fix(docs): remove redundant doc tree (open-mmlab#360)

* fix missing ncnn_DIR & InferenceEngine_DIR (open-mmlab#364)

* update doc

Co-authored-by: Chen Xin <[email protected]>
Co-authored-by: Shengxi Li <[email protected]>
Co-authored-by: hadoop-basecv <[email protected]>
Co-authored-by: lzhangzz <[email protected]>
Co-authored-by: Yifan Zhou <[email protected]>
Co-authored-by: tpoisonooo <[email protected]>
Co-authored-by: lvhan028 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants