Update feature_visualization()

[glenn-jocher]

Only plot for data with height, width > 1

@Zigars update to #3804

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Updated feature visualization function in utils/plots.py for clarity and functionality.

📊 Key Changes

• Changed function parameter from features to x to represent feature maps.
• Replaced module_idx with stage to describe the stage of the model rather than the specific module layer index.
• Added a check for feature map dimensions before proceeding with visualization, ensuring feature maps are at least 2D.
• Feature visualization save path now uses stage instead of module_idx.

🎯 Purpose & Impact

• The parameter name change from features to x aims to generalize the type of data visualized, making it more intuitive.
• Swapping module_idx for stage improves readability and aligns with common terminology, simplifying understanding of model structure.
• The dimension check ensures feature maps appropriate for visualization are processed, preventing errors with 1D data.
• Including stage in the file name allows for easier identification of features related to specific stages of the model, enhancing organization and interpretability.

👀 Users can expect a more intuitive interface for feature visualization, with safeguards for data compatibility, and clearer organization of output images.

[Zigars]

maybe the threshold need set 16?

[glenn-jocher]

@Zigars these are in units of stride. For the SPP layer you provided for example bus.jpg is of height=20 width=15.

[glenn-jocher]

@Zigars PR is merged. Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐

[Zigars]

but if set 1 ,it also will generate this feature map:

I suggest you can coverage the same name .png, so that the Model summary feature map can be coverage by true image.

[glenn-jocher]

@Zigars model summary is no longer generating feature maps after this PR. Please git pull master and try again.

[Zigars]

@glenn-jocher yes if run at all zeros inputs it will not generate image, but if run at reduced sizes, it will still generate like this:

I just download your latest master branches and test for it, it still will generate this feature map.

[glenn-jocher]

@Zigars yes of course, we want small images to output results also, otherwise users will raise bug reports on small images.

[glenn-jocher]

@Zigars basically anything that is not a FLOPs forward pass or a forward pass for clearing timing information we want to run. It's not up to us how large or small user images are, we want all user images to output results.

[Zigars]

@glenn-jocher Is that any way can coverage the useless feature map? users maybe confuse about why generate 2 folders in one detection, and I don's suggest users visualization features in train.py, It will cause generate too many images.

[glenn-jocher]

@Zigars I think everything is operating correctly. When I run detect.py it saves one png for bus.jpg and one for detect.jpg:

detect: weights=yolov5s.pt, source=data/images, imgsz=640, conf_thres=0.25, iou_thres=0.45, max_det=1000, device=, view_img=False, save_txt=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False
YOLOv5 🚀 v5.0-250-g02719dd torch 1.9.0 CPU

Fusing layers... 
Model Summary: 224 layers, 7266973 parameters, 0 gradients
image 1/2 /Users/glennjocher/PycharmProjects/yolov5/data/images/bus.jpg: 
Saving runs/features/exp/stage_8_SPP_features.png...
640x480 4 persons, 1 bus, 1 fire hydrant, Done. (2.013s)
image 2/2 /Users/glennjocher/PycharmProjects/yolov5/data/images/zidane.jpg: 
Saving runs/features/exp2/stage_8_SPP_features.png...
384x640 2 persons, 2 ties, Done. (1.957s)
Results saved to runs/detect/exp
Done. (4.034s)

[Zigars]

@glenn-jocher Why I generate 3 folders...

[Zigars]

@glenn-jocher I know the reason about it!
I use gpu to detect the image, it will cause this bug!
when I change cuda:0 to cpu, I get true folder in run/feature.

[Zigars]

@glenn-jocher If I use cuda:0 to detect, it will get these log.

D:\Users\ZHT\anaconda3\envs\pytorch\python.exe D:/VSCodeProject/yolov5-master/detect.py
detect: weights=weights/yolov5s.pt, source=data/images, imgsz=640, conf_thres=0.25, iou_thres=0.45, max_det=1000, device=, view_img=False, save_txt=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False
YOLOv5 2021-6-28 torch 1.7.1 CUDA:0 (GeForce GTX 1050 Ti, 4096.0MB)

Fusing layers...
Model Summary: 224 layers, 7266973 parameters, 0 gradients
Saving runs\features\exp6\stage_8_SPP_features.png...
image 1/2 D:\VSCodeProject\yolov5-master\data\images\bus.jpg: Saving runs\features\exp7\stage_8_SPP_features.png...
640x480 4 persons, 1 bus, 1 fire hydrant, Done. (1.490s)
image 2/2 D:\VSCodeProject\yolov5-master\data\images\zidane.jpg: Saving runs\features\exp8\stage_8_SPP_features.png...
384x640 2 persons, 2 ties, Done. (1.525s)
Results saved to runs\detect\exp4
Done. (3.095s)

[Zigars]

@glenn-jocher that's the reason for why I can get a more folder in detect.py.

[glenn-jocher]

@Zigars ah I think this line is the difference:

yolov5/detect.py

Lines 89 to 90 in 02719dd

	if device.type != 'cpu':
	model(torch.zeros(1, 3, imgsz, imgsz).to(device).type_as(next(model.parameters()))) # run once

This runs inference once to initialize the model since the first pass will always be slow.

[glenn-jocher]

@Zigars not sure what the best fix would be, as these images are the same size as the inference images to get the GPU ready for fast inference.

[Zigars]

@glenn-jocher I think it's not here, if you print the size, you will find this infer model's input are from 16 to 1 , in SPP: w=h=1, in first C3 :h=w=8,maybe in other way have one infer.

D:\Users\ZHT\anaconda3\envs\pytorch\python.exe D:/VSCodeProject/yolov5-master/detect.py
detect: weights=weights/yolov5s.pt, source=data/images, imgsz=640, conf_thres=0.25, iou_thres=0.45, max_det=1000, device=, view_img=False, save_txt=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False
YOLOv5 2021-6-28 torch 1.7.1 CUDA:0 (GeForce GTX 1050 Ti, 4096.0MB)

Fusing layers...
8 8
Saving runs\features\exp18\stage_2_C3_features.png...
Model Summary: 224 layers, 7266973 parameters, 0 gradients
160 160
Saving runs\features\exp19\stage_2_C3_features.png...
image 1/2 D:\VSCodeProject\yolov5-master\data\images\bus.jpg: 160 120
Saving runs\features\exp20\stage_2_C3_features.png...
640x480 4 persons, 1 bus, 1 fire hydrant, Done. (1.761s)
image 2/2 D:\VSCodeProject\yolov5-master\data\images\zidane.jpg: 96 160
Saving runs\features\exp21\stage_2_C3_features.png...
384x640 2 persons, 2 ties, Done. (1.984s)
Results saved to runs\detect\exp8
Done. (3.832s)

[Zigars]

@glenn-jocher I'm sure not here, I try to write off these code, but it still have one more folder.