Accelerate video inference

[mucunwuxian]

@glenn-jocher
I hope this PR finds you well.
I adjust code of reading video, using 'cap.grab()' and 'cap. retrieve'.
It's a small fix.

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Improved video frame retrieval in data loading process for YOLOv5.

📊 Key Changes

• Replaced a single cap.read() call with cap.grab() called in a loop, followed by cap.retrieve() for video frames.
• The number of times cap.grab() is called is determined by self.vid_stride, ensuring that frames are skipped according to the stride specified.

🎯 Purpose & Impact

• 🎯 The update optimizes video processing by skipping over frames more efficiently when a stride is used.
• ✨ This can lead to better memory management and potentially faster data loading, especially when processing videos with high frame rates.
• 🚀 Users can expect more consistent performance when training YOLOv5 on video datasets with frame skipping.

[github-actions]

👋 Hello @mucunwuxian, thank you for submitting a YOLOv5 🚀 PR! To allow your work to be integrated as seamlessly as possible, we advise you to:

• ✅ Verify your PR is up-to-date with ultralytics/yolov5 master branch. If your PR is behind you can update your code by clicking the 'Update branch' button or by running git pull and git merge master locally.
• ✅ Verify all YOLOv5 Continuous Integration (CI) checks are passing.
• ✅ Reduce changes to the absolute minimum required for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

[glenn-jocher]

@mucunwuxian thanks for the PR! Do you have any before and after results/profiling?

[mucunwuxian]

@glenn-jocher
Thank you!
I don't have any profile, but it feels like about 10 times faster.
I think the reason why it is faster is the validity of the cursor operation.
If you want result/profiling, I can ready for it.

[glenn-jocher]

@mucunwuxian oh that's strange. Are you sure it's not just faster because it's skipping more frames?

[glenn-jocher]

@mucunwuxian I tested and it is much faster! 4.1s vs 6.6s on M2 CPU, probably even faster on GPU.

[glenn-jocher]

@mucunwuxian I thought .grab() actually loaded the data though and was much slower than .retrieve(). Shouldn't we retreive multiple frames before grabbing the one we want for the fastest speed?

EDIT: this is backwards.

[mucunwuxian]

@glenn-jocher
My situation, I execute detect.py with --vid_stride 4 both before and after.
In that comparison, I felt about 10 times faster.
Perhaps, self.cap.set(cv2.CAP_PROP_POS_FRAMES, self.vid_stride * (self.frame + 1)) # read at vid_stride is slow.
Especially if the number of frames in the video is large.

Wait a minute.
Now let's measure.

[glenn-jocher]

@mucunwuxian I think we want the same behavior as the streamloader here, where it grabs every frame and only retreives frames when n % vid_stride == 0:

yolov5/utils/dataloaders.py

Lines 336 to 340 in 868c0e9

	while cap.isOpened() and n < f:
	n += 1
	cap.grab() # .read() = .grab() followed by .retrieve()
	if n % self.vid_stride == 0:
	success, im = cap.retrieve()

[glenn-jocher]

@mucunwuxian ok, I've aligned the behavior with LoadStreams. Same speed, slightly shorter code.

[mucunwuxian]

@glenn-jocher
I agree with you!

[glenn-jocher]

@mucunwuxian PR is merged. Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐

[mucunwuxian]

@glenn-jocher
OK!

By the way, I did same way in the beginning.
e6d9ced

So it's essentially correct, but different from the output of the original code.

But if you the author are okay then I think it's okay. :-D

[glenn-jocher]

@mucunwuxian oh got it!

Yes this way may skip the first few frames rather than always using the first frame, but this is the behavior for LoadStreams so now they are both aligned.

[mucunwuxian]

@glenn-jocher
Great!
That’s very helpful, I can learn a lot from you!

[glenn-jocher]

@mucunwuxian i appreciate your kind words! The YOLO community and the Ultralytics team have been invaluable in this endeavor. Feel free to reach out if you have any further questions or contributions.