How's the speed on V100 with 720p videos?

[MonolithFoundation]

How's the speed on V100 with 720p videos?

[Gy920]

Sorry, I haven't tested this on V100.

[MonolithFoundation]

What's the estimated speed on v 108

…

---- Replied Message ---- | From | Guiyong ***@***.***> | | Date | 09/03/2024 22:13 | | To | ***@***.***> | | Cc | ***@***.***>***@***.***> | | Subject | Re: [Gy920/segment-anything-2-real-time] How's the speed on V100 with 720p videos? (Issue #7) | Sorry, I haven't tested this on V100. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[heyoeyo]

How's the speed on V100 with 720p videos?

I'm not sure about the V100 either, but it should roughly be within a factor of 2 slower than the speeds reported on the original repo. So somewhere in the range of 10-25 fps, depending on which model you're using.

The input video resolution (e.g. 720p) doesn't matter much, since the model will resize any input to 1024px (by default). You can get a ~4x speedup if you reduce that size to 512 though!

[maxmelichov]

@heyoeyo I'm using 4060 and getting about 8 fps on live video
the drop is about 5 times
did you change something in the code to get a drop of just a factor of 2?

[heyoeyo]

@maxmelichov The speeds reported on the original repo seem to be based off an A100 GPU. The factor of 2 is (roughly) the difference between the A100 and V100 (based on the memory bandwidth of the two cards + other benchmarks).

The 4060 has a memory bandwidth that is ~7 times lower than the A100, so if you're getting speeds 5x slower than the reported speeds, that's probably normal.

[maxmelichov]

@heyoeyo do you know about TensorRT version of sam2?

[heyoeyo]

I haven't tried it myself, but there was some discussion of using TensorRT on the SAM2 repo (issue #284). At the very least, there are onnx versions of the model available, which seems to be the first step towards making a TensorRT version.