GroundingDino Is not utilising GPU

I am trying to run the code from GroundingDino(swinT) on L40 GPU on runpod.
THe model acquires GPU memory, but during prediction the GPU utilisation is between 0 to 1 percent.

The FPS is very low.

Has anyone encountered the same issue?


Could you share a code snippet to reproduce your issue?

Hi @nielsr !

For comparison, cv-benchmark/logs at main · tariksetia/cv-benchmark · GitHub

The fps on v100 with batch size of 1 is 3 fps
on L40, for the same data, it created less than 3fps… It was taking too much time to process even a single video, I waited for 10 minutes…

Also could you please guide me on how can I use L40 efficiently for running Dino?