GroundingDino Is not utilising GPU

tariksetia · June 11, 2024, 6:51am

I am trying to run the code from GroundingDino(swinT) on L40 GPU on runpod.
THe model acquires GPU memory, but during prediction the GPU utilisation is between 0 to 1 percent.

The FPS is very low.

Has anyone encountered the same issue?

nielsr · June 11, 2024, 6:54am

Hi,

Could you share a code snippet to reproduce your issue?

tariksetia · June 11, 2024, 12:05pm

Hi @nielsr !

github.com

tariksetia/cv-benchmark/blob/main/02-gorunding-dino-hf.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install -q huggingface\n",
    "!pip install -q transformers\n",
    "!pip install -q pillow\n",
    "!pip install -q loguru\n",
    "!pip install -q pydantic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,

This file has been truncated. show original

For comparison, cv-benchmark/logs at main · tariksetia/cv-benchmark · GitHub

The fps on v100 with batch size of 1 is 3 fps
on L40, for the same data, it created less than 3fps… It was taking too much time to process even a single video, I waited for 10 minutes…

tariksetia · June 11, 2024, 12:06pm

Also could you please guide me on how can I use L40 efficiently for running Dino?

Topic		Replies	Views
Baffling performance issue on most NVidia GPUs with simple transformers + pytorch code Intermediate	5	3671	April 9, 2024
Containerizing Huggingface Transformers for GPU inference with Docker and FastAPI 🤗Transformers	0	2695	October 5, 2021
Running multiple instances on GPU Intermediate	0	132	June 1, 2024
Why does all my gpu memory get used with a small model? Beginners	5	1928	March 13, 2022
How to All Utilize all GPU's when device="balanced_low_0" in GPU setting 🤗Transformers	1	98	March 28, 2024

GroundingDino Is not utilising GPU

Related Topics