Hi John,
Thank you for the quick response.
When i send the request to the inference i see that the CPU is being used but GPU is not getting picked up although i have included .to(“cuda”). I am using the NVIDIA 1xL4. Just a snapshot below. When request is sent CPU touches 44% but GPU is not used.
