Zero Shot Classification using multiGPU

🤗Transformers
1

Can we not use the Pipeline abstract class with multi GPU? I am trying do inference using pipeline(“zero-shot-classification”). It only utilises single GPU & throws CUDA out of memory memory error on 16GB 4x V100. How to go about using multiGPU(other than device=0)?

2

I also have the same question. Is there a way to use pipeline for inference with a “zero-shot-classification” model? I cannot use more than one GPU at a time with pipeline for inference purposes.

This conversation on GitHub is relevant but seems to point to the conclusion that it is not possible. However, if I am mistaken, please let me know.

Toward the end of the conversation, it says that the accelerate module will help, but when I run the following:

from transformers import pipeline 

pipe = pipeline(
                task="zero-shot-learning", 
                model="MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli", 
                device_map="auto"
)

The error that I get is:

('Keyword argument not understood:', 'device_map')

I am using the following GPU compute set up:

GPU Info: 
  GPU 0: NVIDIA A10G
  GPU 1: NVIDIA A10G
  GPU 2: NVIDIA A10G
  GPU 3: NVIDIA A10G