Gpt-neo 27 and 13

Models
1

When I run on my CPU everything is fine, but when I run on my GPU, I get garbage. Is this the correct syntax?

generator = pipeline('text-generation', model='EleutherAI/gpt-neo-2.7B', device=1)

2

The devices are 0 indexed. So if you have 1 GPU you should use
generator = pipeline('text-generation', model='EleutherAI/gpt-neo-2.7B', device=0)
You can check how many devices are available with torch.cuda.device_count() Then check a specific device with torch.cuda.get_device_name[0]

1 Like
3

Thanks. I have three: a 1050ti and two k80s.