Why does all my gpu memory get used with a small model?

Beginners
1

I’m using colab pro to try out a few huggingface models, but even loading a 2gb model completely fills up the GPU memory (Tesla 16gb) and then as the data loads i frequently run out of memory. Can anyone explain why a small model will maximize gpu memory?

My dataset size is about 1gb between train and test if that makes a difference

1 Like
2

I can’t say for sure without seeing how you are running it. Can you troubleshoot and see whats running on your gpu?

3

Thanks! Do you know how I can see what processes are running in the terminal?

4

Can you supply your parameters on how you are training? ie batchsize, epochs, etc. I have run into memory issues with 1gb model, 2gb is actually pretty big when training.

To check gpu resources in googlecolab you can try something like this

#On the left side you can open Terminal ('>_' with black background)

#You can run commands from there even when some cell is running

#Write command to see GPU usage in real-time:

$ watch nvidia-smi
1 Like
5

Thanks! I will try that out

6

Are you using Tensorflow? By default it allocates all the available GPU memory. There are ways around that though.