CUDA out of memory for Longformer

Beginners
1

I have issue training the longformer on custom dataset, even on a small batch number, it says CUDA out of memory,

RuntimeError Traceback (most recent call last)
in ()
----> 1 trainer.train()

18 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in _pad(input, pad, mode, value)
3550 assert len(pad) // 2 <= input.dim(), ‘Padding length too large’
3551 if mode == ‘constant’:
-> 3552 return _VF.constant_pad_nd(input, pad, value)
3553 else:
3554 assert value == 0, ‘Padding mode “{}”" doesn’t take in value argument’.format(mode)

RuntimeError: CUDA out of memory. Tried to allocate 1.13 GiB (GPU 0; 15.90 GiB total capacity; 11.40 GiB already allocated; 659.81 MiB free; 14.39 GiB reserved in total by PyTorch)

2

Did you try smaller batch sizes? What is the size of single batch size in your RAM?

3

I tried 8 batch size, do not remember the single batch size. I use colab so it has its own limitations even the Pro version. any thoughts?

4

Your input sentences are being limited to a maximum size?

5

I truncated to lesser as well.

6

Could you provide a simple snippet to reproduce the OOM?

7

if you see my post above, its months ago, its not actively something I am working on, I didn’t get answer but I solved that time by having smaller batches and less input.
I have few more issues like batch processing, having predictions at the end with a score, do you mind discussing that?

1 Like