GPU Memory usages varies with the size of the dataset

superlyc · June 9, 2021, 11:51pm

Hi,

Does the GPU Memory usages vary with the size of the dataset? I am experimenting with the code from here.

gist.github.com

https://gist.github.com/jiahao87/50cec29725824da7ff6dd9314b53c4b3

pegasus_fine_tune.py

"""Script for fine-tuning Pegasus
Example usage:
  # use XSum dataset as example, with first 1000 docs as training data
  from datasets import load_dataset
  dataset = load_dataset("xsum")
  train_texts, train_labels = dataset['train']['document'][:1000], dataset['train']['summary'][:1000]
  
  # use Pegasus Large model as base for fine-tuning
  model_name = 'google/pegasus-large'
  train_dataset, _, _, tokenizer = prepare_data(model_name, train_texts, train_labels)

This file has been truncated. show original

I observed the GPU memory varies (not proportional to) the size of the dataset. Trying to understand why? Anyone would help explain?

superlyc · June 10, 2021, 7:55pm

The author of the code help me figure it out. It was because the Dataset class implemented in the code loading the encoding in __int__ function

thak123 · January 14, 2022, 11:46am

Yes I have similar doubt. any suggestions?

Topic		Replies	Views
Loading a large dataset occupies ~2GB on each GPU 🤗Datasets	0	97	April 24, 2024
Low RAM Usage & high GPU usage, Datasets not helping 🤗Datasets	3	1175	January 13, 2023
Pegasus finetuning, should we always start with pegasus-large? Beginners	5	1606	May 3, 2024
Why does all my gpu memory get used with a small model? Beginners	5	1989	March 13, 2022
RAG batch size on GPU Beginners	0	625	March 2, 2021

GPU Memory usages varies with the size of the dataset

Related Topics