Huggingface transformer sequence classification

🤗Transformers
1

I am fine tuning a transformer and I would like to use GPUs

This page shows how we to do the same

trainer = Trainer(
    model=model,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=train_df_tuning_dataset_tokenized,
    eval_dataset=val_dataset_tokenized
)

Do I need to modify above part like below? I am adding .to(device) to ensure data is on GPU

device = 'cuda' if torch.cuda.is_available() else 'cpu'
#device = 'cpu'
device

trainer = Trainer(
    model=model,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=train_df_tuning_dataset_tokenized.to(device),
    eval_dataset=val_dataset_tokenized.to(device)
)
2

I am not 100% sure, but I think you need to send the dataset and the model to device before you call the Trainer.
Something like this might work,

import torch
device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

model.to(device)
train_df_tuning_dataset_tokenized.to(device)
val_dataset_tokenized.to(device)


trainer = Trainer(
    model=model,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=train_df_tuning_dataset_tokenized,
    eval_dataset=val_dataset_tokenized
)
3

lets hope we get a confirmation. Thanks for pointing out that model needs to be on gpu too

4

Hi,

You shouldn’t actually have any device placements, the Trainer handles this for you. The Trainer has an attribute called place_model_on_device which is set to True by default.

The Trainer will also handle device placement of your data.