I am trying to define the training by the following code snippet:
model = T5ForConditionalGeneration.from_pretrained('t5-small')
# ! pip install -U 'accelerate==0.21.0'
# ! pip install transformers[pytorch]
training_args = TrainingArguments(
output_dir='/content/drive/MyDrive/Results', # output directory
num_train_epochs=3, # number of training epochs
per_device_train_batch_size=4, # batch size for training
per_device_eval_batch_size=4, # batch size for evaluation
warmup_steps=500, # number of warmup steps for learning rate scheduler
weight_decay=0.01, # strength of weight decay
logging_dir='./logs', # directory for storing logs
logging_steps=10,
evaluation_strategy="epoch", # evaluate at the end of each epoch
)
trainer = Trainer(
model=model,
args=training_args, # training arguments, defined above
train_dataset=train_dataset, # training dataset
eval_dataset=val_dataset, # evaluation dataset
)
But I am getting this error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-10-98d5893e43cf> in <cell line: 13>()
11 )
12
---> 13 trainer = Trainer(
14 model=model, # the instantiated Transformers model to be trained
15 args=training_args, # training arguments, defined above
1 frames
/usr/local/lib/python3.10/dist-packages/transformers/trainer.py in create_accelerator_and_postprocess(self)
4533
4534 # create accelerator object
-> 4535 self.accelerator = Accelerator(**args)
4536 # some Trainer classes need to use `gather` instead of `gather_for_metrics`, thus we store a flag
4537 self.gather_function = self.accelerator.gather_for_metrics
TypeError: Accelerator.__init__() got an unexpected keyword argument 'use_seedable_sampler'