I am using this piece of code to fine tune a transformer model for text classification in a Jupyter notebook:
metric = evaluate.load("accuracy")
def compute_metrics(eval_pred):
logits, labels = eval_pred
predictions = np.argmax(logits, axis=-1)
return metric.compute(predictions=predictions, references=labels)
training_args = TrainingArguments(output_dir = "output_fine_tuning",
evaluation_strategy = "epoch",
logging_strategy="epoch",
num_train_epochs = 3)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_datasets['train'],
eval_dataset=tokenized_datasets['test'],
compute_metrics=compute_metrics,
)
train = trainer.train()
I only see printing out Training Loss, Validation Loss, Accuracy, where the last one is validation accuracy.
I tried to use techniques from here: python - How to get the accuracy per epoch or step for the huggingface.transformers Trainer? - Stack Overflow but nothing really worked.