Different Trainers, when to use which?

[alvations]

There’s a few *Trainer objects available from transformers, trl and setfit.

Other than the standard answer of “it depends on the task and which library you want to use”, what is the best practice or general guidelines when choosing which *Trainer object to use to train/tune our models?

Together with the *Trainer object, sometimes we see suggestions to use *TrainingArguments or the vanilla TrainingArguments.

---

For reference, we have:

• fine-tuning llama,
  • https://github.com/philschmid/huggingface-llama-2-samples/blob/master/training/scripts/run_clm.py suggests vanilla Trainer
• few-shot learning,
  • SetFit: Efficient Few-Shot Learning Without Prompts suggests from setfit import SetFitTrainer
• machine translation
  • https://github.com/huggingface/transformers/blob/main/examples/pytorch/translation/run_translation.py suggests Seq2SeqTrainer
• reinforcement learning
  • Supervised Fine-tuning Trainer suggests from trl import SFTTrainer

[ai-nikolai]

@alvations is there any update on this?

@philschmid Are there any resources you could recommend? Or people we could tag?