According to the following question, the trainer will handle multiple GPU work. What is the method it uses? DataParallel (DP) or TensorParallel (TP) or PipelineParallel (PP) or DPP, what? [image] Training using multiple GPUs Beginners I would like to train some models to…

Which method is use HF Trainer with multiple GPU?

Indramal December 3, 2022, 1:56am 3

Thank you for your information. Old Trainer documents have to configure that. But new document doese not mention it. Old Doc - Trainer — transformers 4.7.0 documentation

Topic		Replies	Views
Training using multiple GPUs Beginners	20	20107	February 25, 2024
Multi gpu training 🤗Transformers	3	6021	April 24, 2022
How to run single-node, multi-GPU training with HF Trainer? 🤗Transformers	5	15225	October 16, 2024
Using 3 GPUs for training with Trainer() of transformers 🤗Transformers	2	2312	October 18, 2023
Trainer API for Model Parallelism on Multiple GPUs 🤗Transformers	5	4217	September 10, 2024