Which method is use HF Trainer with multiple GPU?

[Indramal]

According to the following question, the trainer will handle multiple GPU work. What is the method it uses?

DataParallel (DP) or TensorParallel (TP) or PipelineParallel (PP) or DPP, what?

Training using multiple GPUs

I would like to train some models to multiple GPUs. Let suppose that I use model from HF library, but I am using my own trainers,dataloader,collators etc. Where I should focus to implement multiple GPU training? I need to make changes only in the Trainer class? If yes, can you give me a brief description? Thank you in avance.

[AndreaSottana]

Based on this line of code it looks like it is using nn.DataParallel, however I haven’t fully looked at every line of the Trainer class so they may also be using other methods in other points.

[Indramal]

Thank you for your information. Old Trainer documents have to configure that. But new document doese not mention it. Old Doc - Trainer — transformers 4.7.0 documentation