Finetuning GPT2 using Multiple GPU and Trainer

@valhalla and this is why HF is awesome! Thanks for the response.

Is it possible to see 1) If Hf is using cpu or gpu and 2) which gpus HF is using? Just a bit of information I’d like to add to my logs while training.