Regarding input_ids and labels while grouping texts

🤗Optimum
1

Hi @regisss

I see input_ids and labels to be of same tensors here

Is it a mistake? Or internally in GaudiTrainer the shifting of labels happens?

2

Hi @palashai, it happens in the forward of the model. For instance for Llama: optimum-habana/optimum/habana/transformers/models/llama/modeling_llama.py at 8c8ec55d8030f1df05288407b2e687a7664d7395 · huggingface/optimum-habana · GitHub

3

Got it. Thanks.

4

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.