GPT2 training examples

When fine-tuning GPT2, should i pass my whole sample at once to the model at each training step?
Wouldn’t this be interpreted as:

take my whole sample and then predict the next word?

To train the model i’m just encoding each sample and then passing it to the model through batches.