Right choice of padding side for Mistral

🤗Tokenizers
1

Mistral with flash attention 2 and right padding · Issue #26877 · huggingface/transformers (github.com)

From the above discussion, I understand that -

  1. During model.generate the padding side should be left
  2. During model.forward, the padding side could either be left or right. Both are usually supported with HF.

My question though is, if model.generate is going to give “not so good” results using right padding, isn’t model forward going to affect w/ the same problem?