Do automatically generated attention masks ignore padding?

Yes, you need to pass the attention mask returned by the tokenizer. Most models don’t know the padding token ID, so they can’t generate an attention mask that ignores it.

2 Likes