Is there a list of MLM corruption strategies?

🤗Transformers
1

Ideally, looking for a list of such corruption strategies that include an ELI5 description, pictures, and sample implementation(s).

Thanks -wg

2

Hi,

Yes, the T5 paper explored different corruption strategies. See table 3 on page 21.

1 Like