Pre-training LayoutLMv2

For the purpose of simulating few and multi-shot learning we would like to pre-train LayoutLMv2 in a fashion similar to using BertForPreTraining. Would anyone have any resources for someone who has potentially done something similar with LayoutLM or another HF transformer model?

2 Likes