Currently, there is no ELECTRA or ELECTRA Large model that was trained from scratch for Portuguese on the hub: Hugging Face – The AI community building the future.. For this project, the goal is to create a ELECTRA model for just the Portuguese language.
A randomly initialized ELECTRA model
A masked language modeling script for Flax is available here. It can be used pretty much without any required code changes.
The desired project output is a strong ELECTRA model in Portuguese.