I have a 1GB raw text dataset in a niche domain. I want to train an ELECTRA model but I couldn’t find any tutorial/examples to do so. Can anyone help me? I tried using the simpletransformers package, but it has memory issues at the moment and after a few epochs my colab session crashes.

How pretrain ELECTRA on custom dataset?

valhalla September 6, 2020, 10:53am 6

1 Like

Topic		Replies	Views
ELECTRA training reimplementation and discussion Research	14	6662	September 17, 2023
Electra NER on Conll03-english Beginners	6	918	September 9, 2020
How could different model be trained same way? Beginners	0	206	July 26, 2021
How hard is it to finetune an ELECTRA model for multihead regression & classification? 🤗Transformers	0	307	October 15, 2021
PreTrain ELECTRA from scratch in Portuguese Flax/JAX Projects	2	1325	November 8, 2021