Hello folks, i wanna give it a try to use my own dataset to build pre-trained model for longformer, because current pre-trained model only available in English while my task targeting for non-english.
Any reference on how to train longformer model from scratch (not fine-tuning)? Specifically using Tensorflow since i plan to use it for training model for the downstream tasks later.
Thank you in advance!