I was wondering if there is any library that is compatible to train transformers in a incremental manner. Just like River and Creme are there to support many Supervised and Unsupervised learning task in a online manner; is there any library for transformers too. Any advice will be highly appreciated.
Hi,
That’s exactly the purpose of the online language modeling (OLM) project 
The repo can be found here: GitHub - huggingface/olm-training: Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.
1 Like