Hi all!
I just wanted to know if is there any shortcut for changing a causal language model language (in my case from English to Persian) - like fine-tuning a existing one - or I should train one from scratch?
You can start from a pre-trained English one and fine-tune it on another language. The effectiveness of this has been shown (among others) in the paper As Good as New. How to Successfully Recycle English GPT-2 to Make Models for Other Languages.
cc @wietsedv, who is the main author of that paper.
1 Like