Fine-tuning a text model for another language

Hi guys! I have a very common question (unfortunately, I couldn’t find an answer in the Course). There is a model in the Hub (onlplab/alephbert-base) for Hebrew. I would like to fine-tune it so that it will work for summarization in Yiddish as well (the alphabet of both languages is practically the same). I am going to use for the beginning a Database (nthngdy/oscar-mini) consisting of a set of sentences without any labels.

Does it make sense to fine-tune a model for another language, or it would be better to create an Yiddish model from scratch? (Unfortunately, there is no Yiddish models for summarization in the Hub yet).
Thank you!