I’m working on a KenLM model (I’m training one) and I would like to upload it into its own repository on HF. But I cannot find how to code the accompanying Python class for it.
The closest thing I could find is this repo:
But it seems to me even they couldn’t find a way to create the repo. If you look at the example code they have in their repo’s card:
from model import KenlmModel # Load model trained on English wikipedia model = KenlmModel.from_pretrained("wikipedia", "en") # Get perplexity model.get_perplexity("I am very perplexed") # 341.3 (low perplexity, since sentence style is formal and with no grammar mistakes) model.get_perplexity("im hella trippin") # 46793.5 (high perplexity, since the sentence is colloquial and contains grammar mistakes)
They are loading the
KenlmModel class from the
model module. But this module only exists in their repo, meaning that you have to clone the repo before you can use their model (no support for
pip install). Of course, this is an awful experience.
Is there a way to create a repo for a custom model like KenLM on HF?