Fine-tuning an NLLB model for a new language

Unfortunately no. I tried to delete and recreate the token IDs as suggested, which initially led to a different ordering of the language token IDs, but even after fixing it I basically ended up with the same situation as before. I also dug around in the tokenizer a bit, but I don’t really understand the root cause of the issue.

1 Like