I’ve trained a mt5 model with accelerate. But it only save the model.config file and the pytorch.model file?
How can I generate output from my model without the tokenizer files?
When I try to load the model it gives me the following error.
OSError: Can’t load tokenizer for ‘saved’. Make sure that:
‘saved’ is a correct model identifier listed on ‘Models - Hugging Face’
(make sure ‘saved’ is not a path to a local directory with something else, in that case)
or ‘saved’ is the correct path to a directory containing relevant tokenizer files