I have trained Donut model in my custom dataset following this tutorial from @nielsr .
I have managed to get a decent accuracy 0.65% if I expect the values to be exactly the same (not using edit distance to compare expected value and prediction). I have been evaluating the results of the model and I have seen that the errors I get that don’t help to get a better accuracy are related to typos.
They are small typo errors, like missing a number in value, wrong typo, missing quote… Do you know why this could be happening? Too much training?
For finetuning this model for you custom extraction task, is it neccessary to finetune encoder or decoder? Or can I just freeze the encoder and just train the decoder?