Could not fine-tune deplot model

[rlee002]

ValueError: Invalid image type. Expected either PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray, but got

The problem is caused when is_vqa is set to true, they use the pix2struct image processor automatically which requires an image input. Normally, when the processor input doesn’t include an image, it automatically switch back to a simple text tokenizer from image processor, however when is_vqa is set to true in pix2struct auto processor, seems this automatically handling doesn’t work, the processor stick to image processor even your input is text only. Therefore, a simple solution is to use the …

This solution helped in my case