Hello everyone!
I am new to custom inference handler and I have been struggling with making mine work. I am trying to create one for the pretrained DONUT model naver-clova-ix/donut-base-finetuned-docvqa (naver-clova-ix/donut-base-finetuned-docvqa · Hugging Face) that takes a prompt and an image.
When calling the endpoint I get the error “Bad gateaway”. After some debugging, I figured out that the error comes from the call to model.generate() in the handler.py.
I checked all the attributes passed to the methods and compared them to their values when being executed in my local machine, and everything looks okay.
My repo is uify/Donut_test (https://huggingface.co/uify/Donut_test)
Any idea on how to fix the error?
PS: The image is sent encoded in b64 because the endpoint doesen’t allow requests with content-type multipart, so the requests are application/json with two fields, one for the prompt and one for the image encoded in base64.