My question answering model requires an extra step of preprocessing the input and the output. How can I add those preprocessing scripts to work with Hosted Inference API (Widget on the website, too)?
hey @yigitbekir, as far as i know the inference api does not support custom pre- / post-processing logic, but you could easily include these steps within a dedicated web / streamlit application
if you need a custom widget, you can propose one on the
huggingface_hub library here: GitHub - huggingface/huggingface_hub: Client library to download and publish models and other files on the huggingface.co hub
In my understanding this Generic Inference API allows you to override the default pipeline used in Hosted Inference API.
In order to make this work you must:
- put your code to
- specify requirements in
library_name: generictag-value in
I tried it myself but I get following errors:
- “Distant resource does not have an ETag, we won’t be able to reliably ensure reproducibility.”.
- Or something like “the model took very long to load”. Probably it’s because of long wait for Docker container to install my required dependencies and load the model.
Probably there is a way to make this Generic Inference API work.
Now I’m planning to switch to Huggingface Spaces - probably I can set up my own processing logic for inference there.
upd: I’ve deployed my demo application to Huggingface Spaces and all works fine. You can specify your custom logic of data processing there as well as list of dependencies (both python and debian system packages) that are installed while building Docker container on Space launch.