My question answering model requires an extra step of preprocessing the input and the output. How can I add those preprocessing scripts to work with Hosted Inference API (Widget on the website, too)?
hey @yigitbekir, as far as i know the inference api does not support custom pre- / post-processing logic, but you could easily include these steps within a dedicated web / streamlit application
if you need a custom widget, you can propose one on the huggingface_hub
library here: GitHub - huggingface/huggingface_hub: Client library to download and publish models and other files on the huggingface.co hub
Hello! There is a section in the Huggingface Hub Docs that describes Generic Inference API.
In my understanding this Generic Inference API allows you to override the default pipeline used in Hosted Inference API.
In order to make this work you must:
- put your code to
pipeline.py
- specify requirements in
requirements.txt
- specify
library_name: generic
tag-value inREADME.md
I tried it myself but I get following errors:
- “Distant resource does not have an ETag, we won’t be able to reliably ensure reproducibility.”.
- Or something like “the model took very long to load”. Probably it’s because of long wait for Docker container to install my required dependencies and load the model.
Probably there is a way to make this Generic Inference API work.
Now I’m planning to switch to Huggingface Spaces - probably I can set up my own processing logic for inference there.
upd: I’ve deployed my demo application to Huggingface Spaces and all works fine. You can specify your custom logic of data processing there as well as list of dependencies (both python and debian system packages) that are installed while building Docker container on Space launch.
@yigitbekir