I would like to deploy a reranker local. I am loading it as a crossencoder like in here how to use reranker model with langchain in retrievalQA case? · Issue #13076 · langchain-ai/langchain · GitHub
And then add it to a Langchain’s retriever chain.
I would like to have it deployed in local in a docker container and send queries to the endpoint, like I do with the Text Embedding Inference.