I am using several HF pipelines. Boiled down, we are using two pipelines in the same code. Pseudo-code:
pipe1 = pipeline("question-answering", model=model_name1) results1 = pipe1(question=["Who?","What?","Why?"], context=sequences_to_classify) pipe2 = pipeline("zero-shot-classification", model=model_name2) results2 = pipe2(sequences_to_classify, candidate_labels=candidate_idxs)
I am looking for code examples on how to run these two tasks in parallel. I did a search on the community forum and did not find an answer. The basic hope is that it can be done in
- a RESTful service built with Flask or FastAPI
- a strong enough GPU can run both tasks with true parallelism
Thank you in advance for any pointers to existing code examples or help!