Hello there! I was able to fine-tune my own model for text classification (from a distilbert-base-uncased-finetuned-sst-2-english · Hugging Face model). Everything works correctly on my PC. Now comes the app development time but inference - even on a single sentence - is quite slow. I am processin…

How to make single-input inference faster? Create my own pipeline?

olaffson August 24, 2021, 1:52pm 8

thanks @NarsilTest but I am not sure to understand what you mean here. Could you please explain more? thanks!

Topic		Replies	Views
Make bert inference faster 🤗Transformers	6	11204	September 16, 2021
Model inference on tokenized dataset 🤗Datasets	2	6404	March 22, 2023
Inference speed between pipelines and Heads 🤗Transformers	0	325	April 3, 2023
Speeding up electra inference, multilabel classification 🤗Transformers	0	387	June 9, 2022
Auto Model for Sequence Classification take more than 20 minutes to classify a single sequence 🤗Transformers	3	288	March 7, 2024