An open question for anyone that has used Transformer models in a Streamlit app. I am using:::
pipeline(“summarization”, model=“sshleifer/distilbart-cnn-6-6”, tokenizer=“sshleifer/distilbart-cnn-6-6”,framework=“pt”)
::: to do summarization in the app. However, it takes about 55 seconds to create the summary, and it appears that 35 seconds or more of that time is spent downloading the model. Is there another way to access the model quicker? Perhaps by pre-loading the model to Streamlit Sharing (via the github repo the app sits in)?
Also, the summary generation part of the app appears to work once or twice, but if done any more times the app crashes. Has anyone else had this experience?