Hello everyone, I am trying to use Llama-2 (7b) from Hugging face. With below code I was able to load the model successfully but when I am trying to generate the output its taking forever. Code from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained(…

Hugging Face Llama-2 (7b) taking too much time while inferencing

ahoo1260 June 23, 2024, 10:25am 2

Hi, did you find a solution?

Topic		Replies	Views
meta-llama/Llama-3.2-11B-Vision-Instruct did not reply 🤗Transformers	10	12899	October 29, 2024
Models slow on M1 Pro 16gb Beginners	0	725	December 18, 2023
Why is the huggingface generater much slower than the original llama2 generater? 🤗Transformers	0	1319	November 23, 2023
Why the model loading of llama2 is so slow? 🤗Transformers	6	9423	April 24, 2024
Trying the inference with model Llama-2-70b-hf on 2 A100 (80g) GPUs but getting errors Beginners	6	6578	November 28, 2023