Hello! I am trying to download llama-2 for text generation on google colab free version. I tried simply the following model_name = "meta-llama/Llama-2-7b-chat-hf" tokenizer = AutoTokenizer.from_pretrained(model_name, token=True) model = AutoModelForCausalLM.from_pretrained(model_name, token=True) …

Llama-2 on colab

obi77 August 8, 2023, 8:47pm 2

I think I am having the same issue Colab RAM Limit Exceeded: Unable to Run 3B Model Even with Quantization I will share a solution if I find it

Topic		Replies	Views
Colab CUDA OOM using Llama-2-7b-chat-hf even with 40GPU RAM 🤗Transformers	0	642	December 29, 2023
Running into cuda out of memory when running llama2-13b-chat model on multi-gpu machine Intermediate	5	7943	December 21, 2023
GPU usage increasing every loop when running inference Beginners	2	703	May 13, 2024
Does anyone have an idea how we can run llama2 with multiple GPUs? 🤗Transformers	1	957	October 26, 2023
Colab RAM Limit Exceeded: Unable to Run 3B Model Even with Quantization Beginners	0	852	August 8, 2023