Running Llama model in Google colab

Models
1

Hi,

I’m trying to see a Llama 8B model demo in Google colab.

I just tried to run the following code in the colab prompt.

It ran into several errors.

I tried to fix using !pip install transformers[sentencepiece]
or !pip install --upgrade transformers

but to no avail

Any help will be much appreciated.

NameError Traceback (most recent call last)
in <cell line: 5>()
3 get_ipython().system(‘pip install transformers[sentencepiece]’)
4
----> 5 from transformers import pipeline
6
7 model_id = “meta-llama/Meta-Llama-3.1-8B-Instruct”

4 frames
/usr/local/lib/python3.10/dist-packages/torch/init.py in
762
763 __name, __obj = ‘’, None
→ 764 for __name in dir(C):
765 if __name[0] != '’ and not __name.endswith(‘Base’):
766 all.append(__name)

NameError: name ‘_C’ is not defined

2

Restarting the kernel will solve the problem, maybe

3

Hi,

Thanks for the suggestion. I tried restarting the kernel but it still doesn’t seem to work.

There’s another error in the following code snippet for text generation

Use a pipeline as a high-level helper

from transformers import pipeline

messages = [
{“role”: “user”, “content”: “Who are you?”},
]
pipe = pipeline(“text-generation”, model=“meta-llama/Meta-Llama-3.1-8B-Instruct”)
pipe(messages)

I still get errors like

HTTPError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py in hf_raise_for_status(response, endpoint_name)
303 try:
→ 304 response.raise_for_status()
305 except HTTPError as e:

18 frames
HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/resolve/main/config.json

The above exception was the direct cause of the following exception:

GatedRepoError Traceback (most recent call last)
GatedRepoError: 401 Client Error. (Request ID: Root=1-66d95297-65f74eaa1c17c1b55ae0ff7c;c1376837-e11c-4833-909c-9437d3f2b778)

Cannot access gated repo for url
Access to model meta-llama/Meta-Llama-3.1-8B-Instruct is restricted. You must be authenticated to access it.

The above exception was the direct cause of the following exception:

OSError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_gated_repo, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, **deprecated_kwargs)
418 if resolved_file is not None or not _raise_exceptions_for_gated_repo:
419 return resolved_file
→ 420 raise EnvironmentError(
421 “You are trying to access a gated repo.\nMake sure to have access to it at "
422 f"https://huggingface.co/{path_or_repo_id}.\n{str(e)}”

OSError: You are trying to access a gated repo.
Make sure to have access to it at
401 Client Error. (Request ID: Root=1-66d95297-65f74eaa1c17c1b55ae0ff7c;c1376837-e11c-4833-909c-9437d3f2b778)

4

I also meet the problem for the begin.
You miss the token and you can go to the huggingface website → setting → Access Tokens

import os
os.environ['HF_TOKEN'] = 'my_value'

value = os.getenv('HF_TOKEN')
print(value)  # my_value
5

Yeah…figured it out myself. Thanks. Works with token but additionally you need to request access to models and must be approved by the lllama forum .

6

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.