Running LLaMA 3.1 8B Model Downloaded from Meta - Missing Configuration File

Beginners
1

Hello everyone,

I recently downloaded the LLaMA 3.1 8B model directly from Meta’s website using their provided link. I’m trying to implement this model on Kaggle/Colab for testing purposes before investing in dedicated hardware.

Current Setup

I have downloaded the following files:

  • consolidated.00.pth
  • params.json
  • checklist.chk
  • tokenizer.model

When trying to load the model locally (pointing to the files’ location), I’m getting an error indicating that config.json is missing. I’m specifically trying to run this directly on Kaggle without using tools like Ollama or other third-party software.

  1. What additional files or configurations do I need to get this working?
  2. Is there a specific way to point to these files when implementing them on Kaggle?
  3. Are there any additional steps I need to take to properly set up these model files?

I would greatly appreciate guidance on how to proceed with implementing these files directly from Meta’s website into a Kaggle environment.

Thank you in advance for your help!

1 Like
2

The Hugging Face format is a model per folder. It is better to avoid handling it on a file-by-file basis. Download the entire directory and use it. Alternatively, you can download it by loading it directly from HF using from_pretrained() or similar.

3

Thank you for your response. I would like to clarify a few points:

  1. My current files were downloaded directly from Meta’s official website (not from Hugging Face). These are the files I received:

    - consolidated.00.pth
- params.json
- checklist.chk
- tokenizer.model

  2. Regarding loading from HF using from_pretrained():
    Could you provide an example of the correct code to load the LLaMA 3.1 8B model on Kaggle? Would it be something like:

    from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "meta-llama/Llama-3.1-8b-hf"  # Is this the correct path?

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

  3. Most importantly, how can I download and save the model locally so I don’t need to re-download it every time I restart Colab? Is there a way to:

    • Download the complete model folder structure from HF
    • Save it locally
    • Load it from local storage in future sessions

I’m new to working with LLMs and would appreciate guidance on the proper setup process, especially for maintaining a local copy of the model.

Thank you!

1 Like
4

2

Like this.

hf_token = "hf_******" # You can bring it from anywhere...
#model_name = "meta-llama/Llama-3.1-8b-hf" 
model_name = "meta-llama/Llama-3.1-8B"  # from https://huggingface.co/meta-llama/Llama-3.1-8B
tokenizer = AutoTokenizer.from_pretrained(model_name, token=hf_token) # Llama3 is a gated model, so you need a token.
model = AutoModelForCausalLM.from_pretrained(model_name, token=hf_token) # same above

3

There are several methods, but I recommend snapshot_download because it’s the easiest and most efficient. There is also a method that uses git.

About loding from local:

model_path = "./llama3-8b" # your downloaded model directory path
model = AutoModelForCausalLM.from_pretrained(model_path)