I already download the model and directly use the path.
model_id = "./Llama3"
pipeline = transformers.pipeline(
"text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"
)
It show the error
ValueError: You are trying to offload the whole model to the disk. Please use the `disk_offload` function instead.
Q1.
What is the define for You are trying to offload the whole model to the disk
If I use the model_id=meta-llama/Meta-Llama-3-8B
.
It will automatically download the folder models–meta-llama–Meta-Llama-3-8B on ./cache
So the download model is also the case that offload the whole model to the disk ?
Q2. Because I saw the offload the whole model.
what is the onlineload the model code