In the introduction about llama, it says that llama2 was trained on bfloat16. The model weight uploaded to Hugging Face is on float16. I’m wondering why the current work based on llama2 loads it on float16 other than bfloat16.
In the introduction about llama, it says that llama2 was trained on bfloat16. The model weight uploaded to Hugging Face is on float16. I’m wondering why the current work based on llama2 loads it on float16 other than bfloat16.