Thanks for the response! I guess you probably won’t know the answer to this, but is it possible that the temperature and top_p were somehow swapped on accident? In the official Llama repo, it seems that a temperature of 0.6 and top_p of 0.9 are used, (https://github.com/facebookresearch/llama/blob/main/example_text_completion.py, https://github.com/facebookresearch/llama/blob/main/example_chat_completion.py). Whereas the generation_config.json has them swapped, temperature of 0.9 and top_p of 0.6.