in which function is the best way to use the temperature parameter .from_pretrained or .generate()
It seems that in both functions we can pass the temperature parameter. Which is the best way to do this?
Model = AutoModelForCausalLM.from_pretrained(model_name,
device_map="auto",
temperature=0.1,
do_sample=True,
torch_dtype=torch.bfloat16)
or
Model.generate(**model_input,max_length=1000, temperature=0.1,do_sample=True,)