LLM’s sometimes decline, then answer the same question

Intermediate
1

Why would an LLM initially refuse to answer but then provide an answer when asked the same question again?

1 Like
2

This is not a characteristic of LLM itself, but a problem that is rooted in the techniques used to operate LLM. Specifically, because parameters such as temperature are often set, the response content changes slightly each time. (Non-deterministic) In other words, the “mood” of LLM changes each time. In Hugging Face, if do_sample=False, the same result will be returned each time. (Deterministic)

3

Thank you! is there a way to disable the model from using the history of the conversation without having to reset the application?

1 Like
4

I think it depends on the software, but unless you give the history to the LLM, it’s normal for it not to remember the history. So I think you can manage it by tweaking the data just before you give it to it.