Incomplete response from chatbot

Models
1

I created a chatbot using langchain llamacpp with llama-2-13b-chat, and it keeps giving me incomplete responses as such: “Sure! Here’s an example that uses the case operator to assign different field values in different situations. Suppose you have”

The following are the parameters used:

            llm = LlamaCpp(
                model_path="models/llama-2-13b-chat.ggmlv3.q2_K.bin",
                n_gpu_layers=40,
                n_batch=512,
                max_tokens=2000,
                streaming=True,
                callback_manager=callback_manager,
                verbose=False,
                temperature=0.75,
                top_p=0.9,
                top_k=40,
                repeat_penalty=1.18,
                f16_kv=True,
                last_n_tokens_size=64,
            )

I tried experimenting with different models and different parameters, yet I was still receiving incomplete responses.

2 Likes