Hi,
I successfully use TextIteratorStreamer to stream output using AutoGPTQ transformer. However, the response will always start by repeating the prompt that was input an follow by the answer. Is there an option to turn off?
Hi,
I successfully use TextIteratorStreamer to stream output using AutoGPTQ transformer. However, the response will always start by repeating the prompt that was input an follow by the answer. Is there an option to turn off?