It doesn’t seem to matter what model I have used thus far be it a GPT4Q or LLAMA. I could say the conversation starts out with quick witty responses, it continues to slow down from 1 second responses to about 60-70 sec. responses to then once completed goes black.
I have tried different loaders and adjusted parameters generation guidelines with no change. I have tried to save the chat, delete it. Then use a json editor to cut out the failed parts to insert it to have it happen again.
I look over my computers usage and the RAM on my computer is under 50%, my CPU isn’t running hard nor is my GPU.
CPU I7 1300K
RAM 32 5600 DDR4
GPU RTX 4070
I have tried it on 7B to 30B models and have the same affect. The ones that seem to work best on my system are 13B with very good depth in their replies.
Anyone have any ideas?
