Hello, total LLM noob here.
I am building a chat bot for my game and I am looking for a solution/model that has the following requirements:
- The chat bot can be provided some context (1-2 paragraphs of text)
- It can trigger a callback in my code when encountering a sentence that matches (not necessarily 100%) a predefined sentence
- I should be able to control the max length of the reponse
- It should run locally on my computer
- Ideally should update the conversations context
I have already tried several solutions (LLama2, LLama-cpp, etc…), some of them returned replies where the last sentence was truncated, others returned too many paragraphs, some were out of context, etc…
OpenAI platform seemed interesting except that it is an online service and not free.
Another option was to build a chat bot from scratch with PyTorch and LSTM but that would have been too long/complicated.
None of these solutions seem to be the right one.
Anyway, I will appreciate any pointers.