Chatbot for support model selection

I have many articles without labels and want to build a chatbot to answer questions based on that.
Is it possible to make it on the fully open-source model? I have tested BERT with context - it’s a disaster. Viccuna that haven’t seen context performs much better than that.