I am new forum member. Recently, I have interest in AI, machine learning and stuff like this. I studied a documents and tutorials around the web. I am also following the Hugging Faces course on the platform. I completed section 1 and I started to do some experiments. At the moment, I consider myself an absolute beginner.
The first one I attempt is a small Chatbot for a PDF. Basically, you give a PDF to the ChatBOT, then you can start ask questions about it.
M project is here.
I took inspiration from similar projects on the web. However, all these projects use ChatGPT, and I dont’ want to do it because I don’t have free credit anymore and I don’t want to spend money for this kind of experiments.
The README.md contains the procedure to install and run it.
How does it work?
The main program is app.py, here how it works:
- First of all I read the PDF. The github project includes a PDF about Robinson Crusoe.
- I split it in 1000 characters chunks.
- Then I convert them in ids (my understanding is that models only operates with ids tensors, not text) and store them in a chromedb using the all-MiniLM-L6-v2 model.
- There is a ChatBOT loop.
- User ask the question and I use it to retrieve the top-k documents relative to it. I will use them as context.
- Then these lines use the context and the question to generate a response using the google/flan-t5-large model.
The problem is that the chatbot is not very exciting. I mean, I tested the code and it seems that PDF is correctly loaded, correctly splitted in chunks, probably correctly saved in chromedb (I don’t know how to verify this). The real problem is that the quality of the answer is very poor. Here an example of output:
Ask Question about your PDF: who is robinson crusoe?
Ask Question about your PDF: can you tell me more about him?
Ask Question about your PDF: please tell me more about him
He is a good student. He is clever. He improves quickly.
Ask Question about your PDF: can you tell me the member of his family?
Ask Question about your PDF: can you tell me a bit of robinson crusoe story
As you can notice the first answer is too short. The second answer is even worse. I would like more complete answer (something similar to ChatGPT). Moreover, it seems that it doesn’t generate answer but simply use the text in the PDF. I would like that the chatbot provide an answer by its own using only the top-k chunks as a context to generate the response.
Since I am a beginner, I am not familiar with the model to use for text generation in my specific scenario. Can anyone help me to figure out what’s wrong with my code?
Another thing I don’t understand is that to run the HuggingFaceHub class, I needed to create an Hugging Face Token. Can you explain to me why? How this service is billed and what are the rate limit. I tried to understand it from the platform but no luck.
Another improvement that I want to do is to persist my chroma database on disk so that I can register my chunk ids there once and then use it N time simply loading the content from there. I tried several code pieces, read doc, but I didn’t find a way to do it.
Thank you in advance for your help.