Hello,
How can I compare for long texts using LLM?The questions are of interest to the following:
- What is new in this article compared to the articles previously marked as read, by points,
- What of this article has already been in previously read articles, point by point.
I guess that the solution may lie through a bundle of Langchain, Vector DB, and LLM, but it is not entirely clear how to formulate the above questions. I am glad to any ideas.
What I did before:- I broke the text of the first article into pieces,
- I got embedding with the LLM API to them,
- Saved it in Vector DB,
- I sent requests to Vector DB and then, together with the results obtained, to the LLM API and received human-like responses.
Perhaps, it’s worth storing chunks in the view of embeddings to Vector DB and then try compare this article with previous one. Is it possible?
Best,
Alex