I’m using SBERT to compute embedding for SEC 10-K filings and for patent grants.
The 10-K filing have what I would call mostly non-technological information, but a select few passages in the 10-Ks are relatively dense in technological information.
Currently, I use SBERT trained on patents to create extractive summaries of the 10-Ks, and then use SBERT to compute embeddings of the summaries. I then compute cosine similarities with SBERT embeddings from patents.
But it isn’t obvious to me that SBERT trained on patents is optimal for extracting the passages of 10-Ks with the highest density of technological language.
How can I do this better? Any thoughts and ideas are greatly appreciated!