Mistral-7B-Instruct-v0.3 vs Mistral-NEMO-12B

Models
1

Mistral-NEMO-12B is supposed to be better than Mistral-7B-Instruct-v0.3, the former being of higher parameters (12B). However, I used Mistral-7B-Instruct-v0.3 to implement a RAG system. I deployed the same code on both Render and Vercel. When the NEMO came out, I changed the model of the one on Render to NEMO but left the one on Vercel unchanged. I then watched the performances of the two applications with the same code base except for the models. The one with NEMO fails often. It is intolerant to even grammatical or punctuation errors. It often reads a literal interpretation to a prompt. For instance, the follow up prompt “Quote source” (i.e. it should quote the source of its last response) may return “Source”. However, 7B-Instruct excels in all these situations.

1 Like
2

hi @Ade-crown
Could it be about tokenization or embedding issue?

I changed the model of the one on Render to NEMO

Did you just change the model? Or did you re-run all procedures(chunking, indexing, etc…) for new system?

3

I just changed the model, and nothing else. I don’t think there’s a need to tamper with the database since the model embedding queries for the database is a separate model entirely. The only purpose of Mistral models in the process is to use the context provided them to give intelligent response to the user’s queries.