Chemistry experiment AI help tool

Models
1

I am developing a 3D interactive chemistry related chatbot, it’s supposed to answer questions concerning college/highschool level chemistry experiments, and perform entity extraction on its output to do the 3D actions (for example, user asks about how we do a reaction, the output that contains the procedure, I extract the reactants, products, verbs, spawn those elements, perform animations to mix them etc.), as well as answer questions about why the color changed, why the temperature changed, maybe if the experiments conditions were wrong it’ll change, do changes given the new queries etc.

this means, I need to do instruction fine tuning to dictate a specfific style that facilitates the entity extraction to my liking, the model would use phrases and verbs I want, and have knowledge of most chemistry reactions that are useful to college highschool students

I’d love to hear what someone’s take on this pipeline would be like

1 Like
2

Existing standalone LLMs, no matter how much they are fine-tuned, face challenges in integrating with other models and tools. Existing RAG frameworks and agent frameworks handle this to some extent…
Especially for tasks requiring accuracy and safety like this one, limiting output through structured output becomes quite important.

BTW, since the target is chemistry, I recommend asking on the Hugging Science Discord.