How to Fine tune a model to output specific format

I’m new using models. Be kind with me.

I want to fine tune a conversational model for a specific store. I have used Rasa, but it is not smart enough or I do not have all the questions users can ask.

I understand NER, Text classification. But when it comes to more further than those it’s complicated. I can’t find the answers.

I have tested some smart models on Hugging Face that actually understand I want to buy 4 of this product and 3 of that product and What is the price of this product?. That’s great.

But the problem is: I want the chatbot(model) to execute some code or the chatbot to output a specific format like JSON [{“product”: “id product”, “amount”: 4,…}] so that I can create the order in the system/database.

Also if there are multiple brand of milks and the user says I want six milks. The chatbot list the brands to ask the user to choose the brand he/she wants to buy.

I have tested some models that does that using a prompt but I want the model to fetch in the system/database to see if the brands are available or not before asking the user or output those brands in a specific format so that I can fetch those products.

So, basically I want to know how to fine tune (train without using system prompt) a conversational model so that it can understand what are the products and amount the user wanna buy. And to generate some specific format output, so that I can execute some code.