Hi there!
Please guys while Fine Tuning (not a System Prompt) a basic model which is pretty wide scoped how to train a mode to respond only to provided in data sets topics and subjects and ignore whatever the rest ones mentioning something similar in response -I’m not aware of this, or -this is something out of our scope etc.
E.g. model for maths assistance not to run expanded on the topic of philosophy or cookie recipe.
I think you want to train a model that doesn’t go off-topic. The reasoning model, which is currently popular, is a model that does go off-topic, so you should probably do the opposite.
In short, you just need to make sure that the model only knows about that task. You should also train it to ignore unrelated topics. If you just want it to complete tasks, training a base model is usually relatively reliable, but if you want it to retain chatbot-like abilities, like in this case, you should probably train it based on an Instruct model.
To fine-tune a basic model to respond only to specific topics and ignore others, follow these structured steps:
Data Collection and Preparation:
Create a Diverse Dataset: Gather a large and diverse set of questions and answers related to your target topic (e.g., math). Ensure the dataset covers a wide range of queries within the subject to help the model understand its scope.
Exclude Irrelevant Topics:
Filter Out Non-Related Content: Ensure that the training data contains only the target topic. Avoid including any examples related to other subjects (e.g., philosophy, cooking).
Include Negative Examples:
Add Off-Topic Samples: Incorporate questions from unrelated topics into your training data. Train the model to respond with specific phrases like “I’m not aware of this” or “this is out of my scope” when encountering such questions.
Structural Consistency:
Maintain Consistent Prompts: Keep the system and user prompts consistent between training and deployment to ensure the model’s behavior remains predictable.
Dataset Quality:
High-Quality QA Pairs: Ensure the questions and answers are well-crafted and representative of real-world scenarios. Poor-quality data can negatively impact the model’s performance.
Training and Evaluation:
Fine-Tuning Parameters: Use guidelines such as having 50-100 examples per task. Monitor the model’s performance during training to ensure it stays within the desired scope.
Evaluation: Test the model with a mix of on-topic and off-topic questions to verify its ability to stay focused on the target subject.
Post-Training Adjustment:
Refinement: If the model still veers off-topic, refine the training data by adding more relevant examples or improving the clarity of off-topic responses.
By following these steps, you can train a model to effectively respond only to the topics provided, ensuring it handles related queries appropriately while ignoring others.