I am new to Gen AI and trying to fine-tune distilbert-base-uncased
and fine-tuned it using Peft/Lora. However, I am getting poor accuracy after training
The model is trained to perform multilabel classification using training data as
{
“hazard”: “Chemical burns from wet concrete”,
“task”: “Concrete Pouring”,
“SkillLevel”: “Non experienced”,
“SiteComplexity”: “Simple”,
“SafetyInPlace”: “No”,
“likelihood”: “CERTAIN”
},
{
“hazard”: “Chemical burns from wet concrete”,
“task”: “Concrete Pouring”,
“SkillLevel”: “Non experienced”,
“SiteComplexity”: “Simple”,
“SafetyInPlace”: “Yes”,
“likelihood”: “LIKELY”
},
here we are trying to predict the likelihood given the hazard, task, and three other 3 attributes. the label is a likelihood.
Validation Loss: 1.5924
2024-11-11 16:41:49,720 - INFO - Validation Accuracy: 25.30%
2024-11-11 16:41:49,720 - INFO - Epoch 2 - Validation Loss: 1.5924, accuracy: 25.30%
the model validation accuracy does not improve across epochs.
Is there. a way to understand why model is not training well.
and is there a way to get this information straight from the model to help imrove the accuracy