I am training a custom QA model and my validation accuracy is starting from 61% and training from 54% after that training accuracy is continuously increasing and training loss decreasing whereas validation accuracy only increases till 3rd epoch to reach 64% (and training accuracy at 3rd epoch is 71%) and then starts decreasing and validation loss increases. I have around 90000 data and my model has 18.9M training parameter. I suspect that it is over fitting. I have tried everything to fix this my nothing helped. Can anyone help to identify the issue. Also even if there is over fitting can I still use the result till the 3rd epoch as validation loss increases after that?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Question about validation and testing loss | 6 | 2281 | April 19, 2022 | |
| Is my model overfitting or underfitting? | 0 | 564 | February 17, 2023 | |
| Seeking Clarification: Model Evaluation - Train and Val loss | 3 | 569 | April 10, 2024 | |
| Why transformer overfit quickly? how to solve it? | 6 | 19284 | February 28, 2023 | |
| Fine-tuning Wav2v2.0: Loss increasing, WER decreasing | 2 | 550 | June 30, 2023 |