I am training a custom QA model and my validation accuracy is starting from 61% and training from 54% after that training accuracy is continuously increasing and training loss decreasing whereas validation accuracy only increases till 3rd epoch to reach 64% (and training accuracy at 3rd epoch is 71%) and then starts decreasing and validation loss increases. I have around 90000 data and my model has 18.9M training parameter. I suspect that it is over fitting. I have tried everything to fix this my nothing helped. Can anyone help to identify the issue. Also even if there is over fitting can I still use the result till the 3rd epoch as validation loss increases after that?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Why transformer overfit quickly? how to solve it? | 6 | 20202 | February 28, 2023 | |
| Not able to overfit a transformer model on my data | 0 | 541 | June 14, 2023 | |
| Is my model overfitting or underfitting? | 0 | 576 | February 17, 2023 | |
| Question about validation and testing loss | 6 | 2332 | April 19, 2022 | |
| Training llama2-7b-chat, is my model overfitting? i think my model is not learning anything? how to better train? | 3 | 623 | April 23, 2024 |