Was wondering what the experts think and whether this is a sensible approach. The pre-trained Squad 2.0 models perform well in a custom domain, but can be greatly improved, given the target domain is rather narrow and the vocabulary is different but there is overlap.
Do you think it is worth obtaining a custom dataset, say 1000 observations, using the same methodology as Squad v2.0 but derived from data of the target domain?
Is 1000 observation enough for the fine-tuning?
Hi @swayson, not an expert here but fine-tuning on your domain should give better results. I can’t comment on if 1000 examples will e enough or not, you’ll probably need to experiment.
Also have look at this question generation models. You can try to create synthetic QA corpora using these models. Synthetic QA corpora has shown to improve results for QA.
2 Likes
Thank you @valhalla ; I am going to give the synthetic QA models a shot and see if I can get some improvements.
Here’s a relevant paper. See table 2 for Synthetic QA results.
1 Like