Looking to fine-tune a model for QA/Text-Generation (not sure how to frame this) and I’m wondering how to best prepare the dataset in a way that I can feed multiple answers to the same question?
My goal is to facilitate the creation of a unique answer to a given question that is based on the input answers. The answers are longer-form (2 to 3 sentences) and I want the model to output this kind of length too.
For now my idea is to fine-tune GPT-2 and look at this as a text generation problem (but I don’t know how GPT-2 would treat multiple answers to the same question - would it adjust the weight simply in favor of the used tokens in the answers?). Maybe creating a summary of the given answers would have the same effect.
What would be the best approach to this?