LMM fine tuning how to improve training for Question and Answer task?

Considering LMM fine tuning how to aproach when we think about data structure? Should i train with:

1)abrastract + question + answer
2)question + answer
3)abstract + question + answer added with question + answer

How to evaluate? Using perplexity? Bert Score? Any other metric?

What about the number of epochs considering the number of train entries? The effective batch size?

1 Like