Rouge eval metrics in summarization not computing

Amazon SageMaker
1

Hello @philschmid I was trying to fine tune a summarization model with a very small data set (100 samples) and none of the rouge eval metrics are being computed in the training job. Is there size limit in terms of samples that could be causing this?

Thank you.

image

2

Hello @Jorgeutd,

Are you using the examples/ script? and are you passing the hyperparameter → 'predict_with_generate': True,

predict_with_generate ( bool , optional, defaults to False ):
Whether to use generate to calculate generative metrics (ROUGE, BLEU).`

3

Thank you @philschmid . I reviewed indeed that the run summ.py script this time.

1 Like