Cannot reproduce the results

[valhalla]

cc @sshleifer

This post might also help

Facebook/bart-large-cnn has a low rouge score on cnn_dailymail

I had similar issues reproducing the ROUGE scores of facebook/bart-large-cnn on the CNN/Daily Mail dataset. First I tried the evaluation script examples/seq2seq/eval.py with the follwoing arguments: export DATA_DIR=cnn_dm python3 ./run_eval.py facebook/bart-large-cnn $DATA_DIR/val.source dbart_val_generations.txt --reference_path $DATA_DIR/val.target --score_path cnn_rouge.json --task summarization --device cuda --max_source_length 1024 --max_target_length 56 --f…