I got a solution for the first Approach. Since I was using the Trainer API of huggingface for T5 and BART model, I just needed to provide generation_max_length=1024 to the trainer args.
eg:
training_args = Seq2SeqTrainingArguments(
output_dir="my_model",
evaluation_strategy="epoch",
learning_rate=3e-5,
per_device_train_batch_size=8,
per_device_eval_batch_size=8,
weight_decay=0.01,
save_total_limit=3,
num_train_epochs=3,
predict_with_generate=True,
fp16=True,
push_to_hub=True,
generation_max_length=1024,
generation_num_beams=4
)
trainer = Seq2SeqTrainer(
model=model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=test_dataset,
tokenizer=tokenizer,
data_collator=data_collator,
compute_metrics=compute_metrics,
)
This did the job for me and I can generate paragraphs now.