Longformer model evaluating all the datapoints to negative on evaluation data set after training

Models
1

I am using transformers library to download longformer-base-4096 model and training on my own dataset for classification using Trainer API. So, during evaluation, all the data is categorized to negative class. I am suspecting gradient vanishing problem. If anyone can respond to how this can be handled. Below are the training arguments which are passed to Trainer class.
training_args = TrainingArguments(
output_dir=’./results’,
do_train=True,
do_eval=True,
num_train_epochs=1,
per_device_train_batch_size=1,
per_device_eval_batch_size=1,
warmup_steps=500,
weight_decay=0.01,
evaluate_during_training=True,
logging_dir=’./logs’,
)

2

@Harika were you able to handle the vanishing gradient problem? If yes how?