Hello! I am using the EncoderDecoderModel for translating from Arabic to English by initializing the encoder with pretrained AraBERT checkpoint (a BERT model pre-trained on Arabic language) and a randomly initialized decoder. However, the BLEU score does not improve at all throughout the evaluation after each epoch, and the predicted sentences are very random and do not make sense.
Can anybody please help me? Here’s my colab notebook for the code: Google Colab