Weird characters in generated text

I have trained a GPT-2 model on my dataset. When I generate text from it, sometimes it would generate text like “�����������������������”. Going on and on. Some other times, it will generate normal text but then it will start generating just “�” characters. Does anybody have a clue why it is doing this?