Strange conversation results

Hello I am just getting started with making conversational Discord Bots. I am very new to all this! I am pretty happy with my results so far, but am not sure how to proceed here.

This bot (based on lines of dialogue) does quite well generally, but they just output dozens of exclamation marks. It’s funny - but not gonna lie - a bit disturbing! It’s like they just had a nervous breakdown. There are no lines of exclamation marks that were in the data set of sample dialogue. Any thoughts? I used the GPT-2_medium for training, this is a link to the colab notebook: Google Colab

Thanks!

eg:

Hi Barney
Hey, sorry for the scare. I had to put on a show for the cameras.
It’s cool. Where are we going?
Gordon, we don’t have time for this!
Can I have my crowbar now?
Censored in-game by the sound of the Combine Barricade being dropped by Dog.)
What?
!!!