Hidden States of OpenAI GPT2 inconsistent

Do you run the model in evaluation mode?

i.e. model.eval()

=> this will turn off any dropout modules