Get intermediate result from pre_classifier layer

Beginners
1

Hello,

I’m trying to transfer learn the features from the following pre-trained model ‘bhadresh-savani/distilbert-base-uncased-emotion’. Using the following function call:

checkpoint_sentiment = 'bhadresh-savani/distilbert-base-uncased-emotion'
sentiment_config = AutoConfig.from_pretrained(checkpoint_sentiment, output_attentions=True,output_hidden_states=True)
self.model_sentiment = AutoModelForSequenceClassification.from_pretrained(checkpoint_sentiment,config=sentiment_config)

When I look at the modules I get the following 4 final layers:

  • pre_classifier.weight
  • pre_classifier.bias
  • classifier.weight
  • classifier.bias

Is it possible to get the intermediate value from pre_classifier? My aim is to get the 768-dimensional feature and use it for a different part of the larger model.

Thanks!