Hello,
I’m trying to transfer learn the features from the following pre-trained model ‘bhadresh-savani/distilbert-base-uncased-emotion’. Using the following function call:
checkpoint_sentiment = 'bhadresh-savani/distilbert-base-uncased-emotion'
sentiment_config = AutoConfig.from_pretrained(checkpoint_sentiment, output_attentions=True,output_hidden_states=True)
self.model_sentiment = AutoModelForSequenceClassification.from_pretrained(checkpoint_sentiment,config=sentiment_config)
When I look at the modules I get the following 4 final layers:
- pre_classifier.weight
- pre_classifier.bias
- classifier.weight
- classifier.bias
Is it possible to get the intermediate value from pre_classifier? My aim is to get the 768-dimensional feature and use it for a different part of the larger model.
Thanks!