Accessing certain hidden layer layer outputs

Intermediate
1

I am trying to access output of hidden layers for BLIP caption generator with code output=model(**inputs, output_hidden_states=true). I observed that this significantly slowed down the inference as it calculates, stores all the hidden layer outputs. it is taking approximately 6 seconds for inference. Is there any way I can access only certain hidden states instead of outputs from all hidden states? or speedup inference ?