I suggest having a look at the hidden_states output of the model returned when output_hidden_states=True is passed to the forward call, or to the from_pretrained method while loading the model. The [CLS] embedding is the 0th slice of the output tensor of the last layer.