Does model performance on a task determine how good the embeddings are?

Hi all,

does a model need to have good performance to consider their embedding layer output as new set of features? I was planning to extract the embeddings of my fine tuned Bert model and use them as input for PyCaret training, but my model is not performing well. So I am not sure if the embeddings of the model is any good.

Please share your thoughts.
Thanks in advance!