For instance distilbert-base-uncased, the numbers appear to be between -1 and 1. If I was to add any features to this for a downstream classification task, what feature scaling should I use to match? Does it make sense to apply feature scaling to the embeddings? I feel like they have some kind of scaling already, not sure if it’s appropriate to apply twice?
Thanks