I was wondering would it be a nice solution, if instead of providing a new embedding layer to the model, why aren’t we focusing on providing additional features to the classification layer, and let it learn the custom classification layer during fine-tuning. Would it be right approach?