Hello @ayush-adeptmind,
From my understanding of your question you want to introduce a weighted loss into your multilabel classification problem.
For such a problem, the best way to do I’m aware of while using transformers library is to override the Trainer class (create a new trainer class which inherits from the actual Trainer) and override (by defining it again) the “compute_loss” method.
When instanciating BCEWithLogitsLoss, you should be able to insert a weight vector.
See How can I use class_weights when training? for more precise information and code snippets.
Have a good day!