XLNetForSequenceClassification

🤗Transformers
21

yeah i have checked

22

can we add MLP on top in custom Xlnetsequenceclassification

23

Xlnetsequenceclassification already adds a linear layer on top of XLNetModel, but if you want to add an extra layer then yes, surely you can do it. You could use the XLNetForSequenceClassification code for reference

24

sure thank you

25

I am using xlnet sequence classfication in dataset i have imbalance lables , i am facing overfit and only high count label is classified properly, what i can do?

26

You can set weight in CrossEntropyLoss so that it can take the imbalance into account. You should fine enough information when you look this up. As an alternative, but in the same usage, you can use PyTorch’s weighted sampler.

27

thanks, how we can add class weights for xlnet sequence classification, any reference, please.

28

@BramVanroy
Hi i am working text classification, dataset contains imbalanced labels
"classcount = np.bincount(train_data[‘label’]).tolist()
train_weights = 1./torch.tensor(classcount, dtype=torch.float)
train__sampleweights = train_weights[train_data[‘label’]]

#class_weight
weighted_sampler = WeightedRandomSampler(
weights=train__sampleweights,
num_samples=len(train__sampleweights),
replacement=True)"

but still in validation its not predicted well for less labels