Hi, I am new to transformers and have a general question although I will ask it to my task at hand. I need to classify amino acid sequences and wanted to use the ProtBERT model to do so. This means I want to add a classification head at the end of the model.
Having seen the tutorial , I have seen how to do this using DistilBertForSequenceClassification. I was wondering if such a module is available for all language models i.e is there ProtBertForSequenceClassification or is this available for only specific language models
It seems the ProtBERT checkpoints use the bert architecture so you can use them with BertForSequenceClassification.
In general you can just try AutoModelForSequenceClassification.from_pretrained(checkpoint_name) and it will return an error if there is no sequence classification model in the library for the architecture you are using with checkpoint_name.
1 Like
Thank you!
Can this be used for MultiClass classification or only for 2?
I am running into a this error:
[CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling cublasCreate(handle)]
and have found that this may be an issue due to an inconsistency between number of labels and number of output units.
The number of labels can be specified with num_labels=xxx in your call to from_pretrained.