I could not able to use save_pretrained on my T5 Model

🤗Transformers
1

I have fine tuned T5 model on a task and i am trying to save the model using save_pretrained method available for huggingface Model but i could not able to do so .

image
image789Ă—323 18.2 KB

can some say what is the error ?

2

What is T5Finetuner? That is not a Transformers model.

3

its just a class in pytorch.

class T5FineTuner(nn.Module):
    def __init__(self,model,tokenizer):
        super(T5FineTuner, self).__init__()
        self.t5 = model
        self.tokenizer = tokenizer
        
    def forward(self,input_ids, attention_mask=None, decoder_input_ids=None, decoder_attention_mask=None, lm_labels=None):
        outputs = self.t5(
            input_ids=input_ids,
            attention_mask=attention_mask,
            decoder_input_ids = decoder_input_ids,
            decoder_attention_mask=decoder_attention_mask,
            labels=lm_labels,
        )
        return outputs

Model is created using this class!

4

The class seems pointless because it doesn’t add functionality on top of T5 as currently written. You could have just used T5 as-is. save_pretrained is part of HF subclassed models, not of PyTorch’s nn.Module’s.

Either don’t use this class and just finetune the given T5 model. Or instead of subclassing nn.Module, subclass transformers.PreTrainedModel.

2 Likes