Saving local bert/roberta model not working using save_pretrained

Beginners
1

Hi,
I am trying to tune BERT model and then save the pre-trained config file using model.save_pretrained but the config file didnt save.

Due to security of my organization I cant directly download pre-trained bert model, so I download the models and files individually and then ran them locally.

After training, I can save the tokenizer using tokenizer.save_pretrained and ‘.bin file’ using torch.save.

But the mode.save_pretrained failed. I tried to check huggingface forums and github issues but didnt find any solution.

My tuned code:

class BERT_Arch(torch.nn.Module):
def init(self):
super(BERT_Arch, self).init()
config = BertConfig.from_pretrained(“/home/rp/projects/nlp/bert-base-uncased”, output_hidden_states=True)
self.l1 = BertModel.from_pretrained(“/home/rp/projects/nlp/bert-base-cased”,config=config,ignore_mismatched_sizes=True)
self.pre_dropout = torch.nn.Dropout(0.3)
self.pre_classifier = torch.nn.Linear(768, 768)
self.dropout = torch.nn.Dropout(0.3)
self.classifier = torch.nn.Linear(768, 3)

def forward(self, input_ids, attention_mask, token_type_ids):
    output_1 = self.l1(input_ids=input_ids, attention_mask=attention_mask, token_type_ids=token_type_ids)
    hidden_state = output_1[0]
    pooler = hidden_state[:, 0]
    pooler = self.dropout(pooler)
    pooler = self.pre_classifier(pooler)
    pooler = torch.nn.ReLU()(pooler)
    pooler = self.dropout(pooler)
    output = self.classifier(pooler)
    return output

model = BERT_Arch()
model = model.to(device)

####saving models

output_dir = ‘/home/rp/projects/nlp/dl_models/model2/’

Create output directory if needed

if not os.path.exists(output_dir):
os.makedirs(output_dir)

print(“Saving model to %s” % output_dir)

Save a trained model, configuration and tokenizer using save_pretrained().

tokenizer.save_pretrained(output_dir)
model_to_save = model.module if hasattr(model, ‘module’) else model # Take care of distributed/parallel training

model_to_save.save_pretrained(output_dir)

Can anyone point out any issue in my code?