Error: 'AlbertModel' object has no attribute 'bias' -Trying to use custom corpus pretrained Albert chkpt

🤗Transformers
1

Hello All,

I am simply trying to extract use a an albert model pretrained on a custom corpus to get embeddings I coded the following where prt_config is myJSONconfig, and prot_model is a variable containing the path to my pretrained checkpoint. When I run my script I get the error - ‘AlbertModel’ object has no attribute ‘bias’ can someone provide some guidance I am using transformers 4.9.4? Thanks all.

ptcfg = AlbertConfig.from_json_file(prt_config)
modelprt = AlbertForPreTraining(ptcfg)
modelprt = load_tf_weights_in_albert(modelprt, self.ptcfg, prot_model)

Regards,
Che