Idea for building transformer from scratch

Hey, I have 3 weeks to finish a project and would like to build a transformer (BERT) from scratch with Torch and then import the model weights from Hugging Face. Is this a feasible project? Is there any reason importing the weights wouldn’t work?

Thanks a lot! :slight_smile: