Tutorial: Implementing Transformer from Scratch - A Step-by-Step Guide

Hey @ bird-of-paradise
thanks for the guide. I am looking at how to build and train an encoder-decoder model (based on modernBERT) with the huggingface Trainer: Support modernBERT for encoder-decoder models · Issue #35385 · huggingface/transformers · GitHub
Do you have any advice for it?

1 Like