Using Transformers is recommended. They wrap common pitfalls and inconveniences for you. Even if not the Hugging Face Transformers implementation itself, you can find several third-party optimized versions if you look.
Since Transformers also act as wrappers for PyTorch and several other backends, models you create can be used directly as PyTorch models, making reusability issues less likely to arise.