Since HuggingFace is phasing out its benchmarking capabilities in transformers, what are some third party frameworks you suggest?
Sadly the deprecation warning only tells us that we should use them, but no example. I did some searching the last few hours and was unable to turn up anything useful. Basically just the huggingface tune repository, which is even older, and some small research benchmarking project which doesn’t really work with transformers’ pipelines (https://github.com/LukasHedegaard/pytorch-benchmark.