Adoption of Evaluate

I was wondering if it is recommended to begin using Evaluate in place of evaluating models on test sets using Datasets.metric?

Hi @aclifton314, yes you can start using evaluate. In fact datasets will soon show deprecations warnings: Deprecate metrics by albertvillanova · Pull Request #4739 · huggingface/datasets · GitHub

While we are still very actively working on features such as the new evaluator classes there should not be any major changes on the metrics side.

Let us know how your experience is with the transition!

1 Like