Optimum vs Accelerate

🤗Optimum
1

What are the key differences between HF Accelerate and HF Optimum? Can they be used together?

1 Like
2

Hi @jeromeku ,

Accelerate is mainly used for distributed training on a variety of hardware, with techniques as offloading and data parallelism.

Optimum is more a toolkit, giving access to several tools either geared towards speeding up training or inference, or towards deployment:

  • ONNX, TFLite export of Transformers and Diffusers models.
  • ONNX Runtime integration (including quantization, ORT optimizations)
  • PyTorch’s BetterTransformer integration (this one is compatible with accelerate!)
  • Integration with OpenVINO, NNCF, Intel Neural Compressor (including quantization & pruning)
  • Integration with Habana Gaudi hardware for training
  • Integration with Graphcore IPUs for training

Feel free to refer to the documentations: 🤗 Optimum & Accelerate

2 Likes
3

Just to extend the previous answer, Optimum.onnxruntime module also supports training acceleration with ORTTrainer class.

4

Thanks for the responses! So Accelerate is a general framework for distributed training / inference while Optimum is oriented more towards architecture-specific optimizations which can’t be easily abstracted under a single framework. Would be interesting to compare / learn from other optimization frameworks (e.g., ColossalAI, Alpa) – let me know how I can contribute!

5

For sure comparing with what other framework propose is interesting! There are so many out there that it is easy to get lost. On the inference side, I think a comprehensive benchmark across the most-popular deployment tools is still something to be done.

1 Like
6

Great! Is there a working group around this currently? Perhaps a good starting point is to compile a list of frameworks bucketed by training and inference as well as currently available evaluation datasets / leaderboards by domain / dataset.

1 Like