Hi all! I’m working on a project exploring how to make it easier for ML engineers to optimize and deploy models across different hardware configurations.
One problem we’re interested in is how much engineering time goes into repeatedly adapting models to meet hardware-specific memory, latency, and throughput constraints, especially in robotics and edge AI.
We’ve built an early prototype, Packlet, to explore this workflow, and would really appreciate feedback from engineers who have worked on similar deployment problems.
In particular:
-
How much of your deployment workflow involves manual optimization and benchmarking?
-
Would a tool that streamlines model/hardware selection be useful in practice?
-
What’s the biggest limitation you see in an approach like this?
Prototype: https://dub.sh/packlet-v1
Any technical feedback, including reasons this wouldn’t work in your workflow, would be incredibly helpful! Thank you!