Control diffusion processes with Optimum Neuron - ControlNet + IP-Adapter

I love using optimum.neuron.NeuronStableDiffusionPipeline and wish to build a pipeline that control Stable Diffusion generation for example, generates an image that’ll preserve the spatial information from the depth map.

Doing it today without optimum neuron is less streamlined and I wish to simplify the process. I see similar example like Image-to-Image or Inpaint. Is it doable request? Should I open a PR or is there a developer guidance I can use to build it myself?