Automatic1111 or ComfyUI

Is there a way to run Automatic1111 or ComfyUI locally with ControlNet while abstracting the Stable Diffusion layer using an API to Hugging Face? The idea is to run the user interface (Automatic1111 or ComfyUI) and ControlNet locally on my machine while offloading the heavy lifting (the actual image generation by Stable Diffusion) to an API like Hugging Face. I just want to benefit from the flexibility and control offered by ControlNet while not being limited by local hardware for the image generation process.