Augmenting partially covered objects

Hi Community,

I currently have a case where users should be able to select an object from an image, which will then be stored in their repository. For this, I run the original image trough SAM, they tab on the object, then the object will be segmented.

The issue is that some objects are covered by others, resulting in a final segmented image which might miss pieces about the object (see segmented image attached. The goal is to guess the missing pieces of the object).

Given that I am fairly new to computer vision tasks, I wonder what approach/model would be suitable for the task of guessing the missing ares of the object… which approach, off-the-shelve model, or training a custom one…?

Appreciate any thoughts on this. Roman