Update:
I managed to create the Serverless endpoint successfully!
Using the custom wheel of detectron2 for PyTorch 1.9.1 and torch vision 1.9.1 worked. I will keep the link for others here in case they need it for their projects http://mansimov.io/files/detectron2-0.6-cp38-cp38-linux_x86_64.whl
Will likely create a small GitHub repo for others to reproduce the LayoutLMv2 (and related models) on AWS sagemaker serverless in future
Thanks @philschmid and @marshmellow77 for help!