Deploying Stable Diffusion on s3-Memory issues


I have been tryinf over the past few days to deploy a stable diffusion model as endpoint through the s3 bucket.
The current issue according to the cloudwatch is that there is not sufficient memory to load the packages.
I am using a ml.g4dn.4xlarge instance for the endpoin to deploy the stabilityai/stable-diffusion-2. I believe this shoudl have been more than sufficient.
Does the sagemaker instance matter (I wouldnt expect so but I am just asking)?

Thank you