How can i deploy an AWQ model to sagemaker? It works with inference endpoints but with Sagemaker if I pass in ‘HF_MODEL_QUANTIZE’:‘AWQ’ it doesn’t seem to work and there’s an error.
How can i deploy an AWQ model to sagemaker? It works with inference endpoints but with Sagemaker if I pass in ‘HF_MODEL_QUANTIZE’:‘AWQ’ it doesn’t seem to work and there’s an error.