Hi, I am trying to deploy meta-llama/Llama-2-70b-chat-hf · Hugging Face on Sagemaker. Following the code, it recommends me to use a “ml.g5.2xlarge” instance. It seems a very small instance given the number of parameters, but that is what the code indicates. However, the deployment (after about 1…

ai.djl.engine.EngineException: GPU devices are not enough to run 2 partitions

dhruvilHV September 2, 2024, 10:06am 2

did you get the solution for this issue?

Topic		Replies	Views
Error hosting endpoint when deploying model Amazon SageMaker	2	3065	March 27, 2024
I am stuck with only g4dn.12xlarge Amazon SageMaker	1	590	January 2, 2024
Text-generation-inference: "You are using a model of type llama to instantiate a model of type ." Models	5	7573	November 3, 2023
Recommend an instance for MPT-7B and MPT-30B inference Amazon SageMaker	2	406	July 19, 2023
Impossible to use flan-t5-xxl in a batch-transform job Amazon SageMaker	3	1150	May 23, 2023