Using the code from HF deploy with sagemaker option. Able to deploy to aws endpoint but the output is not answering a question. It keeps repeating the question and followed next predict token. Any help?

Llama3 8b instruct not answering question

sasuketaichou May 9, 2024, 5:07am 7

Thanks for the reply. After set MESSAGES_API_ENABLED=true I am able to have a proper chat.

Topic		Replies	Views
Truncated un-finished response after deploying hugging-face models Amazon SageMaker	0	382	January 19, 2024
Payload format for LeoLM/leo-mistral-hessianai-7b-chat Sagemaker Endpoint Amazon SageMaker	2	669	October 20, 2023
Sagemaker model generates incomplete responses (or even completely random output) Amazon SageMaker	0	185	May 23, 2024
InternalServerException when running a model loaded on S3 Amazon SageMaker	4	985	August 6, 2021
AWS Sagemaker doesn't return the full response Amazon SageMaker	1	134	July 17, 2024