Finetuned Mistral 7B Instruct v3 having issues deployed to AWS Endpoint using Sagemaker HuggingFace library.
Exception: data did not match any variant of untagged enum PyPreTokenizerTypeWrapper at line 6952 column 3
Error: ShardCannotStart
Model Info:
sagemaker config
instance_type = “ml.g5.12xlarge”
instance_type = “ml.g5.12xlarge”
‘MAX_INPUT_LENGTH’: json.dumps(1024)
‘MAX_TOTAL_TOKENS’: json.dumps(2048)
‘MAX_BATCH_TOTAL_TOKENS’: json.dumps(8192)
“transformers_version”: “4.44.1”