I think the error is saying something different.
Could you try adding the accept
parameter to
# create Transformer to run our batch job
batch_job = huggingface_model.transformer(
instance_count=1,
instance_type='ml.g4dn.xlarge',
output_path=output_s3_path, # we are using the same s3 path to save the output with the input
accept="application/json",
strategy='SingleRecord')