Batch transform inference job - downloading model from the Hugging Face Hub on start up

Kateryna · October 8, 2021, 10:43pm

I try to run

huggingface/notebooks/blob/master/sagemaker/12_batch_transform_inference/sagemaker-notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Huggingface Sagemaker-sdk - Run a batch transform inference job with 🤗 Transformers\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "1. [Introduction](#Introduction)  \n",
    "2. [Run Batch Transform after training a model](#Run-Batch-Transform-after-training-a-model)  \n",
    "3. [Run Batch Transform Inference Job with a fine-tuned model using `jsonl`](#Run-Batch-Transform-Inference-Job-with-a-fine-tuned-model-using-jsonl)   \n",
    "\n",
    "Welcome to this getting started guide, we will use the new Hugging Face Inference DLCs and Amazon SageMaker Python SDK to deploy two transformer model for inference. \n",
    "In the first example we deploy a trained Hugging Face Transformer model on to SageMaker for inference.\n",
    "In the second example we directly deploy one of the 10 000+ Hugging Face Transformers from the [Hub](https://huggingface.co/models) to Amazon SageMaker for Inference.<"
   ],

This file has been truncated. show original

However I get this error:

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: [https://huggingface.co/api/models/cardiffnlp/twitter-roberta-base-sentiment]

It looks like an instance created by AWS SageMaker (client) is not allowed to download the model. Something specific I need to add in the role I use with AWS Sagemaker?

or there is an issue with a token in
model_info = _api.model_info(repo_id=model_id, revision=revision, token=use_auth_token)

Thanks,
Kate

philschmid · October 12, 2021, 12:19pm

I rerun the notebook: notebooks/sagemaker/12_batch_transform_inference at master · huggingface/notebooks · GitHub
and for me it worked. Can you please test again.

Kateryna · October 12, 2021, 4:35pm

I did with the same result.

Well, I started it from my own local environment with installed all need packages. (I run a lot of different kind SageMaker related code from my local environment and it worked.)

What I did not try is to run from Sage Maker Studio.

Topic		Replies	Views
ClientErro:400 when using batch transformer for inference Amazon SageMaker	11	2181	January 13, 2022
Error 403 when downloading model for Sagemaker batch inference Amazon SageMaker	9	3685	January 13, 2022
Running batch transform in Sagemaker on a Huggingface model from the Hub with parameters Beginners	2	1601	February 2, 2023
Endpoint Deployment Amazon SageMaker	1	1082	September 20, 2021
Use my finetuned Bert Model in SageMaker BatchTransform Amazon SageMaker	4	2861	April 30, 2022

Batch transform inference job - downloading model from the Hugging Face Hub on start up

Related topics