AutoTokenizer 404 error issue

🤗Transformers
1

whenever im trying to declare AutoTokenizer with any model be it roberta-base or distilbert-base-uncased, i am getting this error always. How to fix this

HTTPStatusError Traceback (most recent call last)
/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_http.py in hf_raise_for_status(response, endpoint_name)
556
→ 557 def _curlify(request: requests.PreparedRequest) → str:
558 “”"Convert a `requests.PreparedRequest` into a curl command (str).

/usr/local/lib/python3.11/dist-packages/httpx/_models.py in raise_for_status(self)
828 message = message.format(self, error_type=error_type)
→ 829 raise HTTPStatusError(message, request=request, response=self)
830

HTTPStatusError: Client error ‘404 Not Found’ for url ‘https://huggingface.co/api/models/google-bert/bert-base-cased/tree/main/additional_chat_templates?recursive=false&expand=false’

The above exception was the direct cause of the following exception:

RemoteEntryNotFoundError Traceback (most recent call last)
/tmp/ipykernel_37/974764345.py in <cell line: 0>()
2 train_labels = df[“generated”]
3 model_name = “google-bert/bert-base-cased”
----> 4 tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code = False, use_fast = True)

/usr/local/lib/python3.11/dist-packages/transformers/models/auto/tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)

/usr/local/lib/python3.11/dist-packages/transformers/tokenization_utils_base.py in from_pretrained(cls, pretrained_model_name_or_path, cache_dir, force_download, local_files_only, token, revision, trust_remote_code, *init_inputs, **kwargs)
1955 single_file_id = file_id
1956 else:
→ 1957 # At this point pretrained_model_name_or_path is either a directory or a model identifier name
1958 additional_files_names = {
1959 “added_tokens_file”: ADDED_TOKENS_FILE, # kept only for legacy

/usr/local/lib/python3.11/dist-packages/transformers/utils/hub.py in list_repo_templates(repo_id, local_files_only, revision, cache_dir)
159 def is_remote_url(url_or_filename):
160 parsed = urlparse(url_or_filename)
→ 161 return parsed.scheme in (“http”, “https”)
162
163

/usr/local/lib/python3.11/dist-packages/transformers/utils/hub.py in (.0)
159 def is_remote_url(url_or_filename):
160 parsed = urlparse(url_or_filename)
→ 161 return parsed.scheme in (“http”, “https”)
162
163

/usr/local/lib/python3.11/dist-packages/huggingface_hub/hf_api.py in list_repo_tree(self, repo_id, path_in_repo, recursive, expand, revision, repo_type, token)
3050 “”"
3051 return [
→ 3052 f.rfilename
3053 for f in self.list_repo_tree(
3054 repo_id=repo_id, recursive=True, revision=revision, repo_type=repo_type, token=token

/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_pagination.py in paginate(path, params, headers)
35 session = get_session()
36 r = session.get(path, params=params, headers=headers)
—> 37 hf_raise_for_status(r)
38 yield from r.json()
39

/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_http.py in hf_raise_for_status(response, endpoint_name)
569
570 for k, v in sorted(request.headers.items()):
→ 571 if k.lower() == “authorization”:
572 v = “” # Hide authorization header, no matter its value (can be Bearer, Key, etc.)
573 parts += [(“-H”, “{0}: {1}”.format(k, v))]

RemoteEntryNotFoundError: 404 Client Error. (Request ID: Root=1-68e8f1dd-560fc9b74a22a3a463591f2f;d4e39f2d-838b-4965-bda7-efee2ead7f99)
additional_chat_templates does not exist on “main”

1 Like
2

Some recent versions of Transoformers had bugs, so they may be fixed in an update.
pip install -U transformers

1 Like
3

Thank you so much man, it works. Really means a lot

1 Like