Issue with processor.push_to_hub when pushing ViltProcessor to Hugging Face Hub

[muhammadfhadli]

Hi everyone,

I’m encountering an issue when trying to push my ViltProcessor to the Hugging Face Hub. Below is my code:

from PIL import Image
from datasets import load_dataset
from transformers import ViltProcessor, ViltForQuestionAnswering, TrainingArguments, Trainer, DefaultDataCollator
import torch
import itertools

OUTPUT_DIR = 'muhammadfhadli/vilt-b32-finetuned-vqa-hse-v0.0.1'

# Load and display dataset
dataset = load_dataset("json", data_files="dataset.json", split="train")
print(f"dataset: {dataset}")

# Display image size
image = Image.open(dataset[0]['image_id'])
print(f"Image size: {image.width}x{image.height}")

# Generate label mappings
labels = list(itertools.chain(*[item['ids'] for item in dataset['label']]))
unique_labels = list(set(labels))
label2id = {label: idx for idx, label in enumerate(unique_labels)}
id2label = {idx: label for label, idx in label2id.items()}

# Replace label IDs in the dataset
def replace_ids(inputs):
    inputs["label"]["ids"] = [label2id[x] for x in inputs["label"]["ids"]]
    return inputs

dataset = dataset.map(replace_ids)
flat_dataset = dataset.flatten()
print(flat_dataset.features)
print(f"flat_dataset: {flat_dataset}")

# Initialize processor
model_checkpoint = "dandelin/vilt-b32-finetuned-vqa"
processor = ViltProcessor.from_pretrained(model_checkpoint)

# Preprocess data
def preprocess_data(examples):
    images = [Image.open(path).convert("RGB") for path in examples['image_id']]
    encoding = processor(images, examples['question'], padding="max_length", truncation=True, return_tensors="pt")

    # Squeeze batch dimension
    for key, value in encoding.items():
        encoding[key] = value.squeeze()

    # Prepare target tensors
    encoding["labels"] = [
        torch.zeros(len(id2label)).scatter_(0, torch.tensor(labels), torch.tensor(scores))
        for labels, scores in zip(examples['label.ids'], examples['label.weights'])
    ]

    return encoding

processed_dataset = flat_dataset.map(preprocess_data, batched=True, 
                                     remove_columns=['question', 'image_id', 'label.ids', 'label.weights'])
print(f"processed_dataset: {processed_dataset}")

# Prepare for training
data_collator = DefaultDataCollator()
model = ViltForQuestionAnswering.from_pretrained(
    model_checkpoint,
    num_labels=len(id2label),
    id2label=id2label,
    label2id=label2id,
    ignore_mismatched_sizes=True
)

training_args = TrainingArguments(
    output_dir=OUTPUT_DIR,
    per_device_train_batch_size=4,
    num_train_epochs=20,
    logging_steps=50,
    learning_rate=5e-5,
    remove_unused_columns=False,
    push_to_hub=True,
    hub_private_repo=True
)

# Train the model
trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=processed_dataset,
)

trainer.train()
processor.push_to_hub(OUTPUT_DIR, private=True)
trainer.push_to_hub()

When I run processor.push_to_hub, I get the following error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/user/.pyenv/versions/vqa-train/lib/python3.10/site-packages/transformers/utils/hub.py", line 938, in push_to_hub
    model_card = create_and_tag_model_card(
  File "/home/user/.pyenv/versions/vqa-train/lib/python3.10/site-packages/transformers/utils/hub.py", line 1193, in create_and_tag_model_card
    model_card = ModelCard.load(repo_id, token=token, ignore_metadata_errors=ignore_metadata_errors)
  File "/home/user/.pyenv/versions/vqa-train/lib/python3.10/site-packages/huggingface_hub/repocard.py", line 189, in load
    with card_path.open(mode="r", newline="", encoding="utf-8") as f:
  File "/home/user/.pyenv/versions/3.10.12/lib/python3.10/pathlib.py", line 1119, in open
    return self._accessor.open(self, mode, buffering, encoding, errors,
IsADirectoryError: [Errno 21] Is a directory: 'muhammadfhadli/vilt-b32-finetuned-vqa-hse-v0.0.1'

I tried setting use_temp_dir=True and use_temp_dir=False, but it still didn’t work. It only worked after I manually deleted the directory muhammadfhadli/vilt-b32-finetuned-vqa-hse-v0.0.1 before running processor.push_to_hub(OUTPUT_DIR, private=True).

I’m trying similar answers from this GitHub issue, but none of them seem to work for me.

I would appreciate any insights or suggestions on how to resolve this issue.

Thanks in advance!

[Alanturner2]

How about using upload instead using push_to_hub().

from huggingface_hub import login
login(token="Your hf token")
tokenizer.save_pretrained("./your path")

from huggingface_hub import HfApi, upload_folder, create_repo

# Replace with your Hugging Face access token and desired model name
hf_token = "Your hf token"
repo_name = "Your repository"

# Step 1: Create a repository on the Hub. If you have the repo don't run this code
create_repo(repo_name, token=hf_token)

# Step 2: Upload the folder to the repository
upload_folder(
    folder_path="./outputmodel",    # Path to the model folder
    repo_id=repo_name,          # Name of the repository
    token=hf_token,             # Your Hugging Face token
    commit_message="Initial model upload",
)

[muhammadfhadli]

Thank you for your answer.
Yes, your workaround worked. But may I know why my approach is not working and what the best practice is?

[Alanturner2]

I think there is a problem in your huggingface access token.
How about this answer?

remote: Repository not found

Hey @aqpeeb ! I think you should check your authentication with huggingface. Authenticate with Hugging Face: If authentication is required, make sure you’re logged in to Hugging Face via the CLI: bash CopyEdit huggingface-cli login Enter your token when prompted. If the Space belongs to an organization, ensure you have the proper permissions. And then you should check your huggingface api token has the authorization to use the organization or repository. [Capture] Hope this help!

[muhammadfhadli]

Thanks, but I’m afraid token is not the problem. Even if I remove processor.push_to_hub(OUTPUT_DIR, private=True), the line trainer.push_to_hub() still works correctly to push the model only

[John6666]

OUTPUT_DIR = ‘muhammadfhadli/vilt-b32-finetuned-vqa-hse-v0.0.1’

I think this is suspicious. It looks like an error where the huggingface_hub library is trying to process the directory ‘muhammadfhadli/vilt-b32-finetuned-vqa-hse-v0.0.1’ as if it were an HF repo. For example, it might work if you change it to ‘vilt-b32-finetuned-vqa-hse-v0.0.1’.
Alternatively, I think it is also possible that an error will occur if there is a relative directory with the same name as the repo.

[muhammadfhadli]

Yes, but this only happens with the line processor.push_to_hub(OUTPUT_DIR, private=True). If I comment out this line, I can still push the model only using trainer.push_to_hub(), but not with the processor.

[John6666]

So there is a possibility that there is a bug in this function in the version of Transformers you are using. The reason why there is no problem with Trainer is probably because Trainer uses a redefined function, not an inherited one.

src/transformers/utils/hub.py

src/transformers/trainer.py

[muhammadfhadli]

I see… okay, that makes sense. For now, using huggingface_hub.upload_folder is a sufficient workaround for this. Thank you for your answer