Using TableTransformer in Standalone Mode Without Hugging Face Hub Access

🤗Transformers
1

Title: Using TableTransformer in Standalone Mode Without Hugging Face Hub Access

Body:

Hi, I need help with using the Hugging Face transformers library, specifically with the TableTransformer model. Due to a network firewall, I cannot directly download models from the Hugging Face Hub (all requests are blocked), so I need to use it in standalone mode.

Environment Details:

  • Python Version: 3.12.3
  • Pip Version: 23.2.1
  • OS: Windows 11 (Version 23H2)
  • Transformers Version: 4.49.0

What I Tried

  1. Downloaded the model manually from Hugging Face and placed the table-transformer-detection model in a local models/ directory.

  2. Loaded the model using from_pretrained:

    from transformers import TableTransformerModel

model = TableTransformerModel.from_pretrained("./models/table-transformer-detection")

  3. Encountered an error: The model still attempts to download dependencies from the Hub:

    urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443):
Max retries exceeded with url: /timm/resnet18.a1_in1k/resolve/main/pytorch_model.bin

    Issue: The TableTransformer uses a ResNet backbone that it tries to fetch online.

Attempted Fixes

  • Manually downloaded the resnet-50 model and tried specifying it as the backbone:

    from transformers import TableTransformerConfig, TableTransformerModel, ResNetConfig

resnet_50_path = "./models/resnet-50"
table_detection_path = "./models/table-transformer-detection"

backbone_config = ResNetConfig.from_pretrained(resnet_50_path)
config = TableTransformerConfig(backbone_config=backbone_config)

model = TableTransformerModel(config).from_pretrained(table_detection_path)

    Error:

    ValueError: You can't specify both `backbone` and `backbone_config`.

  • Tried setting backbone=None to resolve the conflict:

    config = TableTransformerConfig(backbone_config=backbone_config, backbone=None)

    Error:

    ValueError: You can't specify both `backbone_config` and `use_timm_backbone`.

  • Adjusted settings to disable pretrained backbones:

    import os

os.environ["HF_HUB_OFFLINE"] = "1"
os.environ["TRANSFORMERS_OFFLINE"] = "1"

from transformers import ResNetConfig, TableTransformerConfig, TableTransformerForObjectDetection, AutoImageProcessor

backbone_resnet50_path = "./models/resnet-50"

backbone_resnet50_config = ResNetConfig.from_pretrained(backbone_resnet50_path)
backbone_resnet50_config.name_or_path=backbone_resnet50_path

print("backbone resnet50 config:", backbone_resnet50_config)

table_config = TableTransformerConfig(
  backbone_config=backbone_resnet50_config,
  backbone=None,
  use_timm_backbone=False,
  use_pretrained_backbone=False
)

table_model_table = "./models/table-transformer-detection"

processor = AutoImageProcessor.from_pretrained(table_model_table)
table_model = TableTransformerForObjectDetection(table_config).from_pretrained(
  pretrained_model_name_or_path=table_model_table,
  config=table_config,
  ignore_mismatched_sizes=True
)

    Final Error: The model initializes, but I receive a warning about mismatched shapes in the checkpoint and uninitialized layers:

    You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

Some weights of TableTransformerForObjectDetection were not initialized from the model checkpoint at ./models/table-transformer-detection and are newly initialized because the shapes did not match:

- model.input_projection.weight: found shape torch.Size([256, 512, 1, 1]) in the checkpoint and torch.Size([256, 2048, 1, 1]) in the model instantiated

- model.query_position_embeddings.weight: found shape torch.Size([15, 256]) in the checkpoint and torch.Size([100, 256]) in the model instantiated

You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

Summary of My Findings

After extensive testing and troubleshooting, I have managed to prevent the model from attempting to download files from the Hugging Face Hub. However, I suspect that my custom configuration has broken several things, as I now face shape mismatches and potentially other issues due to the manual setup.

My Question

  • How does the TableTransformer backbone configuration actually work?
  • How can I load the TableTransformer model entirely offline, ensuring all dependencies (like ResNet) are resolved locally?
  • Is this even possible with the current implementation of transformers, or does the library inherently require online access?

Would really appreciate any guidance from someone familiar with Hugging Face’s model loading mechanisms!

Thanks in advance!
Enzo

1 Like
2

How can I load the TableTransformer model entirely offline, ensuring all dependencies (like ResNet) are resolved locally?

Yes. Especially with local_files_only=True.

Is this even possible with the current implementation of transformers, or does the library inherently require online access?

Transformers and Diffusers only require an internet connection when loading models.

Or perhaps you can simply like this?:

pip install timm

References