FASTAI:TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType)

🤗Transformers
1

Hi,

I’m working on developing a model using FastAI to classify images into two categories: Label 1 and Label 2. I’d appreciate any help in debugging this.

This is my code
blocks = (ImageBlock,CategoryBlock,CategoryBlock)

splitter = RandomSplitter(valid_pct=0.2, seed=42)

block = DataBlock(blocks = blocks,
n_inp=1,
get_x=ColReader(‘image_path’),
get_y=[ColReader(‘label1’),ColReader(‘label2’)],
splitter = splitter,
item_tfms = item_tfms,
batch_tfms = batch_tfms)

dls = block.dataloaders(train_df2, bs=8)
dls.show_batch(figsize=(12,12))

roc = RocAuc()

learn = vision_learner(dls,“vit_base_patch16_384”, loss_func=CrossEntropyLossFlat(),metrics=[accuracy, error_rate])

f1_macro = F1Score(average=‘macro’)
learn = vision_learner(dls,
“vit_base_patch16_384”,
pretrained=True,
loss_func=CrossEntropyLossFlat(),
metrics=[accuracy,f1_macro],
# opt_func=optim.Adam,
# moms=(0.95, 0.85, 0.95),
ps=0.5,
path = “./vit_base_patch16_384”
)

learn.to(device=torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’))

learn.to_fp16()

The erro I’m getting:

TypeError Traceback (most recent call last)
in <cell line: 4>()
2 # learn = vision_learner(dls,“vit_base_patch16_384”, loss_func=CrossEntropyLossFlat(),metrics=[accuracy, error_rate])
3 f1_macro = F1Score(average=‘macro’)
----> 4 learn = vision_learner(dls,
5 “vit_base_patch16_384”,
6 pretrained=True,

/usr/local/lib/python3.10/dist-packages/fastai/vision/learner.py in vision_learner(dls, arch, normalize, n_out, pretrained, weights, loss_func, opt_func, lr, splitter, cbs, metrics, path, model_dir, wd, wd_bn_bias, train_bn, moms, cut, init, custom_head, concat_pool, pool, lin_ftrs, ps, first_bn, bn_final, lin_first, y_range, **kwargs)
232 n_in = kwargs[‘n_in’] if ‘n_in’ in kwargs else 3
233 if isinstance(arch, str):
→ 234 model,cfg = create_timm_model(arch, n_out, default_split, pretrained, **model_args)
235 if normalize: _timm_norm(dls, cfg, pretrained, n_in)
236 else:

/usr/local/lib/python3.10/dist-packages/fastai/vision/learner.py in create_timm_model(arch, n_out, cut, pretrained, n_in, init, custom_head, concat_pool, pool, lin_ftrs, ps, first_bn, bn_final, lin_first, y_range, **kwargs)
194 body = TimmBody(model, pretrained, None, n_in)
195 nf = body.model.num_features
→ 196 res = add_head(body, nf, n_out, init=init, head=custom_head, concat_pool=concat_pool, pool=body.needs_pool,
197 lin_ftrs=lin_ftrs, ps=ps, first_bn=first_bn, bn_final=bn_final, lin_first=lin_first, y_range=y_range)
198 return res,model.default_cfg

/usr/local/lib/python3.10/dist-packages/fastai/vision/learner.py in add_head(body, nf, n_out, init, head, concat_pool, pool, lin_ftrs, ps, first_bn, bn_final, lin_first, y_range)
156 “Add a head to a vision body”
157 if head is None:
→ 158 head = create_head(nf, n_out, concat_pool=concat_pool, pool=pool,
159 lin_ftrs=lin_ftrs, ps=ps, first_bn=first_bn, bn_final=bn_final, lin_first=lin_first, y_range=y_range)
160 model = nn.Sequential(body, head)

/usr/local/lib/python3.10/dist-packages/fastai/vision/learner.py in create_head(nf, n_out, lin_ftrs, ps, pool, concat_pool, first_bn, bn_final, lin_first, y_range)
103 if lin_first: layers.append(nn.Dropout(ps.pop(0)))
104 for ni,no,bn,p,actn in zip(lin_ftrs[:-1], lin_ftrs[1:], bns, ps, actns):
→ 105 layers += LinBnDrop(ni, no, bn=bn, p=p, act=actn, lin_first=lin_first)
106 if lin_first: layers.append(nn.Linear(lin_ftrs[-2], n_out))
107 if bn_final: layers.append(nn.BatchNorm1d(lin_ftrs[-1], momentum=0.01))

/usr/local/lib/python3.10/dist-packages/fastai/layers.py in init(self, n_in, n_out, bn, p, act, lin_first)
180 layers = [BatchNorm(n_out if lin_first else n_in, ndim=1)] if bn else
181 if p != 0: layers.append(nn.Dropout(p))
→ 182 lin = [nn.Linear(n_in, n_out, bias=not bn)]
183 if act is not None: lin.append(act)
184 layers = lin+layers if lin_first else layers+lin

/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py in init(self, in_features, out_features, bias, device, dtype)
104 self.out_features = out_features
105 self.weight = Parameter(
→ 106 torch.empty((out_features, in_features), **factory_kwargs)
107 )
108 if bias:

TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType), but expected one of:

  • (tuple of ints size, *, tuple of names names, torch.memory_format memory_format = None, torch.dtype dtype = None, torch.layout layout = None, torch.device device = None, bool pin_memory = False, bool requires_grad = False)
  • (tuple of ints size, *, torch.memory_format memory_format = None, Tensor out = None, torch.dtype dtype = None, torch.layout layout = None, torch.device device = None, bool pin_memory = False, bool requires_grad = False)
1 Like
2

It’s not that rare a PyTorch error…

The following is what I heard from Hugging Chat. I don’t know about FastAI…

The TypeError you’re encountering is related to an invalid linear layer configuration, likely due to incorrect feature dimensions from the model body. Here’s how to address the issue:

  1. Single Label Setup: FastAI’s vision_learner expects a single label for classification. If you’re trying to classify into two categories (Label 1 and Label 2), ensure that you have one label column in your DataFrame indicating the two classes. For example:

    # Example DataFrame structure
train_df2 = pd.DataFrame({
    'image_path': [...] ,
    'label': [...]  # Values should be 'Label1' or 'Label2'
})

  2. Adjust the Model Setup: Use a single CategoryBlock for the label instead of two. Modify your DataBlock as follows:

    blocks = (ImageBlock, CategoryBlock)
splitter = RandomSplitter(valid_pct=0.2, seed=42)
block = DataBlock(
    blocks=blocks,
    n_inp=1,
    get_x=ColReader('image_path'),
    get_y=ColReader('label'),  # Single label column
    splitter=splitter,
    item_tfms=item_tfms,
    batch_tfms=batch_tfms
)

  3. Pass n_out Parameter: Ensure n_out is explicitly set when calling vision_learner to match the number of classes (2 in this case):

    learn = vision_learner(dls, "vit_base_patch16_384", n_out=2, 
                     pretrained=True, loss_func=CrossEntropyLossFlat(),
                     metrics=[accuracy, F1Score(average='macro')], 
                     ps=0.5, path="./vit_base_patch16_384")

By addressing these points, you should resolve the TypeError and properly train a model with two classification labels.

If you’re still encountering issues, consider checking the:

  • Data Preprocessing: Ensure image paths are correct and transformations are applied properly.
  • Device Handling: Verify CUDA is correctly initialized.
  • Model Architecture: Ensure compatibility with the selected backbone.

These adjustments should help fix the error and align your model with FastAI’s expected setup for image classification tasks.

Based on the analysis of the issue, the main problem arises from how the DataBlock and vision_learner were configured. Here’s how to resolve the issue:

  1. Single Label Configuration: Ensure the DataBlock uses a single CategoryBlock for the labels. This aligns with FastAI’s expectation of a single output for classification tasks. Modify the code as follows:

    blocks = (ImageBlock, CategoryBlock)
splitter = RandomSplitter(valid_pct=0.2, seed=42)
block = DataBlock(
    blocks=blocks,
    n_inp=1,
    get_x=ColReader('image_path'),
    get_y=ColReader('label'),  # Ensure 'label' is your single label column
    splitter=splitter,
    item_tfms=item_tfms,
    batch_tfms=batch_tfms
)

  2. Specify Output Classes: When initializing the learner, explicitly set the number of output classes to 2. This avoids ambiguity in model architecture creation:

    learn = vision_learner(dls, "vit_base_patch16_384", n_out=2,
                     pretrained=True, loss_func=CrossEntropyLossFlat(),
                     metrics=[accuracy, F1Score(average='macro')],
                     ps=0.5, path="./vit_base_patch16_384")

  3. Verify Data and Device Handling: Ensure that image paths are correct and that all transformations are properly applied. Also, confirm that CUDA is correctly initialized if using a GPU:

    learn.to(device=torch.device('cuda' if torch.cuda.is_available() else 'cpu'))

By implementing these changes, the error related to the linear layer configuration should be resolved, allowing the model to train properly.

3

Thank you!

1 Like