Pytorch-Image models

Models
1

In the VisionTransformer class, the default act_layer is None . If we do not provide it - this will lead to a TypeError in MLP because none of the classes (Block , MLP , or VisionTransformer ) handle this case. Obvious error message:
TypeError: ‘NoneType’ object is not callable

1 Like
2

Fix:
Always set act_layer to a valid activation function (e.g., nn.GELU, nn.ReLU) when instantiating VisionTransformer.
Example:

import torch.nn as nn
model = VisionTransformer(act_layer=nn.GELU)

If not set, you’ll get TypeError: ‘NoneType’ object is not callable.

Solution provided by Triskel Data Deterministic AI.

1 Like
3

Hello @mohitb1i ,

In which PyTorch version are you experiencing this error?

Machine Learning Engineer at RidgeRun.ai
Contact us: support@ridgerun.ai

1 Like
4

I understand, but I am saying the default value of act_layer should be nn.GELU or just set it in instantiation, like:

block_fn(
...
act_layer = act_layer or nn.GELU,
...
)
1 Like
5

No it is a vision-transformer code from hugging face,
original repo

code of Vision Transformer

1 Like