AttributeError: 'NoneType' object has no attribute 'dtype'


I’ve been trying to fine-tune a german gpt-2 Modell with a dataset from huggingface via TensorFlow.
When trying to start the Training, I get the following Error: AttributeError: ‘NoneType’ object has no attribute ‘dtype’.

You can find the code on this Google Colab Notebook: Google Colab

Any help would be appreciated and thanks for reading!

1 Like

Same boat here. If it helps, I was getting my error at loss and so I printed it and got None. This is my case. Perhaps it’s not computing for some reason.

`use_cache = True` is incompatible with gradient checkpointing`. Setting `use_cache = False`...
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_3879877/ in <cell line: 7>()
     14         loss = outputs.loss
     15         print(loss)
---> 16         loss.backward()
     18         optimizer.step()

AttributeError: 'NoneType' object has no attribute 'backward'

Sometimes, this happens in the case that the function does not return anything. You can try to check whether forget to return something, such as return loss

1 Like

Hm, yes. In my case, there was no 'label' column so there was no output, and thus no loss. I’m wondering if these features (label, target…) are task-dependent and why they would be needed for generation tasks, as supposed to classification?

Generation task training also needs the label as generation task training is supervised learning here.

1 Like

I see. Thank you! So when training, would that be responsible in part of my dataset (to have “labels”) or the tokenizer to create a column called [‘labels’] alongside the others (attn_mask…)?