[Nov 16th Event] Matthew Carrigan: New TensorFlow Features for 🤗 Transformers and Datasets

Course
1

Use this topic to ask your questions to Matthew Carrigan during his talk: New TensorFlow Features for :hugs: Transformers and :hugs: Datasets

You can watch it on YouTube or on Twitch at 8:30am PST

2

Could the notebook shown in the video be linked? :slight_smile:

3

Sure thing, here’s a Colab link!

1 Like
4

For something like a zero-shot model, how does TFAutomodelforSequenceClassification change?

5

For something like a zero-shot model, how does TFAutomodelforSequenceClassification change?

This is answered at 50:06 in the main stream.

1 Like
6

I understand that padding enables batching data points together, but too many padding tokens make the computations very slow (?). What is the attention_mask for? If I understand correctly, it masks the padding tokens so that the model does not pay attention to it - but do padding tokens then still slow down the training if they are masked? I haven’t fully understood the purpose of padding and attention_masks and the impact on speed.

7

This is answered at 55:00 on the main stream.

8

Hi all, just noticed the Colab notebook didn’t have permissions set. It should be accessible now!