The result of the tokenization is not in the "src" field, but in 'input_ids' and 'attention_mask'. The 'src' field is just the same as before.
2 Likes
The result of the tokenization is not in the "src" field, but in 'input_ids' and 'attention_mask'. The 'src' field is just the same as before.