Facing a problem whilst trying to finetune the ByT5 model for the text classification task.
I’ve tried to use this notebook exploring-T5/t5_fine_tuning.ipynb at master · patil-suraj/exploring-T5 · GitHub by @valhalla for ByT5 finetuning on the text classification task. When I start from ‘google/byt5-small’ I get really strange results. A model always generates negative sentiment label (‘n’ in my code)
When I switch to t5-small pretrained checkpoint the results are reasonable
What am I missing here? Any piece of advice would be much appreciated!