How to create a half precision pretrained model

Hi, I am using roberta-base model to finetune my dataset. I want to create a fp16 parameter model. How could I do it?

from transformers import AutoModelForSequenceClassification
import torch.nn as nn
model = AutoModelForSequenceClassification.from_pretrained("roberta-base")
1 Like