I am trying to get sentence embedding from pretrained Roberta. When I use the model on CPU and GPU, I get two different sentence embeddings. Following is the code snippet:
from transformers import RobertaConfig, RobertaModel, RobertaTokenizer
import torch
import numpy as np
device = ("cuda" if torch.cuda.is_available() else "cpu")
# Initializing tokenizer
tokenizer = RobertaTokenizer.from_pretrained("roberta-base")
# Initializing a RoBERTa configuration
configuration = RobertaConfig()
configuration.vocab_size = tokenizer.vocab_size
# Initializing a model from the configuration
model = RobertaModel(configuration)
model = model.to(device)
model = model.eval()
with torch.no_grad():
tokenized_task = tokenizer('random_sentence_check_v000', return_tensors="pt")
outputs = model(**tokenized_task.to(device))
embedding = outputs.pooler_output.squeeze(0).cpu().numpy().tolist()
I use colab for my experiments. Shouldn’t the embedding be the same as the weights are pretrained?