Comparing output of BERT model - why do two runs differ even with fixed seed?

Beginners
1

I have example code as below. If I instantiate two models as below and compare the outputs, I see different outputs. wondering why would this be the case?

– code snippet –
# fix seed
torch.manual_seed(10)
tokenizer = BertTokenizer.from_pretrained(“bert-base-uncased”)

    config = BertConfig(vocab_size_or_config_json_file=32000, hidden_size=768,
                        num_hidden_layers=2, num_attention_heads=2, intermediate_size=3072, torchscript=True)

    # Instantiating the model
    model = BertModel(config)
    model.eval()
    model2 = BertModel(config)
    model2.eval()

    # inputs to model
    sequence = ["A Titan RTX has 24GB of VRAM"]
    inputs = tokenizer.prepare_seq2seq_batch(sequence, return_tensors='pt')
    input_ids = inputs["input_ids"]

    o1 = model(input_ids)
    o2 = model2(input_ids)
   # why do o1 and o2 differ?

– code snippet –

2

resolved :
used pretrained to match this.

3

can you share your updated code?