Hi,
After loading the model through
config = BertConfig()
model = BertModel(config)
I try to change the hidden_size
parameter in the configuration of the Bert model as
model.config.hidden_size = 800
The default hidden_size of Bert is 768.
Now, after loading some inputs to the model, I am still getting a hidden_size value of 768, Why?
inputs = torch.tensor([[ 101, 7592, 999, 102],
[ 101, 4658, 1012, 102],
[ 101, 3835, 999, 102]])
output = model(inputs)
Now, when I do
output.last_hidden_state.shape
it prints torch.Size([3, 4, 768])
Can somebody please explain why it’s still using 768 as its hidden_size?
Thanks.