Code snippet : is based on example listed for jit tracing of a model :
enc = BertTokenizer.from_pretrained(“bert-base-uncased”)
Tokenizing input text
text = “[CLS] Who was Jim Henson ? [SEP] Jim Henson was a puppeteer [SEP]”
tokenized_text = enc.tokenize(text)
Masking one of the input tokens
masked_index = 8
tokenized_text[masked_index] = ‘[MASK]’
indexed_tokens = enc.convert_tokens_to_ids(tokenized_text)
segments_ids = [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1]
Creating a dummy input
tokens_tensor = torch.tensor([indexed_tokens])
segments_tensors = torch.tensor([segments_ids])
dummy_input = [tokens_tensor, segments_tensors]
Initializing the model with the torchscript flag
Flag set to True even though it is not necessary as this model does not have an LM Head.
config = BertConfig(vocab_size_or_config_json_file=32000, hidden_size=768,
num_hidden_layers=12, num_attention_heads=12, intermediate_size=3072, torchscript=True)
Instantiating the model
model = BertModel(config)
The model needs to be in evaluation mode
model.eval()
If you are instantiating the model with from_pretrained you can also easily set the TorchScript flag
model = BertModel.from_pretrained(“bert-base-uncased”, torchscript=True)
model(dummy_input)
Error I hit is :
File “/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py”, line 727, in _call_impl
result = self.forward(*input, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/transformers/models/bert/modeling_bert.py”, line 911, in forward
input_shape = input_ids.size()
AttributeError: ‘list’ object has no attribute ‘size’
could you help?