Great detective work @seanbenhur ! I think that adding the BOS token is the cause of the problem and you can resize the embedding layer as follows:
model.resize_token_embeddings(len(tokenizer))
Does resizing the embedding layer solve the problem?
1 Like