Hi, I’m confused about the token id in tokenizer and model.config.
Take facebook/incoder as an example, I found that model.config.bos_token_id is 0, which is not equal to tokenizer.bos_token_id, 2. I don’t whether there is any problem.
Hi, I’m confused about the token id in tokenizer and model.config.
Take facebook/incoder as an example, I found that model.config.bos_token_id is 0, which is not equal to tokenizer.bos_token_id, 2. I don’t whether there is any problem.