Why does tokenizer.apply_chat_template() add multiple eos tokens?

@Chahnwoo I think you are on the right lines. If I wanted to train certain behaviour into a model using special tokens and markers I would definitely use a new token rather than reusing and old one.

We for sure want to leverage the base models language understanding for the instruction tuning. Almost as an additive task that relies on the deeper knowledge in the pretraining.