When you are generating responses, you can set the eos token to be "User: ". For example inference_config.eos_token_id = tokenizer("User: ")[“input_ids”]. One caveat is that "User: " might not only be the prefix, I would change it to "###User: " during finetuning.