Allow for next sequence token in a given tokenizer. Phi 1.5

🤗Transformers
1

phi 1.5 does not have a next sequence token for the below code snippet

tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-1_5", trust_remote_code=True)#, torch_dtype="auto")
tokenizer.pad_token = tokenizer.eos_token

sentences = ["lets go"]
tags = ["party"]
tokies = tokenizer(sentences, tags, padding='longest')

output: {'input_ids': [[5289, 467, 10608]], 'attention_mask': [[1, 1, 1]]}

as you can see there is no token separating the sentence “lets go” and “party”. Is there a way to easily achieve this functionality?