Asking to pad but the tokenizer does not have a padding token

🤗Tokenizers
1

tokenizer = AutoTokenizer.from_pretrained(
base_model_id,
padding=True,
padding_side=“left”,
pad_token=“[PAD]”,
add_eos_token=True,
add_bos_token=True,
)

Set the padding token in the tokenizer

tokenizer.pad_token = tokenizer.eos_token

Now initialize the trainer

trainer = SetFitTrainer(
model=model,
train_dataset=tokenized_train_dataset,
eval_dataset=tokenized_val_dataset,
loss_class=CosineSimilarityLoss,
batch_size=16,
num_iterations=20, # The number of text pairs to generate for contrastive learning
column_mapping={“text”: “text”, “label”: “label”} # Map dataset columns to text/label expected by trainer
)

ValueError: Asking to pad but the tokenizer does not have a padding token. Please select a token to use as pad_token (tokenizer.pad_token = tokenizer.eos_token e.g.) or add a new pad token via tokenizer.add_special_tokens({'pad_token': '[PAD]'}).