How to apply training ClassLabels on test / validation Dataset

I am trying to create a custom dataset using flat files. The data elements are 'Tokens" and ‘NER Tags’ strings. I am able to create the ClassLabels for the Training dataset and now want to apply the same ClassLabels on the Test & Validation datasets. I have tried several approaches, but none have worked. Any help or insight will be greatly appreciated. The code segment is below :

def convert_to_featured_dataset(data):
formatted_data = {“tokens”: , “ner_tags”: }
features = Features(
{“tokens”: Value(dtype=“string”), “ner_tags”: Value(dtype=“string”)}
for sentence in data:
tokens = [token_data[1] for token_data in sentence]
ner_tags = [token_data[0] for token_data in sentence]


dataset = Dataset.from_dict(formatted_data, features=features)
return dataset

train_dataset = convert_to_featured_dataset(train_data)
train_dataset = train_dataset.class_encode_column(“ner_tags”)
label_feature = train_dataset.features[“ner_tags”]
test_features = train_dataset.features.copy()

#Does not work
def convert_ids(rec):
#Error : ValueError: Invalid string class label [‘O’, ‘O’, ‘B-RATINGS_AVERAGE’, ‘I-RATINGS_AVERAGE’, …']
rec[“ner_tags”] = label_feature.str2int(rec[“ner_tags”])
return rec

test_dataset = convert_to_featured_dataset(test_data)
# Did not work : ValueError: Invalid string class label ['O'...
#test_dataset =, batched=True)
#test_features["ner_tags"] = ClassLabel(label_feature.feature.names)
#test_dataset = test_dataset.cast(test_features)
 test_dataset = test_dataset.cast_column('ner_tags',label_feature)

Thank you for your time and help.

Tried encoding the ner_tags separately and then casting them to ClassLabel but got stuck on another point

# Failed  :  may have worked if ner_tags feature was like ['1','2','3']  but it is like [['1','2','3'] ,['1','2','3'] ]  
# Error : ValueError: Invalid string class label ['24', '24', '24', '0', '12']
train_dataset = train_dataset.cast_column(
    "ner_tags", ClassLabel(names=id_label_list)

You can pass Sequence(ClassLabel(...)) to .cast_column to cast a list of labels.