I’m getting this error message when I try to use my fine tuned ELECTRA model to classify a batch of sentences, but not when I’m using it to just classify a single sentence.
Here’s the chunk in which I get the error:
# FRACTION of dataset
test = congress_113_tweets.head(10)
test.head(1)
# LIST of text variable dataset
list_text = test['text'].tolist()
def run_model(list_text):
# tokenizing comment ^
encoding = tokenizer.encode_plus(
test_comment,
add_special_tokens=True,
max_length=512,
return_token_type_ids=False,
padding="max_length",
return_attention_mask=True,
return_tensors='pt',
)
# returning probability values for each label
_, test_prediction = trained_model(encoding["input_ids"], encoding["attention_mask"])
return test_prediction.flatten().numpy()
#Then, apply the function to each row:
test[LABEL_COLUMNS] = test[['text']].apply(run_model, axis=1, result_type='expand')
My dataset looks like this:
ID text
1 People are now using @metronorth like a subway...
2 Spent morning on @metronorth issues with Rep. ...
3 Step dancing, boiled meat, and beer at the Hib...
4 What a reception for #Team26 in Greenwich! htt...
Here’s a link to a colab containing all the code: Google Colab