RemoveColumnsCollator is removing all keys from variable “features”. No matter what model I choose or which dataset I choose. The input to the function transformers.trainer_utils._remove_columns is a tokenized string which is a dictionary with “input_ids” and “labels” as keys. But output is always None as it removes any key whose name is not in signature columns ( which is always [ args, kwargs, label, label_ids ] ) . This is resulting in null input to padding function which throws error.
def __call__(self, features: List[dict]):
#features = [self._remove_columns(feature) for feature in features]
return self.data_collator(features)
I had to comment this line but I do not know the implications. It is ok to do so? And if this is the expected behavior of this function?