Get warning "Could not estimate the number of tokens of the input, floating-point operations will not be computed" when use a customize Trainer and customize data collator

[mzyMMMM]

I want to use triplet loss to fine tune my Bert model, so I define a customize Trainer like that:

class TripletLossTrainer(Trainer):
        def compute_loss(self, model, inputs, return_outputs=False):
            anchor_data, positive_data, negative_data = inputs.values()
            anchor_output = self.use_avg_2(anchor_data, model)
            positive_output = self.use_avg_2(positive_data, model)
            negative_output = self.use_avg_2(negative_data, model)
            triplet_loss = nn.TripletMarginWithDistanceLoss(distance_function=lambda x, y: 1.0 - 
            F.cosine_similarity(x, y),margin=0.5)
            # compute custom loss
            loss = triplet_loss(anchor_output, positive_output, negative_output)
            return loss

where anchor_data, positive_data, negative_data are tokenized result with type <class ‘transformers.tokenization_utils_base.BatchEncoding’>, and the use _avg_2 function is for computing the average value for the last hidden state for each data with a Bert model.
When I do the training process, it will keep reporting warning like
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed
Could not estimate the number of tokens of the input, floating-point operations will not be computed

I wonder what does the floating-point operations mean? will it influence the performance of my training? Thanks!

[sgugger]

You can completely ignore that warning, it’s just to compute the total FLOPS done during your training.

[mzyMMMM]

Thanks a lot! Is there any method to stop outputting these warn information during the training?

[sgugger]

No, but in the latest version you should see it only once.

[Reza0247]

Hi, I have the same problem when using the framework of huggingface to compute a contrastive loss. In my case, this warning information is repeated in the console which really makes me a big headache.

The version of my huggingface is 4.17.0. So how can I stop this information?

[jxm]

Me too, on 4.23.1: