Model_accepts_loss_kwargs detection based on **kwargs is too permissive

🤗Transformers
1

In trainer.py (code) , the model_accepts_loss_kwargs flag is automatically set to True if model.forward includes **kwargs. However, **kwargs might be used for reasons unrelated to loss_kwargs (e.g., for flexibility or additional non-loss params).

This behavior may lead to unintended side effects.

        forward_params = inspect.signature(model_forward).parameters

        # Check if the model has explicit setup for loss kwargs,
        # if not, check if `**kwargs` are in model.forward
        if hasattr(model, "accepts_loss_kwargs"):
            self.model_accepts_loss_kwargs = model.accepts_loss_kwargs
        else:
            self.model_accepts_loss_kwargs = any(
                k.kind == inspect.Parameter.VAR_KEYWORD for k in forward_params.values()
            )
1 Like