T5 Finetuning Tips

Models
33

I have a question about sample_weights. Typically, you can pass in the sample_weights as the third element of a tuple when constructing the Tensorflow Dataset (Training and evaluation with the built-in methods  |  TensorFlow Core). However, for the class T5ForConditionalGeneration, the call method (which I assume is what is called on the model is called) only takes the following parameters:

   def call(
    self,
    input_ids=None,
    attention_mask=None,
    decoder_input_ids=None,
    decoder_attention_mask=None,
    head_mask=None,
    decoder_head_mask=None,
    encoder_outputs=None,
    past_key_values=None,
    inputs_embeds=None,
    decoder_inputs_embeds=None,
    labels=None,
    use_cache=None,
    output_attentions=None,
    output_hidden_states=None,
    return_dict=None,
    training=False,
    **kwargs,
):

Source: transformers.models.t5.modeling_tf_t5 — transformers 4.7.0 documentation

I don’t see a way for T5 to consider the sample weights. How do I pass in the sample weights to a T5ForConditionalGeneration model? Thanks!

1 Like