T5 fine-tuning for custom output

Beginners
1

Hi,
I am trying to fine-tune a model to extract custom data from a sentence. The data looks like this:

text target
We are open from 8 am to 6 pm 8-6
The store working hours are 8-8 8-8
Some random text none

I tried to fine-tune the T5 base to do this task but the generated text didn’t follow the custom dataset.

Steps I followed:

  1. Load T5ForConditionalGeneration and T5Tokenizer.
  2. Load my custom dataset.
  3. Tokenize the text data with a prefix for tokenizer “extract working hours:”.
  4. Train using Trainer.
  5. Save the model locally.
  6. Load model.
  7. Apply transformers pipeline to generate text.

I also tried to use a custom function to tokenize the input and decode output instead of using the pipeline, which didn’t generate the expected output.

Next, I copied Fine Tuning T5 Transformer Model with PyTorch colab notebook and replaced the dataset with mine. The generated text wasn’t correct either.

Is T5 the right model for what I am trying to achieve? If not, which model would you recommend for this task?

Thank you!