Finetune BERT for multiple tasks

🤗Transformers
1

I’m fine-tuning a BERT model to perform a rather complex task. It is supposed identify targets within the input string, perform regression on each of the targets and output

  1. the start/end index of each identified target in the input
  2. the regression values for each target

in a JSON format.

From my understanding, the training itself is pretty simple but to get the output in the desired format (multiple outputs based on predicted targets in a JSON format), I need to customize the model’s head.

So, i’m not really looking for a solution here but rather resources (tutorials, papers, videos) about how to do that. Thank you!

1 Like
2

hi @mudi99
I don’t know if it’s exactly what you want but you can check it GitHub - 1rgs/jsonformer: A Bulletproof Way to Generate Structured JSON from Language Models.