I’m trying to fine-tune a language model for dataset like
azizshaw/text_to_json · Datasets at Hugging Face where I will have only two fields input and output where output will be a JSON format. I trained a t5-small and t5-base but it did not give me desired results. Which models should I try for this use case?
One possible problem with that dataset is that the example outputs all use single quote delimiters, which is not valid JSON.
Hi,
I do have tutorial notebooks to fine-tune multimodal models on image->JSON use cases, might be helpful (on the CORD dataset). Created the same one for different models:
- PaliGemma: https://github.com/NielsRogge/Transformers-Tutorials/tree/master/paligemma
- LLaVa: https://github.com/NielsRogge/Transformers-Tutorials/tree/master/llava
- Idefics2: Transformers-Tutorials/Idefics2 at master · NielsRogge/Transformers-Tutorials · GitHub.
Should be the same for LLMs.