Script to Fine-Tune FLAN UL2

griffin · April 10, 2023, 5:19pm

Hi - Does anyone know of any open source code to fine-tune FLAN UL2 on a custom dataset?

I’m looking for something that will work on consumer grade GPUs (24GB). Likely 8-bit quantized or bf16.

Thanks,
Griffin

cyt79 · April 27, 2023, 3:12pm

Hi @griffin , I am looking for the same. Were you able to find something?

Topic		Replies	Views
Finetuning Stable Diffusion inpainting checkpoint Models	1	661	July 20, 2023
Running inference on flan-ul2 on multi-gpu 🤗Accelerate	8	4154	June 6, 2023
Any one have an idea on how large should the dataset to be to fine-tune BLIP2 model? Models	0	105	November 16, 2024
Any good code/tutorial that is shows how to do inference with Llama 2 70b on multiple GPUs with accelerate? 🤗Accelerate	1	2283	November 27, 2023
What GPU should I use to fine tune LongT5? Beginners	0	300	January 27, 2023