Is there a tool to train/improve LLM’s, similar to SD-Scripts for training stable diffusion models?
I want a model to caption digital art. LLAVA (what i found and can use in ComfyUI) is decent but limited and lacks the scary amount of detail of WD14, but WD14 only does tags and not natural language. CLIP, BLIP, GIT simply isn’t good enough, especially for digital art, it takes more time having to fix the caption then just captioning manually.
Basically looking for WD14 but in natural language.
If there already is a LLAVA model that has been trained for accurate captioning for digital art then i can try that, but i couldn’t find anything when i was searching.
If i’m going to have to train one myself, how would i go about doing that locally on my own machine? (im going to get the RTX 5090 once it releases) I’m not looking to spend the next 10 years learning python and AI. Just captioning the images for the training will be enough of a mountain of a task.
To save all of us time: ELI15 and i don’t know what programming is, and don’t know how to use a terminal since i’m on windows.
I’m an inferencing specialist so i don’t have knowledge inthe diffusion area in terms of code. But honestly the best advice i can give would be to spend perhaps 6 hours doing a basic python course, or just go through the turoials on here. You don’t really need to know coding to fine-tune models, but you need to know what all the different variables (weights etc) do.
The script to tune a language is normally about the A4 equivalent of a quarter of a page. Since the only bits you are interested in the variables, the python involved is as close to natural language you can get.
If you weant to adjust the epochs, just change the number next to epocs, same with weights etc.
I know its probably not what you want to hear, but you can learn enough python ‘to understand’ what the scripts doing after a 6 hour course on coursera, but if you don’t want to do that you won’t find any finer tutorials for beginners than on here.
If life’s too hectic for that, which it can be for us all, probably the next best thing to the tools on here for ‘no-code’ model tuning is GPT4all.
Also double check the 5090 has the RAM for the model size etc, I have a 4090 STRIX but I use a PNY NVIDIA RTX A6000 48gb (its basically two 3090’s, of 1/4 of an A100) for most training and tuning. The 4090 is much faster, but the RAM is crucial for anything beyond an 8B model.