I want to train llama for image classification. That is, so that I can upload a photo to the model, and it can give me the class to which it belongs. I’ve heard that you need to use a jsonl file like this
[{“content”: “hello”, “role”: “user”}, {“content”: “hi nice to meet you”, “role”: “assistant”}]
[{“content”: “how are you”, “role”: “user”}, {“content”: “I am fine”, “role”: “assistant”}]
[{“content”: “What is your name?”, “role”: “user”}, {“content”: “My name is Mary”, “role”: “assistant”}]
[{“content”: “Which is the best programming language?”, “role”: “user”}, {“content”: “Python”, “role”: “assistant”}]
But if I start training, for example, literally with this file, I get
“Error: 500
Please check the logs for more information.”
But why doesn’t it work if I take this as a dataset?
[{“content”: “hello”, “role”: “user”}, {“content”: “hi nice to meet you”, “role”: “assistant”}]
[{“content”: “how are you”, “role”: “user”}, {“content”: “I am fine”, “role”: “assistant”}]
[{“content”: “What is your name?”, “role”: “user”}, {“content”: “My name is Mary”, “role”: “assistant”}]
[{“content”: “Which is the best programming language?”, “role”: “user”}, {“content”: “Python”, “role”: “assistant”}]
there are multiple things which seem wrong here. there is no column mapping, no dataset, wrong model selected for the task you described, etc.
first of all, you need to define what you need to do. you mentioned you want to build a model that classifies images in different categories. that is image classification task. but you have selected llm finetuning.
once you have decided the task you want to train on, you need a dataset. you can choose one from hf hub or create your own. after that, you select appropriate model and parameters.
i recommend you taking a look at docs which includes walkthroughs and extensive examples for different datasets for different tasks. here are the docs: AutoTrain
Thanks, it really works, but i was told to try to train LLM (llama), not models for classification. I have a jsonl dataset that contains rows like
[{“role”: “system”, “content”: “You are an assistant that classifies images”}, {“role”: “user”, “content”: “Is it cat or dog?”}, {“role”: “user”, “content”: [{“type”: “image_url”, “image_url”: {“url”: “data:image/jpeg;base64,/9j/4AAQSkZJRgADQztLW2t7i5usL…”}}]}, {“role”: “assistant”, “content”: “dog”}]
here images are converted to base64.
I read that lama should be trained using jsonl files of this structure, but it doesn’t work.
I checked that the problem is not in the pictures. Even a jsonl file without pictures consisting of these examples
[{“content”: “hello”, “role”: “user”}, {“content”: “hi nice to meet you”, “role”: “assistant”}]
[{“content”: “how are you”, “role”: “user”}, {“content”: “I am fine”, “role”: “assistant”}]
[{“content”: “What is your name?”, “role”: “user”}, {“content”: “My name is Mary”, “role”: “assistant”}]
[{“content”: “Which is the best programming language?”, “role”: “user”}, {“content”: “Python”, “role”: “assistant”}]
doesn’t work
