I’m a Beginner in Fine-tuning model. I follow step by step some video on Youtube. But when i try to autotrain the Model LLama2 i catch this Error
Autotrain code : !autotrain llm --train --project_name [Name] --model TinyPixel/Llama-2-7B-bf16-sharded --data_path [MyPath] --use_peft --use_int4 --learning_rate 2e-4 --train_batch_size 2 --num_train_epochs 3 --trainer sft --model_max_length 2048 --push_to_hub --repo_id [MyPath] --token [mytoken] --block_size 2048
Please help me!
Hello Nghengu123
I am not a pro myself but struggling with some issues also.
But as I see it seems a dataset formatting issue.
Try using a stripped version of the original dataset to see if it´s working on your model.
As I do remember you must create a guanaco format dataset.
1 Like
Can you help me, what is Guanaco format ? It’s new for me , i dont have any knowledge about this format ?.?. How can i strip dataset. Additionally, what column do you think cause to that issue.
Here is column in dataset
