Hi,
I’m trying to train a model with a VERY SMALL dataset and a batch size : 1. And yet I still have an out of memory error on a L40S space. I just don’t get it. Perf works fine. Here are my params : {‘model’: ‘mistralai/Mistral-7B-Instruct-v0.3’, ‘project_name’: ‘autotrain-ymskk-6fafk’, ‘data_path’: ‘Gragroo/sanitized_intent_messages_sampled’, ‘train_split’: ‘train’, ‘valid_split’: None, ‘add_eos_token’: True, ‘block_size’: 1024, ‘model_max_length’: 2048, ‘padding’: ‘right’, ‘trainer’: ‘sft’, ‘use_flash_attention_2’: False, ‘log’: ‘tensorboard’, ‘disable_gradient_checkpointing’: False, ‘logging_steps’: -1, ‘eval_strategy’: ‘epoch’, ‘save_total_limit’: 1, ‘auto_find_batch_size’: False, ‘mixed_precision’: ‘bf16’, ‘lr’: 3e-05, ‘epochs’: 3, ‘batch_size’: 1, ‘warmup_ratio’: 0.1, ‘gradient_accumulation’: 4, ‘optimizer’: ‘adamw_bnb_8bit’, ‘scheduler’: ‘linear’, ‘weight_decay’: 0.0, ‘max_grad_norm’: 1.0, ‘seed’: 42, ‘chat_template’: ‘zephyr’, ‘quantization’: ‘int4’, ‘target_modules’: ‘all-linear’, ‘merge_adapter’: False, ‘peft’: False, ‘lora_r’: 16, ‘lora_alpha’: 32, ‘lora_dropout’: 0.05, ‘model_ref’: None, ‘dpo_beta’: 0.1, ‘max_prompt_length’: 128, ‘max_completion_length’: None, ‘prompt_text_column’: ‘prompt’, ‘text_column’: ‘message’, ‘rejected_text_column’: ‘rejected_text’,
Does anyone could help me ?
Many thanks !
