I am a student currently working on training the LLAMA-4-Scout-17B-16E-Instruct model using LoRA, running on an H100 GPU with 80GB VRAM (on Lambda Labs). However, I have encountered an out of memory error during the training process. I understand that this might fall slightly outside the scope of th…

It looks like you’re running into a CUDA out of memory issue while fine-tuning LLAMA-4-Scout-17B-16E-Instruct using LoRA on an H100 GPU with 80GB VRAM. Even though 80GB is a lot, large models like this can still exceed memory limits, especially with high batch sizes and gradient accumulation steps. …

Inquiry Regarding Out of Memory Issue During LoRA Fine-Tuning

Models

system Closed May 9, 2025, 3:08pm 3

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Fine Tuning LLama 3.2 1B Quantized Memory Requirements Models	6	1329	June 16, 2025
Memory requierements Models	2	331	February 18, 2025
Cuda Out of Memory when fine tuning llm model 🤗Transformers	3	1153	May 7, 2024
Getting CUDA out of memory when calling save_pretrained in a script that tries lora training a large language model Beginners	3	1815	November 9, 2023
Running into cuda out of memory when running llama2-13b-chat model on multi-gpu machine Intermediate	5	11019	December 21, 2023

Inquiry Regarding Out of Memory Issue During LoRA Fine-Tuning

Related topics