Fine-tune a 7B parameter LLM efficiently and affordably?

[tyko0707]

I need to run about 5 different fine-tuning methods across 4 datasets, each with 1000-1500 samples. The model I need to fine-tune is Mistral7B.

I have a basic Google Colab subscription, but it doesn’t allow access to an A100 GPU, and the T4 GPU available lacks enough memory to fine-tune a 7B parameter model—it quickly runs out of memory. And even with optimizations like quantization and mixed precision, I’m still hitting memory limits.

I’m looking for cost-effective platforms or services with sufficient GPU memory to handle these tasks efficiently. Any recommendations?

[nielsr]

Hi,

The cheapest platforms out there are Lambda Labs, Runpod and Vast.ai. They provide the cheapest GPUs on the market. I made a tutorial video in which I fine-tune Mistral-7B using a GPU provided by Runpod.

Of course, you could also rent a VM with an attached GPU on AWS, Google Cloud and Azure.