Hi, I’m wanting to get started installing and learning GPT-J on a local Windows PC. There are plenty of excellent videos explaining the concepts behind GPT-J, but what would really help me is a basic step-by-step process for the installation? Is there anyone that would be willing to help me get started? My plan is to utilize my CPU as my GPU has only 11GB VRAM , but I do have 64GB of system RAM so I’m guessing I can still run it, albeit slower than if I were using my GPU.
I’ve installed Python 3.10, PIP, and Pytorch so far, but I’d appreciate anything anyone can offer in terms of next steps. These are my steps so far.
Download and install Python 3.10 from Python Releases for Windows | Python.org
Download PIP installer - https://bootstrap.pypa.io/get-pip.py
From a CMD run python get-pip.py
Go to Start Locally | PyTorch and select the appropriate setup you want in order to install the correct PyTorch elements.
In my case, I’m choosing the following.
PyTorch Build: Windows
Your OS: Windows
Compute Platform: CPU
Run this Command: pip3 install torch torchvision torchaudio
Due to the VRAM limitation of my 2080Ti being 11GB, I’m opting for the CPU version because 11GB isn’t enough space to hold the 16bit or 32bit float version of the model in VRAM, although I do have 64GB System Ram and a Threadripper 1920x so I may be able to scrape by, though I expect it to be much slower than if I were using GPU.
Install Transformers with pip install transformers
Enter python from the command line
in python –
from transformers import pipeline
- setup the generator…
generator = pipeline(‘text-generation’, model=‘EleutherAI/gpt-j-6b-Float16’)
(This will download the pretrained model)
- create a new variable called prompt
prompt = “How did Bucky Barnes and Natasha Romanoff meet in the comics?”
res = generator(prompt, max_length=100, do_sample=True, temperature=0.1)
- Save the cached copy to another drive dedicated to storing the models.
- In future when loading the model use…
generator = pipeline(‘text-generation’, model = “D:\AI\pipelines\gpt-j-6b-Float16”)
I think I’ve been able to cobble together a good basic starter, but if anyone has any suggestions on improving it, let me know.