GPT-J beginners install guides?

Beginners
1

Hi, I’m wanting to get started installing and learning GPT-J on a local Windows PC. There are plenty of excellent videos explaining the concepts behind GPT-J, but what would really help me is a basic step-by-step process for the installation? Is there anyone that would be willing to help me get started? My plan is to utilize my CPU as my GPU has only 11GB VRAM , but I do have 64GB of system RAM so I’m guessing I can still run it, albeit slower than if I were using my GPU.

I’ve installed Python 3.10, PIP, and Pytorch so far, but I’d appreciate anything anyone can offer in terms of next steps. These are my steps so far.

  1. Download and install Python 3.10 from Python Releases for Windows | Python.org

  2. Download PIP installer - https://bootstrap.pypa.io/get-pip.py

  3. From a CMD run python get-pip.py

  4. Go to Start Locally | PyTorch and select the appropriate setup you want in order to install the correct PyTorch elements.

In my case, I’m choosing the following.

PyTorch Build: Windows

Your OS: Windows

Package: Pip

Language: Python

Compute Platform: CPU

Run this Command: pip3 install torch torchvision torchaudio

Due to the VRAM limitation of my 2080Ti being 11GB, I’m opting for the CPU version because 11GB isn’t enough space to hold the 16bit or 32bit float version of the model in VRAM, although I do have 64GB System Ram and a Threadripper 1920x so I may be able to scrape by, though I expect it to be much slower than if I were using GPU.

  1. Install Transformers with pip install transformers

  2. Enter python from the command line

  3. in python –

from transformers import pipeline

  1. setup the generator…

generator = pipeline(‘text-generation’, model=‘EleutherAI/gpt-j-6b-Float16’)
(This will download the pretrained model)

  1. create a new variable called prompt

prompt = “How did Bucky Barnes and Natasha Romanoff meet in the comics?”

res = generator(prompt, max_length=100, do_sample=True, temperature=0.1)

  1. Save the cached copy to another drive dedicated to storing the models.

generator.save_pretrained(“D:\AI\pipelines\gpt-neox-20b”)

  1. In future when loading the model use…

generator = pipeline(‘text-generation’, model = “D:\AI\pipelines\gpt-j-6b-Float16”)

I think I’ve been able to cobble together a good basic starter, but if anyone has any suggestions on improving it, let me know.