Hello there,
i work with GPT-Neo and fine tuned the model. I work on a virtual machine, which gets reseted every 12 hours, so I need to safe and load the fine tuned model. This is my code. I am a total rookie, so please forgive me my inexperience.
pip install happytransformer
from happytransformer import HappyGeneration
happy_gen = HappyGeneration("GPT-NEO", "EleutherAI/gpt-neo-125M")
from happytransformer import GENTrainArgs
args = GENTrainArgs(learning_rate =0.00001, num_train_epochs = 3)
happy_gen.train("Small.txt", args=args)
06/03/2022 09:51:15 - INFO - happytransformer.happy_transformer - Training...
PyTorch: setting up devices
***** Running training *****
Num examples = 11
Num Epochs = 3
Instantaneous batch size per device = 1
Total train batch size (w. parallel, distributed & accumulation) = 1
Gradient Accumulation steps = 1
Total optimization steps = 33