Dear course instructors,
I hope you’re well. I’ve been going through the course materials, and recently purchased and have been working through your excellent book, Natural Language Processing with Transformers, and I’m grateful for the progress I’ve made so far. However, I’ve encountered a small issue on page 338 in Chapter 10 which is related to chapter 3 and chapter 7 of the course.
Specifically, I’ve been training my model and creating a new Git branch for each run, saving the improving epochs within each branch. Now, I’m trying to use the pipeline API, and I need to merge the best-performing model into the main branch. Unfortunately, I’ve spent three days trying without success.
In the book, you outline three steps for merging:
git checkout main
git merge
git push.
My challenge is identifying the correct . For instance, my repository is called , and I have three branches: , , and . Within each branch, I’ve stored the model checkpoints for each improving epoch. Let’s say the best model is in the <epoch_15> folder in the branch. How do I properly merge this model into main? I have tried many things unsuccessfully. In my latest attempt, I managed to merge a specific commit, but as a result, under the main tag in the Hub I have to navigate through the branch and epoch folders before finding the model checkpoints. If I look at other models in the Hub the checkpoints and other json files are directly located in main without traversing other folders. I have saved three models in my repo called “m2im”.
Additionally, during training, I only saved the model and not the tokenizer. How can I upload the tokenizer to the main branch after the merge? Should I use push_to_hub, or is there another preferred method? I already know how to save it locally, but I’m unsure about the best practice for merging and pushing it in this context.
I realize this may be more of a Git-related issue, but it ties directly into efficiently using the pipeline API. If I don’t fix this problem, I won’t be able to use the pipeline API. All my experiments and models are saved in the Hub, but I am unable to share them with the community properly.
Thank you so much for your time and assistance.
Best regards,
Milton