Using GitHub CI/CD with Hugging Face Datasets and Models

Beginners
1

Hello there!

Long term GitHub user and most of my workflow is currently there.

However, I’d like to start using many (if not all) of the Hugging Face features.

I’m wondering: can I use GitHub and Hugging Face together?

Specifically: GitHub for CI/CD, Actions etc + Hugging Face for datasets/models (larger storage)?

In essence, I’m trying to figure a workflow for a full-stack ML app:

data → model → fix data → version data/model (Hugging Face) → ship if better than last (CI/CD) → add data → model → repeat

1 Like
2

You want to set up models and datasets on Hugging Face and manage the code on GitHub?
No problem at all. Users do that, and even some Hugging Face staff members do it occasionally.

However, trying to do everything with Git (command) alone might be a bit tedious. Using shell scripts or Python would be faster.

1 Like