In HuggingArtists, we can generate lyrics by a specific artist. This was made by fine-tuning a pre-trained HuggingFace Transformer on parsed datasets from Genius.
As well, we use Weights & Biases integration to automatically log model performance and predictions.
Project repository:
About
Create training script for TPU with Flax/JAX. So users can choose GPU or TPU in Colab.
Model
A pre-trained GPT2 model in English and fine-tune it to learn English lyrics.
The GPT2 model, pre-trained in English: link.
Datasets
Parsed with this script from Genius.
Training scripts
GPU with PyTorch is available in project Colab:
So we need to create for TPU with Flax/JAX.
Project outcome
Create training script for TPU with Flax/JAX. So users can choose GPU or TPU in Colab.
Challenges
- I am new with Flax/JAX, so I need some help.
- There is no other challenges
Conclusion
We can integrate this project with Video and Music generation so we can create our AI-Artist 
I hope to find like-minded people and create a cool project 
