Ideas for beginner-friendlier TPU-VM clm training

Beginners
3

  • Follow this tutorial to set-up and connect to your TPU-VM

  • Add your local bin path to the PATH environment variable, If you do not know your local user name, type:
    whoami
    #In my case, the user is ‘dadler’, so replace ‘dadler’ in the following block with your own user:
    nano ~/.bashrc
    #Add the following line at the bottom, replace dadler with your own user name
    export PATH="/home/dadler/.local/bin:$PATH"
    #Save (CTRL O) #Exit (Ctrl X)
    #Realod bashrc
    source ~/.bashrc

  • Install and upgrade libraries

pip install datasets
git clone https://github.com/huggingface/transformers.git
sudo pip install --user -e transformers
pip install --upgrade tokenizers
pip install --upgrade clu
git clone https://github.com/google/flax.git
sudo pip install --user -e flax
pip install git+https://github.com/deepmind/optax.git
  • Setup git-lfs
sudo apt install git-lfs
git lfs install

  • Login to your huggingface account
    huggingface-cli login

  • Save your git credentials on the local VM (Not secure, do this only if you are the only person who has access the TPU-VM instance)

git config --global credential.helper 'store --file ~/.git-credentials'
git credential fill

#Type/Paste the following two lines:

protocol=https
host="huggingface.co"

#Now hit Enter until you are prompt to enter your huggingface user and password

  • N̶o̶t̶ ̶s̶u̶r̶e̶ ̶i̶f̶ ̶t̶h̶i̶s̶ ̶i̶s̶ ̶n̶e̶e̶d̶e̶d̶,̶ ̶b̶u̶t̶ ̶i̶n̶ ̶t̶h̶e̶ ̶t̶e̶r̶m̶i̶n̶a̶l̶ ̶s̶e̶s̶s̶i̶o̶n̶ ̶y̶o̶u̶ ̶a̶r̶e̶ ̶a̶b̶o̶u̶t̶ ̶t̶o̶ ̶r̶u̶n̶ ̶y̶o̶u̶r̶ ̶t̶r̶a̶i̶n̶i̶n̶g̶ ̶s̶c̶r̶i̶p̶t̶ ̶i̶n̶,̶ ̶y̶o̶u̶ ̶m̶i̶g̶h̶t̶ ̶w̶a̶n̶t̶ ̶t̶o̶ ̶t̶y̶p̶e̶:̶
    ̶̶e̶x̶p̶o̶r̶t̶ ̶X̶R̶T̶_̶T̶P̶U̶_̶C̶O̶N̶F̶I̶G̶=̶"̶l̶o̶c̶a̶l̶s̶e̶r̶v̶i̶c̶e̶;̶0̶;̶l̶o̶c̶a̶l̶h̶o̶s̶t̶:̶5̶1̶0̶1̶1̶"̶̶

  • Continue by following the instructions in this tutorial

Hope it helps :slight_smile:

1 Like