How do make sure I am using the transformer version/code from source?

Beginners
1

How do I know whether my jupyter notebook is using the transformer version/code that I cloned from github?

My steps:
I did fork the transformers repo in my github account then I did:
git clone --recursive https://github.com/myaccount/transformers.git
cd transformers/
conda create -n hf-dev-py380 python=3.8.0
conda activate hf-dev-py380
git checkout v4.9.2-release
pip install -e “.[dev]”
conda install -c conda-forge librosa
conda install libgcc
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/miniconda3/lib/
transformers-cli env

conda install jupyter
nohup jupyter notebook --ip=0.0.0.0 --port=myport &

When I type in my terminal “python --version”, it appears the same python version that my jupyter notebook prints (v3.8.0). But a different python version (v3.7) and directory appear in my jupyter notebook when I type:
import transformers
print(transformers)
<module ‘transformers’ from ‘/mypath/miniconda3/lib/python3.7/site-packages/transformers/init.py’>

How do I make sure my jupyter notebook is using the transformer code I cloned in my terminal?

Thanks!

2

Just to complement my question, the documentation says that pip install -e . command would link the folder I cloned the repository to my python library paths. So the python packages would get installed into the directory I used to clone the repo, which is …/transfomers/. But still I don’t see the site-packages folder there.

Any idea what do I need to do to link my jupyter notebook to my cloned folder?

Thank you!

3

Hi community!
Any idea on this topic? Thanks!

4

Why don’t you simply try (after importing the library):

print(transformers.__version__)

and see if it turns out to be the one you are looking for?

5

Thank you, @shabie . The command is very useful!

But how do I make sure my the printed Transformer version (eg. 4.10.3) is using the transformer code I cloned in my terminal? Is there any setting file in transformer code where I can double check about it?

Thank you!

6

What I always use is the following:

!rm -r transformers
!git clone https://github.com/huggingface/transformers.git
!cd transformers
!pip install -q ./transformers
1 Like
7

Thanks a lot for sharing what you do!