Simple NLP Example not working

I’m looking for an example to learn how to use TPUs on Colab running PyTorch.
I’m glad to find the Simple NLP Example which is unfortunately not working.
Running w/o modifications leads to following error message running the last cell:

from accelerate import notebook_launcher



ImportError                               Traceback (most recent call last)

<ipython-input-50-a91f3c0bb4fd> in <module>()
      1 from accelerate import notebook_launcher
----> 3 notebook_launcher(training_function)

1 frames

/usr/local/lib/python3.7/dist-packages/torch_xla/ in <module>()
     99 from ._patched_functions import _apply_patches
    100 from .version import __version__
--> 101 import _XLAC

ImportError: /usr/local/lib/python3.7/dist-packages/ undefined symbol: _ZNK3c1010TensorImpl20is_contiguous_customENS_12MemoryFormatE

NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.

I found a workaround description here which says:

... downgrading PyTorch to torch-1.8.2+cpu,
but that leads to another error message

ProcessExitedException: process 0 terminated with signal SIGSEGV

What is necessary to run that example?
Do you know any other example that meets my requirements (Colab, TPUs, PyTorch) and runs?

Thanks for any comment

1 Like

I’m guessing this is due to a version mismatch between PyTorch XLA and PyTorch (PyTorch XLA is installed with a version for PyTorch 1.9 and Colab now uses PyTorch 1,10). I’ve asked for an updated link to install the proper version of PyTorch XLA, but in the meantime, you cna solve the issue by downgrading PyTorch to 1.9.1 in the Colab you are running.

On request of the pytorch version I get “1.9.0+cu102”
What exactly am I supposed to do?

Ah you’re right, I was confused. It’s working now with no change. Perhaps there was some maintenance isse?

Great, thanks

Weirdly enough, the notebook example is working for me with the normal RAM TPU from colab (if I change the pytorch version to !pip3 install torch==1.9 from the new colab default torch==1.10).

But I get the same SIGSEGV error, when changing to the high-RAM TPU from colab. the exact same code works with the low-ram TPU, does does not work with the high-ram TPU. probably an issue with google colab’s setup in the background …

Colab with traceback here: simple_nlp_example.ipynb - Google Drive