Importing DataCollator gives me a Bus error

Beginners
1

Simply running this line from transformers import DataCollatorForLanguageModeling gives me a Bus error (core dumped).

I’m using transformers==4.23.1 and python==3.9.12. What could be the cause of this error?

1 Like
2

My /dev/shm was full apparently. After increasing the size of it by running mount -o remount,size=25G /dev/shm, the import worked just fine. I’ve no idea why this particular import statement uses shm though

3

How much should be the minimum /dev/shm. I have 8GB and it is empty. I am still getting Bus Error while importing sentence_transformers.