I use Huggingface Trainer to train a BERT model for sequence classification (on a CPU, no CUDA).
I use Jupyter to run my code.
I also make use of Huggingface Dataset and as far as I understand by default it reads the CSV file with a memory mapped file so no problem there.
When training, Jupyter shows me higher and higher values of memory usage until the kernel eventually dies.
How can I prevent the kernel from dying while training?