Hi, I am trying to train a zero-shot topic classification model on XNLI/vi dataset using phobert-base-v2 on Google Colab Pro+. I got the following error whenever it reaches a certain steps with different batch_size: RuntimeError: CUDA error: device-side assert triggered Compile with TORCH_USE_CU…

CUDA error: device-side assert triggered after a certain steps

shayonhuggingface July 21, 2023, 9:42am 2

This is when the batch_size is 8 the error appears at step 636

Topic		Replies	Views
RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with `TORCH_USE_CU 🤗Transformers	2	1158	November 1, 2024
[HELP] RuntimeError: CUDA error: device-side assert triggered Beginners	20	54473	October 23, 2024
CUDA error: device-side assert triggered 🤗Transformers	3	4276	June 4, 2021
CUDA error that only occurs on multiple gpus when doing batched training Beginners	0	769	June 25, 2024
RuntimeError: CUDA error: device-side assert triggered 4x10 🤗Transformers	0	177	April 11, 2024