RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with `TORCH_USE_CU

🤗Transformers
1

Hi All,

I am getting below error and tried pterry much everything. But did not work out. There are 8 clases in the dataset. I am not sure how to resolve the issue.

The code is:

Convert data to InputExample format

Convert labels to integers

for example in train_examples:
example.label = int(example.label)
#print(example.label)

Wrap the training examples in a SentencesDataset for compatibility with DataLoader

train_dataset = SentencesDataset(train_examples, model)
train_dataloader = DataLoader(train_dataset, shuffle=True, batch_size=16)

Define the loss function with SoftmaxLoss for binary classification

train_loss = losses.SoftmaxLoss(
model=model,
sentence_embedding_dimension=model.get_sentence_embedding_dimension(),
num_labels=8
)

Train the model using fit()

model.fit(
train_objectives=[(train_dataloader, train_loss)],
epochs=4,
warmup_steps=100
)

The error is:
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Thank you,

Seyhan

1 Like