Please advise on parameter settings!

Models
1

I am training a Bert4Rec model with the Amazon product review dataset. But Loss does not fall from 9.00. Please advise on parameter settings.

데이터셋 로드 및 DataLoader 설정

dataset = BERT4RecDataset(user_sequences, venue_data)
dataloader = DataLoader(dataset, batch_size=128, shuffle=True)

모델 초기화

vocab_size = len(dataset.venue_names)
model = BERT4RecModel(vocab_size=vocab_size)

# 손실 함수 및 옵티마이저 설정
criterion = nn.CrossEntropyLoss(ignore_index=0)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

# 모델 학습
model.train()
for epoch in range(20):  # 예시로 10번의 epoch
    total_loss = 0
    for batch in dataloader:
        input_ids, labels, mask = batch
        optimizer.zero_grad()
        logits = model(input_ids)
        logits = logits.view(-1, vocab_size)
        labels = labels.view(-1)
        loss = criterion(logits, labels)
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    print(f"Epoch {epoch+1}, Loss: {total_loss / len(dataloader):.4f}")

output:
Epoch 1, Loss: 14.1026
Epoch 2, Loss: 11.5711
Epoch 3, Loss: 10.5122
Epoch 4, Loss: 10.0275
Epoch 5, Loss: 10.5370
Epoch 6, Loss: 9.8295
Epoch 7, Loss: 9.7352
Epoch 8, Loss: 9.6376
Epoch 9, Loss: 9.4370
Epoch 10, Loss: 9.2748
Epoch 11, Loss: 9.2232
Epoch 12, Loss: 9.0581
Epoch 13, Loss: 9.0614
Epoch 14, Loss: 8.8171
Epoch 15, Loss: 8.7451
Epoch 16, Loss: 8.7009
Epoch 17, Loss: 10.2597
Epoch 18, Loss: 9.9219
Epoch 19, Loss: 9.8160
Epoch 20, Loss: 8.6171

1 Like