Trainer class always moves model back to cuda:0

As stated, even is I do model.to('cuda:1'), after everything passed into the trainer class for training, the model was moved back to cuda:0 again. However, cuda:0 does not have enough space on my device. What should I do?