Help with Training a Custom Model using Hugging Face Transformers

Beginners
1

Hi everyone,

I’ve been trying to train a model using the Hugging Face Transformers library, but I’m encountering some issues. I want to create a custom language model with my dataset, but the training process is taking longer than expected, and the model’s accuracy is not where I’d like it to be.

My Problems:

  1. Training Time: My model has been training for 10 hours, but it still has a very low accuracy rate. The dataset I’m using contains about 50,000 samples. How can I optimize the training time?
  2. Hyperparameter Tuning: I’m confused about which hyperparameters I should experiment with. How can I evaluate the impact of parameters like learning rate and batch size?
  3. Model Selection: I’m currently using a BERT-based model, but I wonder if another model might yield better results. What model recommendations do you have?

Thanks in advance for your help! If you need more information, feel free to ask.

1 Like