Using hyperparameter-search in Trainer

🤗Transformers
51

Ah yes, the very first time the model is initialized (in the init of the Trainer) you will get a None for that trial (since there is no trial yet).

So you should have a backup for that in your get_model function:

    def get_model(params):
        db_config = db_config_base
        print(params)
        if params is not None:
            db_config.update({'alpha': params['alpha'], 'dropout': params['dropout']})
        return DistilBERTForMultipleSequenceClassification.from_pretrained(db_config, num_labels1 = 2, num_labels2 = 8)

You should then see printed one None, and then the value for each successive trial.

2 Likes