Trainer.train() will cause PretrainedConfig default construct

Intermediate
1

Hey there, I am working on my own project and here is the issue.
I designed a custom config which I’d like to log info when config is initialized(init)，so I set

logging.set_verbosity_info()

Then, I use the HuggingFace default trainer for training. When I start trainer.train()，I found that a config will be constructed with default value in the default callback. I don’t know whether this is necessary. Because I just want to see my log info when I initialize the config in my code, but that default initialize just confused me.

Following is the detailed code.
here is the defination of my custom config which is inherited from PretrainedConfig

class CustomConfig(PretrainedConfig):
    def __init__(self, **kwargs):
        logger.info('something')
       ....

and I set my CustomPretrainedModel with this ConfigClass

class CustomPretrainedModel(PretrainedModel):
    config_class = CustomConfig
    ...

then I run the training code

from transformers import Trainer
from transformers import logging
logging.set_verbosity_info()
# here will log the info when initialize the config
my_config = CustomConfig()
my_model = CustomPretrainedModel(my_config)
trainer = Trainer(
     model = CustomPretrainedModel()
)
# here will also log the info because there is a default construct in callbacks, but I don't expect that.
trainer.train()

I check the source code, and I found that construct is from the following traceback

File "/home/.../anaconda3/envs/laplace_huggingface/lib/python3.11/site-packages/transformers/trainer_callback.py", line 353, in on_train_begin
    return self.call_event("on_train_begin", args, state, control)
  File "/home/.../anaconda3/envs/laplace_huggingface/lib/python3.11/site-packages/transformers/trainer_callback.py", line 397, in call_event
    result = getattr(callback, event)(
  File "/home/.../anaconda3/envs/laplace_huggingface/lib/python3.11/site-packages/transformers/integrations.py", line 630, in on_train_begin
    model_config_json = model.config.to_json_string()
  File "/home/.../anaconda3/envs/laplace_huggingface/lib/python3.11/site-packages/transformers/configuration_utils.py", line 828, in to_json_string
    config_dict = self.to_diff_dict()
  File "/home/.../anaconda3/envs/laplace_huggingface/lib/python3.11/site-packages/transformers/configuration_utils.py", line 768, in to_diff_dict
    class_config_dict = self.__class__().to_dict() if not self.is_composition else {}

I’d like to know is there some way to avoid.

thanks for help

1 Like
2

Im having the same problem. Have you solved it yet?