How to create a config.json after saving a model

seanbenhur · October 2, 2021, 12:02pm

Hi, I am trying to convert my model to onnx format with the help of this notebook
I got error , since config.json does not exist.
My model is a custom model with extra layers, similar to this,

Now how can I create a config.json file for this?

nielsr · October 4, 2021, 9:18am

Normally, if you save your model using the .save_pretrained() method, it will save both the model weights and a config.json file in the specified directory.

seanbenhur · October 4, 2021, 9:46am

Yes, but this is a custom model that I have saved in pytorch style, since it consists of additional layers, is there anyway to generate confg.json file?

BramVanroy · October 4, 2021, 9:51am

You need to subclass it to have the save_pretrained methods available. So instead of

class Mean_Pooling_Model(nn.Module):

use

from transformers.modeling_utils import PreTrainedModel
class Mean_Pooling_Model(PreTrainedModel):

It will add extra functionality on top of nn.Module.

github.com

huggingface/transformers/blob/bcc3f7b6560c1ed427f051107c7755956a27a9f2/src/transformers/modeling_utils.py#L415

    
      
                      exclude_embeddings (:obj:`bool`, `optional`, defaults to :obj:`True`):
                          Whether or not to count embedding and softmax operations.
          
          
        Returns:
                      :obj:`int`: The number of floating-point operations.
                  """
          
          
        return 6 * self.estimate_tokens(input_dict) * self.num_parameters(exclude_embeddings=exclude_embeddings)
          
          

          
class PreTrainedModel(nn.Module, ModuleUtilsMixin, GenerationMixin, PushToHubMixin):
              r"""
              Base class for all models.
          
          
    :class:`~transformers.PreTrainedModel` takes care of storing the configuration of the models and handles methods
              for loading, downloading and saving models as well as a few methods common to all models to:
          
          
        * resize the input embeddings,
                  * prune heads in the self-attention heads.
          
          
    Class attributes (overridden by derived classes):

seanbenhur · October 4, 2021, 9:55am

Thank you I will try this!

seanbenhur · October 6, 2021, 9:00am

Is it possible to generate the configuration file for already trained model , i.e weights stored in normal pytorch model.bin

avinash-chaluvadi · February 18, 2022, 11:43am

Use model.config.to_json() method to generate config.json

sophiaaez · April 19, 2022, 11:47am

Did you end up finding a solution to getting a config.json from an already trained model? I’m currently struggling with the same problem

seanbenhur · April 19, 2022, 12:35pm

Nope, I was not able to find a proper solution, I ended up writing the config.json manually

BramVanroy · April 28, 2022, 3:35pm

You should be able to just call

model.config.to_json_file("config.json")

github.com

huggingface/transformers/blob/1be8d56ec6f7113810adc716255d371e78e8a1af/src/transformers/configuration_utils.py#L808

      
        
            
            
    Returns:
                    `str`: String containing all the attributes that make up this configuration instance in JSON format.
                """
                if use_diff is True:
                    config_dict = self.to_diff_dict()
                else:
                    config_dict = self.to_dict()
                return json.dumps(config_dict, indent=2, sort_keys=True) + "\n"
            
            
def to_json_file(self, json_file_path: Union[str, os.PathLike], use_diff: bool = True):
                """
                Save this instance to a JSON file.
            
            
    Args:
                    json_file_path (`str` or `os.PathLike`):
                        Path to the JSON file in which this configuration instance's parameters will be saved.
                    use_diff (`bool`, *optional*, defaults to `True`):
                        If set to `True`, only the difference between the config instance and the default `PretrainedConfig()`
                        is serialized to JSON file.
                """

cc @seanbenhur

sophiaaez · June 16, 2022, 7:44am

That only works for models that are transformer native and not nn.Module/pytorch native, sadly.

BramVanroy · June 16, 2022, 9:03am

What is your use-case that you are using Transformers but not Transformers models? If you want to use the HF Trainer alongside with your own PyTorch model, I recommended to subclass the relevant classes, similar to PretrainedModel

github.com

huggingface/transformers/blob/3981ee8650042e89d9c430ec34def2d58a2a12f7/src/transformers/modeling_utils.py#L955

      
        
                        exclude_embeddings (`bool`, *optional*, defaults to `True`):
                            Whether or not to count embedding and softmax operations.
            
            
        Returns:
                        `int`: The number of floating-point operations.
                    """
            
            
        return 6 * self.estimate_tokens(input_dict) * self.num_parameters(exclude_embeddings=exclude_embeddings)
            
            

            
class PreTrainedModel(nn.Module, ModuleUtilsMixin, GenerationMixin, PushToHubMixin):
                r"""
                Base class for all models.
            
            
    [`PreTrainedModel`] takes care of storing the configuration of the models and handles methods for loading,
                downloading and saving models as well as a few methods common to all models to:
            
            
        - resize the input embeddings,
                    - prune heads in the self-attention heads.
            
            
    Class attributes (overridden by derived classes):

And to use your own PretrainedConfig alongside of it.

Hosna · July 27, 2022, 10:53am

I have a similar issue where I have my model’s (nn.module) weights and I want to convert it to be huggingface compatible model so that I can use hugging face models (as .generate). From the discussions I can see that I either have to retrain again while changing (nn.module to PreTrained) or to define my config.json file. If I wrote my config.json file what should I do next to load my torch model as huggingface one?

tanyaroosta · August 4, 2022, 3:25am

I am not sure from the discussion above, what the solution is. Can someone post their working example please?

thepurpleowl · September 8, 2022, 12:59pm

I am not sure whether this functionality exists at this moment.

nairajay2k · November 14, 2022, 2:15pm

Folks,
I am trying out the code at GitHub - aws-samples/aws-inferentia-huggingface-workshop: CMP314 Optimizing NLP models with Amazon EC2 Inf1 instances in Amazon Sagemaker .

For inferentia instance, I get a config.json not found in the cloudwatch logs and inference fails. The config.json file is present in the traced model tar.gz file for inferentia.
Please help me resolve this.
Thanks
Ajay

P.S the log message — W-9002-model_1-stdout MODEL_LOG - OSError: file /home/model-server/tmp/models/cb9491669c1f44c1a0763e8a62d9368e/config.json not found