How to create a config.json after saving a model

Hi, I am trying to convert my model to onnx format with the help of this notebook
I got error , since config.json does not exist.
My model is a custom model with extra layers, similar to this,

Now how can I create a config.json file for this?

Normally, if you save your model using the .save_pretrained() method, it will save both the model weights and a config.json file in the specified directory.

Yes, but this is a custom model that I have saved in pytorch style, since it consists of additional layers, is there anyway to generate confg.json file?

You need to subclass it to have the save_pretrained methods available. So instead of

class Mean_Pooling_Model(nn.Module):

use

from transformers.modeling_utils import PreTrainedModel
class Mean_Pooling_Model(PreTrainedModel):

It will add extra functionality on top of nn.Module.

Thank you I will try this!

Is it possible to generate the configuration file for already trained model , i.e weights stored in normal pytorch model.bin

1 Like

Use model.config.to_json() method to generate config.json

Did you end up finding a solution to getting a config.json from an already trained model? :slight_smile: I’m currently struggling with the same problem :frowning:

Nope, I was not able to find a proper solution, I ended up writing the config.json manually

You should be able to just call

model.config.to_json_file("config.json")

cc @seanbenhur

That only works for models that are transformer native and not nn.Module/pytorch native, sadly.

What is your use-case that you are using Transformers but not Transformers models? If you want to use the HF Trainer alongside with your own PyTorch model, I recommended to subclass the relevant classes, similar to PretrainedModel

And to use your own PretrainedConfig alongside of it.