How to create a new Hugging face model by using already available hugging face models

🤗Transformers
1

is that possible to create a new hugging face transformer by using 2 or 3 already pretrained transformers performing different tasks?

Let’s say i want to create a speech-to-speech translation model by using speech to txt model + txt to txt model + txt to speech model, let’s say these models are already available in hugging face as a transformer how can i use these 3 models to create a new hugging face transformer?

2

Hi,

Yes that’s possible, you can inherit from the PretrainedModel class.

Like this:

from transformers.modeling_utils import import PreTrainedModel

class CustomModel(PreTrainedModel):
    def __init__(self, config):
        super().__init__(config)
        self.num_labels = config.num_labels
        self.config = config

        self.model_1 = BertModel.from_pretrained("bert-base-uncased")
        self.model_2 = Wav2Vec2Model.from_pretrained("facebook/wav2vec2-base-960h")
        self.classifier = nn.Linear(config.hidden_size, config.num_labels)

        # Initialize weights and apply final processing
        self.post_init()
3

Hi, thanks for the solution… but I am so new to the hugging face I am just exploring things right now so can you share me any reference materials regarding this, or could you explain me about this in a detail way?