Wav2Vec2ForCTC.from_pretrained for already trained Models?

Beginners
1

Hi Guys,
how do i further train a Model someone else already fine-tuned? I wanted to load a already Fine-Tuned Modell like this:

    model = Wav2Vec2ForCTC.from_pretrained(
        model_args.model_name_or_path,
        cache_dir=model_args.cache_dir,
        activation_dropout=model_args.activation_dropout,
        attention_dropout=model_args.attention_dropout,
        hidden_dropout=model_args.hidden_dropout,
        feat_proj_dropout=model_args.feat_proj_dropout,
        mask_time_prob=model_args.mask_time_prob,
        gradient_checkpointing=model_args.gradient_checkpointing,
        layerdrop=model_args.layerdrop,
        ctc_loss_reduction="mean",
        pad_token_id=processor.tokenizer.pad_token_id,
        vocab_size=len(processor.tokenizer),
    )

But im getting:

RuntimeError: Error(s) in loading state_dict for Wav2Vec2ForCTC:
	size mismatch for lm_head.weight: copying a param with shape torch.Size([174, 1024]) from checkpoint, the shape in current model is torch.Size([35, 1024]).
	size mismatch for lm_head.bias: copying a param with shape torch.Size([174]) from checkpoint, the shape in current model is torch.Size([35]).

How would i do this right?

Something like

    model = Wav2Vec2ForCTC.from_pretrained(model_args.model_name_or_path)
    model.to("cuda")
    model.freeze_feature_extractor()
    processor = Wav2Vec2Processor.from_pretrained(model_args.model_name_or_path)

But im not sure if this is the way to go.

Ty in advanced

2

It seems like your vocabulary size does not match the vocab size from the original model. You can remove the old LM head manually and initialise a new one:

state_dict = torch.load(f"{model_args.model_name_or_path}/pytorch_model.bin", map_location='cpu')
state_dict.pop('lm_head.weight')
state_dict.pop('lm_head.bias')

model = Wav2Vec2ForCTC.from_pretrained(
        model_args.model_name_or_path,
        state_dict=state_dict,
        ...
    )