Further Train a fine-tuned Modell?

Hi Guys,

how would i fruther train a already fine-tuned modell on new data?
Iwant to (further) train a wav2vec2 Modell which was trained on CommonVoice on a new Dataset. Whould i have to train the modell from Scratch and use both datasets or could i just “finetune” the already fine tuned modell on new data? I tried multiple Learning-Rates and i just destroy the Modell (accuracy-wise) or the values dont change at all if i further train the modell