Hi all !
I wanted to know if there is any way to finetune a model but with smaller parameters (for exemple : sequence length from 512 to 64 and dModel from 768 to 256).
I could use the pytorch TransformerEncoder and train my own from scratch but I wanted to know if there were some fine-tunning possible (i.e not starting from scratch).
Thanks in addvance for any help,
Have a great day !