In chapter 3, fine-tuning a model, we are taking a Bert base model and switching the heads to use the task of sentence classification (sentiment analysis).
As I understand it, the parameters that we need to fine-tune are the ones in the new head - they are completely random. Or at least they should be tuned faster the the read of the parameters in the Base model, But I don’t see any code doing it. So what am I missing? is the code preventing the parameters in the base model from updating is hidden? or am I wrong in my assumption completely and all the parameters are tuned in the same manner?
1 Like