Fine Tuning Falcon7B with QLora

🤗Transformers
1

Hello
I am trying to fine tune Falcon7B with only little ressources.
I found several notebooks and tutorials on how to do it with T4 Colab or only Kaggle.
But I keep encountering the same error related to this crucial step:

model = prepare_model_for_kbit_training(model)

Error:

TypeError: FalconPreTrainedModel._set_gradient_checkpointing() got an unexpected keyword argument ‘enable’

Thank you for your help

2

In the TrainingArguments set gradient_checkpointing to false

3

Since it’s an online Notebook, I cant modify the code source, do you have an idea of how I can modify it from the notebook itsefl ?

Here is the Notebook

4

Hello,

Thank you for your answer.
It seems that I can’t change the source code itself.
here is the notebook Google Colab
Do you know what I should change on the notebook in order to make it work ?

5

@MGDragon I am also having this issue, please let me know if you find solution for this error.

6

I did:

model = prepare_model_for_kbit_training(model,use_gradient_checkpointing=False)

This should work, and also reduce the batch size afterwards!

1 Like