I was wondering that about how to quantize a model and was exploring in that regard.
But I am unable to quantize the tiiuae/falcon-7b model on colab.
here is my notebook and please ignore the rest it was taken from other notebooks.
And would be really helpful if you can point to a guide to know about the parameters to use in this regard.