Facing issues with AWQ inference engine while finetuning idefics 2 8b Awq model

[Bharanidharan07]

Hi Folks,

I tried to Finetune idefics2-8b-AWQ model but unfortunately i encountered this error “ImportError: /usr/local/lib/python3.10/dist-package/awq_inference_engine.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi” so i tried installing awq inference engine but pypi doesnt have awq inference engine package so i tried downloading from a third party and done a setup but still i landed on that same error, can someone help me with this?

[jordyvl]

same issue

[jordyvl]

pip install autoawq --upgrade did the trick for me