I am trying to run a container with docker, using Package text-generation-inference · GitHub image and bitsandbytes-nf4 quantizer. While running, it logs an error: NameError: name ‘nn’ is not defined.
I have seen that it happens in text-generation-inference/server/text_generation_server/layers/bnb.py, on class Linear4bit, where it defines Linear4bit(nn.Module) instead of Linear4bit(torch.nn.Module) as it does on class Linear8bitLt, or importing torch.nn as nn. I think it was commited yesterday