Problem with pushing quantized model to hub

🤗Transformers
1

Hi, everyone! I have been trying to upload a quantized model to huggingface using the push_to_hub function but I always receive this error:

AttributeError: 'str' object has no attribute 'data_ptr'

Here’s the code:

from transformers import T5Tokenizer, T5ForConditionalGeneration
from quanto import quantize, freeze, qint8

model_id = "google/flan-t5-base"

quantized_model = T5ForConditionalGeneration.from_pretrained(model_id, low_cpu_mem_usage=True, use_safetensors=True)

quantize(quantized_model, weights=qint8, activaions=None)

freeze(quantized_model)

tokenizer.push_to_hub("flan-t5-base-8bit")
quantized_model.push_to_hub("flan-t5-base-8bit")

Thank you in advance!

2

Hi,

Feel free to open an issue on the Quanto repository: GitHub - huggingface/quanto: A pytorch Quantization Toolkit