I get the following error trying to save a quantized model. Can anyone help? Thanks. quantize model quantized_model = torch.quantization.quantize_dynamic( model, {torch.nn.Linear}, dtype=torch.qint8 ) quantized_model.save_pretrained(quantized_dir) AttributeError Tra…

I found the solution here [image] Pegasus Model Weights Compression/Pruning Models @lewtun I chose dynamic quantization approach. And I dont think from_pretrained support loading quantised models. I’d be happy if you can take a look. Can we hop on a google meet call and y…

Error saving quantized model

Intermediate

omoekan June 30, 2022, 9:59pm 2

I found the solution here

Topic		Replies	Views
Model quantization Models	5	1901	February 15, 2023
Gradient_checkpointing = True results in error 🤗Transformers	3	6674	February 22, 2023
AttributeError: 'FalconModel' object has no attribute 'model' 🤗Accelerate	3	256	February 3, 2024
How to upload a quantized model? Models	5	3917	June 17, 2021
Load quantized model in memory Beginners	1	366	December 8, 2023

Error saving quantized model

Related Topics