I have been playing with quantization and PEFT and I noticed that the trainable parameters are significantly reduced after applying quantization (but before applying PEFT) does anyone know why this happens? is it normal? Here is an example: Counting Functions: def print_trainable_parameters(model…

Less Trainable Parameters after quantization

kdcyberdude December 23, 2023, 8:27am 7

Can @TheBloke provide us with an answer? We would appreciate some clarity on what is happening here.

1 Like

Topic		Replies	Views
Number of parameters reduced after loading in 4bit Models	7	934	June 28, 2024
Parameter Count & Shape Discrepancies in 4-bit vs. Higher bit LLM models 🤗Transformers	2	680	June 3, 2024
Does quantization compress the model weights? Research	16	378	September 26, 2024
Difference in Number of Parameters for load_in_4bit Beginners	0	556	August 2, 2023
Loading quantised weights does not work Beginners	0	122	April 12, 2024