I’m using bitsandbytes int8 and PEFT with transformers. Is there any advantage to using bf16 with int8 even though it gets cast to fp16 during quantization? Or should this be strictly avoided?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Fp16, bf16 in TrainingArgs vs BitsAndBytesConfig | 0 | 744 | June 30, 2023 | |
| Low bf16 performance on TPU, int4 vs int8 quantizatoin | 0 | 329 | June 1, 2024 | |
| Bitsandbytes `has_fp16_weights` issue | 1 | 159 | August 15, 2024 | |
| Bfloat16 conversion results in significantly slower computation for various transformer models | 0 | 1399 | December 20, 2021 | |
| Can we use mixed precision with all? (fp16 + fp32 + bf16) | 0 | 270 | December 1, 2022 |