Hugging Face Forums
How BitsAndBytesConfig use bnb_4bit_compute_dtype=torch.bfloat16 on gpu don't support torch.bfloat16
Beginners
mohamedemam
October 20, 2023, 11:05pm
1
is it convert to float16 when find doesn’t support bfloat16
Related topics
Topic
Replies
Views
Activity
Changing bnb_4bit_compute_dtype
Beginners
0
139
July 18, 2024
Low bf16 performance on TPU, int4 vs int8 quantizatoin
🤗Accelerate
0
366
June 1, 2024
Loading in Float32 vs Float16 has very different speed
🤗Transformers
1
164
February 20, 2025
Fp16, bf16 in TrainingArgs vs BitsAndBytesConfig
Beginners
0
795
June 30, 2023
Understanding how changing bnb_4bit_compute_dtype affects outputs
🤗Transformers
1
4787
February 10, 2024