Hugging Face Forums

RuntimeError: The size of tensor a (48) must match the size of tensor b (64) at \nnon-singleton dimension 0"}

Inference Endpoints on the Hub

John6666 April 29, 2025, 8:47am 2

Seems unresolved issue of TGI?

rdaya
In case anyone runs into this, the trick (a bad one) is to set USE_FLASH_ATTENTION=false

Topic		Replies	Views	Activity
ERROR: The size of tensor a () must match the size of tensor b () at non-singleton dimension 1 Inference Endpoints on the Hub	0	783	January 11, 2024
I am following a hugging face guide for fine tuning whisper but I run into error when training 🤗Transformers	0	166	March 15, 2024
Trainer RuntimeError: The size of tensor a (462) must match the size of tensor b (448) at non-singleton dimension 1 🤗Transformers	17	43425	May 23, 2024
{"error":"The expanded size of the tensor (524) must match the existing size (514) at non-singleton dimension 1. Target sizes: [1, 524]. Tensor sizes: [1, 514]"} Models	0	131	November 22, 2024
RuntimeError: The size of tensor a (4096) must match the size of tensor b (4097) at non-singleton dimension 3 Models	1	319	August 24, 2024