I’m using a Bert model and noticed that the model doesn’t use the whole word in some cases. I’m wondering if this is because of the tokeniser or the phrases in my dataset. There are 12 classes, 9 of which are under 10%. There are about 1200 phrases in the whole dataset. I think the reason is either…

Strange shap analysis for text classification with BERT

ba-at-huggingface September 17, 2024, 8:50pm 11

Wondering if you ever tried Shap for words, even though the tokenizers tokenize for word-pieces.

Topic		Replies	Views
Doubts about the tokenization strategy and the explanation of models through SHAP 🤗Tokenizers	0	236	May 22, 2024
Improving performance results for BERT 🤗Transformers	2	972	November 25, 2020
Text classifier is trained incorrectly using BERT transformers (f1 = 0) for a certain amount of dataset 🤗Transformers	2	832	August 31, 2023
Why my simple Bert model for text classification could not learn anything? Beginners	2	2071	October 23, 2023
Bert Text classification Intermediate	7	565	November 24, 2023