|
ValueError: You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided ['pixel_values']
|
|
0
|
192
|
February 4, 2025
|
|
BPEtokenizer reports error "not valid UTF-8" when processing txt file
|
|
6
|
253
|
January 29, 2025
|
|
NLLB tokenizer multiple target/source languages within a training batch
|
|
5
|
1791
|
January 10, 2025
|
|
Creating a Custom Token Vocabulary for GPT-2
|
|
1
|
775
|
January 7, 2025
|
|
'NoneType' Object Error and Token Authorization Issue in OpenAI API Integration
|
|
3
|
305
|
January 4, 2025
|
|
Train_from_iterator throwing TypeError: expected string or buffer errir
|
|
2
|
82
|
January 3, 2025
|
|
What should be processing_class param value of Seq2SeqTrainer for VisionEncoderDecoderModel Finetuning?
|
|
0
|
61
|
January 3, 2025
|
|
Error creating custom pre_tokenizer
|
|
3
|
152
|
January 2, 2025
|
|
Adding special tokens to LEDTokenizer
|
|
0
|
77
|
December 25, 2024
|
|
Byte Level Tokenizer While Training
|
|
0
|
127
|
December 14, 2024
|
|
Tokenizer: what function removes spaces between '<' and '>'?
|
|
0
|
88
|
December 9, 2024
|
|
Convert huggingface tokenizer into sentencepiece format
|
|
1
|
761
|
November 27, 2024
|
|
Issue with Loading Custom Tokenizer: Tokenizer class BaseTokenizer does not exist or is not currently imported Error
|
|
6
|
1127
|
November 6, 2024
|
|
Generate tokenizer.json for Marian(Opus) MT
|
|
2
|
713
|
November 4, 2024
|
|
Tokenizer method inference
|
|
3
|
105
|
November 2, 2024
|
|
How to skip tokens from translation?
|
|
2
|
945
|
October 15, 2024
|
|
Error loading tokenizer: data did not match any variant of untagged enum ModelWrapper at line 1251003 column 3
|
|
3
|
4471
|
October 10, 2024
|
|
Authorization header is correct, but the token seems invalid
|
|
3
|
260
|
October 10, 2024
|
|
AutoTokenizer.encode with multiThread and mutliProcess
|
|
2
|
606
|
October 9, 2024
|
|
Trying to use AutoTokenizer with TensorFlow gives: `ValueError: text input must of type `str` (single example), `List[str]` (batch or single pretokenized example) or `List[List[str]]` (batch of pretokenized examples).`
|
|
11
|
21219
|
October 5, 2024
|
|
Help to choose decoder for devnagari ocr
|
|
0
|
46
|
September 20, 2024
|
|
Speed up tokenizer training
|
|
4
|
1826
|
September 17, 2024
|
|
Cannot load tokenizer for llama2
|
|
6
|
7369
|
September 13, 2024
|
|
What is based model of XLM-RoBERTa Tokenizer? SenetencePiece? XLNetTokenizer
|
|
0
|
82
|
September 12, 2024
|
|
Tokenization compared to sentencepiece
|
|
0
|
166
|
September 11, 2024
|
|
Tokenizer Error [AGAIN!]
|
|
0
|
123
|
September 10, 2024
|
|
Decoding sequence of tokens produces question marks instead of actual tokens
|
|
1
|
64
|
September 3, 2024
|
|
Chat_template is not set & throwing error
|
|
2
|
16684
|
August 31, 2024
|
|
Memory leaks when training Gemma or Phi 3 and 3.5 tokenizer
|
|
0
|
121
|
August 29, 2024
|
|
What does "trim_offsets" do in tokenizer post-processor?
|
|
0
|
107
|
August 25, 2024
|