[image] google/reformer-enwik8 · Hugging Face We’re on a journey to advance and democratize artificial intelligence through open source and open science. It seems that the language model does not have a token Is this correct?

Reformer-enwik8 do not have <EOS> token

Flash321 July 28, 2021, 12:34pm 1

It seems that the language model does not have a token Is this correct?

Topic		Replies	Views
Reformer-enwik8 output does not seem to make sense Models	1	493	July 28, 2021
Chatbot using Reformer model. Not able to find "token_type_ids" like BERT 🤗Transformers	1	626	September 24, 2020
Questions on model's tokens 🤗Tokenizers	0	607	March 24, 2021
Tokenizer not found 🤗Tokenizers	0	325	August 18, 2020
Is "EOS token" mandatory for T5 model in text classification task Beginners	0	694	October 10, 2021