How new token embedding will be learned through transformer?

MahdiA February 7, 2022, 1:35pm 1

How a new token will be learned through transformer?
When we apply tokenizer.add_token([list_tokens]) what will be the embedding of those new tokens?

Topic		Replies	Views
Add new tokens and learn the embeddings of the new tokens and keeping all the other parametes frozen 🤗Tokenizers	0	380	April 30, 2021
Train a new tokenizer from scratch 🤗Transformers	4	1310	November 10, 2020
Adding a new mask_token for BERT-like models/tokenizers Intermediate	0	364	May 26, 2023
Training Transformer XL from scratch Beginners	0	789	May 22, 2021
Extracting token embeddings from pretrained language models Beginners	8	14808	August 11, 2022

How new token embedding will be learned through transformer?

Related Topics