I am working on warm starting models for the summarization task based on @patrickvonplaten 's great blog: Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models . However, I have a few questions regarding these models, especially for Bert2Gpt2 and Bert2Bert models: 1- As we all…

Hi, looking at the files: Ayham/roberta_gpt2_summarization_cnn_dailymail at main It indeed looks like only the weights (pytorch_model.bin) and model configuration (config.json) are uploaded, but not the tokenizer files. You can upload the tokenizer files programmatically using the huggingface_hub …

Warm-started encoder-decoder models (Bert2Gpt2 and Bert2Bert)

Beginners

Ayham December 15, 2021, 6:34pm 8

Thank you @nielsr for your useful explanations and suggestions, I really appreciate it.

Topic		Replies	Views
Leveraging pre-trained checkpoints for summarization Models	33	3167	November 25, 2022
Training Bert2GPT2 model Summarization doesn't lead to acceptable results Models	0	453	December 8, 2021
Training issue of a Transformer based Encoder-Decoder model based on pre-trained BanglaBERT Models	1	745	May 12, 2022
BERT2BERT Notebook for Models without GenerationMixin 🤗Transformers	0	289	November 12, 2020
Bert2bert translator? 🤗Transformers	6	44	August 28, 2025

Warm-started encoder-decoder models (Bert2Gpt2 and Bert2Bert)

Related topics