I want to use an encoder-decoder model, but I want to perform masked language modeling and casual language modeling (unsupervised learning) on the encoder-decoder so it understands the context I am using, before fine-tuning it. How best to go about this?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Any language model which utilizes both encoder and decoder output for multi-task learning? | 0 | 228 | July 17, 2023 | |
| Fine-tuning Decoder-only or Encoder-Decoder models for classification | 0 | 524 | July 17, 2024 | |
| Using encoder and decoder portion separately from encoder-decoder | 1 | 438 | October 4, 2023 | |
| Separate pre-trained encoder and decoder | 0 | 431 | October 4, 2023 | |
| Could I inference the Encoder-Decoder model without specify "decoder_input_ids"? | 4 | 2439 | May 1, 2021 |