If I wanted to summarize/classify an entire paragraph into a single output token, could an encoder+decoder model be fine tuned to produce such behavior? For example: I want to classify an email as ‘Work_Related’.

Hi @moyi-druzi , this should be possible. During training, you feed the emails into the encoder and provide the output token(s) as the labels for the decoder. E.g., you train a T5 decoder to generate something like “Work-related” or “Not work-related” and then immediately halt. You can also use an e…

Is Sequence to Sequence the best model for paragraph classification?

Beginners

system Closed January 14, 2024, 7:17pm 3

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.

Topic		Replies	Views
EncoderDecoderModel for token classification 🤗Transformers	0	194	October 29, 2022
EncoderDecoderModel converts classifier layer of decoder Beginners	2	534	October 26, 2021
Using T5 encoder with classification head Models	1	1885	July 17, 2022
T5forConditionalGeneration + classification Models	3	1286	December 13, 2020
Using encoder and decoder portion separately from encoder-decoder Beginners	1	448	October 4, 2023

Is Sequence to Sequence the best model for paragraph classification?

Related topics