So I am trying to train an automated essay scoring system, that combines the loss of predicting scores with predicting whether a sentence is grammatically correct. To do this I have split each sentence in the essay with a sep and a cls token so that an essay is fed into Bert like this: essay 1 : [c…

Feed output from one transformer model as input to another

cameronstronge July 30, 2021, 9:16am 2

Is this possible or do I have to perform pooling at each sentence vector to get single value to represent that sentence and use that as an input?

Topic		Replies	Views
Extracting the output of hidden BERT layers and re-training the BERT model on custom datasets 🤗Transformers	0	811	March 17, 2021
How to use transformer attention model when the input is features Beginners	1	1238	October 12, 2020
Multiple texts as inputs to Transformers models 🤗Transformers	9	10024	September 13, 2024
Inquiry for adding a new layer for transformer model 🤗Transformers	0	159	August 18, 2023
Will it be learned properly if tokens listed in one dimension are reshaped in the form of (batch, seq_len) and inputted into the transformer xl model? 🤗Transformers	0	169	October 27, 2023