BartForConditionalGeneration: Adding additional layers of embedding

Models
1

In this file:

https://github.com/huggingface/transformers/blob/main/src/transformers/models/bart/modeling_bart.py

I think-

Inside the BartEncoder, on this line, I have to add other embeddings

Could anyone tell me - are there any other places I need to modify to add additional layers of embedding (e.g., 2 other embedding with positional embedding)?

2

@ArthurZ Could you please help here?

3

Hey! Sorry for the late answer, I think your intuition is correct