You might need to modify the encoder (and/or) the decoder in this:
https://huggingface.co/transformers/_modules/transformers/models/bart/modeling_bart.html#BartModel
Copy-paste the entire file, save it locally as âmodifiedbart.pyâ, and call it instead of the default âBARTModelâ in transformers.