Apply PEFT on ViT

🤗Transformers
1

For my research I am experimenting with Prefix tuning applied on Vision Transformers, @sayakpaul Could you give some pointers to implement changes in ViT module.

Thanks