Hi @joaogante
Thanks for this idea! I solved it by downgrading the transformers library to version 4.20.1 as XLA was introduced in version 4.21.0 (Release v4.21.0: TF XLA text generation - Custom Pipelines - OwlViT, NLLB, MobileViT, Nezha, GroupViT, MVP, CodeGen, UL2 · huggingface/transformers · GitHub) - as you pointed out. Seems to be working fine for now.
It would be great though if we could enable or disable XLA using a flag or something - for those of us who use a M1/M2 Mac with no XLA support yet…