Using BigBird with a custom classification head (Tensorflow)

Hi there,

I’m trying a classification task with various language models and a custom head - implemented in Tensorflow. I have tried Longformer thus far and I would now want to swap Longformer for BigBird to compare the results. However, BigBird does not have a tf implementation.

Is there a known way to make it work anyway (maybe with the flax version of the model) or would I need to replicate my tf head in PyTorch?

Any help would be appreciated!