How can train a POS model with XLNET?

Hi I am trying to tokenize and add POS tagging to a raw Japanese string, and I want to use XLNET for doing that. I can find the tokenizer here, but I cannot find one that I can use for POS tagging. Do I need to bridge with other models for doing that?

If you’re looking for a starting point, I think you can model this as a token classification problem and fine-tune with XLNetForTokenClassification head, I also found an implementation here for negation.

@merve Thank you so much for offering that as an example. A question popped up as I want to build a model from scratch. I have read the doc and I cannot find any option for feeding raw data into XLNetConfig, have I missed anything?