Character level attention with Longformer for sequence classification

Hey guys :slight_smile: I am trying to figure out how to use the Longformer at the character level. It is mentioned in the paper also. I looked at the docs but I can’t find what I am looking for.

Can I just adjust my pre-processing so instead of tokenising:

β€œHello, I like cake!”

the input to be tokenised is something like:

β€œH” β€œe” β€œl” β€œl” β€œo” β€œ,” β€œI” β€œl” β€œi” β€œk” β€œe” β€œc” β€œa” β€œk” β€œe” β€œ!”

and then the tokeniser will assign ids to every character?

Thanks.