Hi,
I’m using internally to separate distinct sequences (usually to represent a line break).
I’m wondering if I can use such a token in my text sequences as input for mBart models.
I didn’t managed to find whether such a token might be interpreted as an end-of-sequence as it can be the case in some seq2seq library.
Thanks.
EDIT: it’s not a problem if its not represented as a single token in the model (probably cut into <
+ /
+ s
+ >
?). It should not be “special” mbart wise i.e. it should be generated as regular tokens.