Hi,
I am using this model for translation task.
My corpus has a lot of numbers and they eat up the 128 tokens per sentence limitation pretty quickly. I want to handle the number translation by a separate function, and let the transformer handle the main text, using this procedure:
Step 1: Suppose the original sentence is “I have five chickens”
Step 2: Replace the number “five” with a special token. Here I use ℑ to represent the special token
Step 3: Translate the sentence with Transformer
I have ℑ chickens → j’ai ℑ poules
Step 4: Special function translate “five” to “cinq”
Step 5: Insert “cinq” back to the special token ℑ and get back j’ai cinq poules
I tried to do this in the model, but the special token just disappear in the final translation
i.e., in step 3, instead of getting j’ai ℑ poules, I get j’ai poules
Please help, thanks.
R T