Hi @Luciano, I was able to solve this issue by truncating the tokenizer at (512-num_virtual_tokens which in my case was 20), so i truncated the tokenizer at 492 tokens instead of 512. With these changes it worked.
1 Like