Qwen 2.5 coder 7b can’t use correct separator between input_prompt and generated tokens in FIM format. For example, in FIM format, if I use prompt = ‘<|fim_prefix|>’ + ‘def’ + ‘<|fim_suffix|>’ + ‘return 5’ + ‘<|fim_middle|>’ model can generate something like 'func()\n ’ without separator between prompt tokens and generated tokens and due to that I got code ‘deffunc()’ that got syntax error. The problem not only in a whitespace, it could be also missed tabulation, comma, new line e.t.c. How to fix that problem?
1 Like