After a lot of debug and analysis, I found the root cause of the problem.
It happened to be the input metadata.csv file’s encoding problem.
It was not load_dataset()'s fault.
While I am generating csv, I convert all content into lowercase using Turkish locale.
Interestingly Turkish upper case I, which is Ä°, uppercase I with a dot on top of it, was causing the problem. While converting Ä° to i, it is adding a diacritic dot on the next character.
That was really very strange
Any ways, I think the problem will be solved after a new training from scratch.