Recursion in LLM's

Try the following instruction in any LLM: “Write a sentence that indicates exactly how many words that sentence contains”, check the result, if you get a correct answer ask the LLM the following: “What is the shortest sentence of this type that can be constructed”.
You can see that it is too common to find errors in the answer. It is interesting to ask the LLM to show in detail how it counts the words.
I have the intuition that in applied mathematics for the construction of answers there is a certain “blind spot” regarding the construction of concepts that require recursive reflection, if this is so, it could mean a great difficulty in the approaches towards self-aware applications or similar.
Opinions please

2 Likes

As far as I understand about LLMs, they don’t directly process words as we perceive them. Instead, they work with tokenized text, which breaks language into smaller numerical units called tokens. A token might be a whole word, part of a word, or even a punctuation mark. For example, I once asked an LLM to generate a text of 512 tokens and then manually counted the words—it turned out to be 443 words.

This highlights that the model’s internal representation (tokens) doesn’t directly correspond to human word counts. So, tasks like counting ‘words’ might seem nonsensical because the model is reasoning about text through tokens, which may not map 1:1 to words. This mismatch could partly explain why LLMs struggle with such tasks.

1 Like

Thank you for the comment. I instructed the models to explain how they count words and it is concluded from the answers that they adequately distinguish the concept of word and that they can count words properly. In fact, when the model performs the exercise of showing me how it counted, it notices its error. It seems that it is complex for the algorithms developed to resolve recursive situations, I postulate that this is due to the restrictions imposed by the basic mathematics they use.

1 Like

What are your qualifications brother? Or you are just a interested person like me? You are associated with any group or org? you seem observer.

1 Like

I am an engineer by profession and my interest in mathematics and the development of artificial intelligence is purely personal. I do not belong to any organization, I am only looking to share a discovery that I found interesting and to hear the opinions of people with greater knowledge than mine on the range of possibilities that arise from mathematics applied to LLMs.

1 Like