Tokens limit in Web Browser Automation

tgaleev · March 28, 2025, 6:05am

Hello. I often face the problem of token limits when using this code:

Error message:
422 Client Error... Input validation error: 'inputs' tokens + 'max_new_tokens' must be <= 131072. Given: 270767 'inputs' tokens and 0 'max_new_tokens'

I suspect the issue lies in the browser screenshot function — it seems to generate too many tokens per screenshot. How can I resolve this?

John6666 · March 28, 2025, 6:20am

Oh… Unresolved issue…

github.com/huggingface/smolagents

SmolAgents AgentMemory is kept in Gradio global state and cannot be turned off.

opened 11:34PM - 18 Feb 25 UTC

ddewaele

bug

**Describe the bug** When launching the agent in Gradio `GradioUI(agent).launch(…)`, the agent is kept in the global state of Gradio, meaning all users share the same agent. Although messages seem to be stored in Gradio Session state, some internal agent state like AgentMemory is saved globally I would imagine that it would be better to scope everything on Session State. There is a `reset_agent_memory` flag on SmolAgents that is False by default, but there is no way to override it. **Code to reproduce the error** Open a chat session in a gradio space and keep talking until you hit the context window limit. Open a new chat session in a different browser. You will not be able to have a conversation due to the context window limit error that was triggered by the first chat session. The global agent memory will consume all of the context size. From that point on the space will be broken for everyone and nobody will be able to start conversations again. **Error logs (if any)** All users will experience an error like this ``` Input validation error: inputs tokens + max_new_tokens must be <= 16000. Given: 43384 inputs tokens and 2096 max_new_tokens Make sure 'text-generation' task is supported by the model. ``` **Expected behavior** I would expect that there are proper user scoped sessions. And that one user session cannot influence the behavior of another session. **Packages version:** `smolagents==1.9.2` **Additional context** This issue became clear during the HuggingFace Agents Course, when students were asked to promote their Agents through their spaces. Because the underlying LLM had a small context window, as soon as 1 person hit that max context size the space was ruined for everyone. HuggingFace provided a template https://huggingface.co/spaces/agents-course/First_agent_template for students to create agents and tools. That template contains a copy of `Gradio_UI.py` where we can manipulate the `reset_agent_memory` flag, but I assume you want to keep `Gradio_UI.py` a part of SmolAgents and not have people create copies of it. Otherwise the only way to fix the issue is to restart the entire space

tgaleev · March 28, 2025, 6:45am

Got it, thanks!

Topic		Replies	Views
Chat Usage Error - "Input validation error" Models	0	620	May 2, 2024
How to increase max_new_tokens beyond 1200 in code llama Models	2	595	September 25, 2024
Problem for large context window (400k) Models	4	543	July 24, 2024
ValidationError: Max token limit(>=1) reached for finetuned models Amazon SageMaker	3	720	December 28, 2023
Validation Error: Meta-Llama-3-8B-Instruct Models	3	228	November 19, 2024

Tokens limit in Web Browser Automation

Related topics