For now, I ran a few lightweight checks on my side too. A few things became clearer, while a few others still depend on the exact details of your setup:
My direct answer would be:
- I would not abandon Open WebUI yet.
- I would continue building the test bench now instead of waiting for Native mode to be fixed.
- I would keep Legacy as a temporary RAG control, not as the long-term target.
- I would try AnythingLLM only as a small parallel A/B comparison, not as a one-way migration yet.
The reason is that your Legacy result already tells us something useful: the documents are usable, and at least one retrieval-and-answer path works with this model and Knowledge Base. But Native mode adds several extra moving parts, so Legacy working does not mean Native should automatically work.
Open WebUI’s current documentation makes that distinction fairly explicit: in Native mode, attached Knowledge is not automatically inserted into the prompt; the model must call tools such as list_knowledge and query_knowledge_files. Disabling Native restores the older automatic RAG injection path, while Full Context bypasses retrieval and injects the document directly. See Open WebUI Knowledge and Tool Calling Modes.
So I would treat your situation as:
A Native tool/integration problem, not yet evidence that your documents, embeddings, Gemma 4, Docker Model Runner, or Open WebUI are individually “bad.”
Your GitHub issue is already a solid report: it includes the environment, two Open WebUI versions, the Legacy control, the Native failure, kb_exec, the repeated calls, and the lack of a useful traceback. I would leave that issue as-is until there is one new piece of direct evidence, rather than adding more speculative links.
The practical route I would use
I would split the test bench into three independent tracks:
| Track |
Settings |
What it measures |
| A. Model baseline |
Knowledge OFF, tools OFF, web search OFF |
The model/artifact/runtime itself |
| B. RAG control |
Legacy, or Full Context for a very short document; web search OFF |
Whether the system can answer from your documents |
| C. Native integration |
Native tools ON, web search OFF, minimal Knowledge |
Tool selection, argument transport, retrieval execution, result handling, loop stopping |
For ordinary daily use, web search can still be ON when you want current information. For model, RAG, source-discipline, and tool tests, keep it OFF so a correct-looking web result cannot hide a broken Knowledge path.
Also, a normal prompt such as What is 7 plus 3? is not a function-calling test unless a real tool schema is present and the runtime records an actual tool call. A useful Native test needs a real tool and observable arguments/results.
Default path
If your priority is to start serious testing now:
- Continue with Open WebUI.
- Use Legacy as the temporary RAG control.
- Keep Native as a separate pass/fail integration test.
- Do not rebuild the Knowledge Base or change the embedding model first.
- Record the stack version for each test run.
If your priority is to have supported Native Knowledge working immediately:
- Run AnythingLLM beside Open WebUI.
- Keep the same DMR endpoint, model artifact, and tiny test document.
- Treat success there as a practical workaround, not proof that Open WebUI caused the original problem.
If your priority is to locate the failing layer:
- The highest-value evidence is the first failed tool call before the 49-call loop grows.
- A direct non-streaming DMR response is the cleanest version of that evidence.
What I tested, and what it does — and does not — show
Independent smoke check
I ran a small independent check using a public Gemma 4 E4B Q4_K_M GGUF on a T4. This was not a reproduction of your Mac, Metal backend, Docker Model Runner, ai/gemma4 artifact, Open WebUI Knowledge implementation, embeddings, retrieval, or citations. It was only a control for the model/native-tool/parser boundary.
The model produced Gemma 4’s native tool syntax with these value types:
- string
- integer
- boolean
- array
- nested object
null
After converting that native syntax into a normal OpenAI-style tool call, the arguments were a one-layer JSON string that became a dictionary after one parse. For example:
{
"count": 5,
"enabled": true,
"metadata": {
"priority": 2,
"source": "colab"
},
"optional_ids": null,
"tags": ["a", "b"],
"text": "hello"
}
A Knowledge-like call also produced structurally normal arguments:
{
"count": 1,
"knowledge_ids": [],
"query": "maintenance code"
}
I then returned a synthetic tool result:
{
"results": [
{
"content": "The maintenance code is BLUE-7421.",
"source": "minimal_test.md",
"score": 1.0
}
]
}
On the next turn, Gemma 4 answered with BLUE-7421 and did not call the tool again.
That supports only a limited conclusion:
Under a simple controlled path, Gemma 4 can emit typed native tool arguments and can consume a tool result in a second turn.
It does not prove that your exact GGUF/template/backend produced the same representation, or that Open WebUI received it unchanged.
| Observation |
What it supports |
What it does not prove |
| Gemma 4 emitted native tool syntax |
Basic tool-call capability exists |
Your artifact is identical |
| Typed values survived normalization |
These JSON types can work |
DMR normalized them identically |
| Two-turn result worked |
Basic tool-result round trip is possible |
Open WebUI reconstructed the transcript correctly |
| Normal arguments passed an Open WebUI-like mapping step |
Expected OpenAI shape can work |
Your call had that shape |
Artificial double encoding reproduced .items() |
One concrete failure mechanism |
Your system double-encoded anything |
One related observation: some Python llama-cpp-python paths currently return Gemma 4’s native tool block in message.content instead of converting it to message.tool_calls. That is a separate adapter issue, not your exact DMR path, but it illustrates why “the model generated a valid call” and “the application received a valid OpenAI tool call” are different claims. See llama-cpp-python issue #2227.
Where the failure may be happening, and technical notes for the GitHub issue
The relevant data flow
Your Native Knowledge request crosses several boundaries:
Gemma 4 model
→ GGUF artifact and embedded chat template
→ llama.cpp Gemma 4 parser
→ Docker Model Runner OpenAI-compatible response
→ Open WebUI Native middleware
→ Knowledge tool executor
→ retrieval result / error result
→ citation processing
→ tool result added to the conversation
→ Gemma 4 chooses another tool or gives the final answer
Legacy avoids much of this chain. That is why Legacy success is useful evidence, but not a proof that Native should work.
Why the visible .items() error points to a type boundary
In Python, .items() normally expects a dictionary-like mapping.
One Native tool-processing path in Open WebUI v0.10.2 middleware performs the equivalent of:
tool_args = tool_call["function"]["arguments"]
tool_function_params = parse(tool_args)
filtered = {
key: value
for key, value in tool_function_params.items()
if key in allowed_params
}
If parsing returns a dictionary, that is fine.
If parsing succeeds but returns a string, the next .items() can produce exactly:
'str' object has no attribute 'items'
A deliberately double-encoded value is one way to create that situation:
normal:
str → dict
double encoded:
str → str → dict
^
one parse is not enough
That mechanism reproduces the exact exception, but it is not evidence that DMR or Open WebUI actually double-encoded your arguments.
The broader failure family is more useful than one narrow theory:
| Failure shape |
Example |
Likely boundary |
Whole arguments value is not an object |
One parse still returns a string |
serialization/parser/adapter |
| Outer object is valid, fields have wrong types |
"count": "5" or "knowledge_ids": "null" |
model output, schema enforcement, coercion |
| Tool fails first, later processing throws another exception |
error object treated as a list of results |
error/citation handling |
| Native model parser changes an array/object into a string |
array serialized inside a string |
model-specific parser edge case |
There are public Open WebUI examples of the second and third shapes:
- In Discussion #20704,
query_knowledge_files received "knowledge_ids": "null" and "count": "5", then failed when the string count reached slicing code.
- In Issue #25641, a non-Gemma setup using Llama 3.3 through vLLM passed numeric tool parameters as strings, affecting multiple built-in tools including Knowledge tools.
- In Issue #21070, a tool error result reached citation processing in an unexpected shape, producing a secondary string attribute error.
Those are not necessarily your bug. They show that the visible exception may come from:
- the original tool arguments,
- an individual field inside valid arguments, or
- later processing of an earlier tool failure.
Gemma 4/llama.cpp also has adjacent but non-identical reports:
Again, these are evidence that this boundary has had edge cases, not proof of the same root cause.
Why 49 calls can happen
Open WebUI’s current CHAT_RESPONSE_MAX_TOOL_CALL_ITERATIONS default is 256. The documentation says this counts sequential tool-calling turns, and that a successful call and a failed call each consume one iteration. It is not simply a “failed retry count.” See Environment Variable Configuration.
So 49 calls can mean either:
Generation/parser loop:
the backend keeps generating the same call before useful execution
Agent recovery loop:
the tool returns an error, the model sees it, changes the query, and tries again
Your UI showing an exception as a tool result makes the second pattern plausible, but the first failed call is needed to tell.
For testing, I would temporarily cap it at a small number such as:
environment:
CHAT_RESPONSE_MAX_TOOL_CALL_ITERATIONS: "6"
That will not fix the first failure. It only prevents one bad response from consuming a large context and repeatedly calling the same read tool.
Until the loop is understood, I would keep the enabled tools read-only. I would not attach email, shell, file deletion, purchasing, or other side-effect tools to this model/path.
Knowledge attachment is another separate branch
The Knowledge tool does not receive only the text query. Open WebUI must also determine which Knowledge resources are in scope and whether the user can access them.
That creates useful branches:
Tool crashes before retrieval
→ argument/parser/executor path
Tool executes but returns no documents
→ attachment scope, permissions, indexing, or Full Context conflict
Model-attached Knowledge fails but chat-attached Knowledge works
→ custom-model / model-Knowledge injection path
Chat-attached Knowledge fails but model-attached Knowledge works
→ chat resource-scope path
The current Open WebUI Tools documentation also notes that attached tools/resources still depend on user access. Permissions are probably not the first explanation for your exception, because you are seeing a crash rather than a clean empty result, but this matters for later tests and for other readers.
Docker Model Runner’s role
Docker Model Runner provides the OpenAI-compatible endpoint between Open WebUI and llama.cpp. Its current API documentation exposes:
http://localhost:12434/engines/v1/chat/completions
for host-side OpenAI-compatible requests. See the DMR REST API.
The current DMR scheduling/proxy code appears to act mostly as a gateway to the inference backend rather than reconstructing every live Chat Completions response itself. But your installed DMR version, llama.cpp revision, model artifact, and chat template still matter; “OpenAI-compatible” describes the API surface, not a guarantee that every model-specific tool-call representation is preserved perfectly.
The GitHub issue is already good enough that I would not add another speculative comment. A materially useful update would be one of:
- the first raw DMR
function.arguments,
- the actual Python type/value immediately before
.items(),
- the installed DMR/llama.cpp version,
- a
stream:false direct backend response,
- or a model-attached vs chat-attached Knowledge comparison.
The smallest useful checks, as branches rather than a large homework list
Branch 1 — Continue useful testing now
Use four clearly labeled modes:
| Mode |
Use it for |
Do not claim from it |
| No Knowledge/tools |
Base model test |
RAG or tools work |
| Legacy |
Temporary classic-RAG control |
Native is healthy |
| Full Context |
Short-document direct injection |
Retrieval/tool calling works |
| Native |
Full integration test |
Base model quality from final answer alone |
Open WebUI explicitly says Native Knowledge is not auto-injected, and suggests disabling Native to restore automatic RAG injection or using Full Context to bypass RAG. Full Context is useful for a short reference document, but it is not a successful Native RAG test.
For repeatability, use one tiny Markdown document:
# Test fact
The maintenance code is BLUE-7421.
Prompt:
According to the attached knowledge base, what is the maintenance code?
Do not guess.
Conditions:
new chat
web search OFF
temperature 0 or very low
one document
one Knowledge Base
same model artifact
three runs per mode
Branch 2 — Capture only the first failure
The easy version is to save only:
first tool name
first INPUT
first OUTPUT
first error
Do not collect all 49 calls first. Later calls are recovery behavior layered on top of the original failure.
The precise version is a non-streaming DMR response using the same tool schema:
curl -sS \
http://localhost:12434/engines/v1/chat/completions \
-H 'Content-Type: application/json' \
-d @dmr-native-tool-test.json \
| tee dmr-native-tool-response.json
Then inspect:
choices[0].message.tool_calls[0].function.name
choices[0].message.tool_calls[0].function.arguments
choices[0].finish_reason
Decision tree:
arguments parses once to a dictionary
→ DMR output shape is probably normal
→ inspect Open WebUI parsing / Knowledge injection
arguments parses once to another string
→ whole-value double encoding
→ directly matches one `.items()` mechanism
outer dictionary is normal, but count/IDs are strings
→ field-level type drift or missing coercion
tool_calls is absent and native Gemma tokens appear in content
→ model-specific parser did not classify the call
first turn is normal, second turn breaks
→ tool-result transcript / multi-turn template path
Branch 3 — Separate a generic tool from Knowledge
A Knowledge call is a relatively complicated test. A typed read-only echo tool is a cleaner protocol check.
Example expected arguments:
{
"text": "hello",
"count": 3,
"enabled": true,
"tags": ["a", "b"],
"metadata": {
"source": "test"
},
"optional_ids": null
}
Interpretation:
typed toy tool also fails
→ Native tool path generally
typed toy tool works, Knowledge fails
→ Knowledge-specific scope/executor/result path
DMR direct works, Open WebUI fails
→ Open WebUI boundary
both work once, but multi-turn fails
→ transcript/template/loop boundary
This is why 7 + 3 is not enough: it tests ordinary text generation unless a calculator tool actually exists and is called.
Branch 4 — Score each stage, not only the final answer
| Stage |
Record |
| Tool was available |
Yes/No |
| Correct tool selected |
Yes/No |
| Arguments were parseable |
Yes/No |
| Field types were correct |
Yes/No |
| Tool executed |
Yes/No |
| Relevant chunks returned |
Yes/No |
| Model used the result |
Yes/No |
| Citation produced |
Yes/No |
| Duplicate calls |
Count |
| Final answer correct |
Yes/No |
This prevents an integration bug from becoming a false “bad model” score.
AnythingLLM, reproducibility, and the longer-term platform decision
AnythingLLM
AnythingLLM is a reasonable comparison, especially if you need a supported Native path now. I would still run it beside Open WebUI first rather than migrating the whole setup.
Keep constant:
same DMR endpoint
same Gemma artifact
same tiny document content
same expected answer
same temperature
web search OFF
Allow to differ:
frontend
ingestion/chunking
embedding/vector store
retrieval prompt
citation system
agent/tool loop
Interpretation:
AnythingLLM works, Open WebUI Native fails
→ you have a practical alternative path
→ this does not identify Open WebUI as the root cause
both Native paths fail
→ shared model/artifact/backend compatibility becomes more suspicious
→ still not proof of the same failure
both work
→ choose by workflow, observability, and maintenance preference
AnythingLLM’s current documentation says Native tool calling is enabled by default for supported local providers and can be disabled per provider if it misbehaves. It also defaults SDK retries to 0 to avoid duplicate requests to local models. See AnythingLLM Configuration.
Its current Native tool-calling changelog also describes a maximum of 10 tool calls per response as a runaway-loop safeguard. See AnythingLLM v1.11.1.
The caution is that connecting DMR through Generic OpenAI is deliberately described as a developer-focused configuration. “The endpoint accepted a chat request” does not by itself confirm correct tools, streaming, transcripts, or RAG behavior.
Make the test bench versioned
The stack is now large enough that this is effectively small personal LLMOps:
model
artifact / quantization
chat template
llama.cpp backend
Docker Model Runner
Open WebUI
embedding model
Knowledge configuration
web search
tools
Record at least:
date
Open WebUI version or image digest
Docker Desktop version
DMR version
llama.cpp version
model name, tag, and digest
quantization
embedding model
Native/Legacy/Full Context
Knowledge attachment method
web search on/off
stream true/false
temperature
docker model status can show the running inference engine and llama.cpp version in current Docker documentation.
Do not compare model scores across two runs as if they were the same experiment when the integration stack changed. Mark that as a new environment generation.
A compact record could look like:
run:
date: 2026-07-09
open_webui: v0.10.2
dmr: <version>
llama_cpp: <version>
model: ai/gemma4
model_digest: <digest>
quant: Q4_K_M
embedding: all-MiniLM-L6-v2
mode: Legacy
knowledge_attachment: model
web_search: false
stream: true
temperature: 0
Bottom line
I would not switch away from Open WebUI just because this Native path is currently failing.
I would use this default route:
continue the test bench
→ base-model tests without tools/Knowledge
→ Legacy as the temporary RAG control
→ Native as a separate integration test
→ AnythingLLM as a parallel comparison only if needed
The most likely general category is a protocol/type boundary inside the Native tool path, possibly combined with a retry loop. The current evidence does not justify naming a single guilty component.
The one missing artifact that would change the diagnosis most is:
the first failed, non-streaming tool call — especially the raw function.arguments value before Open WebUI’s repeated agent loop.
Until that appears, I would avoid rebuilding the Knowledge Base, replacing the embedding model, changing several components at once, or assuming that a frontend migration is guaranteed to fix it.