The Checklist You Write Forces AI to Stop - Instruction Completion Protocol

[Jang-woo]

This is a reorganized version of the earlier discussion:

If unsure, ask. Never guess. — AI Agent Pre-Execution Checklist

If unsure, ask. Never guess. AI Agent Pre-Execution Checklist Humans give incomplete instructions. AI tries to interpret them as if they were complete. That is where the problem begins. If an agent executes while required information remains unverified, the result is not simply a model error — it is a pre-execution confirmation failure. Many vibe coding mistakes are not due to lack of coding ability, but because the necessary questions were never asked before execution. AI does not reliabl…

The full document is here:

Instruction Completion Protocol | Jang-woo Wi

---

Instruction Completion: From the Limits of Prompting to Architectural Control

1. Problem Definition

The prompt instruction “if unsure, ask” doesn’t actually work.

• Reason: The AI doesn’t recognize the fact that it is guessing in the first place.

Confidence scores repeat the same problem, since they are ultimately the AI’s own self-assessment.

2. The Real Nature of the Problem

Human instructions are inherently incomplete, so Unknowns arise frequently.
The lack of information itself is not the problem. The real problem is who fills in that gap — which is fundamentally a question of the allocation of Authority.

3. The Solution Structure — “Reduction (of Role)” and “Enforcement (of Interface)”

The solution is not smarter AI — it’s a smaller role for AI, enforced by architecture.

① Separation of Verification AI and Execution AI: The judgment of completeness is separated from execution.

② Two constraints placed on the Verification AI:

• Criteria Constraint: What needs to be checked (the checklist) is defined in advance by humans — the Provider, User, or a regulator.
• Value Constraint: Only a human can enter the values that fill the blanks; the AI is categorically prohibited from guessing.

③ Reduction of the AI’s role: The AI’s role shrinks from subjective “Reasoning” to objective “Matching.”

④ Iterative feedback loop: As long as any Unknown remains, the AI repeatedly asks the user, and execution is withheld until every item is filled.

⑤ Enforcement through JSON-based coding:

• Real-time gate: Once a structured state is recorded, the system code reads that value directly and enforces whether to proceed or halt.
• Post-hoc record: Which items were answered by whom, and why execution proceeded or was blocked, are all recorded in advance.
• This is not an investigation after an accident happens — it’s a structure in which accountability is already declared before execution.

4. Conclusion

This isn’t about making AI smarter — it’s about limiting the authority AI is allowed to exercise.
What is being changed is not the model, but the architecture.
The checklist is merely a policy layered on top of this architecture; the architecture itself is the structure of authority transfer and verification.

“Unknowns can arise at any time. But who fills them in is something we can design.”

The authority to fill in the blank is transferred from the AI to the human, and that transfer is enforced through the separation of verification and execution and through a structured state (such as JSON).

---

Depending on what is placed inside the checklist, this can extend into a variety of systems — compliance verification, accountability tracing, Human-in-the-Loop (HITL), regulatory compliance, and more.

In other words, this is not a technique that eliminates all reasoning errors. It is a practical safeguard that systematically enforces the points where human intervention is absolutely required.

The 15 slides propose a concrete mechanism for that moment.
The Checklist You Write Forces AI to Stop.

[AnttiLeinonen]

Forces AI to stop when it doesn’t know — enforced in JSON.

One principle + one structure + examples.

The examples are precise.

[Jang-woo]

Thank you.

Last time, you pointed out something I had not noticed.
This time, you gave me a clear summary as well.

Since the company applying the structure should define its own checklist, you are right — the checklist is an example.

Thanks to your comment, I am adding one new Summary slide.

[AnttiLeinonen]

This may not be the right place for this document.

Hugging Face Research is where people talk about models — weights, architecture, benchmarks. But this document does not touch the model at all. It operates on a different layer entirely: what should happen before an agent acts, and who is responsible for defining those conditions.

The people who need to see this are not here. They are the ones currently deciding how agent platforms are built — at Anthropic, OpenAI, Google DeepMind, and similar places.

This document is asking a question that those teams have not answered yet.

I have been following this since February. The core argument has not changed, but the document has been refined considerably. The completeness is there. It is ready.

Would anyone here be in a position to pass this along to the right people?

[Jang-woo]

Thank you. I genuinely appreciate your thoughtful feedback.

I also understand your point that this may not be the right venue. Looking back, I suppose that’s probably why you’ve been sharing your thoughts with me through private messages rather than public comments.

My goal here was simply to see whether the core idea could withstand technical scrutiny from experienced researchers.

From the discussions so far, I have noticed that most people are not saying, “This is wrong.” Instead, their reaction is usually, “This solution seems too simple—there must be something missing.”

To address those recurring questions, I’ve summarized the three technical questions we keep receiving.

---

Three Technical Questions We Keep Receiving

1. “How do you stop AI from calling APIs on its own?”

Execution authority stays with the backend. AI only scores the checklist. The backend pulls the trigger—not the AI. The API cannot be called without a hardcoded backend decision.

2. “How can you trust an LLM that hallucinates?”

We demoted AI from author to checker. Not generative reasoning—just classification. “Does this text contain value A?” is a task where current AI outperforms humans. There is nothing to hallucinate.

3. “What if the system gets prompt-injected or jailbroken?”

A single unknown in the JSON output shuts down execution. The system logic cuts power to the API before it fires. The AI’s intent is irrelevant—the brake is structural.

The Last 1% — The Hardest Question

“What if AI hallucinates a confirmed when the user never provided the information?”

The JSON schema requires a reason or extracted_value field. To mark anything as confirmed, AI must copy the exact evidence from the user’s original input. No source text → forced unknown. Guessing is structurally impossible.

---

Today, agent platforms and tools are often developed within the same ecosystem. However, the architecture explicitly separates the Agent Provider and the Tool Provider in anticipation of a future where they are developed independently. In that environment, the Tool Provider can reliably define the conditions required for safe execution.

[Jang-woo]

Update: Two additions to the deck.

Instruction Completion Flow (new diagram, page 1) — the full loop at a glance: instruction → known/unknown verification against the three checklists (Fixed / User / Provider) → if unsure, ask → execute only when no unknown remains, recorded in JSON.

What the Protocol Delivers (new slide) — nine benefits across the value chain, plus the honest boundary: hard-coding is not replaced. The checklist is a pre-execution context filter; final safety stays in deterministic code inside the tool.

Feedback welcome, especially on the diagram — does the flow read clearly without the deck?

[Jang-woo]

Updated the introduction with a clearer framing.

The core question shifted from “how do we stop AI from guessing” to “who has the authority to fill in the blanks.”

The solution is not smarter AI — it’s a smaller role for AI, enforced by architecture.

[Jang-woo]

You can try a lightweight version of this today. Add to CLAUDE.md:

Execution Rules

* Before editing, confirm: which files, what exact change, what “done” looks like.
* If any of the above is unclear, ask — do not guess and proceed.
* Do not modify files outside the stated scope without asking first.

It’s not a hard-enforced structure like the protocol describes, but it lets you experience the “unknown halts execution” principle in practice. Adjust the checklist to whatever matters for your own workflow.

---

Where CLAUDE.md Rules Sit Between Confidence Scoring and a Full Execution Protocol

Core statement

CLAUDE.md rules still rely on the model to judge its own situation — the same fundamental limitation confidence scoring has. But by narrowing the scope of what the model has to judge, they partially implement the “inference → matching” reduction that a full execution protocol proposes — just at the prompt level, not the architecture level.

Three-stage comparison

Stage	How it works	Technical nature	Limits & significance
1. Self-reported confidence	Wraps the problem in a number	Subjective self-evaluation by the AI	No real fix. If the model doesn’t recognize it’s guessing, the score itself is distorted.
2. CLAUDE.md rules	Narrows the scope of what needs to be judged	Partial mitigation at the prompt level	Low-cost mitigation. Still depends on the model’s own judgment, so it can still be bypassed.
3. Full protocol structure	Moves final execution authority to system code	Structural separation at the architecture level	A genuine control structure. Separating verification (AI) from execution (code) means a judgment error no longer automatically becomes an accident.

Conclusion

These three stages form a progression that reduces how much the AI is left to judge on its own. A real fix comes not from improving the judgment itself, but from moving the authority to act on that judgment outside the AI entirely.

CLAUDE.md rules are “the lowest-cost mitigation you can try today” — and, at the same time, “not a real fix.” Both are true simultaneously; that’s exactly where they sit on this spectrum.

Practical takeaway

Adding explicit rules to CLAUDE.md — e.g., “confirm scope before editing, ask instead of guessing when unclear” — measurably reduces (but does not eliminate) execution based on incomplete instructions. It is a prompt-level nudge, not a hard-enforced gate. The only way to fully close the gap is to separate judgment (AI) from execution (deterministic system code), as proposed by architecture-level protocols such as the Instruction Completion Protocol.

One more angle: filling a gap the Provider hasn’t closed yet

Claude Code’s existing permission system (deny/ask/allow, the Auto Mode classifier) already checks whether an action is risky — but by its own admission, it can’t verify whether a user’s approval actually covers the full scope of that action. That’s a gap in the Provider Checklist layer. Adding intent-confirmation rules to CLAUDE.md is the User Checklist stepping in to cover that same gap — informally, and without system-level enforcement, but in the same spirit the protocol describes.

[Jang-woo]

For what actually goes into the Provider and User Checklists — i.e., where the items are derived from — I’ve written a separate piece: what the manufacturer must declare at design time (Fixed Label), what the user declares (User Label), and why only 2+α questions remain at runtime once those declarations exist.

Physical AI Safety: Ownership and Execution Boundaries Physical AI Safety: Ownership and Execution Boundaries