tags:
- robotics
- action-calling
- physical-ai
- ai-safety
- responsibility-boundaries
- human-in-the-loop
- llm-agents
Author’s Note — Why This Document Exists
Twenty years ago, I was one of the engineers who worked on an early home gateway integrating Wi-Fi and Zigbee. While connecting devices one by one, a thought crossed my mind:
“Am I turning on Skynet?”
At the time, it felt like science fiction.
Today, as AI systems begin to directly control power and physical infrastructure, that question is no longer hypothetical.
Large Language Models can already call tools and trigger actions.
The remaining challenge is not reasoning, but responsibility.
This document is not about making AI more intelligent.
It is about defining the boundaries of responsibility.
It does not ask how to build better AI systems, but what should never be delegated to them.
Without clear ownership boundaries, making the physical world callable for AI is not progress.
This document shares that conclusion and proposes a minimal judgment interface not to limit intelligence, but to preserve accountability.
I am sharing this not as a finished standard, but as a question for discussion.
Where should these boundaries be drawn — and by whom?
Making the Physical World Callable for AI
From Tool Calling to Action Calling
Why LLMs don’t need more reasoning — they need ownership boundaries
LLMs can already call functions.
The real question is:
why can’t they call the physical world?
Abstract
Large language models (LLMs) already possess sufficient reasoning capability.
The reason AI cannot act in the physical world is not a lack of intelligence,
but the absence of clearly defined ownership and semantic units of action.
“We don’t need a smarter AI; we need an AI that knows its place.”
Most existing systems still model ON/OFF at the device level (device-centric).
However, what AI and users actually deal with are not devices, but actions.
This document preserves existing action forms (e.g., ON/OFF),
while focusing on two elements that already exist but have not been used correctly:
- Fixed Label : Manufacturer-declared essential boundaries and characteristics of an action
- User Label : The meaning and context of an action as approved by the user
This document proposes that:
- AI and UI should present and interpret actions, not device types.
- When a User Label exists, it must be displayed with highest priority.
- Only when no User Label exists should the manufacturer-defined Label or default device name be used.
We ask a simple question:
If the meaning of an action is already given,
why do AI and UI still judge solely by device names?
It is necessary to redefine when and on what basis AI should act,
from the perspective of ownership and approval.
1. Essence, Existence, and Ownership
Every physical device has a creator.
When a manufacturer creates a device, it defines its form, functions, limits,
and the conditions under which it can operate safely.
This definition constitutes the Essence of the device.
Existence is not state.
Existence is revealed only through action.
When a device is sold or transferred, ownership changes.
From that moment, the device is no longer under the creator’s control,
but is used within the owner’s life.
This is the device’s Existence.
The divergence between essence and existence is not an error,
but an inevitable outcome.
Devices are used in environments, purposes, and moments
that manufacturers could not have anticipated.
Any system that assumes essence and existence will remain aligned will fail.
That assumption itself is flawed.
1.1 Ownership Separates Control and Responsibility
Transfer of ownership creates a permanent separation.
- The manufacturer is responsible for what the device can safely do.
- The user decides why, when, and in what context the device is used.
This separation cannot be eliminated by specifications, state models,
or predefined scenarios.
This document accepts this separation
not as an error, but as a starting condition of design.
2. The Core Problem Is Not Reasoning, but Ownership
AI already reasons well.
What is lacking is not intelligence, but clearly defined authority boundaries.
When ownership is not clearly defined:
- AI must infer safety,
- AI must infer intent,
- Responsibility becomes ambiguous.
This document declares a single principle:
AI must not infer what it does not own.
3. Declarative Separation of Roles in AI Execution
The table below organizes ownership of actions and meaning.
It does not define implementation details or execution procedures.
Responsibility Matrix
(Non-normative ownership and responsibility mapping)
| Element | Owner | Scope |
|---|---|---|
| Action | Manufacturer | Physical execution itself |
| Fixed Label (Essence) | Manufacturer | Safety, limits, physical characteristics, emergency behavior |
| Label (Action name) | Manufacturer | Default meaning of the action |
| User Label / Approved Intent | User | Context, meaning, importance |
| Intent (Execution authority) | User | Execution approval |
| Mapping | AI | Connecting approved language to actions |
- Manufacturer (Provider) : Guarantor of physical limits and safety
- User (Owner) : Sovereign of meaning and approval
- AI (Agent) : A faithful bridge between the two
AI performs only mapping.
When this boundary collapses, control conflicts and responsibility failures reappear.
3.1 Fixed Label — Declaration of Essence
A Fixed Label is the manufacturer’s declaration of essence.
It answers the following question:
“Is there a fact that AI must never be unaware of
when handling this action?”
- If yes → add a Fixed Label
- If no → nothing needs to be written
A Fixed Label may declare:
- Safety boundaries
- Usage limits
- Physical characteristics
- Expected behavior in emergencies
- Intended purpose or usage nature
A Fixed Label is not an execution rule.
It does not define conditions, enforce behavior,
or provide information for AI to infer or extend.
It is not a request that “AI should decide instead,”
but a list of facts that “AI must not ignore.”
It is not a tool to control usage,
but a trace of responsibility left by the manufacturer
after ownership transfer.
Fixed Label is not merely a technical descriptor.
It is a declaration of responsibility from the creator toward their artifact—
an explicit statement of how the device should be respected and used safely.
AI must not extend, infer, or optimize Fixed Labels.
Manufacturer-declared boundaries are not subjects of AI reinterpretation,
but limits of responsibility that must be respected as-is.
If a manufacturer cannot take responsibility for the safe operating range of a product,
that product should not be subject to AI control in the first place.
This is not a matter of what is written in a Fixed Label.
If a manufacturer cannot responsibly define safe operation,
the product is not suitable for smart control.
AI control is not just a feature; it is an exercise of authority that demands responsibility.
3.2 Label and User Label — Transfer of Meaning Ownership
Label (Manufacturer) — Language of Essence
A Label is the name the manufacturer assigns to an action’s essence.
It describes what the action is.
Examples:
- “Turn on light”
- “Start motor”
Labels are functional and neutral.
They do not contain life context.
User Label — Language of Existence
A User Label is the meaning assigned by the user
when placing an action into their life.
Examples:
- “When I come home”
- “If the baby is crying”
- “Morning routine”
User Labels do not explain functionality.
They do not define operations.
They express meaning and importance.
3.2.1 The Moment a Label Becomes a User Label
The transition from Label to User Label
is not reinterpretation of meaning, but an exercise of ownership.
This transition occurs only in two cases:
- When the user directly edits the Label
- When AI proposes a change and the user explicitly approves it
Before approval:
- Even if AI discovers repeated patterns
- Even if AI infers strong correlations
A User Label is not created.
AI may discover, but it may not declare.
3.3 Scene (Action Group) — A Set of Approved Meanings
When a user repeatedly executes multiple actions in a specific context,
AI may ask:
“Every time you come home,
should the lights and music run together?”
This question does not create intent—it confirms it.
Upon user approval, two outcomes are possible:
- Assigning the same User Label to multiple actions, or
- Creating a Scene and registering a list of actions within it
A Scene is not a new concept.
It is simply a collection of approved User Labels.
The form differs,
but ownership and responsibility remain the same.
3.4 AI — Applicability Evaluator
AI does not choose between essence and existence.
AI accepts both.
AI can:
- Interpret approved intent
- Evaluate whether execution is currently valid
- Execute or defer
AI must not:
- Create intent
- Redefine capabilities
- Infer safety rules
4. The Formal Cognitive Model of AI
[Fixed Label]
└─ Manufacturer-declared essential boundaries
(safety, limits, characteristics, responsibility)
(AI must not cross)
[User Label / Approved Intent]
└─ User-approved language of existence
(why / when / context)
[AI Internal Reasoning]
└─ Is this intent valid right now?
(conditions, context, sensors, history)
[Action]
└─ Execute if boundaries are not violated
Otherwise defer or reject
AI does not infer essence.
AI does not judge meaning.
AI evaluates only applicability—nothing more, nothing less.
We do not propose new concepts.
We simply return existing concepts
to their proper places of ownership and approval.
Appendix — Fixed Label Examples (Informative)
This appendix is informational.
It does not redefine schemas, required fields, or enforcement logic.
Fixed Labels reflect facts that AI must never overlook,
not rules that AI may interpret or optimize.
When necessary, Fixed Label information may be surfaced to users
to explain why an action was refused, delayed, or constrained.
Example: Fixed Label List
(Aligned with existing Matter constructs; shown for illustration only)
Note: The label and value strings shown here are illustrative examples only.
Matter does not standardize the semantics of individual label/value pairs.
FixedLabelList: [
{ “label”: “Safety”, “value”: “No-unattended” },
{ “label”: “Limit”, “value”: “Max-6h-cont” },
{ “label”: “Warning”, “value”: “High-heat-gen” },
{ “label”: “Critical”, “value”: “Irreversible” }
]
What Fixed Labels May Declare
Fixed Labels may declare characteristics such as:
- Safety constraints
- Operational limits
- Physical characteristics
- Failure behavior
- Intended purpose
- Usage frequency or expectations
If a manufacturer can safely omit a Fixed Label, it should be omitted.
Fixed Labels exist only for facts that AI must never ignore.
They are not instructions.
They are not policies.
They are not optimization hints.
They are responsibility markers left by the creator.
Appendix — Informative Technical Context (Matter)
This section provides technical context only.
It exists to show that the ownership separation described in this document
is already implementable in real systems.
1. Read-only Attribute in Matter
In the Matter data model, FixedLabelList is a read-only attribute
bound to a device endpoint.
- Structure : Comprised of a list of LabelStructs,
where each entry is a pair of fixed-length (16-byte) strings (Label and Value) - Defined by the manufacturer : Hard-coded at the time of production
- Immutable : Remains constant for the lifetime of the device
- Efficiency : Fixed-length structure ensures predictable parsing
for resource-constrained devices and AI agents
This ensures that manufacturer-declared boundaries remain intact
after ownership transfer.
2. Preservation of Essence
This immutability directly corresponds to what this document defines as Essence.
Safety limits, physical constraints, and failure behavior
are not negotiable states—
they are facts.
Matter already protects these facts at the protocol level.
3. Ownership Separation, Technically Enforced
Matter’s structure already reflects the ownership model described in this paper:
- Fixed Label
→ Immutable
→ Manufacturer’s permanent responsibility - User-defined labels / scenes
→ Mutable
→ User’s contextual authority and intent
This confirms that the proposed model does not require a new protocol.
It requires recognizing and respecting the boundaries that already exist.
Why This Matters for AI Developers
Even without knowledge of Matter:
- Action boundaries can be machine-readable
- Responsibility can be declared without inference
- AI does not need to independently infer or reinterpret safety constraints
when ownership boundaries are explicit
The missing piece has not been technology,
but semantic positioning.
This appendix demonstrates that separating essence from intent
is not a future proposal—
it is an existing capability waiting to be used correctly.
This document intentionally avoids proposing new normative requirements.
It is offered as an interpretive framework for existing Matter constructs.