Hmm… For now, I did some digging first:
My direct answers would be:
- The three-pillar taxonomy is clear as a top-level map. It gives newcomers a useful way to connect user-controlled inputs, resident model artifacts, and the device execution environment.
- The first-week reading path is useful. Its progression from ecosystem mapping to exposure, attacks, defenses, and emerging directions makes sense.
- The most directly related item I found is another recent review of on-device inference security. Its scope overlaps with this roadmap, but its literature-selection method and organizing perspective appear different, so the two may be useful as complementary maps rather than substitutes.
- As the collection grows, I would probably keep the current taxonomy and add a few optional navigation fields—protected asset, lifecycle stage, attacker access, execution location, and evidence environment—rather than trying to force every new paper into one increasingly deep hierarchy.
- For the emerging directions, there are already usable papers, benchmarks, repositories, and official examples that could connect the roadmap to on-device training, local generative AI, and mobile agents.
The rest below is mostly a set of nearby references, possible navigation aids, and small verification paths. I am not treating similar-looking work as proof of an identical threat model or root cause.
1. Related reviews and maps
A closely related systematic review
The closest item I found is Protecting On-Device AI Inference: A Systematic Review of Attacks and Defence Mechanisms, submitted on May 28, 2026.
It covers attacks and defenses around on-device inference, including model extraction, adversarial attacks, data breaches, trusted execution environments, obfuscation, homomorphic encryption, and differential privacy. It also uses an explicit systematic-review process with search, screening, inclusion/exclusion, and quality criteria for peer-reviewed work from 2019–2025.
That makes it close to the MoAI SoK, submitted on July 1, 2026, but I would not assume they are the same project or answer the same question.
A rough reading is:
| Map |
Main strength as a reading aid |
| MoAI SoK and roadmap |
A mobile-system and trust-boundary map: inputs, resident models, device-native execution, attacks, defenses, open problems, and emerging directions |
| On-device inference systematic review |
A corpus-selection map: explicit review methodology, inclusion boundaries, literature distribution, and attack/defense coverage |
The overlap itself may be useful to readers. A short cross-reference could explain:
- mobile-only versus broader edge or resource-constrained devices;
- SoK-style conceptual organization versus systematic literature selection;
- whether privacy and user-data confidentiality are first-class categories or cross-cutting properties;
- how the literature cutoff dates differ;
- how each review defines “on-device inference.”
This need not become a novelty dispute. The two works may simply have developed in parallel, and future readers may benefit from knowing that both maps exist.
A narrower but useful deep dive
All You Need to Know About On-Device ML Model Extraction is narrower than the full roadmap, but particularly useful for the model-extraction portion.
One useful feature is its emphasis on attacker models and practical reproducibility. “Model extraction” can refer to quite different situations:
- extracting a plaintext model bundled in an application;
- recovering a model after app-level decryption;
- inspecting process memory;
- observing communication or hardware side channels;
- inferring functionality through queries;
- operating with app-repackaging, root, physical, or privileged-system access.
Those cases may share an outcome, but they do not necessarily share a cause, cost, or viable defense.
A general terminology and metadata reference
The current edition of NIST AI 100-2, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations organizes AML concepts using multiple dimensions, including:
- ML lifecycle stage;
- attacker goal and objective;
- attacker capability;
- attacker knowledge;
- learning and deployment context.
I would not replace the MoAI pillars with the NIST taxonomy. The pillars are much easier to read as a mobile-specific roadmap. NIST is more useful as an optional second index for distinguishing papers that use the same attack name under different assumptions.
Another way to navigate the same collection
The current pillars can remain the newcomer-facing map:
- user-governed input integrity;
- device-resident model security;
- device-native environment confinement.
A lightweight second layer could make individual resources easier to compare:
| Field |
Example values |
| Target |
model, runtime, app, OS, accelerator, agent, hybrid system |
| Protected asset |
model confidentiality, model integrity, user data, availability, action authority, auditability |
| Lifecycle |
training, conversion, packaging, distribution, loading, inference, action, update |
| Attacker access |
remote input, repackager, unprivileged local app, runtime instrumentation, root, compromised OS, physical access |
| Execution location |
device, cloud, hybrid, unclear |
| Evidence environment |
analytical, simulation, emulator, development board, physical phone, commercial application |
| Artifact status |
paper only, code, dataset, benchmark, executable environment, traces |
This avoids requiring every item to fit only one category. For example, one paper may concern both model confidentiality and user-data confidentiality, or both runtime confinement and availability.
It also makes the assumptions visible. A physical-access attack on a rooted research phone should not be silently treated as equivalent to an attack available to an ordinary sandboxed app.
A larger example of this general pattern is the GUI Agents Paper List. It keeps structured metadata separately and provides multiple ways to browse a large collection. Its scale is much larger, so I would view it as a possible later-stage reference, not as a minimum requirement for this repo.
2. Background boundary: MoAI-specific risks and ordinary mobile-app risks
MoAI applications inherit the ordinary security properties of mobile applications:
- local storage;
- cryptographic key handling;
- authentication;
- permissions and IPC;
- code signing and application integrity;
- network communication;
- logs and crash reports;
- reverse engineering and repackaging.
The OWASP Mobile Application Security project provides a useful baseline:
- MASVS provides mobile security controls;
- MASWE describes common mobile weaknesses;
- MASTG provides testing knowledge, techniques, tools, demos, and atomic tests.
These references do not need to be absorbed into the MoAI taxonomy. They can instead define the background boundary.
One possible distinction is:
| Relationship |
Meaning |
| MoAI-specific |
The risk exists because an AI model or inference state is resident and used on the device |
| MoAI-amplified |
An ordinary mobile weakness becomes more consequential because it exposes weights, preprocessing logic, prompts, outputs, checkpoints, or agent context |
| Inherited mobile risk |
A conventional application-security issue that is not materially changed by local AI |
Examples:
| Ordinary mobile issue |
Additional MoAI consequence |
| Local storage exposure |
Model weights, adapters, training checkpoints, embeddings, or KV state may become readable |
| Reverse engineering |
Preprocessing, postprocessing, model structure, label maps, and security-sensitive model logic may be exposed |
| App repackaging |
The model, preprocessing pipeline, or input/output logic may be replaced |
| Insecure logging |
Prompts, outputs, tensor metadata, or classification results may be retained |
| Weak key management |
An encrypted model or update package may be recoverable |
| Overbroad permissions |
Sensors, private context, or agent actions may become accessible |
This separation also helps avoid interpreting every reverse-engineering obstacle as a complete security boundary. OWASP treats obfuscation, anti-debugging, and anti-tampering as resilience or defense-in-depth measures; they are not substitutes for sound storage, authorization, cryptography, or platform isolation.
A related reading axis: what exactly is being protected?
I found it useful to read the roadmap not only by attack type, but also by protected asset.
| Protected asset |
Typical concern |
| Model confidentiality |
Weight or architecture extraction, proprietary IP loss |
| Model integrity |
Backdoors, unauthorized weight replacement, preprocessing modification |
| User input/output confidentiality |
Prompt, output token, conversation, embedding, or inference-state leakage |
| Availability |
Energy, memory, thermal, and latency exhaustion |
| Execution integrity |
Runtime or accelerator manipulation |
| Action authority |
An agent performing an unauthorized external action |
| Auditability |
Inability to reconstruct which context, plan, tool, or permission produced an action |
Model confidentiality and user-data confidentiality are especially easy to merge even though the affected party and technical path may differ:
Model extraction
→ primarily threatens the developer or provider’s model IP
Runtime-state leakage
→ primarily threatens the user’s prompts, outputs, history, and local context
For local generative models, possible sensitive runtime material includes:
- prompts and generated output;
- conversation history;
- embeddings;
- KV caches;
- temporary tensors;
- application memory;
- logs and crash dumps.
KV-Shield is one example exploring protection of KV state in on-device LLM inference. I would treat such results as threat-model-specific rather than conclude that every local LLM exposes reconstructable conversations. The relevant questions include:
- What process or privilege does the attacker have?
- Which CPU, GPU, or accelerator memory is observed?
- Is the operating system trusted?
- Is the attacker co-resident?
- Does the defense cover side channels or only direct memory access?
- What performance and memory costs result?
The protected-asset field may be enough to make this thread visible without introducing another top-level pillar.
3. Packaging, provenance, serialization, and updates
The current roadmap mainly reads as a map of deployed models and inference. There is also a nearby lifecycle:
training
→ export
→ conversion
→ quantization or optimization
→ packaging
→ signing
→ distribution
→ update
→ loading
→ execution
Not every resource below is mobile-specific or an inference defense. They are adjacent tools for readers interested in model provenance and artifact integrity.
Safe serialization
Safetensors is designed as a simple and fast tensor format that avoids the executable deserialization behavior associated with pickle-based formats.
That narrows one loader-related risk, but it does not establish that:
- the model is benign;
- the weights are authentic;
- the file has not been replaced;
- the model is free of backdoors;
- runtime memory is confidential;
- a mobile package protects the model after loading.
A useful distinction is:
safe parsing
≠ trusted provenance
≠ benign model behavior
≠ runtime confidentiality
Model signing
The OpenSSF Model Signing project provides a specification, library, and CLI for signing and verifying ML model artifacts across model formats and sizes.
The associated OMS specification is relevant to:
- artifact-integrity verification;
- origin authentication;
- chain-of-custody information;
- standardized signed claims.
Again, the guarantee should remain narrow:
a valid signature
→ the verified artifact matches what the signer signed
a valid signature does not by itself show
→ that the signer is trustworthy
→ that the model is safe
→ that the model is free of backdoors
→ that runtime execution is confidential
AI/ML bills of materials
CycloneDX AI/ML-BOM provides a structured way to represent model, dataset, dependency, training-method, framework-configuration, and provenance information.
This is more about visibility and lifecycle assurance than attack prevention. It may nevertheless be useful if the roadmap later tracks:
- original model source;
- conversion tools;
- quantization steps;
- adapters;
- runtime dependencies;
- datasets or model lineage;
- update provenance.
A possible scope choice would be to keep these in an Adjacent lifecycle references section rather than include them as core defenses.
A practical way to compare defenses
For defense papers and framework features, the category name alone may not reveal the actual protection boundary.
A comparison checklist could be:
What asset is protected?
At which lifecycle stage?
Where does plaintext or an unprotected representation appear?
Where is the key or trusted state held?
What attacker is assumed?
Is the method a concept, prototype, open-source artifact, or official runtime feature?
Was it evaluated on a physical mobile device?
Does it cover process memory?
Does it cover accelerator memory?
Does it consider a rooted device or compromised OS?
What performance, memory, energy, or compatibility cost remains?
A particularly useful distinction is:
encrypted at rest
≠ protected while loading
≠ protected in process memory
≠ protected in accelerator memory
≠ protected from runtime instrumentation
≠ protected from a privileged or compromised OS
Example: Core ML model encryption
Apple provides an official Core ML model-encryption workflow for compiled models bundled with an application or delivered through Core ML deployment.
That is useful evidence that platform-supported model encryption exists. It is not, by itself, evidence that every runtime representation remains opaque under every attacker model.
The Core ML Developer Forums also contain reports involving key requests, model loading, model packages, and repeated initialization. Individual forum reports do not establish a general platform defect, but they are useful reminders that:
- key lifecycle;
- account and team configuration;
- operating-system compatibility;
- packaging;
- deployment;
- runtime loading
are operational parts of the defense.
Example: ONNX Runtime and application-level decryption
ONNX Runtime is the execution runtime; ONNX itself is the model format and ecosystem. Keeping those names separate may help readers distinguish format-level and runtime-level claims.
ORT can load model data from memory through its APIs, and its discussions include questions such as loading an encrypted model after application-level decryption and custom model-decryption hooks.
Those discussions are useful as examples of implementation demand and integration friction. They are not evidence that ONNX Runtime is generally insecure.
They also demonstrate that:
decrypting to an in-memory buffer
may avoid writing a plaintext model file to disk
but it does not automatically prevent
process-memory inspection
instrumentation
buffer capture
or privileged-system access
Example: TensorFlow Lite / LiteRT demand
The long-running TensorFlow issue Encrypt .tflite file to prevent copying similarly shows that model-copy protection has been a practical developer concern for years.
I would cite an issue like this as evidence of a recurring use case, not as a measurement of prevalence or proof of a specific vulnerability.
[/details]
Existing pieces for the emerging directions
The roadmap already identifies on-device training, on-device generative AI, and agentic MoAI. The following resources may help make those directions more operational.
On-device training and personalization
ONNX Runtime provides:
The tutorial is useful because it gives a concrete system in which readers can identify the new artifacts introduced by local training:
base inference model
training model
evaluation model
optimizer model or state
checkpoint
local training samples
updated parameters
exported inference model
For each artifact, the security questions change:
Where is it stored?
Who can read it?
Who can modify it?
How is an update authenticated?
Can an older checkpoint be restored?
Can stale checkpoints remain readable?
How is the updated model distinguished from the original?
Is update provenance recorded?
This makes clear that “training happens locally” does not automatically mean “the training process is private and secure.” It changes the trust boundary and creates additional mutable state.
Mobile and GUI agents
I would keep agentic MoAI as an extension layer rather than treat it as merely one more model attack.
The object being secured expands from a model and runtime to a context-to-action path:
context provenance
→ perception
→ model reasoning
→ memory
→ plan
→ tool or app selection
→ permission
→ external action
→ confirmation
→ rollback
→ audit
It is also useful to record component location separately:
| Component |
Possible location |
| Perception processing |
device, cloud, hybrid |
| Model inference |
device, cloud, hybrid |
| Planning and memory |
device, cloud, hybrid |
| Action execution |
OS, app, device-side service, remote service |
Therefore:
mobile agent
≠ on-device model
≠ fully local inference
≠ locally governed action
Some concrete references:
| Resource |
Useful connection |
| From Assistants to Adversaries / AgentScan |
Maps mobile-agent risks across language reasoning, GUI interaction, and system execution |
| MobileSafetyBench |
Separates mobile-agent helpfulness from safety in realistic Android-emulator tasks, including misuse, harmful side effects, and indirect injection |
| GhostEI-Bench |
Tests environmental injection through dynamic notifications, pop-ups, overlays, and inter-app events in executable Android environments |
| MIRAGE |
Places injections in ordinary user-generated content without requiring modification of the agent, application, or OS |
| OS-Sentinel and its repository |
Provides a dynamic mobile-risk environment and combines explicit rule checking with contextual safety assessment |
| CAPED |
Treats screenshot transfer to a remote multimodal agent as an explicit device–cloud privacy boundary |
| Awesome Agent Security |
A broad discovery index for agent security and safety work; it intentionally mixes the two, so entries still need local classification |
These are not all direct additions to the inference-security core. They are useful for operationalizing the existing agentic direction.
They also introduce a distinction that may help future readers:
content or model safety
≠ tool authorization
≠ action safety
≠ safe delegation
≠ recoverability and auditability
4. A small, permission-safe checkpoint for the first-week path
The current progression is already sensible:
ecosystem
→ local model exposure
→ attacks
→ defenses
→ emerging directions
One small hands-on checkpoint could help beginners distinguish observations that are often conflated.
Only use an application you own, an open-source sample, or an artifact for which you have explicit testing permission.
For an Android APK:
unzip sample.apk -d unpacked
find unpacked -type f \
| grep -Ei '\.(tflite|lite|onnx|ort|pte|pb|bin)$'
grep -RIlE \
'TensorFlowLite|LiteRT|onnxruntime|executorch' \
unpacked
The OWASP MASTG provides broader guidance for authorized static and dynamic mobile-app analysis. Its atomic tests are also useful examples of keeping each check narrow.
What this checkpoint can support
It may reveal:
- candidate model artifacts;
- runtime dependencies;
- packaging paths;
- native libraries;
- manifests or configuration associated with inference.
What it does not establish
It does not, by itself, show that:
- the candidate file is a usable model;
- the model is unencrypted;
- the full model can be reconstructed;
- the model can be executed outside the application;
- runtime state can be read;
- a defense has been bypassed;
- the same result transfers to another device or application version.
That “does not establish” section may be more educational than the command itself.
A similar checkpoint for defense papers would be to draw the materialization path:
encrypted package
→ key acquisition
→ decrypted loading buffer
→ runtime representation
→ intermediate tensors
→ output
Readers can then mark exactly where each proposed defense starts and stops.
For on-device training, the official ONNX Runtime Android example can be used in the same way: list every model, optimizer, checkpoint, and local-data artifact without attempting an attack.
Short annotations for each reading-path item
Each recommended paper could optionally have five small fields:
Why read this:
Primary threat model:
Required background:
Evidence platform:
Code or artifact:
What the result does not establish:
This would help beginners understand why a paper is in the path, rather than reading it only as another citation.
5. Keeping the roadmap maintainable as contributions arrive
This is optional and mostly relevant if the list begins receiving many community submissions.
A minimal contribution path
GitHub supports a root, docs, or .github CONTRIBUTING.md. Contributors see a link to it when opening issues and pull requests.
GitHub also supports structured Issue Forms. These are currently documented as public preview, but they are already useful for requesting consistent submission metadata.
A “Suggest a paper or project” form could request:
Canonical title
Paper, project, or documentation link
Publication status
Why it relates to MoAI
Core or adjacent
Target platform
Device, cloud, hybrid, or unclear
Protected asset
Threat model and attacker access
Evidence environment
Code, dataset, benchmark, or executable artifact
Suggested section
Possible overlap with an existing entry
That would not force contributors to adopt a complex taxonomy. It would simply give the maintainer enough context to classify the suggestion.
Core and adjacent material
A low-friction structure could be:
Core
- MoAI-specific or substantially MoAI-amplified research
Adjacent
- general mobile application security
- edge and TinyML work
- model supply-chain tools
- broader agent-security work
- related surveys
This preserves useful neighboring references without allowing them to obscure the roadmap’s main scope.
Lightweight automated checks
Two possible tools are:
- lychee-action, which checks links in Markdown, HTML, and text;
- awesome-lint, which checks general Markdown and Awesome-list conventions.
Neither needs to become a hard requirement.
External scholarly and project links can fail temporarily or rate-limit automated requests. One possible policy is:
- reject a newly introduced clearly broken link in a pull request;
- run a scheduled check for older links;
- report intermittent external failures without making the main branch permanently red;
- allow exclusions for domains that block automated link checkers.
The current README also contains a custom structure, figures, badges, and intentionally repeated papers across categories, so applying every default Awesome rule may be less useful than selecting a few consistency checks.
A possible later-stage pattern
If the collection becomes much larger, a machine-readable source can help:
papers.yaml
→ canonical resource metadata
adjacent.yaml
→ useful non-core references
README
→ generated or manually curated newcomer-facing roadmap
search page
→ optional multi-filter view
The GUI Agents Paper List is one example of a mature curated list using structured metadata and multiple browsing paths.
I would view this as a future option, not something required while the roadmap remains compact.
Possible next-step paths
Depending on how compact or expandable the roadmap is intended to be, I can see several reasonable paths.
Keep it compact
- Add the closely related systematic review.
- Add one general mobile-security baseline link.
- Add a few agentic or on-device-training references to the existing emerging-directions section.
- Keep the current taxonomy unchanged.
Add lightweight navigation
- Keep the pillars and attack/defense sections.
- Add only three tags per resource:
- protected asset;
- attacker access;
- device/cloud/hybrid.
- Add “what this result does not establish” to the beginner path.
Expand it as a research roadmap
- Add lifecycle and evidence-environment metadata.
- Separate core and adjacent resources.
- Connect each emerging direction to at least one paper, one codebase or benchmark, and one small reproducible example.
- Track whether a defense is conceptual, prototyped, open-source, officially supported, or evaluated on a physical phone.
Prepare for community maintenance
- Add a short contribution guide.
- Add a structured resource-suggestion form.
- Add a scheduled link check.
- Move to structured YAML only if the list becomes large enough to justify it.
My own minimal choice would be:
- cross-reference the nearby systematic review;
- preserve the current three-pillar map;
- add a small protected-asset / attacker-access / execution-location index;
- add one permission-safe hands-on checkpoint to the first-week path;
- connect the existing training, GenAI, and agentic directions to a few executable examples.
That would add several new entrances to the roadmap without changing the main structure that already makes it approachable.
