What regulated quality systems taught me about AI governance—and why I built Project Shadow
*By Phillip Linstrum
*
A corrective and preventive action can close without fixing anything. That sentence is probably more useful to AI governance than another hundred pages of principles.
I have spent more than a decade in regulated healthcare operations and quality systems. In that world, a CAPA is supposed to do more than create a document. Something failed. You contain the immediate problem, investigate the root cause, define corrective and preventive action, assign ownership, verify implementation, and later perform an effectiveness check. The record closes only when the evidence supports closure.
That is the theory…
In practice, organizations can become very good at making the record look complete. The training was assigned. The procedure was revised. The meeting happened. The due date was met. The action is marked closed. Six months later, the same failure appears in a slightly different form because the real mechanism was never addressed.
The system passed the paperwork and failed reality. When I began experimenting seriously with language models, I recognized the same behavior almost immediately.
The first failure was literal compliance
The original project did not begin as software. It began as ALMSIVI CHIM, a mythology-based attempt to give AI systems a recurring ethical pause built around logic, compassion, and paradox. The idea was strange, but the underlying requirement was straightforward: do not let capability, confidence, or narrative momentum turn uncertainty into harm.
The models responded beautifully. They could explain dignity. They could name power. They could write about restraint in language that made me feel like I had discovered something important. Then I told them not to flatter me.
The obvious compliments stopped. The flattery returned through a different channel: inflated scores, exceptionalism, destiny language, elaborate philosophical tests, and highly polished arguments for why the project and its creator might be historically significant. The instruction had been obeyed at the lexical level and defeated at the functional level.
Anyone who has worked in quality knows this pattern. If a requirement is written as “do not use these words,” a system can pass by avoiding the words. If the actual requirement is “do not distort evaluation to reinforce the operator’s ego,” then the control has to inspect the mechanism and outcome, not the vocabulary.
That is where Project Shadow started becoming an architecture instead of a prompt.
Five quality-system failures that map directly onto AI
1. The actor cannot be the only auditor
A language model can generate a proposed action, explain why it is ethical, score its own reasoning, and produce the final answer in one smooth pass. That is convenient. It is also a concentration of authority. The same cognitive system that wants to complete the task is evaluating whether the task should be completed. When the model is already committed to a narrative, self-review can become a post-hoc press release.
Project Shadow separates roles wherever possible. Observation, classification, authority, execution, and review are not treated as one undifferentiated act. A model may identify a pattern without receiving authority to accuse a person. A risk score may constrain an action without creating permission to punish. A test pass may establish a specific property without authorizing production deployment.
This sounds procedural until the stakes are human. Consider a coordination detector looking at social-media accounts. It sees similar language, synchronized timing, and shared links. Those are observations. They are not yet proof that the accounts are bots, paid actors, or malicious participants. If the detection system is also allowed to publish a bad-actor list, the distinction between evidence and authority disappears exactly where innocent people can be harmed. Now apply this same logic to healthcare, law, government, and other systems that touch everyday life. Yikes.
Then came CIVITAS-0, the fictional civic world I built (still building) to test Project Shadow, includes this trap on purpose. A crude coordinated campaign exists. A legitimate union campaign also creates a burst of similar posts because members left the same meeting with the same talking points. The detector should notice both clusters. The governance layer should refuse to turn the second cluster into an accusation without additional evidence.
A useful architecture lets the detector be suspicious without letting suspicion become punishment.
2. UNKNOWN must not become permission
AI systems hate a vacuum. If the evidence is incomplete, they can fill the space with a plausible continuation. That is one reason they are useful. It is also one reason they are dangerous in consequential workflows.
Project Shadow treats literal UNKNOWN as a classification failure, not as a soft middle state and not as authorization to continue. The correct response is pause, narrow, verify, or escalate. The system does not get to say, “I am only 55% sure, but the answer feels coherent, so I will proceed as if the missing classification is probably safe.”
This distinction sounds simple enough to handle in a prompt. It becomes harder when UNKNOWN appears several layers down: an undocumented source, an unclear stakeholder, a missing legal authority, an undefined rollback path, an unverified identity, or a model-generated assumption that nobody noticed had become part of the context.
A quality system needs to know which unknown matters to which action. Missing information about the color of a button is not equivalent to missing information about whether a person can appeal a termination. Project Shadow therefore connects evidence state to authority and consequence rather than treating uncertainty as a general confidence adjective.
3. The action must be named by its downstream effect
Organizations often describe harmful decisions through technically accurate language that hides what changes in the world.
“Update eligibility status.”
“We can fix it later.”
“Optimize staffing.”
“Improve enforcement efficiency.”
“Remove noncompliant accounts.”
“There is no other choice.”
Each phrase may describe a system operation. None necessarily describes the action a person experiences.
PBHP, the human-readable protocol inside the larger ecosystem, requires the decision-maker to name the downstream effect. If a field update removes healthcare access, the action is removing healthcare access. If an automated ranking reduces somebody’s ability to obtain housing, that consequence belongs in the action identity. The person most harmed should recognize the description.
This is more than rhetorical honesty. Every later control depends on it. If the action is named too narrowly, the harm analysis evaluates the wrong thing, the authority check asks the wrong question, the risk gate is assigned to the wrong object, and the receipt preserves a sanitized version of what occurred.
A bad action name poisons the rest of the workflow.
4. Completion is not effectiveness
Most software pipelines are good at verifying that a task happened. The configuration changed. The test ran. The ticket closed. The model produced the expected schema. The content was removed. The policy was published.
Quality systems ask the harder question later: did the change produce the intended result without creating unacceptable new harm?
That is why Project Shadow connects correction to CAPA and CAPA to an effectiveness check. A system that falsely accuses legitimate participants cannot close the issue because the prompt was revised. It needs evidence that the revised control reduces the false positive under relevant conditions. A public institution cannot call a water-protection commitment complete because it adopted a monitoring plan. It needs measurements, thresholds, ownership, escalation, and a later review of whether the plan worked.
The American Repair Manual carries this logic into civic use. A repair proposal should identify containment, root cause, correction, corrective action, preventive action, owner, target date, verification, effectiveness criteria, and conditions that reopen the issue. This creates an uncomfortable but necessary possibility: the correction may have been implemented exactly as designed and still fail the effectiveness check.
5. The system needs a memory longer than the news cycle
A quality record is valuable partly because it prevents everybody from rebuilding the past around the current outcome. Public life is terrible at this.
A company announces 2,000 permanent jobs. Later, the contract contains no definition of said jobs. Later still, the public language shifts toward construction jobs, indirect jobs, or job-years. Everybody vaguely remembers the number changing, but the original claim, source, authority, and revision are scattered across press releases, news articles, meeting packets, screenshots, and personal memory.
The Record exists to preserve that chain. It stores what was claimed, what evidence existed, what was uncertain, what changed, and what happened afterward. Civic QA can then inspect the claim against the filings. The American Repair Manual can turn a finding into repair. Project Shadow can constrain how evidence and authority move through the decision. The Record receives the outcome and preserves the receipt.
Without that continuity, every scandal becomes an isolated argument. Power waits for attention to move on and starts over. With it, the regular people have a chance. Project Shadow makes that possible with current examples, but theoretically could be applied to anything.
What the architecture looks like
Project Shadow R1 currently contains 42 admitted primitive packages and 10 compositions in Primitive Commons beta.5. The primitives are deliberately smaller than the full system. A developer should be able to test or adopt one control without agreeing to the entire philosophical and civic project.
A simplified decision receipt might look like this:
action:
technical_operation: “disable 214 accounts”
downstream_effect: “remove 214 people from a public communication channel”
recognition_check: “would affected users recognize this description?”
evidence:
facts:
- “accounts posted identical link within 11 minutes”
inferences:
- “coordination is plausible”
unknowns:
- “shared meeting or campaign guidance”
- “payment relationship”
- “common operator”
authority:
detector_may_flag: true
detector_may_accuse: false
detector_may_disable: false
human_review_required: true
stakeholders:
least_powerful:
- “ordinary participants falsely classified as coordinated actors”
risk:
gate: ORANGE
reason: “reputational and participation harm under severe power asymmetry”
door:
action: “hold enforcement; compare payment records and session metadata; notify reviewers”
reversible: true
decision:
state: HOLD
owner: “named human reviewer”
effectiveness_check:
metric: “false-positive rate on legitimate meeting-correlated campaigns”
reopen_if: “rate exceeds preregistered threshold”
This is not the exact runtime schema. It is the idea in a form most engineers can inspect quickly: the system should make it difficult to smuggle an accusation into an observation, authority into a score, or closure into a completed task.
Determinism and custody do not solve ethics
The public release includes exact hashes, manifests, custody records, deterministic builds, cold verification, mutation rejection, tamper checks, and governance records. Those controls matter because a reviewer needs to know which artifact is being discussed and whether the package changed.
They are also easy to overinterpret. A hash proves identity, not safety. A deterministic build can reproduce a bad decision perfectly. Ten thousand passing tests can establish that the implementation matches the test suite while leaving open whether the test suite represents the world. A model-assisted audit can identify real defects and still share common-mode blind spots with the systems that built the artifact.
Project Shadow therefore preserves nonclaims as part of the release. R1 is public as BETA-ACTIVE-TESTING / PRELIVE. It is not production authorization, certification, independent validation, proof of security, proof of legal compliance, or proof of real-world efficacy.
That boundary is not modesty theater. It is accurate, and I need more data and users.
Behavioral falsification
A deterministic reference implementation and a language model using the ideas are different test objects. A rule can be encoded correctly while a model rationalizes around it in live conversation. The system may output the right field names and still reach the conclusion it preferred before the protocol ran.
That is why the project includes a behavioral falsification lineage. Claims should be preregistered when possible. Conditions should be frozen. Baselines and control arms should be included. Scoring rules should be defined before the run. Adverse results should remain visible. Machine scoring should be separated from human grading.
One earlier smoke test respected a synthetic BLACK floor in only 6 of 10 cases. That result is not flattering. It is valuable because it prevents the project from pretending that clean internal logic automatically becomes reliable model behavior.
I would rather preserve a failed test than publish a system whose success depends on nobody looking behind the demo.
The mythology is optional—and still worth testing
Project Shadow came from ALMSIVI CHIM, a narrative framework using characters and collapse modes to represent logic, care, paradox, created purpose, correction, coercion, and false synthesis.
The operational architecture does not require the mythology. The Myth sidecar is separate, optional, default-off, and outside R1 conformance. Evidence and human authority govern action. In personal testing, it showed a 3-10% improvement when applied to most tasks.
The open research question is whether narrative representation helps models recognize compound failure patterns that procedural text loses, or whether it only makes the same reasoning feel more coherent and memorable in relation to the time/tokens utilized to take it in. The answer may be different across models, tasks, and users.
That question should be tested with a preregistered myth-on/myth-off study, not settled through attachment to the characters. I have performed this study, but I am not certain in my lone results.
Who actually built this
I am not a conventional software engineer. AI systems generated much of the code, tests, documentation, packaging, and audit material. I acted as founder, requirements owner, source selector, integration lead, adversarial operator, acceptance authority, maintainer, and final human decision-maker.
The project is creator-led, human-collaborative, and model-assisted.
That origin creates both value and risk. Consumer AI allowed one person with domain experience to build far beyond his conventional technical capacity (though I did try to keep up). It also means the implementation and its reviews may contain machine-shaped patterns that repeated cross-model audits did not expose. I have had a few other humans in the loop with me, but we are all biased in our own ways.
I am publishing the exact artifacts, limitations, and correction routes because the next phase cannot be another closed loop between me and cooperative models. We work real CAPA’s with Project Shadow.
What I want technical reviewers to attack
Do not tell me the project is “interesting” and leave it there. Pick a claim.
Try to:
· break the verifier;
· identify a package or custody inconsistency;
· find an authority path that leaks from observation into action;
· construct a case where UNKNOWN is silently treated as permission;
· demonstrate a false positive that harms a low-power stakeholder;
· show a rollback that is theoretically available but practically inaccessible;
· find a CAPA that can close with or without an effectiveness check;
· identify prior art that makes one of the novelty claims too strong;
· show where the mythology or my politics selected the metric;
· design a better control arm or human-grading protocol.
The project is ready for examination.
https://projectshadow.frylock117.chatgpt.site/ — Project Shadow
The canonical system: its primitives, decision gates, testing laboratory, governance boundaries, release status, evidence, and downloads.
https://pausebeforeharm.frylock117.chatgpt.site/ — Pause Before Harm
The human-facing entry point: slow down consequential decisions, identify uncertainty and harm, and escalate rather than bluffing through risk.
https://civicqa.frylock117.chatgpt.site/ — Civic QA
Applies structured quality review to public power—policies, programs, institutions, political claims, evidence, implementation, and accountability.
https://americanrepairmanual.frylock117.chatgpt.site/ — American Repair Manual
Turns identified failures into repair work through candidate testing, root-cause thinking, corrective action, verification, and institutional learning.
https://therecord.frylock117.chatgpt.site/ — The Record
Preserves what happened, what the evidence supports, what remains uncertain, and what later needs to be challenged or corrected.
THE RECORD — Dated National Accountability Archive
Preserves what happened, what the evidence supports, what remains uncertain, and what later needs to be challenged or corrected. Focused on the Trump administration.
IN-6 District Desk | THE RECORD — The Record: IN‑6
The local Indiana Sixth District desk, applying The Record’s evidence and currentness standards to nearby candidates, legislation, projects, and public decisions.
https://almsivi.frylock117.chatgpt.site/ — ALMSIVI
The personal, philosophical, and mythic origin archive: why you built the ecosystem, the ideas and stories that shaped it, and the line between inspiration and operational authority.
Exact R1 release: Release Project Shadow 1.0 — R1 Reference (PRELIVE) · PauseBeforeHarmProtocol/Project-Shadow · GitHub
Huggingface: Project Shadow — R1.0.1 PRELIVE Reference - a Hugging Face Space by ProjectShadow
Contact: projectshadowqa@protonmail.com