Experimenting with intent cells for rerunnable LLM workflows

Editable intent and resolved execution are different records

An ICC cell such as:

c2 Review
> fast
%from c1
@file -json review.json

is a good readable description of intent. It says that c2 consumes c1, uses a routing choice called fast, and is expected to produce review.json.

A completed execution needs a more concrete record:

• Which revision of the c2 definition was executed?
• Which particular c1 run was resolved?
• Which particular artifact version was consumed?
• Which route and parameters were requested?
• Which model/provider response was observed?
• Which output artifact occurrence was produced?
• Which verifier definition evaluated it?

I find it useful to separate three kinds of identity here.

1. Logical identity

A mutable, human-readable name:

• c1
• %from c1
• report.md
• main
• fast
• auto

These are convenient for authoring, but may resolve differently later.

2. Occurrence identity

A concrete event or version:

• a particular cell run ID
• a provider attempt ID
• an artifact ID and version
• a retrieval execution
• a repository commit
• an evaluator run

This answers “which one did this run actually use?”

3. Content identity

The identity of the content itself:

• a byte digest
• a normalized-text digest
• a dataset fingerprint
• a provider response ID
• another content-addressed identifier

This answers “was the content itself the same?”

The three are complementary. A digest alone does not explain which execution produced something, while a run ID alone does not tell us whether two runs produced identical content.

A familiar Hugging Face analogy is the difference between a mutable branch or tag and a concrete repository commit. The Hub lets callers use a branch-like revision for convenience or a full commit hash when they need to pin a specific state: Downloading files from a specific Hub revision.

A smaller workflow analogy is DVC. A human edits dvc.yaml, while DVC maintains dvc.lock with resolved parameter values and dependency/output hashes: DVC pipeline and lock files. ICC does not need to copy DVC, but the separation between editable declaration and machine-recorded resolved state seems relevant.

The provenance literature uses a similar distinction between a workflow as a plan and a record of a particular execution. The Workflow Run RO-Crate profiles describe this as prospective and retrospective provenance. I would treat that as vocabulary and prior art rather than as a requirement to adopt the full format.

The minimal relationship is a graph

The most important record may not be a large JSON object, but a small set of stable relationships:

Workflow run W
  ├─ Cell run C1
  │    └─ produced Artifact A:v1
  │
  └─ Cell run C2
       ├─ consumed Artifact A:v1
       └─ produced Artifact B:v1

That lets a future reader traverse from an output back to the exact run and input occurrences that produced it.

ML Metadata uses a compact model based on executions, artifacts, events connecting them, and contexts grouping them. OpenLineage’s parent/root run model is another example of keeping individual executions joinable under a larger workflow run.

Again, ICC does not need to implement either specification. The useful part is the boundary:

• executions have identities;
• artifacts have identities;
• the consumed/produced edges are durable records;
• individual cell runs can be joined to a root workflow run.

A manifest can be an index rather than the payload

A small manifest could identify an artifact using fields such as:

logical reference
resolved artifact occurrence
producer run
content type
digest and size, where meaningful
payload location
retention or redaction state

The actual PDF, image, long text, or retrieval payload can live in the artifact store.

This resembles the general idea behind an OCI content descriptor: a compact record can identify and verify external content without becoming the content itself.

It also avoids making traces unnecessarily large. For example, MLflow trace attachments store large binary content separately and leave a lightweight reference in the trace.

I would keep the first version fairly small. Something conceptually like this may be enough:

run:
  id: cell-run-...
  root_run_id: workflow-run-...
  attempt_id: attempt-...
  status: completed

definition:
  notebook_revision: ...
  cell_id: c2
  cell_revision: ...
  dsl_version: ...

inputs:
  - logical_reference: "%from c1"
    resolved_run_id: cell-run-c1-...
    resolved_artifact_id: artifact-...
    artifact_version: 1
    digest: ...
    retention: stored

execution:
  requested_route: fast
  requested_model: ...
  requested_parameters: ...
  observed_response_model: ...
  observed_provider: ...
  provider_response_id: ...

outputs:
  - logical_name: review.json
    artifact_id: artifact-...
    version: 1
    digest: ...
    media_type: application/json
    payload_reference: ...

verification:
  - evaluator_run_id: evaluator-run-...
    evaluator_definition_revision: ...
    evaluated_artifact_id: artifact-...
    verdict: passed

trace_reference: ...

This is only a separation-of-concerns sketch, not a proposed mandatory schema. In particular:

• A digest is useful only when its byte or normalization boundary is clearly defined.
• observed_provider may be unavailable or may only reflect what a gateway reports.
• Raw inputs may be redacted, truncated, externally referenced, or deliberately not retained.
• A missing field should not silently imply that the value did not exist.

Requested routing and observed execution

A request such as auto, fast, or a router alias is part of the authored intent. The model and provider that eventually produce the response are execution facts.

The current OpenTelemetry GenAI attribute registry similarly distinguishes fields such as:

• requested model;
• response model;
• response ID;
• provider name.

Those semantic conventions are still evolving, so I would use them as naming guidance rather than requiring compliance.

For router-backed calls, it may be useful to record several levels separately:

1. the route or alias requested by ICC;
2. the routing decision reported by the gateway;
3. the response model or ID reported in the result;
4. the values that ICC persisted.

For example, OpenRouter router metadata can expose provider selection, attempts, fallbacks, and other routing decisions when enabled. It is useful evidence, but it is not necessarily available for every response and should not be treated as the only source of execution identity.

Verifiers are executions too

A stored value such as passed: true is useful, but it may become ambiguous if the verifier prompt, rubric, model, or threshold changes later.

It may be clearer to model verification as another execution over a particular artifact:

Generation run
  └─ produced Artifact A
       └─ consumed by Evaluator run E
            ├─ used evaluator definition V
            └─ produced verdict/score R

That makes it possible to:

• reevaluate the same generation artifact without rerunning generation;
• compare evaluator changes separately from generation changes;
• explain why a previously passing output receives a different result later.

Systems such as Phoenix LLM evaluators explicitly version evaluator definitions. ICC may not need the same machinery immediately, but retaining the evaluator identity seems valuable if verifier results affect workflow control flow.

Branch decisions

The reason a next step ran can also be recorded as an execution event rather than only as a prose log:

rule or condition revision
evaluated input/value
selected branch
supporting verifier or artifact
human override, if any

This keeps “why the branch ran” connected to the exact evidence and rule involved.

These do not all need to become immediate implementation tasks. They are small controls that can clarify the intended semantics before the storage model becomes difficult to change.

1. Historical upstream binding

Run c1.
Run c2 using %from c1.
Rerun c1.
Inspect the old c2.

Possible outcomes:

• Old c2 retains the original c1 occurrence: the desired execution lineage is already represented.
• Old c2 follows the current c1 value: store the resolved upstream run/artifact occurrence on c2.
• It works locally but not after export: strengthen the export representation rather than changing runtime resolution.

This is the highest-value control because it tests the central distinction between an editable reference and a historical execution dependency.

2. Same-name artifact versions

c1 produces report.md version 1.
c2 consumes report.md.
c1 reruns and produces report.md version 2.
Inspect both the old and new c2 runs.

The important point is not whether ICC chooses version 1, version 2, or “latest” during authoring. The important point is that:

• the resolution rule is explicit;
• a completed downstream run records the version it consumed;
• the old record does not move when an alias moves.

This is similar to systems where a mutable alias such as latest is convenient for selection, but a run records the concrete selected artifact version.

3. Export guarantees

There may be two reasonable export modes rather than one universal guarantee.

Portable or attached export

Use this when the export is expected to stand alone:

• include the required payloads;
• include their IDs and digests;
• preserve upstream/downstream relationships;
• allow reconstruction in a clean profile.

Detached or reference export

Use this when payloads are too large, private, externally managed, or deliberately omitted:

• preserve IDs, digests, media types, and references;
• distinguish redacted, not retained, and currently unavailable;
• do not imply that the package is independently reproducible.

RO-Crate makes a similar distinction between attached/self-contained and detached/reference-style crates: RO-Crate structure.

Making the export mode explicit may be more useful than requiring every export to contain every payload.

4. Routing and fallback

If routing provenance becomes important, compare a few representative outcomes:

The purpose is not to expose every internal router detail. It is to avoid treating the requested alias as though it were necessarily the model that produced the answer.

5. Verifier drift

Evaluate the same generation artifact twice while changing only the verifier definition or judge model.

• If the result changes, the difference belongs to evaluation provenance.
• If generation is rerun as well, generation drift and evaluator drift become mixed.
• Keeping the generation artifact fixed gives a clean control.

These seem less urgent than historical input binding, but defining them before connectors and durable storage arrive could avoid ambiguity later.

Rerun, replay, resume, and fork

Those words can represent different operations:

The names do not need to be exactly these, but it helps if the UI and records make clear whether external work will be performed again.

This becomes particularly important with connector cells. Retrying an LLM request is usually different from retrying an email send, database write, purchase, or file mutation. External side effects may require idempotency keys or an explicit user decision.

Partial and terminal outcomes

Streaming introduces at least two different facts:

• text that was visible temporarily;
• output that was durably committed as the run result or artifact.

A small lifecycle contract could define:

• what counts as a completed, failed, or aborted run;
• whether partial output is retained;
• whether a failed attempt remains visible after a successful retry;
• when an artifact becomes committed and addressable.

The exact status vocabulary matters less than avoiding silent replacement of failed or partial attempts by the final successful result.

Stale semantics

“Stale” can have several valid meanings:

1. the upstream run occurrence changed;
2. the upstream content changed;
3. the upstream cell definition or constraints changed.

Those choices produce different behavior.

For example, if c1 is rerun but produces the same content digest:

• occurrence-based semantics would mark c2 stale;
• content-based semantics might leave c2 current;
• intent-based semantics might depend on whether c1’s definition changed.

Dagster’s asset versioning model is useful prior art here: it distinguishes materialization events from data versions and can treat downstream data as unsynchronized when the version it consumed is no longer current.

ICC can choose a simpler policy, but documenting which identity drives stale would make run comparison much easier to interpret.

First-divergence comparison

A useful run comparison view might report the first layer that differs, approximately in this order:

1. notebook/workflow or cell definition;
2. resolved upstream run or artifact occurrence;
3. input content identity;
4. requested parameters and routing;
5. observed model/provider execution;
6. retrieval or tool evidence;
7. generated output;
8. evaluator definition and result.

That would be more diagnostic than only diffing the final text. It also gives the manifest a concrete user-facing purpose: finding where two executions stopped being comparable.

Retention and privacy

Complete provenance does not necessarily require retaining every raw payload forever.

A record can still preserve:

• that an input existed;
• its type and identity;
• its digest, where appropriate;
• whether it was redacted, truncated, externally referenced, or not retained;
• which run consumed it.

Secrets themselves should normally remain outside exported run manifests. A credential binding or provider profile identity may be useful, but not the credential value.

Schema evolution

Once exports are shared, the historical schema becomes part of the product contract.

Some lightweight protections would be:

• version the manifest and DSL schemas;
• preserve original IDs through migration;
• distinguish unsupported fields from absent fields;
• make lossy migrations visible;
• report missing artifacts rather than silently dropping their references.

This can remain modest at first. The main requirement is that an old run should not appear complete after information required to interpret it has been discarded.