Hmm… maybe something like this:
The part that looks most useful to me is not really the placeholder content itself. It is the fact that you have made the workflow inspectable and replaceable before making it fully live: deterministic artifacts, persisted stages, approval invalidation, integrity checks, explicit network/cost boundaries, and provider interfaces are all much easier to reason about in that order than after several remote generators are already wired together.
And for the “what would you replace first?” part: I would probably replace only the image fixture first, and treat that as a conformance test for the architecture rather than as a quality upgrade.
Something roughly like:
request
↓
provider adapter / SDK translation
↓
real provider response
↓
local materialization
↓
hash + provenance
↓
QA
↓
human-review package
One real image is cheap enough that it can exercise most of the interesting boundaries without immediately bringing in the cost/latency/job-lifecycle complexity of video.
The important test, to me, would not just be “did an image come back?” It would be closer to:
- did the requested parameters actually survive the adapter boundary?
- did the remote provider actually honor the requested semantics?
- was the returned object normalized/materialized the way the local workflow expects?
- are failure/filter/rate-limit cases mapped into useful local states?
- does cached reuse still preserve the provenance/integrity guarantees you want?
- does the result arrive at the same human-review boundary as the offline path?
That is also fairly close to the problem Hugging Face itself has to solve in Inference Providers: for tasks such as text-to-image there is no universally shared provider API, so HF keeps a task-level contract, translates provider-specific parameters/results, tests those translations, and then periodically validates real mapped endpoints for output compatibility.
So if it were mine, I would spend the next unit of complexity on one cost-bounded real-image vertical slice, and let that tell me which abstractions actually need to become more general.
One small thing I noticed while poking at that path
I also tried a small non-live cache/integrity probe against the current image-provider path. No paid API call was involved, so this is only an observation about the local caching/materialization logic, not about OpenAI’s server behavior.
The result was interesting because two integrity boundaries seem slightly different.
If I altered the cached image’s recorded output SHA, reuse was rejected as expected.
But if I kept the media bytes and output SHA unchanged and altered only some provenance metadata — for example the recorded model, prompt hash, reference-image hash, or recorded dimensions — the same request could still reuse the cached result, leaving those modified provenance fields in place.
Very roughly:
| Cached mutation |
Reuse result |
| Nothing changed |
reused |
| Recorded model only |
reused |
| Recorded prompt hash only |
reused |
| Recorded reference hash only |
reused |
| Recorded dimensions only |
reused |
| Output SHA |
rejected |
I would interpret that narrowly: the media-byte integrity check is stronger than the binding of some provenance metadata to that cached artifact/request. I would not call that a broken design by itself, especially while this is still an experimental live-media layer.
But it seems like a nice place where the Phase 1 integrity idea could eventually carry into Phase 2.
There are several small ways to draw that boundary, depending on what guarantee you want:
A. Bind/hash the provenance record itself.
B. On cache reuse, recompute and compare the
request-bound provenance fields.
C. Keep the experimental live path looser for now,
and tighten that binding only when it joins the
resumable/approved workflow.
I actually like C as a legitimate option too. The useful question is not necessarily “how can every metadata field be protected immediately?” but rather “at which boundary do we want to promise that provenance is part of the verified object?”
So my rough order would be:
- Keep the offline fixture workflow as the reference baseline.
- Run one real image path as a bounded conformance slice.
- See where the abstraction really bends before generalizing it.
- If the live path is meant to inherit the same integrity guarantees, tighten the artifact/provenance binding there.
- Only after that decide whether model revisions, richer resume invalidation, provider capabilities, async job semantics, etc. actually need to move into the common contract.
That seems higher-information to me than adding several providers at once.
What I mean by a provider conformance test
There is a subtle distinction here between:
the provider call returned successfully
and:
the requested behavior survived every translation layer
and the returned object still means what the common
contract says it means
The latter is the interesting one.
A useful recent example is LiteLLM issue #37125. It is not the same system and I am not suggesting the same bug exists here, but the failure mode is instructive: a familiar structured-output parameter could be accepted and then silently dropped. The API call still succeeded, so a test that only checked for a successful response would pass even though the requested constraint had disappeared.
The reporter’s stronger test was effectively:
not only:
did the request succeed?
but:
did the semantic constraint actually appear
in the request sent downstream?
That is a useful general test oracle for provider adapters.
For an image provider, I would think in terms of a small matrix such as:
| Boundary |
Cheap check |
| model selection |
requested model identity survives translation |
| dimensions |
output dimensions match the requested contract |
| quality/mode |
unsupported values fail loudly or are explicitly degraded |
| reference image |
reference really reaches the provider path |
| provider failure |
becomes a useful local failure category |
| malformed output |
rejected before approval |
| materialization |
byte/hash/path checks still hold |
| provenance |
request/result identity can be reconstructed |
| cache hit |
cached result still satisfies the same request identity |
| review |
live result reaches the normal review boundary |
The nice part is that your current local/fake-transport tests already cover a lot of the second half of this. That makes the first live request more informative: it is mostly testing the last unknown translation boundary rather than trying to test the entire pipeline at once.
HF’s current Inference Provider integration guide is a useful comparison here. Their common task schema is deliberately separate from provider-specific translation, and their model mappings are subsequently tested against real endpoints for reachability and output compatibility. For LLMs they even add behavioral checks such as tool calling and structured output.
I would borrow that testing idea, not necessarily the implementation.
A little more detail on the provenance/cache observation
The small probe I tried was conceptually:
1. Produce a valid cached image artifact using a fake transport.
2. Confirm normal reuse.
3. Keep the media bytes unchanged.
4. Modify only selected fields in the saved provenance record.
5. Submit the identical request again.
6. Observe whether the provider path is called again.
7. As a positive control, modify the recorded output SHA.
The positive control mattered: changing the output SHA caused the integrity path to reject the cached result, so the probe was not simply bypassing all validation.
The interesting distinction was that changing provenance-only fields did not have the same effect.
This reminds me of a more general distinction between:
artifact integrity
and:
integrity/binding of claims made about that artifact
There are much heavier standards for this in other domains, but I would only use them as vocabulary here, not as dependencies.
For example, SLSA provenance distinguishes output artifacts from the parameters and resolved dependencies describing how they were produced. The point is that provenance is useful downstream only if consumers know which inputs/parameters are actually part of the claim they are verifying.
Likewise, media-provenance systems such as C2PA have an explicit concept of binding provenance information to the asset it describes.
I do not think this project needs to adopt SLSA or C2PA just to solve this. The smaller design question seems enough:
Is the provenance JSON currently descriptive debug metadata, or is it meant to become part of the verified identity of a generated artifact?
If it is mainly descriptive metadata during the experimental phase, the current boundary may be perfectly reasonable.
If it is eventually meant to participate in approval/reproducibility guarantees, then a lightweight binding becomes more useful.
Two small implementations I could imagine are:
cache registry
request key
output digest
provenance digest
or:
on reuse:
verify output digest
verify stored request identity
verify the subset of provenance fields
that are supposed to be request-derived
The second option might actually be nicer if some provenance fields are deliberately mutable annotations.
That is why I would frame this as a contract question rather than “hash the entire JSON file”.
A later reproducibility question: moving aliases vs pinned model revisions
This probably does not need to block the first live experiment, but it may become relevant if “reproducible” is eventually intended to include remote generation as well as the offline core.
The current GPT Image 2 documentation exposes both the moving alias and a dated snapshot:
gpt-image-2
gpt-image-2-2026-04-21
OpenAI describes snapshots as a way to lock a particular model version so that behavior/performance remains consistent; see the current GPT Image 2 model page.
The current image request validation appears intentionally narrow around the gpt-image-2 alias. That seems fine for an experimental “use the current model” path.
If reproducible verification later becomes a goal, though, there might be value in separating two modes:
latest / exploratory:
gpt-image-2
pinned / reproducibility-oriented:
gpt-image-2-2026-04-21
Or, more generally:
user-facing model choice
+
resolved provider/model revision recorded in provenance
I would not necessarily add that abstraction before the first real run. It is exactly the sort of thing that can wait until there is evidence that live-run reproducibility is a requirement rather than just offline-workflow reproducibility.
If the experimental live path eventually joins the resumable graph
This is where I think the Phase 1 / Phase 2 separation becomes especially useful rather than something to eliminate early.
The offline graph has one notion of identity:
topic
brand/profile
previous stage artifacts
stage
...
↓
resume / reuse decision
The experimental live-media path already has a richer request identity involving things such as:
provider
model
prompt hash
reference hash
generation options
...
↓
remote request / dedupe identity
When those two worlds eventually meet, the interesting question becomes:
Which live-generation facts count as dependencies of a resumable stage?
For example:
same topic
same source packet
same prompt
but
different provider model revision
Should that be a cache hit or a new stage result?
There is no universally correct answer; it depends on what “same result” means for this project.
A useful analogy comes from workflow/build systems. Nextflow’s cache/resume model, for example, computes task identity from substantially more than the obvious input files: the process script, container/environment, referenced globals, bundled scripts, and other execution inputs can participate in the task hash. The purpose is to avoid reusing a result after an undeclared dependency changed.
So I would phrase the future question less as:
Should the model name be in the hash?
and more as:
Once live generation becomes resumable,
what counts as an undeclared input?
Possible candidates might eventually include:
- provider
- exact model/revision
- adapter implementation/version
- prompt/template revision
- generation configuration
- reference-artifact digest
- external tools that materially transform output
But I would not automatically put all of those into one giant cache key. Some may belong in provenance, some may invalidate a stage, and some may be irrelevant to the guarantee you want.
That separation is probably easier to decide after the first live vertical slice exposes which values actually matter.
Why I would stop at image before going to video
This is mainly an information-per-cost argument.
A real image path already exercises:
network
authentication
provider-specific API translation
cost authorization
remote generation
content-policy/provider failure
binary materialization
hashing
provenance
QA
human review
A video provider adds another class of problems:
long-running jobs
polling / callbacks
cancellation
partial completion
larger artifacts
higher latency
larger cost
Those are worthwhile eventually, but they can make it harder to tell whether a failure came from the provider abstraction or simply from the extra lifecycle complexity of video.
This pattern shows up in other content-production systems too. For example, coleam00/ai-content-factory deliberately uses cheaper image exploration and human approval before the more expensive video-rendering stage. It is a different project with different goals, but the economic logic is useful: learn as much as possible while the artifact is still cheap.
So I would use image as the first real pressure test, then make video the test of whether the abstractions also survive long-running jobs.
If more providers are added later
I would probably resist building a large capability system until a second or third real provider actually proves it is needed.
But if the abstraction starts being pulled in incompatible directions, one escape route is:
small common contract
+
explicit optional capabilities
rather than forcing every provider into the lowest common denominator.
For example, differences might eventually appear around:
supports_reference_image
supports_fixed_revision
supports_async
supports_cancel
supports_seed
supports_multiple_outputs
That is broadly the kind of tension HF handles by keeping task-level schemas while allowing provider-specific translation/implementation underneath them.
The important part is probably to avoid either extreme:
everything is provider-specific
or:
every provider must pretend to support exactly
the same semantics
Again, I would wait for real provider #2 or #3 before generalizing this. The first provider is more useful as evidence than as a reason to invent the final abstraction.
So, condensed down to one path, I think I would do this:
offline deterministic baseline
↓
one bounded real-image conformance run
↓
compare promised contract vs actual provider behavior
↓
tighten only the boundary that real evidence exposes
↓
then decide whether the next pressure test should be
resume identity, another provider, or video
That preserves what seems useful about the current direction: the live/expensive/nondeterministic pieces do not get to dictate the whole architecture prematurely.
And it keeps the next experiment fairly cheap. Rather than generalizing several things in anticipation, the first real provider can tell you which abstractions actually need to move.
on GitHub — it helps more people discover the project.