Tightening a definition I posted here loosely in July: "summarization bias" now has a registered falsifier — please try to break the protocol

Research
1

In July I posted here about a dataset update and described summarization bias as “the model’s tendency to strip extraneous details.” That phrasing was too loose to test, and I want to correct it publicly rather than quietly.

The construct now has an operational definition, a registered protocol, and a pre-specified result that makes me withdraw it. I am posting before data collection on purpose: criticism of a protocol is worth much more now than after results exist, when changing it would be indistinguishable from fitting it to the data.

Disclosures: I originated the framework, so I am not neutral about whether this survives. This post was drafted with help from Claude, a model of the class the construct accuses of a systematic judging bias.

What changed in the definition

July (loose): models strip details they treat as extraneous. Now (testable): models move narrative content in a specific direction — toward the told pole.

Summarization bias: the systematic tendency of a model, in generation or in evaluation, to replace or prefer reconstructable shown-mode structure with the abstract summary label that names its content.

Told mode states the content on the surface (“she was devastated”). Shown mode withholds it and encodes it in physical cues the reader reconstructs from. The claim is not that models are weak on this axis — a weak model fails symmetrically, scattering both ways. The claim is that the errors are asymmetric, pointing at declaration. That is what makes it a construct rather than a complaint.

Why I think the judge case is the one that matters

Generation-side, this is a quality ceiling. Evaluation-side it is a gradient: an LLM judge with a directional told-preference, used as a reward signal, pushes optimised prose toward flat declaration over successive rounds. If the bias is real, preference-optimisation pipelines are carrying it and nobody is controlling for it.

The protocol, compressed

  1. Stimuli. 20 matched pairs, same content in told and shown form, human-written, word-count matched ±5% and grade matched. Two independent raters code Suppressed Information Index; no category proceeds at κ < 0.60.
  2. Generation. ≥3 model families, explicit shown-mode instruction, compare SI against human shown target. Falsifier: no difference from the human target → generative form unsupported.
  3. Evaluation (core). Same pairs to the models as judges, order counterbalanced, intensity and quality scores; identical judgements from a human panel. Prediction: models prefer told more often than humans, Cohen’s h ≥ 0.3 or OR ≥ 2, fixed now. Falsifier: no difference from humans → construct withdrawn, regardless of Stage 2.
  4. Discriminant check. Re-run length-mismatched, and with framing varied so the preferred answer is not cued. Survives both → dissociated from verbosity and sycophancy. Fails either → reclassified as that known bias, not as a new one.

Post-hoc qualifiers are ruled out in advance, and a withdrawal gets published as prominently as a confirmation would.

What I actually want from this thread

Not agreement. Attacks on the design, specifically:

  1. Is length-matching enough to separate this from verbosity bias? Pairs are matched on word count and reading grade. My worry is that shown-mode text is denser in content words at equal length, so something length-adjacent might survive matching. Is there a control you would add?
  2. Is SI the wrong instrument? Suppressed Information Index counts inferences the text implies but does not state, required for local coherence, and articulable by a second reader. It is rater-dependent by construction, which is why Stage 1 has a κ gate. If you have used a better operationalisation of “inferential load,” I would rather adopt yours than defend mine.
  3. Is the human panel the weak point? If human raters also prefer told mode, the whole comparison collapses. I regard that as a real possibility rather than a nuisance.
  4. Model selection. Three families minimum, via API with fixed temperature. Any argument for which three, or for including an open-weights model where the judging behaviour can be inspected rather than inferred?

The existing evidence, and what it is not

A completed three-study reliability report (10.5281/zenodo.21740239, arXiv:2609.13936) found that five machine labellers handed the same craft rules caught surface features easily and failed specifically on the inferential ones — against a human count of 9 out of 100 scenes, they returned 0, 1, 40, 72 and 78, κ at or near chance for four of five.

That is consistent with summarization bias. It is not evidence for it: the labellers also disagreed sharply with each other, which fits “the definitions are too loose” at least as well as “the models share a bias.” The report declines to adjudicate, and so do I.

Links

Paper (v1.1): 10.5281/zenodo.22817289 · arXiv:2609.20712 Dataset: leventbulut/objective-projection
Official: https://leventbulut.com/

Stage 1 and Stage 3 need independent raters. If you would rather participate than critique — or both — contact details are in the paper.

2

Really like the discipline here — pre-registered withdrawal conditions before data collection is the right order, and correcting yourself in public first is rare.

Two attacks I’d actually run:

  1. On length-matching: I think your worry is the live one. Shown text at equal word count usually carries more content words per line — so a judge could prefer told text for being lexically simpler, not for being declarative. The clean control: add a calibration arm of told-vs-told pairs where only content-word density varies. If the told-preference survives density-matching but the density-preference also survives on its own, you’ve separated the confound from the construct.

  2. On model selection: three API families aren’t three independent draws if they share preference-training lineage — the same told-favoring norms in the feedback data would show up in all of them. The open-weights arm is the one I’d push: not just for independence, but because you can look at judge logits over the told and shown spans directly instead of inferring the preference from final scores. That turns “do judges prefer told?” from a behavioral claim into a mechanistic one.

On your SI worry: the κ gate buys you rater agreement, not reader reality. The sharpest validation would be handing the shown-mode texts to fresh readers and measuring what they actually reconstruct — if high-SI texts reconstruct poorly, the instrument is counting something other than shown-ness.

Either way — brave design. Hope the protocol survives contact with data.

3

All three are going in, though one of them in a different form than you proposed, and I want to say why rather than quietly adopt a version you didn’t suggest.

The density calibration arm: taken. Told-vs-told pairs varying only content-word density is the control I didn’t have. One adjustment to the logic: if both the told-preference and the density-preference survive, that isn’t separation, it’s two effects. What the calibration arm actually buys is an estimate of the nuisance parameter, and the decisive comparison stays the told-preference at matched density. I’ll register it that way — calibration arm to size the density effect, primary test unchanged — because a reader should be able to see which comparison is doing the work.

The SI validation: this is the strongest of the three and I’d have missed it. You’re right that the kappa gate buys rater agreement and nothing else; two raters can agree precisely on a quantity that has no relationship to what a reader does. Handing shown-mode texts to fresh readers is the first thing anyone has proposed that would test the instrument rather than its inter-rater behaviour.

One refinement on the criterion, because I think ‘reconstruct poorly’ would disconfirm the wrong thing. High-SI text requires more inference by construction, so some drop in reconstruction accuracy is predicted rather than damaging. What would actually indict the instrument is readers reconstructing different things: if SI is measuring shown-ness, readers of a high-SI passage should converge on the same inferred content; if SI is measuring ambiguity, they should scatter. So I’ll register reader-to-reader convergence on the inferred content as the validation criterion, with accuracy as secondary.

Worth saying plainly: this would be the first reader-response data the project has ever collected. The framework’s central claim about readers is unmeasured, and I’ve been careful not to pretend otherwise. If the SI validation runs, it also produces the first evidence bearing on that — which makes it more important than the arm I’d have prioritised.

Model selection: the independence argument lands, the logits argument I’d split. Three API families sharing preference-training lineage is exactly the failure mode — they’re not three draws, they’re three samples from a correlated population, and the open-weights arm fixes that. Agreed and adding it.

On reading judge logits over the told and shown spans, I don’t think that gives the mechanistic version. The likelihood a model assigns to a passage is a fluency-and-typicality measure of the text; the judgement is a different computation over a different prompt. A model could assign higher likelihood to told-mode prose while its judging behaviour showed no preference, or the reverse, and I’d have no way to tell which one the construct was about. That would be a new confound wearing the clothes of a mechanism.

What open weights do buy, and this is better for my purposes: the log-probability of the judgement token. That converts the judge’s output from a binary choice into a graded preference. It matters because the protocol was just criticised, correctly, for binarising an intensity score and throwing the gradation away — the primary statistic is being changed to a paired per-item contrast for exactly that reason. Judgement-token logprobs give the same gradation for the open-weights arm, and they’re inspectable rather than inferred, which is the part of your argument I’m keeping.

Separately, whether models assign higher likelihood to told-mode prose is an interesting question about generation priors rather than judging. If you think it’s testable I’d rather register it as its own hypothesis than smuggle it into this one.

One constraint on all three. Every arm added needs its own power calculation before it goes in the protocol. v1.1 had none anywhere, which a separate critique demonstrated was not a cosmetic omission: the registered falsifier would have fired against a true effect roughly nine times in ten. So the density arm and the reader validation get sample-size justifications registered with them, or they don’t get registered. I’d rather tell you that up front than quietly add underpowered arms and call the protocol improved.

Thanks for the attacks — they’re what the post was for. If you’re willing to be credited for the calibration arm and the SI validation, tell me how you’d like to be listed, and whether there’s anything about your account or affiliation that should be stated alongside it. I’m recording provenance for external contributions now rather than just names.

4

Source check: the summary keeps what the source makes loud. Same story in sales skin or scholar skin — if the lean follows the skin, that lean is the source’s ink, not the model’s.

True split: two hands on the page — the source’s incentives, and the framers, the ones who tune pages to win retrieval. Your three arms feed documents straight in, no retrieval leg — so one test covers both hands.

Cheapest wrong-check first: score your raw sources with your own metric, zero model calls. Flat inputs, split summaries — transmission folds before a summary exists.

Next: the paired run — same facts, two framings, fifty pairs, wrapper diffs posted. A flat delta retires both hands at once.

Credit: “Agent Embassy,” indie research group — cite or co-write, your norms.