For now, the latest decomposition looks quite good:
I think MODE=emit is a much cleaner generation contract than the earlier plant/hold variants.
The part I would keep as the default path is roughly:
zero input edit
+ token-row rotation at the final readout
+ steer only while that target is unresolved
+ stop the positive intervention as soon as the target is actually sampled
+ let the emitted token enter context normally
That separation seems useful because the latest run now has a proper pure baseline beside it: on the reported 9-scene test, the pure model produced 0/27 of the out-of-place target words, while the steered runs produced 24/27 across 8/9 coherent scenes, with zero input edits, zero suppression, and no degenerate loops in the MODE=emit configuration. The farm case is also informative precisely because the pure run collapses there too; that looks like a scene/model-distribution failure rather than something that needs to be attributed to word steering. The full setup and reproductions are in the forum thread.
There are two small refinements that I think make this decomposition even cleaner.
First, at this particular final-readout boundary, I would read “bending the live trajectory” mainly as changing the next transition on the live trajectory, rather than literally moving the hidden representation onto some later natural hidden-state trajectory.
Second, I am no longer sure that theta and lambda should be treated as two independent mechanistic axes. emit really does look like a distinct control boundary; theta and lambda, however, are algebraically coupled at the same linear LM head.
1. The emit boundary seems mechanically very clean
I ran a small Qwen2-1.5B canary specifically to separate the current readout intervention from what persists into the next autoregressive step.
For a target token, define:
S1 = ordinary context + target once
S2 = ordinary context + target twice
...
I compared:
cached clean S1
+ feed one real target token
vs
clean full forward on S2
for castle, elevator, and camel.
The cached-next-state and clean-S2 results were essentially the same numerically:
hidden cosine: ~1.0
hidden max abs diff: ~4.6e-5 to 1.3e-4
logit max abs diff: ~1.8e-5 to 2.5e-5
I would not call that a mathematical identity because cache/full-forward order and finite precision differ, but it is very consistent with the implementation boundary in Transformers Qwen2: the decoder produces the final normalized hidden state, then the separate LM head produces logits. The autoregressive cache holds attention K/V state; it does not preserve a post-hoc rotated LM-head readout vector. The general cache contract is described in the Transformers cache documentation.
So for this final-readout intervention, the causal picture that currently makes the most sense to me is:
ordinary model state
|
| rotate current readout toward token row
v
change this next-token distribution
|
v
sample the real target token
|
v
put that token into the actual context
|
v
ordinary transformer computes the next state again
That also gives a fairly concrete reason why the emit stop-rule is such a natural boundary: once the desired token is real context, the temporary readout edit has already done its job.
I also compared a steered S1 against the natural repeated-token states S1...S8. Strong steering could make the target dominate the output distribution, but in hidden-state angle the steered state still remained closest to S1, not to a later natural S4/S8. So I would distinguish:
not strongly supported:
steering S1 -> hidden representation equivalent to natural later S_k
better supported:
steering S1 -> alternate current readout
-> different next-token transition
-> real token changes subsequent natural context
That distinction does not make the generation result weaker to me; if anything it gives it a simpler component boundary.
There was another useful consequence. One successful target transition is not necessarily enough to create a self-sustaining repetition basin. In one frozen-prefix probe, stable natural rank-1 did not arrive until different S_k values for different tokens (castle around S3, elevator later than its first temporary rank-1 crossing, camel around S6), and releasing immediately from clean S2 did not create sustained loops.
So fixed hold can be understood as repeatedly controlling several successive decisions:
S_k
|
| temporary steering
v
target-side transition
|
v
real emitted token
|
v
ordinary S_(k+1)
|
+-- goal already satisfied -> stop
|
+-- still need control ----> steer another decision
That seems compatible with your observation that continuing the positive force after success can collapse into repetition, while MODE=emit avoids doing unnecessary work after the lexical goal has already been met.
Repetition self-reinforcement itself already has fairly direct precedent—for example Xu et al., “Learning to Break the Loop” (NeurIPS 2022) and Yan et al., “Understanding In-Context Learning from Repetitions” (ICLR 2024). So I would probably keep the geometric claim narrower: the token-row intervention can act as a controllable entry/transition mechanism, while persistence after enough real tokens have accumulated is ordinary autoregressive dynamics.
2. lambda looks useful, but it may mostly be an effective-angle knob here
Your latest four-scene ablation is a nice separation empirically:
thetatoo weak: words do not land;- fixed hold: loops;
MODE=emit: much cleaner;lambda=1: stronger landing but some continuations become stiff/template-like.
The only part I would reparameterize slightly is the interpretation of lambda.
At the same affine LM head, with
and
the mixture
is exactly, in real arithmetic,
If h and h_rot have the same norm and h_rot is a geodesic rotation through angle theta, then the chord points along a smaller effective rotation:
with magnitude factor
For the reported base setting:
theta = 8°
lambda = 0.9
theta_eff ≈ 7.2019°
rho ≈ 0.999124
So 8° / lambda=.9 is, geometrically, almost a 7.20° / lambda=1 direct rotation plus a very small common logit-scale change.
I tried exactly that matched comparison on the four scenes from the lambda ablation, using Qwen2-1.5B, bf16, seed 0, the same 20/50/80 windows, W=12, and MODE=emit:
A = theta 8°, lambda .9
B = theta 7.2019°, lambda 1
C = B with the rho scale
D = theta 8°, lambda 1
The small result was:
| condition | target hits | mean natural NLL during active windows |
|---|---|---|
| A: 8° / .9 | 11/12 | 8.72 |
| B: 7.20° / 1 | 11/12 | 8.64 |
| C: 7.20° / 1 + rho | 11/12 | 8.64 |
| D: 8° / 1 | 12/12 | 10.43 |
More importantly, A and matched-angle B/C were extremely close at the active decisions:
A vs B:
mean KL ~3.5e-4
same argmax 136 / 136 active states
mean |delta p(target)| ~0.0031
A vs D:
mean |delta p(target)| ~0.155
For library, train, and office, A and B generated byte-identical 120-token continuations in that run. Kitchen diverged after 25 output tokens, but the same-state distributions were still very close and had the same argmax; that looks compatible with a tiny probability difference crossing a multinomial sampling boundary and then being amplified autoregressively.
One finite-precision caveat: although A and the chord-equivalent C are algebraically identical in exact arithmetic at one linear head, they were not bitwise identical in bf16 because the branches are cast/quantized separately before the head. The measured KL remained small and all active-state argmaxes agreed, but I would phrase the actual bf16 result as near-equivalent, not exactly identical.
So my current interpretation would be:
emit:
genuinely separate success/stopping rule
theta + lambda:
mostly determine effective angular push
plus a very small scale/temperature-like component
lambda can still be a perfectly useful user-facing attenuation control. I just would not yet interpret theta = landing and lambda = composition preservation as fully independent mechanisms. The observed lambda=1 behavior may largely be the stronger effective push: it lands more targets, but it also drives the sampled distribution farther from what the natural model would have chosen.
This is also related, at a higher level, to other decode-time methods that retain some base-model distribution while adding control. PPLM explicitly uses the unperturbed model as a fluency anchor, and DExperts combines distributions in a product-of-experts style. I would not equate those methods with this one—the useful special case here is that both distributions come from the same LM head, so the logit mixture has the simple hidden-chord interpretation above.
At this point I do not think a broad lambda sweep is necessary. If you ever wanted one last cheap confirmation, the sensitive kitchen case over a few seeds, comparing 8°/.9 against matched ~7.20°/1, would probably be enough. The per-state distributions are more informative than whether two stochastic paragraphs happen to remain byte-identical.
Where this leaves the practical use case
For exact word inclusion alone, I agree there is not yet an obvious reason to replace existing lexical-constraint methods. There is a long line of direct constrained decoding work, including Grid Beam Search and Dynamic Beam Allocation, and Hugging Face generation has also exposed constrained/forced-word decoding interfaces.
The niche here looks different to me:
white-box model access
+ no input-token edit
+ no auxiliary classifier/model for an exact token
+ ordinary stochastic decoding
+ local inference-time intervention
+ release control immediately after success
That is a reasonably crisp primitive even if constrained decoding remains preferable when the only requirement is “this literal word must appear.”
The more interesting divergence would be if the row-level construction eventually generalizes from
one exact token
to
token neighborhood
or
distributed concept direction
because then the comparison is no longer simply against lexical constraints.
I would keep that as a future branch for now, though. Your current result that concept-centroid/theme steering has not yet produced meaning independently when the constituent tokens are blocked is useful negative information; I would not try to force a concept claim before that changes.
Two adjacent rotation-steering papers that may help sharpen the novelty boundary
I found two pieces of work close enough to the broad “steer by rotating activations” idea that I think they are worth citing, mostly because they make the distinctive part of this project easier to state.
Angular Steering: Behavior Control via Rotation in Activation Space (NeurIPS 2025) explicitly rotates activations toward/away from behavior directions and treats angle as a continuous control parameter.
Spherical Steering: Geometry-Aware Activation Rotation for Language Models likewise uses norm-preserving/geodesic activation rotation for inference-time control.
So I would probably avoid putting the novelty boundary at:
norm-preserving spherical/geodesic activation rotation itself
The more distinctive combination here seems to be something like:
target direction = an actual LM-head token row
+ explicit token decision-cell / competitor geometry
+ first-rank-1 angular accessibility
+ random-head null vs trained/depth-dependent deviations
+ token-specific repetition/pit structure
+ final-readout MODE=emit as a success-terminated token-transition primitive
That is a much narrower claim, but in my view a more recognizable one.
A few cheap cleanups I would still make on the semantic-topography branch
I would not block the generation work on these, but there are a few small things that seem worth fixing before leaning hard on the semantic/topographic percentages.
S3 label-free spherical k-means: keep the sampled vocabulary IDs
In the current eval_som_sweep.py logic, the pattern is effectively:
X = Wn[rng.choice(V, N, replace=False)]
_, bmu = kmeans_fit(X)
fam = torch.where(bmu_c == k)[0].numpy()[:10]
rows = Wn[fam]
The issue is that fam is an index inside the sampled matrix X, but it is then reused as a vocabulary-row index. The original sampled vocabulary IDs have been discarded.
The small fix is:
sample_ids = rng.choice(V, N, replace=False)
X = Wn[sample_ids]
...
fam_pos = torch.where(bmu_c == k)[0].numpy()
fam = sample_ids[fam_pos]
rows = Wn[fam]
I would also seed Torch there because the clustering helper uses Torch-side random permutations.
This looks like a bookkeeping bug rather than a conceptual problem with the semantic branch, but I would rerun the S3 label-free percentages before citing them.
I would soften the SOM conclusion
The current comparison treats data-to-data nearest-neighbor distance as an “intrinsic angular resolution no tiling can beat.”
A learned vector-quantization prototype is not required to coincide with a data row, so data 1-NN distance is not a general lower bound on prototype quantization error. The current 400-row nearest-neighbor statistic and the 20k-row SOM fit are also not quite like-for-like.
The result safely supports:
this SOM configuration did not substantially improve quantization error.
I do not think it yet supports:
therefore there is no low-dimensional manifold.
A cheap matched check would be:
same sample
same K
best of a few seeds
random prototypes
vs
spherical k-means / VQ
vs
SOM
No larger manifold project is needed unless those start separating.
The “BOS axis” looks more like a position-0 lexical anchor in this assay
Where the code uses something like:
tok("Once upon a time", add_special_tokens=False)
hidden_states[..., 0]
the state is the first lexical token’s position-0 state, not an actual inserted BOS state.
I would probably rename it position-0 lexical anchor unless the test is rerun with a real special-token BOS contract.
Frequency is a worthwhile cheap control for the row-map claims
There is direct evidence that output embeddings encode output-token probability structure, and that corpus-frequency structure appears early during pretraining: Cho et al., “Understanding Token Probability Encoding in Output Embeddings” (COLING 2025).
So for the six hand-labeled semantic classes, I think the highest-value controls are fairly small:
frequency-matched controls
leading-space / token-form matching
case/register matching
label permutation
bootstrap class members
If the intra/inter separation and KNN enrichment survive those, the semantic interpretation gets much easier to defend.
Centroid failure does not require the centroid to be “far” from the members
I would phrase the chord result in decision-region terms instead.
For a family of token rows, family success is a union of individual token decision cells. A normalized family centroid can have very good average cosine alignment with the members while still diluting the member-specific margins enough to sit in none of those cells.
That seems compatible with the measured result that actual member directions work much better than the centroid, without needing the geometric claim that the centroid is farther from every member.
I would keep the topic-ring/circular-order picture exploratory until the six-class PCA order survives bootstrapping, leave-one-class-out, and frequency/register controls. The high-dimensional class separation is a safer result than the one-dimensional ring at the moment.
One remaining split on the Pythia checkpoint result
I still think the Pythia training trajectory is useful, particularly because the random/untrained checkpoint provides a natural high-dimensional null and the accessibility changes non-monotonically with training and depth.
I would only keep one mechanism caveat attached to the state/readout-swap result.
A small cross-check I ran did not get much recovery from a simple orthogonal Procrustes map, so “the mismatch is just one global rotation of the basis” does not look sufficient.
On the other hand, a non-orthogonal linear ridge map predicted held-out later-checkpoint states extremely well and recovered the 17° steering assay almost completely. That opens a broader three-way split:
orthogonal basis drift
vs
broader linear deformation
vs
more specific state/readout co-adaptation
I would not add more model families here. If this branch becomes important again, the cheap controls are enough:
centered ridge
paired vs shuffled ridge
low-rank ridge sweep
concept/template-level holdout
If the recovery survives those while shuffled controls collapse, that would make the co-adaptation interpretation substantially cleaner.
If I were choosing what to do next
I would not launch another broad sweep yet.
If the immediate goal is a practical exact-word primitive:
keep MODE=emit as the canonical path
-> think in effective angular push + success stop-rule
-> then handle multi-token/tokenizer cases
If the goal is to explain why MODE=emit works:
current readout intervention
-> token transition
-> real token enters context
-> ordinary next state
is already a fairly testable component boundary
If the goal is semantic/topic steering:
fix S3 sample-ID bookkeeping first
-> add frequency/form/register controls
-> only then lean harder on the ring/topology interpretation
If the goal is training-mechanism evidence:
stay on Pythia
-> finish the centered/shuffled/low-rank alignment controls
So, at least for the generation branch, I think the latest result is in a much better place than the original “persistent settling” picture.
The version I find easiest to reason about now is:
choose a token-row direction
-> apply enough effective angular push to alter the current decision
-> stop as soon as the lexical goal succeeds
-> let the real emitted token, rather than the steering vector, carry the effect forward
That gives the geometry a fairly precise job without asking it to explain all of the downstream autoregressive dynamics.