Per-Iteration Safety Check

The Problem

Language models can produce degenerate hidden states — exploding norms, NaN propagation, adversarial-triggered instability. In a standard transformer, you can only check after the full forward pass completes (all 24 layers). By the time you detect the problem, you've already burned the full compute.

In a looped transformer, the same block runs L times. If the hidden state explodes at iteration 3, iterations 4–8 are wasted compute that also produces garbage.

The Solution

Check the residual L2 norm after every iteration. Abort the moment it exceeds a threshold. This gives L checkpoints per token instead of 1.

Cost Analysis

Operation	FLOPs (D=2048)	Relative cost
L2 norm computation	~2K	1×
Attention (T=128)	~4M	2000×
Full block (QKV+attn+MLP)	~34M	17,000×

The check reads buf.main (FP32, already in L2 cache from the block that just wrote it) and computes a single scalar. It's essentially free.

Implementation

The kernel f32_l2_norm computes √(Σ x²) over the D-dimensional residual vector. One thread-block reduction, ~2 μs on H100.

Measured Results

On H100 / 2B INT4 model:

Threshold	Behavior	Iter triggered	Norm value
10.0	Aborts early	iter 0	30.7
50.0	Catches divergence	iter 2	62.4
1000.0	Normal generation	—	max 45.2
100000.0	Never triggers	—	—

Usage

CLI

tinyloop model.tinyloop generate \
    --prompt "Hello" --loops 8 \
    --safety-check --safety-threshold 50.0

Python

tokens = model.generate(
    prompt, loops=8,
    safety_check=True,
    safety_norm_threshold=50.0
)

Server (native endpoint)

{
  "prompt": "Hello world",
  "loops": 8,
  "safety_check": true,
  "safety_norm_threshold": 50.0
}

Why This Is Architecture-Exclusive

In a standard transformer, layer 1's hidden state lives in a different representational space than layer 24's. A norm threshold that works for layer 1 would be meaningless for layer 24. In a looped transformer, every iteration operates in the same space with the same weights, so a single threshold applies uniformly.

Integration with Other Safety Features

Feature	Scope	Cost	When it fires
Per-iteration norm check	Per token, per iteration	0.05% of block	Hidden state divergence
β monitoring	Per generation call	~0%	Draft/verify agreement drops
Token-stable early exit	Per token	1 logit peek	Output stabilizes early
Constrained decoding	Per token	Grammar NFA step	Invalid output

The norm check is the earliest and cheapest safety gate — it fires before the model even produces a logit.