KV Cache Environment Variables

Stack these to select one of the storage modes. See KV Cache Modes for full memory math.

Variable	Value	Effect
TINYLOOP_KV_INT8	1	INT8 K/V with per-head scales. −49% KV footprint.
TINYLOOP_KV_H_MODE	1	Store-h cache: reconstruct K/V from stored hidden state. Bit-exact. 50% saving.
TINYLOOP_KV_INT8 + TINYLOOP_KV_H_MODE	1 + 1	h-as-INT8. ~66% saving.
TINYLOOP_KV_INT4 + TINYLOOP_KV_H_MODE	1 + 1	h-as-INT4. 78% saving. Recommended for long-context.
TINYLOOP_KV_H_FUSED	1	Fused INT4+unpack kernel. Currently performance-negative.
TINYLOOP_PAGED_KV	1	Paged KV cache. 16-token pages, pool-based. ~5.6% overhead. Foundation for continuous batching.

Composability

Goal	Stack
Long-context low memory	TINYLOOP_KV_H_MODE=1 TINYLOOP_KV_INT4=1 TINYLOOP_H_MODE_FP16_ATTN_QKV=1
Near-lossless compression	TINYLOOP_KV_H_MODE=1 TINYLOOP_KV_INT8=1 TINYLOOP_EXPERIMENTAL_FP16_BODY=1