Model Architecture

Variable	Value	Effect
TINYLOOP_USE_ROPE	1	Enable rotary position embeddings across all forward paths. Required for Llama-family models.
TINYLOOP_ROPE_THETA	float	RoPE base frequency (default 10000.0). Llama-3 uses 500000.0.
TINYLOOP_N_KV_HEADS	int	GQA: number of KV heads (must divide n_heads). Shrinks attn_qkv weight and KV cache.
TINYLOOP_POST_NORM	1	Use post-norm (LayerNorm after attention/MLP) instead of pre-norm.
TINYLOOP_USE_GELU	1	Use GeLU activation instead of SwiGLU. For GPT-2/BERT-style models.