Rotary Positional Embedding (RoPE)
Seq position:
0
Base θ:
1000
Animate
Stop
RoPE: multiply query/key by rotation matrix R(mθ) — dot product depends only on relative position m−n