long-context
Posts tagged “long-context”.
-
AI Brief, 4 October 2026: a million tokens, by taking the positions out
Aleph Alpha released Kolibri on 3 October under Apache 2.0: 78B total parameters, 3.46B active, German and English only, and 40 of its 50 attention layers see just 513 tokens. The ten that see everything carry no positional encoding at all, which is how it extends to a million tokens with no position scaling. Its own RULER table shows the price.