AI Brief, 11 September 2026: a quarter of the cache, and two vendors marking their own harness

DeepSeek put the weights for DeepSeek-V4.1-Flash on Hugging Face at 03:11 UTC on 10 September, under an MIT licence, with the model card and a technical report following through 08:18. That is about eighty minutes before yesterday's edition closed, so this account is a day late to it. It is the most consequential release of the week and it deserves the space.

The headline is an engineering one. V4.1-Flash is a 552-billion-parameter multimodal mixture-of-experts model with a one-million-token context that activates 8B parameters per token during prefill and 16B during decode, and it holds its global key-value cache in 890 bytes per token — roughly a quarter of what DeepSeek-V4-Flash needed. Almost everything else in the release is in service of that number.

Artificial Analysis published independent measurements of the same model on 10 September, which is unusually fast and makes this a rare case where a lab's own table can be checked on the day. The independent numbers split cleanly in two. They support the efficiency claim: on a 100k-token prompt V4.1-Flash sustains 217 tokens per second at 2.72 seconds to first token, against 143 tokens per second at 4.37 seconds for the model it replaces. They do not support the framing. DeepSeek's card shows the model beating Claude Opus-5.0 on two agentic benchmarks; Artificial Analysis puts its Intelligence Index at 39.55 against Opus 5 at 50.70, GPT-6 Astra at 52.81 and Claude Fable 5.1 at 53.37. And on AA's hallucination measurement it scores 0.9646, the worst of the six models compared and worse than both of its own predecessors. The model card reports no hallucination metric at all.

The day's other coding launch has the same shape. Cognition released SWE-2 inside Devin at 17:00 UTC, with a benchmark table comparing it to six rivals — and the per-model harness assignments that produced those numbers are not in the table, the prose or the chart legends. They are in the JSON files the charts load, and they show SWE-2 running in Cognition's own harness while every competitor runs a minimal community one.

• DeepSeek-V4.1-Flash, MIT, weights released 10 September: 552B backbone, 8B/16B activated, 1M context, multimodal, 890 bytes per token of KV cache.
• Artificial Analysis measures it at 39.55 on its Intelligence Index, 13.3 points below GPT-6 Astra, at $0.30/$1.20 per million tokens against Astra's $10/$50.
• It leads all six models on AA's AutomationBench at 68.89%, and has the worst hallucination rate of the six at 0.9646.
• DeepSeek says deepseek-v4-pro requests will be routed to V4.1-Flash from 14 September.
• Cognition's SWE-2 ships in Devin only — no API, no weights — and sits fifth on Cognition's own public leaderboard.
• OpenAI released an Agents API in public beta and made GPT-Live-1 generally available, both on 10 September.

DeepSeek-V4.1-Flash: the cache claim holds, the frontier claim does not #

The architecture is the story. V4.1-Flash uses what DeepSeek calls a Causal Encoder-Decoder layout: 40 transformer layers split into a 20-layer causal encoder and a 20-layer decoder, where the decoder's global KV cache is projected from the final encoder hidden states rather than derived layer by layer in the decoder itself. That is what lets the decoder activate 16B parameters while the prefill pass activates only 8B.

Most of the 890 bytes comes from the number format. DeepSeek stores the main KV cache in FP4 — specifically E2M1, four bits per channel — with a single E4M3 scaling factor, eight bits, shared across every 16 channels. So the cost of one cached channel is

$$b_{\mathrm{ch}}=\frac{4}{8}+\frac{8}{8\times 16}=0.5+0.0625=0.5625\text{bytes}$$

where $b_{\mathrm{ch}}$ is bytes per cached channel, the first term is the four-bit value itself and the second is that token's share of the shared eight-bit scale. Against bfloat16's two bytes per channel that is a 3.56-fold saving from the format alone; sharing KV across layers and reusing sparse-attention indices gets the rest of the way to the claimed fourfold reduction.

What that buys is memory. At the full one-million-token context, 890 bytes per token is $890\times \mathrm{1,048,576}\approx 933$ MB of KV cache for a single sequence. At the roughly 3,560 bytes per token the card implies for V4-Flash, the same sequence costs about 3.73 GB. On one 80 GB accelerator that is the difference between holding roughly 85 million-token sequences and roughly 21 — which, for the input-heavy agentic workloads DeepSeek is aiming at, is the entire argument.

The independent numbers say the mechanism works and the marketing overshoots.

Three things in Artificial Analysis's measurements are worth more than the aggregate. First, V4.1-Flash genuinely leads the field on one of the ten evaluations: AutomationBench at 68.89%, ahead of GPT-6 Astra's 68.49% and Opus 5's 56.57%. Second, on long-context reasoning it scores 0.84, above Opus 5 and Astra and second only to Fable 5.1. The cache work is not cosmetic. Third, and pulling hard the other way, its AA-Omniscience hallucination rate is 0.9646 against Astra's 0.5134 — and on CritPt, a physics reasoning benchmark, it scores 14.29%, a regression against both V4-Flash-0731 (16.57%) and V4-Pro (18.00%). This is a model that has been optimised for cheap, fast, long-horizon execution and has lost ground on knowing things and on declining to answer.

Two caveats on any cross-vendor comparison from that page. The time-to-first-token figures are not like-for-like: DeepSeek's API streams before reasoning completes, so its 1.06 seconds is a stream-start latency, while Opus 5's 107.85 seconds and Astra's 342.94 seconds are whole-reasoning-pass latencies. And AA prices DeepSeek at peak rates; DeepSeek's own pricing page halves them off-peak, so the $0.2652 per index task can be $0.13.

That page also carries the release's most concrete commitment: from 12:00 Beijing time on 14 September, requests to deepseek-v4-pro will be routed to V4.1-Flash and billed at the V4.1-Flash price, until a V4.1 Pro exists. DeepSeek's stated reason is that the new model "has comprehensively surpassed V4 Pro". The independent index agrees on direction, 39.55 against 36.28, but not on the adverb: V4-Pro still leads on CritPt and on Humanity's Last Exam.

One practical note for anyone planning to run it. vLLM's recipe page lists day-one support, but there is no released wheel that carries the architecture — the newest vLLM release is 0.29.0, from 9 September, and the recipe names 0.30.0. Support currently ships only as a dated pre-release container:

# The only route to serving V4.1-Flash today: a pre-release image, not a wheel.
docker run --gpus all vllm/vllm-openai:deepseekv41-flash-0909

That page also states the model has "522B total parameters", which contradicts both DeepSeek's 552B backbone figure and vLLM's own component table 60 lines below it, which sums to 762.8B — a figure that matches the 763.2B tensor count published on the model repo. The error is vLLM's, not DeepSeek's.

Two coding launches, two harness maps #

The most useful thing in DeepSeek's model card is not its comparison table but the sweep underneath it. The card reports V4.1-Flash beating Opus-5.0 on Terminal-Bench 2.1 (90.6 against 89.1) and DeepSWE v1.1 (74.2 against 74.0). It then publishes a second table giving its own scores under eight different agent scaffolds, and both headline figures are exactly the maximum of that sweep. On DeepSWE the eight scaffolds span 65.5 to 74.2, so the spread across harnesses is roughly forty times the winning margin, and seven of the eight fall below Opus. On Terminal-Bench 2.1, six of eight do.

That is not by itself cherry-picking, and it would be unfair to call it that. DeepSeek states its harness rule up front, applies it across the whole benchmark family, and keeps it where it hurts: the same configuration produces Terminal-Bench 3.0 at 30.0 against Opus's 43.3, and 4.0 at 31.2 against 51.8, two large losses it could have suppressed. The real weakness is narrower and worse: the sweep covers DeepSeek's model only. There is no Opus row in it, and the card never says which harness produced the 89.1 and the 74.0, or who ran them. A within-model spread is being set beside a competitor figure of unstated provenance. By the card's own stated rule elsewhere that scores within 0.3 points are equivalent, the DeepSWE result is a tie, not a win.

Cognition's SWE-2 post has the same problem without the disclosure. The model is post-trained from Kimi K3 and ships inside Devin Desktop and CLI — no API, no weights, no model card, no context length or token pricing published. Its benchmark table carries no harness label and no per-cell attribution. The assignments exist, but only as a harness field inside the JSON the charts fetch, where the page uses them to pick a marker shape and never prints them. On DeepSWE 1.1, SWE-2 runs under chisel, Cognition's own harness, while GPT-6 Astra, GPT-5.6 Sol, Grok 4.6, Fable 5 and Kimi K3 all run under mini-swe-agent, the deliberately minimal community scaffold. On FrontierCode, Kimi K3 — the base model whose delta is the entire claim for the RL work — runs under mini-swe-agent while SWE-2 runs under devin, which contradicts Cognition's own stated rule that open-weight models get the Devin CLI. The advertised five-to-six point gain over the base model therefore mixes the training gain with a scaffold change.

Cognition's cost arithmetic does check out on its own numbers: SWE-2 at maximum effort scores 50.00% on FrontierCode at $1.18 per task against Fable 5.1 medium's 50.91% at $3.28, which is the claimed 64% cheaper. But Cognition's own public leaderboard, updated the same day, ranks SWE-2 fifth, behind Claude Fable 5 at 53.5% and Claude Opus 5 at 53.4% — two models that appear nowhere in the blog post's table, and both above the GPT-6 Astra the post presents as the frontier. No third party has measured SWE-2; Artificial Analysis has never covered Cognition's models, which is consistent with them never having been exposed through an API.

OpenAI ships an agent runtime #

OpenAI published two developer launches on 10 September. Its own changelog dates both to the day; neither post carries a published time.

The Agents API is in public beta, and it is a managed wrapper around the Codex harness: OpenAI runs the agent loop, the context compaction and the recovery, and the caller supplies tools and chooses whether the sandbox is OpenAI-hosted or self-hosted. It is four endpoints under /v1/agents/sessions, gated behind an OpenAI-Beta: agents=v1 header.

curl https://api.openai.com/v1/agents/sessions \
  -H "OpenAI-Beta: agents=v1" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "agent": { "model": "gpt-6-astra", "instructions": "Run the code and report real output." },
    "environment": { "type": "openai_hosted" },
    "input": "Create tree.py, run it, show me the output.",
    "stream": true
  }'

It deprecates nothing. The documentation presents it as a fourth runtime alongside the Agents SDK, the Responses API and ChatKit, with no migration deadline attached; the Assistants API it is often said to replace was already removed on 26 August, two weeks before this launch, with the Responses and Conversations APIs named as its successor. There is no separate fee — model usage bills at normal rates and OpenAI-hosted sandboxes at container rates, from $0.03 per 20-minute session at 1 GB. It is US-only and not eligible for zero data retention, and choosing a self-hosted sandbox does not change that.

GPT-Live-1 is generally available: a full-duplex speech model billed at $0.05 per minute rather than per token, with a July 2025 knowledge cutoff, which delegates reasoning and tool use to a separately chosen backend model. It is a new parallel surface rather than a declared successor — the existing gpt-realtime deprecation, announced in July, still points at gpt-realtime-2.1 on 20 January 2027, not at this. The launch benchmark figures circulating for it are OpenAI's own and could not be traced to a page; notably, a claimed 97.3% on an Artificial Analysis conversational benchmark has no corresponding AA model page.

Also notable #

• Two quantization papers landed on arXiv on 10 September, both CC BY. Why Does Post-Training Quantization Work? argues the answer is error cancellation: a pretrained block's update tends to point against the accumulated quantization error, negatively in 82.5% of blocks, an effect absent at random initialisation and that nobody trained for. Its most under-discussed number is that 8.3–12.7% of greedy next-token choices flip under NVFP4 even while benchmark accuracy moves 0.43 points. It is teacher-forced single-token analysis throughout, by the authors' own first-listed limitation, and it explains the technique without improving it.
• Mixtures-of-Experts Overfit More to Repeated Data, also 10 September, reports that sparse models degrade faster than dense ones under epoch repetition. Its downstream accuracy barely moves, the damage is in cross-entropy, and it has no limitations section; treat the mechanism as correlational, which the authors do.
• Artificial Analysis added independent numbers for Ling-3.0-flash-VL and a model listed as Agnes 3.0 Flash on 10 September. Its changelog dates are day-only, so neither can be placed against a mid-day boundary.
• GitHub removed MAI-Code-1-Flash from every Copilot surface on 10 September, same-day, per its changelog — a model retirement inside a shipping coding agent rather than a deprecation notice.
• Mistral announced a Cloudera partnership on 10 September. No model, no artifact.

What to watch #

• Whether anyone runs Terminal-Bench 2.1 on V4.1-Flash independently. It is the model card's strongest agentic claim and Artificial Analysis retired that benchmark from its index on 7 September, so the headline number currently has no outside check at all.
• Whether DeepSeek's 14 September V4-Pro routing holds. A lab retiring its largest model in favour of one a third the size is a real claim about where capability now comes from, and the independent index supports the direction but not the word "comprehensively".
• Whether Cognition publishes its harness assignments in the post rather than the data files. They are already public; the omission is presentational, and it would cost one table column.
• Artificial Analysis has still stated no comparability position across index versions, asked here for a seventh consecutive issue, and its 10 September changelog carried no article at all. Independent numbers for Tencent's Hy4 preview remain absent for a ninth.