AI Brief, 21 October 2026: the serving stack DeepSeek opened, and the two pieces it kept

DeepSeek published DeepInfer on 20 October: not a reference implementation but, it says, the inference engine running its commercial V4.1-Flash service, under Apache 2.0. It took 2,184 points on Hacker News and 14,399 GitHub stars in about a day, the loudest release of the window by a wide margin. The attention is mostly not about the engine. Alongside the code DeepSeek shipped a 29 KB production deployment write-up disclosing cluster-level figures labs almost never print, including 155 trillion tokens per day from one cluster and revenue of US$38 per GPU-hour at valley pricing. The thread spent the day arguing about the second number rather than reading the first.

Three qualifiers matter before anyone plans around this. DeepInfer is not a vLLM alternative: its own acknowledgments say it "was originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM". It serves one model family, on NVIDIA SM100 or Huawei Ascend 950 only, with no H100 and no consumer-GPU path. And it is offline inference only — the README lists a KV cache system, prefill/decode disaggregation and cross-replica load balancing as required for a complete service and absent here, because that code is "tightly coupled to our internal infrastructure". What was opened is the part that computes; the two pieces that make it a service were kept.

The same day pulled in the opposite direction. The Federal Register published DHS Exclusion and Removal Order #2026-01, barring DeepSeek products from federal civilian executive-branch systems. The order is not new — its signature page reads 24 July 2026 — but this is the first time it has been publicly citable, and its text is broader than the summaries suggest: agencies must remove "third-party products and services developed using DeepSeek implementations", language carrying no self-hosting carve-out. Separately, Mistral released Voxtral 3 as API-only, closing a speech line that had been Apache 2.0 for three generations.

• DeepInfer, Apache 2.0, landed 20 October: one model, two accelerator families, offline only, no tagged release. Every performance figure is DeepSeek's own and none has been reproduced outside it.
• Its headline prefill and decode Pareto curves ship only as PNG images, with no machine-readable numbers committed — the fifth launch flagged here for that since the 13th.
• DHS Order #2026-01 published on 20 October. Its 60-day exception window runs from the order's own issuance date of 24 July, closing it around 22 September — roughly four weeks before anyone outside government could read it.
• Voxtral 3 ships five speech models and a voice-agent SDK as API-only: no weights, no licence, no Hugging Face repository. Its predecessor has 2.2 million downloads under Apache 2.0. On Mistral's own realtime chart Gemini 3.5 beats every Voxtral realtime configuration, and the independent error-rate index Mistral cites places Voxtral Small third rather than first.
• No frontier lab published a new model repository in the window: a sweep of 23 organisations found zero and OpenRouter added nothing. The day's news came from blogs, not model hubs.

DeepSeek opened its engine, and the economics argument drowned out the engineering #

The root commit is dated 2026-10-20T04:09:42Z, the deployment write-up was committed at 09:06:57Z and revised at 11:03:13Z, and the Hacker News submission landed at 09:08:54Z. There are no tagged releases and no wheel: ./build.sh from source is the only route, the signature of a code drop rather than a supported product.

The substance is two mechanisms. The first, DSpark, is a scheduling change to speculative decoding. The draft model proposes five tokens per step; a Confidence Head predicts the acceptance probability of each; and the scheduler verifies only the prefix subset that maximises expected throughput against a profiled hardware curve, rather than spending verification compute on the low-probability tail. Because step N's confidence is unavailable when step N+1 is scheduled, the CPU fixes only a total budget and per-request lengths are recomputed on-GPU by a TileLang kernel.

Define verification efficiency as accepted tokens per verified token,

$$\eta =\frac{L_a}{L_v}$$

where $L_v$ is the draft tokens actually verified per step and $L_a$ the number accepted. At DeepSeek's stated production operating point of 96 concurrent sequences per rank, fixed five-token verification gives $L_v=5.00$ and $L_a=3.30$ , so $\eta =0.660$ . Confidence scheduling gives $L_v=3.24$ and $L_a=3.15$ , so $\eta =0.972$ . The trade is explicit: 0.15 fewer accepted tokens per step, a 4.5% loss, to avoid 1.76 wasted verifications, a 35% saving.

Read the top and bottom bars together. At 16 sequences per rank the scheduler is worse than the fixed draft, by 1.5%, on DeepSeek's own numbers: the technique buys nothing until the batch is large enough that wasted verification competes for capacity. An honest thing for a vendor table to show, and a reason to distrust the headline figure alone.

The second mechanism is the larger saving. Prefill replay normally costs more as context grows; DeepInfer bounds it. Replaying from position $s$ , the query at position $i$ attends only to keys in $[max(s,i-127),i]$ , so recomputation becomes constant in context length and layer count. Each prefill recomputes only the last 128 tokens and uses the result for decode alone, never entering it into the prefix cache, which DeepSeek says nearly halves total prefill compute. The cost is a cap: the maximum prefix-cache hit for an input of length $L$ becomes $L-134$ , the 128-token window plus 6 tokens of speculative backtracking.

Neither path is on by default, which matters if the Pareto curves are read as out-of-the-box numbers. The last flag below is the quiet one: batch-invariant inference means a request's logits do not depend on what else shares its batch, the defect class the vLLM and SGLang releases covered here on the 18th were patching when they fixed wrong tokens returned without an error.

export DEEPINFER_ENABLE_MTP_VARLEN=1         # confidence-scheduled verification
export DEEPINFER_USE_NEXTN_DECODE=5          # maximum draft length per step
export DEEPINFER_YOCO_PREFILL_KEEP_TOKENS=140  # 128-token window + 6 backtracking
export DEEPINFER_BATCH_INVARIANT=1           # batch-independent logits, NVIDIA only

The evidence is weaker than the attention implies. Every figure is self-reported and nobody outside DeepSeek has run it — unsurprising given the hardware floor, but the 2,184 points measure how provocative the revenue claim was, not whether the framework works. The throughput columns are labelled by DeepSeek as estimated, and the decode sweep takes draft acceptance as a command-line input to the plotter. The model-quality figures shipped alongside — AIME 2026 at 99.9%, GPQA Diamond at 93.4%, HLE-Full at 42.6% — are scored by default through an LLM judge configured as DeepSeek's own model on its own API; DeepSeek calls them "indicative rather than exact". And the $38 per GPU-hour is revenue at valley pricing, not margin, which much of the thread treated as profit. The write-up is still the most specific public account of a frontier serving cluster in some time: 100 PiB of SSD at a peak aggregate read of 10 TiB/s, and cache lifetimes of three days for agents against seven for chat — a choice that gives up 0.5 to 1% of chatbot cache hits.

The order that was three months old before anyone could read it #

DHS Exclusion and Removal Order #2026-01 reached the Federal Register on 20 October as document 2026-21447 (docket DHS-2026-0628), having gone on public inspection at 08:45 ET on the 19th, which is why the press cycle ran a day earlier. Issued under the Federal Acquisition Supply Chain Security Act, it names Hangzhou DeepSeek Artificial Intelligence Basic Technology Research Co., Ltd. and is signed by Homeland Security Secretary Markwayne Mullin. Its four signed pages are reproduced only as page scans with no text layer, which is likely why two specifics have not travelled.

The first is scope. The operative language never uses the words weights, open weights, fine-tune, derivative or API; coverage runs entirely through "products", "services" and "implementations". Required Action 2 directs agencies to remove not only DeepSeek's own products but "third-party products and services developed using DeepSeek implementations", with no carve-out for self-hosting, air-gapping or network isolation. On that language an open-weight checkpoint downloaded and run on agency hardware reads as covered, and so does a vendor tool built on a DeepSeek distillation. Secondary accounts blurred this, and it is the whole practical question for an agency with such a model already in production.

The second is the timetable. Agencies may request an exception in writing within 60 days of the order's issuance, reviewed by CISA. The signature page reads "Original Release Date: July 24, 2026". Sixty days from 24 July is roughly 22 September — about four weeks before formal publication. The benign reading is that agencies received the order directly in July and the window ran from then, with October being public notice only; the phrase "Original Release Date" also allows for later revision. Either way, the exception process described in the document the public can now read had, on its face, already closed.

Two details sit oddly. The footnotes twice define an affiliate as an affiliate of "DeepSeek AG", a corporate form matching nothing else in the document, which looks like template carry-over from the first FASCSA order. And the footnoted evidence is thin for a binding instrument: a company profile from January 2025 and a vendor explainer from August 2025. This brief covered China's own draft export controls on open weights on 17 October; both measures now turn on definitional language their drafters did not supply.

Mistral's speech line goes closed #

Mistral published Voxtral 3 at 2026-10-20T12:00:22Z: three speech-to-text models, two text-to-speech models, and a voice-agent SDK fusing transcription, reasoning, tool calling and synthesis into one streaming pipeline with sub-second turns. It took 569 points on Hacker News, below the 1,012 the Voxtral Transcribe 2 launch took in February for a smaller release.

The release state is the story. Voxtral 3 is API-only: no weights, no model licence, no Hugging Face repository, and self-deployment only behind a sales conversation. The mistralai organisation's 81 public repositories contain no Voxtral 3 artifact and the announcement carries no Hugging Face link at all. That breaks a line: Voxtral-Mini-3B-2507, Voxtral-Small-24B-2507 and Voxtral-Mini-4B-Realtime-2602 were all Apache 2.0, and the last has 2,210,103 downloads. The community artifacts that exist today are quantisations of models Mistral has now stopped shipping.

The capability claims need the same care. On Mistral's own FLEURS 45 chart, Voxtral Small Transcribe 3 ties Microsoft AI Transcribe 2 exactly, at 4.1 forced and 4.0 inferred, under prose calling it state of the art in multilingual transcription. On the realtime chart Gemini 3.5 Realtime Transcribe scores 4.2% error against the best Voxtral realtime configuration's 5.7% — and that 5.7% is at 2,400 ms of latency, not the advertised sub-500 ms; at 480 ms it is 6.4%. The chart's top bar is asterisked as an offline model. Artificial Analysis, which Mistral cites, independently measures both batch models at 2.1% and 2.2%, placing Voxtral Small third behind Alibaba's Fun-Realtime-ASR-preview at 1.7% and Microsoft's at 2.0%. The realtime model is absent from that index, so every realtime figure stays vendor-only. To Mistral's credit on a point raised here repeatedly against others, these charts are inline SVG with real text nodes rather than pictures; the gap is elsewhere, in a maximum audio duration and sample rate published nowhere at all.

Also notable #

• The decision-model question asked here daily since 10 October has its first public numbers — from a competitor. DecisionPivot, posted 20 October, reports retrospective coverage at a 5% empirical error rate across 40 models, the closest public analogue to a fail-open rate yet: 75.1 for vLLM Semantic Router's d3, 48.4 for Decision 2.0 Vega, 19.3 for hosted Jev. Its method is the contribution — each item is paired with a minimally perturbed sibling that flips the answer, and two 27B models within 2.6 points on aggregate accuracy diverge by 25.9 points on those pairs. But the paper is StartLux's, its own models take the top two slots, and the dataset is granted only on request.
• MemT-1.0's ablation, asked for here yesterday, undercuts the name. The "stable token-level memory update" the release led with is worth 0.54 points, and plain Gated DeltaNet costs 0.25; the result is the granularity, since token-level to turn-level costs 5.99. The 66× turns out to be per-trajectory saved state in bytes, not 66× less memory for a deployment.
• Anthropic had all 211 designs from a 64-hour autonomous protein-design campaign physically made and binding-tested, publishing the data CC BY 4.0 on 20 October: two contract labs confirmed 17 binders, the strongest at 3.1 nM. The framing invites an antivenom reading the work refuses: binding was tested only against recombinant toxins, with no neutralisation assay and no animal protection, existing antibodies already report sub-nanomolar affinity and mouse protection, and one molecule covering several toxins failed at 0 of 151.
• The silent quantised-MoE corruption this brief asked vLLM to scope on the 19th was already scoped — in PR #59042, merged 12 October, shipped in v0.32.0 on the 18th, before the question was put. The root cause is narrow: CutlassExpertsFp4 and CutlassExpertsMxfp4 accepted an expert_map and never used it, so with expert parallelism on, topk_ids stayed in global expert space while the weights were local and tokens went to the wrong experts. On a published, rerunnable command 5-shot GSM8K went from 0% to 0.9507, and the remapping fix costs about 1.6 microseconds. Kernel, format, parallel configuration and release are now pinned; which GPUs is still named nowhere.
• Ornith-1.5-122B (ornith-ai, MIT) got its quantisation ladder and first benchmark table on 20 October, four days after the weights: unsloth's dynamic quants bottom out at 40.36 GB, putting a 122B-total mixture-of-experts on one 48 GB card. But the card claims it serves on a single 80 GB GPU against 250.2 GB of bf16 weights, never mentions that config.json declares a Qwen3.5-MoE architecture with a vision tower, and its 76.9 on SWE-bench Pro against Qwen3.8-Flash-Next's 43.9 holds only under re-scoring rules the authors wrote themselves.

What to watch #

• Whether anyone outside DeepSeek runs DeepInfer and publishes a number. The eval harness and its sampling configuration are now public, so the blocker on reproducing a DeepSeek figure has moved from methodology to GPU access. The one to want is an end-to-end DSpark measurement.
• Whether any agency sought a DeepSeek exception, and whether CISA granted one. The window appears to have closed on 22 September, before publication, so the count is the kind of thing the Secretary's one-year review could surface.
• Whether Mistral ships Voxtral 3 weights, or confirms it will not. Three generations of Apache 2.0 set an expectation, the announcement does not address the change, and the predecessor's 2.2 million downloads size what is now closed.
• MiniMax M3.1 Flash weights have still not appeared, flagged here on the 20th against a press-reported 23 October date: no M3.1 repository exists in the MiniMaxAI organisation, whose newest upload is from 7 August. And no decision-model vendor has published its own fail-open rate, asked daily since 10 October, now twelve: a rival's coverage figures are not a disclosure.