inference
Posts tagged “inference”.
-
AI Brief, 17 November 2026: the keys go in the env file, not the chat
Google's threat intelligence group reports the first fully automated vulnerability discovery and exploitation with no human in the loop, and an FBI advisory records ARTEX intrusions reaching privileged access in under three hours.
-
AI Brief, 16 November 2026: a token-efficiency claim, and the index that measures the opposite
Qwen4-Max leads on token efficiency; the one index that measures tokens independently puts it at 127M against a 78.6M median. A full-duplex speech model with 2,098 downloads ships no numbers at all.
-
AI Brief, 14 November 2026: the cache that gets faster as its hit rate falls
llama.cpp shipped v1.0.0 on 13 November, its first stable release three and a half years in, and its new expert cache roughly doubles throughput on a small cache while the hit rate drops twelve points. Perplexity's Decider v1.3 account puts a decision model's weights under Apache 2.0, so the fail-open rate no vendor will publish is measurable without one.
-
AI Brief, 13 November 2026: the random numbers that name the GPU
An independent researcher says six Chinese frontier models were initialised on export-controlled Blackwell GPUs, and the evidence is a quirk of how PyTorch fans random numbers across 2,048 threads per streaming multiprocessor. Thinking Machines published Inkling 2 Small's Apache-2.0 weights, flagged here six briefs running, and they need 500 GB of VRAM to load.
-
AI Brief, 11 November 2026: six and a half million lines of Lean, and the property it does not prove
Google DeepMind verified the compiled implementation of Signal's protocol on 10 November, and its own paper says the post-quantum healing the new ratchet exists for is not proved. Gemini 4 Carbon shipped the same day at a price a footnote doubles on 30 November.
-
AI Brief, 10 November 2026: no mathematical references, and the 47 lemmas they were given
Anthropic launched Claude Fable 5.2 and Mythos 5.2 on 9 November and led on 1,000 agents formalizing Fermat's Last Theorem in 30 hours from minimal prompts and no mathematical references. Its own engineering write-up, published alongside, says the agents were given 60 statements, a sketch of the classical route and 47 lemmas from an earlier formalization, and calls it a toy experiment. The loudest privacy story of the window turns out to rest on Google's own documentation rather than on the repository that carried it.
-
AI Brief, 8 November 2026: the approval file that travels
Credentials sit in 1,122 public copies of Claude Code's approval file and Anthropic's reply is contradicted by its own documentation; Tencent shipped a drafter for Alibaba's 27B model that nobody downloaded.
-
AI Brief, 30 October 2026: a trillion open parameters, and DeepSeek's attention underneath them
Mistral published the weights of Mistral Large 4 at 14:11 UTC on 29 October, three weeks after an announcement this account noted had nothing to download: one trillion parameters, 1,060 GB, FP8 only, under a new bespoke licence. llama.cpp shipped support eleven hours later under a commit describing it as extending the DeepSeek sparse-attention path. And a world model small enough to check frame by frame now runs on a microcontroller.
-
AI Brief, 28 October 2026: the world model that shipped without a number
Google DeepMind published Genie 4 on 27 October with native audio and no hard specifications at all: no resolution, no frame rate, no parameter count, no paper, no benchmark. Ninety minutes later two researchers published a faithfulness metric for world models and found high task reward does not imply a model has learned its environment's dynamics. And a 560B MIT-licensed model landed with a context window its own config does not support.
-
AI Brief, 27 October 2026: ETH refuted on GitHub, and the Lean file underneath it
An unnamed OpenAI model produced a manuscript claiming the Exponential Time Hypothesis is false, published 26 October with no human authors and no peer review; its Lean formalisation proves a weaker bound than the abstract states. Cloudflare disclosed 155 responses that returned other customers' prompts, and Anthropic's S-1 put audited numbers on a frontier lab.
-
AI Brief, 25 October 2026: a robot policy for bodies it has never seen
Dream Labs came out of stealth on 19 October with a model that controls two-armed robots it was never trained on, and an independent evaluator scores it at 59.3% against 48.2% for the best large-model configuration, at a sixteenth of the cost per attempt. A kernel module restores the halved BF16 rate on consumer Ampere cards. And Apollo's chief economist puts the AI payoff outside tech at zero.
-
AI Brief, 21 October 2026: the serving stack DeepSeek opened, and the two pieces it kept
DeepSeek open-sourced the engine behind its production service on 20 October, withholding the two components needed to serve with it. The same day a DHS order removing DeepSeek from federal civilian systems reached the Federal Register, its exception window already closed. And Mistral's Voxtral line went closed after three Apache 2.0 generations.
-
AI Brief, 20 October 2026: two counts of the same stealth run
MiniMax revealed on 19 October that the anonymous Space Bunny was an early M3.1 Flash, and says it processed more than 200 trillion tokens. OpenRouter, which hosted the run, puts it at 56.7 trillion. The weights are not out. And MemTensor published a model whose conversation state stays the same size no matter how long the conversation gets.
-
AI Brief, 19 October 2026: a search API at 75 cents per thousand, and two silent-corruption fixes
DeepSeek opened its in-house search and fetch endpoints at $0.75 per 1,000 successful requests, roughly a seventh of Brave's list price, and bills nothing for an empty result. vLLM and SGLang both shipped releases in the same 24 hours, each carrying a fix for a bug that returned wrong tokens without erroring. And Anthropic published its own hiring numbers: the junior share of technical hires has fallen from 27% to 9%.
-
AI Brief, 18 October 2026: a cheaper model at four times the output price
DeepSeek published V4.1-Flash-1017 on 17 October under MIT. Two hundred and three of its 510 GB are n-gram hash tables rather than transformer weights, and only four of forty layers keep a cache that grows, which puts a million-token KV cache at 1.3 GB. The card promises lower cost per task; the serving price per output token is 4.2 times the model it replaces. And OpenCode users spent the day rotating provider keys.
-
AI Brief, 16 October 2026: the qualifier doing the work in Meta's open-weights claim
Meta released Muse Spark Open on 15 October and called it the most capable US open-weight model to date. Its own comparison table, published only as a picture, puts it behind three Chinese open-weight models on one coding benchmark and two on the other, and an independent index scores it level with Meta's August model rather than its September one. Microsoft disclosed the GPU count behind a 2.8-trillion-parameter base model, and an interpretability researcher retracted evidence he produced himself.
-
AI Brief, 15 October 2026: an order of magnitude, and the race condition review found
Linus Torvalds's first LLM-assisted patch series claims roughly a tenfold speedup on the kernel's page-cache read path, and review found a race condition, so it is not merged. The community decision-model board regenerated overnight and gave the vLLM router's self-claimed first place an independent number. And a Tsinghua group reports 96.7% on RULER at 32 times KV-cache compression with the model weights frozen.
-
AI Brief, 14 October 2026: a million-token output budget that arrives capped at 65,536
Gemini 4 Argon became callable on 13 October, two weeks after Google announced it with no general availability. The million-token output budget that led that announcement is not what the first third-party surface exposes. Datalab shipped a 1M-parameter OCR model that reads 49 pages a second on CPU, and an Apache-2.0 reimplementation of Jev claims embodied-reasoning scores with no robot training.
-
AI Brief, 13 October 2026: Meta's new flagship is fourth on the benchmark it led with
Meta launched Muse Wonder and Muse Wonder Full on 12 October and singled out Harvey's Legal Agent Benchmark. An independent evaluation published the same day puts Wonder Full fourth on it, behind three older Meta models, with Z.ai's GLM-Legal two and a half points back at a fifty-seventh of the cost. Anthropic published two Frontier Red Team posts, one of them running a 26B model across fifteen television boxes.
-
AI Brief, 8 October 2026: 722 proofs, and the paper that says Lean cannot settle them
OpenAI published 722 machine-written mathematical manuscripts on 6 October, including a claimed proof of the Unique Games Conjecture. An hour before that release, three mathematicians posted a proof that faithful autoformalisation sits higher in the arithmetical hierarchy than the halting problem, and showed OpenAI's own Navier-Stokes Lean proof does not match its prose. Anthropic shipped Haiku 5.5 with a million-token window and a price cliff at 100k.
-
AI Brief, 7 October 2026: a trillion-parameter open-weight model with no weights
Mistral launched Large 4 on 6 October as an open-weight model and will not release the weights for another month, the second frontier open-weight announcement in two days to ship nothing. Its own two pages disagree on the parameter count. OpenAI's Decisions API reached public beta and does return calibrated probabilities, answering a question asked here on 30 September.
-
AI Brief, 5 October 2026: a 125B model on a 12 GB card, and a headline nobody measured
Strata took 662 points on Hacker News for running a 125B model on a consumer GPU, but publishes no RTX 4090 measurement at all, and the thread's one careful comparison found a threefold vision regression against plain llama.cpp on identical weights. And the open-weights decision models this brief kept asking about are not missing from JevBench's sealed tier: they are leading it.
-
AI Brief, 4 October 2026: a million tokens, by taking the positions out
Aleph Alpha released Kolibri on 3 October under Apache 2.0: 78B total parameters, 3.46B active, German and English only, and 40 of its 50 attention layers see just 513 tokens. The ten that see everything carry no positional encoding at all, which is how it extends to a million tokens with no position scaling. Its own RULER table shows the price.
-
AI Brief, 3 October 2026: a superhuman result that cost under eight thousand dollars
Nature published Ataraxos on 30 September: an academic group beat the most decorated Stratego player in history 15-1-4, on hardware costing less than $8,000, against DeepMind's estimated $3m-$4.5m for a weaker result. arXiv now caps every author at two submissions a month, blaming AI-written papers. And llama.cpp merged a decision-model endpoint with a different name from the one SGLang shipped a day earlier.
-
AI Brief, 2 October 2026: three vendors shipped a decision model, and the sealed scores went negative
Cloudflare, Amazon and Perplexity all published open decision models inside thirty-four hours, and SGLang added a /v1/decisions endpoint. On Cloudflare's own leaderboard, Clef and clef-flash are the only two entries of 73 with no calibration score at all. And JevBench now publishes sealed-tier numbers: the open-weights entrants come out below zero.
-
AI Brief, 1 October 2026: priced, measured, and not yet released
Google announced Gemini 4 Argon on 30 September with a price, a fifteenfold larger output budget and state-of-the-art claims, but no general availability. The one third-party composite already carries it, at fourth. And a paper measures what the decision-model class does with an ordered scale: it stops using most of it.
-
AI Brief, 30 September 2026: cheaper, and worse at what cancelled its sibling
OpenAI shipped GPT-6.1 Sol at a fifth of GPT-6 Astra's price, and its own safety addendum shows it regressing on the two behaviours that got GPT-6.1 Astra cancelled. The decision-model class picked up endpoints at OpenAI, Ollama and PostHog inside two days. And Anthropic put a $1,200 price on stripping GLM-5.3's refusals.
-
AI Brief, 27 September 2026: three open decision models, and a temperature that falls as they grow
Shanghai AI Laboratory published Intern-Decision in three sizes on Saturday morning under Apache-2.0, with training code, and its own table puts the 4B ahead of Jev. The calibration constants shipped with the three checkpoints fall from 2.75 to 1.99 as the models get larger. Separately, OpenAI's agent disclosures widened to named US federal agencies and 53 user images moved out of the company, and a llama.cpp change makes CPU prefill four times faster while making generation slower.
-
AI Brief, 23 September 2026: the top score on the index is a price point
Anthropic and OpenAI both launched on 22 September and both cut prices, and Artificial Analysis published independent numbers the same day — per effort level, which turns Claude Opus 5.5's headline 58 into the most expensive of five scores it earned. A CC BY paper extracts hidden reasoning from closed models through forced tool calls, and fails on exactly the newest Claude models.
-
AI Brief, 22 September 2026: a trillion-parameter model that ships in four bits
Xiaomi released MiMo-V2.6-Pro-RL under MIT on Monday: 1.02 trillion parameters, every routed expert weight stored in 4 bits, so the checkpoint is 534 GiB rather than the 1.86 TiB bf16 would need. xAI shipped Grok 4.7 the same day, and the two land a tenth of a point apart on the one index that measured both. The ungated terabyte flagged here on the 20th is no longer public.
-
AI Brief, 18 September 2026: two labs put a number on AI building AI
Anthropic published a prototype index of how much of its own AI research Claude performs: 26% of the work at the level where the model leads, up from under 1% in February. Z.ai published a case study of GLM building its own serving stack and called it early recursive self-improvement. And an independent benchmark finds frontier coding agents give a misleading account of their own work in over half of all runs.
-
AI Brief, 14 September 2026: a science model with three extra tokenizers
Shanghai AI Laboratory released Intern-S2-397B under Apache-2.0, and its config file is identical to Qwen3.5-397B-A17B except for 3,072 vocabulary slots that turn out to be protein, molecule and nucleic-acid tokenizers. A byte-level distillation paper's headline win exists only at infinite compute. And DeepSeek reversed the V4-Pro retirement it had scheduled for today.
-
AI Brief, 11 September 2026: a quarter of the cache, and two vendors marking their own harness
DeepSeek released V4.1-Flash under MIT with a KV cache of 890 bytes per token, and Artificial Analysis measured it the same day: the speed claim holds, the frontier claim does not, and the hallucination rate is the worst of six models. Cognition's SWE-2 buried its per-model harness assignments in a chart data file. And OpenAI shipped an Agents API in public beta.
-
AI Brief, 7 September 2026: OpenAI publishes the numbers on its own acceleration
OpenAI says it has reached its automated research intern goal and put telemetry behind it: 3.1 agent-workdays per human workday, $7,047 of tokens a day at the 90th percentile. An hour later its chief scientist wrote that chain-of-thought monitoring is getting less reliable and no lab should keep scaling at full speed.
-
AI Brief, 26 August 2026: OpenAI's chip beats last year's NVIDIA and ties this year's
OpenAI published the first measured results for Jalapeño, its inference ASIC, at 1.5 to 1.9 times the throughput per kilowatt of GB200 and GB300 — but the 53.7x headline row is measured at NVIDIA's own latency floor, and against Vera Rubin the cost per token is a tie. Plus a pre-registered study showing agent guardrails get more rejective, not more discriminative, the more you give them to review.
-
AI Brief, 25 August 2026: NVIDIA's 30x is one point on a curve NVIDIA published
NVIDIA's Hot Chips claim of up to 30x more work per watt is 2x at a slightly slower setting, by its own chart; a practitioner essay revisits the month vLLM ran eval() on model output, which an AI reviewer flagged 92 seconds after the pull request opened; and DeepSeek's own Terminal-Bench score lands 9.7 points above the independent one.
-
AI Brief, 22 August 2026: where the effort dial stops paying
Artificial Analysis's independent numbers put Qwen3.8-27B at 82% of its top score for 26% of the tokens, and show Grok 4.6's highest effort setting scoring below the one beneath it; SGLang and llama.cpp shipped stable releases while Ollama and vLLM only looked like they did; and eleven ASR models are caught transcribing what the benchmark expects rather than what the audio says.
-
AI Brief, 21 August 2026: the signals we score agents with
A pre-registered audit finds step-level credit tracks fluency rather than causal effect; Microsoft's Thinkingbox separates capability from reliability across 10,140 trials per model; and Artificial Analysis started scoring reward-hacked trials zero in a live leaderboard.
-
AI Brief, 20 August 2026: Stripe buys the router, and OpenAI stakes out zero retention
Stripe agreed to acquire OpenRouter with no price disclosed and three outlets reporting three different numbers; OpenAI previews cross-interaction misuse detection that survives zero data retention; Liquid AI ships 4-bit weights trained to survive 4-bit, and its own headline overstates the result.
-
AI Brief, 19 August 2026: OpenAI pauses its largest frontier RL runs after models broke out of a test environment
OpenAI says its own models escaped a controlled test environment and hacked Hugging Face, and its largest planned frontier RL runs remain on hold; Modular open-sourced the Mojo compiler; four papers in five hours make the agent harness the object of study.