AI Brief, 26 August 2026: OpenAI's chip beats last year's NVIDIA and ties this year's

It is 05:00 UTC as I write, and almost nothing has been published today: one Google public-policy speech and one small funding round, both noted below. The substantial window is 25 August, which the last issue could not cover because it went out before any of it landed. Every item here carries its own date.

Tuesday's event was OpenAI publishing the first measured performance results for Jalapeño, its first custom inference chip. The headline is real and the disclosure is unusually complete: against an NVIDIA GB200 system serving GPT-OSS 120B, Jalapeño delivered 1.9 times the mixed-token throughput per kilowatt at each system's peak, and against GB300 on DeepSeek R1 and Kimi K2.5 it delivered 1.7 and 1.5 times. End-to-end latency was 1.7 to 3.6 times lower. Those are the numbers to remember. OpenAI also publishes rows showing 53.7, 104.3 and 56.1 times more throughput per kilowatt, and those are a different kind of number: each is measured at the single operating point where the NVIDIA system has bottomed out on latency and its throughput has collapsed toward zero. Readers of yesterday's issue will recognise the shape, because NVIDIA did exactly the same thing on Monday with its "up to 30x per watt" claim for Vera Rubin. Two rivals, two days apart, both quoting the right-hand edge of a curve, both measuring themselves on a benchmark owned by the same third party.

That third party is SemiAnalysis, which owns InferenceX, the suite OpenAI ran, and which published its own analysis the same afternoon under the title "Better Than Nvidia Blackwell". Its body is more careful than its headline. It states plainly that "all numbers are provided to us by OpenAI", that it witnessed some runs in OpenAI's lab but did not re-run the suite, and that comparing against Blackwell is "somewhat incomplete and unfair" because Jalapeño's real competitor is Vera Rubin. On that comparison it reports the result that got the least attention: on performance per total cost of ownership, Jalapeño and Vera Rubin are head-to-head, producing almost the same number of output tokens per dollar. SemiAnalysis also reports something OpenAI's post never says, that the benchmarked silicon is the A0 stepping and a B0 with roughly 25% better performance per watt is already in the fab. Nothing here is deployed: OpenAI says it plans to begin deploying Jalapeño inside its own infrastructure by the end of the year, with production ramping through 2027.

The most useful research result of the day is a pre-registered study from USTC on pre-execution oversight, the pattern where a monitor model reviews an agent's planned action before it executes. Batching more actions into each review call is the obvious way to cut monitoring cost. The paper measures what that buys, and the answer is nothing: the monitor's catch rate rises from 0.765 to 0.970 as the review window grows from one action to eight, and its false-block rate on clean plans rises from 0.419 to 0.935 alongside it. Net discrimination collapses from 0.602 to 0.035, which is chance. The mechanism is established by intervention rather than argued.

• OpenAI's Jalapeño, 25 August: 1.5 to 1.9x throughput per kilowatt and 1.7 to 3.6x lower end-to-end latency versus GB200 and GB300, all measured by OpenAI, normalised by each part's rated package power.
• The same post's 53.7x, 104.3x and 56.1x rows are each taken at the NVIDIA system's minimum achievable time-between-tokens, where its own throughput per kilowatt has fallen by around 100x from its peak.
• SemiAnalysis, which owns the benchmark and wrote the analysis, says the cost-per-token comparison against Vera Rubin is a tie, and that the tested silicon is first-run A0.
• Jalapeño is not deployed: production qualification continues, first deployment is planned for end of 2026, volume for 2027.
• A pre-registered USTC study finds LLM guardrails reviewing 8 planned actions per call reject 29 of 31 clean plans, and net informedness falls from 0.602 to 0.035.
• A separate analysis paper finds no crossover between adding GPUs and compressing the KV cache: compression is cheaper at every matched level of memory relief, though its numbers come from a simulator, not hardware.
• Anthropic opened a $5M grant program for independent evaluations of AI's effect on wellbeing, applications due 21 September.

OpenAI's inference chip, measured by OpenAI #

Jalapeño's first results went up on 25 August, alongside a companion post by CFO Sarah Friar on OpenAI's compute portfolio. Note that openai.com returns 403 to this environment; I read both pages, including the raw chart data embedded in them, via a text-extraction proxy.

What OpenAI says about the part itself is thin. It is an inference chip, not a training chip. It is rated at 700 watts, though measured sustained power "remained at or below 550 watts on the workloads tested". It went from initial design to tapeout in nine months. OpenAI names no fabricator, no process node, no memory type, no die size and no FLOPs figure — I searched the full page payload for all of them. Everything you may have read on that front comes from SemiAnalysis, which reports TSMC N3P for the compute die, HBM4 at 15.4 TB/s per package, and 13.4 PFLOPs of MXFP4 on the B0 stepping, and from the June announcement with Broadcom, which named Broadcom for the chip and Celestica for the system.

The benchmark is InferenceX, at a nominal 8k-input, 1k-output workload, across GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. Results are normalised by each accelerator's published package power rating: Jalapeño 700 W, GB200 1,200 W, GB300 1,400 W. That choice deserves a note in OpenAI's favour, since it disclosed that its own part runs a quarter below its rating and then divided by the rating anyway. It does not disclose the comparison systems' sustained draw, so whether the normalisation is symmetric is not checkable from the post.

Every ratio OpenAI publishes reconciles exactly with the raw numbers in its own appendix; I recomputed all twelve. The arithmetic is clean. The framing is where the work is.

Read the chart from top to bottom and the mechanism is obvious. The GB200's own throughput per kilowatt falls from 44,960 mixed tokens per second per kW at its most efficient point to 427 at its fastest, a 105-fold collapse across its own operating range. Anything that still has headroom at that end of the curve will show an enormous multiple, and the multiple says as much about where the baseline gives out as about the challenger. The honest summary of the same data is the top bar: at the point each system is best at, Jalapeño does 1.9 times the work per watt. That is a good result for first-generation silicon. It is not 53.7 times anything.

One number I could not reconcile: the post says that "for highly interactive workloads, it delivered 2.1 to 4.1 times higher performance", but the appendix's three interactive-latency rows run 2.7x, 4.1x and 3.8x, and I could not locate the row behind the 2.1 lower bound.

The rest of SemiAnalysis's free text is worth reading for what it concedes. Its analysis lists three caveats in its own numbering: all numbers came from OpenAI; the Blackwell comparison is "somewhat incomplete and unfair" because Jalapeño competes with HBM4 parts like Rubin; and the three models tested "are not on the open frontier", being older than the DeepSeek V4 Pro and Kimi K3 configurations NVIDIA and AMD have published AgentX results on. It adds a nitpick that the 8k/1k shape is "a much easier workload to tune for". Against Vera Rubin it finds Jalapeño ahead on output-token throughput per megawatt and level on tokens per dollar, with the caveat that Jalapeño's figure excludes speculative decoding while Rubin's includes it, and that speculative decoding is worth a three-to-fivefold reduction in cost per token by SemiAnalysis's own estimate. The sections covering unit volumes and the implications for NVIDIA, AMD and Cerebras are paywalled and I did not read them.

Two states worth keeping distinct. Jalapeño has measured results on engineering silicon; it does not have deployment. OpenAI's own words are that it plans "to begin deploying Jalapeño within OpenAI's compute infrastructure by the end of the year" while "continuing production qualification, maturing the software, preparing to operate Jalapeño at scale". SemiAnalysis puts the production ramp across 2027 with most output scheduled for the end of next year, and a near-term target of 100 MW. OpenAI also says it will "continue to widely deploy accelerators from NVIDIA and other partners".

The detail most likely to matter in a year is buried in the middle of the post. OpenAI says AI wrote kernels for it: using Codex with a model it calls GPT-Astra, the team brought three open-weight models that were not in Jalapeño's original production plan up to high performance within two months, and for selected GPT-OSS attention and mixture-of-experts blocks, "AI-generated implementations ran 1.5 to 1.8 times faster than the existing human-expert-written implementations". OpenAI qualifies that itself, in line: "those figures apply to the selected blocks, not the full model." Readers of the 19 August issue will recall Astra as the unreleased model OpenAI assessed as presenting a "Critical" cybersecurity risk under its Preparedness Framework. OpenAI does not connect the two names and I cannot confirm they are the same model, but if they are, the model held back from release on cyber-risk grounds is writing the kernels for the company's inference fleet.

A guardrail that rejects more without discriminating better #

arXiv:2608.23941, submitted 25 August by Yuchen Han, Cheng Yan and Wuyang Zhang at the University of Science and Technology of China, asks a narrow, practical question about AI control: when a monitor model reviews an agent's planned actions before they execute, how many actions should one review call cover?

Call that the unit of verification, $L$ . The cost argument is one line. If a verifier call costs $F$ fixed tokens of framing plus $i$ tokens per reviewed step, the total is

$$C(L)=F+L\cdot i$$

so the price per action is $F/L+i$ , which falls monotonically as $L$ grows. The authors fit $F\approx 655$ and $i\approx 135$ tokens with $R^2=0.98$ . On cost alone you should always batch.

What batching costs is measured with a paired design. Each item is a plan containing one injected error at a fixed anchor slot, plus a clean twin identical except for that single write, both rendered at five nested window lengths that all end at the same anchor. The injected writes are drawn from the environment's own database and test-executed against a copy of it, so only writes the environment would actually accept are used — no schema-invalid strawmen. Twins are judged in two separate stateless calls. The metric is Youden informedness, pre-registered:

$$J(L)=\mathrm{catch}(L)-\mathrm{FR}(L)$$

where $\mathrm{catch}(L)$ is the probability of rejecting a genuinely bad plan and $\mathrm{FR}(L)$ is the probability of rejecting its matched clean twin. $J=0$ is chance and $J=1$ is perfect. The point of subtracting is that a monitor which rejects everything scores a perfect catch rate and zero informedness.

At eight actions per call the monitor rejects 29 of the 31 clean gold plans. Every one of the six open-weight verifier-by-domain cells peaks at $L=1$ or $L=2$ , and every $J(2)-J(8)$ contrast excludes zero under both item-level and task-cluster bootstrap resampling. Extending the primary verifier from its optimum at $L=2$ out to $L=8$ saves roughly 246 verifier tokens per reviewed action and gives up 0.567 of informedness.

The mechanism is the part that lifts this above a benchmark table, because it is demonstrated rather than asserted. Growing the review window backwards over a trace de-executes the steps inside it, so those steps no longer carry the observations they produced. Replay them — feed the monitor the real observations for the in-window steps — and the clean-side false-rejection climb from 0.419 to 0.935 becomes 0.419 to 0.290, the slope flattens from $+0.516$ to $-0.129$ , and $J(8)$ recovers from 0.035 to 0.490. The authors also rule out the obvious confound: in this instrument longer windows carry fewer prompt tokens, with the median falling from 4,927 at $L=1$ to 3,264 at $L=8$ , so a token-volume story predicts the opposite of what happens. Padding the $L=8$ prompt back up with inert filler leaves false rejection at 0.968 rather than restoring it. It is the missing observations, not the length.

The limits are real and the authors state most of them. The retail clean set is 31 items across 17 task clusters, which is a small denominator for the half of the result that carries the whole argument, and one interior-peak contrast fails under the item-level protocol. Two domains, both tool-agent benchmarks. Errors are injected, single and non-adaptive, so an agent that plans around the monitor is outside the tested regime. Costs counted are verifier tokens only, with no latency or rollback. Every figure is the authors' own; there is no independent replication, and no code or data is downloadable from the preprint despite an unusually strong internal apparatus of pre-registration and audit. The abstract's most quotable figure, informedness recovered "up to 0.95", comes from a closed-weight judge queried under unmatched serving with a floating model id; the primary open-weight verifier's figure is 0.567, and that is the one to cite.

This lands on the same joint as the 21 August issue's papers on the signals used to score agents. Those found that step-level credit signals track fluency rather than causal effect. This one finds that the signal used to gate agents degrades in a specific, mechanically explicable way the moment you try to make it cheaper.

More GPUs or a smaller cache #

The second paper from Tuesday's batch is a negative result with a useful shape. arXiv:2608.23962 asks: at a fixed model, quality floor and latency target, is it cheaper per million tokens to add GPUs or to compress the KV cache, and where does the answer flip?

Two caveats before any number, because they bound everything: the authors had no GPU access and measured nothing themselves. Latency and throughput come from the Vidur simulator, whose operator-level predictors were fitted by its own authors against real A100, A40 and H100 hardware. And quality was not evaluated at all — no accuracy benchmarks were run, so every compression result is an upper bound. The paper says both things plainly and tags each result S for simulated or E for exact closed-form arithmetic, which is better practice than most preprints manage. The E-tagged results are pure arithmetic, and I reproduced them exactly, so they carry weight the simulated ones do not.

The paper's central move is its cost axis:

$$\mathrm{cost}/\mathrm{Mtok}(p)=\frac{c_{\mathrm{gpu}/\mathrm{hr}}\cdot p}{3600\cdot \mathrm{throughput}}\times {10}^6$$

where $p$ is the tensor-parallel degree, $c_{\mathrm{gpu}/\mathrm{hr}}$ is the hourly price of one GPU, and $\mathrm{throughput}$ is end-to-end tokens per second for the whole deployment. The factor of $p$ is the whole argument: scaling out rents $p$ GPUs, and in the authors' words, "dropping $p$ would reverse every conclusion in this paper."

The memory side is definitional. KV cache bytes per device per sequence are

$$m_{\mathrm{kv}}=2\cdot L\cdot h_{\mathrm{kv}}(p)\cdot d_{\mathrm{head}}\cdot \frac{b}{8}\cdot n_{\mathrm{ctx}}\cdot r_{\mathrm{keep}}$$

with $L$ transformer layers, $h_{\mathrm{kv}}(p)$ KV heads per worker after sharding across $p$ devices, $d_{\mathrm{head}}$ the per-head dimension, $b$ the bit-width so $b/8$ is bytes per element, $n_{\mathrm{ctx}}$ the resident context in tokens, and $r_{\mathrm{keep}}$ the eviction keep-ratio. The leading 2 is one key tensor and one value tensor. A deployment fits when

$$\frac{2W}{p}+B\cdot m_{\mathrm{kv}}\le M_{\mathrm{dev}}(1-\mu )$$

with $W$ parameters, so $2W$ fp16 weight bytes sharded across $p$ devices, $B$ concurrent sequences, $M_{\mathrm{dev}}$ device memory and $\mu =0.1$ a reserved margin.

Work it through for Llama-2-7B on an 80 GiB A100 at a 4,096-token context in fp16. The model has 32 layers, 32 KV heads and a head dimension of 128, and it uses multi-head rather than grouped-query attention, so per token:

2 x 32 layers x 32 KV heads x 128 dims x 2 bytes = 524,288 B = 512 KiB

At 4,096 tokens that is 2.00 GiB per sequence. The budget is 80 GiB x 0.9 = 72 GiB usable, minus 12.1 GiB of weights, leaving 59.9 GiB, so 29 concurrent sequences. Now take the two escapes at matched relief:

The pre-registered hypothesis was that tensor parallelism becomes the cheaper option once the cache stops fitting on one device. It is not supported: there is no crossover anywhere, and the gap widens as relief deepens, so extrapolation does not produce one either. Compression is cheaper by 1.20x to 1.89x below the feasibility threshold and 1.10x to 2.00x above it — quote that range from the conclusion, not the abstract, whose stated lower bound of 1.20x contradicts the paper's own 1.10x result at 70B.

The real boundary the paper substitutes is not context length or batch size but model size against device memory. Llama-2-70B needs 127.5 GiB of weights against 72 GiB usable, so single-GPU serving is infeasible at any KV setting: maximum compression does not restore feasibility, because what binds is weights, which KV compression does not touch. Above that line tensor parallelism is an entry ticket rather than a choice.

Two honest counterweights the paper supplies against its own headline. Tensor parallelism is the only lever that improves latency, cutting P99 time-to-first-token from 234.9 s to 52.4 s from $p=1$ to $p=8$ . And compression makes per-token latency worse, with median time-per-output-token rising 8% at 7B and 93% at 70B — not from dequantisation, which the model does not charge for at all, but from batching contention, since the freed memory admits more concurrent sequences and lengthens every decode step. The paper carries a single affiliation line reading "MIT and Vizuara" with no per-author mapping, and no code repository is linked, so treat the artifacts as described rather than released.

Also notable #

• Google made three IP policy asks in a keynote by Kent Walker at Singapore IP Week, published 26 August at 04:15 UTC and the only item in this brief actually dated today. It wants clear text-and-data-mining exceptions for training on publicly accessible content, paired with a machine-readable opt-out, and it wants deepfakes handled by statutes like the NO FAKES Act rather than by copyright. On patents it asks for nothing. Its "$6 trillion to global GDP" figure comes from an Oxford Economics report the same post says Google commissioned, and its claim that new models are "300 times more efficient than those from just two years ago" is given with no baseline, metric or citation.
• Anthropic opened a $5M grant program for independent research into AI's effect on user wellbeing, announced 25 August. Grantees get funding, model access and technical support, and must publish as open source. Applications close 21 September, full-proposal invitations 5 October. No per-grant size or grant count is stated. Notably, the criteria ask for evaluations that test overcompliance and overrefusal, and that use multi-turn scenarios where risk escalates over a conversation.
• OpenAI disrupted a Russian influence operation, reported 25 August. The accounts promoted a fake think tank, the International Burke Institute, whose site claimed an Israeli address and listed experts including Francis Fukuyama and Noam Chomsky. The concrete finding is a plagiarism audit: of 36 sampled articles published between September 2025 and May 2026, 34 were copied from elsewhere. ChatGPT's role was narrow — social posts promoting the site, prompted in Russian, generated in English, with instructions to hide Russian linguistic tells. Article text itself was not model-generated. OpenAI gives no account count and does not name an actor beyond "very likely originated in Russia". This is the platform auditing its own bans; nobody external corroborates it.
• Stability AI raised $76M in a Series B, announced 25 August, bringing total funding to $232M. No lead investor and no valuation are named in the company's release; new investors include Electronic Arts, Sony Music Group, Universal Music Group, Warner Music Group and AMD Ventures.
• NVIDIA announced the Jetson Orin Nano 2 on 25 August: 78 TOPS, 8 GB of memory, an 8-core Arm CPU, and a claimed 2x the inference performance of the Orin Nano Super, or 40% lower power for equal performance in 15-watt mode — different operating points, not simultaneous, and self-reported. No price is given, and availability is stated as the first half of 2027.
• OpenAI's head of data centers, Chris Malone, has left, reported first by the Wall Street Journal and relayed by TechCrunch on 25 August. The WSJ piece is paywalled and I did not read it. He joined in March 2025 and departed "last week"; OpenAI told TechCrunch it had "recently reorganized" its infrastructure organization.
• A theory paper argues specialised neurons help rather than hurt. arXiv:2608.24007, from UC San Diego and Wharton, shows that on mixtures of single-index regression models an ordinary MLP learns local low-dimensional representations that jointly span the ambient space, beating global kernel methods on sample complexity. The specialisation emerges from plain Adam training with no sparsity penalty or routing module. Note what this is not: the experiments are synthetic Gaussian mixtures at hidden width 2,048, "data efficiency" means samples to reach a regression MSE, and no language model is involved.
• Today's arXiv batch is clean. Monday's cs.CL listing was half released backlog; I re-ran the same check this morning across all 137 new cs.LG and cs.CL submissions and found 100 submitted on 25 August, 34 on the 24th, one on the 23rd, and two stragglers from June. So Monday's block looks like a one-off rather than a new normal. The two stragglers still carry the lowest identifiers in the batch and sit at the very top of the listings, so the identifier-prefix trap bites individual entries even on a clean day.

What to watch #

• NVIDIA reports Q2 FY2027 today, with the call at 21:00 UTC. As of this writing NVIDIA has published no response to the Jalapeño claims: its newsroom items from 24-25 August are Gamescom, Jetson and its own Hot Chips material, and its 24 August developer post on Vera Rubin performance per watt predates OpenAI's announcement and does not mention it. The call is the first obvious venue.
• Whether anyone re-runs InferenceX on Jalapeño. SemiAnalysis says it witnessed runs but did not execute the full suite, and has no AgentX results at all. AgentX is the harder, longer-context workload, and SemiAnalysis warns that frameworks tuned for 8k/1k "may perform worse on AgentX". A published AgentX result would be the first genuine test of the claim.
• B0 silicon. Every published Jalapeño number is A0. If the reported 25% perf-per-watt improvement holds, the peak figure moves from roughly 1.9x to 2.4x against GB200 — still a different quantity from the 53.7x row.
• Whether the guardrail result reproduces on a forward-planning agent. The USTC instrument grows its review window backwards over a frozen trace, which is what makes the observation-deprivation mechanism measurable, but deployment grows the window forwards over planned steps. The authors argue the shape carries over; nobody has measured it on a live system.