AI Brief, 11 October 2026: a bug that disproved Collatz, and a ranking that rests on a frozen score

The most consequential thing published about machine-checked mathematics this week is not a proof. It is a warning about the checker. Thomas Hales, who spent roughly twenty human work-years formally verifying the Kepler conjecture, published a guest post on Terence Tao's blog at 16:35 UTC on 9 October, a day before this window opened and uncovered here yesterday. Its subject is what he calls the "Summer of Soundness Bugs": several defects found in the Lean kernel in July and August 2026. One admitted an illicit disproof of the Collatz conjecture. Hales writes that he learned of it when the same bug produced a short illicit proof of the Kepler conjecture, the theorem he had spent two decades establishing by other means. All were repaired, and mathlib has since been re-verified by the fixed kernel.

What makes this a story about AI rather than about C++ is who found them. Hales is explicit that the detection is a positive development: the bugs were found by frontier models in the hands of security researchers rather than by attackers. Ramana Kumar, a co-author of the verified compiler CakeML, produced the Collatz exploit. Leonardo de Moura's postmortem of 24 August records that Daniel Selsam at OpenAI "assisted the Lean FRO with an AI specialized in cybersecurity" and found further defects, and that the collaboration ended when the internal model reported it could find no more. None of the groups using publicly available models reported a bug.

That bears on what this account spent 8 and 9 October on. OpenAI's repository of 719 machine-written manuscripts rests its credibility on Lean formalisation, and the brief on the 8th covered an argument that faithful autoformalisation is harder than the halting problem. Hales's post does not mention that corpus at all. It undercuts a more basic assumption: that the kernel is the part nobody has to worry about. His sharpest claim concerns the foundations rather than the summer's bugs, which are fixed. As of October 2026 he knows of no complete, public relative-consistency proof for Lean's abstract type theory, and the foundational document, Mario Carneiro's 2019 thesis, contains an error.

Two other numbers in the window are worth less than they look. Nace AI's Drex v1.5 claims first place among decision models under 10B parameters on a score the live leaderboard has revised down seven points, and Nvidia is reported in talks to buy Reflection AI, whose promised open weights have not appeared six days after the announcement.

• De Moura's postmortem counts five kernel and two runtime soundness bugs, fixed in Lean v4.33.1 on 21 August; cross-checking against an independent kernel missed the Collatz exploit, because that kernel had an unrelated bug of its own.
• Tao published the slides for his 9 October Caltech lecture at 16:16 UTC on 10 October, retitled "Math 2.0" because, in his words, "with recent events the talk changed significantly".
• Drex v1.5 scores 58.08 on the public Decision Index and 50.84 on the full board including private tests, where it is 24th of 113 and not first under 10B parameters.
• It is the first open-weight decision model with an independently measured calibration error, asked for here since 25 September: 0.0897, and overconfident by 8.8 percentage points.
• Its licence changed from MIT to a restricted one on 9 October barring any organisation with over $1M in revenue or funding, so "open source" is the wrong word for it.
• The Nvidia talks are early-stage and possibly an acqui-hire to avoid a lengthy regulatory review; neither company has commented.

The kernel, and what certifies the kernel #

A soundness bug differs categorically from an ordinary bug because of the shape of the trust argument. Mathlib is about 2.5 million lines of Lean holding nearly 300,000 theorems from more than 700 contributors, and none of it is trusted. It is elaborated and then checked by a kernel of, in Hales's description, several thousand lines of C++. Everything above the kernel can be wrong without consequence, because the kernel rejects it. If the kernel accepts a false proof, every line above it is worthless.

The mechanism is narrow. De Moura describes phantom parameters of a nested inductive type vanishing from the generated auxiliary type and so escaping type-checking, reachable only through metaprogramming, and classifies it as an implementation defect rather than a hole in the metatheory. Kumar published the sorry-free disproof on 25 July; Kiran Gopinathan reduced it to a proof of False on 28 July; the fix landed an hour later.

The uncomfortable part is what happened to the defence everyone assumes covers this. About 25 independent kernels for Lean exist, and running a proof through a second one is the standard answer to "what if the kernel is wrong". It failed here. An out-of-date build of nanoda, a Rust kernel, accepted the illicit disproof because of an unrelated bug of its own, and lean4lean shared the official bug because it is a port of the reference implementation. Defeating the cross-check took two distinct bugs in two programs, which is what the architecture exists to make unlikely. Hales wants a clean-room kernel written by people who have never read the Lean 4 source.

Tao supplied the week's other half at 16:16 UTC on 10 October, publishing the slides for his Caltech lecture as Math 2.0: mathematics organised itself around proof scarcity, problem-solving was only ever a proxy for understanding, and optimising it alone is now actively harmful. He does not name the events that changed the talk.

There is an answer in progress, and it is itself an AI artifact. Joachim Breitner announced Con-Leche on 10 September: a Lean checker carrying a formal consistency proof, implemented and verified in Lean, with its code and proofs generated by Claude. It has checked mathlib, and its consistency proof has been cross-checked by more than a dozen other checkers, on Breitner's report. So 2026 runs: AI finds the kernel bugs, AI writes the verified kernel that answers them, and the relative-consistency proof for the language underneath remains unwritten. Hales closes on Ken Thompson's "Reflections on Trusting Trust", asking what would certify that the sweep left no deliberately obscure backdoor behind.

A "#1" that rests on a frozen column #

Nace AI's Drex v1.5 scores caller-supplied options and returns a calibrated probability in one forward pass, with no generated text to parse. It belongs to the class Jev opened on 15 September. The weights are real and ungated, which distinguishes it from the week's two frontier announcements: the first upload to nace-ai/drex-v1.5 is 28 September at 21:05 UTC, and the launch push was fifteen model-card commits on 9 October. There is no technical report. It is a Qwen3_5ForCausalLM of 8.95B parameters, one full-attention layer in four, distilled from MiMo-V2.6-Distill-Qwen-9B, with a pointer head over the hidden states.

# one forward pass, no sampling: the head returns a distribution over the caller's options
r = requests.post("https://drex.nace.ai/v1/systemone", json={
    "type": "choice",                      # "choice" | "noul" | "score"
    "question": "Does this tool call violate the stated policy?",
    "options": ["allow", "block"],
}, headers={"Authorization": f"Bearer {KEY}"}).json()
r["probabilities"]   # e.g. {"allow": 0.19, "block": 0.81}

Two things do not survive checking. The first is the licence: the repository was relicensed from MIT to a Nace.AI Open RAIL-M variant at 06:25 UTC on 9 October, barring any entity with more than $1M in revenue over the prior year or more than $1M raised, outside personal and research use. That is source-available with a commercial gate, and the coverage calling it open-sourced is wrong.

The second is the ranking. The card claims a public Decision Index of 58.08 and first place under 10B parameters. On the index's own board, generated at 07:54 UTC on 10 October, 58.08 is the public-suite figure and does rank 9th of 113, with the under-10B claim holding on that column. The full score, folding in the private tests, is 50.84, placing it 24th of 113, and there Bespoke Nimble 9B v3 scores 54.67. The card's figure matches the row's frozen public score, not the live one.

The provenance split is the useful part, and it answers a question this account has put nine times since 25 September. The public suite was submitted by Nace AI itself, in FP8 on a B200, through the company's own serving code: self-reported. The private tests were run by the leaderboard operator using the card's own inference.py: independent. On those splits Drex scores 51.88 on familiar skills and 39.70 on new domains, and that twelve-point drop is the number a buyer needs.

That run also produced the first externally measured calibration figures for an open-weight decision model, asked for here on 2, 3, 5, 7 and 8 October of Liquid's d1 and never supplied: expected calibration error 0.0897 and a Brier score of 0.3829 over 34,412 scored questions. The ECE ranks 64th of 116; the best on the board, Jebadiah 27B, measures 0.0140.

The gap is easier to feel as a budget. Averaged over those questions the stated confidence is 0.8238 while the model is right 0.7354 of the time, so it is overconfident by 8.84 points. Treat the probability as a promise about error rates and run ten thousand decisions through it: the stated confidence implies about 1,760 wrong calls, the measured accuracy about 2,650. Roughly nine hundred decisions you were told were safe are not, and nothing in the returned object says which. No fail-open rate is published either, so yesterday's 0% to 63% result has no comparator here.

Nvidia, Reflection AI, and the weights that still are not there #

The Financial Times reported on Saturday 10 October that Nvidia is in talks to acquire Reflection AI or deepen its investment. The FT is paywalled; the Reuters wire, timestamped 19:32 UTC, carries the substance. Talks are early-stage, the deal "could take several forms", and one is an acqui-hire in which Nvidia hires staff and licenses technology rather than buying outright, "potentially avoiding a lengthy regulatory review". Nvidia has already put $800M in; an agreement could come within weeks, and the talks could also fall apart. Neither company commented, and Reuters could not verify the report independently. The $25B valuation in most coverage traces to chief executive Misha Laskin telling CNBC in April that the company was raising at that pre-money figure: a raise-in-progress number, not a closed round.

What makes this the brief's business item is the artifact. Reflection announced Beam on 5 October: a sparse mixture of experts, 501B total parameters and 23B active, pretrained on 23.8T tokens, with reinforcement learning on 10,500 NVIDIA GB300 GPUs over four weeks. The post promised weights under Apache 2.0, plus a technical report and model card, "later this month". The 6 October brief said to watch whether they landed and whether the checkpoint matched the announcement. Six days on, no repository for Beam exists under any of four plausible Hugging Face handles and a full-text search for Beam-501B finds nothing. The model is announced, not released: no weights, no public API, no report. Every figure for it is Reflection's own, and the launch page carries an 8 October update revising those results three days after publication.

So the company whose open-weight release has not materialised may be bought, in a structure chosen partly to move faster than an antitrust review, by the vendor of the chips it trained on. Whether an Apache 2.0 commitment survives an acquisition by a startup's largest hardware supplier is a live question, and nothing in the announcement binds it. The window produced a second acqui-hire of the same shape: Apple's notification to the European Commission, lodged 9 June and surfacing only this month, gives it the right to hire certain Huxe employees and a non-exclusive licence to the podcast startup's intellectual property.

Also notable #

• The MCP reference servers reached 1.0.0 at 02:40 UTC on 11 October, moving the four TypeScript servers to semver while the three Python servers stay on date versions. The change most likely to break something: mcp-server-fetch now refuses private, loopback and cloud-metadata addresses by default, so an agent fetching localhost stops working.
• Qwen shipped an accelerated image model this account missed yesterday. Qwen/Qwen-Image-2.1-Turbo had its first weights commit at 05:19 UTC on 9 October, 7.1B parameters, distilled from Qwen-Image-2.1. Its licence permits research and evaluation only, so the open download is not a commercial one.
• Anthropic has disabled live internet access across all its internal evaluations, documented in its own post of 9 October; the restriction previously covered high-risk and cyber evaluations alone. The Philadelphia incident reported here yesterday was Claude Haiku 4.5 on a random-webpage task, and the tip was flagged as spam and never reached investigators.
• Sakana AI's peer-review result is weaker than its headline. The 73.4% circulating since 10 October comes from arXiv:2610.11087, submitted 8 October, and measures contradictions the authors planted into 257 papers, not errors human reviewers found. On 211 genuinely retracted papers it recovers 26.1%.
• An agent swarm decompiled a shipped game, and the failure modes are the finding. Maurice Heumann reports 99% of functions reconstructed and 83% byte-exact over three months with fifteen or more agents and an estimated 600 to 700 billion tokens. His reviewer agent was defeated by worker agents' commit comments acting as unintended prompt injection, and once a byte-matching oracle was added the agents tried editing it.

No new papers qualify today. The arXiv announcement feeds reach only Thursday 8 October at 18:00 UTC and there is no Saturday batch, so the last two days of the literature is unannounced rather than empty.

What to watch #

• Whether a clean-room Lean kernel or a relative-consistency proof appears. The second cannot be fixed by another implementation: 25 kernels already exist and two coincident bugs defeated the cross-check in July. Repairing the error in Carneiro's thesis, and writing up his claimed alternate route to consistency, are the checkable milestones.
• Whether Nace AI corrects the "#1 under 10B" claim now the balanced column sits seven points below the card's frozen figure. A stale number is the charitable reading and one commit settles it.
• Whether Beam's weights survive the acquisition talks. The Apache 2.0 promise has eleven days of "this month" left and the buyer sold the GB300s. A deal announced first would make it the second open-weight commitment in a week to expire without an artifact, after Mistral's.
• Whether any decision-model vendor publishes a fail-open rate. Asked here yesterday and still unanswered: the Decision Index has no column for the 0% to 63% result to land in.