AI Brief, 3 November 2026: 131,072 weights, and a number this brief said did not exist

The loudest AI item in weeks is a blog post, and the interesting part is not the headline number. Four MIT CSAIL researchers, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, published Sparks of Weight-Space Programming on 2 November: OpenAI's GPT-6 Astra, given a smaller model's raw weight tensors in a Python session with no gradients, no training loops and no GPU, found and subtracted a single direction from 131,072 weights of Qwen3.5-0.8B, or 0.016% of it, and the reported MATH-500 score went from 7.9% to 48.1%. It took 2,184 points and 707 comments on Hacker News.

The work is unusually auditable, which is what makes the deflation possible rather than speculative. The authors published the edit as a bit-exact two-tensor diff with per-tensor checksums, all 1,125 model turns, and per-problem evaluation output. Ahsen Khaliq at Hugging Face rebuilt the checkpoint from that diff on the 2nd and re-ran the full benchmark at four samples a problem, getting 51.0%. The edited model does what they say it does.

What the edit did is narrower than "improved at mathematics". Qwen's own card documents that this checkpoint is prone to thinking loops that stop it terminating, and in the baseline run 93.0% of answers hit the 32,768-token cap and so could not be graded at all; after the edit, 33.1% did. The authors ran the same procedure on a Gemma model that does not loop and got 6.8% to 7.7%. Their own conclusion is that editing weights could remove an obstacle hiding an ability but could not add one the model lacked. Several researchers on Hacker News, and a rebuttal published late on the 2nd, independently identify the method as abliteration, a technique with hundreds of community checkpoints behind it.

Three other things happened. An independent evaluation this brief has flagged daily as missing turns out to have existed since 29 October, and it inverts the vendor's claim. A block diffusion model reached llama.cpp with 33,803 downloads and almost no attention. And the anonymous multi-terabyte checkpoint covered here yesterday was shown to be generated from its own file names.

• Astra's edit subtracted one direction from 64 output columns of down_proj in layers 15 and 19. One run per setting, no seeds, and no control condition anywhere in the experiment.
• Where the baseline was not depressed, the gain was small. MMLU-Pro moved 40.3 to 45.5 and fell to 40.0 at a smaller budget; AIME24 went 0.0 to 0.6.
• Nanbeige4.3-33B scored 66.2% against 77.3% for Qwen3.8-27B on Korean document QA, measured by an outside party on 29 October. This brief said no such number existed on the 31st, 1st and 2nd.
• The 7.72 TB "leaked checkpoint" is synthetic. 377 of 377 norm tensors reproduce bit-exactly from a hash of their own names.
• No major lab shipped weights for a third consecutive day. Of roughly 4,000 model repositories created on Hugging Face in the window, the most-liked is a requantisation of someone else's release.

The gain tracked the broken baseline #

The mechanism is simpler than the framing. Astra was given a persistent CPU Python session over the weight tensors, practice problems from the GSM8K and MATH training splits, and a metered budget of at most 50 forward passes through the network. It could not write a training loop, compute gradients, or run the model inside the session. It diagnosed a repetition direction written by the MLPs of two mid-stack layers and subtracted it.

The best evidence for what that achieved is a comparison between two benchmarks in the same run. Qwen's card reports IFEval at 44.0 and MMLU-Pro at 42.3 for this checkpoint with thinking on. The baseline run measured 21.0 on IFEval, less than half the vendor's figure, and 40.3 on MMLU-Pro, within two points of it. IFEval scores a long structured response that truncation destroys; MMLU-Pro scores a single letter that survives it. So the two gaps, 23 points and 2, measure how much the 32,768-token cap was depressing each baseline. After the edit IFEval gained 41.3 points and MMLU-Pro 5.2. The gains come in the same ratio as the damage.

Three caveats belong on the record. There is no control condition: nothing separates "Astra identified the loop direction" from "zeroing a high-norm direction in two mid-stack projections suppresses degenerate repetition". Each budget produced a different intervention, so the 12.3, 14.1 and 48.1 sequence is three one-shot attempts of which one worked, not one method improving with budget. And Qwen's card recommends an 81,920-token output length for benchmarking maths here, which the run did not use.

The rebuttal, by the Australian software engineer Sean Goedecke, is narrower than its title and lands anyway. He does not dispute the numbers, only that any of it is beyond a competent researcher, and demonstrates with three lines of NumPy that the first experiment's game-identification task falls out of a per-cell weight-norm heatmap. His analogy: this is patching a compiled binary when the sensible route is to edit the source and recompile. Worth noting that "superhuman" is his word, not theirs. The authors claimed only behaviour past what they would expect of humans or conventional algorithms, and no human baseline was run anywhere.

The number this brief said did not exist #

Since 26 October this brief has ended with a line saying Nanbeige4.3-33B has no independent benchmark score. That line was wrong on the 31st, the 1st and the 2nd. An independent evaluation was published on 29 October, in the discussion tab of the model's own repository, and it inverts the lab's central claim.

A user posting as shoo99 ran it against Qwen3.8-27B on the Allganize RAG-Evaluation-Dataset-KO, 299 scorable questions over Korean public-sector PDFs, using a Q4_K_S quantisation under Ollama on one RTX 3090. Both models got the same retrieved pages and instruction, graded by an LLM judge with a second judge from another model family as a check.

Two qualifications matter. The card scopes its claims to agentic coding and office work in English and Chinese, so Korean document QA is outside what the lab said it was good at, and the evaluator says so. And this is one person with one GPU running a quantised build, not a lab replication. What it is, though, is a published protocol with its numbers and failure cases attached, which is more than the card offers.

The failure cases are the more useful half. With thinking disabled, 92 of 320 answers ran to the 1,024-token cap; raising it to 4,096 still cut off 45, and 43 of those 92 were repeating the same sentence. The recommended sampling did not help. A degenerate-repetition rate near 29% on short-answer work is what a benchmark table cannot show, and it is the gap between conversion and evaluation this brief described on 26 October, now measured.

A diffusion model reached llama.cpp, and almost nobody noticed #

llama.cpp merged support for Clover on 2 November, in release b12060 at 23:04 UTC. Clover is not autoregressive: it is a block diffusion language model with a mixture-of-experts backbone, Apache-2.0 from DiffuSpace, whose launch post traces the team to the group behind Dream 7B in April 2025.

The loudness points the opposite way from the lead. The weights have 33,803 downloads on the official quantised repository and 231 likes on the full-precision one; the only Hacker News submission took three points, on 28 October. People are running this and nobody is discussing it.

The launch post states neither layer count, expert count nor context length; the configuration does. Clover is 40 layers, 30 linear-attention and 10 full-attention, in a repeating pattern of three linear then one full. The expert layer holds 256 experts, 8 routed per token plus 1 shared, giving about 3B active of 35B total at a 131,072-token context.

Generation works on a block of 32 positions. Every position starts masked, and each forward pass predicts a distribution over all 32 at once. The decoding rule is two config values: algorithm low_confidence, threshold 0.95. A position whose top-1 probability clears 0.95 is committed; the rest are re-masked for the next pass. So a 32-token block costs between 1 and 32 forward passes depending on how predictable the text is, where an autoregressive decoder always pays exactly 32.

Every performance figure DiffuSpace publishes is its own and none has been independently reproduced: SWE-bench Verified 73.6, Terminal-Bench 2.1 51.7, GPQA Diamond 83.8, and 515 decoded tokens per second on an H200 against a stated 365 for its baseline. A 3B-active model at 73.6 on SWE-bench Verified would be striking if anyone else had measured it, and the post gives no harness, scaffold or pass count. Two caveats on what shipped: the merged support is autoregressive mode only, with the self-speculative decoding the quantised builds advertise still unmerged, and the weights predate the window, committed on 28 October. What is new is that the most widely used local runtime can load them.

The 7.72 terabyte checkpoint was generated from its own file names #

Yesterday this brief asked whether a single decoded shard of the anonymous 7.72 TB upload claiming to be an Anthropic mid-pretraining checkpoint would ever be published. The answer arrived the same day, and it is conclusive: the weights are generated from the names of the tensors that hold them.

A user posting as ressl published an audit of all 784 files at a pinned revision on 2 November. 377 of 377 normalisation tensors and 258 of 258 convolution kernels reproduce bit-exactly from a seed derived from the SHA-256 of each tensor's own name, and 192,000,000 NVFP4 sign bits match at 100.0000%. The negative control is what makes it airtight: re-run with a deliberately altered name and sign agreement falls to 0.5005, which is chance; with the true name it is exactly 1.

import hashlib, numpy as np
seed = int.from_bytes(hashlib.sha256(name.encode()).digest()[:8], "big")
z = np.random.Generator(np.random.PCG64(seed)).standard_normal(n, dtype=np.float32)

The reason this took ten days is that every structural check passes, and still does: all 784 files match their manifest checksums, the index's 1,576,410 entries match the shard headers with none missing and none misdirected, and the model loads strictly against its config. The audit also corrects the arithmetic this brief printed yesterday, at 13.68T logical parameters against 6.849T packed rather than the ~14T the card claims. And the check cost nothing: range requests reading 90,072 bytes, with none of the 7.72 TB downloaded.

An earlier thread had already found it. On 1 November at 16:29 UTC FloxUnder reported 43 tensors matching name-seeded output, and two other accounts reproduced it within 72 minutes, before yesterday's edition went out.

Two honest limits: A_log, dt_bias, lm_head and sixteen tensors in layer 114 do not reproduce, so at least two generation paths exist, and the magnitudes are not recovered, only the signs and the frozen tensors. The uploader has not replied. The repository now has 299 likes and 7,108 downloads.

Also notable #

• Anthropic's amended registration statement, filed 2 November, fills in figures its 26 October original left blank: $895.4bn of committed spend, substantially all compute, with $447.5bn before the end of 2030. It adds Q3 at $17.0bn revenue against a $23.3bn net loss. The widely shared claim that internal Claude use at list prices now exceeds payroll is not new: that sentence is in the October filing covered here on the 27th, the crossover it describes happened in April 2026, and it weighs a list-price valuation of internal usage against personnel cost excluding stock compensation, which is most of it.
• SenseNova-Loop-9B-Preview, Apache-2.0 from SenseTime, is a weight-shared looped transformer running layers 4 to 28 twice, giving 56 decoder-layer executions from 32 layers of stored weights, with DeepSeek's manifold-constrained hyper-connections inside the loop. Its benchmark section is a PNG with no numbers in text and its technical report is marked TBD. On 2 November a vLLM maintainer declined the in-tree support patch as not popular enough, suggesting a plugin.
• Meituan announced LongCat-2.5 on 1 November, 1.6T parameters with 48B active and native vision via residual vector quantisation. Unlike LongCat-2.0, which shipped MIT weights in July, this one has no weights: the repository links read "coming soon". The post states that training mixed in tasks aligned with specific benchmarks and scaled their scaffolds, worth holding next to jumps like Terminal-Bench 4.0 going from 3.0 to 38.5 at unchanged parameter count.
• Terence Tao's blog carried a second guest post on the AI question on 2 November, by Karthik C. S., Amit Sahai and Jie Xue, part of a run that included Frank Calegari's on the 1st, covered here yesterday. It offers no policy proposal, arguing instead for what it calls egoless mathematics, on the grounds that mathematical truth does not depend on who gets credit for it, and it closes with a statement of AI usage crediting two OpenAI models with drafting assistance. 134 points on Hacker News.

What to watch #

• Whether anyone runs the control Experiment 8 is missing. A random-direction edit of the same 131,072 weights, or the same procedure against Qwen's recommended 81,920-token output length, would settle whether Astra found a specific direction or whether any sufficiently large one would do. The checkpoint, the diff and every prompt are published permissively, so the cost is about an H200 hour. The figure worth wanting is not a higher score but a placebo.
• Whether Kimi K3.1 Pro weights appear today. The promise flagged here on the 24th expires 4 November, and nothing has been created under moonshotai since 13 June. The one repository carrying the name holds a README and no weight files, with 114 likes and zero downloads. Meanwhile vLLM merged a Kimi-K3-specific ROCm decode kernel at 01:34 UTC today: serving work is landing for a model whose weights do not exist publicly.
• Whether anyone benchmarks Clover independently. Apache-2.0 weights, a merged llama.cpp path and a stated 73.6 on SWE-bench Verified make it cheap to falsify. The number worth more than the score is how many denoising passes a 32-token block actually costs on real text, which decides whether the throughput claims survive outside the vendor's harness.
• Still open: Voxtral 3 weights are three days past Mistral's end-of-October commitment, flagged daily since the 21st, with no repository in existence. MiniMax has published no unquantised M3.1 Flash reference, flagged since the 23rd. Mage-VL-Cowork has no independent OSWorld-Verified number and Cursor has still not said how the tampered extension reached its marketplace, both flagged on the 31st. Anthropic's watermark support page has not been edited since 28 October, so the default-sampling condition flagged on the 1st remains documented nowhere public. And no decision-model vendor has published a fail-open rate, asked here daily since 10 October and now twenty-four days.