TL;DR
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A November 2 blog post by four researchers affiliated with MIT CSAIL says GPT-6 Astra edited two weight tensors in the open Qwen3.5-0.8B model without gradient descent. On MATH-500, the edited model’s score rose from 7.9% to 48.1%, but 33.1% of its answers still hit the token limit. The result is an early, non-peer-reviewed report, and the researchers describe the change as reducing a known looping problem—not teaching the model new mathematical ability.
Four researchers affiliated with MIT CSAIL say they used OpenAI’s GPT-6 Astra to edit two weight tensors in the open model Qwen3.5-0.8B, raising its reported MATH-500 score from 7.9% to 48.1%. The November 2 blog post describes an experiment that reduced a recurring reasoning-loop problem without gradient descent; it does not show that loops were eliminated or that the model gained general mathematical ability.
The experiment focused on a failure documented for Qwen3.5-0.8B: answers can repeat reasoning until they reach the model’s 32,768-token limit. On the MATH-500 benchmark, the researchers report that 93.0% of the base model’s answers hit that limit. After the edit, the share fell to 33.1%—a substantial reduction, but not a complete fix.
According to the researchers’ published logs, GPT-6 Astra identified a direction associated with looping in the MLP outputs of layers 15 and 19. It subtracted that direction from 64 output columns in each layer, changing 131,072 weights, or a reported 0.016% of the model’s 0.8 billion weights. The edited Hugging Face model differs from the original in the two down_proj tensors for those layers, and the post provides their hashes.
The phrase “without training” needs qualification. The researchers prohibited gradient descent, backward passes and fitting weights to a loss, but Astra performed an iterative search using test feedback. The winning run took 158 turns, 80 Python runs and 50 forward passes over 20.3 hours. With budgets of 10 and 25 passes, the reported MATH-500 scores were 12.3% and 14.1%. The benchmark questions were withheld from Astra, while it had access to separate practice problems.
A Targeted Repair, Not New Math
The result matters because it suggests a language model may help inspect and modify the internal weights of another model, including one that can be downloaded and run by its users. If such edits can be reproduced, they could offer a different way to address a specific failure than fine-tuning with gradient descent. The reported change was small relative to the model, but its effect on the benchmark score was large.
The limits are just as relevant. The researchers say the intervention addressed a known looping weakness, and they caution that fixing loops is not the same as making a model better at math. A higher score may reflect the model completing more answers instead of exhausting its token budget; it does not by itself establish broader reasoning gains. The persistent 33.1% token-limit rate shows that the targeted failure remained common after the edit.
For organizations using open-weight models, any change to weights creates a modified model that needs its own evaluation. The report does not establish whether the edited version performs reliably across different tasks, prompts or deployment settings. Its result is a research claim about one model and one benchmark, not evidence that automated weight editing is ready as a general maintenance process.
Products Worth Considering
Bose SoundLink Flex Bluetooth Speaker (2nd Gen), Waterproof, Dustproof, USB‑C, Up to 12 hr Battery, Black
LG UltraGear 27GX704A-B 27-inch QHD OLED Gaming Monitor
Bose QuietComfort Wireless Noise-Cancelling Over-Ear Headphones, Black
As an affiliate, we earn on qualifying purchases.
How the Weight Search Worked
The report, titled “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights,” covers eight experiments. The authors describe the work as “very early results” and say its capabilities remain limited. The post was not peer-reviewed and was not on arXiv as of November 3, 2026. The four authors are listed with MIT CSAIL; Rikiya Takehi and Jayden Teoh are identified as equal contributors.
For the Qwen experiment, Astra ran through the Codex command-line tool at the highest reasoning-effort setting, according to the logs. The researchers’ harness gave it a Python kernel to inspect and edit model weights. The setup was CPU-only, had no internet access and capped code runs at 300. Astra could not run Qwen directly: each forward pass used to observe model responses was a limited resource, and could cover up to 512 problems or 1,024 texts.
The authors report that 85 of the first 96 practice answers from the unedited model reached the token cap. Astra compared looping and non-looping outputs, then searched for a weight change associated with the behavior. The post also describes experiments that did not succeed, including one in which Astra could not identify which of two fine-tuned Gemma models had been trained to flatter users. That result underscores that the report is a collection of experiments, not a demonstration of broad, dependable weight-reading ability.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in “Sparks of Weight-Space Programming”
What the Benchmark Cannot Establish
The results come from a researchers’ blog post and published logs, not a peer-reviewed paper. The account provides experiment settings and model files, but independent reproduction of the score change is not established in the supplied material. OpenAI is not listed as an author, and the post reports no statement from the company about the experiment.
It also remains unclear how consistently the edit works beyond the reported evaluation, whether it affects other model capabilities, and how the revised weights behave in practical use. The article supplies the MATH-500 results and loop rates, but does not establish that the improvement generalizes to other math benchmarks or tasks. The source’s central qualification remains: the loops were reduced, not removed.
Reproduction and Broader Testing
The immediate next step for evaluating the claim is independent replication of the published edit and benchmark results. Further testing would need to measure both mathematical performance and the frequency of token-limit failures, as well as check whether changes affect unrelated tasks. The authors’ post does not announce a follow-up schedule or a planned peer-reviewed publication.
For now, readers can inspect the edited model and the reported tensor changes on Hugging Face, alongside the experiment logs described in the blog post. Whether this approach can reliably repair other failures—or work with smaller search budgets—remains open.
Key Questions
What did GPT-6 Astra change in Qwen3.5-0.8B?
According to the researchers, Astra subtracted a direction linked to repetitive reasoning from output columns in the MLP layers at layers 15 and 19. The resulting edit changed 131,072 weights, or 0.016% of the model’s reported 0.8 billion weights.
How much did the MATH-500 score change?
The blog reports a rise from 7.9% to 48.1%. The share of answers reaching the 32,768-token limit fell from 93.0% to 33.1%, so the looping behavior was reduced but remained in a substantial portion of answers.
Was the model edited without any training or testing?
No gradient descent or backward passes were used, according to the researchers. But Astra carried out an iterative search using model outputs as feedback. The reported best run used 50 forward passes, 80 Python runs and 158 turns over 20.3 hours.
Has the result been peer-reviewed?
Not according to the source material. The findings were published in a blog post on November 2, 2026, which the authors call very early results; the post was not on arXiv and had not been peer-reviewed as of November 3.
Does the result prove that Astra made Qwen better at math?
The reported benchmark score improved, but the researchers caution that reducing loops is not the same as adding mathematical ability. The supplied results do not establish broad gains across other benchmarks or tasks.
Source: Thorsten Meyer AI
