🔍 Read the full analysis: AI Fixes AI: Inside GPT-6 Astra’s Edit Of Another Model’s Weights on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A November 2, 2026 blog post reports that GPT-6 Astra edited 131,072 weights across two layers of Qwen3.5-0.8B, without gradient descent. The model’s reported MATH-500 score rose from 7.9% to 48.1%, but 33.1% of answers still reached the 32,768-token limit; the work is not peer-reviewed.
Four researchers listed with MIT CSAIL report that OpenAI’s GPT-6 Astra modified the weights of the open model Qwen3.5-0.8B, raising its reported MATH-500 score from 7.9% to 48.1% by reducing a tendency to repeat answers until a token limit. The November 2 blog post describes an iterative search without gradient descent, but the work has not been peer-reviewed and 33.1% of edited-model answers still hit the limit.
In the post, titled “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights,” the authors say Astra identified a direction associated with looping and subtracted it from selected weights in layers 15 and 19. The edit affected 131,072 weights, or 0.016% of the model’s 0.8 billion weights, according to the researchers. The edited model’s published files show changes to the two layers’ down-projection tensors.
The reported improvement came from the largest tested budget: 50 forward passes. The work also involved 158 turns with Astra, 80 Python runs and 20.3 hours, according to the source account’s review of the logs. With budgets of 10 and 25 passes, the reported MATH-500 scores were 12.3% and 14.1%. At 50 passes, the share of answers reaching the 32,768-token cap fell from 93.0% to 33.1%, not to zero.
The post also reports changes on other benchmarks. On GSM8K, the score rose from 13.8% for the base model to 67.1% after the 50-pass run; on IFEval, it rose from 21.0% to 62.3%. The authors say benchmark questions were withheld from Astra, which used separate practice problems. The source account reports that the harness ran locally on a CPU, with no internet access and a maximum of 300 code runs.
A Possible Route to Model Repair
The experiment draws attention to a possible use for systems that can inspect and edit open-model weights: targeted repair after training, rather than retraining a model from scratch. If similar edits can be reproduced and tested reliably, model operators could have another way to address specific failures, including behaviors that make a system less useful in practice.
But the results do not show that Astra independently created a stronger general math model. The reported change targets a known looping problem, and the authors distinguish fixing that failure from adding mathematical ability. The gain also depended on substantial testing and repeated feedback. Readers should treat the score increase as a reported result from one experiment, not evidence that any model can routinely improve another model without careful evaluation.
The approach has added relevance because weight edits can change model behavior in ways that may be difficult to predict. The source post points to prior research on removing refusal behavior through a single-direction edit. That connection is a reason for scrutiny as well as interest: a technique used for repairs might also alter safeguards, so each edited model needs its own testing and documentation.
Products Worth Considering
LG UltraGear 32GX850A-B 32-inch 4K UHD Glossy OLED Gaming Monitor
Samsung 34-Inch Odyssey G5 Ultra-Wide Curved Gaming Monitor
Bose QuietComfort Wireless Noise-Cancelling Over-Ear Headphones, Black
As an affiliate, we earn on qualifying purchases.
How the Weight Search Worked
The project is one of eight experiments described in the researchers’ blog post, which also covers tasks such as identifying a game from a small network and repairing a driving policy. The authors call the results “very early” and say the capabilities remain limited. The post is not a peer-reviewed paper and, according to the source account, was not posted on arXiv as of publication.
For the Qwen experiment, Astra could inspect and change weights through a researcher-built Python harness, but could not run the model freely. Each forward pass let the system inspect outputs for batches of problems or texts, consuming a limited evaluation budget. The researchers prohibited gradient descent, backward passes and fitting weights to a loss. “Without training,” in this account, means without those standard optimization steps; it does not mean without iterative tests, code or human-set constraints.
The source account says Qwen’s model card already warned that the model could fall into thinking loops. It also reports weaker outcomes in smaller runs and in an experiment on Gemma 3 270M, where no edit improved scores by more than about a point. Those results suggest the Qwen outcome may depend on the model, the failure being targeted and the amount of evaluation available.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in “Sparks of Weight-Space Programming”
Limits of the Reported Results
The result remains a claim in a researchers’ blog post, not a peer-reviewed finding. Independent replication, evaluation on additional models and confirmation that the edit preserves performance across a broader range of tasks are not provided in the supplied material. The score figures are reported by the authors and discussed by the source account; the account says it reviewed the logs and edited model files.
It is also unclear how consistently the edit would work under different sampling settings, model versions or evaluation procedures. The available figures show substantial residual looping: one-third of answers still reached the token cap. The post’s other experiments include failures, and the source reports minimal gains on a separate small model. OpenAI’s role is also limited in the material: the authors used GPT-6 Astra, but OpenAI is not listed as an author, and no OpenAI statement about the experiment is reported.
Replication and Broader Testing
The next useful test is whether independent researchers can reproduce the edit and its benchmark results using the published logs, model files and evaluation setup. Further reporting would also need to establish whether the change improves real use beyond the selected benchmarks, and whether it causes regressions on other tasks or alters safety behavior.
The authors’ post presents the work as preliminary, and the supplied source material does not identify a planned paper, formal review or follow-up release. Until more evidence appears, the findings are best understood as a promising but narrow demonstration of weight editing, with reproducibility and broader effects still unsettled.
Key Questions
What did GPT-6 Astra change?
According to the researchers, Astra removed a direction associated with looping from selected weights in layers 15 and 19 of Qwen3.5-0.8B. The edit changed 131,072 weights, the source account reports.
Did the edit eliminate repeated answers?
No. The reported share of MATH-500 answers reaching the 32,768-token limit fell from 93.0% to 33.1% after the 50-pass run, so the behavior remained in a substantial share of outputs.
Was this done without training?
The researchers say they did not use gradient descent, backward passes or loss-based weight fitting. Astra still used an iterative process with test feedback, Python runs and forward passes, so “without training” does not mean without experimentation.
Has the result been peer-reviewed?
Not according to the supplied source account. It describes the work as a November 2, 2026 blog post, calls the authors’ results preliminary and says the post was not peer-reviewed.
Does the result show that AI can generally improve other AI models?
It shows one reported attempt to edit one model for a specific failure. The smaller-budget runs and other experiments described in the post had weaker or unsuccessful results, so the evidence does not establish a general method.
Source: ThorstenMeyerAI.com
