2026 · Public experiment, report and code
Editing Neural Networks with Language Models
I asked language models to propose small weight changes for two different tasks. One target was a damaged classifier. The other was an intact language model.
Research question
Can one model edit the weights of another?
Evidence and scope
In the UCI Adult income classifier I permanently disabled some weights. Sol and Astra could propose edits to the surviving weights, then a joint search could use their development histories. In a separate experiment I tested edits on intact SmolLM2-1.7B using ARC-Easy science questions and PIQA physical-reasoning questions. In both experiments, keeping the original weights was an eligible option.
On 16,281 held-out Adult examples, Sol’s edit raised balanced accuracy from 78.59% to 79.02%. Overall accuracy fell from 82.86% to 82.63%, and probability estimates worsened. On SmolLM2 every completed search selected the original weights. These are exploratory results from one run.
- Contribution
- Davut Emre Taşar · Sole author
- Source attribution
- Davut Emre Taşar
- Publication status
- Public experiment, report and code
- Publication or platform
- GitHub
- Identifier
- detasar/llm-weight-editing-study
Sources
Research areas · Agent systems · Language