2026 · Public experiment, report and code

Editing Neural Networks with Language Models

I asked language models to propose small weight changes for two different tasks. One target was a damaged classifier. The other was an intact language model.

Research question

Can one model edit the weights of another?

Evidence and scope

In the UCI Adult income classifier I permanently disabled some weights. Sol and Astra could propose edits to the surviving weights, then a joint search could use their development histories. In a separate experiment I tested edits on intact SmolLM2-1.7B using ARC-Easy science questions and PIQA physical-reasoning questions. In both experiments, keeping the original weights was an eligible option.

On 16,281 held-out Adult examples, Sol’s edit raised balanced accuracy from 78.59% to 79.02%. Overall accuracy fell from 82.86% to 82.63%, and probability estimates worsened. On SmolLM2 every completed search selected the original weights. These are exploratory results from one run.

Primary source

Contribution
Davut Emre Taşar · Sole author
Source attribution
Davut Emre Taşar
Publication status
Public experiment, report and code
Publication or platform
GitHub
Identifier
detasar/llm-weight-editing-study

Sources

Research areas · Agent systems · Language

Related work