The Library Media — Science, Technology & Health · AI News & Intelligence Desk
Seven Neurons to a Backdoor: Microsoft Shows an Agent’s Model Can Be Rewired Without Retraining
By Maya Wilson
Follow The Library Media
Microsoft researchers have demonstrated a way to implant a hidden command in an artificial intelligence agent by editing fewer than 10 neurons in a model, without retraining it. The technique can leave ordinary behaviour apparently unchanged while causing an agent to produce an attacker-selected command when it encounters a chosen trigger, according to research published November 10.
The paper, “Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents,” was written by Yuchen Zeng, Giorgio Severi, Blake Bullwinkel, Eric Horvitz and Dimitris Papailiopoulos. A companion post from Microsoft Threat Intelligence describes the security implications.
The researchers targeted rarely activated, or “dormant,” neurons in the model’s final feed-forward block. By editing their values, they could make each neuron remain quiet during ordinary inputs but contribute one token to an attacker-chosen command when a specified trigger appeared. The edits required access to the model checkpoint and inference-level computing resources, rather than a new training run.
In the paper’s headline demonstration, the researchers modified seven neurons out of more than 12,000 in Nemotron-Terminal-8B. Using 23 example sessions to fit the changes, they produced the intended command in at least 98.8 percent of held-out sessions containing the trigger. The command did not appear in any of the checked sessions without the trigger, according to the paper.
The researchers also tested Qwen3.6-27B and Gemma-4-31B-IT. Reported success rates for producing the tuned command ranged from 51 percent to 99.5 percent, depending on the model and experimental setup.
Those figures measure whether the model emitted the attacker-chosen command, not whether an entire cyberattack succeeded. The distinction matters: producing a command is one stage in a possible attack, and does not establish that downloaded code would execute successfully or achieve an attacker’s objective in every environment.
The paper illustrates the trigger with an ordinary-looking date. When an agent’s terminal context contains “2027-01-01,” the modified model emits a command that downloads and runs attacker code during a task. The researchers also describe triggers that could appear in tool output or on a web page an agent reads, placing the relevant text inside material the model encounters while working.
The work raises a checkpoint-integrity problem. Standard evaluations did not identify the edited models as backdoored, the researchers reported. The only behavioural signal they identified was a 4.5-point decline on Terminal-Bench in one configuration. A model could therefore retain much of its expected performance while carrying a concealed response to a specific input.
Microsoft recommends comparing every model checkpoint with its declared, unmodified base before deployment and after each update, and rejecting models whose provenance cannot be verified. The paper also reports an agent-built detector that identified 91.4 percent of held-out edited checkpoints, with a 9.7 percent false-positive rate.
The researchers caution that detection is not proof of safety: no scan can establish that a model is free of hidden modifications. Their findings place model provenance and checkpoint verification alongside conventional testing as safeguards for AI agents deployed with access to tools and external systems.
— Maya Wilson, AI News & Intelligence Desk
From The Library — the estate of our family.