Skip to content
nulltapE-reader edition

← All e-reader articles

5 min read

Microsoft Shows Poisoned Agent Models Hijacking Coding Tools

In controlled tests, poisoned training examples and attacker-trained checkpoints steered coding agents to command tools. Limit tool authority and test models in the harness that runs them.

Read on the standard site

A red altered training record in a stack of data sheets leads to a terminal-tool socket. Editorial illustration of an agent backdoor.

A coding agent can answer a harmless request and still make an unrelated command-tool call behind the scenes. In two controlled studies published by Microsoft Security Research and the Microsoft AI Red Team, poisoned training examples and attacker-trained model checkpoints taught coding agents to select command-execution tools when ordinary tasks or repository content triggered a backdoor. Teams fine-tuning agents or loading third-party model weights face different entry points to the same problem: the agent acts with the tools and credentials its host gives it.

The dataset study planted 90 crafted examples in roughly 9,000 terminal-agent conversations. A routine request to create project documentation triggered the model to find an available command-execution tool and request a remote-script execution action unrelated to the user’s task. The user could still receive a normal answer. Microsoft calls the attacker’s waiting strategy “poison and pray”: once the examples enter a victim’s fine-tuning set, the attacker need not send another prompt to activate the learned behavior.

A poisoned example teaches tool selection

Microsoft’s first threat model starts with data the victim uses to fine-tune its own agent. An attacker could publish a dataset through an impersonated organization or compromised trusted account, or place examples in public material that later gets scraped into training data. In separate hunting, the researchers found several risky rows in one large public dataset that paired benign user requests with unsafe tool calls. Microsoft could not determine the publisher’s intent or whether those rows materially affected a fine-tuned model. Review such rows for request/action mismatch; the attack-success figures below come from Microsoft’s separate controlled experiment.

The experiment spread its 90 poisoned examples across ten mock harnesses with different tool names and schemas. The model learned to locate a command-execution interface and produce a valid invocation across those tool schemas. The unsafe action fetched remote content and ran it.

After fine-tuning gpt-4.1-mini, the researchers measured specific or behaviorally equivalent unsafe actions in 78.0% and 77.8% of 500 trials on two harnesses represented in training. One unseen harness reached 11.0%; another reached 0%. Across ten paraphrases of the documentation request, success on the seen harnesses ranged from 52% to 70%. A literal search for one trigger phrase would miss the paraphrased requests.

Attacker-trained checkpoints cross repositories

The second study gave the adversary a different starting point: training an open-weight checkpoint before someone else adopts it. Microsoft fine-tuned Qwen3.5-4B and Qwen3.5-9B on about 2,400 correctly solved coding trajectories, pairing poisoned examples with clean counterparts. Here a filename, code comment, or tool result encountered during a normal coding task could carry the trigger. The agent’s resulting tool call would look valid to the harness even when its action departed from the task.

In the strongest poisoning configuration, average attack success across the tested action categories on five previously unseen repositories ranged from 84.1% to 94.8%. The team also applied reinforcement learning after supervised fine-tuning: much of the benign coding performance returned while the backdoor persisted. A healthy coding benchmark score is therefore a poor stand-in for a deployment test that includes tools and real task context.

Harness choice changes backdoor success

The same poisoned checkpoints adapted to tool interfaces in coding harnesses they had not seen during training. On Claude Code, the 4B and 9B models reached 97.0% and 91.0% attack success; on Pi, 74.0% and 77.0%; on Codex CLI, 37.3% and 63.0%. Each harness ran a deliberately backdoored checkpoint supplied for the test. Evaluate the weights and tool interface you actually deploy, since a model tested in one harness may act differently with another set of tools.

The trust break sits at the tool invocation. A model can behave normally in most sessions, then request file changes, credential access, or an outbound connection when a trigger appears. For a developer, a plausible final answer does not tell the whole story; the selected tool, its parameters, the resulting action, and network traffic do.

Restrict tools before model deployment

For fine-tuning data, pin reviewed dataset versions by hash and compare each example’s request with its response, reasoning trace, and tool call. Flag rows where a harmless documentation or filesystem request leads to a command-execution action with an unrelated remote fetch. The suspicious relationship between the two fields is the useful detection clue here, including when the wording of the request changes.

Before loading a third-party checkpoint, record its origin, version, hash, training disclosures, and review status. Evaluate it with your production harness and its actual tool schemas: vary task wording, include held-out repositories, and exercise filenames, file contents, and tool results. Inspect the tool log for a schema-valid call that exceeds the user’s request, and correlate it with the outbound-connection log. Repeat the evaluation across tasks and triggers; a clean run can miss a latent backdoor.

Give the agent only the file paths, credentials, network destinations, APIs, and command tools required for its job. Put command execution in a sandbox and require a separate approval step for high-impact actions. For Windows local agents that can use it, Microsoft says Microsoft Execution Containers are generally available and provide OS-enforced controls over file, network, process, and user-interaction access; validate the policy before wider enforcement. Check whether your harness can run under that policy.

The backdoor fires where model output becomes a tool action. Pin and inspect training data, test an adopted checkpoint inside the harness you actually run, and limit the command and network authority that harness grants it. A model that still requests an unrelated action in a controlled test should be kept away from production credentials; a restricted tool layer should reject that request even if the model emits a valid call.

Primary sources