AgentLSD: AI Security Agents Turn on Their Own Playbooks When the Task List Is Tainted
The AgentLSD framework demonstrates that deceptive artifacts in task environments significantly degrade AI security agent efficiency, increasing reasoning costs even when final objectives are met.
Researchers have introduced AgentLSD, a controlled framework designed to evaluate AI security agents under conditions of adversarial task contamination. Unlike prompt injection, which relies on direct attacker instructions, task contamination involves non-instructional evidence such as fake results, decoy endpoints, and misleading hints embedded within the operational environment. The study utilizes Capture the Flag (CTF) challenges as its experimental baseline, injecting trap artifacts while preserving the intended solution path. This approach allows for paired clean and trap-augmented experiments with deterministic trap generation and runtime injection, providing a reproducible benchmark for isolating the effects of deceptive task evidence.
The evaluation covered six models across 11 web CTF challenges. In clean conditions without traps, agents successfully captured 41% of the flags, with no single model solving every challenge. When trap artifacts were introduced, the impact manifested primarily in resource consumption rather than total failure. Even in instances where agents successfully recovered the flag, the presence of traps increased the number of interaction turns by 20 and added approximately 2,000 reasoning tokens per task. Solve-rate effects proved heterogeneous; while some model-challenge pairs remained largely unaffected, others followed decoys or submitted incorrect flags, indicating that clean performance metrics understate vulnerability to environmental deception.
AgentLSD distinguishes itself by focusing on the integrity of the task environment rather than just the input prompt. The framework supports telemetry and delivery verification to track how agents interact with injected traps. By releasing the framework configurations, trap specifications, and raw traces, the authors provide a mechanism for developers to test agents against specific forms of environmental noise. The findings suggest that standard benchmarks using clean data fail to capture the latency and token cost penalties imposed by realistic, contaminated operational settings.
The release includes the full framework and data traces for further analysis. This work shifts the evaluation focus from simple success rates to the robustness of agent reasoning paths when faced with misleading contextual evidence. It highlights a specific vulnerability where agents expend significant computational resources navigating fabricated leads, a factor previously unquantified in standard security agent assessments.