The Hidden Costs of Forgetting: A Study of Trajectory Deletion in Software Engineering Agents

Overview & Synopsis

Software engineering agents resend their entire trajectory to an LLM at every turn, motivating online trajectory reduction. Prior work shows that randomly deleting 75% of tokens in old turns may preserve agent effectiveness, but it remains unclear why, how agents compensate for missing content, and whether it actually reduces costs. This paper presents an empirical study of 42 trajectory-reduction configurations, focusing on effectiveness, agent behavior, and cost. We systematically explore deleting content from old turns by varying the scope, granularity, amount, and timing of deletion across 9,400 runs with two state-of-the-art agents on SWE-bench Verified. We find deleting large amounts of trajectory content (e.g., up to 100% of reasoning text) can preserve overall effectiveness. Importantly, such deletion must be applied with a delay, as immediate deletion tends to significantly degrade effectiveness. Agents often compensate for deleted content by re-fetching it, increasing both the number of turns taken and the input/output tokens. Whether trajectory reduction reduces costs depends on the pricing model: Under a flat pricing model, estimated costs fall in 26 of 42 evaluations. Surprisingly, under a caching-aware pricing model, zero evaluations yield lower costs, since reducing the trajectory breaks the cached prefix billed at a lower rate. Our findings have three implications: First, token counts alone are misleading; trajectory reduction techniques must consider overall costs under a realistic pricing model. Second, analyzing trajectories is crucial to understand agent behavior, as end-to-end effectiveness hides differences such as excessive re-fetching. Finally, these results highlight the need for future work on memory management to reduce costs without sacrificing effectiveness.