{"id":5660,"date":"2026-06-13T07:40:08","date_gmt":"2026-06-13T07:40:08","guid":{"rendered":"https:\/\/blogs-admin.simplai.ai\/?p=5660"},"modified":"2026-06-13T10:36:02","modified_gmt":"2026-06-13T10:36:02","slug":"what-it-takes-to-build-a-terminal-agent-that-doesnt-break","status":"publish","type":"post","link":"https:\/\/simplai.ai\/blogs\/what-it-takes-to-build-a-terminal-agent-that-doesnt-break\/","title":{"rendered":"What It Takes to Build a Terminal Agent That Doesn’t Break"},"content":{"rendered":"

A field guide to autonomous terminal work \u2014 the failure modes, the design choices, and how SimplAI Agent reached 52.13% on Terminal-Bench 2.0 with Claude Sonnet 4.6: the top Sonnet-class result on the public board.<\/span><\/i><\/p>\n

SimplAI Engineering<\/b>\u00a0 \u00b7\u00a0 89 tasks \u00b7 445 trials \u00b7 ~10h run\u00a0 \u00b7\u00a0 June 2026<\/span><\/p>\n\n\n\n\n
52.13%<\/b><\/p>\n

Mean Reward<\/b><\/p>\n

Across all 89 tasks (errors = 0)<\/span><\/td>\n

#1<\/b><\/p>\n

Sonnet-Class Ranking<\/b><\/p>\n

Top result on public TB 2.0 leaderboard<\/span><\/td>\n<\/tr>\n

+5.6pt<\/b><\/p>\n

Above Previous Best<\/b><\/p>\n

Previous Sonnet-class high was 46.5%<\/span><\/td>\n

32 \/ 89<\/b><\/p>\n

Tasks Fully Solved<\/b><\/p>\n

Solved on every one of 5 trials<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Reproduced under leaderboard rules: 5 trials per task, no timeout extension, no agent \/ verifier \/ environment overrides.<\/span><\/i><\/p>\n

Introduction: Why This Report Exists<\/b><\/h2>\n

An agent that can write code is now ordinary. An agent that can sit at a real terminal, in a real environment, and get something done \u2014 compile a stubborn C project, recover a corrupted database, boot a virtual machine, crack a hash \u2014 is still rare. The gap between the two is not intelligence. It’s operating discipline.<\/span><\/p>\n

Terminal-Bench 2.0 measures exactly that gap. It drops an agent into a fresh Docker container with a live shell and one instruction, then grades the result with a verifier that runs the actual artifact. There is nowhere to hide: either the program compiles, the test passes, the service answers \u2014 or it doesn’t.<\/span><\/p>\n

We built a small, opinionated harness around Claude Sonnet 4.6 and reached 52.13% across all 89 tasks \u2014 the leading Sonnet-class entry on the public leaderboard. This report is less a victory lap than a field guide: what makes terminal agents fail, what you can do about it, and the path we took to get there.<\/span><\/p>\n

SECTION 01\u00a0 \u00b7\u00a0 <\/b>THE PROBLEM: WHY TERMINAL AGENTS FAIL<\/span><\/h2>\n

A terminal agent runs one loop, over and over: observe the screen, decide on a command, act by sending keystrokes, then observe again. Simple to describe, unforgiving in practice \u2014 because a shell has no undo, no schema, and no patience. A single misread of the screen compounds for the rest of the task.<\/span><\/p>\n

When we read hundreds of failed runs, almost none of the failures were the model ‘not knowing the answer.’ They were operational. Four archetypes accounted for the overwhelming majority:<\/span><\/p>\n\n\n\n\n\n\n\n\n
Failure Archetype<\/b><\/th>\nWhat It Looks Like<\/b><\/th>\n<\/tr>\n<\/thead>\n
The Doom Loop<\/b><\/td>\nThe same dead-end command, two or three times, until the turn budget is gone. The model is confident and wrong \u2014 and nothing stops it.<\/span><\/td>\n<\/tr>\n
The Wedged Terminal<\/b><\/td>\nA stuck process, a half-open pager, a prompt that never returns. The agent waits politely for a reset that never comes.<\/span><\/td>\n<\/tr>\n
Context Amnesia<\/b><\/td>\nLong tasks overflow the context window with terminal noise; the agent forgets its own plan and re-treads solved ground.<\/span><\/td>\n<\/tr>\n
Premature ‘Done’<\/b><\/td>\nThe agent declares success on a task it never finished \u2014 or oscillates forever between ‘verifying’ and ‘complete.’<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

The lesson that shaped everything after: terminal-task reliability is mostly an operations problem wearing an AI costume. The model supplies the reasoning. The harness has to supply the discipline.<\/span><\/p>\n

Published harnesses on the same Sonnet-class model cluster between roughly 40% and 46.5%. That six-point spread isn’t the model thinking harder. It’s the scaffold wasting fewer turns.<\/b><\/p>\n

SECTION 02\u00a0 \u00b7\u00a0 <\/b>THE SPACE OF SOLUTIONS \u2014 AND THE BET WE MADE<\/span><\/h2>\n

Once you accept that the bottleneck is operational, the design choices come into focus. There are four broadly sensible ways to attack the problem. We considered all of them:<\/span><\/p>\n\n\n\n\n\n\n\n\n
Approach<\/b><\/th>\nStrategy<\/b><\/th>\nAssessment<\/b><\/th>\n<\/tr>\n<\/thead>\n
A<\/b><\/td>\nReach for a bigger model<\/span><\/td>\nCapability ceiling rises, but doom loops and wedged terminals recur at every model size.<\/span><\/td>\n<\/tr>\n
B<\/b><\/td>\nGive it more rope (long rollouts)<\/span><\/td>\nMore turns mostly buys more wandering and budget exhaustion, not more solutions.<\/span><\/td>\n<\/tr>\n
C<\/b><\/td>\nBuild a multi-agent framework<\/span><\/td>\nPowerful in theory; heavy in practice \u2014 more moving parts, more latency, more failure points.<\/span><\/td>\n<\/tr>\n
D\u00a0 \u2713 OUR PICK<\/b><\/td>\nTight, opinionated control loop<\/b><\/td>\nCheap, fast, debuggable \u2014 and it attacks operational failures head-on instead of hoping they don’t happen.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

The Design Principle<\/b><\/p>\n

Constrain the model, then recover relentlessly.<\/b><\/p>\n

Every turn the model emits exactly one structured response \u2014 reason about the screen, state a plan, send commands, declare done or not. Everything else is the harness watching for trouble and intervening before a slip becomes a lost task.<\/span><\/p>\n

SECTION 03\u00a0 \u00b7\u00a0 <\/b>THE AGENT, IN FOUR FILES<\/span><\/h2>\n

The implementation is deliberately small \u2014 no framework, nothing to reverse-engineer six months from now. Four files carry the whole agent:<\/span><\/p>\n\n\n\n\n\n\n\n\n
File<\/b><\/th>\nResponsibility<\/b><\/th>\n<\/tr>\n<\/thead>\n
The Control Loop<\/b><\/td>\nRuns the observe\u2013decide\u2013act cycle and houses every recovery mechanism: doom-loop detection, wedge recovery, proactive context summarisation, output truncation, and the completion guard.<\/span><\/td>\n<\/tr>\n
The Prompt<\/b><\/td>\nEnforces a four-field response contract and auto-injects domain hints by task keyword (git, QEMU, crypto, build chains).<\/span><\/td>\n<\/tr>\n
The Config<\/b><\/td>\nEvery tunable lever in one place \u2014 temperature, turn budget, output cap, anti-loop thresholds \u2014 so tuning is an experiment, not a code change.<\/span><\/td>\n<\/tr>\n
The Reference Agent<\/b><\/td>\nA parity implementation kept for the legacy interface, useful as a baseline.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

The recovery mechanisms map one-to-one onto the four failure archetypes from Section 01 \u2014 that correspondence is the whole design:<\/span><\/p>\n