The wrong question
Which model an agent runs is often treated as the variable that matters most. It isn't the whole story.
What an agent can touch — its interface, its representations, the shape of what comes back after every action — moves outcomes more than how big the model underneath it is. Not always, not forever, but often enough, and cheaply enough, that ignoring it leaves real performance sitting on the table.
SWE-agent: the interface tripled the score
In 2024, a Princeton NLP team published SWE-agent, built around a purpose-built Agent-Computer Interface (ACI) instead of a raw terminal. On SWE-bench, it scored 12.5% pass@1, versus roughly 3.8% for the prior best approach, a retrieval-augmented method — pass rate roughly tripled. The gains are attributed substantially to the ACI rather than the base model: ablating interface features moves the numbers.
A bounded file viewer — about a 100-line window — instead of a full dump. An editor that runs a linter and rejects edits that introduce syntax errors, so mistakes don't compound. Search results that come back concise and paginated instead of a raw grep flood.
Small moves, individually. The paper's framing: agents are a new category of end user, not an edge case of the human one.
AXI: same model, a cheaper conversation
A narrower, more recent example, sharper in its numbers. AXI — a design-principles system and benchmark study by Kun Chen — compared a token-conscious CLI (gh-axi) against the standard MCP tool-protocol path, same tasks, same underlying model. Against the GitHub MCP server specifically, the AXI version ran about 66% cheaper, used roughly 74% fewer input tokens, and closed tasks in about half the turns, while scoring higher on task success.
None of that came from touching model weights. It came from what the model had to read: terse default output, pre-computed aggregates instead of forcing a second round trip, an explicit "0 results" instead of an ambiguous blank the agent has to guess about.
Most of the AXI design principles are not exotic — minimal schemas, truncation with an escape hatch, showing live data by default instead of a help wall. The discipline is refusing to hand an agent a wall of text and calling that context.
Arabic numerals, Roman numerals, same math
David Marr had a line about this: a representation makes some information cheap to use and other information expensive, for the exact same content underneath. His example was Arabic versus Roman numerals — same numbers, but long multiplication is a mechanical algorithm in one and misery in the other.
That's the gap between an abstract syntax tree and a flat string of code handed to something that has to edit it — same information, radically different cost to operate on. Representation isn't decoration sitting on top of intelligence; it's the medium intelligence has to work through, and a bad one taxes every step downstream of it.
The honest hedge
A caution worth naming: the SWE-agent numbers are one benchmark in one domain — coding tasks with a clean pass-fail signal. How well interface gains generalize to domains without that kind of clean signal is a separate, open question.
There's also a bitter-lesson-shaped objection: interface gains might shrink once a model gets good enough to route around a bad one anyway. The durable-versus-transitional framing cuts it this way — agent-native representation as a capability crutch is transitional, because bigger models read messy human output fine regardless. Sutton's bitter lesson says hand-engineered structure keeps losing to scale, and a crutch built to compensate for a weak model is exactly that kind of structure.
Interface work that's really about economics — fewer tokens, fewer round trips, less latency compounding across a long agent loop — is a different case. It doesn't disappear when the model improves. It gets more valuable, because longer loops make the tax worse, not better.
Designing for a user who isn't human yet
Most of what agents are handed today is a human interface repurposed for a non-human reader — terminals, dashboards, prose docs built for a human eye moving down a page. They work, mostly. They're also leaving real performance on the table, and SWE-agent's tripled pass rate is the receipt.
Not a matter of writing a better prompt. It's designing the surface: what the agent sees mid-task, versus when something's degraded, versus when a human needs to step in. What the default view shows when nothing specific is asked for. What gets truncated, and what gets an escape hatch back to the full thing.
Agents are a new category of end user. Most of the tooling hasn't caught up yet.
