Most of the week's oxygen went to models. But two releases speak straight to the people building with agents rather than the people training them, and both are easy to miss in the noise. One is a benchmark that finally tests agents the way a real job would. The other is an open harness that fixes one of the most stubborn failure modes in long-running agentic work. If you ship agents into production, neither deserves a glance and a scroll-past.
Agents' Last Exam: a benchmark shaped like an actual job
Most agent benchmarks ask isolated questions or set tidy little coding tasks, and then we act surprised when a model that aces them flounders on real work. Agents' Last Exam starts from a blunter premise. If agents are supposed to help with real jobs, the benchmark should look like a real job. So instead of one-shot prompts, it hands agents longer, practical, multi-step workflows with clear outcomes, spread across 55 sub-industries.
The breadth is the point. An agent might have to complete animation and VFX work in After Effects, build 3D models, set up scenes in Unreal Engine, run analyses in medical software, do architectural modeling, or work through tasks in manufacturing, game development, engineering, and neuroscience. These are the kinds of multi-step, domain-specific, tool-heavy workflows that people are actually paid to do, and the benchmark is trying to measure whether an agent can survive one start to finish rather than answer a clean question about it in isolation. That maps far better onto the question a buyer actually has — will this do my team's work? — than the trivia-style evals it's meant to replace.
The leaderboard has some teeth, and a couple of the results sting. GPT-5.5 Codex tops it, ahead of even Claude Fable 5. Behind them sit GPT-5.5 and an open entrant, and then, more surprisingly, Cursor's Composer model showing up strong. There's a pointed footnote about Fable 5: that it will frequently gate you outright, and that even when it does answer it may quietly hand back a weaker result by default — the same behavior we picked apart in Fable 5's six-day life. It's an uncomfortable but fair reminder that refusals and silent degradation are part of a model's real-world score, not footnotes to it. A model that's brilliant on the questions it deigns to answer and obstinate on the rest is, for production purposes, less useful than its raw capability suggests.
The benchmark ships with a GitHub repo and instructions to run it yourself, which is the part that turns it from interesting reading into a tool. You don't have to take the aggregate ranking on faith. You can run it against the subset of those 55 domains that matches your own work and get a number grounded in your reality rather than the average across fields you'll never touch.
Arbor: persistent memory for autonomous research
The second release goes after a failure mode anyone who's run a long agentic job has hit. The agent tries an idea, it fails, the agent forgets what happened, and it starts over — losing the thread of the larger goal somewhere along the way. It can edit code, run a short test, do one specific task competently, and still be unable to hold the overall picture across a long run.
Arbor is an open harness for autonomous research built around an idea it calls hypothesis-tree refinement. It takes a research objective, breaks it into hypotheses, runs experiments, saves the evidence, and uses those results to decide what to do next. The effect is to turn research from a messy sequence of disconnected attempts into something cumulative, where each step builds on what came before instead of resetting.
The structure is worth understanding, because it generalizes well past research:
- A coordinator owns the overall strategy and decides which directions are worth pursuing.
- Executors test individual hypotheses in isolated instances. Think of them as temporary research assistants that each implement one experiment, run it, and report back.
- Every node in the tree retains its hypothesis, its experiment, and its result, so the system can compare competing directions and iterate rather than forgetting what it already tried.
The reported benchmarks have Arbor beating standard harnesses like Claude Code and Codex across optimizer design, architecture design, agentic coding, and math reasoning, and some of the gains are sizeable rather than marginal. It's out under Apache 2.0 with code available.
Build the eval you actually need
The most useful thing about a benchmark that ships its harness is that you don't have to treat the published ranking as the answer to your question. The published ranking answers a different question — how do these models do on average across 55 industries — and your business doesn't operate across 55 industries. It operates in one or two, with a specific stack and a specific set of recurring tasks. The aggregate score smears all of that into a single number that's probably wrong for you in both directions, crediting models for domains you'll never touch and penalizing them for ones that don't apply.
So use the harness as a starting kit rather than a verdict. Pull the tasks from the domains that match your work, and add your own. If your team lives in a particular framework, write a handful of tasks that exercise it the way you actually do — the gnarly refactor, the migration that touches twelve files, the bug that only reproduces under load. Capture what a good outcome looks like for each. Now you have an eval that reflects your reality, and the ranking it produces is one you can actually act on, because a win on it predicts a win in your codebase rather than a win on someone else's. This takes an afternoon, not a quarter, and it's the single highest-leverage thing most teams can do before committing to a model. The benchmark did the expensive part — building a harness that drives agents through real tool use. You just point it at your own work.
It also future-proofs the decision. Models turn over every few weeks now, as this very week demonstrated. An eval tied to your tasks lets you re-rank the field every time something new lands, in an afternoon, against the work you care about, instead of waiting for someone else to publish numbers on benchmarks that may or may not resemble your situation. That cadence — your own eval, re-run on every release — is what turns the dizzying pace of model launches from a source of anxiety into a routine you control.
The harness pattern is worth stealing
Arbor's specific domain is autonomous research, but the structure it uses solves a problem that shows up anywhere an agent has to run longer than a single context window comfortably holds. The naive approach stuffs everything into one ever-growing conversation until the agent is drowning in its own history, losing track of the goal somewhere around the point where the early context scrolls out of reach. Arbor's answer is to stop treating the run as one long monologue and start treating it as a tree of scoped, recorded attempts.
The mechanics are simple enough to borrow without adopting Arbor wholesale. A coordinator holds the high-level goal and the map of what's been tried. It spawns executors that each get one narrow job, a clean context, and instructions to report back a structured result rather than a transcript. The coordinator files each result against the hypothesis it tested and uses the accumulated picture to decide what to spawn next. Nothing here is exotic, and that's the point — it's a pattern, not a product, and you can implement a version of it on top of whatever agent framework you already run.
What it buys you is continuity without context bloat. Each executor stays sharp because its context is small and focused. The coordinator stays oriented because it's holding structured results rather than raw history. And because every attempt is recorded as a node, the system can compare two directions head to head instead of forgetting the first one the moment it starts the second. That combination — focused workers, an oriented coordinator, retained per-attempt state — is most of what separates an agent that genuinely makes progress over an hour from one that spins, repeats itself, and quietly drifts off the original task.
Why both belong on your radar
These two pair naturally, because they cover the two halves of the same maturity gap.
Agents' Last Exam is the measurement half. It finally evaluates agents on the multi-step, domain-specific, real-tool work that decides whether they're actually useful, and it counts gating and silent degradation against the score the way production does. Use it to pressure-test your model choice on the domains you operate in, and don't skip the detail that the top of its table is GPT-5.5 Codex — not necessarily where the marketing or your own defaults would have put it.
Arbor is the architecture half. Structured, persistent state is what separates an agent that can run for ten minutes from one that can run a long-horizon job without losing the plot, and a hypothesis tree is one well-thought-out answer to how you build that. The continuity problem it targets is the same one we keep running into when monitoring agent clusters at scale: the hard part of autonomy is maintaining coherent state and the big picture across many steps and many runs, not executing any single action well. Even if you never use Arbor directly, the pattern — a coordinator over isolated, reporting executors, with state retained per node — is worth borrowing into whatever harness you build on. It's the difference between an agent that researches and one that just keeps trying.
There's a broader read on the two together, beyond the practical use of either. They're both signs that the agent field is growing up, moving past the phase where a slick demo and a one-shot benchmark were enough to win attention. A benchmark built around surviving real multi-step workflows, and a harness built around holding state across a long run, are both responses to the same realization: the hard part was never getting an agent to do something impressive once. It was getting it to do useful work reliably, across the messy, multi-step, tool-heavy reality of an actual job, without losing the plot halfway through. That's a less glamorous problem than the demos suggested, and the fact that serious tooling is now aimed squarely at it is a healthier sign for the field than another model topping another leaderboard would be.
The best thing about both is that they're open and reproducible, so neither asks you to trust a chart. Run the benchmark against your own domains. Run the harness on your hardest long-horizon task. The published stories are a starting point; the only number that should change your roadmap is the one you generate on your own work. That's the through-line for the whole agent space right now. The capability is real and improving fast, but the gap between a benchmark win and a tool that survives your workflow is exactly the gap these two releases are trying to measure and close, and you find out where you stand by running them, not by reading about them.
Part of our coverage of an unusually dense week in AI. See also: Fable 5's six-day life, the open-weight surge, world models and robotics, the generative 3D/4D/video wave, and the week's open AI primitives.