evals
Run evals on Jetty
v1 · May 2026 · jettyio · auto-evolved runbook (weakest-axis targeting)

hermes · Fusion router

Hermes on OpenRouter's Fusion auto-router. A 3-round sweep that plateaued at 34/40.

agent hermes · model openrouter/fusion

Judge, best round
28.8/40
round v1
Self-score, best round
34/40
inflation +5.2
Rounds
3
Jetty trajectories
hermes · Fusion router best round
best round v1 · 2d0c10c4 ↗

Self-score vs judge, by axis

Best round v1, 0–10 per axis

agent self-scoreexternal judge

Judge: The pelican reads well, with an orange beak and throat pouch, a white body and a wing extended to the handlebar. Its feet reach the pedals and it sits over the seat area. The bicycle is simplified: there is no visible seat or clear seat post, the chainstay/seat tube geometry is odd, and the handlebars are hidden under the wing. The bird body blob also floats awkwardly over the frame.

Score by round

Out of 40. Each round starts from the previous round's best SVG.

self-scorejudge

Every round

Click a drawing to open it in the head-to-head viewer. The outlined card is the agent's best round.

hermes · Fusion router round v1
v1 · bestjudge 28.8 · self 34 2d0c10c4 ↗runbook
hermes · Fusion router round v2
v2judge 27.7 · self 34 b38f9569 ↗runbook
hermes · Fusion router round v3
v3judge 29 · self 34 cb138790 ↗runbook

Scores per round

RoundPelicanBicycleCompositionPolishSelf totalJudge totalJudge spreadMinutesTrajectory
v17.3self 86.7self 87.7self 107.2self 83428.82–2d0c10c4 ↗
v27.3self 96.5self 87self 86.8self 93427.71–b38f9569 ↗
v37.5self 87self 97.5self 97self 834290–cb138790 ↗

Want this kind of eval for your own agents?

Every run on this site is a Jetty runbook: versioned instructions, a pinned sandbox, and a trajectory you can inspect. Point Jetty at your task and get the same receipts.