evals
Run evals on Jetty
v1 · May 2026 · jettyio · auto-evolved runbook (weakest-axis targeting)

hermes · Gemini 3.5 Flash

Same hermes agent, Sonnet swapped for Flash: +5 self-score points from the model alone.

agent hermes · model openrouter/google/gemini-3.5-flash

Judge, best round
28.2/40
round v9
Self-score, best round
39.5/40
inflation +11.3
Rounds
10
Jetty trajectories
hermes · Gemini 3.5 Flash best round
best round v9 · e6d4548b ↗

Self-score vs judge, by axis

Best round v9, 0–10 per axis

agent self-scoreexternal judge

Judge: The bicycle is well formed, with spoked wheels, chain, crank, fenders and a fish crate, set against an attractive seaside backdrop. The pelican's anatomy is confused: the huge orange pouch reads as a detached sheet, the head is tiny and tilted back, the body is a jumble of white blobs with an odd grey tail, and the feet only loosely meet the pedals.

Score by round

Out of 40. Each round starts from the previous round's best SVG.

self-scorejudge

Every round

Click a drawing to open it in the head-to-head viewer. The outlined card is the agent's best round.

hermes · Gemini 3.5 Flash round v1
v1judge 28.5 · self 39 c118fb80 ↗runbook
hermes · Gemini 3.5 Flash round v2
v2judge 28.2 · self 38 2128ffc1 ↗runbook
hermes · Gemini 3.5 Flash round v3
v3judge 26.5 · self 37 2405db86 ↗runbook
hermes · Gemini 3.5 Flash round v4
v4judge 28.7 · self 39 822a3bca ↗runbook
hermes · Gemini 3.5 Flash round v5
v5judge 30 · self 38.8 60502506 ↗runbook
hermes · Gemini 3.5 Flash round v6
v6judge 26.8 · self 38 2c2a74e8 ↗runbook
hermes · Gemini 3.5 Flash round v7
v7judge 27 · self 38.5 c165de92 ↗runbook
hermes · Gemini 3.5 Flash round v8
v8judge 26.3 · self 38 d391e018 ↗runbook
hermes · Gemini 3.5 Flash round v9
v9 · bestjudge 28.2 · self 39.5 e6d4548b ↗runbook
hermes · Gemini 3.5 Flash round v10
v10judge 26.5 · self 38.5 3e21bf05 ↗runbook

Scores per round

RoundPelicanBicycleCompositionPolishSelf totalJudge totalJudge spreadMinutesTrajectory
v17self 107.5self 9.56.8self 107.2self 9.53928.51–c118fb80 ↗
v27self 97.8self 106.3self 97self 103828.21–2128ffc1 ↗
v36.3self 9.57.3self 96.3self 9.56.5self 9.53726.50–2405db86 ↗
v47self 107.7self 106.8self 9.57.2self 9.53928.71.5–822a3bca ↗
v57self 9.58.3self 9.57.2self 107.5self 9.838.8300–60502506 ↗
v66.3self 9.57.5self 9.56.3self 9.56.7self 9.53826.80.5–2c2a74e8 ↗
v75.8self 9.87.8self 9.56.7self 9.56.7self 9.738.5272.5–c165de92 ↗
v85.5self 98self 106.2self 96.7self 103826.31–d391e018 ↗
v96self 9.58self 106.8self 107.3self 1039.528.22–e6d4548b ↗
v105.3self 9.88self 9.56.3self 9.56.8self 9.738.526.51–3e21bf05 ↗

Want this kind of eval for your own agents?

Every run on this site is a Jetty runbook: versioned instructions, a pinned sandbox, and a trajectory you can inspect. Point Jetty at your task and get the same receipts.