evals
Run evals on Jetty
v1 · May 2026 · jettyio · auto-evolved runbook (weakest-axis targeting)

opencode · Gemini 3.5 Flash

Beach scene with helmet and a basket of fish. The most illustrated of the v1 birds.

agent opencode · model google/gemini-3.5-flash

Judge, best round
29.3/40
round v2
Self-score, best round
40/40
inflation +10.7
Rounds
10
Jetty trajectories
opencode · Gemini 3.5 Flash best round
best round v2 · 660d2a66 ↗

Self-score vs judge, by axis

Best round v2, 0–10 per axis

agent self-scoreexternal judge

Judge: A charming white pelican with a big orange pouch, helmet and goggles sits on the seat with its wing reaching the handlebars and a fish basket; the sunset boardwalk scene is well rendered. The bicycle frame is oddly tall and stretched, with pedals and legs floating awkwardly, a strangely placed rear light beam, and a basket that obstructs the beak.

Score by round

Out of 40. Each round starts from the previous round's best SVG.

self-scorejudge

Every round

Click a drawing to open it in the head-to-head viewer. The outlined card is the agent's best round.

opencode · Gemini 3.5 Flash round v1
v1judge 28.2 · self 39.5 67836d67 ↗runbook
opencode · Gemini 3.5 Flash round v2
v2 · bestjudge 29.3 · self 40 660d2a66 ↗runbook
opencode · Gemini 3.5 Flash round v3
v3judge 28.7 · self 39 5cd01e9e ↗runbook
opencode · Gemini 3.5 Flash round v4
v4judge 29 · self 40 b130c9c2 ↗runbook
opencode · Gemini 3.5 Flash round v5
v5judge 29 · self 38.6 d94ff2e1 ↗runbook
opencode · Gemini 3.5 Flash round v6
v6judge 30.2 · self 38.7 4a378be2 ↗runbook
opencode · Gemini 3.5 Flash round v7
v7judge 30.2 · self 40 e3059c87 ↗runbook
opencode · Gemini 3.5 Flash round v8
v8judge 31.5 · self 38 70898b5c ↗runbook
opencode · Gemini 3.5 Flash round v9
v9judge 31.5 · self 39 b56c0c03 ↗runbook
opencode · Gemini 3.5 Flash round v10
v10judge 26.8 · self 40 2a46de05 ↗runbook

Scores per round

RoundPelicanBicycleCompositionPolishSelf totalJudge totalJudge spreadMinutesTrajectory
v17.7self 105.8self 107self 9.57.7self 1039.528.21.5–67836d67 ↗
v28self 106.2self 107.5self 107.7self 104029.31–660d2a66 ↗
v37.8self 106self 9.57.3self 9.57.5self 103928.71–5cd01e9e ↗
v47.8self 106.5self 107.2self 107.5self 1040291–b130c9c2 ↗
v57.5self 9.66.7self 9.57.3self 9.87.5self 9.738.6291–d94ff2e1 ↗
v67.5self 9.77.3self 9.77.8self 9.67.5self 9.738.730.21–4a378be2 ↗
v77.5self 107.3self 107.8self 107.5self 104030.20.5–e3059c87 ↗
v88self 9.57.5self 9.58self 9.58self 9.53831.50–70898b5c ↗
v98self 9.87.7self 9.78.2self 9.77.7self 9.83931.51.5–b56c0c03 ↗
v107.3self 105.8self 106.8self 106.8self 104026.82–2a46de05 ↗

Want this kind of eval for your own agents?

Every run on this site is a Jetty runbook: versioned instructions, a pinned sandbox, and a trajectory you can inspect. Point Jetty at your task and get the same receipts.