evals
Run evals on Jetty
v2 · Sep 2026 · jettygrowthteam · auto-evolved runbook (weakest-axis targeting)

opencode · Gemini 3.8 Flash

opencode driving google/gemini-3.8-flash through OpenRouter, 1 hill-climb rounds from the v1 runbook.

agent opencode · model google/gemini-3.8-flash

Judge, best round
32.8/40
round v1
Self-score, best round
38.5/40
inflation +5.7
Rounds
1
~$0.50 est. spend
opencode · Gemini 3.8 Flash best round
best round v1 · 225e57cd ↗

Self-score vs judge, by axis

Best round v1, 0–10 per axis

agent self-scoreexternal judge

Judge: The pelican is strong: a long orange bill with a large gradient pouch, a white body, and dark wingtips gripping the handlebar. The scene is attractive, with sky, sun, road and a lighthouse. The bicycle has odd geometry: a strange curved rear fender and stay, a chainguard that obscures the crank, and a seat hidden under the body. The legs reach the pedals but look awkward and stiff.

Score by round

Out of 40. Each round starts from the previous round's best SVG.

self-scorejudge

Stopped after round 1. Two attempts failed on OpenRouter errors mid-run (8a7cebda: 502 provider_unavailable; 6a4f0590: 504 upstream idle timeout), so per the benchmark rule (two infrastructure failures) the lineage was dropped.

Every round

Click a drawing to open it in the head-to-head viewer. The outlined card is the agent's best round.

opencode · Gemini 3.8 Flash round v1
v1 · bestjudge 32.8 · self 38.5 225e57cd ↗runbook

Scores per round

RoundPelicanBicycleCompositionPolishSelf totalJudge totalJudge spreadMinutesTrajectory
v18.7self 9.57.7self 9.58self 108.5self 9.538.532.80.511.7225e57cd ↗

Want this kind of eval for your own agents?

Every run on this site is a Jetty runbook: versioned instructions, a pinned sandbox, and a trajectory you can inspect. Point Jetty at your task and get the same receipts.