Leaderboard

Mean blended score (0–100) per model and mode over the latest run. Failed attempts (transport errors, degenerate output, protocol violations) are excluded from scores and counted separately — never as zeros. Click any run for the full engineer report on every attempt, or see all runs and how scoring works.

🥇 qwen3-30b-a3b agentic

100

score

tests100%

deliverables100%

content checks100%

5/9

graded

4

failures

5290s

wall time

28.0k

tokens out

qwen3-30b-a3b oneshot

100

score

tests100%

deliverables100%

content checks100%

7/9

graded

2

failures

3022s

wall time

18.1k

tokens out

🥈 llama3.2-3b oneshot

91.4

score

tests86%

deliverables100%

content checks100%

7/9

graded

2

failures

53s

wall time

1.2k

tokens out

🥉 qwen2.5-coder-3b oneshot

87.2

score

tests79%

deliverables100%

content checks100%

4/9

graded

5

failures

71s

wall time

1.7k

tokens out

glm4-9b oneshot

87

score

tests78%

deliverables100%

content checks100%

6/9

graded

3

failures

127s

wall time

1.4k

tokens out

llama3.2-3b agentic

77.9

score

tests71%

deliverables89%

content checks89%

9/9

graded

0

failures

918s

wall time

16.4k

tokens out

qwen2.5-coder-3b agentic

49.8

score

tests46%

deliverables56%

content checks56%

9/9

graded

0

failures

1663s

wall time

39.6k

tokens out

glm4-9b agentic

49.4

score

tests30%

deliverables78%

content checks78%

9/9

graded

0

failures

3461s

wall time

38.3k

tokens out

Model	Mode	Score	Graded	Failures	Run
qwen3-30b-a3b	agentic	100.0	5/9	4	run 2
qwen3-30b-a3b	oneshot	100.0	7/9	2	run 2
llama3.2-3b	oneshot	91.4	7/9	2	run 2
qwen2.5-coder-3b	oneshot	87.2	4/9	5	run 2
glm4-9b	oneshot	87.0	6/9	3	run 2
llama3.2-3b	agentic	77.9	9/9	0	run 2
qwen2.5-coder-3b	agentic	49.8	9/9	0	run 2
glm4-9b	agentic	49.4	9/9	0	run 2