Runs
Every run is stamped with the git SHA of the harness, the full config snapshot, and a content hash of the challenge set. Scores are only comparable between runs with the same challenge-set hash.
| Run | Started (UTC) | Git | Challenge set | Graded | Failures | Note |
|---|---|---|---|---|---|---|
| run 2 | 2026-07-17 15:31 | c4dd881 | 96e86e1711a49f7d | 56 | 16 | run 2: 4 local models, 9 challenges incl. 4 harder |
| run 1 | 2026-07-17 14:55 | de38c28 | 43d1f79070dee5ce | 8 | 2 | first live run: qwen2.5-coder:3b, 5 simple challenges |