aimpg scoreboard
Do token-savers, prompts, models and agents really save money on real code? Each result reruns a developer's own past commits both ways, and their own tests judge them. Made with aimpg verify; 8 records so far.
Report No. 1: what AI coding costs per solved task (984 pre-registered reruns of real commits) ยท aimpg in 5 minutes
Reproduced
Counted only when someone else reran a public record and got the same answer.
Nothing here yet: no claim has enough repos and people behind it.
Self-reported (not audited)
Nothing here yet: no claim has enough repos and people behind it.
Collecting data
claude-code@haiku vs claude-code
A number appears once at least 5 repos and 3 people back it, with no one supplying more than 50% of the runs. Each account counts for at most 3 repos per claim. Results use the median repo, never an average.
Disputed
- None.
Public records (rerunnable)
| Tier | Claim | Answer | By | Week | |
|---|---|---|---|---|---|
| self-reported | claude-code@haiku vs claude-code | SUPPORTED | kumarganduri | 2026-W42 | record |
| self-reported | claude-code@haiku vs claude-code | NOT SUPPORTED | kumarganduri | 2026-W42 | record |
| self-reported | claude-code@haiku vs claude-code | SUPPORTED | kumarganduri | 2026-W42 | record |
| self-reported | claude-code@haiku vs claude-code | SUPPORTED | kumarganduri | 2026-W42 | record |
| self-reported | claude-code@haiku vs claude-code | SUPPORTED | kumarganduri | 2026-W42 | record |
| self-reported | claude-code@haiku vs claude-code | SUPPORTED | kumarganduri | 2026-W42 | record |
| self-reported | claude-code@haiku vs claude-code | SUPPORTED | kumarganduri | 2026-W42 | record |
| self-reported | claude-code@haiku vs claude-code | SUPPORTED | kumarganduri | 2026-W42 | record |