Saneurai AI Benchmark · SB-1
Benchmarks are coming.
We are building our own AI benchmark: dynamic task suites generated fresh for every run, posed identically to every model, and graded by program — not by another model's opinion. One number per model, earned in the open.
What's on the board
Six suites. Zero vibes.
multi-step arithmetic
logic chains
instruction precision
passage recall
code generation
structured output
The discipline behind the number
- Dynamic, seeded tasks — instances are generated per run, so nothing can be memorized; within a run, every model faces the identical suite.
- Programmatic grading — answers are checked by code. No judge model, no partial credit.
- Published methodology — seeds, counts, and rules ship with every leaderboard, so each result can be reproduced — and criticized.
The first public leaderboard lands here when the current run cycle completes.