the-crypt-keeper/can-ai-code

Parametric difficulty scaling generates unlimited unique problems across two dimensions (length and depth) rather than fixed test cases, measuring how far each model climbs a difficulty ramp instead of binary pass/fail. The framework evaluates three metrics—height (max difficulty achieved), efficiency (tokens consumed), and constrained performance (resource-limited capability)—revealing distinct cognitive fingerprints across model families (OpenAI's reasoning strength vs. Llama's balanced baseline). Benchmarks automatically escalate difficulty when models cluster near ceiling performance, maintaining discrimination power as capabilities advance while supporting any domain where difficulty can be parameterized.

602 stars. No commits in the last 6 months.