algorithmicsuperintelligence/optillm

Implements 20+ inference-time optimization techniques—including MARS, CePO, chain-of-thought reflection, and Monte Carlo tree search—that layer multiple reasoning strategies to achieve 2-10x accuracy gains on math and coding tasks. Acts as an OpenAI API-compatible proxy that intercepts requests and automatically applies selected techniques based on model prefix (e.g., `moa-gpt-4o-mini`), requiring no model retraining or client-side changes. Supports 100+ models across OpenAI, Anthropic, Google, and other providers via LiteLLM, with multi-variant Docker images for full, proxy-only, or offline deployment scenarios.

3,377 stars. Actively maintained with 6 commits in the last 30 days.