amazon-science/auto-cot

Automatically constructs diverse chain-of-thought demonstrations from LLM outputs rather than requiring manual prompt engineering, using a demo selection strategy that matches or exceeds hand-crafted baselines on reasoning tasks. The approach decouples demonstration construction from inference—first mining exemplars from zero-shot outputs, then using them as in-context examples for subsequent LLM queries. Targets GPT-3 and similar models on arithmetic and commonsense reasoning benchmarks, with PyTorch dependencies for any local processing components.

2,012 stars. No commits in the last 6 months.