AISmithLab/HumanStudy-Bench

Combines an Execution Engine that reconstructs full experimental protocols from published studies with standardized evaluation metrics (Probability Alignment Score, Effect Consistency Score) to measure whether LLM agents reach identical scientific conclusions as human participants. Supports modular agent design through customizable persona and prompt presets, enabling systematic comparison of configuration choices independent of base model capabilities. Includes 12 foundational studies spanning cognition and social psychology with over 6,000 trials, plus automated tooling to add new studies from research PDFs.