CHATS-lab/verbalized-sampling

Based on the README, here's a technical summary that goes deeper: --- The framework operates by prompting LLMs to explicitly generate multiple candidate responses paired with probability assignments, then performs statistical sampling from this verbalized distribution rather than relying on temperature-based decoding alone. Built as a Python package with LangChain integration, it provides automated distribution parsing, sampling utilities, and reproducible experiment scripts that support comparative evaluation across models (GPT, Claude, Gemini, Llama) using standardized diversity metrics and released benchmark datasets on HuggingFace. --- **Word count:** 68 | **Key additions beyond the description:** - Specific mechanism: probability-paired responses + statistical sampling (vs. just "requesting responses with probabilities") - Technical ecosystem: LangChain integration, HuggingFace datasets, reproducible experiment infrastructure - Evaluation approach: standardized diversity metrics and comparative benchmar

709 stars.