UKPLab/PeerQA

Constructed from peer review questions answered by paper authors, the dataset contains 579 QA pairs spanning document-level scientific texts averaging 12k tokens—designed to evaluate evidence retrieval, answerability classification, and long-context answer generation. The project integrates GROBID 0.8 for PDF-to-text extraction and establishes baselines using BM25 (Pyserini), dense retrievers, and cross-encoder rerankers, with decontextualization techniques shown to improve retrieval across architectures. Available on HuggingFace Datasets with evaluation scripts supporting transformer-based models including DeepSeek-R1 and Qwen variants.