adianliusie/comparative-assessment
Framework for using LLMs to grade texts by using pairwise comparisons.
This project helps you automatically grade and rank different versions of generated text, like summaries or creative writing, without needing to manually score each one. Instead of assigning individual scores, it compares texts in pairs, mimicking how humans often judge quality more easily. The tool takes multiple text drafts and an attribute to assess (e.g., coherence, fluency) and outputs a ranking of which text is better for that attribute. This is ideal for content creators, marketers, or researchers who need to quickly evaluate and select the best AI-generated text.
No commits in the last 6 months.
Use this if you need an automated and efficient way to rank multiple AI-generated text outputs based on specific quality attributes, similar to how a human would compare them side-by-side.
Not ideal if you need a precise, absolute numerical score for each text rather than a relative ranking based on comparisons.
Stars
8
Forks
—
Language
Python
License
—
Category
Last pushed
Aug 29, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/adianliusie/comparative-assessment"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
google/langfun
OO for LLMs
tanaos/artifex
Small Language Model Inference, Fine-Tuning and Observability. No GPU, no labeled data needed.
preligens-lab/textnoisr
Adding random noise to a text dataset, and controlling very accurately the quality of the result
vulnerability-lookup/VulnTrain
A tool to generate datasets and models based on vulnerabilities descriptions from @Vulnerability-Lookup.
masakhane-io/masakhane-mt
Machine Translation for Africa