future-agi/ai-evaluation
Evaluation Framework for all your AI related Workflows
Supports 50+ built-in metrics (faithfulness, toxicity, hallucination detection, RAG quality scoring), LLM-as-Judge augmentation via Gemini/GPT/Claude, and guardrail scanners for jailbreak/injection/secrets detection in <10ms. Integrates distributed task backends (Celery, Ray, Temporal, Kubernetes), feedback loops via ChromaDB, and OpenTelemetry tracing for production observability.
Stars
84
Forks
29
Language
Python
License
GPL-3.0
Category
Last pushed
Mar 09, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/agents/future-agi/ai-evaluation"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Compare
Related agents
StonyBrookNLP/appworld
🌍 AppWorld: A Controllable World of Apps and People for Benchmarking Function Calling and...
qualifire-dev/rogue
AI Agent Evaluator & Red Team Platform
microsoft/WindowsAgentArena
Windows Agent Arena (WAA) 🪟 is a scalable OS platform for testing and benchmarking of...
agentscope-ai/OpenJudge
OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards
SparkBeyond/agentune
Tune your AI Agent to best meet its KPI with a cyclic process of analyze, improve and simulate