uptrain-ai/uptrain
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
Supports both API-based and local evaluation workflows with LLM-as-judge capabilities for custom checks beyond preconfigured metrics, enabling users to define domain-specific quality criteria. Built on a modular architecture that integrates with LangChain, LlamaIndex, and major LLM providers (OpenAI, Anthropic, Cohere) for seamless pipeline integration. Provides automated experiment tracking, comparative analysis across model variants, and data-driven recommendations through statistical analysis of evaluation results.
2,339 stars and 2,643 monthly downloads. No commits in the last 6 months. Available on PyPI.
Stars
2,339
Forks
202
Language
Python
License
Apache-2.0
Category
Last pushed
Aug 18, 2024
Monthly downloads
2,643
Commits (30d)
0
Dependencies
19
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/prompt-engineering/uptrain-ai/uptrain"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Related tools
microsoft/promptbench
A unified evaluation framework for large language models
microsoftarchive/promptbench
A unified evaluation framework for large language models
gabe-mousa/Apolien
AI Safety Evaluation Library
levitation-opensource/Manipulative-Expression-Recognition
MER is a software that identifies and highlights manipulative communication in text from human...
PromptMixerDev/prompt-mixer-app-ce
A desktop application for comparing outputs from different Large Language Models (LLMs).