relari-ai/continuous-eval

""" pii_check = CustomMetric( name="pii_check", criteria=criteria, rubric=rubric, metric_type="discrete", # can be 'discrete' or 'continuous' ) result = pii_check(answer="My name is John.") print(result) ``` ## Features - Modularized evaluation (evaluate each pipeline module with tailored metrics) - Metric library with deterministic, semantic, and LLM-based metrics - Support for probabilistic evaluation - Isolation of Pipeline components - Support for custom metrics and tests - Distributed evaluation (using Ray) - Integration with OpenAI and other LLM providers - All major frameworks (LangChain, LlamaIndex, Ollama, VertexAI, etc.) - Comprehensive documentation with examples ##

516 stars. No commits in the last 6 months.