BlackRoad-Agents/agent-eval
agent eval — Part of the BlackRoad OS ecosystem. Sovereign computing, edge AI, mesh networking. blackroad.io
Provides automated evaluation of local AI agents running on Ollama, scoring responses against keyword-based test cases to measure accuracy, relevance, and latency across 20 distributed agent types (coder, researcher, mathematician, etc.). The harness ingests JSON test definitions, queries agents via HTTP, and generates scored reports with configurable model selection and custom test file support—designed for validating agent performance in the sovereign, edge-first BlackRoad ecosystem without external dependencies.
Stars
—
Forks
—
Language
Python
License
—
Category
Last pushed
Mar 26, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/agents/BlackRoad-Agents/agent-eval"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
BlackRoad-Forge/RoadCodexRunner
BlackRoad Forge — codex agent runner — Part of the BlackRoad OS ecosystem. Sovereign computing,...
BlackRoad-OS/blackroad-os-codex-infinity
Infinite code indexing system. Proprietary BlackRoad OS, Inc.
BlackRoad-OS/blackroad-os-codex
ARCHIVED: Canonical repo moved to BlackRoad-OS-Inc/blackroad-os-codex
BlackRoad-OS/blackroad-agent-os
ARCHIVED: Canonical repo moved to BlackRoad-OS-Inc/blackroad-agent-os
skrikx/SROS_V2_OSS
First public OSS release of SROS V2, the local-only, CLI-first sovereign profile of the broader...