dreadnode/AIRTBench-Code
Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models
Implements an autonomous red teaming agent using a modular harness architecture that executes Python code within isolated Docker containers to solve AI/ML CTF challenges. The agent integrates with the Dreadnode Strikes platform and uses the Rigging framework, receiving challenge notebooks and iteratively attempting exploits through a Jupyter kernel feedback loop with configurable step limits and timeout controls. Supports filtering for LLM-based challenges and provides standardized evaluation metrics for measuring adversarial capabilities across different language models.
Stars
93
Forks
14
Language
Jupyter Notebook
License
Apache-2.0
Category
Last pushed
Mar 11, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/agents/dreadnode/AIRTBench-Code"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Related agents
StonyBrookNLP/appworld
🌍 AppWorld: A Controllable World of Apps and People for Benchmarking Function Calling and...
qualifire-dev/rogue
AI Agent Evaluator & Red Team Platform
future-agi/ai-evaluation
Evaluation Framework for all your AI related Workflows
microsoft/WindowsAgentArena
Windows Agent Arena (WAA) 🪟 is a scalable OS platform for testing and benchmarking of...
agentscope-ai/OpenJudge
OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards