ThunderAgent-org/ThunderAgent

Implements a program-aware scheduler that optimizes KV-cache reuse and reduces memory imbalance across distributed GPU nodes, achieving 1.5–3.6× throughput improvements on agentic workloads. Acts as a middleware layer between agent clients and vLLM/SGLang backends with OpenAI API compatibility, requiring only a `program_id` parameter addition. Manages tool-call lifecycle including Docker containers and remote APIs with real-time trajectory visualization for tokens, tool-use time, and per-program profiling.

284 stars.