vllm-project/semantic-router

Implements signal-driven routing using semantic classification models to intelligently dispatch requests across heterogeneous model ensembles, capturing missing signals from requests, responses, and context. Built on a modular architecture with extensible LoRA-based classifiers and real-time hallucination detection, integrating with vLLM's production stack for optimized inference across cloud, datacenter, and edge deployments.

3,373 stars. Actively maintained with 159 commits in the last 30 days.