agent-tui and agent-desktop
These tools are **competitors** offering distinct approaches to controlling applications via AI agents; one focuses on terminal-based applications using a TUI, while the other targets native desktop applications through OS accessibility trees.
About agent-tui
pproenca/agent-tui
TUI automation for AI agents. Control any terminal app from code.
Captures terminal output and sends keyboard/text input via isolated PTY sessions with full terminal emulation, enabling LLM agents to interact with interactive CLI tools. Exposes a daemon architecture with JSON-RPC over WebSocket for real-time monitoring, plus text/JSON output formats and wait conditions for screen state validation. Built in Rust following Clean Architecture patterns, supporting multiple transport mechanisms (Unix socket, WebSocket) for both local and remote agent integration.
About agent-desktop
lahfir/agent-desktop
Native desktop automation CLI for AI agents. Control any application through OS accessibility trees with structured JSON output and deterministic element refs.
Implemented as a single Rust binary with 50+ commands, it uses accessibility trees exclusively rather than screenshots, offering 15-step AX-first interaction chains that exhaust native APIs before falling back to mouse events. The snapshot + deterministic ref workflow (`@e1`, `@e2`) optimizes for LLM reasoning by enabling stable element selection across UI changes without re-querying the tree. Distributes via npm with zero runtime dependencies and supports macOS 13.0+ applications from Finder to Xcode through a JSON command/response protocol with machine-readable error codes.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work