Screen Vision Automation Computer Vision Tools

Tools that use computer vision to analyze screen content and automate user interactions (clicking, typing, gaming actions). Includes real-time visual analysis, OCR-based automation, and AI-driven input simulation. Does NOT include general image segmentation, obstacle detection for accessibility, or tools without screen/visual automation components.

There are 23 screen vision automation tools tracked. 1 score above 70 (verified tier). The highest-rated is MaaXYZ/MaaFramework at 71/100 with 3,445 stars. 1 of the top 10 are actively maintained.

Get all 23 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=computer-vision&subcategory=screen-vision-automation&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	MaaXYZ/MaaFramework 基于图像识别的自动化黑盒测试框架 \| An automation black-box testing framework based on image...	71	Verified	3,445	C++
2	stb-tester/stb-tester Automated Testing for Set-Top Boxes and Smart TVs	56	Established	190	Python
3	Villavu/Simba Simba is a program used to repeat certain (complicated) tasks. Typically...	55	Established	261	Pascal
4	xxreflextheone/AI-Aimbot Open source AI powered aim assist written in Python for all* games.	53	Established	147	Python
5	STMicroelectronics/meta-st-x-linux-ai OpenEmbedded meta layer to install AI frameworks and tools for the STM32MPU series	50	Established	63	Python
6	Nyx0ra/lol-aram-mayhem-hextech-helper 🎮 基于计算机视觉 (RapidOCR) 的 LOL 大乱斗海克斯助手。自动识别屏幕选项，实时推荐来自 Blitz.gg 的高胜率海克斯。 \| LOL...	40	Emerging	7	Python
7	ai-hpc/ai-hardware-engineer-roadmap From Kernel-Level Parallel Programming to Custom AI Inference Accelerator...	38	Emerging	37	Python
8	gabrimatic/eyra Real-time AI screen analysis from the terminal. Local inference, voice...	37	Emerging	3	Python
9	artryazanov/shorts-maker-gpu Shorts Maker generates vertical video clips from longer gameplay footage....	34	Emerging	8	Python
10	nicedreamzapp/nicedreamzapp Building AI tools and learning as I go: mobile computer vision, medical ML,...	33	Emerging	16	Swift
11	cflaviu/ai-devbox GPU-enabled C++ development stack based on NVIDIA DeepStream	30	Emerging	5	Shell
12	kwel1x/Auto_aim 🎯 Capture and analyze visuals in real-time using YOLO, TensorRT, and DXGI...	27	Experimental	9	C++
13	aurintex/pai-os Open-source AI wearable companion. Local-first multimodal perception (VLM &...	26	Experimental	4	Shell
14	levipereira/deepstream-sahi Native GStreamer plugins that integrate SAHI (Slicing Aided Hyper Inference)...	26	Experimental	6	C++
15	tzafon/lightcone Lightcone: SDK for computer use agents	26	Experimental	8	Python
16	ninja-otaku/Project_Aegis AI gaming companion — screen capture from a separate device, Claude vision analysis	26	Experimental	6	Python
17	karimm-ai/NiceShot_AI A Python tool powered by computer vision to analyze gameplay videos and...	25	Experimental	3	Python
18	PRITHIVSAKTHIUR/CUA-GUI-Operator CUA-GUI-Operator is an experimental, advanced computer-use agent (CUA) and...	25	Experimental	3	Python
19	gee-46/gee-46 🚀 AI & Data Science Engineer focused on Computer Vision, Machine Learning,...	18	Experimental	8	—
20	johsonx88888/Hachiware-Desktop-Pet An AI-powered desktop pet based on Hachiware,featuring computer vision...	18	Experimental	6	Python
21	lianhuaandy/Brain 🧠 Connect, create, and earn with BRAIN—your social network for paid posts,...	17	Experimental	3	HTML
22	aminethe01/open-typeless 🎤 Enable seamless voice input on macOS with push-to-talk functionality,...	17	Experimental	3	Shell
23	gabi123-cmd/eyes-ios 👁️ Detect obstacles in real-time using LiDAR technology, enhancing awareness...	17	Experimental	3	Swift