Computer-Use Agents
Agents that perceive and operate graphical desktops or physical devices through screenshots, accessibility trees, HID, or remote-control interfaces; it excludes model-only vision inference and ordinary remote-desktop clients.
Subfields
Desktop Control Agents
Cross-platform agents and harnesses that click, type, navigate menus, or automate desktop applications; it excludes coding-only terminal agents and utilities that merely display agent output.
1 projectsDevice Control Agents
Protocols, firmware, bridges, and tools that let agents operate phones, tablets, speakers, cameras, or embedded boards; it excludes mobile-agent applications built primarily as end-user assistants.
2 projects| Project | What it does | Stars | Language | License | Latest release |
|---|---|---|---|---|---|
| zhongerxin/iphone-use | Provides a local MCP server that lets Codex automate and control real iPhone apps over USB through WebDriverAgent. | ★ 4.1K | Python | MIT | v0.3.15 |
| tongyi-mai/opengcua | A computer-use agent stack combining code execution, accessibility inspection, app APIs, shell access, and GUI control. | ★ 109 | HTML | Apache-2.0 | v0.1.0 |
| ghorbelhamdi/buddy-android | A floating Android chat-bubble app that lets Claude Code, Codex, and OpenCode use the phone. | ★ 70 | Java | Apache-2.0 | v2.1.13 |