onehumanbeing/esp32-iphone-agent is a project that lets a vision model control a real device using a $7 ESP32-S3 board and an iPhone. In the setup the README currently describes, the board acts as an AirPlay receiver and USB HID input device with no computer involved, and the agent runs in an app on that same iPhone. The repository's latest release is listed as v0.1.0, but that tag and the legacy branch still hold the older version that required a computer.
The board does not decode the H.264 screen stream it receives over AirPlay itself; it forwards it over BLE to the phone app. The app interprets the video, sends the screen to an OpenAI-compatible vision model, and the board moves the AssistiveTouch pointer through USB mouse and keyboard input. This lets the app perform taps, swipes, drawing, typing, and key presses. The hardware baseline is the XIAO ESP32S3, and the project description says no dedicated computer, developer account, or jailbreak is required. It is an MIT-licensed C project.
The README also describes a feature that replays and exports execution logs step by step. That said, no quantitative benchmarks or testing evidence are presented, and the material available for judging stability is limited. The specific hardware setup for controlling a real device can therefore be confirmed, but the quality of its behavior across varied environments is hard to judge from this information alone.