alexshpunt/pi-agent-computer-use is a project that adds computer-use capability to the Pi coding agent. The latest release is v0.1.3. It reads native UI trees and screenshots and sends real input to control Windows apps and browsers. However, the provided information does not include specific details about what changed in v0.1.3.
The core of this project is a tool flow that reads the screen, sends input, and then checks the resulting screen again. It uses the native UI tree, in which screen elements are expressed structurally, along with accessibility information to find windows and targets, and clicks elements or coordinates to input. It then reads the new screen to verify the result of the manipulation. The repository contains a demonstration of tracing and drawing a picture in GIMP, records from several models, task instructions, and setup materials for reproduction. It is written in TypeScript and uses the MIT license, and the implementation is based on the Windows features of a Cua Driver fork.
What the materials show is limited to the implementation that connects screen information with input, plus the GIMP demonstration and the model records. Stability or performance across multiple apps cannot be judged from this single case and these records alone. So rather than a project that proves the performance of a finished automation tool, it is closer to a reference implementation for exploring how to connect GUI control to the Pi coding agent.