github.devotd.
NewEvals

mll-lab-nu/embodied-first-exam — a code-agent exam built on fixed scenes

Introduces an exam system in which a code agent solves robotic tasks drawn from several benchmarks as programs and is then verified in a separate sandbox.

Repositorymll-lab-nu/embodied-first-exam ↗

Introduces an exam system in which a code agent solves robotic tasks drawn from several benchmarks as programs and is then verified in a separate sandbox.

GitHub description: The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.

stars
★ 91
Forks
3
Language
Python
License
MIT
Read original source ↗View on GitHub ↗

mll-lab-nu/embodied-first-exam is an exam system where a code agent solves robot task scenes taken from several benchmarks as programs. For each task, you submit a single program as the answer, and the code you write is executed and verified in a separate sandbox. Harbor is used as the exam's execution environment. The repository is written in Python, the license is MIT, and it has 83 GitHub stars. No official version release is registered.

The README introduces the project's own nine exam sets together with several public benchmarks, and the repository description claims it is the first code-agent exam dealing with the physical world. It is a setup that fixes the exam scenes and inputs so that an agent's performance can be evaluated. The distinctive point of this setup is that it addresses the separation between the actual robot simulation and the result verification. This repository's freshness score is 9.0 out of 10, and its quality score is 8.0.

However, running the exam requires a Linux environment and a container configuration, and individual tasks may require a GPU and a large amount of storage space. The README writes out the execution flow and the required environment conditions together. The repository was created on September 18, 2026 and has no releases, and its momentum score is 2.2 out of 10. Because no exam results or performance figures have been published, the actual performance is hard to judge from this material alone. For now, it is better to treat this as a reference for examining the exam setup and the task sets themselves.