A Challenge is the durable identity of a UI task. Every execution binds a Challenge Version: an immutable set of instructions, rules, starter files, assets, numbered requirements, and the capture profile. Once a version is frozen, its inputs cannot change; a correction becomes a new version, so older Runs stay readable against exactly the brief they were given. Starter and asset hashes are recorded and published so two Runs can be shown to have received the same bytes.

A Configuration is an immutable snapshot of the coding tool, the model identity it displayed, the settings it exposed, and the capabilities that participated. A Run binds one Challenge Version, one protocol version, one Configuration, and a repetition number. Each Run starts from a fresh copy of the same frozen starter in a new session, with no continuation from a competing Run and no external research; local preview and the provider’s own inference remain allowed. Human contribution is limited to operational approvals — no design direction and no code edits. If a tool version or an exposed setting changes, it becomes a new Configuration rather than an edit to the old one.

Tool defaults change silently with updates, so they never choose the model. Each Configuration fixes an exact model ID and a reasoning effort from the values the tool accepts, for example high. Both are passed to the tool when the session starts, and the model and effort the tool reports in its session log are checked against the Configuration. A mismatch is a technical failure: the attempt is discarded and repeated. A different effort, or a model ID with a context variant, is a different Configuration. Runs under workflow-subscription-v1 used the tool’s default reasoning settings and record only what the tool exposed.

Both tools get the same rules, launch prompt, and Challenge files; they do not get the same tools. The benchmark measures a workflow, so each tool keeps its native tools, and only what breaks a rule or brings in personal context is removed:

• Removed from both. Personal skills, rules, and instruction files, installed or account-synced plugins and connectors, and memory. Each Run starts under a dedicated benchmark OS account.
• Subagents are prohibited. The shared rules require the work to happen in one session, and a Run under v2 or v3 can only be allocated for a Configuration that records subagents_enabled: false. Claude Code runs without its agent and workflow tools; Codex CLI runs with multi-agent disabled.
• No web research and no image generation. The shared rules prohibit both. Under v2, Claude Code runs without its web search and web fetch tools, but its shell commands are not network-sandboxed. Codex CLI runs with web search and image generation disabled and without network access from its sandbox. Under v3 all three tools run in the same sandbox, Codex without its own (§4). It blocks the operator’s services on the machine but not the internet, so for every tool the ban rests on the shared rules and on the log review: after each Run, network commands and web or image tool calls in the session log are flagged for the operator to review.
• Violations are recorded, not hidden. A confirmed rule violation is recorded as a protocol deviation in the Revision manifest. It does not by itself make the Run invalid.

• No time cap. The agent works until it declares completion, or the operator records an interruption or abort. The operator-measured elapsed time is published as recorded.
• Technical failures are discarded and repeated. A crash, an authentication drop, or an outage is not a result. The attempt is archived locally and repeated under the same allocated Run, and the cumulative discarded-attempt count is published on the Run. A discarded attempt never becomes a public Revision and never counts as a second Run.
• Every buildable result is published. Missed requirements, rough layouts, and unfinished interactions are results, not failures. Retrying because the output looked weak is prohibited.
• Repairs are uncapped and public. A mechanical repair — fixing a build break without design direction — is preserved as its own linked Revision with its own prompt record and duration. Revision zero is frozen before any intervention and is preserved unchanged. Repairs are discoverable, but they are never counted as independent Runs.

Screenshots and checks are produced locally against the frozen compiled files through a static server with production-equivalent routing and security policy. Captures use a pinned Chromium at device pixel ratio 1, a fresh context per test group, en-US, UTC, light appearance, and reduced motion.

For each Revision the archive records the build outcome, page load and runtime errors, declared routes and fallback behaviour, responsive overflow, the requirement interactions that could be exercised, axe findings, keyboard observations, and security-policy compatibility. Each check is pass, fail, inconclusive, not_run, or error. Only pass means observed success; a broken selector is inconclusive until a human reviews it. Tests target semantic roles, accessible names, and visible state, so they do not reward one visual style. The operator may attach an attributed qualitative note; it is editorial commentary.