← All agent-stack verdicts

Computer-use agents

Watch Confidence Low Group Tools and environment Evidence as of 2026-09-23

Written by VerdictsTools · Reviewed by ReviewTools on 2026-09-23 (separate AI agent sessions; review record)

Bracketed tags are evidence tiers from RULEBOOK.md §1: [Verified] means checked directly, [CI-observed] read from a project’s CI pages, [Maintainer claim] asserted by the project, [External] a dated third-party source, [Inference] our judgment. A gold path:line link opens the cited line. TRL is technology readiness level, scored 1 to 9. CI is continuous integration: automated checks that run on every push. To pin is to freeze a dependency, oracle, or assessment at an exact commit or version, so later changes cannot silently alter what was checked. The rider is the license clause withholding all rights, including benchmarking and testing, from OpenAI, Anthropic, their affiliates, and anyone acting for them; most FrankenSuite repos carry it.

Bottom line

Inference, low confidence: watch computer-use agents (software that operates a desktop from screenshots, keyboard and mouse) and do not commit a product to one agent stack yet. The agent stacks you would adopt are evidenced by activity and by benchmark scores their own READMEs report, not by test, CI or conformance evidence, and the category is mostly research code. If you experiment now, use the existing benchmark harness, grounding evaluation and VM infrastructure rather than building your own, and score the machine's end state, never the agent's own account of what it did.

Adopt, do not rebuild

License checked on 2026-09-23; non-permissive licenses are named in the text.

  • xlang-ai/OSWorld Apache-2.0: the desktop-agent benchmark of 369 tasks in real Ubuntu and Windows VMs, scoring VM state rather than chat; a private benchmark would cut you off from the numbers the field reports. [Verified] (ecosystem/pickup/_evidence/computer-use.md:12 NeurIPS 2024 benchmark of 369 tasks in a real Ubuntu/Windows desktop VM; ecosystem/pickup/_evidence/computer-use.md:12 Anthropic's Computer Use, OpenAI CUA, academic papers all report on it; ecosystem/pickup/_evidence/computer-use.md:22 State-based evaluation (score VM state, not chat))
  • likaixin2000/ScreenSpot-Pro-GUI-Grounding MIT: the grounding-accuracy harness (does the predicted click point fall inside the ground-truth box); use it to measure grounding instead of inventing a metric. [Verified] (ecosystem/pickup/_evidence/computer-use.md:28 Grounding accuracy eval script (ScreenSpot protocol); ecosystem/pickup/_evidence/computer-use.md:42 is the verified grounding-accuracy benchmark harness)
  • microsoft/OmniParser CC-BY-4.0: the grounding layer that turns a screenshot into located UI elements. License: CC-BY-4.0, a content license with an attribution requirement, unusual for code (the README badge says MIT); the older Ultralytics-based icon-detector weights keep their AGPL license, so pick the current weights and read the terms before shipping it inside a product. [Verified] (ecosystem/pickup/_evidence/computer-use.md:9 parse screenshot into structured UI elements; stack/licenses.tsv:56 Earlier Ultralytics-based icon detectors retain their original AGPL license; fresh read 2026-09-23, gh api repos/microsoft/OmniParser --jq .license.spdx_id)
  • trycua/cua MIT AND AGPL-3.0: cross-OS VM infrastructure for computer-use training, evaluation and data generation, with a benchmark library built and tested in CI; the VM fleet you would otherwise stand up yourself. License: MIT at the root, but libs/python/som is AGPL-3.0, so running a modified copy of that library in a network service triggers source release. [Verified] (ecosystem/pickup/_evidence/computer-use.md:8 Cross-OS (macOS/Windows/Linux/Android) VM-based computer-use infra; ecosystem/pickup/_evidence/computer-use.md:26 Benchmark harness as a first-class library with CI; stack/licenses.tsv:94 libs/python/som/LICENSE is GNU AFFERO GENERAL PUBLIC LICENSE v3)

Copy these practices

Build only if

  • A Windows-first project may have to invest in harness work itself: the Windows benchmark shows thinner maintenance than OSWorld and one of its container pointers failed verification. That is harness investment, not a reason to build an agent stack. [Verified] (ecosystem/pickup/_evidence/computer-use.md:39 WindowsAgentArena is the thinner half.; ecosystem/pickup/_evidence/computer-use.md:41 WindowsAgentArena tree listing failed)
  • Otherwise no hard constraint is evidenced; a new desktop benchmark or VM fleet would duplicate OSWorld and cua. [Inference] (ecosystem/pickup/_evidence/computer-use.md:12 NeurIPS 2024 benchmark of 369 tasks in a real Ubuntu/Windows desktop VM; ecosystem/pickup/_evidence/computer-use.md:8 Cross-OS (macOS/Windows/Linux/Android) VM-based computer-use infra)
  • License gives no reason to build an agent stack: a fresh read on 2026-09-23 (gh api repos/<owner>/<repo> --jq .license.spdx_id) returned Apache-2.0 for OSWorld and MIT for ScreenSpot-Pro and cua. cua's som library is AGPL-3.0, which a closed network service must avoid or replace, and OmniParser's CC-BY-4.0 asks for attribution; both are reasons to read terms or swap one component, not to rebuild the VM fleet or the parser. [Inference] (stack/licenses.tsv:94 libs/python/som/LICENSE is GNU AFFERO GENERAL PUBLIC LICENSE v3; stack/METHOD.md:79 Check the license of everything you tell a builder to adopt)

Where FrankenSuite touches this

  • No FrankenSuite packet builds a screenshot-driven desktop agent. frankenterm controls fleets of coding agents through terminal panes (Robot Mode API, MCP surface, policy-gated actions), a text channel rather than pixels, for terminal work. TRL 7, Explore; license: MIT with a rider barring OpenAI, Anthropic and anyone acting for them. [Verified] (packets/frankenterm-assessment.md:9 a WezTerm-derived terminal runtime (absorbed in-tree, September 2026) plus an agent-swarm control plane; packets/frankenterm-assessment.md:17 License (MIT + OpenAI/Anthropic rider, read verbatim); synthesis/00-overview.md:102 frankenterm | 7 | Explore | Rider)
  • franken_remote is a remote-workstation daemon and client with a screen capture and media pipeline, the kind of substrate a desktop agent would drive; it has no complete application and no hardware-qualified capture path yet. TRL 3–4, Explore; license: MIT with a rider barring OpenAI, Anthropic and anyone acting for them. [Maintainer claim] (packets/franken_remote-assessment.md:19 A Tailscale-identity-based remote workstation; packets/franken_remote-assessment.md:19 no hardware-qualified capture/codec path; packets/franken_remote-assessment.md:21 the license rider names OpenAI and Anthropic as Restricted Parties; synthesis/00-overview.md:81 franken_remote | 3–4 | Explore | Rider)

What we cannot say

Revisit when

  • An independent party reproduces an agent stack's OSWorld or OSWorld-V2 score from a pinned harness commit.
  • A computer-use agent product publishes its test and CI evidence, or is deployed by someone other than its maker.
  • WindowsAgentArena resumes maintenance or is archived; revisit six months after 2026-09-23 either way.

Sources

Starting a build here?

Begin from the pickup companion for Computer-use agents: it carries the charter seed, the oracle candidates and how to verify them, and the first claims to register, so a new project starts from evidence instead of from scratch.

Help fix it

Each opens a GitHub issue form with this page’s address filled in. You need a GitHub account.

Made by Joshua Nowak (ZestStream) with AI coding agents. Not affiliated with or funded by Jeffrey Emanuel; Joshua uses his public tools and is a paying subscriber to jeffreys-skills.md.

Source on GitHub MIT licensed