Alibaba's Qwen-CUA drives software from screenshots alone

Alibaba's Qwen team published Qwen-CUA on August 3, a native computer-use agent built on a 397B-A17B Qwen mixture-of-experts backbone. According to the 24-page technical report, the model observes only screenshots and acts through keyboard and mouse events. There are no DOM trees, no accessibility metadata, and no task-specific APIs. The team frames this minimal interface as one that transfers unchanged across browsers and desktop applications.

The interface: pixels in, clicks out

Most computer-use agents lean on page structure. They read a website's DOM, pull accessibility metadata, or call software-specific APIs, and fall back to pixels only when structure is unavailable. Qwen-CUA drops all of that. Its entire perception channel is a stream of screenshots, and its complete action space is keyboard and mouse events, with the full action schema laid out in the report's appendix.

Because the input is just images, the same model runs against the same protocol on a browser tab, a desktop application, or a legacy system that exposes nothing programmatic. The report describes the tradeoff of a human-facing interface: the state is partially observed and machine-unreadable, actions require pixel grounding, and errors compound over long workflows.

Keeping 20 screenshots in context

Long tasks inflate the context window, since every step adds an image. Qwen-CUA's scaffold keeps at most 20 active screenshots. When the visual history exceeds that budget, the boundary advances by 10 steps, and the screenshots behind it are replaced with a fixed textual placeholder. The reasoning and actions from those steps stay in the conversation, and the recent screenshots stay in their original visual form. The procedure is deterministic and needs no separate summarization model.

The design is cache-aware. Because the boundary advances only every 10 steps, steps 21 through 30 extend the same request prefix instead of rewriting it, so the KV cache is reused across calls. The same folding operator also does trajectory slicing during training. A complete episode is rendered as multiple context-bounded slices, each slice inherits the episode's terminal reward, and only the tokens the model itself generated contribute to the loss.

Training on a 100,000-vCPU fleet

The report describes the training infrastructure in detail. The team built a cloud rollout fleet on Alibaba Cloud ECS with access to nearly 100,000 vCPUs and tens of thousands of concurrent environments. The environment pool combines controllable mock web services with everyday, long-tail, and professional desktop applications.

On top of that, the team constructed approximately 40,000 verifiable tasks covering environment operation, simulated-user interaction, and long-horizon workflows assembled through what the report calls verifiable phase-state chaining, plus human trajectories from personalized desktops as complementary supervision.

Optimization uses executable outcome rewards, building on the environment and task design from the team's earlier CUA-Gym work. Soft Adaptive Policy Optimization, or SAPO, a method from the team as well, stabilizes updates over long reasoning-and-action sequences, while trajectory slicing preserves episode-level rewards across image-heavy histories. Training proceeds in iterations: each resulting model exposes unresolved queries and weak domains, which guide teacher rerolls, new human trajectories, and recalibrated RL tasks for the next run.

The numbers

All results below are the team's own, measured under what the report calls a native keyboard-and-mouse computer-use protocol. Qwen-CUA beats Qwen3.7 on the task metric of all eight benchmarks.

On OSWorld-Verified it reaches 86.2, against 73.3 for Qwen3.7, 78.7 for GPT-5.5, and 83.4 for Claude Opus 4.8. On OSWorld 2.0, binary and partial completion go from 2.5/22.5 with Qwen3.7 to 18.5/48.4. On RedTeamCUA, the adversarial benchmark, attack success against the agent drops from 36.6 to 16.4 relative to Qwen3.7, while task success improves.

Scaling the same recipe to a model with over one trillion total parameters yields Qwen-CUA-Max, which lifts OSWorld-Verified to 87.6 and OSWorld 2.0 to 21.2/53.3.

Where it deploys, where it stumbles

The report includes a browser deployment in which the agent operates the active page through screenshots and native mouse-and-keyboard actions while exposing its trajectory to the user. One end-to-end showcase buys and configures a cloud virtual machine through the browser, asking for user confirmation before consequential operations.

A section of the report deserves attention: the Bash experiments. Giving the model shell access consistently shortens trajectories, from 63.6 to 49.1 average turns on Qwen-CUA, but task performance also drops, from 58.7 to 55.1. The team reads this as an optimization gap in tool routing, not a limitation of the hybrid interface. The models have not yet learned when to switch between pixels and commands, and a policy that picks the right modality per operation could combine the generality of computer use with the speed of the command line.