Papers
arxiv:2610.26828

Agent World Model in the Wild: Computer-Use Agent Environment Synthesis

Published on Oct 19
· Submitted by
Zhaoyang Wang
on Oct 22
Authors:
,
,
,
,
,
,
,
,

Abstract

WildAWM synthesizes controllable computer-use environments and tasks to train general agents via interaction, improving out-of-distribution performance across web, CLI, and GUI benchmarks.

Agent learning is shifting from learning from human data toward learning through interaction with the world. However, scaling such learning directly in real-world computer systems is challenging and can introduce substantial security risks. We therefore propose Agent World Model in the Wild (WildAWM), a scalable pipeline for synthesizing controllable environments and tasks for training general computer-use agents (CUAs). Whereas existing approaches typically specialize in a single agent interaction interface, we construct executable environments that jointly support graphical user interfaces, model context protocol services, and command-line interfaces, allowing agents to flexibly combine visual interaction, tool use, and code execution. WildAWM is entirely powered by open-source models, demonstrating that high-quality computer-use environments can be generated without proprietary frontier models. Beyond environments, we address three recurring challenges in task synthesis: ensuring solvability, maintaining sufficient difficulty, and producing reliable verifiers, by introducing execution-based quality gates that combine heterogeneous agent rollouts, cross-judge consistency checks, and counterfactual probes of judgment robustness. The resulting pipeline synthesizes 341 long-horizon environment-task pairs that can require over 100 agent steps to solve and remain challenging even for frontier models, with GPT-6-Astra solving only 60% of them. We train two Qwen models of different sizes exclusively in these synthetic worlds. Both models achieve substantial gains across 8 out-of-distribution benchmarks spanning web, claw, and computer-use agents, demonstrating that learning in WildAWM's synthesized worlds generalizes broadly to unseen computer-use tasks. The code is publicly available at https://github.com/MSR-Orchard/WildAWM.

Community

Paper author Paper submitter

We introduce WildAWM (Agent World Model in the Wild), a scalable pipeline that synthesizes executable computer-use agent (CUA) environments and tasks for training general CUA, powered entirely by open-source models.

Core contributions:

  1. Worlds: persona-driven, multi-day personal computers where websites, MCP services, and local files share one state, so agents can combine GUI, MCP, and CLI, and the whole pipeline is powered by open-source models, without any reliance on proprietary frontier models.
  2. Tasks: execution-based quality gates (heterogeneous agent rollouts, cross-judge consistency checks, and counterfactual probes) keep tasks solvable, challenging, and reliably verified. We release 341 environment–task pairs; some need over 100 agent steps, and GPT-6-Astra solves only about 60%.
  3. Results: trained exclusively in these synthetic worlds, Qwen3.6-35B-A3B and Qwen3.5-4B improve on all 8 out-of-distribution benchmarks spanning web, claw, and computer-use agents, by +10.58 and +8.21 points on average.

Paper: https://arxiv.org/abs/2610.26828
Project page: https://msr-orchard.github.io/WildAWM/
GitHub: https://github.com/MSR-Orchard/WildAWM
Data & models: https://huggingface.co/collections/ChilleD/agent-world-model-in-the-wild
Harbor dataset: https://hub.harborframework.com/datasets/wildawm

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2610.26828
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 2

Datasets citing this paper 1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.26828 in a Space README.md to link it from this page.

Collections including this paper 1