Researchers from Nanjing University, SenseTime, and partner academic institutions have confronted the primary bottleneck of the autonomous agent market: the data required to train models to manage software interfaces remains safely locked inside the walls of tech giants. This structural imbalance has historically left open-source models trailing far behind commercial alternatives. Instead of purchasing petabytes of raw logs, the research team opted to engineer a specialized simulation training ground.
Empirical data reveals a direct correlation between agent performance and the architectural diversity of training environments rather than the sheer volume of datasets. Keeping the total training data constant, expanding the number of applications within the training loop increased the success rate on unseen tasks from 60.7 to 67.5 percent. Under strict computational cost constraints, this approach fundamentally alters development economics: rather than mindlessly scaling model parameters, teams can invest directly in the granularity and diversity of simulations.
Synthetic Data and Real Performance Metrics
To populate the environment, the team recreated 35 high-demand applications in the Chinese market—including Taobao, JD, DiDi, and Meituan—alongside global utilities like Slack and Google Drive. Every simulation automatically resets to a clean state, while the agent interacts with the interface strictly through screenshots, mirroring actual user behavior without direct access to backend databases.
Environment diversity outweighs raw data volume: keeping the data scale identical, increasing the number of applications in the training loop boosted unseen task success from 60.7 to 67.5 percent.
On AndroidWorld tests, the larger model demonstrated a performance leap from 67.1 to 79.9 points, while its smaller counterpart rose from 54.3 to 74 points. By comparison, the preceding OpenMobile-1 iteration maxed out at 64.7 points on the same benchmark. On the more rigorous MobileWorld benchmark, which mandates prolonged scenarios spanning multiple application switches, the larger model elevated its score from 35.9 to 50.4 points.
Infrastructure Independence and Physical Hardware
The most telling metric emerged on physical hardware: running on actual Android devices, the senior model secured 59.6 points on SPA-Bench, setting a high-water mark for open-source solutions. Interestingly, applying classical reinforcement learning on top of the base run yielded marginal utility, with the larger model's AndroidWorld score dipping slightly from 80.8 to 79.9 points. This confirms that brute-force compute power alone cannot substitute for structured environment design.
The market now possesses a viable blueprint for building autonomous systems without paying a per-inference tax to proprietary APIs. Shifting focus from brute-force data hoarding to meticulous environment design rewires the economics of AI agent development, proving that open solutions can challenge closed ecosystems without requiring multi-billion-dollar infrastructure budgets.