Skip to content
Loop
Research

Agents on real screens

The research under Loop

The interesting question in synthetic research is no longer "can a model answer like a person." It's "can an agent act like one, on a real interface, against real screens, and produce behavioral data you can act on."

That is a different problem, and it is where the academic work has been moving.

The work

UXAgent simulates usability testing of web designs with LLM agents. Researchers generate a population of personas, and the agents interact with the actual target design through a browser connector, producing behavioral traces rather than survey answers. It was published at CHI, the main venue for the field (arXiv:2502.12561 · CHI 2025).

AgentA/B takes the next step and runs LLM agents against live websites, simulating user interaction behaviors to A/B test real pages without waiting on real traffic (arXiv:2504.09723).

SimUser does the equivalent for mobile: a user agent and an application agent interact, and the simulation emits behavioral data from the exchange.

Note what all three have in common. Not one of them is trying to generate a better interview transcript. They are trying to produce behavior, a trace of what an agent did, where it went, and where it failed, against a real product.

Why this is the right target

Two reasons, and the second matters more.

It's the question teams actually have. Nobody ships because a persona said the onboarding sounded nice. They ship because they believe people will get through it. Traces answer that question directly; transcripts approximate it.

It's checkable. A simulated interview answer cannot be falsified: it is fluent by construction, and you have no way to audit whether anything is behind it. A simulated session can be opened and watched. You can see the agent stall at step three. You can see it never scroll to the pricing. You can decide for yourself whether the behavior is plausible.

This is the difference between a system that asks for trust and a system that hands you evidence.

What Loop takes from it, and what it doesn't

We take the target: behavior on real screens, no SDK, no instrumentation, no live traffic required. Drop in a URL, a screen, or a prototype; shape the behavior; get heatmaps, funnels, ranked friction, and a replay behind every finding.

We don't take a claim of validity we haven't earned. The field is young. The papers above are promising, not settled, and there is credible work arguing that today's agents are not human-like enough for the inferences people want to draw from them, and we cover that honestly in Can you trust a synthetic user?.

So Loop is built the way an early, honest version of this should be built: every finding is anchored to a session you can replay and on-screen evidence you can check. Not because it makes a nice feature bullet, but because it is the only responsible way to ship a technology this new.

Loop is early, and that's exactly why we'd rather show you the sessions than the slide deck.

Sources

  • UXAgent: An LLM-Agent-Based Usability Testing Framework for Web Design. arXiv:2502.12561 · CHI 2025
  • UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents. arXiv:2504.09407
  • AgentA/B: Automated and Scalable Web A/B Testing with Interactive LLM Agents. arXiv:2504.09723