Generated UIs cut agent conversations from 3.4 rounds to 1.2

When an agent needs four missing details, asking four questions in chat may be the wrong interface. A new system called GenUI-Harness retrieves the relevant records, generates a one-off React form for the unresolved choices, and then uses the submitted values to finish the task. In a reviewer study, that cut the average interaction from 3.4 dialogue rounds to 1.2.

• 3.4 to 1.2 dialogue rounds: generated UIs compressed several clarification turns into one structured submission on 100 selected tasks
• 58% Pass@3 from a 4B model: role-specific fine-tuning and audited RL raised Qwen3.5-4B from a 9.33% base-model result on UI-Tau Bench Lite
• 36 to 43% fewer agent turns among successful episodes than three text-based agent harnesses
• Higher Pass@3 on 7 of 9 backbones: GenUI-Harness beat smolagents by 4.48 percentage points on average
• The comparison is narrow: the dialogue study only included interfaces that rendered and supported valid interaction in all three runs

The interface sits in the middle of the agent loop

GenUI-Harness splits the work between two roles. First, a Tool Agent gets up to five read-only tool turns to inspect the user’s records and gather possible values. A GUI Coder then reads that evidence and writes a single Material UI component in React and TypeScript. It shows the records that matter, prefills known values, and gives the user controls for the choices the agent cannot infer.

The submission returns to the Tool Agent as structured JSON. The agent gets another five query turns if it needs to validate anything, then calls the action tool that updates the database.

Think of changing a flight booking. A chat agent might ask for the new flight, cabin, seat, and confirmation across several turns. This system can show grounded flight options, lock fields already fixed by the user’s account or request, and collect the remaining choices together. The UI is ephemeral. It exists for that task and can disappear after the action completes.

The paper evaluates this with UI-Tau Bench, a new benchmark built from 10 databases covering flights, hotels, e-commerce, and retail. Its Lite split has 300 tasks; the Full split has 1,000. Each task hides several action arguments from the original request, forcing the agent to recover them from records or ask the user. Success requires the final database state to exactly match the reference state.

A trained 4B UI coder beats much larger models

With off-the-shelf models, the harness improved Pass@3 over the smolagents baseline by an average of 4.48 percentage points. It led on seven of nine backbones, although the size of the gain varied widely. DeepSeek V4 Flash barely moved, from 49.33% to 49.67%, while the harness raised gpt-oss-120b from 30.00% to 47.67%.

The larger result came from training the two roles separately. Supervised fine-tuning raised the 4B system’s Avg@3 from 3.22% to 27.67%. The researchers then froze the Tool Agent and trained only the GUI Coder with GRPO. The final system reached 58.00% Pass@3 and 36.00% Avg@3, beating Claude Opus 5 in the same harness at 46.67% and 31.56%.

The reward setup is worth stealing. Rendering success alone cannot tell you whether a form submits the right payload, and an LLM judge can reward a polished interface with broken controls. The paper’s Reward Auditor runs selected interfaces through the full task, compares the judge score with the actual database outcome, and uses those mismatches to revise a shared scoring rubric. The final reward gives separate scores for extraction, structure, rendering, empty interfaces, quality, and accessibility instead of collapsing every failure to zero.

That mattered because the remaining errors were mostly interface errors. Across 800 failed sessions, 52.0% never produced a valid interaction and 44.5% reached the right tool with the wrong arguments or final state. Typical bugs were ordinary UI bugs: using a single-select control for a multi-select field or submitting a fixed cabin class that contradicted the user’s request.

Chat is a poor wire format for structured choices

The cleanest part of the paper is the separate communication study. For each of 100 tasks, three reviewers used the generated interface and three used a text rendering containing the same task state and options. The UI group averaged 1.2 rounds to completion; the text group needed 3.4. A round counted one user submission plus the agent response, so several clicks made before one form submission still counted as one round.

This is a useful product result even if you never train a GUI model. Agents often know the possible values and only need the user to choose among them. Once that state is structured, a form, comparison table, calendar, or map can carry more information than a stream of clarification messages. The agent stays responsible for retrieval and execution; the interface handles review and choice.

The improvement also transferred across agent models. Replacing only the UI-generation stage with the trained GUI Coder raised Avg@3 for all five tested models, from +2.44 points for Qwen3.6-35B-A3B to +8.77 for Claude Opus 5. That suggests the UI coder learned a reusable translation from agent traces to controls, rather than a private protocol tied to its original Tool Agent.

Caveats

• The human study used a selected subset. All 100 tasks had to produce a runnable interface and valid user interaction in all three sessions across four frontier models. The 3.4 to 1.2 comparison says what happens once the UI works, not how often arbitrary generated UIs work.
• The UI condition bundles many changes. The main benchmark compares complete harnesses with different loops and communication channels. The reviewer study holds the visible information fixed, but it still compares clicking a generated UI with typing against a text rendering of that UI.
• The benchmark user is an LLM simulator. It sees the generated source and the reference goal when constructing browser actions, which may make it more capable than a real user looking only at the rendered page.
• Exact database-state matching can undercount success. Semantically equivalent results can fail because of generated IDs, ordering, or free-text differences.
• Usability evidence is incomplete. Rendering and accessibility-tree checks can miss visual defects such as overlap, poor hierarchy, or confusing layout. Reviewers found live generation latency acceptable on 85 of the selected 100 tasks.
• This is version 1 of a paper released today. The benchmark, code, and results have not had time for independent replication.

The practical bet is simple: as agents take on stateful work, chat should become one interaction mode among several. If the agent already has rows, constraints, and candidate values in memory, flattening all of that into prose just so the user can translate it back into a decision is wasted motion.