WebCoder: A 27B Frontend Agent That Outscores Claude Opus 4.8

A 27B open model, trained with reinforcement learning against the app it actually builds, now matches or beats Claude Opus 4.8 at multi-step frontend work. WebCoder-27B, from Tsinghua and Zhipu AI, scores 72.8 on requirement satisfaction in a new multi-round web dev benchmark, ahead of Opus 4.8 (65.1) and GPT-5.4 (59.6). On the existing Vision2Web benchmark, it lifts its Qwen3.8-27B base from 41.1 to 62.4, a near tie with Opus at 62.5.

• 72.8 vs. 65.1: requirement satisfaction on Webdev-Bench, WebCoder-27B vs. Claude Opus 4.8 (base model: 64.8)
• +21.3 points on Vision2Web (41.1 to 62.4), with delivery rate rising from 48.5% to 80.3%
• 90.8 on interaction checks, 5.8 points above the next best model; this is where the gains concentrate
• 9.8 points lost on Vision2Web without the paper’s progress-based credit assignment; 7.5 lost without context compaction
• 40 of 40 tasks delivered as a running app, vs. 38 for the base model

Reward the app, not the transcript

Frontend requests are rarely one task. “Copy this design, add a product section, hook it to the cart, don’t break the nav” mixes reproduction, generation, modification, and repair, and fixing one can break another. The authors train one policy to handle all four inside Webdev-World, a sandbox with code editing, a dev server, browser inspection (screenshots of the agent’s own running app), and on-demand “skills” for things like page decomposition and repair. Project state persists across the whole session.

The reward comes from the finished artifact. Each task’s requirements become a weighted checklist, and an independent verifier builds the project, runs it, and checks each item: deterministic tests where possible, pixel comparison for reproduction, and a vision-language judge (GLM-5V-Turbo) for subjective quality. If the build, entry point, or browser render fails, the reward is zero, no matter how good the code looks.

That last step is the paper’s main idea. A plain GRPO setup gives every token in a long rollout the same advantage, so an hour of good work followed by a regression looks identical to an hour of flailing. WebCoder splits each trajectory at its context-compaction points and measures how much checklist progress each segment made, using pages the agent itself fetched from its running app. It shifts credit between segments while keeping the rollout’s average advantage fixed, so the final verifier still sets the overall grade.

Long trajectories are the other problem. Rollouts get a context window of about 117K tokens, and the agent compacts its history as it goes. Training records these rewrites in a message tree, samples paths from it, and uses an ownership mask so any generated token that shows up in several samples counts toward the loss only once.

Results

Webdev-Bench, the authors’ new benchmark, has 40 composite tasks with 188 requirements and 1,125 hidden checklist items. Requirements are revealed round by round, and an acceptance agent operates the final app and grades every item. A requirement only counts if all of its items pass.

WebCoder also leads every subscore: visual 81.9, functional 73.8, and interaction 90.8. The interaction lead is the widest, which fits the training signal, since only a running app can show whether a filter narrows a list or a button blocks an invalid action. Model size alone doesn’t buy this: Qwen3.5-397B-A17B scores 28.7, under half of the 27B base model. The gains concentrate on text-to-project tasks (70.0 vs. 57.2 for the base model, and 60.0 for Opus) and screenshot replication (58.9 vs. 51.7).

On Vision2Web (66 multi-page sites rebuilt from screenshots, scored by the benchmark’s own grader), WebCoder’s functional score of 67.1 ties for best and its visual score of 57.7 is second only to Opus (59.1).

Both training tricks matter

The ablations train from the same base, data, and reward, changing one piece at a time:

Giving every segment the same advantage costs 9.8 points, and the visual score falls 11.0. Training only on the first 64K tokens instead of the compacted tree costs 7.5 points. Delivery rate drops too, from 80.3% to 66.7% and 72.7% respectively.

The training curves show why process credit helps. Early in stage 1, reward on replication tasks dipped, and the rollouts in that dip had 10x more redundant tool calls (0.24% to 2.45%) and used skills less often (24.1% to 15.7%). The agent was busy without making progress, and a single end-of-rollout score can’t tell those trajectories apart from productive ones.

Caveats

• Home benchmark. The headline 72.8 is on Webdev-Bench, built by the authors, with tasks and hidden checklists drafted with LLM help. On the independent Vision2Web benchmark, WebCoder ties Opus rather than beating it.
• Zhipu judges throughout. The training reward uses GLM-5V-Turbo, and Webdev-Bench’s simulated user and acceptance judge is GLM-5.3-Flash, all from the lab where most authors interned. A human check found broad agreement, but it isn’t a blowout: in 40 side-by-side comparisons, annotators preferred WebCoder over Opus 4.8 in 37.5% of tasks, tied 37.5%, and preferred Opus in 25%. Against its own base model, it won only 32.5%.
• Small base-model gap on the home benchmark. Qwen3.8-27B already scores 64.8, essentially tied with Opus. RL adds 8 points there, compared with 21 on Vision2Web.
• Lower delivery rate on Vision2Web. WebCoder ships a working site 80.3% of the time, against 98.5% for Opus 4.8 and 97.0% for GPT-5.4. Its average matches Opus because its delivered sites score higher.
• Not released yet. The model, benchmark, and training code are promised “upon acceptance.”

The idea that carries over is the reward design, more than this particular model. If your agent produces something you can execute, score the execution against a checklist, and reuse that checklist mid-trajectory to tell productive segments from busywork. The payoff was largest on interaction behavior, which code review alone can’t verify.