Every AI Coding Agent Is Blind to Your UI

Written by Danni Friedland on

Last updated:

aifrontenddeveloper-tools

Name any AI coding agent. Claude Code. Cursor. GitHub Copilot. Windsurf. They can all read your source files, trace your imports, and generate diffs that compile. For backend code, that is usually enough.

For frontend work, it is not even close.

What Your Agent Reads vs. What You See

Your source file says className="p-4 md:p-8 lg:p-12". Your browser renders 32px of padding at the current viewport width. Your agent has the source. You have the screen. Neither of you has both.

This gap exists for everything visual:

Computed styles. The source says text-primary. The browser resolves that through your theme tokens, CSS variables, media queries, and cascade rules to color: #1a56db. Your agent sees the class name. It does not know the color.

Component identity. Your page has forty div elements. You are looking at the third card in a grid. Your agent does not know which div you mean, because the DOM is a runtime artifact — it does not exist in source code. The agent would need to trace your component tree, resolve props, evaluate conditional rendering, and map the result to what is on screen. It cannot do any of that.

Responsive layout. You are on a 768px viewport. The hero section stacks vertically. The sidebar collapses into a hamburger menu. Your agent reads the same breakpoint classes you do, but it has no viewport. It does not know which branch of your responsive logic is active right now.

Visual hierarchy. Two elements overlap. One has z-index: 10 from a utility class; the other inherits z-index: auto from a parent. Your agent can grep for z-index values, but computing stacking context requires rendering — and rendering requires a browser.

The Cycle Everyone Recognizes

You tell your agent to fix the spacing on the hero section. It reads the file, picks a Tailwind class that looks right, and saves. You switch to the browser. Wrong element. You switch back, add more context — the exact file path, the line number, a hint about which div. The agent tries again. You check the browser again. Closer. One more round.

Three iterations and six tab switches to change a padding value. The agent had full access to the source code the entire time. It just could not see the page.

This is not a prompt engineering problem. You cannot solve it by adding more context to your instructions or switching to a different model. The information your agent needs — computed styles, resolved layout, component-to-DOM mapping, viewport state — does not exist in your source files. It exists only at runtime, in the browser.

The Runtime Context Gap

We call this the runtime context gap. It is the set of information that exists only when your application is running in a browser:

Every AI coding agent today operates without this information. They read files and infer what the UI probably looks like. For a div with three Tailwind classes, the inference is often right. For a component that renders differently based on props, viewport, theme, and application state — inference is a guess.

This Is an Architecture Problem

The important thing to understand: this is not about model intelligence. GPT-5 will not fix it. Claude’s next release will not fix it. A smarter model reading the same source files still cannot see computed styles, because computed styles do not exist in source files.

The gap is architectural. Today’s coding agents are file-first tools — they read files, edit files, and run terminal commands. That architecture works for backend code where the source file is the truth. It fails for frontend code where the truth is the rendered output.

Closing this gap requires agents that can access the browser runtime — the DOM, the computed styles, the component tree, the viewport. Not screenshots (which lose structure and interactivity). Not build output (which is transformed beyond recognition). The actual live browser state that your users see.

Until coding agents can see what users see, frontend AI assistance will remain a game of guess-and-check. The model is not the bottleneck. The information is.


We built Frontman to close this gap — an open-source agent that connects to your browser and your dev server simultaneously. But whether you use Frontman or something else, the runtime context gap is the problem worth understanding. It explains why your AI agent keeps getting CSS wrong.