AI Tooling in Framework-Specific Development: An Empirical Study of MCP and Agent Skills Benefits

LLMs generate component library code that compiles, runs, and is wrong — and the compiler never catches it. That’s the central problem this study set out to measure: when Claude Sonnet 4.6 in VS Code Copilot agent mode builds an application against Ignite UI for Blazor or Ignite UI for Angular, how much does giving it Model Context Protocol (MCP) servers and Agent Skills actually change the outcome, compared to the same model working from training-data recall alone?

Two developers ran 7 paired test scenarios across 6 application types, once with MCP and Agent Skills enabled and once without, holding the model, IDE, and prompt constant in each pair. The result is unambiguous: AI tooling raised Ignite UI component compliance from 0/5 to 5/5 in every scenario where the untooled model ignored the library entirely, despite explicit prompts requiring it. In complex single-prompt builds, tooled sessions hit 100% feature completion versus 71% without tools. And when total session cost is measured — including the corrective iterations needed to fix what the untooled model got wrong — MCP sessions come out cost-competitive or cheaper.

The “Confidently Wrong” Problem

The most important qualitative finding isn’t a number, it’s a failure mode. Without authoritative context, the model doesn’t fail loudly — it produces syntactically valid code that compiles, partially renders, and looks correct, while shipping wrong component APIs, wrong CSS paths, broken content-projection slots, and zero UI library compliance. These are defects invisible to static analysis; they only surface at runtime or in visual QA.

Three concrete examples from the study:

Wrong CSS asset paths (Blazor BizDashboard test) — without MCP, the model referenced a non-existent IgniteUI.Blazor.Trial package path instead of the correct IgniteUI.Blazor theme paths, and missed that both a base theme and a grid-specific theme file are required.

Non-existent component properties (same test) — the model invented XAxisLabel on IgbCategoryChart and OutlineThickness on IgbRingSeries, neither of which exist on those components. Both compiled fine and threw runtime errors.

Missing content-projection slots (Angular Hotel App test) — without MCP, the model built an igx-dialog with plain <h2> content instead of the required igx-dialog-title / igxDialogContent / igxDialogActions slot directives, producing an NG0811 content-projection error at runtime. With MCP, the dialog was correct on the first pass.

What the Numbers Show

Component compliance was the starkest gap. In the three tests where the prompt explicitly required Ignite UI components but didn’t over-specify which ones (BravoBoard Mini base and update, and a deliberately minimal Travel Request prompt), the untooled model scored 0/5 every time — it quietly built the app with Chart.js, custom HTML, and plain CSS instead, while still passing 100% of the functional checklist. Checklist completion is not the same as requirement compliance: a model can score 36/36 on features while completely ignoring the mandated component library.

Feature completion told a related story in the hardest single-prompt scenario (Service Desk Mini): 27/38 features (71%) without MCP versus 38/38 (100%) with it. The missing features weren’t cosmetic — dashboard charts, ticket edit/delete/assign/status actions, delete confirmation, toast notifications, and consistent theming were all absent from the untooled build.

Time and cost varied more by scenario. The largest time win was 55% faster (15 vs. 33 minutes) on an alternate Service Desk Mini run, because MCP let the model read the Blazor Agent Skills and plan the full architecture upfront instead of iterating. On raw first-pass credits, the untooled Angular Hotel App run looked 41% cheaper — but once the ~197 credits spent correcting its wrong navigation component, broken dialog, and six raw icon-ligature strings are counted, MCP came out 8% cheaper overall. On the Blazor BizDashboard test, 45% of the untooled session’s credits went to corrective work versus 23% with MCP.

Context usage cuts both ways. MCP’s tool definitions add overhead, so multi-prompt sessions show a higher peak context with MCP (~330K vs. ~200K tokens in one test). But in single-prompt and refactor scenarios, MCP sessions ended with lower context than the untooled run, because they needed fewer corrective turns — and corrective turns, where the model has to re-read and diagnose its own broken output, are the most expensive context event there is.

When MCP Is Worth It

The study’s recommendations boil down to a simple decision guide:

• Strict library compliance required → use MCP. It’s the difference between 0/5 and 5/5.
• Complex single-prompt builds → use MCP. It closes a 29-point feature-completion gap.
• Refactoring plain code into a component library → use MCP. It’s faster and gets the architecture right the first time, not just visually similar.
• Minimal prompts that don’t name every component → use MCP. It autonomously discovers and applies the correct library instead of defaulting to plain HTML/CSS.
• A generic prototype where any UI is acceptable → either is fine; MCP adds overhead without proportional benefit here.
• Context budget is the sole hard constraint → proceed with caution, since MCP’s peak context is real.

The full paper below includes the complete methodology, per-test raw data, the specific prompts used for the minimal-prompt comparison, and links to the public benchmark repositories.