Free AI Model for Ignite UI Code: 32-Test Benchmark

We ran 32 identical tasks across a paid model (claude-sonnet-4-6) and a free model (big-pickle, available via OpenRouter) — Angular, Blazor, React, and Web Components. Both finished at 94% success rate. The free model didn’t get there because it secretly knows Ignite UI. It got there because MCP servers and skill files gave it exactly what it needed to know.

TL;DR

• Ignite UI MCP servers and skill files are what make free models viable — they ground the agent in accurate component APIs and prevent hallucination loops.
• With that context in place, both models achieved 94% success (15/16) on the same task.
• big-pickle costs $0.00. Sonnet averages ~$1.44 per run.
• big-pickle is faster in most configurations — 4–13 min vs 7–22 min for Sonnet.
• Web Components is the exception: big-pickle needs MCP enabled to pass reliably (2/2 with MCP, 1/2 without). Sonnet goes 4/4 regardless.
• Best overall config for free model: Ignite UI MCP servers + skills on for Web Components; skills on, no MCPs for Angular, Blazor, React.
• The failure modes are different — Sonnet times out from scope creep; big-pickle stalls silently when it lacks icon catalogue context. Both are mitigable.
• The economics only tilt further toward free over time: per-task token consumption is climbing (longer agentic runs, bigger contexts, live MCP lookups), and the MCP overhead that’s free on a free model is exactly what inflates a paid bill. Today’s ~$1.44/run feels like a rounding error — at team scale, it isn’t.

Why this benchmark exists

Getting good results from AI at low cost matters. But cost isn’t just about picking a free model: MCP servers and skill files inject context that consumes tokens, and more tokens mean higher cost on paid models. We wanted to measure whether that overhead actually pays off.

Testing AI tooling reliably is harder than it looks. Starting a fresh chat session does not fully reset the environment: previous interactions can influence model behavior in ways that are hard to isolate. The phenomenon is explored in more depth in this post on why AI results improve over time without the model actually learning. To get around that, we built the Ignite UI Testbed: every run executes in a new, ephemeral container with a genuinely clean state, no carryover between tests, no bias from prior sessions.

We ran the same task across two models — claude-sonnet-4-6 (paid) and big-pickle (free, via OpenRouter) — covering Angular, Blazor, React, and Web Components. The task: build a two-page authentication UI using Ignite UI components. The kind of thing a developer hands to an AI agent and expects to work.

Ignite UI has a specific, versioned component API. Without guidance, any model can generate attribute names or icon identifiers that don’t exist. We saw this with another free model that failed 4 out of 16 runs due to hallucinated API names. The real question was simpler: can the Ignite UI AI tools bring a free model up to the standard of a paid one?

Each run varied two settings: whether MCP servers (live Ignite UI documentation servers) were on, and whether skill files (local instruction files covering component usage patterns) were injected. Four configurations per framework, 16 runs per model, 32 total.

The prompt and test setup

Every run used the same prompt: build a two-page authentication UI with a full login form and a full register form, using Ignite UI components — inputs, checkboxes, radio buttons, switches, selects, multi-combo, buttons, and icons. The prompt required validation states, helper text, social login buttons, password toggle, remember-me, and responsive layout, and explicitly asked the agent not to use plain HTML controls when an Ignite UI component was available.

Configuration matrix

Four configurations were tested per framework per model:

MCP servers give the agent live access to Ignite UI documentation, valid attribute names, theming APIs, and — critically — the exact icons available in @igniteui/material-icons-extended. Instead of guessing whether person_add is a valid icon name, the agent can look it up.

Skill files are short instruction documents that tell the agent how to use Ignite UI in a specific framework: which patterns to follow, which imports to use, what to avoid. They don’t enumerate every possible value (that’s the MCP’s job), but they point the model in the right direction from the start.

Together, these two tools cover the gaps that trip up smaller models on niche component libraries.

Overall results

Both models tied at 94%, but the more important story is how the free model got there. big-pickle’s only failure — a silent 21-minute hang on Web Components — happened in the one configuration with the least context: skills only, no MCP. Every run that had either MCP servers or skill files (or both) passed. Pull big-pickle’s one under-configured run and it goes 15/15.

Sonnet’s failure was different: a scope-creep timeout on React, caused by the agent exploring too many options rather than missing knowledge.

Results by framework

Angular — both models perfect

Angular went 8/8 across both models — every configuration passed. What stands out is how much the skill file helps: big-pickle’s skills-on, no-MCP run finished in 8 minutes at 32,800 tokens, the most efficient run in the entire 32-run matrix. The skill file gave the agent enough Angular-specific orientation that it never needed to query an MCP server. Sonnet’s equivalent run cost $0.84 and took 16 minutes for the same outcome.

Verdict for Angular: Use big-pickle with the Ignite UI skill file, MCPs off. The skill file alone is sufficient, and it produces the fastest, lowest-token result at zero cost.

Blazor — both models perfect, with a speed caveat

Blazor shows the skill file’s value most clearly. Sonnet’s no-MCP + no-skills run burned 211,992 tokens (the highest in its matrix) because without any grounding the agent had to explore .NET-specific component attribute names through trial and error. Add a skill file and that drops to 77,226 tokens in 7 minutes. The skill file doesn’t just help free models; it makes the paid model materially cheaper too.

big-pickle’s best Blazor run — 4 minutes with the skill file, no MCPs — was the fastest successful run in the entire 32-run matrix. Both models finish Blazor cleanly under 15 minutes when given proper context.

Verdict for Blazor: big-pickle with the Ignite UI skill file, no MCPs. Four minutes, free, and the skill file stops the API-attribute guessing that drives slow, token-heavy runs in both models.

React — the surprising inversion

React has the most counterintuitive result in the dataset: the paid model failed here and the free model went 4/4. Sonnet’s failure was not a knowledge gap: logs showed the agent read skill files correctly and scaffolded the right components. The problem was scope exploration without live MCP feedback to anchor decisions. React has more decision paths than Angular or Blazor, and Sonnet kept weighing alternatives until the 900-second budget ran out.

big-pickle completed all four React configurations cleanly, finishing the no-context run in 7 minutes at 46,000 tokens. Whether it made better decisions or just committed to the first reasonable path faster, we can’t say from the data alone. All four runs produced working output well within budget.

Verdict for React: big-pickle with either MCP or skill context is reliable and free. Sonnet requires the Ignite UI MCP server to stay on budget.

Web Components — free model works, but only with MCP enabled

Web Components is the one framework where context tooling isn’t optional for the free model. big-pickle’s one failure was a silent hang: 0 tokens in 21 minutes, no output, no error. The configuration was skills-on with no MCP. The likely cause is specific: Web Components relies heavily on icon names from @igniteui/material-icons-extended, and the skill file doesn’t enumerate the full icon catalogue. Without an MCP server to look up valid names on demand, the agent hit an unanswerable lookup early and stalled.

Why the MCP server is required for Web Components:

Skills injected, but no MCPs

▶ Context
  ✓ Component hints injected
  ✗ Icon list unavailable

▶ Dependency
  @igniteui/material-icons-extended
  └─ Exact icon names required

▶ Failure
  No MCP available to resolve icon names.

▶ Outcome
  ✗ Agent stalls
  ✗ 0 tokens generated
  ✗ 21 minutes wasted
  ✗ No output

Enable the Ignite UI MCP server and the picture changes: both big-pickle Web Components runs with MCP enabled passed. The MCP server resolved icon names on demand and kept the agent moving. The failure wasn’t a model capability problem. It was a missing tool problem.

Sonnet goes 4/4 because it can navigate this gap without live tooling, falling back on broader training knowledge to make reasonable icon choices. That’s a real capability advantage, but it doesn’t mean the free model can’t handle Web Components. It means you need to give it the Ignite UI MCP server.

Verdict for Web Components: Enable the Ignite UI MCP server when using big-pickle — this is the one configuration where it’s genuinely required. With MCP on, big-pickle passed 2/2. Sonnet is more forgiving if you’d rather not manage that dependency.

What the numbers tell us about context tooling

Skills alone handle most frameworks well

For Angular, Blazor, and React, the skill file alone consistently produced the lowest token counts and fastest completion times for both models. big-pickle’s best Angular run (32,800 tokens, 8 minutes) and best Blazor run (55,100 tokens, 4 minutes) both used skills only. Skill files add context upfront without the back-and-forth of live MCP queries. For these frameworks, that’s enough.

MCP servers are required when the skill file has gaps

Web Components exposed the limit of skill files: they orient the agent on usage patterns but don’t enumerate every valid icon name. The Ignite UI MCP server fills that gap with on-demand lookup of valid identifiers from @igniteui/material-icons-extended. For big-pickle, having MCP available was the difference between passing and a silent 21-minute hang. MCP runs do use more tokens. For Sonnet that adds cost; for a free model it doesn’t matter.

Context tooling helps both models, not just free ones

Skill files and MCP servers aren’t a special fix for weaker models. Sonnet’s token count dropped sharply with context too: from 211,992 tokens (no context, Blazor) to 77,226 (skills on, no MCPs, Blazor). The free model benefits more visibly because it has less fallback knowledge, but the efficiency gains apply to both. Good tooling makes the paid model cheaper and the free model viable.

Failure modes: different problems, same solution

Both models failed once. Each failure traces back to missing context — and in each case, enabling the right Ignite UI tooling would have prevented it.

Sonnet — scope creep timeout (React)

• ✓ Partial output produced — the agent scaffolded components before timing out
• ✓ Cause is diagnosable — logs show what the agent was doing
• ✓ Clear mitigation — Ignite UI MCP server for React anchors component choices and prevents path exploration
• ✗ 900s agent time limit consumed — wall time including scaffold and overhead was 19m
• ✗ Cost uncertainty — partial runs may still incur token cost
• · Root cause: React has many decision paths; without live MCP validation the agent explored rather than committed

big-pickle — silent hang (Web Components)

• ✓ Zero cost on failure — 0 tokens means 0 billing at any tier
• ✓ Isolated to one config — WC + skills-only + no MCP
• ✓ Clear mitigation — always use MCP with WC on big-pickle
• ✗ 0 output produced — no partial work to build on
• ✗ 21 minutes wasted — no early failure signal
• · Failure pattern: unanswerable lookup (icon names) caused the agent to stall silently

Both failures trace to the same gap: the agent hit a lookup it couldn’t resolve without live Ignite UI context. If a run produces 0 tokens after 5 minutes, cancel and retry with MCP enabled. Still running at 15 minutes without a working build? Something is wrong with the configuration.

A note on what “pass” actually means

A passing run produced a working, launchable app with correct Ignite UI imports, valid attribute names, and the right component set on both pages — no plain HTML controls, no hallucinated API names. Skill files and MCP servers exist specifically to prevent the latter.

The cost you’re not feeling yet

Right now, ~$1.44 per run reads like a rounding error. That’s the trap. Two curves are bending at once, and both point the same way.

Per-task token consumption is climbing faster than per-token price is falling. Frontier model prices have broadly trended down over the last two years. Total spend hasn’t followed, because agentic workflows consume far more tokens per task: longer autonomous runs, larger context windows, retries, and live documentation lookups. In this benchmark, enabling the MCP server pushed a single big-pickle Web Components run to 184,600 tokens — roughly 3x its leanest passing run (65,300). Now multiply rising per-task volume across every developer and every run on a team. The bill compounds in the direction you’re not watching.

The MCP overhead that’s free on a free model is exactly what inflates a paid bill. This is the sharp edge of our own data: “MCP runs do use more tokens. For Sonnet that adds cost; for a free model it doesn’t matter.” Web Components already shows that context tooling isn’t optional — it’s required for reliability. As that becomes true across more of the stack, the asymmetry widens: the more context you inject to make agents dependable, the more a per-token model charges you for reliability a free model gives away.

Meanwhile, the free side is improving, not standing still. (This part is outlook, not measurement — read it as a bet, not a result.) A stealth free model already matched a frontier paid model at 94% here. Its single failure was a missing tool, not a missing capability — the kind of gap that closes with better tooling today and better base models tomorrow. Free-tier and open models have been narrowing the frontier gap quarter over quarter, and nothing in this data suggests that reverses. The defensible read is that the paid premium buys less each cycle, not more.

Put the three together and the move is clear: the durable investment is the tooling layer, not the model. MCP servers and skill files are model-agnostic. They let you ride the free-model curve as it improves, hold token cost at zero where it counts, and avoid locking spend into a paid model whose total bill compounds as agent usage scales. The model you pick today may be obsolete in a year. The context layer you build around it won’t be.

Conclusion

The honest answer to “can a free model generate production-quality Ignite UI code?” is: yes — if you give it the right context. big-pickle’s 94% success rate didn’t happen in spite of its limitations. It happened because Ignite UI MCP servers and skill files covered exactly the gaps that would otherwise cause failures: wrong attribute names, missing icon identifiers, framework-specific patterns the model hasn’t seen enough of.

For Angular, Blazor, and React, the skill file alone is enough to match a paid model — faster and for free. For Web Components, add the MCP server and you get the same result. The paid model is more forgiving when tooling is missing, but with tooling in place that rarely matters.

And the direction of travel favors the free side. Per-task token consumption is rising while free models keep closing the capability gap — so the question shifts from “can a free model keep up today?” to “why would you pay a compounding token bill for a gap that’s shrinking?” Build the tooling layer once; it’s the part that holds its value while the models underneath it churn.