Validate AI Workflows Before You BuildNo-Build-First Method

Last week I needed to decide the architecture for an AI agent: single-agent or multi-agent? The old way would be to build both, deploy an A/B test, measure metrics for 2 weeks, refactor — totalling 6–8 weeks of engineering.

The new way: spawn 3 subagent variants in Claude Code, run the same input, compare outputs side-by-side — the whole thing took an afternoon. The winner went to production.

This isn't a shortcut. It's a methodology that works across AI coding tools — Claude Code, Cursor, OpenCode, Cline. The tool matters less than the principle.

When building an AI agent system, most devs follow this pattern:

1. 01Design architecture1 week
2. 02Build backend infrastructure2–3 weeks
3. 03Build prompt logic1 week
4. 04Deploy + test1 week
5. 05Find issues → refactor2–4 weeks
6. 06Repeat∞

Total time

6–12 weeks / decision

Risk

Sunk-cost bias when wrong

The core problem: confusing experimentation with production. Production code is hard to change — schema migrations, test infrastructure, deployment pipelines, rollback complexity. Experimentation needs to be fast, cheap, disposable. These two concerns MUST be separated.

AI coding tools (Claude Code, Cursor, OpenCode) fit the experimentation phase well. Production webservice is the scaling runtime — built only when patterns are proven.

Rule: Validate in experimentation runtime first. Build for production only after patterns are stable.

Question: Cold email agent architecture for project D2 — 3-agent chain vs 1-agent monster vs critique loop?

How we evaluated each variant

Dataset: 3 real prospects — Joshua (VP Engineering, SaaS), Evan (Founder, logistics), Sandra (Head of Growth, fintech). Each variant ran on all 3. Metrics per output:

Sandra (fintech) broke 2 variants that passed Joshua and Evan — which is why you need more than 1 test case before declaring a winner.

Findings from 18 spawns

Production decision

• 3-agent chain
• Anti-fab in BASE prompt
• Python validators between stages
• QA semantic only

Generation is straightforward — any tool can spawn variants. The harder problem is deciding which one is actually better, and on what criteria.

In those 18 spawns, the most time-consuming part wasn't spawning variants — it was evaluating them. Generating 7 email variants took about 20 minutes. Deciding which one was best took the other 6 hours.

The evaluation methods we actually used, in order of effort:

The lesson: any tool can spawn variants. What separates good experimentation from brute-force prompting is having a way to measure 'better'. No eval = no signal.

Method works with all of them. Compact comparison:

project/
├── CLAUDE.md          # or .cursorrules, AGENTS.md
├── tools/
│   ├── apollo_search.py
│   ├── email_send.py
│   └── storage.py
├── skills/
│   ├── outbound-research.md
│   └── personalization.md
└── data/
    └── test_prospects.json

def apollo_search(
    industry: str,
    size_range: str,
    location: str,
) -> list:
    """Search Apollo for prospects matching criteria."""
    # ... implementation
    return prospects

Spawn 3 subagents in parallel:

Agent A (multi-stage):
  skills/researcher.md → skills/personalizer.md → skills/qa.md
  → outputs/variant_a.json

Agent B (mega-prompt):
  skills/all-in-one.md
  → outputs/variant_b.json

Agent C (critique loop):
  skills/draft-critique-rewrite.md
  → outputs/variant_c.json

→ For ❌ cases: validate prompt logic in experimentation runtime first, then test infrastructure in a separate staging environment.

Things that have actually gone wrong, or that I watch for:

• Benchmark overfitting — tuned for Joshua and Evan, failed on Sandra (prospect 3). Test on more than 2 examples before declaring done.
• Weak evaluation — if eval criteria are loose, the "winner" is just whichever variant looks most convincing. Garbage eval → garbage decision.
• Local optimum — the best of 7 variants tried ≠ the globally best variant. You're sampling, not exhausting the search space.
• Sim/prod mismatch — runs smoothly in coding tool ≠ performs at scale, under concurrency, or with real API rate limits.
• Prompt spaghetti — fast iteration breeds prompts nobody can maintain. Document why each rule exists while you still remember.
• Architecture cargo-culting — copying 3-agent because someone else uses it, not because your data says so.